What is the difference of syscall and call in Assembly?

Question

The following simple code I wrote, based on a code I read in a book, was not compiling:

;myhello
section .data
    msg db "Boa tarde",0
    NL db 0xa
section .bss
section .text
    global main
main:
    push rbp
    mov rbp, rsp
    mov rax,1
    mov rdi,1
    mov rsi, msg
    mov rdx,9
    call
    mov rax,1
    mov rdi,1
    mov rsi, NL
    mov rdx,1
    call
    mov rax, 60
    mov rdi,0
    call

And I was printing the following mistake:

myhello.asm:14: error: invalid combination of opcode and operands
myhello.asm:19: error: invalid combination of opcode and operands
myhello.asm:22: error: invalid combination of opcode and operands

How the problem was in call, I switched to syscall and the program started working. However, I don’t know why.

What is the difference between call and syscall in Assembly?

Answer 1

Near Relative Call

call rel16/rel32

This is the call we use has no secret. It basically gets a negative or positive number indicating the number of bytes that should be diverted

Near Absolute Call

Different from the relative call indicating a number of bytes to be summed with RIP, in an absolute call you pass the exact address from where you want to make the call. You can try making a call like this:

mov  rax, rotulo
call rax

Far Call

call seg16:off16   ; Em 16-bit
call seg16:off32   ; Em 32-bit

call mem16:16  ; Em 16-bit
call mem16:32  ; Em 32-bit
call mem16:64  ; Em 64-bit

Far calls are all absolute and receive in the operand a value following the format of specifying an offset followed by the 16-bit segment. In nasm, an immediate value can be passed as follows:

call 0x1234:0xabcdef99

Where the value on the left specifies the segment and the one on the right the offset. Note that this instruction is not supported in 64-bit. The second type of far call, supported in 64-bit, is what it takes to operate a value in memory. But note that we have a near call that gets the same kind of argument. By default the nasm will assemble the instructions as near and not far. But you can avoid this ambiguity by making explicit with Keywords from nasm, which are very intuitive:

call [rbx]       ; Próximo e absoluto
call near [rbx]  ; Próximo e absoluto
call far [rbx]   ; Distante

near expects offset address in memory, no secret. But far expects offset followed by segment.

Syscall

A system call, or syscall (abbreviation for system call), is something very similar to a call but with the very subtle difference that it is the operating system kernel that will run the code. The kernel, if you don’t know, is the main part of an operating system. It is the basis of the entire rest of the system that runs on kernel control. Linux is actually a kernel, a "Linux" operating system is actually an operating system that uses the Linux kernel.

In x86-64 there is an instruction that has been made specifically to make system calls and its name is intuitively syscall. It does not receive any operand and the specification of which code it will execute and with which arguments it is defined by a call convention as in the case of functions.

Syscall convention x86-64

The convention to make a system call on Linux x86-64 is quite simple, just set RAX to the syscall number you want to run and other 6 registers are used to pass arguments. In syscalls that receive less than 6 arguments, it is not necessary to define the value of the remaining registers because they will not be used. The syscall return is in RAX as well as in the C language call convention.