How to handle the FPU stack in the x86 Assembly using NASM and functions in c?

Question

I’m trying to do this operation (y = 0.299*red + 0.587*green + 0.114*blue) in Assembly mode protected nasm integrating c with Assembly in windows, but regardless of the input values the result is always 0.

extern printf  ;a função printf em C é chamada
extern scanf   ;a função scanf em C é chamada

SECTION .data

msg1 db "Pixel vai ficar preto!", 10,0
msg2 db "Pixel vai ficar branco!", 10,0
msg3 db "Digite os valores para as cores vermelho, verde e azul:", 10,0
scan db "%d", 0
;msg4 db "Os numeros sao %d, %d, %d", 0
msg5 db "valor final %.3f",10,0
constante1 dd 0.299
constante2 dd 0.587
constante3 dd 0.114
constante4 dd 0.713
constante5 dd 128
constante6 dd 0.564
;y dw 0

SECTION .bss
red resd 1
green resd 1
blue resd 1
y resd 1
cr resd 1
cb resd 1

SECTION .text


_start:
push msg3
call printf
add esp, 4

;pega o valor de vermelho
push red
push scan
call scanf
add esp, 8

;pega o valor de verde
push green
push scan
call scanf
add esp, 8

;pega o valor de azul
push blue
push scan
call scanf
add esp, 8


;imprime os valores lidos (só pra teste)
;push dword[blue]
;push dword[green]
;push dword[red]
;push msg4
;call printf
;add esp, 16

calcula_y:


fld dword[red]          ;coloca na pilha o valor de red
fld dword[constante1]       ;coloca na pilha o valor da constate1
fmulp st1,st0           ;faz a multiplicação e salva em st1

fld dword[green]        ;coloca na pilha o valor de green
fld dword[constante2]       ;coloca na pilha o valor da constate2
fmulp st1,st0           ;faz a multiplicação e salva em st1

fld dword[blue]         ;coloca na pilha o valor de blue
fld dword[constante3]       ;coloca na pilha o valor da constate3
fmulp st1,st0           ;faz a multiplicação e salva em st1

faddp st1, st0          ;faz a adição de st1 e st0 e salva em st1 e dá um pop em st0
fadd st1            ;faz a adição de st1 e st0 e salva em st0
fst dword[y]            ;pega o valor st0 e coloca em y

push dword[y+4]
push dword[y]
push msg5
call printf
add esp, 12



ret

Answer 1

The logic in using FPU is correct.

There are only two problems in the program:

1) Data load for FPU

When performing the function scanf to read the data, the informed format string is %d, so the read result will store a whole number in the target memory.

When loading this number to the FPU as, for example, in the line fld dword[red], FPU expects to find a number in format Floating Point, and not a whole.

The solution is to change the format string scan for "%f".

2) Storage of calculated result in memory

FPU is trying to store a result in format qword, however, you specified the destination address as dword.

Since the value to be stored is greater than the target, FPU triggers an exception of the type #P, which conforms to the Intel Handbook:

"Value cannot be represented Exactly in Destination format."

The solution is to change the prefix dword for qword.

Follow below, the program already with these corrections:

global _main
extern _printf  ;a função _printf em C é chamada
extern _scanf   ;a função _scanf em C é chamada

SECTION .data

msg1        db  "Pixel vai ficar preto!", 10,0
msg2        db  "Pixel vai ficar branco!", 10,0
msg3        db  "Digite os valores para as cores vermelho, verde e azul:", 10,0
scan        db  "%f", 0 ; tem que ser %f!!!!

;msg4 db "Os numeros sao %d, %d, %d", 0
msg5        db "valor final %.3f",10,0
constante1  dd 0.299
constante2  dd 0.587
constante3  dd 0.114
constante4  dd 0.713
constante5  dd 128
constante6  dd 0.564
;y dw 0

SECTION .bss

red         resd 1
green       resd 1
blue        resd 1
y           resd 1
cr          resd 1
cb          resd 1

SECTION .text

_main:
            ; Aqui, precisa salvar o stack frame
            push        ebp
            mov         ebp, esp

            push        msg3
            call        _printf
            add         esp, 4

            ;pega o valor de vermelho
            push        red
            push        scan
            call        _scanf
            add         esp, 8

            ;pega o valor de verde
            push        green
            push        scan
            call        _scanf
            add         esp, 8

            ;pega o valor de azul
            push        blue
            push        scan
            call        _scanf
            add         esp, 8


;imprime os valores lidos (só pra teste)
;push dword[blue]
;push dword[green]
;push dword[red]
;push msg4
;call _printf
;add esp, 16

calcula_y:

            fld         dword[red]          ;coloca na pilha o valor de red
            fld         dword[constante1]   ;coloca na pilha o valor da constate1
            fmulp       st1, st0            ;faz a multiplicação e salva em st1

            fld         dword[green]        ;coloca na pilha o valor de green
            fld         dword[constante2]   ;coloca na pilha o valor da constate2
            fmulp       st1, st0            ;faz a multiplicação e salva em st1

            fld         dword[blue]         ;coloca na pilha o valor de blue
            fld         dword[constante3]   ;coloca na pilha o valor da constate3
            fmulp       st1, st0            ;faz a multiplicação e salva em st1

            faddp       st1, st0            ;faz a adição de st1 e st0 e salva em st1 e dá um pop em st0
            faddp       st1                 ;faz a adição de st1 e st0 e salva em st0
            fst         qword[y]            ; Aqui, tem que ser QWORD

            push        dword[y+4]
            push        dword[y]
            push        msg5
            call        _printf
            ;call        _scanf
            add         esp, 12

            mov         esp, ebp
            pop         ebp
            ret

The program was compiled with NASM version 2.11.08, through the command:

nasm -g -f win32 xx.asm

And linked with Microsoft (R) Incremental Linker Version 14.00.23506.0, through the command:

link /debug:FULL /libpath:"caminho_para_o_dir_lib_do_msvc" /subsystem:CONSOLE xx.obj "libcmt.lib" "advapi32.lib" "legacy_stdio_definitions.lib"

Barter path para_o_dir_lib_do_msvc to the directory lib within the MSVC facility.

For the debug process during the analysis, the program was used: x64dbg (32-bit version).

After execution, the result of the program was:

Digite os valores para as cores vermelho, verde e azul:
100.0
101.0
102.0
valor final 100.815

Note: To perform the error-free link process, it was necessary to add the underline '_' before duties.

For further reference, please refer to the manuals (in PDF):
Intel® 64 and IA-32 Architectures Software Developer Manuals