Numerical accuracy in Sqlite

Asked

Viewed 246 times

2

About numerical accuracy in Sqlite v3.20.0. See the sequence of commands below and the result of .dump. The value associated with the INSERT declaration is not the same value obtained. I enter the value 123456.789 and get 123456.7890000000043. How can this occur?

SQLite version 3.20.0 2017-08-01 13:24:15
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> CREATE TABLE Teste (Campo Number(9,3));
sqlite> INSERT INTO Teste Values (123456.789);
sqlite> SELECT * From Teste;
123456.789
sqlite> .dump
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE Teste (Campo Number(9,3));
INSERT INTO Teste VALUES(123456.7890000000043);
COMMIT;
sqlite>

1 answer

1

Binary floating point mathematics is like this. In most programming languages (not only in Sqllite), it is based on the IEEE 754 standard. For example, Javascript uses the 64-bit floating-point representation, which is the same as double Java. The crux of the problem is that numbers are represented in this format, since an integer has a power of two; rational numbers (like 0.1, which is 1/10) whose denominator is not a power of two cannot be exactly represented. Follow the references: Wiki - Float Binary Representation

  • More Has How To Solve It?

  • 1

    In these cases the best is to treat without decimals, that is always as whole, and at the time of displaying the data, makes the division, including the decimals you want. Ex: 123,45 = 12345/100

Browser other questions tagged

You are not signed in. Login or sign up in order to post.