Most voted "floating-point" questions
Floating point numbers are approximations of real numbers that can represent intervals greater than whole numbers using the same amount of memory, however at the cost of less precision.
Learn more…25 questions
Sort by count of
-
111
votes7
answers13788
viewsHow to represent money in Javascript?
What is the best way to represent monetary units (i.e. money) in Javascript? I know it is not good to use floating point, given the rounding problem, but I do not know what would be the best…
-
56
votes4
answers3125
viewsPrecise math division
In Windows calculator: 9/3,0001 = 2,999... In my program: 9/3,0001 = 3; I would like to get that level of accuracy using C. Watch excerpts from my program: double op(double num1, double num2, char…
-
39
votes3
answers6437
viewsInaccurate result in broken numbers calculation
Problem Test there in the consoles of your browsers: 1067.11-1000 = 67.1099999999999 The right thing would be 67.11 Could someone explain this to me? And how do I fix it?…
-
28
votes2
answers2065
viewsWhat is "positive zero" and "negative zero" in float and double types?
In response of this link the operation of the atan2(), and its translation of documentation by Victor Stafusa, there are some excerpts that I highlight below: (...) If the first argument is positive…
-
23
votes2
answers1397
viewsHow to calculate the value of a polynomial function at an arbitrary point?
I need to calculate the value of a polynomial function of arbitrary grade in arbitrary values of the realm. I would like to know how to do it in a way that is not naive when doing the floating point…
-
13
votes3
answers8540
viewsJavascript floating point account realization with absolute precision
I am working with financial values and when it comes to performing mathematical operations in Javascript, the values are not as expected. console.log(11.8-10.2) 1.6000000000000014 I would like to…
-
12
votes4
answers1519
viewsWhy is 0.1 + 0.05 not equal to 0.15? What solutions can be used in R?
In the R: 0.1+0.05 == 0.15 [1] FALSE Why does this happen? How to get around the situation (functions and packages to handle floating points)? Editing: This other question already contains general…
-
12
votes2
answers360
viewsWhy are primitive types with floating points divided or multiplied by certain multiples of 10 displayed in scientific notation?
In this answer we can see that Java has a peculiarity when displaying the result in certain types of operations with primitive types that have floating point, such as division by multiples of 10,…
-
11
votes2
answers355
viewsNegative variance in R? Floating point error propagation
Suppose the following formula to calculate the variance: variancia <- function(x) { n <- length(x) (1/(n^2-n))*(n*(sum(x^2))-(sum(x)^2)) } Note that it is equivalent to the function var in…
-
9
votes3
answers4910
viewsSplit result equals zero in decimals
In division 1 by 3, my program is printing the following: number e' of 0.00 What is the error in the code? #include <stdio.h> int main(){ float numero; numero = 1/3; printf("o valor do numero…
-
9
votes3
answers235
viewsWhy is it not possible to use comparison operators with Nan?
Nan - The global property Nan is a special value meaning Not-A-Number (not a number). The curious thing is that it is not possible to perform operations of comparison with this almost mystical…
-
7
votes1
answer115
viewsWhy is the expression (double.Minvalue == double.Minvalue + 1) true?
The following code C#: public class Program { public static void Main(string[] args) { //Your code goes here Console.WriteLine(double.MinValue == double.MinValue + 1); Console.WriteLine(int.MinValue…
-
5
votes1
answer426
viewsWarning compared between floats how to proceed
In an exercise I did, you are asked to make a compound interest calculation with for for the values of 5%, 6%, 7%, 8%, 9%, and 10%. As I could not use a check per counter being the counter a type…
-
4
votes2
answers1659
viewsWhat is and what is the reserved word "strictfp" for?
I don’t know this keyword, I’ve never seen an example of code in which it was used. Until today I discovered her existence by reading some books about Java, however it was not clear what the utility…
-
4
votes2
answers475
viewsWhy in Python 0.03 % 0.01 = 0.009999999999998 and not 0?
>>> 0.03 % 0.01 0.009999999999999998 Why gives this result, the rest of the division being 0? And also, instead of 3, give: >>> 0.03 // 0.01 2.0…
-
4
votes1
answer110
viewsHow do computers store and interpret floating point numbers in binary?
In my course of systems analysis and development I started the matter of computational mathematics the subject was numerical bases and conversions, something that caught my attention is that they…
-
3
votes2
answers222
viewsProblems computing on average in Java
I’m trying to make a program that calculates the average between 4 values. Code: private void botao1ActionPerformed(java.awt.event.ActionEvent evt) { // TODO add your handling code here: int n1 =…
-
2
votes1
answer2162
viewsShow a specific number of decimals in floating point numbers in Lua language
I started to learn a little about the language Lua and I’m doing some algorithms to practice and one of them I should show in the output of floating point values with 4 decimal places. In C++ I…
-
2
votes1
answer610
viewsHow to handle the FPU stack in the x86 Assembly using NASM and functions in c?
I’m trying to do this operation (y = 0.299*red + 0.587*green + 0.114*blue) in Assembly mode protected nasm integrating c with Assembly in windows, but regardless of the input values the result is…
-
1
votes1
answer471
viewsError: cannot Convert from double to float
Why float is not accepted for numbers 0.197 and 0.185? Code: float salBruto, salLiquido; Scanner ent = new Scanner (System.in); System.out.println("Informe o seu salário:"); salBruto =…
-
1
votes3
answers1588
viewsPython Floating Point Problem 3
I’m writing an algorithm where variable increment float in 0.2, however, after a few increments instead of incrementing 2.2 for 2.4, for example, the program increments to 2.4000000000000004 I’ve…
-
1
votes1
answer57
viewsHow to remove value after floating point
I have the code section exposed below: var calcularSoma = function (primeiroValor, segundoValor) { var soma = primeiroValor + segundoValor; if (soma < 10) { console.log(soma, "Unidades."); return…
-
0
votes3
answers3861
viewsHow to replace the comma
I was wondering how can I replace the comma for a period. Example: I have the following number 2.32 wanted it to be 2.32. How can I do that? I’ve tried using the…
-
0
votes4
answers1240
viewsI want to take only the first digit after the point of a number that in python 3.7.4
I want to take only the number after the dot, type in the number 3.76443 I want to print only the 7, how do I do? how will I also make to print it? I have to create a variable for it?
-
-2
votes1
answer50
viewsPeople need to put this JS that would be a floating Whatsapp on a PHP page!
People I’m trying to put this JS on my site that is in PHP but I tried the ways I know but nothing worked I would like the help of you I believe that just put in a page I can already put in all a…