Programming: Java

Floating point precision

It didn’t occur to me that computer cannot store floating points accurately.

float a = 1.0;
float b = 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1;

a != b

The value of b is infact 0.999999999999999

Pretty weird. That is why I failed one of the test data sets for my latest lab #3. Due to the inprecision of floating point calculation.

Workaround? Easy. Include a tolerance value / epsilon.

e.g
(Math.abs(a – b) < 0.00001) this will be true whereas (a == b) this will be false

Don’t know if this happens to ASP. Erik wanna try it out and tell me about the floating point computation of ASP and PHP?

4 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments