Microsoft KB Archive/61436

= Problem of Testing Floating-Point Equality, IF n=VAL(&quot;n&quot;) =

Article ID: 61436

Article Last Modified on 8/16/2005

-

APPLIES TO


 * Microsoft QuickBasic 4.0
 * Microsoft QuickBASIC 4.0b
 * Microsoft QuickBasic 4.5 for MS-DOS
 * Microsoft BASIC Compiler 6.0
 * Microsoft BASIC Compiler 6.0b
 * Microsoft BASIC Professional Development System 7.0
 * Microsoft BASIC Professional Development System 7.1

-



This article was previously published under Q61436



SUMMARY
A floating-point constant passed in a string to the VAL function can return a slightly different floating-point result compared to the same floating-point constant in an IF statement, resulting in an apparent inequality. This floating-point difference may seem like a software problem, but it is actually a design limitation. This behavior is demonstrated in the program below.

To reliably test for floating-point equality (in any binary floating-point format, such as IEEE or Microsoft Binary Format), you must subtract the two floating-point numbers being compared and test whether their difference is less than a value at the limits of significance for single or double precision.



MORE INFORMATION
You might expect the conditions in the IF statements in the program below to both be true and PRINT their messages. Instead, only the second IF is true and PRINTs its message: a$ = &quot;0.1&quot; IF .1 <= VAL(a$) THEN PRINT &quot;This is false, and does not print.&quot; s = VAL(a$)  ' This time use a temporary variable s.   IF .1 <= s THEN PRINT &quot;This is true and does print.&quot; There is an inequality in the first IF statement because the numeric constant .1 in the IF is assigned at compile time, whereas the VAL(A$) function is calculated at run time and different numeric storage is allocated internally so that a tiny difference exists between the numbers at the limits of single precision. In the second IF statement, the comparison of .1 and &quot;s&quot; is compiled differently than in the first IF, and the comparison luckily expected a &quot;true&quot; result. However, you should avoid both of the above numeric comparison techniques for testing the equality of floating-point numbers.

Instead, to reliably test for floating-point equality (in any binary floating-point format), you must subtract the two floating-point numbers being compared and test whether their difference is less than a value that is about 7 significant digits smaller than the value being compared for single precision (in other words, divide by 10^7 to find the comparison value), or about 15 significant digits smaller than the value being compared for double precision (divide by 10^15). For example: A$ = &quot;.1&quot; IF .1 - VAL(A$) <= .1 / 10^7 THEN PRINT &quot;Equal within single precision&quot;

B$ = &quot;.1#&quot; IF .1# - VAL(B$) <= .1# / 10^15 THEN PRINT &quot;Equal within double precision&quot;

