Microsoft KB Archive/259562

= INFO: Visual FoxPro Digits of Precision =

Article ID: 259562

Article Last Modified on 5/12/2003

-

APPLIES TO


 * Microsoft Visual FoxPro 3.0 Standard Edition
 * Microsoft Visual FoxPro 3.0b Standard Edition
 * Microsoft Visual FoxPro 5.0 Standard Edition
 * Microsoft Visual FoxPro 5.0a
 * Microsoft Visual FoxPro 6.0 Professional Edition

-



This article was previously published under Q259562



SUMMARY
Visual FoxPro is documented as having 16 digits of precision. This is an approximation: the actual maximum exactly representable number is 9007199254740992 (2^53).



MORE INFORMATION
Floating point numbers are stored in 8-byte or 64-bit representations. There are 12 bits of overhead, leaving 52 bits to store the number. There is one more implied bit that gives you 2^53 as the maximum. The maximum number that can be stored by Visual FoxPro is 2^1023. The highest power of two that is printed out exactly using the ? command with the default setting of SET DECIMALS TO 2 is 2^43.

The following code demonstrates this: * Start of Code SET DECIMALS TO 2 ? 2^43            && All digits displayed ? 2^44            && Scientific notation

SET DECIMALS TO 5 ? 2^53            && Maximum exact number

? 2^53 - 1        && Correct result ? 2^53 + 1        && Incorrect result: rounded in floating point

? 2^1023          && Cannot display: *'s will be printed ? 2^1022          && Can display
 * End of Code

