Article ID: 263213
Article Last Modified on 1/27/2007
APPLIES TO
- Microsoft Office Excel 2003
- Microsoft Excel 2002 Standard Edition
- Microsoft Excel 2000 Standard Edition
- MSPRESS Microsoft Excel 97 Developer's Kit ISBN 1-57231-498-2
- Microsoft Excel 97 Standard Edition
This article was previously published under Q263213
SUMMARY
Microsoft Excel may change the precision mode of the floating-point control word during its execution. Because of this, C/C++ DLLs called from Excel that use the standard run-time library may return slightly different results than expected. The difference is typically less than 10E-15.
MORE INFORMATION
C/C++ DLLs or XLLs designed to be used from Excel should link to the FP10.OBJ library. The Microsoft C/C++ run-time library sets the default internal precision of the math coprocessor (or emulator) to 64 bits. By linking your project with FP10.OBJ, you override this default and set the chip to utilize 80-bit precision. FP10.OBJ must appear before Libc.lib, Libcmt.lib, or Msvcrt.lib on the linker command line. By linking to FP10.OBJ, you should see more consistent results. However, even when utilizing 80-bits, Excel is still limited to 15 digits of precision because of its adherence to the IEEE 754 specification.
REFERENCES
78113 XL: Floating-Point Arithmetic May Give Inaccurate Results
125056 INFO: Precision and Accuracy in Floating-Point Calculations
Visual C++ Programmer's Guide, under "Floating-Point Support".
Additional query words: _controlfp _control87 _PC_53 _PC_64 double formula function decimal places
Keywords: kbfloatpoint kbinfo KB263213
