Microsoft KB Archive/299341

= FIX: Rounding Error When Saving Calculated Value into Integer Field =

Article ID: 299341

Article Last Modified on 5/12/2003

-

APPLIES TO


 * Microsoft Visual FoxPro 3.0 Standard Edition
 * Microsoft Visual FoxPro 3.0b Standard Edition
 * Microsoft Visual FoxPro 5.0 Standard Edition
 * Microsoft Visual FoxPro 5.0a
 * Microsoft Visual FoxPro 6.0 Professional Edition

-



This article was previously published under Q299341



SYMPTOMS
If a calculated value is written into an integer field in a table, the value may be rounded improperly and stored incorrectly. For example, if X = 2.05 * 100, when X is added to an integer field, the actual value stored to the table is 204.



STATUS
This problem was corrected in Microsoft Visual FoxPro version 7.0 for Windows.



Steps to Reproduce Behavior
  Copy the following code to a new program: CLEAR LOCAL lnCalcedVal lnCalcedVal = 2.05 * 100

CREATE CURSOR TstCurs (iFLD1 i) INSERT INTO TstCurs VALUES(lnCalcedVal) ? &quot;2.05 * 100&quot; ? &quot;&quot; ? &quot;Calculated Value:&quot;,lnCalcedVal ? &quot;Value inserted into table:&quot;,TstCurs.iFLD1

USE IN TstCurs  Save and run the code (the program name does not matter). When the code runs, 2.05 * 100 is stored to a local variable, lnCalcedVal. This variable is then inserted into an integer field in a cursor. The output on the screen shows the value of lnCalcedVal and the contents of the cursor's integer field. Note that the two values are different.

(c) Microsoft Corporation 2001, All Rights Reserved. Contributions by Trevor Hancock, Microsoft Corporation.

Keywords: kbbug kbfix kbdatabase kbvfp700fix KB299341

-

[mailto:TECHNET@MICROSOFT.COM Send feedback to Microsoft]

© Microsoft Corporation. All rights reserved.