Microsoft KB Archive/93437

= FIX: Result Differs When Comparing Single w/ Double Precision =

Article ID: 93437

Article Last Modified on 10/30/2003

-

APPLIES TO


 * Microsoft Visual Basic 2.0 Standard Edition
 * Microsoft Visual Basic 2.0 Professional Edition

-



This article was previously published under Q93437



SYMPTOMS
When you compare a real number stored as a Single precision variable to the same real number stored as a Double precision variable, the result may be that they are not equal. Storage of real numbers is different within the two data types. Therefore, the number may be represented differently, so a check for equivalence can return false.



WORKAROUND
In Microsoft Visual Basic version 2.0 for Windows, this problem occurs only on computers that do not have coprocessors. If your computer does not have a coprocessor, add some extra code when comparing data stored in Single data types to those stored in Double data types.



STATUS
Microsoft has confirmed this to be a problem in the Standard and Professional Editions of Microsoft Visual Basic version 2.0 for Windows. It was corrected in Microsoft Visual Basic version 3.0 for Windows.



Steps to Reproduce the Problem
 Start VB.EXE.  Add the following in the Form_Click event procedure of Form1: Print 12.3! = 12.3# '** Note the '#' sign disappears.  Press the F5 key to run the example, and click Form1. If the result is '0' then a coprocessor is not installed, if the result is '-1' then a coprocessor is installed or you are on a 486 with a built in coprocessor.

Note if you run this same example in Microsoft Visual Basic for Windows, version 1.0, the result is '0' with or without a coprocessor installed.

Additional query words: buglist2.00 fixlist3.00 2.00 3.00

Keywords: kbbug kbfix KB93437

-

[mailto:TECHNET@MICROSOFT.COM Send feedback to Microsoft]

© Microsoft Corporation. All rights reserved.