Microsoft KB Archive/824835

= Certain IDL constructs are marshaled and unmarshaled incorrectly by RPC =

Article ID: 824835

Article Last Modified on 10/26/2006

-

APPLIES TO


 * Microsoft Windows Server 2003, Datacenter Edition for Itanium-Based Systems
 * Microsoft Windows Server 2003, Enterprise Edition for Itanium-based Systems
 * Microsoft Windows Server 2003, Datacenter Edition (32-bit x86)
 * Microsoft Windows Server 2003, Enterprise Edition (32-bit x86)
 * Microsoft Windows Server 2003, Standard Edition (32-bit x86)
 * Microsoft Windows Server 2003, Web Edition
 * Microsoft Windows XP for Itanium-based Systems Version 2003
 * Microsoft Windows XP Professional x64 Edition
 * Microsoft Windows XP Professional
 * Microsoft Windows XP Home Edition
 * Microsoft Windows XP Tablet PC Edition
 * Microsoft Windows XP Media Center Edition 2002

-





Important This article contains information about how to modify the registry. Make sure to back up the registry before you modify it. Make sure that you know how to restore the registry if a problem occurs. For more information about how to back up, restore, and modify the registry, click the following article number to view the article in the Microsoft Knowledge Base:

256986 Description of the Microsoft Windows registry



SYMPTOMS
Programs that use remote procedure calls (RPC) to communicate with other Microsoft Windows-based computers over a network may fail. For example, when you use a custom Microsoft Visual Basic program between two remote computers, and the Visual Basic program uses user-defined type marshaling, you may find that certain Interface Definition Language (IDL) constructs are incorrectly marshaled and unmarshaled. For example, you may experience one of the following symptoms:
 * The remote call fails, and an RPC_X_BAD_STUB_DATA error code is returned by the Visual Basic program.
 * Incorrect data is unmarshaled.



CAUSE
This problem may occur if the RPC component incorrectly marshals or unmarshals certain IDL constructs. Versions of Windows that may incorrectly marshal and unmarshal IDL constructs include the following:  Microsoft Windows XP (original version) and Windows XP Service Pack 1 (SP1) installations that include the MS03-039 security update. This update is documented in the following Microsoft Knowledge Base article:

824146 MS03-039: A buffer overrun in RPCSS could allow an attacker to run malicious programs

 Microsoft Windows 2000 Microsoft Windows NT 4.0 Microsoft Windows 98

This problem may occur in a Visual Basic program if both of the following conditions are true:  The IDL file uses a structure that contains an 8-byte field that uses any one of the following types:

DATE

Double

CY

VARIANT

DECIMAL

INT64/UINT64

</li> The 8-byte field is a non-first field, and there is a field that is 4 bytes or smaller before the 8-byte field that is contained in the same structure.</li></ul>

For example, this problem may occur if the IDL file uses a structure that is similar to the following: Public Type myType

member1 As Long

member2 As Double

End Type

<div class="resolution_section">

Hotfix information for Windows Server 2003
To communicate with versions of Windows that incorrectly marshal and unmarshal IDL constructs, follow these steps: <ol> Do one of the following:  Install the latest service pack for Windows Server 2003. For more information, click the following article number to view the article in the Microsoft Knowledge Base:

889100 How to obtain the latest service pack for Windows Server 2003

</li> Install the security update that is described in security bulletin MS05-051. For more information about security bulletin MS05-051, click the following article number to view the article in the Microsoft Knowledge Base:

902400 MS05-051: Vulnerabilities in MS DTC and COM+ could allow remote code execution

</li></ul> </li> Add the UDTAlignmentPolicy registry entry. To do this, follow these steps:

Warning Serious problems might occur if you modify the registry incorrectly by using Registry Editor or by using another method. These problems might require that you reinstall your operating system. Microsoft cannot guarantee that these problems can be solved. Modify the registry at your own risk. <ol style="list-style-type: lower-alpha;"> Click Start, click Run, in the Open box, type regedit, and then click OK.</li> Locate the following registry subkey:

</li> On the Edit menu, point to New, and then click DWORD value.</li> Type UDTAlignmentPolicy, and then press ENTER.</li> Double-click UDTAlignmentPolicy, and then type a value to determine the data format that RPC will use based on the following information:

If the UDTAlignmentPolicy value is set to 0, RPC will marshal by using the &quot;old&quot; incorrect data format. RPC will communicate with versions of Windows that incorrectly marshal and unmarshal IDL constructs. For example, RPC will communicate with Windows 2000 or Windows XP SP1 installations that include the MS03-039 update.

If the UDTAlignmentPolicy value is set to 0, RPC can communicate with Windows 2000-based computers.

If the UDTAlignmentPolicy value is set to 1, RPC will marshal by using the &quot;new&quot; correct wire format. RPC will communicate with versions of Windows that correctly marshal and unmarshal IDL constructs. For example, RPC will communicate with Windows Server 2003 or Windows XP installations that include the MS03-026 update.

If the UDTAlignmentPolicy entry does not exist, the remote call will fail, and RPC will return an RPC_X_BAD_STUB_DATA error code when RPC tries to marshal or unmarshal affected types.

Important When the UDTAlignmentPolicy value is set to 0, your Windows Server 2003-based computer can exchange data for the IDL construct with versions of Windows that incorrectly marshal and unmarshal IDL constructs. However, the IDL construct is unmarshaled to the incorrect format. Using incorrect marshaling and unmarshaling code may prevent the interoperability of this same IDL construct and other IDL constructs when programs communicate with computers that run versions of Windows that correctly marshal and unmarshal IDL constructs.</li></ol> </li></ol>

Service pack information for Windows XP
To resolve this problem, obtain the latest service pack for Windows XP. For more information, click the following article number to view the article in the Microsoft Knowledge Base:

322389 How to obtain the latest Windows XP service pack

In Windows XP Service Pack 2, you can modify the UDTAlignmentPolicy registry entry to adjust the way RPC communicates with different versions of Windows. To do this, follow step 3 in the &quot;Hotfix information for Windows Server 2003&quot; section.

Hotfix information for Windows XP
To communicate with versions of Windows that incorrectly marshal and unmarshal IDL constructs, such as Windows 2000 or Windows XP SP1 installations that include the MS03-039 update, install the MS03-039 update on your Windows XP SP1-based computers. For more information about the MS03-039 update, click the following article number to view the article in the Microsoft Knowledge Base:

824146 MS03-039: A buffer overrun in RPCSS could allow an attacker to run malicious programs

A supported hotfix is not currently available for Windows XP-based computers that communicate by using RPC with versions of Windows that correctly marshal and unmarshal IDL constructs. These versions are Windows Server 2003 and Windows XP installations that include the MS03-026 update.

<div class="status_section">

STATUS
Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the &quot;Applies to&quot; section. This problem was first corrected in Microsoft Windows XP Service Pack 2.

<div class="moreinformation_section">

MORE INFORMATION
Versions of Windows that correctly marshal and unmarshal the IDL construct that is mentioned in the &quot;Cause&quot; section include the following:  Microsoft Windows Server 2003 and later versions</li> Microsoft Windows XP Service Pack 1 (SP1) installations that include the MS03-026 update but do not include the MS03-039 security update. The MS03-026 update is documented in the following Microsoft Knowledge Base article:

823980 MS03-026: Buffer overrun in RPC interface may allow code execution

</li> Microsoft Windows XP Service Pack 2 (SP2)</li></ul>

Note A supported hotfix is not currently available for Windows 2000-based computers that communicate by using RPC with versions of Windows that correctly marshal and unmarshal IDL constructs. These versions are Windows Server 2003 and Windows XP installations that include the MS03-026 update.

Additional query words: UDT

Keywords: kbhotfixserver kbqfe kbtshoot kbwinxpsp2fix kbwinxppresp2fix kbfix kbbug KB824835

-

[mailto:TECHNET@MICROSOFT.COM Send feedback to Microsoft]

© Microsoft Corporation. All rights reserved.