Microsoft KB Archive/911554

From BetaArchive Wiki

Article ID: 911554

Article Last Modified on 12/4/2007



APPLIES TO

  • Microsoft Internet Security and Acceleration Server 2004 Enterprise Edition
  • Microsoft Internet Security and Acceleration Server 2004 Standard Edition




SYMPTOMS

Consider the following scenario:

  • A client computer that is behind a network address translation (NAT) device uses an Internet Protocol security (IPsec) NAT traversal (NAT-T) connection to access resources.
  • These resources are behind a computer that is running Microsoft Internet Security and Acceleration (ISA) Server 2004.
  • ISA Server 2004 is configured as the perimeter firewall.

In this scenario, the client computer cannot access the resources that are behind the ISA Server 2004 computer. Additionally, the following error code is logged to the ISA Server Firewall service log:

0xc0040031 FWX_E_BAD_TCP_CHECKSUM_DROPPED

Note This problem occurs if IPsec is used to help secure inbound traffic on the ISA Server computer.

CAUSE

This problem occurs when there is a mismatch between the TCP checksum and the checksum that is calculated by ISA Server.

When IPsec NAT-T is used, the TCP checksum and the checksum that is calculated by the ISA Server do not match. Therefore, ISA Server blocks the connection.

RESOLUTION

Service pack information

ISA Server 2004, Enterprise Edition

To resolve this problem, obtain the latest service pack for ISA Server 2004, Enterprise Edition. For more information, click the following article number to view the article in the Microsoft Knowledge Base:

891024 How to obtain the latest ISA Server 2004 service pack



After you apply the service pack, follow the steps that are described in the "How to set up an exclusion list" section.

ISA Server 2004, Standard Edition

To resolve this problem, apply the hotfix that is described in the following Microsoft Knowledge Base article:

915461 ISA Server 2004 blocks UDP traffic between two networks



After you apply the hotfix, follow the steps that are described in the "How to set up an exclusion list" section.

STATUS

Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.

MORE INFORMATION

How to set up an exclusion list

After you install the hotfix or the service pack, configure an exclusion list for the TCP checksum validation. This configuration prevents the resource access problem because inbound traffic is examined at the IPsec layer. To configure an exclusion list, follow these steps:

  1. Start Notepad on the ISA Server computer.
  2. Copy and then paste the following script into Notepad. This script adds a Vendor Parameter set for the IP address and port combinations that are to be excluded from the TCP checksum validation for ISA Server.

    ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
    '
    ' Copyright (c) Microsoft Corporation. All rights reserved.
    ' THIS CODE IS MADE AVAILABLE AS IS, WITHOUT WARRANTY OF ANY KIND. THE ENTIRE
    ' RISK OF THE USE OR THE RESULTS FROM THE USE OF THIS CODE REMAINS WITH THE
    ' USER. USE AND REDISTRIBUTION OF THIS CODE, WITH OR WITHOUT MODIFICATION, IS
    ' HEREBY PERMITTED.
    ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
    
    ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
    ' This script adds a new VendorParametersSets under the array root.
    ' Used to added new parameters that are required for hotfixes or service packs.
    ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
    
    Sub AddIpsecXsumBypass()
    
        ' Create the root object.
        Dim root  ' The FPCLib.FPC root object
        Set root = CreateObject("FPC.Root")
    
        'Declare the other objects needed.
        Dim array       ' An FPCArray object
        Dim VendorSets  ' An FPCVendorParametersSets collection
        Dim VendorSet   ' An FPCVendorParametersSet object
    
        ' Get references to the array object
        ' and the network rules collection.
        Set array = root.GetContainingArray
        Set VendorSets = array.VendorParametersSets
    
        On Error Resume Next
        Set VendorSet = VendorSets.Item( "{143F5698-103B-12D4-FF34-1F34767DEabc}" )
    
        If Err.Number <> 0 Then
            Err.Clear
    
            ' Add the item
            Set VendorSet = VendorSets.Add( "{143F5698-103B-12D4-FF34-1F34767DEabc}" )
            CheckError
            WScript.Echo "New VendorSet added... " & VendorSet.Name
    
        Else
            WScript.Echo "Existing VendorSet found... value- " & VendorSet.Value("IpAddrToBypassIPSecXsum")
        End If
    
        WScript.Echo "Initial IpAddrToBypassIPSecXsum- " & VendorSet.Value("IpAddrToBypassIPSecXsum")
    
        Err.Clear
    '
    '       The format for setting an exception is: sourceip:sourceport>destinationip:destinationport
    '       Any field (entire ip or port) which is 0 is ignored.
    '       For example- 12.0.0.0:0>14.15.16.17:80;
    '       means that traffic from 12.0.0.0, any source port, to 14.15.16.17
    '       port 80 is not subject to validation of tcp checksum. The ';' character
    '       must end each address pair, even if there is only one pair.
    ''
    '
    '   THIS FOLLOWING LINE IS THE ONE THAT YOU NEED TO CHANGE  
    '
        VendorSet.Value("IpAddrToBypassIPSecXsum") = "12.34.55.66:45587>54.127.232.0:80;"
    '
    '   Use the following value to reset the exception list.
    '
    '    VendorSet.Value("IpAddrToBypassIPSecXsum") = ""
    
        If Err.Number <> 0 Then
            CheckError
        Else
            VendorSets.Save false, true
            CheckError
    
            If Err.Number = 0 Then
                WScript.Echo "Done with IpAddrToBypassIPSecXsum, saved!"
            End If
        End If
    
       WScript.Echo "Final IpAddrToBypassIPSecXsum- " & VendorSet.Value("IpAddrToBypassIPSecXsum")
    
    End Sub
    
    Sub CheckError()
    
        If Err.Number <> 0 Then
            WScript.Echo "An error occurred: 0x" & Hex(Err.Number) & " " & Err.Description
            Err.Clear
        End If
    
    End Sub
    
    AddIpsecXsumBypass
    
    'End of Script
  3. Amend the line in the script that starts with “VendorSet.Value("IpAddrToBypassIPSecXsum")” to include the exception list. Use the following format:

    SourceIp:SourcePort>DestinationIp:DestinationPort

    For example, assume that traffic is directed from address 12.0.0.0 and from any source port to address 14.15.16.17 and to port 80. This traffic is not validated by ISA Server for the TCP checksum when you include the following line in the script:

    "12.0.0.0:0>14.15.16.17:80”

    Notes

    • Full wildcard values are supported to exclude an IP address and a port. For example, IP address 0.0.0.0 is supported and will add every client IP address to the exclusion list.
    • Multiple entries are supported. Each entry must end by using a colon (;). For example, the following address and port notation is supported:

      "12.0.0.0:0>14.15.16.17:80;12.0.0.0:1>14.15.16.17:80;"

  4. On the File menu, click Save As.
  5. In the File name box, type ISA_script_name.vbs, and then click OK.
  6. At a command prompt, type the following command, and then press ENTER:

    ISA_script_name.vbs

    Note You must run the script from the location where you saved the ISA_script_name.vbs file.
  7. Restart the Firewall service.

For example, when you want to help secure a Web listener by using IPsec, you must use an exclusion list for the TCP checksum validation. This configuration may be required for RPC over HTTP traffic when client certificate authentication is required but is not supported by Microsoft Office Outlook. In this scenario, you can use IPsec to provide certificate authentication at the network layer.

In this scenario, the exclusion list must include every client IP port and every client source port that sends traffic to the ISA Server computer on port 443. For example, the ISA_script_name.vbs script must include the following line:

VendorSet.Value("IpAddrToBypassIPSecXsum") = "0.0.0.0:0>ISAWebListenerIP:443;"


Note ISAWebListenerIP is the IP address that ISA Server uses for its Web listener. ISA Server listens on port 443.

For more information, click the following article number to view the article in the Microsoft Knowledge Base:

824684 Description of the standard terminology that is used to describe Microsoft software updates


Keywords: kbbug kbfix kbqfe kbpubtypekc KB911554