Microsoft KB Archive/838438

= Clients may receive an &quot;Error 792: The L2TP connection attempt failed because security negotiation timed out.&quot; error message when they try to complete a VPN connection to ISA Server 2006 or to ISA Server 2004 =

Article ID: 838438

Article Last Modified on 12/4/2007

-

APPLIES TO


 * Microsoft Internet Security and Acceleration Server 2004 Standard Edition
 * Microsoft Internet Security and Acceleration Server 2004 Enterprise Edition
 * Microsoft Internet Security and Acceleration Server 2006 Enterprise Edition
 * Microsoft Internet Security and Acceleration Server 2006 Standard Edition

-





SYMPTOMS
Virtual private network (VPN) clients may be unable to connect to a network through a VPN server that is running Microsoft Internet Security and Acceleration (ISA) Server 2006 or ISA Server 2004. In this scenario, the VPN clients may receive the following error message:

Error 792: The L2TP connection attempt failed because security negotiation timed out.



CAUSE
This issue may occur if both the following conditions are true:
 * The VPN clients use Layer 2 Tunneling Protocol (L2TP) to create the VPN connection.
 * ISA Server is configured to block IP fragments.



RESOLUTION
To resolve this issue, turn off the option that blocks fragmented IP packets. To do this, follow these steps:
 * 1) Start the ISA Server Management tool.
 * 2) Expand  , where   is the name of your ISA Server computer.
 * 3) Expand Configuration, and then click General.
 * 4) Under Additional Security Policy, click Define IP Preferences.

Note In ISA Server 2006, click Configure IP Protection.
 * 1) Click the IP Fragments tab, click to clear the Block IP fragments check box, and then click OK.
 * 2) Click Apply to update the firewall policy, and then click OK.



MORE INFORMATION
IPSec uses the Internet Key Exchange (IKE) protocol for mutual computer authentication and for the exchange of session keys in an L2TP VPN connection. The IKE negotiation information cannot fit inside a Maximum Transmission Unit (MTU). Because of this, the IKE negotiation packet is fragmented or broken into smaller multiple datagrams. When you filter fragmented packets in ISA Server, the IKE negotiation packets are dropped by ISA Server. Therefore, the VPN connection cannot be completed successfully.

Note IKE negotiation is always used regardless of your IPSec authentication mechanism, such as preshared keys, Kerberos protocol, or certificates.

For additional information about why you might want to filter IP fragments, search on &quot;packet fragments&quot; in ISA Server Help.

Additional query words: L2TP VPN

Keywords: kbfirewall kbnetwork kbprb kbwinservnetwork kbisa2006swept KB838438

-

[mailto:TECHNET@MICROSOFT.COM Send feedback to Microsoft]

© Microsoft Corporation. All rights reserved.