Microsoft KB Archive/908573

= A POST or PUT request may fail when you use the HttpWebRequest class to send lots of data on a computer that is running the .NET Framework =

Article ID: 908573

Article Last Modified on 11/22/2007

-

APPLIES TO


 * Microsoft .NET Framework 1.1
 * Microsoft .NET Framework 1.0

-





SYMPTOMS
When you use the HttpWebRequest class to send lots of data by using a POST or PUT request, the request may fail on a computer that is running the Microsoft .NET Framework. Additionally, you may receive an out-of-memory or time-out error message.

If you do not receive an out-of-memory or time-out error message, you may notice that the application that uses the HttpWebRequest class uses lots of memory. When you use Performance Monitor to monitor the application that uses the HttpWebRequest class, the Private Bytes count will continue to increase as data is sent. Therefore, you may also experience slow performance on the computer and in other applications because increased memory and resource utilization occurs.

Note The amount of data that can be uploaded by default will vary according to the memory and the resources that are available on the computer.



CAUSE
This issue occurs because the .NET Framework buffers the outgoing data by default when you use the HttpWebRequest class.



WORKAROUND
To work around this issue, set the HttpWebRequest.AllowWriteStreamBuffering property to false. However, you may receive an error message that is similar to the following when you set the HttpWebRequest.AllowWriteStreamBuffering property to false:

This request requires buffering of data for authentication of redirection to be successful.

To successfully send lots of data by using a POST or PUT request when the HttpWebRequest.AllowWriteStreamBuffering property is set to false, use one of the following methods, depending on the authentication method that you want to use.

Anonymous authentication
If the Web server is configured to use Anonymous authentication, set the HttpWebRequest.AllowWriteStreamBuffering property to false. No other changes are needed.

Basic authentication
If the Microsoft Internet Information Services (IIS) Web server is configured to use Basic authentication, and you must set the HttpWebRequest.AllowWriteStreamBuffering property to false, you must send a HEAD request to pre-authenticate the connection before you send the POST or PUT request. You must also set the HttpWebRequest.PreAuthenticate property to true. Then, send the POST or PUT request, and then receive the response. To do this, use code that is similar to the following code example. public void test(Uri URL) {           HttpWebRequest WRequest; HttpWebResponse WResponse; //preAuth the request // You can add logic so that you only pre-authenticate the very first request. // You should not have to pre-authenticate each request. WRequest = (HttpWebRequest)HttpWebRequest.Create(URL); // Set the username and the password. WRequest.Credentials = new NetworkCredential(user, password); WRequest.PreAuthenticate = true; WRequest.UserAgent = &quot;Upload Test&quot;; WRequest.Method = &quot;HEAD&quot;; WRequest.Timeout = 10000; WResponse = (HttpWebResponse)WRequest.GetResponse; WResponse.Close; // Make the real request. WRequest = (HttpWebRequest)HttpWebRequest.Create(URL); // Set the username and the password. WRequest.Credentials = new NetworkCredential(user, password); WRequest.PreAuthenticate = true; WRequest.UserAgent = &quot;Upload Test&quot;; WRequest.Method = &quot;POST&quot;; WRequest.AllowWriteStreamBuffering = false; WRequest.Timeout = 10000; FileStream ReadIn = new FileStream(c:\\testuploadfile.txt, FileMode.Open, FileAccess.Read); ReadIn.Seek(0, SeekOrigin.Begin); // Move to the start of the file. WRequest.ContentLength = ReadIn.Length; // Set the content length header to the size of the file. Byte[] FileData = new Byte[ReadIn.Length]; // Read the file in 2 KB segments. int DataRead = 0; Stream tempStream = WRequest.GetRequestStream; do           { DataRead = ReadIn.Read(FileData,0,2048); if (DataRead > 0) //we have data {                   tempStream.Write(FileData,0,DataRead); Array.Clear(FileData,0, 2048); // Clear the array. }           }while(DataRead > 0); WResponse = (HttpWebResponse)WRequest.GetResponse; // Read your response data here. // Close all streams. ReadIn.Close; tempStream.Close; WResponse.Close; } Note Depending on how the application is designed, you may not have to pre-authenticate each request by sending a HEAD request.

Integrated Windows authentication
You can configure a computer on which IIS is installed to respond by using Negotiate or NTLM Windows authentication. If IIS is configured to use Negotiate for Windows authentication, the client can use either Kerberos or NTLM to authenticate. If IIS is configured to use NTLM authentication, only NTLM authentication can be used, and Kerberos authentication is not supported.

If Negotiate with Kerberos authentication is used, use the following workaround. The workaround will fail if NTLM is used.

Negotiate with Kerberos authentication
If the IIS Web server is configured to use Negotiate authentication and you must set the HttpWebRequest.AllowWriteStreamBuffering property to false, you must send a HEAD request to pre-authenticate the connection before you send the POST or PUT request. You must also set the HttpWebRequest.PreAuthenticate property to true. Additionally, you may have to set the HttpWebRequest.UnsafeAuthenticatedConnectionSharing property to true. Then, send the POST or PUT request, and then receive the response. To do this, you can use code that is similar to the following code example.

Note This workaround will fail if the client cannot use Kerberos with Negotiate authentication. You also must make sure that the HttpWebRequest.KeepAlive property is set to true. By default, the setting for the HttpWebRequest.KeepAlive property is true. The logic for Kerberos and Basic authentication is almost the same. public void test(Uri URL) {           HttpWebRequest WRequest; HttpWebResponse WResponse; CredentialCache myCredCache = new CredentialCache; myCredCache.Add(URL,&quot;Negotiate&quot;,(NetworkCredential) CredentialCache.DefaultCredentials); // Pre-authenticate the request. WRequest = (HttpWebRequest)HttpWebRequest.Create(URL); // Set the username and the password. WRequest.Credentials = myCredCache; // This property must be set to true for Kerberos authentication. WRequest.PreAuthenticate = true; // Keep the connection alive. WRequest.UnsafeAuthenticatedConnectionSharing = true; WRequest.UserAgent = &quot;Upload Test&quot;; WRequest.Method = &quot;HEAD&quot;; WRequest.Timeout = 10000; WResponse = (HttpWebResponse)WRequest.GetResponse; WResponse.Close; // Make the real request. WRequest = (HttpWebRequest)HttpWebRequest.Create(URL); // Set the username and the password. WRequest.Credentials = myCredCache; // This property must be set to true for Kerberos authentication. WRequest.PreAuthenticate = true; // Keep the connection alive. WRequest.UnsafeAuthenticatedConnectionSharing = true; WRequest.UserAgent = &quot;Upload Test&quot;; WRequest.Method = &quot;POST&quot;; WRequest.AllowWriteStreamBuffering = false; WRequest.Timeout = 10000; FileStream ReadIn = new FileStream(&quot;c:\\ testuploadfile.txt &quot;, FileMode.Open, FileAccess.Read); ReadIn.Seek(0, SeekOrigin.Begin); // Move to the start of the file. WRequest.ContentLength = ReadIn.Length; // Set the content length header to the size of the file. Byte[] FileData = new Byte[ReadIn.Length]; // Read the file in 2 KB segments. int DataRead = 0; Stream tempStream = WRequest.GetRequestStream; do           { DataRead = ReadIn.Read(FileData,0,2048); if (DataRead > 0) // We have data. {                   tempStream.Write(FileData,0,DataRead); Array.Clear(FileData,0, 2048); // Clear the array. }           }while(DataRead > 0); WResponse = (HttpWebResponse)WRequest.GetResponse; // Read your response data here. // Close all streams ReadIn.Close; tempStream.Close; WResponse.Close; } Note Depending on how the application is designed, you may not have to pre-authenticate each request by sending a HEAD request.

NTLM authentication
If the IIS Web server is also configured to use NTLM authentication with Windows Integrated authentication, and you must set the HttpWebRequest.AllowWriteStreamBuffering property to false, you can set the authentication type to NTLM in the client code. After you configure IIS to use both Negotiate and NTLM authentication and you set the authentication type to NTLM in the client code, you can configure how IIS handles authentication requests by setting the AuthPersistSingleRequest property in the IIS metabase to false.

Note For more information about how to configure IIS to support both Negotiate and NTLM authentication, see the &quot;References&quot; section.

You must also send a HEAD request to pre-authenticate the connection before you send the POST request and set the HttpWebrequest.UnsafeAuthenticatedConnectionSharing property to true. Then, set the HttpWebRequest.PreAuthenticate property to false. Finally, send the POST or PUT request, and then receive the response. To do this, use code that is similar to the following code example. public void test(Uri URL) {           HttpWebRequest WRequest; HttpWebResponse WResponse; CredentialCache myCredCache = new CredentialCache; myCredCache.Add(URL,&quot;NTLM&quot;,(NetworkCredential) CredentialCache.DefaultCredentials); // Pre-authenticate the request. WRequest = (HttpWebRequest)HttpWebRequest.Create(URL); // Set the username and the password. WRequest.Credentials = myCredCache; // For NTLM authentication, you must set the following property to true // so the connection does not close. WRequest.UnsafeAuthenticatedConnectionSharing = true; WRequest.UserAgent = &quot;Upload Test&quot;; WRequest.Method = &quot;HEAD&quot;; WRequest.Timeout = 10000; WResponse = (HttpWebResponse)WRequest.GetResponse; WResponse.Close; // Make the real request. WRequest = (HttpWebRequest)HttpWebRequest.Create(URL); // Set the username and the password. WRequest.Credentials = myCredCache; // For NTLM authentication, you must set the following property to true // so the connection does not close. WRequest.UnsafeAuthenticatedConnectionSharing = true; WRequest.UserAgent = &quot;Upload Test&quot;; WRequest.Method = &quot;POST&quot;; WRequest.AllowWriteStreamBuffering = false; WRequest.Timeout = 10000; FileStream ReadIn = new FileStream(&quot;c:\\ testuploadfile.txt&quot;, FileMode.Open, FileAccess.Read); ReadIn.Seek(0, SeekOrigin.Begin); // Move to the start of the file. WRequest.ContentLength = ReadIn.Length; // Set the content length header to the size of the file. Byte[] FileData = new Byte[ReadIn.Length]; // Read the file in 2 KB segments. int DataRead = 0; Stream tempStream = WRequest.GetRequestStream; do           { DataRead = ReadIn.Read(FileData,0,2048); if (DataRead > 0) // We have data. {                   tempStream.Write(FileData,0,DataRead); Array.Clear(FileData,0, 2048); // Clear the array. }           }while(DataRead > 0); WResponse = (HttpWebResponse)WRequest.GetResponse; // Read your response data here. // Close all streams. ReadIn.Close; tempStream.Close; WResponse.Close;

} Note Depending on how the application is designed, you may not have to pre-authenticate each request by sending a HEAD request.

