Microsoft KB Archive/818894

= Changes to increase the number of open files in Windows 2000, in Windows 2003, and in Windows XP =

Article ID: 818894

Article Last Modified on 12/3/2007

-

APPLIES TO


 * Microsoft Windows Server 2003, Standard Edition (32-bit x86)
 * Microsoft Windows Server 2003, Enterprise Edition (32-bit x86)
 * Microsoft Windows Server 2003, Datacenter Edition (32-bit x86)
 * Microsoft Windows XP Home Edition
 * Microsoft Windows XP Professional
 * Microsoft Windows 2000 Datacenter Server
 * Microsoft Windows 2000 Server
 * Microsoft Windows 2000 Advanced Server
 * Microsoft Windows Small Business Server 2003 Standard Edition
 * Microsoft Windows Small Business Server 2003 Premium Edition

-



SUMMARY
This article describes a memory management change that was introduced in Windows 2000 Service Pack 2 (SP2) that increases the scalability and performance of Windows. This change is included in all later versions of Windows 2000, Microsoft Windows XP, and Windows Server 2003.



MORE INFORMATION
Starting with Windows 2000 SP2, the prototype Page Table Entry (PTE) algorithm has been changed to permit the creation of data sections for larger files than was possible with earlier versions of Windows. This change was introduced to increase the scalability of Windows, making it possible to create much larger data sections for large files, even on computers with a relatively small amount of physical memory. This allows the backing up of very large databases and also increases the potential sizes of Web servers and mail servers in your organization.

Before this change, in Windows 2000, all prototype PTEs that are required for mapping files in memory are created at the time of section (file mapping) creation. This is a costly algorithm in terms of paged pool memory. With the new algorithm, introduced in Windows 2000 SP2, and used in later versions of Windows 2000, Windows XP, and Windows Server 2003, no prototype PTEs are created at section creation time. Instead, they are created when any view of a subsection is mapped (for example, the mapping of the address space of a process, the system cache, or the system or session view spaces). To restate this:
 * In Windows 2000 SP1 and earlier, the memory manager allocates paged pool memory for all the PTEs to describe a section when the section is created. This consumes large amounts of paged pool memory.
 * In Windows 2000 SP2 and later, Windows XP, and Windows Server 2003, the memory manager allocates the PTEs to describe a view when the view of the section is mapped.

The following scenario example helps to illustrate this change:
 * hSection = CreateFileMapping( size = 100 GB)
 * MapViewOfFile(hSection, size = 1 GB)

In the first case, by using the original prototype PTE algorithm in Windows 2000 SP1 and earlier, the Windows memory manager allocates about 100 megabytes (MB) of paged pool memory to map the whole 100 gigabytes (GB). In the second case, by using the new algorithm in Windows 2000 SP2 and later, only about 1 percent of paged pool memory is used (1 MB in this example).

Additionally, when all the mapped views for a particular subsection are unmapped, the Windows memory manager does not remove the corresponding prototype PTEs immediately. This is to increase performance. The PTEs are just marked as unused. If the same subsection is mapped again later, the Windows memory manager will reuse the same prototype PTEs. If Windows starts to run out of paged or non-paged pool memory, the memory manager removes the unused prototype PTEs to free the memory. The PTEs are then dynamically re-created if the subsection is accessed again.

Note When Windows is configured to use Physical Address Extensions (PAE) memory addressing (and also with the AMD64 architecture specification), the prototype PTE cost is actually doubled from the cost that is used with an x86-based computer or an IA64-based computer. Using the previous example, in the first case (Windows 2000 SP1 and earlier), 200 MB of paged pool memory would be consumed, and for the second case (Windows 2000 SP2 and later, Windows XP, and Windows Server 2003), 2 MB of paged pool memory would be consumed.

This behavior occurs because this usage of paged pool memory is based on a ratio of PTE width (in bytes) to the page size (in kilobytes). x86 architecture has 4-KB pages and 4-byte PTEs for a ratio of 1024:1. IA64 architecture has 8-KB pages and 8-byte PTEs for a ratio of 1024:1. PAE and AMD64 architecture has 4-KB pages and 8-byte PTEs for a ratio of 512:1.

