Microsoft KB Archive/197728

= Creating a Thread Pool for MFC-Based ISAPI Extensions =

Article ID: 197728

Article Last Modified on 8/5/2004

-

APPLIES TO


 * Microsoft Internet Server Application Programming Interface 4.0
 * Microsoft Internet Information Services 5.0
 * Microsoft Visual C++ 6.0 Professional Edition
 * Microsoft Visual Basic 6.0 Enterprise Edition

-



This article was previously published under Q197728



SUMMARY
Afxpool.exe is a self-extracting executable that contains the necessary files for a Microsoft Visual C++ 6.0 project that demonstrates the technique of creating a thread pool for an MFC-based ISAPI extension.

This article explains the steps to create a worker thread pool within an ISAPI extension. Often when an ISAPI extension requires a significant amount of processing time to complete a request, it is necessary to create a separate thread of control to handle the processing. For additional information, please see the following article in the Microsoft Knowledge Base:

185518 HOWTO: Implement Worker Threads Using MFC ISAPI Extension

However, creating a new thread of control for each new request can cause a significant performance impact on the server. For this reason, it's advantageous to create a pool of worker threads from which a thread can be "recycled." The affect of this is that the cost of creating a new thread is removed from the cost of handling a new request.

The ISAPI extension will incur the cost of creating the threads when it first initializes but for each request it will merely request an available worker thread from the pool.



MORE INFORMATION
The following files are available for download from the Microsoft Download Center:

Afxpool.exe

For additional information about how to download Microsoft Support files, click the following article number to view the article in the Microsoft Knowledge Base:

119591 How to Obtain Microsoft Support Files from Online Services

Microsoft scanned this file for viruses. Microsoft used the most current virus-detection software that was available on the date that the file was posted. The file is stored on security-enhanced servers that help to prevent any unauthorized changes to the file.   FileName               Size -

ExtThreadPool.aps      18,436 ExtThreadPool.clw      420 ExtThreadPool.cpp      6,184 ExtThreadPool.def      173 ExtThreadPool.dsp      4,332 ExtThreadPool.dsw      549 ExtThreadPool.h        1,560 ExtThreadPool.ncb      66,560 ExtThreadPool.opt      54,784 ExtThreadPool.plg      723 ExtThreadPool.rc       3,242 ExtThreadPool.rc2      405 Resource.h             322 StdAfx.cpp             215 StdAfx.h               669 ThreadPool.cpp         3,130 ThreadPool.h           863 The thread pool uses completion ports available under the Windows NT and Windows 2000 family of operating systems. The basic idea behind this thread pool is to create a single completion port, have all of the workers wait on this completion port, and when a new request comes in, post the request to this completion port. The completion port mechanism ensures that only one thread of control will be notified of this new request.

It possible to create and manage the thread pool without using completion ports with other synchronization objects, such as semaphores. However with completion ports the operating system takes a burden of scheduling and releasing threads in the most efficient manner (that is, Last In First Out).

In a typical ISAPI extension, there are two virtual functions created by the ISAPI MFC Wizard: GetExtensionVersion and TerminateExtension. The initialization of the completion port is accomplished in the GetExtensionVersion with the call to CreateIoCompletionPort. Note that the last parameter to this function allows you to specify how many threads from the pool can run concurrently. The most efficient way is to have as many running threads as CPUs. Last parameter 0 will allow the operating system to pick the number of threads matching the number of CPUs. BOOL CExtThreadPoolExtension::GetExtensionVersion(     HSE_VERSION_INFO* pVer) {     // Call default implementation for initialization CHttpServer::GetExtensionVersion(pVer);

// Load description string TCHAR sz[HSE_MAX_EXT_DLL_NAME_LEN+1]; ISAPIVERIFY(::LoadString(AfxGetResourceHandle, IDS_SERVER, sz, HSE_MAX_EXT_DLL_NAME_LEN)); _tcscpy(pVer->lpszExtensionDesc, sz);

m_phThreads = new HANDLE[THREAD_COUNT];

m_nThreadCount = THREAD_COUNT; m_hIoPort = CreateIoCompletionPort((HANDLE)INVALID_HANDLE_VALUE,        NULL, 0, 0);

// NOTE: If a CWinApp object is not declare, this call should be      // changed to CreateThread instead. for(long n = 0; n < m_nThreadCount; n++) {        CWinThread* pWinThread = AfxBeginThread(            (AFX_THREADPROC)ThreadProc, m_hIoPort); m_phThreads[n] = pWinThread->m_hThread; }

return TRUE; } Here, you have three variables defined in the CExtThreadPoolExtension class: m_nThreadCount, m_hIoPort, and m_phThreads. m_nThreadCount is a type long which defines the number of threads to create within our thread pool (this value is hard coded in the ThreadPool.h file to the THREAD_COUNT variable). The m_hIoPort is the single completion port handle you will send to all of the worker threads to block on. And m_phThreads is the array of thread handles from the worker threads.

It should be noted that this sample is built using MFC in a shared library. If you create an MFC ISAPI Extension with the wizard as statically linked library, the instance of CWinApp won't be available. Because of this, the AfxBeginThread call should be changed to CreateThread instead.

Although the MFC ISAPI Wizard creates a GetExtensionVersion and TerminateExtension method, it is necessary to manually override the HttpExtensionProc method of the CHttpServer as well. This can be accomplished by adding the method in the CExtThreadPoolExtension declaration: class CExtThreadPoolExtension : public CHttpServer {  public: CExtThreadPoolExtension; ~CExtThreadPoolExtension;

// Overrides // ClassWizard generated virtual function overrides // NOTE - the ClassWizard will add and remove member // functions here. //   DO NOT EDIT what you see in these blocks of generated code ! //AFX_VIRTUAL virtual BOOL TerminateExtension(DWORD dwFlags);

// TODO: Add handlers for your commands here. // For example:

void Default(CHttpServerContext* pCtxt);

DECLARE_PARSE_MAP

//AFX_MSG

protected: HANDLE*          m_phThreads; HANDLE           m_hIoPort; long             m_nThreadCount; }; This is necessary because you must override the return value from the CHttpServer::HttpExtensionProc to let IIS know that you're still processing this request, do not recycle the EXTENSION_CONTROL_BLOCK.

The overridden HttpExtensionProc is fairly simple: DWORD CExtThreadPoolExtension::HttpExtensionProc(     EXTENSION_CONTROL_BLOCK *pECB) {     // The result of the base class's (CHttpServer) HttpExtensionProc // must be checked. If it's successful, you will need to return a     // HSE_STATUS_PENDING. This is so that IIS will not shutdown the // connection and not destroy the ECB. // If the result of the base class is not successful, you should // simply propagate the error message up to IIS.

DWORD dwResult = CHttpServer::HttpExtensionProc(pECB); if(dwResult == HSE_STATUS_SUCCESS) return HSE_STATUS_PENDING; else return dwResult; } You merely check the return call from the CHttpServer::HttpExtensionProc and return pending if successful. Otherwise, just propagate the error up.

In the Default implementation (or whatever routine your MFC ISAPI Extension calls in your message map), take the EXTENSION_CONTROL_BLOCK (note this is not the same as the CHttpServerContext object; you can't pass the CHttpServerContext object to the worker thread since that object's existence is only within the scope of the CHttpServer::HttpExtensionProc) and post it to the completion port you created in GetExtensionVersion: void CExtThreadPoolExtension::Default(CHttpServerContext* pCtxt) {     if(!PostQueuedCompletionStatus(m_hIoPort, (DWORD) pCtxt->m_pECB, 0, NULL)) {        char  szBuffer[4096] = { 0 };

sprintf(szBuffer, "Error posting to completion port!           Win32 Error = %i", GetLastError); DWORD dwBufLen = strlen(szBuffer); pCtxt->m_pECB->WriteClient(pECB->ConnID, szBuffer,           &dwBufLen, 0); }  } If the completion port post fails, you need to generate an error message and report it back to the client. Please note that to output the error WriteClient callback directly from the ECB instead of the MFC wrapper is used.

This sample also demonstrates a way to pass various parameters to the worker thread. In addition to Default, you also have ProcessName and ProcessSample. ProcessNametakes as its parameters three strings: First name, Last name, and Middle name. ProcessSample takes two parameters: Data and String. Data will be a type of integer and String is any string data.

The idea behind this is to take advantage of the parse map support from MFC ISAPI extension and allow for the worker threads to handle the appropriate requests. To accomplish this, you need to create two structures: struct ProcessNameData {     EXTENSION_CONTROL_BLOCK*   pECB; CString                   sFirstName; CString                   sLastName; CString                   sMiddleName; };

struct ProcessSampleData {     EXTENSION_CONTROL_BLOCK*   pECB; DWORD                     nValue; CString                   sString; }; ProcessNameData structure will be used to temporarily hold the parameters passed to ProcessName function by the MFC parse map. Within the ProcessName function, you dynamically allocate a new ProcessNameData structure, set the EXTENSION_CONTROL_BLOCK and the parameters passed to us and pass it off to the worker thread. But when you pass the data off to the worker thread, you have to let the worker thread know that the data you are passing is of type ProcessNameData. This is accomplished by third parameter in the PostQueuedCompletionStatus API. In the Default function, you passed 0 as the third parameter. For ProcessName, you're passing 1: void CExtThreadPoolExtension::ProcessName(CHttpServerContext* pCtxt,                                            LPCTSTR szFirstName,                                             LPCTSTR szLastName,                                             LPCTSTR szMiddleName) {     ProcessNameData*  pData = new ProcessNameData;

pData->pECB = pCtxt->m_pECB; pData->sFirstName = szFirstName; pData->sLastName = szLastName; pData->sMiddleName = szMiddleName;

if(!PostQueuedCompletionStatus(m_hIoPort, (DWORD) pData, (DWORD) 1, NULL)) {        char  szBuffer[4096] = { 0 };

sprintf(szBuffer,           "Error posting to completion port! Win32 Error = %i",            GetLastError); DWORD dwBufLen = strlen(szBuffer); pCtxt->m_pECB->WriteClient(pCtxt->m_pECB->ConnID, szBuffer,           &dwBufLen, 0); }  } By passing a value of 1 to the worker thread, the worker thread will know that it needs to process the data differently than from the Default routine.

Similarly, you pass a value of 2 for the ProcessSample function. And in this function, you would create a new instance of ProcessSampleData structure.

Within the ThreadProc (the actual routine that the worker thread executes), you need to have all of the threads block on a single completion port (which is passed to the thread routine from GetExtensionVersion). This is done when the thread calls GetQueuedCompletionStatus. If no status is posted, the call will block until one is posted. When the completion port is posted, one of the worker threads will "awaken" and begin processing: unsigned int ThreadProc(void* p)  { HANDLE                       IoPort   = (HANDLE*)p; unsigned long                pN1, pN2; OVERLAPPED*                  pOverLapped;

while(GetQueuedCompletionStatus(IoPort, &pN1, &pN2, &pOverLapped, INFINITE)) {        if(pOverLapped == (OVERLAPPED*)0xFFFFFFFF) break; else {           switch(pN2) {           case 0: DoWork((EXTENSION_CONTROL_BLOCK*)pN1); break; case 1: DoWork((ProcessNameData*)pN1); break; case 2: DoWork((ProcessSampleData*)pN1); break; default: // Huh?!?!? break; }        }      }

return 0; } As you can see, in the switch statement, you test for the correct pN2 value (this is the third parameter you passed in the PostQueuedCompletionStatus call). It should also be pointed out that there are three versions of DoWork. One version will take a EXTENSION_CONTROL_BLOCK pointer, another takes a ProcessNameData pointer, and a third takes ProcessSampleData pointer. By explicitly casting pN1, you can make sure you are calling the correct version for the data provided.

It is necessary for the version of DoWork, which takes the ProcessNameData and ProcessSampleData, to destroy those structure before exiting. This is because those structures were created for temporarily holding the parse map parameters: void DoWork(ProcessNameData* pData) {     EXTENSION_CONTROL_BLOCK*   pECB = pData->pECB; CString sBody;

// Build the HTML body... sBody.Format(        "This is thread %i in thread pool.  Hi there!",         GetCurrentThreadId); sBody += "I'm processing this for "; sBody += pData->sFirstName; sBody += " "; if(pData->sMiddleName.CompareNoCase("~") != 0) {        sBody += pData->sMiddleName; sBody += " "; }     sBody += pData->sLastName; sBody += "";

// Pretend we're spending some time processing this request // this sample assumes 3 seconds. Sleep( 3000 );

SendData(pECB, sBody);

delete pData; }

void DoWork(ProcessSampleData* pData) {     EXTENSION_CONTROL_BLOCK*   pECB = pData->pECB; CString sBody;

// Build the HTML body... sBody.Format(        "This is thread %i in thread pool.  Hi there!",         GetCurrentThreadId); sBody += "This is the sample data: "; sBody += pData->sString;

CString sTmp; sTmp.Format(" (%i)</H2>", pData->nValue);

sBody += sTmp;

// Pretend we're spending some time processing this request // this sample assumes 3 seconds. Sleep( 3000 );

SendData(pECB, sBody);

delete pData; } SendData is simple function that sends back a CString to the client.

However, before you start processing, you need to make sure the thread isn't being signaled to terminate. This is done by testing the value of the pOverLapped pointer. If the overlapped pointer's value is 0xFFFFFFFF (an invalid handle value), you know that the thread is suppose to terminate. So the routine will break out of the while loop.

To signal the worker threads to terminate, use the third virtual function created by the MFC ISAPI Wizard: TerminateExtension. In this function, you need to signal all of the threads to close and close out our completion port as well: BOOL CExtThreadPoolExtension::TerminateExtension(DWORD dwFlags) {     // extension is being terminated for(long n = 0; n < m_nThreadCount; n++) PostQueuedCompletionStatus(m_hIoPort, 0,           0, (OVERLAPPED*)0xFFFFFFFF);

WaitForMultipleObjects(m_nThreadCount, m_phThreads, TRUE, 120000); CloseHandle(m_hIoPort); delete [] m_phThreads; return TRUE; } To signal all of the threads to close, post with an invalid handle value an overlapped pointer. Do this for the number of threads available. This ensures that each thread receives the notification. Once that's done, wait for all of the threads to exit by calling WaitForMultipleObjects call on the handles returned from the AfxBeginThread call (the handles were copied from the CWinThread object in GetExtensionVersion). In this sample, you will wait two minutes (120,000 milliseconds) for all of the threads to exit.

This will ensure that when you terminate the ISAPI extension, you are cleaning up before you map the DLL out of memory.

NOTE: All threads created do not run in the same security context as that of the ISAPI extension thread. This is because the security context of the ISAPI extension thread is valid only for that thread. To pass the security context over, you will need to pass the security token along with the parse map parameters. For example, if you need to pass the thread security token to the worker for ProcessName, you would need to modify the ProcessNameData structure to: struct ProcessNameData {     EXTENSION_CONTROL_BLOCK*   pECB; CString                   sFirstName; CString                   sLastName; CString                   sMiddleName; HANDLE                    hThreadToken; }; And the ProcessName function needs to be changed to the following: void CExtThreadPoolExtension::ProcessName(CHttpServerContext* pCtxt,                                            LPCTSTR szFirstName,                                             LPCTSTR szLastName,                                             LPCTSTR szMiddleName) {     ProcessNameData*  pData = new ProcessNameData;

OpenThreadToken(GetCurrentThread, TOKEN_ALL_ACCESS, FALSE,        &(pData->hThreadToken)); pData->pECB = pCtxt->m_pECB; pData->sFirstName = szFirstName; pData->sLastName = szLastName; pData->sMiddleName = szMiddleName;

if(!PostQueuedCompletionStatus(m_hIoPort, (DWORD) pData, (DWORD) 1, NULL)) {        char  szBuffer[4096] = { 0 };

sprintf(szBuffer,           "Error posting to completion port! Win32 Error = %i</H1>",            GetLastError); DWORD dwBufLen = strlen(szBuffer); pCtxt->m_pECB->WriteClient(pCtxt->m_pECB->ConnID, szBuffer,           &dwBufLen, 0); }  } Simiarly, in the DoWork, we need to impersonate the thread context: void DoWork(ProcessNameData* pData) {     ImpersonateLoggedOnUser(pData->hThreadToken);

EXTENSION_CONTROL_BLOCK*  pECB = pData->pECB; CString sBody;

// Build the HTML body... sBody.Format(        "This is thread %i in thread pool.  Hi there!</H1>",         GetCurrentThreadId); sBody += "<HR>I'm processing this for "; sBody += pData->sFirstName; sBody += " "; if(pData->sMiddleName.CompareNoCase("~") != 0) {        sBody += pData->sMiddleName; sBody += " "; }     sBody += pData->sLastName; sBody += "</H2>";

// Pretend we're spending some time processing this request // this sample assumes 3 seconds. Sleep( 3000 );

SendData(pECB, sBody);

delete pData; CloseHandle (pData->hThreadToken);

} The included sample does not implement this. The sample's worker threads will all run in the context of the process. If the ISAPI extension is executed in process, it will run as Local System. If the ISAPI extension is executed out of process, it will run as the IWAM_<MachineName> user.

<div class="references_section">