Microsoft KB Archive/154116

= How To Debug OLE Applications =

Article ID: 154116

Article Last Modified on 7/1/2004

-

APPLIES TO

 The Integrated Debugger, when used with:  Microsoft Visual C++ 2.0 Professional Edition

 Microsoft Visual C++ 2.1

 Microsoft Visual C++ 2.2

 Microsoft Visual C++ 4.0 Standard Edition</li></ul>

 Microsoft Visual C++ 4.1 Subscription</li></ul>

 Microsoft Visual C++ 4.2 Enterprise Edition</li></ul>

 Microsoft Visual C++ 4.2 Professional Edition</li></ul> </li></ul>

-

<div class="notice_section">

This article was previously published under Q154116

<div class="notice_section">

<div class="summary_section">

SUMMARY
This article provides a tutorial to assist you in learning how to use the Visual C++ integrated debugger to debug OLE applications.

The Visual C++ integrated debugger supports simultaneous debugging of OLE client and server applications. You can easily step across and into OLE clients and servers and have the ability to step across OLE Remote Procedure Calls. A second instance of the debugger is automatically spawned the first time an OLE client calls into an OLE server.

When building OLE servers, you may want to debug them in the context of being activated by an OLE container, thus debugging both the container and the server at the same time. This tutorial provides an example of how to debug an OLE server when the main debuggee is an OLE container. It shows how a second instance of the debugger is automatically spawned when an OLE client calls into an OLE server. The tutorial is designed to lead you step-by-step through the code that creates the OLE server object and establishes the connection between the OLE container and the OLE server. This is useful in tracking down problems that occur when the OLE server does not get created or initialized correctly.

All regular debugging facilities are available as you debug your OLE application.

<div class="moreinformation_section">

Preparing to Debug
To prepare for OLE Client/Server debugging:


 * 1) Open the project for the OLE application and build a version with symbolic debugging information.
 * 2) From the Tools menu, choose Options. Select the Debug tab. Ensure that the OLE RPC Debugging check box and the Just-In-Time (JIT) Debugging boxes are checked. (You must have Windows NT administrator privileges to enable OLE RPC Debugging.)
 * 3) Choose OK. The information is now stored in the registry.
 * 4) Set breakpoints at the points in the source files for your OLE application where you want to determine the state of the application.
 * 5) Start a debugging session.

Viewing Derived-Most Types
The QuickWatch dialog box provides support for the automatic downcast of pointers in OLE and MFC debugging. The debugger automatically detects when a pointer is pointing to a subclass of the type it is required to point to. When the pointer is expanded, Visual C++ will add an extra member that looks like another base class and indicates the derived-most type. For example, if you are displaying a pointer to a CObject and the pointer really points to a CComboBox, the QuickWatch expression evaluator will recognize this and introduce a pseudo CComboBox member so you can access the CComboBox members.

The rest of this tutorial takes you through a debug session, using MFC sample code.

Creating the Object
<ol> Build debug versions of the Microsoft Foundation Classes (MFC) samples in the development environment. For Visual C++ 2.x, the projects to work with are in the following directories on the CD-ROM:

 MSVC20\SAMPLES\MFC\CONTAIN\STEP2</li> MSVC20\SAMPLES\MFC\SCRIBBLE\STEP7</li></ul>

For Visual C++ 4.x, the projects to work with are in the following directories on the CD-ROM:

 MSDEV\SAMPLES\MFC\TUTORIAL\CONTAIN\STEP2</li> MSDEV\SAMPLES\MFC\TUTORIAL\SCRIBBLE\STEP7</li></ul>

For Visual C++ 5.0 and later, use the online Help system to copy the samples and use the default directories for copying. The sample names are same, CONTAINER and SCRIBBLE.</li> Run the SCRIBBLE.EXE file built in step 1 to update the registry to point to this executable file.</li> Load the CONTAIN\STEP2 project into the development environment.</li> From the Tools menu, choose Options. Select the Debug tab, and make sure OLE RPC Debugging is enabled as described in step 4 of the "Preparing to Debug" section in this article.</li> <li>Open the CONTRVW.CPP file in CONTAIN\STEP2, and set a breakpoint on the line that contains a call to CreateItem.</li> <li>Start a debugging session to run CONTAIN.EXE. In the CONTAIN main menu, choose Insert New Object from the Edit menu.</li> <li>From the resulting Object Type list, choose Scrib Document, and then choose the OK button. At this point, the debugger should stop at the breakpoint you set in step 5. This is the call to the Insert Object dialog's CreateItem function. The purpose of CreateItem is to create and initialize an object of the type you selected from the dialog box. The CreateItem function is passed a CCntrItem object and uses the object to handle this process.

NOTE: For a brief overview of what CCntrItem does, see its class definition in CNTRITEM.H. Then look at the definition of COleClientItem (from which CCntrItem is derived) in AFXOLE.H.</li> <li>Step into the call to CreateItem. You are in the MFC source file OLEDLGS1.CPP. Step over instructions until you get to the call to pNewItem->CreateItem.</li> <li>Step into the call to pNewItem->CreateNewItem. Then step over instructions until you get to the call to OleCreate. While stepping through the code, read the comments and note that storage is allocated for the object and its rendering format is established.</li> <li>Step into the call to OleCreate. At this point, execution proceeds through the RPC mechanism to the server code itself. Therefore, as the server code begins to execute, a new instance of the debugger is created in which to debug the server. A pseudo project for SCRIBBLE.EXE is loaded (as in JIT debugging), and the instruction pointer is set at the call to COleServerDoc::XPersistStorage::InitNew in OLESRV1.CPP. If you installed the .DBG files (see the "NT System Symbols Setup" icon in your Visual C++ program group), your callstack will include fully decorated names.</li> <li>Step over instructions in Scribble's InitNew until you reach the call to pThis->OnNewEmbedding. Then step into OnNewEmbedding.</li> <li>Step over lines until you reach the call to OnNewDocument. Then step into the call to COleServerDoc::OnNewDocument. You are now inside COleLinkingDoc::OnNewDocument. (COleServerDoc is derived from COleLinkingDoc.) Note the code in this small function: It creates a new document object and attaches it to the server (Scribble).</li> <li>Step out twice to get back into COleServerDoc::XPersistStorage::InitNew, where you first came into Scribble.</li> <li>Step out one more time. This will cause the container to return from its call to OleCreate, the function that first took you into Scribble. At this time, the instance of the debugger that has Contain loaded gets the focus, and you are back in OLECLI1.CPP immediately following the call to OleCreate. The embedded Scribble object has now been created but it is not yet fully initialized.</li> <li>Step into the next line, which is the call to FinishCreate. Step through the FinishCreate code to see how OLE finalizes the connection between the container and the server, and then step out of FinishCreate.</li></ol>

A Scribble object has now been created and initialized in memory, but it is not yet editable in the container; Scribble hasn't been activated. In fact, Contain still has only an IUnknown interface to Scribble. You can see this by expanding the lpClientSite variable in the Locals Window.

Activating the Object
<ol> <li> Step out two more times to get back to Contain's OnInsertObject function in CONTRVW.CPP. </li> <li>In Contain's OnInsertObject function, step over five times to get to the call to DoVerb. This function activates Scribble.</li> <li>Step into the call to DoVerb. Then step over a few lines until you come to the call to Activate.</li> <li>Step into Activate. You are now in OLECLI3.CPP. Before going on, scan through the code for Activate. Notice that a rectangle is first created for the embedded Scribble item to live in. GetClientSite is then called to establish an interface back to the container for the Scribble server. Then the server's DoVerb function is called to pass both of these to the server.</li> <li>Step into the call to DoVerb. At this point, execution proceeds once again through the RPC mechanism to the server code itself. As you would expect, the instruction pointer is pointing to the first instruction in the server's COleServerDoc:: XOleObject::DoVerb function.</li> <li>Step to the call to OnDoVerb. Then step into OnDoVerb. OnDoVerb consists of a switch statement that executes the command (verb) passed to it. In this case, the command is OLEIVERB_SHOW. Step over instructions until you get to OnShow.</li> <li>Step into OnShow. Then step three lines to ActivateInPlace, and step into ActivateInPlace.

The ActivateInPlace function does many things and is worth examining in detail. While it is beyond the scope of this tutorial to go into all the details, it is worthwhile to step through the code and observe the comments. At this point, step over each instruction until you get to the call to OnFrameWindowActivate in OLESRV1.CPP. Among other things, you will see the following tasks accomplished:

<ul> <li>Get the document type used in SetActiveObject calls.</li> <li>Get the in-place client-site interface.</li> <li>See if the container wants to go in-place right now.</li> <li>Get the parent window to create the COleIPFrameWnd.</li> <li>Create the inplace frame window.</li> <li>Send an activate notification to the container.</li> <li>Get the frame and doc window interfaces as well as other information.</li> <li>Set up the shared menu.</li> <li>Allow the server to install frame controls in the container.</li> <li>Update the zoom factor information before creating control bars.</li> <li>Resize the window to match the object.</li> <li>Set the active object.</li> <li>Add the frame and the document-level frame controls.</li> <li>Show any hidden modeless dialogs.</li> <li>Attempt toolbar negotiation.</li> <li>Install the menu and a hook that forwards messages from the menu to the inplace frame window.</li> <li>Make sure the object is scrolled into view.</li> <li>Show the inplace frame window and set the focus.</li></ul>

As you can see, ActivateInPlace does a lot of work and is very RPC-intensive.</li> <li> Step into the first line of OnFrameWindowActivate. Observe that this function sends the final notifications via the container to activate the server. </li> <li>Continue the debugging session to finish executing OnFrameWindowActivate. Under normal circumstances, Scribble would come up already activated in place within Contain. However, in this case, you have already executed past the code that set the focus to the activated server. Remember the call to pFrameWnd->SetFocus in the ActivateInPlace function in OLESRV1.CPP? By continuing to step through the code, you have reset the focus back to the debugger. Therefore, you must now switch tasks manually to Contain.</li> <li>Switch tasks to Contain. You will see Scribble's menu and toolbar within Contain, and you will be able to draw in the embedded item's rectangle.</li></ol>

Finishing the Debug Session
When you finish with Scribble, close the document window. Contain's menu and toolbar reappear and the Scribble debugging session ends within the second instance of the debugger.

Although Scribble has terminated, the second instance of the debugger is still running. To avoid complications, terminate the second instance of the debugger. You need to do this because whenever you call into a server to embed a new item or activate an existing one, the debugger will start another instance to debug this server, even if it is the same server that is already attached to another item in the document. Therefore, it is possible (but not desirable) to have multiple instances of the debugger debugging multiple instances of the same server all connected to the same container.

Once the second instance of the debugger has been terminated, it is safe to embed another Scrib Document object in Contain. It is not necessary to terminate the container before doing so.

Trying New Things in Future Debug Sessions
Try stepping into (rather than over) some of the function calls. The flow of control goes back and forth between the container and the server many times, and the debugger will track this flow accurately, bringing you into container code, then server code, and so forth. Functions of interest include:

CanInPlaceActivate

OnInPlaceActivate

GetWindow

OnUIActivate

GetWindowContext

SetActiveObject

SetMenu

ShowObject

The server maintains several pointers (interfaces) into the container's code through which it calls the container's member functions. These pointers include m_lpClientSite, lpInPlaceSite, and pFrameWnd.

Keywords: kbhowto kbbug kbinterop kbide kbctrlcreate kbdebug kbgdi KB154116

-

[mailto:TECHNET@MICROSOFT.COM Send feedback to Microsoft]

© Microsoft Corporation. All rights reserved.