Microsoft KB Archive/122680

Tutorial: Debugging OLE Applications

PSS ID Number: Q122680 Article last modified on 04-20-1996

2.00

WINDOWS NT

= SUMMARY =

This article provides a tutorial to assist you in learning how to use the Visual Workbench Integrated Debugger.

The Visual C++ integrated debugger supports simultaneous debugging of OLE client and server applications. You can seamlessly step across and into OLE clients and servers, with the ability to step across OLE Remote Procedure Calls. A second instance of the debugger is automatically spawned the first time an OLE client calls into an OLE server.

When building OLE servers, you may want to debug them in the context of being activated by an OLE container, thus debugging both the container and the server at the same time. This tutorial provides an example of how to debug an OLE server when the main debuggee is an OLE container. It shows how a second instance of the debugger is automatically spawned when an OLE client calls into an OLE server. The tutorial is designed to lead you step by step through the code that creates the OLE server object and establishes the connection between the OLE container and the OLE server. This is useful in tracking down problems that occur when the OLE server does not get created or initialized correctly.

All regular debugging facilities are available as you debug your OLE application.

= MORE INFORMATION =

Preparing to Debug
To prepare for OLE Client/Server debugging:


 * 1) Open the project for the OLE application and build a version with symbolic debugging information.
 * 2) Choose Options from the Tools menu.
 * 3) Select the Debug tab.
 * 4) Ensure that the OLE RPC Debugging check box and the Just-In-Time (JIT) Debugging boxes are checked. (You must have Windows NT administrator privileges to enable OLE RPC Debugging.)
 * 5) Choose OK. The information is now stored with your project.
 * 6) Set breakpoints at the points in the source files for your OLE application where you want to determine the state of the application.
 * 7) From the Debug menu, choose Go or press the F5 key to start the debugger.

Viewing Derived-Most Types
The QuickWatch dialog box provides support for the automatic downcast of pointers in OLE and MFC debugging. The debugger automatically detects when a pointer is pointing to a subclass of the type it is required to point to. When the pointer is expanded, Visual C++ will add an extra member, which looks like another base class and indicates the derived-most type. For example, if you are displaying a pointer to a CObject and the pointer really points to a CComboBox, the QuickWatch expression evaluator will recognize this and introduce a pseudo CComboBox member so you can access the CComboBox members.

The rest of the tutorial takes you through a debug session, using MFC sample code.

Creating the Object
  Build debug versions of the Microsoft Foundation Classes (MFC) samples in the development environment:   MSVC20   MSVC20    Run the SCRIBBLE.EXE file, built in step 1, to update the registry to point to this executable file.   Load the CONTAIN project into the Visual Workbench.   Choose Options from the Tools menu, select the Debug tab, and make sure OLE RPC Debugging is enabled, as described in step 4 of the “Preparing to Debug” section in this article.   Open the CONTRVW.CPP file in CONTAIN, and set a breakpoint on line 139, which contains a call to CreateItem. </li>  Press the F5 key to run CONTAIN.EXE. In the CONTAIN main menu, choose Insert New Object from the Edit menu. </li>  From the resulting Object Type list, choose Scrib Document, and then choose the OK button. At this point, the debugger should stop at the breakpoint you set in step 5. This is the call to the Insert Object dialog’s CreateItem function. The purpose of CreateItem is to create and initialize an object of the type you selected from the dialog box. The CreateItem function is passed a CCntrItem object and it uses the object to handle this process. For a brief overview of what CCntrItem does, see its class definition in CNTRITEM.H. Then look at the definition of COleClientItem (which CCntrItem is derived from) on line 399 in AFXOLE.H. </li>  Press the F8 key to step into the call to CreateItem. You are in the MFC source file OLEDLGS1.CPP. Press the F10 key to step over instructions until you get to the call to pNewItem->CreateItem on line 104. </li>  Press the F8 key to step into the call to pNewItem->CreateItem. Then press the F10 key to step over instructions until you get to line 594, which contains a call to OleCreate. While stepping through the code, read the comments and observe that storage is allocated for the object and its rendering format is established. </li>  Step into the call to OleCreate. At this point, execution proceeds through the RPC mechanism to the server code itself. Therefore, as the server code begins to execute, a new instance of the debugger is created in which to debug the server. A pseudo project for SCRIBBLE.EXE is loaded (as in JIT debugging), and the instruction pointer is set at the call to COleServerDoc::XPersistStorage::InitNew in OLESRV1.CPP. If you installed the .DBG files (see the “NT System Symbols Setup” icon in your Visual C++ program group), your callstack will include fully decorated names. </li>  Step over instructions in Scribble’s InitNew until you reach the call to pThis->OnNewEmbedding on line 1707. Then step into OnNewEmbedding. </li>  Step over lines until you reach the call to OnNewDocument on line 856. Then step into the call to COleServerDoc::OnNewDocument. You are inside COleLinkingDoc::OnNewDocument. (COleServerDoc is derived from COleLinkingDoc.) Notice the code in this small function. It creates a new document object and attaches it to the server (Scribble). </li>  Press SHIFT+F7 twice to step out twice to get back into COleServerDoc::XPersistStorage::InitNew, where you first came into Scribble. </li>  Step out one more time. This will cause the container to return from its call to OleCreate, the function that took you into Scribble in the first place. At this time, the instance of the debugger that has Contain loaded gets the focus and you are back in OLECLI1.CPP immediately following the call to OleCreate. The embedded Scribble object has now been created, but it is not yet fully initialized. </li>  Step into the next line, the call to FinishCreate. Step through the FinishCreate code to see how OLE finalizes the connection between the container and the server, then step out of FinishCreate. </li></ol>

Now a Scribble object has been created and initialized in memory, but it is not yet editable in the container; Scribble hasn’t been activated. In fact, Contain still has only an IUnknown interface to Scribble. You can see this by expanding the lpClientSite variable in the Locals Window.

Activating the Object
<ol style="list-style-type: decimal;">  Step out two more times to get back to Contain’s OnInsertObject function in CONTRVW.CPP. </li>  In Contain’s OnInsertObject function, step over five times to get to the call to DoVerb on line 149. This function activates Scribble. </li>  Step into the call to DoVerb. Then step over a few lines until you come to the call to Activate on line 71. </li>  Step into Activate. Now you are in OLECLI3.CPP. Before going on, scan through the code for Activate. Notice that a rectangle is first created for the embedded Scribble item to live in. Then GetClientSite is called to establish an interface back to the container for the Scribble server. Then the server’s DoVerb function is called to pass both of these to the server. </li>  Step to line 75. Then step into the call to DoVerb. At this point, execution proceeds once again through the RPC mechanism to the server code itself. As you would expect, the instruction pointer is pointing to the first instruction in the server’s COleServerDoc:: XOleObject::DoVerb function. </li>  Step to line 2064, the call to OnDoVerb. Then step into OnDoVerb. OnDoVerb consists of a switch statement that executes the command (verb) passed to it. In this case, the command is OLEIVERB_SHOW. Step over instructions until you get to OnShow. </li> <li> Step into OnShow. Then step three lines to ActivateInPlace, and step into ActivateInPlace. The ActivateInPlace function does many things, and is worth examining in detail. While it is beyond the scope of this tutorial to go into all the details, it is worthwhile to step through the code and observe the comments. At this point, step over each instruction until you get to line 1098 in OLESRV1.CPP – the call to OnFrameWindowActivate. Among other things, you will see the following tasks accomplished: <ul> <li> Get the document type used in SetActiveObject calls. </li> <li> Get the in-place client-site interface. </li> <li> See if the container wants to go in-place right now. </li> <li> Get the parent window to create the COleIPFrameWnd. </li> <li> Create the inplace frame window. </li> <li> Send an activate notification to the container. </li> <li> Get the frame and doc window interfaces as well as other information. </li> <li> Set up the shared menu. </li> <li> Allow the server to install frame controls in the container. </li> <li> Update the zoom factor information before creating control bars. </li> <li> Resize the window to match the object. </li> <li> Set the active object. </li> <li> Add the frame and the document-level frame controls. </li> <li> Show any hidden modeless dialogs. </li> <li> Attempt toolbar negotiation. </li> <li> Install the menu and a hook that forwards messages from the menu to the inplace frame window. </li> <li> Make sure the object is scrolled into view. </li> <li> Show the inplace frame window and set the focus. </li></ul>

As you can see, ActivateInPlace does a lot of work. It is also very RPC-intensive. </li> <li> Step into the first line of OnFrameWindowActivate. Observe that this function sends the final notifications via the container to activate the server. </li> <li> Press the F5 key or choose the GO button to finish executing OnFrameWindowActivate. Under normal circumstances, Scribble would come up already activated in place within Contain. However, in this case, you have already executed past the code that set the focus to the activated server. Rremember the call to pFrameWnd->Set Focus on line 1094 in the ActivateInPlace function in OLESRV1.CPP? By continuing to step through the code you have reset the focus back to the debugger. Therefore, you must now switch tasks manually to Contain. </li> <li> Switch tasks to Contain. You will see Scribble’s menu and toolbar within Contain, and you will be able to draw in the embedded item’s rectangle. </li></ol>

Finishing the Debug Session
When you finish with Scribble, close the document window. Contain’s menu and toolbar reappear, and the Scribble debugging session ends within the second instance of the debugger.

Although Scribble has terminated, the second instance of the debugger is still running. To avoid complications, terminate the second instance of the debugger. You need to do this because whenever you call into a server to embed a new item or activate an existing one, the debugger will start another instance to debug this server, even if it is the same server that is already attached to another item in the document. Therefore, it is possible (but not desirable) to have multiple instances of the debugger debugging multiple instances of the same server all connected to the same container.

Once the second instance of the debugger has been terminated, it is safe to embed another Scrib Document object in Contain. It is not necessary to terminate the container before doing so.

Trying New Things in Future Debug Sessions
If you have more time in a subsequent debugging session, try stepping into (rather than over) some of the function calls. The flow of control goes back and forth between the container and the server many times, and the debugger will track this flow accurately, bringing you into container code, then server code, and so forth. Functions of interest include:

CanInPlaceActivate OnInPlaceActivate GetWindow OnUIActivate GetWindowContext SetActiveObject SetMenu ShowObject

The server maintains several pointers (interfaces) into the container’s code through which it calls the container’s member functions. These pointers include m_lpClientSite, lpInPlaceSite, and pFrameWnd.

Additional reference words: kbinf 2.00 KBCategory: kbtool kbole kbinterop KBSubcategory: WBDebug VCx86 ============================================================================= Copyright Microsoft Corporation 1996.