Minimalist In-Process Interface Marshaling

• Download demo project - 27.7 Kb (warning, not very spectacular!)
• Download source - 39.1 Kb

Introduction

Creating and registering a separate proxy/stub DLL for in-process marshalling only, unnecessarily impacts the system by requiring registry entries and additional files. This article describes a method of implementing in-process marshalling without the need for any registry changes or type libraries, bundled with the main executable.

Background

Background "worker threads" are a well-known construct used when multiple tasks need to be executed simultaneously, such as servicing the user and performing a requested action. Sometimes, the object that needs to be notified is a COM object.

Roughly speaking, COM objects come in two flavours when it comes to threading: single-threaded and multi-threaded. Single-threaded objects belong (or "live in") so-called STAs (Single-Threaded Apartments), meaning that access to them is synchronized by COM. This is the simplest and most widespread threading model since COM guarantees that calls into these types of objects always happen in the same apartment. Practically, this means that any method call into such an object is guaranteed to be in the context of the thread in which the object was created (via CoCreateInstance, etc.). So far, so good, for a single-threaded application.

When an application is multi-threaded, we have to get our hands dirty. COM treats a worker thread as a different apartment. This means that calls into objects belonging to the main apartment cannot be executed directly (that would contradict the fact that calls into a single-threaded object only occur in the STA to which they belong). COM must, therefore, somehow transfer control to the STA of the called object, perform the call, and return control to the caller. This procedure must also handle any parameters, along with any data returned by the callee. This is referred to as "marshalling".

COM marshalling is a rather complex topic, because it was designed to be very flexible. COM (well, DCOM) supports calls from one apartment to another, in the most general sense. For instance, it is possible to remotely create and access a COM object over a network. To COM, this is no different than accessing an object in a different thread of the same process. This means that COM requires a method of serializing parameters, sending the request over the network, then deserializing the data returned, in a uniform fashion.

Implementing the IMarshal interface is one way of providing this functionality. However, for most uses, this is too low-level. The next method, and probably the most common, is to have the MIDL compiler generate so-called "proxies" and "stubs" from your IDL file. Essentially, this means that COM handles most of the marshalling, except that you provide information about the types and sizes of the data being transferred as part of the method calls.

Conventionally, this information is provided in a separate DLL from the main project. This DLL has the sole purpose of helping COM to serialize parameters (the proxy's responsibility) at the caller, and deserialize it at the callee (the stub's responsibility). For COM to be able to find it, the DLL is usually registered in the registry under HKEY_CLASSES_ROOT\Interfaces, under the IID of the interface whose methods will be invoked remotely. The proxy/stub objects also have their own CLSIDs, which means they are registered under HKEY_CLASSES_ROOT\CLSID.

While this is appropriate for inter-process calls or calls over the network, where the remote component has no way of knowing where to find the serialization information, it could be considered overkill for in-process marshalling. This article demonstrates a minimalist implementation of marshalling in the same executable as the main application, using CoRegisterClassObject and CoRegisterPSClsid to avoid registry changes, suitable for lightweight in-process marshalling.

Using the code

The first step is to cause the MIDL compiler to generate the proxy/stub code based on the method signatures you provide in the IDL file. This is done using the object interface attribute. Note also that methods of such interfaces may only have the return type HRESULT.

[
    object, // Causes MIDL to generate proxy/stub code
    uuid(6DE7CAC2-1D1C-43B8-A91C-D0B3495007CD),
]
interface ITest : IUnknown {
    HRESULT OnEvent([in] LONG ev);
};

As a result, the MIDL compiler generates four files: dlldata.c, basename_h.h, basename_i.c, and basename_p.c:

• dlldata.c provides the entry-points for what would be a separate proxy/stub DLL. Instead of exporting them, we rename them by adding the ENTRY_PREFIX define (documented in <rpcproxy.h>), and we register them locally using CoRegisterClassObject.
• basename_h.h and basename_i.c provide the usual C++ declarations for your interfaces, CLSIDs, etc.
• basename_p.c contains machine-usable definitions of the format of the parameters of your interfaces. This data is used during the marshalling process.

In each apartment, before attempting to marshal or unmarshal any interface pointers, we need to register the proxy/stub factories with COM, using the renamed proxy/stub DLL entry point and CoRegisterClassObject. This would normally be done by registering the proxy/stub DLL (via regsvr32, i.e., DllRegisterServer and DllGetClassObject upon object creation).

IUnknown *punk;
ProxyDllGetClassObject(iid, IID_IUnknown, (void **)&punk);
CoRegisterClassObject(iid, punk, CLSCTX_INPROC_SERVER, 
                      REGCLS_MULTIPLEUSE, cookie);

(Note that we arbitrarily chose the IID of the interface being marshaled as the CLSID of the proxy/stub object.)

COM still hasn't associated the interface being marshaled (ITest) with its proxy/stub, which it would normally do using an entry in HKCR\Interface. This is done process-wide using the CoRegisterPSClsid API:

CoRegisterPSClsid(IID_ITest, IID_ITest);

At this point, we're ready to use the marshalling features of COM. CoMarshalInterThreadInterfaceInStream is a simple but handy wrapper around CoMarshalInterface.

CoMarshalInterThreadInterfaceInStream(IID_ITest, 
                   test->GetUnknown(), &stream);

In the worker thread, given the IStream pointer, we un-marshal the interface pointer:

ITest *test;
CoGetInterfaceAndReleaseStream((LPSTREAM)param, 
                    IID_ITest, (void **)&test);

Lastly, don't forget to add rpcrt4.lib as a link-time dependency, since the proxy/stub code relies heavily on system-provided helpers.

And that's it! COM transparently handles any method calls on this new interface pointer, calls the appropriate proxy to help serialize parameters, context switches into the STA of the object, calls the stub to deserialize parameters, and performs the actual method call. Once the method returns, the process happens in reverse: the stub serializes any return data, it is sent over the network, it's deserialized by the proxy, and finally control is returned to the caller. Phew.

Compatibility

Since COM, DCOM, and RPC (Remote Procedure Call, the default underlying implementation of cross-apartment calls over a network) have evolved significantly from the first releases of Windows until now, there are tweaks that allow for a trade-off between compatibility and performance or stability.

Currently, the code is set to compile using one of the most permissive settings, in which it is compatible with NT 4.0 and later. This is represented by the preprocessor defines WINVER=0x400, _WINNT_WIN32=0x400, and _WINNT_WINDOWS=0x400. The MIDL compiler must also be told to favour compatible code, using the /no_robust switch. You can read all about its meaning in the MSDN page for /robust. Bottom line, specifying /robust makes the proxy/stub only compatible with Windows 2000 and later, but provides additional runtime checking.

Also note that this technique isn't limited to any particular version or C++ compiler. It has been tested on Microsoft Visual C++ 6.0 and .NET 2003 (project files are included for each), and on Windows 2000, XP, and Server 2003.

Points of Interest

One particular thing that surprised me is the need to call CoRegisterClassObject to register the proxy/stub object in both the main thread and the worker thread. At first, I thought this meant CoRegisterClassObject registrations were per-apartment. However, registering using the CLSCTX_LOCAL_SERVER flag leads to not only a process-wide, but a machine-wide registration. Interestingly, IUnknown pointers now returned by CoGetClassObject exhibit the same "stub" vtable as those returned from CoGetInterfaceAndReleaseStream. Effectively, we are marshalling part of the marshaller. Unfortunately, since marshalling handlers for the proxy/stub object haven't been registered, querying for any interface fails.

You might also ask yourself, "the IStream pointer is passed raw between the two threads - should it also be marshaled?" The answer is, no, because the system automatically provides marshalling for "standard" interfaces, such as IStream, IPersist, IStorage, etc. Another possible reason for this could be that system-provided implementations of these objects are multithreaded, meaning that they can be called in the context of any apartment (thread) without special handling on the caller's part. However, the objects themselves do need to manage synchronization as in any other multi-threaded design.

Lastly, for the curious, how does COM practically implement the context switch? What's the most common way in Windows to message (hint, hint) a foreign thread? Messages. COM creates a hidden window in the STA to which messages are sent from any foreign calling apartments. This also explains the need for a message pump in STAs.

Thanks

This article would not have been possible without the professionals of microsoft.public.win32.programmer.ole. Many thanks for your suggestions and help.

History

• 21/04/2006: Added more information about CoRegisterClassObject registrations. Initial release.
• 20/04/2006: Initial version created.