Writing a .NET debugger (part 1) – Starting the debugging session

After having analyzed the mdbg sources, I decided that the best way to learn how the .NET debugging services are working will be to implement my own small debugger engine (named mindbg). In a series of posts, I will try to explain each part of the debugger engine (such as starting/stopping debuggee, setting breakpoint, walking the stack, etc.). The whole project is hosted on CodeProject and you may access the sources here.

We will start from the most basic task which is opening the debugging session either by creating a new process or attaching to the existing one. Both of these operations are done via ICorDebug interface which is a COM interface defined in cordebug.idl (this is the file where you may find all necessary GUIDs). In .NET 1.x, an instance of this interface was created using CoCreateInstance (like a casual COM class):

NativeMethods.CoCreateInstance(ref debuggerGuid,
                               IntPtr.Zero, // pUnkOuter
                               1, // CLSCTX_INPROC_SERVER
                               ref NativeMethods.IIDICorDebug,
                               out rawDebuggingAPI);

In .NET 2.0, you needed to use global static CreateDebuggingInterfaceFromVersion:

ICorDebug rawDebuggingAPI;
rawDebuggingAPI = NativeMethods.CreateDebuggingInterfaceFromVersion(
                                (int)CorDebuggerVersion.Whidbey,debuggerVersion);

In .NET 4.0, acquiring the ICorDebug instance is not so easy and requires usage of CLR hosting interfaces. We will begin with ICLRMetaHost, which will give us access to all installed runtimes or all CLRs loaded in a specified process. An instance of the ICLRMetaHost interface is created using the CLRCreateInstance static global method (guids from metahost.h):

Guid clsid = new Guid("9280188D-0E8E-4867-B30C-7FA83884E8DE");
Guid riid = new Guid("D332DB9E-B9B3-4125-8207-A14884F53216");

ICLRMetaHost metahost = NativeMethods.CLRCreateInstance(ref clsid, ref riid);

Depending on the way how we start the debugging session (creating a new process or attaching to the running one), we need to use either EnumerateInstalledRuntimes or EnumerateLoadedRuntimes. Firstly, I would like to discuss the situation when we start the debuggee from inside the debugger and so we may decide which runtime to load. Attaching to the running process is quite similar and I will briefly explain it later.

One step I haven’t described yet is how we import the COM interfaces to our project. I usually use tlbimp command and then reflector to extract from the generated interop assembly only interfaces that I need. For example the ICLRMetaHost imported from metahost.tlb has following structure:

[ComImport, Guid("D332DB9E-B9B3-4125-8207-A14884F53216"), 
	InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
internal interface ICLRMetaHost
{
    [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType = MethodCodeType.Runtime)]
    IntPtr GetRuntime([In, MarshalAs(UnmanagedType.LPWStr)] string pwzVersion, 
			[In] ref Guid riid);

    [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType = MethodCodeType.Runtime)]
    void GetVersionFromFile([In, MarshalAs(UnmanagedType.LPWStr)] 
	string pwzFilePath, [Out, MarshalAs(UnmanagedType.LPWStr)] 
	StringBuilder pwzBuffer, [In, Out] ref uint pcchBuffer);

    [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType = MethodCodeType.Runtime)]
    IEnumUnknown EnumerateInstalledRuntimes();

    [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType = MethodCodeType.Runtime)]
    IEnumUnknown EnumerateLoadedRuntimes([In] IntPtr hndProcess);

    [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType = MethodCodeType.Runtime)]
    void RequestRuntimeLoadedNotification([In, MarshalAs(UnmanagedType.Interface)] 
	ICLRMetaHost pCallbackFunction);

    [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType = MethodCodeType.Runtime)]
    IntPtr QueryLegacyV2RuntimeBinding([In] ref Guid riid);

    [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType = MethodCodeType.Runtime)]
    void ExitProcess([In] int iExitCode);
}

With all the interface definitions ready, we may finally use metahost variable to find the ICLRRuntimeInfo interface which represents CLR v4.0:

IEnumUnknown runtimes = metahost.EnumerateInstalledRuntimes();
ICLRRuntimeInfo runtime = GetRuntime(runtimes, "v4.0");

And GetRuntime method is defined as follows:

///
/// Steps through the enumerator and returns the ICLRRuntimeInfo instance
/// for the given version of the runtime.
///
///
runtimes enumerator (taken from Enumerate*Runtimes method)
///
/// the desired version of the runtime - you don't need to
/// provide the whole version string as only the first n letters
/// are compared, for example version string: "v2.0" will match
/// runtimes versioned "v2.0.1234" or "v2.0.50727". If null
/// is given, the first found runtime will be returned.
///
///
private static ICLRRuntimeInfo GetRuntime(IEnumUnknown runtimes, String version)
{
    Object[] temparr = new Object[3];
    UInt32 fetchedNum;
    do
    {
        runtimes.Next(Convert.ToUInt32(temparr.Length), temparr, out fetchedNum);

        for (Int32 i = 0; i < fetchedNum; i++)
        {
            ICLRRuntimeInfo t = (ICLRRuntimeInfo)temparr[i];

            // version not specified we return the first one
            if (String.IsNullOrEmpty(version))
            {
                return t;
            }

            // initialize buffer for the runtime version string
            StringBuilder sb = new StringBuilder(16);
            UInt32 len = Convert.ToUInt32(sb.Capacity);
            t.GetVersionString(sb, ref len);
            if (sb.ToString().StartsWith(version, StringComparison.Ordinal))
            {
                return t;
            }
        }
    } while (fetchedNum == temparr.Length);

    return null;
}

Now we can call GetInterface method from the ICLRRuntimeInfo object with interface id and class id of the ICorDebug COM objects (you may find them in cordebug.idl):

clsid = new Guid("DF8395B5-A4BA-450B-A77C-A9A47762C520");
riid = new Guid("3D6F5F61-7538-11D3-8D5B-00104B35E7EF");

Object res;
runtime.GetInterface(ref clsid, ref riid, out res);
ICorDebug codebugger = (ICorDebug)res;

There are two more things that need to be done after constructing a brand new ICorDebug instance. First you need to initialize it – using the Initialize method. Second you need to set the managed event handler. Managed event handler is a special object through which the debuggee will communicate with the debugger. Its simplest representation would be a class that implements interfaces: ICorDebugManagedHandler and ICorDebugManagedHandler2 and has all methods empty:

public class ManagedCallback : ICorDebugManagedCallback, ICorDebugManagedCallback2
{
// here all interface methods with nothing inside
}

Finally, we are ready to start the debuggee process by calling CreateProcess method on the ICorDebug instance:

codebugger.Initialize();
codebugger.SetManagedHandler(new ManagedCallback());

STARTUPINFO si = new STARTUPINFO();
si.cb = Marshal.SizeOf(si);

// initialize safe handles
si.hStdInput = new Microsoft.Win32.SafeHandles.SafeFileHandle(new IntPtr(0), false);
si.hStdOutput = new Microsoft.Win32.SafeHandles.SafeFileHandle(new IntPtr(0), false);
si.hStdError = new Microsoft.Win32.SafeHandles.SafeFileHandle(new IntPtr(0), false);

PROCESS_INFORMATION pi = new PROCESS_INFORMATION();

//constrained execution region (Cer)

ICorDebugProcess proc;
codebugger.CreateProcess(
                    appname,
                    appname,
                    null,
                    null,
                    1,
                    (UInt32)CreateProcessFlags.CREATE_NEW_CONSOLE,
                    new IntPtr(0),
                    ".",
                    si,
                    pi,
                    CorDebugCreateProcessFlags.DEBUG_NO_SPECIAL_OPTIONS,
                    out proc);

You should always run debuggee in a separate console window so the application does not steal key strokes that were passed to the debugger. We may now discuss another scenario for starting the debugging session which is attaching to the running process. Most of steps presented above do not change. Only instead of calling EnumerateInstalledRuntimes, we need to call EnumerateLoadedRuntimes and in place of CreateProcess method, we will use DebugActiveProcess. Below is the code snippet showing how to attach debugger to the process:

/// <summary>
/// Attaches debugger to the running process.
/// </summary>
/// <param name="pid">Process id</param>
public static void AttachToProcess(Int32 pid)
{
    Process proc = Process.GetProcessById(pid);

    Guid clsid = new Guid("9280188D-0E8E-4867-B30C-7FA83884E8DE");
    Guid riid = new Guid("D332DB9E-B9B3-4125-8207-A14884F53216");

    ICLRMetaHost metahost = NativeMethods.CLRCreateInstance(ref clsid, ref riid);

    // get the v4.0 runtime
    IEnumUnknown runtimes = metahost.EnumerateLoadedRuntimes(proc.Handle);
    ICLRRuntimeInfo runtime = GetRuntime(runtimes, "v4.0");
    if (runtime == null)
    {
        throw new Exception("Only v4.0 supported");
    }

    ICorDebug codebugger = CreateDebugger(runtime);

    ICorDebugProcess coproc;
    codebugger.DebugActiveProcess(Convert.ToUInt32(pid), 0, out coproc);

    Console.ReadKey();
}

In the next part, we will discuss debugging events and we will take some control over the debuggee.

Just a reminder: all sources are available under mindbg.codeplex.com.