Click here to Skip to main content
15,034,404 members
Articles / Containers / Virtual Machine
Posted 23 Nov 2010


14 bookmarked

Setting Up Kernel Mode Debugger in Windows

Rate me:
Please Sign up or sign in to vote.
4.60/5 (4 votes)
23 Nov 2010CPOL4 min read
This article will guide you through the process of setting up kernel mode debugger in Windows


Whenever there is a bug in your program, you usually open a debugger (GDB, Visual Studio debugger, etc.) to fix it, but how do you debug a bug in the operating system? Do you load the running OS into the debugger? Is it possible? The simple answer is no, Because even the debugger works with the help of OS. It's a Catch 22 situation.


Earlier developers used two machines; one is the defective OS(slave) and other machine is the one containing debugger software(master). Now the defective slave is connected to master machine using a high speed cable, and then both slave and master machines are started, with the help of debugger the execution of slave machine is paused. But this solution has some drawbacks.

  1. The connection speed between the machines is too slow, because the data and commands should be passed to and fro between master and slave.
  2. Requires extra hardware like cable and two separate machines

Current Process

Fortunately, we now have much better options for beginners who want to study the internals of the OS by debugging. With the help of virtual machines, now we do not require two separate machines. The slave machine can be thought of as a guest VM and master machine can be thought of as a host computer(your real physical machine). The connection between these host and guest have been made even simpler with the help of a software called VirtualKD(Virtual Kernel Debugger)[without this tool, we have to manually set up a named pipe in the guest and modify boot.ini to enable some special options. It's a little time consuming]. So in this tutorial, I will help you set up kernel mode debugger.

I will be using the following tools:

  1. WinDbg (Windows Kernel Debugger)
  2. Virtual Box (Virtual Machine Manager)
  3. VirtualKD (Tool to enable very high speed kernel debugging between host and just machines)

Hereafter, whenever I refer to OS, it will be one version of Windows.

  • First thing we have to do is install virtual box and then install a guest OS of your choice. Here I would like to demo using Windows XP as my guest [installing a guest VM].
  • Second, extract the virtualKD to some folder.

    Rename the VBoxDD.dll file in your VirtualBox program files folder to VBoxDD0.dll.Copy modified VBoxDD.dll from VirtualKD archive to VirtualBox directory (Ensure that you have selected correct version (x86 or x64) of VBoxDD.dll).


  • Third, install WinDbg (simple next next install).

    For any debugger to properly work, we should have symbol files of the program being debugged, think of these files as extra information about your program which helps debugger in displaying meaningful information to the user. If we have correct symbol files, we will have the extract function names and the line numbers, etc. getting shown in the debugger, without that we will see some hard to understand Hex address. So the following few lines will help you configure symbol files for your operating system.

    If you have internet connection all the time, follow this:

    • Create an environment variable named _NT_SYMBOL_PATH and set its value to say srv*c:\symbols*
    • Symbols will be downloaded from the Microsoft symbol server on demand to c:\symbols

    Else follow this:

    • Create an environment variable named _NT_SYMBOL_PATH and set its value to say c:\symbols
    • C:\program files\<windbg>symchk.exe /r c:\windows\system32\* /s SRV*C:\symbols\*
    • The above command will fetch all the symbols for the files in system32 at once (takes time and space).
    • It's a one time task.

    When WinDbg is launched, it will check this variable to know the path of symbol files.

  • Fourth, run VirtualKD in the guest machine and then reboot the guest as instructed. 2.png
  • Once you are at the boot prompt, open:


  • Choose the highlighted one and then open vmmon.exe from VirtualKD which will automatically launch WinDBG and connect to the currently running VM and pauses its execution:



  • In the above illustration, if you see the first highlighted rectangle, Windbg is connected to the guest using named pipe \\.\pipe\kd_XP_2.
  • Symbol files are picked up from c:\symbols, this is where our _NT_SYMBOL_PATH points:


  • Most of the windbg commands are explained very clearly at


  1. Windows Internals 4th Ed By Mark E. Russinovich, David A. Solomon

Happy debugging.


  • 23rd November, 2010: Initial post


This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


About the Author

Vineel Kovvuri
Software Developer Microsoft
India India
Hello, This is Vineel Kovvuri, I am interested in Systems Programming, love to tinker with low-level functionality of Windows. Works for Microsoft.

Comments and Discussions

GeneralThanks man it help a lot Pin
rayps22-Dec-10 9:02
Memberrayps22-Dec-10 9:02 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.