Click here to Skip to main content
15,893,190 members
Articles / Programming Languages / C++
Article

Intercepting WinAPI calls

Rate me:
Please Sign up or sign in to vote.
4.83/5 (29 votes)
31 May 20063 min read 151.1K   3.7K   80   42
An article about intercepting WinAPI calls.

Introduction

API calls interception is the task that allows to get access to some parts of other's programs. Lots of programmers spend time on developing and describing various methods which allow that access. Such methods are used in many anti-viruses and anti-spyware. Besides, sometimes, intercepting can help you to find errors in your application. However, it is not a secret that some viruses use it too. I spent much time to find and understand the technique of interception. I would like to describe here the results of my research.

Method description

First of all, you need to read the following article to understand the basics of the interception mechanism: HookSys (written by Ivo Ivanov). It was very helpful for me, and I used the sample code from it. However, it does not solve all my problems because Ivo's samples sometimes miss very important API calls. It happens when the application starts up too fast and the intercepting service has no time to inject the DLL. After some research, I found the actual problem, and it was related to using the kernel mode function, SetCreateProcessNotificationRoutine. This function is used to receive notification events about any new process creation. Such a notification is often fired when the process has already been started. Therefore, I needed to find a way to improve Ivo's code.

As far as I know, the execution of all Windows processes consists of the following steps:

  • initial process loading;
  • creating the main thread for the process in the suspended state;
  • mapping of the NT.DLL into the address space of the process;
  • mapping all needed DLLs, and calling their DllMain with the DLL_PROCESS_ATTACH reason;
  • resuming the main process' thread.

The step right before the main thread resuming looks like the most comfortable for injection because the process is in suspended state and none of its instructions have been executed yet.

Most of the work on the process creation is done in the kernel mode, so to change this algorithm, you need to intercept the kernel mode functions NtCreateProcess() and NtCreateThread(). The CONTEXT structure, the pointer to which is passed to the function NtCreateThread(), contains a member called EAX. I found that it equals to the process' start address in user mode, so if you can change it, then you can get the control right after process creation and before starting. To solve this task, I wrote a kernel mode driver. It starts while the system starts up.

There are some initialization steps:

  1. starting;
  2. receiving configuration from the user mode;
  3. intercepting kernel mode functions such as: NtCreateProcess(), NtCreateThread(), NtTerminateProcess(), NewNtCreateProcessEx() - for Windows 2003 Server.

A handler to the NtCreateThread() function contains code that will do most of the interesting jobs. Here is a brief description of its algorithm:

  1. allow access to the creating process by calling ObReferenceObjectByHandle();
  2. remember the main thread start address (ThreadContext->EAX);
  3. "jump" to the context of the creating process by calling KeAttachProcess();
  4. allocate memory for my code by calling ZwAllocateVirtualMemory(), similar to the well known technique for CreateRemoteThread() in user mode;
  5. copy the small code to the allocated memory that will load my DLL. This code looks like:
    ASM
    push pszDllName
    mov  ebx, LoadLibraryAddr
    call [ebx]
    mov  eax, Win32StartAddr
    push eax
    ret
    pszDllName: db 'example.dll';
  6. "jump" to the initial process;
  7. change the thread start address (ThreadContext->EAX) so it will point to the allocated memory.

That is all. You can download and compile the complete source code for this article. Note: the sample is fully functional and quite enough for basic understanding, but for real usage, it might be rewritten.

Compiling the code

You need the NTDDK to be installed on your computer. I'm using MSVS 6.0 for compiling NtProcDrv, and MSVS 7.1 for the rest of the projects.

History

  • 2006-03-06 - Submitted.
  • 2006-05-31 - Source codes and binaries are updated.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here


Written By
Chief Technology Officer
Ukraine Ukraine
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

 
QuestionProblem in Release Build Pin
noumantariq28-Oct-09 23:49
noumantariq28-Oct-09 23:49 
GeneralCorrection in assembly code [modified] Pin
MattsUserName27-Jul-09 15:04
MattsUserName27-Jul-09 15:04 
QuestionVista compatibility? Pin
Amit Vilas Shinde7-Mar-08 1:36
Amit Vilas Shinde7-Mar-08 1:36 
GeneralHi, tezm. I ran into the same question as yours. Pin
mybayern19742-Aug-07 22:02
mybayern19742-Aug-07 22:02 
GeneralRe: Hi, tezm. I ran into the same question as yours. Pin
tezm4-Aug-07 6:55
tezm4-Aug-07 6:55 
GeneralRe: Hi, tezm. I ran into the same question as yours. Pin
mjmim10-Oct-08 12:34
mjmim10-Oct-08 12:34 
QuestionError installing service, how to solve it? Pin
tezm27-Jul-07 8:29
tezm27-Jul-07 8:29 
AnswerRe: Error installing service, how to solve it? Pin
tezm28-Jul-07 1:28
tezm28-Jul-07 1:28 
GeneralI want to support vista O.S., please help me Pin
Hsiao, Tsu Tair25-Jul-07 16:46
Hsiao, Tsu Tair25-Jul-07 16:46 
GeneralInjection works fine, but impossible to subclass using this method? [modified] Pin
ehaerim5-Jan-07 11:05
ehaerim5-Jan-07 11:05 
I've modified the dllhookapi.cpp to see
1) if I can subclass Notepad.exe successfully within DllMain.
2) if I can subclass multiple instances of Notepad.exe successfully.

The code works this way:
1) Call EnumProcesses to find out all the running processes.
2) Call Subclass with the process id array obtained above and the name of notepad for subclassing.
3) For each process id, Subclass will call EnumProcessModules and GetModuleFileNameEx to get the full module name and repeat this until we find notepad.
4) Now if we have found notepad, call EnumThreadWindows with current thread id to find out the main window to subclass.

Here the problem rises. EnumThreadWindows returns successfully, but EnumThreadWindowsProc NEVER get called. I couldn't understand why. Now I guess it is because somehow the current thread did not create any windows yet. So EnumThreadWindows return TRUE but no EnumThreadWindowsProc call!

But then, how can I subclass notepad right after the creation?

haerim


<br />
// dllhookapi.cpp : Defines the entry point for the DLL application.<br />
//<br />
///////////////////////////////////////////////////////<br />
// Andriy Oriekhov. 2006. Toleron Sofware.<br />
// www.toleron.com<br />
///////////////////////////////////////////////////////<br />
<br />
#include "stdafx.h"<br />
#include "windows.h"<br />
<br />
#include <psapi.h><br />
<br />
<br />
<br />
HWND g_hwnd = NULL;<br />
BOOL CALLBACK EnumThreadWindowsProc(HWND hwnd, LPARAM lParam);<br />
// New & old window procedure of the subclassed window<br />
WNDPROC OldProc = NULL;	<br />
LRESULT CALLBACK NewProc(HWND, UINT, WPARAM, LPARAM);<br />
<br />
<br />
void Subclass(DWORD* paProcess, DWORD nCount, LPCTSTR pName);<br />
<br />
<br />
BOOL WINAPI DllMain(HANDLE hModule, DWORD ul_reason_for_call, LPVOID lpReserved)<br />
{<br />
	switch (ul_reason_for_call) {<br />
	case DLL_PROCESS_ATTACH:<br />
	{<br />
		WriteLogDebug("Before starting process.");<br />
<br />
		// Get the list of process identifiers.<br />
		DWORD aProcesses[1024], cbNeeded, cProcesses;<br />
		if (EnumProcesses(aProcesses, sizeof(aProcesses), &cbNeeded) == 0)<br />
		{<br />
			DWORD dwErr = GetLastError();<br />
			WriteLogDebug("DllMain: dwErr=%d", dwErr);<br />
			return TRUE;<br />
		}<br />
<br />
		// Find the matching process and subclass it.<br />
		cProcesses = cbNeeded / sizeof(DWORD);<br />
		Subclass(aProcesses, cProcesses, "C:\\Windows\\System32\\notepad.exe");<br />
<br />
		break;<br />
	}<br />
	case DLL_PROCESS_DETACH:<br />
	{<br />
		WriteLogDebug("Before ending process.");<br />
		if (::SetWindowLongPtr(g_hwnd, GWL_WNDPROC, (long)OldProc) == 0)<br />
		{<br />
			WriteLogDebug("Failed to unsubclass back from NewProc(0x%X) to OldProc(0x%X)", GetWindowLongPtr(g_hwnd, GWL_WNDPROC), (long)OldProc);<br />
		}<br />
		<br />
		break;<br />
	}}<br />
<br />
	return TRUE;<br />
}<br />
<br />
void Subclass(DWORD* paProcess, DWORD nCount, LPCTSTR pName)<br />
{<br />
	WriteLogDebug("Subclass: nCount=%d, pName=%s", nCount, pName);<br />
	char szProcessName[MAX_PATH] = "";<br />
	<br />
	unsigned int i;<br />
	for (i = 0; i < nCount; i++)<br />
	{<br />
		// Get a handle to the process.<br />
		DWORD dwPID = paProcess[i];<br />
		HANDLE hProcess = OpenProcess(PROCESS_QUERY_INFORMATION | PROCESS_VM_READ, FALSE, dwPID);<br />
<br />
		if (NULL == hProcess)<br />
		{<br />
			WriteLogDebug("Subclass: dwPID=0x%X(%d), OpenProcess failed!", dwPID, dwPID);<br />
			CloseHandle(hProcess);<br />
			continue;<br />
		}<br />
		<br />
		// Get modules for this process<br />
		DWORD cbNeeded = 0;<br />
		HMODULE hMod;<br />
		if (EnumProcessModules(hProcess, &hMod, sizeof(hMod), &cbNeeded) == 0)	// <br />
		{<br />
			WriteLogDebug("Subclass: hProcess=0x%X, EnumProcessModules failed!", hProcess);<br />
			continue;<br />
		}<br />
		DWORD nModCount = cbNeeded / sizeof(HMODULE);<br />
		WriteLogDebug("Subclass: nModCount=%d", nModCount);<br />
<br />
		// Get the process name.<br />
		GetModuleFileNameEx(hProcess, hMod, szProcessName, sizeof(szProcessName));<br />
		WriteLogDebug("Subclass: dwPID=0x%X(%u), cbNeeded=%d, %s\n", dwPID, dwPID, cbNeeded, szProcessName);<br />
		if (strcmpi(szProcessName, pName) != 0)<br />
		{<br />
			CloseHandle(hProcess);<br />
			continue;<br />
		}<br />
/*<br />
		// Go through with all the windows for the current thread<br />
		// => There is no windows for dwPID yet. So, EnumWindowsProc will NEVER be called for dwPID.<br />
		EnumWindows(EnumWindowsProc, (LPARAM)dwPID);<br />
*/<br />
		DWORD dwTID = GetCurrentThreadId();<br />
		WriteLogDebug("Subclass: dwTID=0x%X", dwTID);<br />
		// Now, let's subclass the main window. => This seems not working.  See comments below.<br />
		// The below EnumThreadWindows seems to NEVER call EnumThreadWindowsProc because there is no windows for the current thread created yet.<br />
		// So, EnumThreadWindows call succeeds returning TRUE but no EnumThreadWindowsProc gets called at all.<br />
		// Then how can I subclass the main window of this thread?  I don't know the answer yet.<br />
		if (EnumThreadWindows(dwTID, EnumThreadWindowsProc, (LPARAM)dwPID) == 0)<br />
		{<br />
			DWORD dwErr = GetLastError();<br />
			WriteLogDebug("Subclass: dwTID=0x%X, dwErr=%d, EnumThreadWindows failed!", dwTID, dwErr);<br />
			CloseHandle(hProcess);<br />
			continue;<br />
		}<br />
<br />
		CloseHandle(hProcess);<br />
	}<br />
}<br />
<br />
BOOL CALLBACK EnumThreadWindowsProc(HWND hwnd, LPARAM lParam)<br />
{<br />
	WriteLogDebug("EnumThreadWindowsProc(0x%X, %d)", hwnd, lParam);<br />
<br />
	// Let's find hwnd's process is the same as DllMain's calling process<br />
	DWORD dwPID_hwnd = 0, dwTID_hwnd = 0;<br />
	dwTID_hwnd = GetWindowThreadProcessId(hwnd, &dwPID_hwnd);<br />
<br />
	TCHAR wmfn[MAX_PATH + 1] = {0};<br />
	UINT nLen = GetWindowModuleFileName(hwnd, wmfn, MAX_PATH);	// Huh, this does not work for other processes, but only for the calling process!!!<br />
	WriteLogDebug("EnumThreadWindowsProc: hwnd=0x%X, dwPID_hwnd =0x%X(%d), dwTID_hwnd=0x%X(%d)\n"<br />
								"                       nLen=%d, wmfn=%s",<br />
																				hwnd,	dwPID_hwnd, dwPID_hwnd, dwTID_hwnd, dwTID_hwnd,<br />
																				nLen, wmfn);<br />
	return TRUE;<br />
//	// Finally, let's subclass hwnd<br />
//	WNDPROC OldProc = (WNDPROC)::SetWindowLongPtr(hwnd, GWL_WNDPROC, (long)NewProc);<br />
//	if (OldProc == 0)<br />
//		WriteLogDebug("EnumThreadWindowsProc: Failed to subclass hwnd(0x%X)", hwnd);<br />
//	else<br />
//		WriteLogDebug("EnumThreadWindowsProc: Succeeded in subclassing hwnd(0x%X)'s OldProc(0x%X) to NewProc(0x%X)",<br />
//									hwnd, (long)OldProc, (long)NewProc);<br />
//<br />
//	return FALSE;<br />
}<br />
<br />
//-------------------------------------------------------------<br />
// NewProc<br />
// Notice:	- new window procedure for the START button;<br />
//			- it just swaps the left & right muse clicks;<br />
//	<br />
LRESULT CALLBACK NewProc(<br />
													HWND hwnd,      // handle to window<br />
													UINT uMsg,      // message identifier<br />
													WPARAM wParam,  // first message parameter<br />
													LPARAM lParam   // second message parameter<br />
												)<br />
{<br />
//	WriteLogDebug("wParam=%d, lParam)=%d", wParam, lParam);<br />
	switch (uMsg) <br />
	{<br />
		//WM_COMMAND nNotifyCode:0 (Menu) wID:34795 <br />
		case WM_COMMAND:<br />
			WriteLogDebug("wParam=%d, lParam, HIWORD(wParam)(nNotifyCode)=%d, LOWORD(lParam)(id)=%d=%d", wParam, lParam, HIWORD(wParam), LOWORD(lParam));<br />
			if (HIWORD(wParam) == 0)<br />
			{<br />
//				WriteLogDebug("2,HIWORD(wParam)(nNotifyCode)=%d, LOWORD(lParam)(id)=%d", HIWORD(wParam), LOWORD(lParam));<br />
				switch(LOWORD(wParam))<br />
				{<br />
				case 34795:<br />
					// display main dialog <br />
					::MessageBox(NULL,"NULL","hStart",MB_OK);<br />
					return S_OK;<br />
				case 34796:<br />
					return S_OK;<br />
				case 34797:<br />
					return S_OK;<br />
				default:<br />
					break;<br />
				}<br />
			}<br />
	}<br />
	<br />
	return CallWindowProc(OldProc,hwnd,uMsg,wParam,lParam);<br />
}<br />
<br />
<br />
// Usage<br />
//WriteLog("int value=%d \n str value=%s", 19, "test");<br />
#include <STDIO.H><br />
void inline WriteLog(char *ch, ...)<br />
{<br />
    char buf[1024]; <br />
    va_list arg_list;<br />
 <br />
    va_start( arg_list, ch );<br />
    wvsprintf( buf, ch, arg_list );<br />
    va_end( arg_list );<br />
 <br />
    FILE *file;<br />
    file = fopen("dllhookapi.log", "a+");<br />
    fprintf(file, "%s", buf);<br />
    fclose(file);<br />
}<br />
<br />
void inline WriteLogDebug(char *ch, ...)<br />
{<br />
    char buf[1024]; <br />
    va_list arg_list;<br />
 <br />
    va_start( arg_list, ch );<br />
    wvsprintf( buf, ch, arg_list );<br />
    va_end( arg_list );<br />
 <br />
		OutputDebugString(buf);<br />
}<br />
<br />
<br />
<br />



-- modified at 17:11 Friday 5th January, 2007
QuestionHow to inject dll into specific processes, not all the processes? [modified] Pin
ehaerim30-Dec-06 21:28
ehaerim30-Dec-06 21:28 
AnswerRe: How to inject dll into specific processes, not all the processes? Pin
ehaerim5-Jan-07 10:35
ehaerim5-Jan-07 10:35 
QuestionNewNtTerminateProcess called with NULL as the value of ProcessHandle. Why? Pin
ehaerim30-Dec-06 20:59
ehaerim30-Dec-06 20:59 
GeneralSecond MessageBox does not show up when process termination but only sound. Pin
ehaerim2-Dec-06 14:56
ehaerim2-Dec-06 14:56 
GeneralRe: Second MessageBox does not show up when process termination but only sound. Pin
tezm28-Jul-07 1:25
tezm28-Jul-07 1:25 
QuestionVC++ 6.0 Version available? Pin
ehaerim2-Dec-06 10:52
ehaerim2-Dec-06 10:52 
QuestionHow to hook multiple instances of Notepad? [modified] Pin
ehaerim30-Nov-06 15:10
ehaerim30-Nov-06 15:10 
GeneralWhere can I get info about SetCreateProcessNotificationRoutine? [modified] Pin
ehaerim30-Nov-06 13:42
ehaerim30-Nov-06 13:42 
Generalstarting programs by clicking on document icons Pin
richardmoss15-Nov-06 5:43
richardmoss15-Nov-06 5:43 
GeneralRe: starting programs by clicking on document icons Pin
Andriy Oriekhov23-Nov-06 0:25
Andriy Oriekhov23-Nov-06 0:25 
QuestionHow to get the executable name from the created process?? Pin
girm13-Sep-06 1:41
girm13-Sep-06 1:41 
AnswerRe: How to get the executable name from the created process?? Pin
Andriy Oriekhov14-Sep-06 2:53
Andriy Oriekhov14-Sep-06 2:53 
GeneralRe: How to get the executable name from the created process?? Pin
girm24-Oct-06 20:40
girm24-Oct-06 20:40 
GeneralRe: How to get the executable name from the created process?? Pin
Andriy Oriekhov25-Oct-06 11:32
Andriy Oriekhov25-Oct-06 11:32 
GeneralRe: How to get the executable name from the created process?? Pin
ehaerim5-Jan-07 10:32
ehaerim5-Jan-07 10:32 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.