Click here to Skip to main content
15,881,852 members
Articles / Desktop Programming / Windows Forms

Popup Window Finder and Mouse Tracker in C#

Rate me:
Please Sign up or sign in to vote.
4.80/5 (20 votes)
26 Nov 2009CPOL7 min read 68.2K   4.8K   75   17
An article on retrieving information from popup windows by using low level keyboard and mouse hooks, user32.dll and kernel32.dll APIs, etc.
Image 1

Contents

Introduction

I tried but did not find a utility which could retrieve information from a popup window. Microsoft Spy++ is the closest I could find, but it works only on static windows because it is not possible for a user to open a popup window when using the window finder in Spy++. This is why I decided to create my own popup window finder and share with you.

The application named Mouse Tracker provides the functions:

  • Record and display mouse locations and actions while the mouse is being moved or the left or right mouse button is clicked.
  • Record and display information of both static and pop-up windows. The information contains the hWnd (handle of Window), class name, text, rectangle, styles, names and styles of elements, and the hWnd of parent, child, previous and next window. The information of parent, child, previous and next window can be displayed by clicking its hWnd.
  • Suspend a popup window to allow Spy++ to retrieve its information.
  • Edit and modify the recorded mouse actions for the simulation.
  • Simulate mouse actions via the recorded mouse locations and actions.

Mouse Tracker can run on both 32bit and 64bit machines.

System Requirement

  • .NET Framework 3.5

Using the Code

Image 2 Starts mouse tracking. The mouse location and action will be stored and displayed in the mouse action grid view. The number of default rows is set at 20, which can be changed by clicking Image 3 and Image 4 buttons. Upon reaching the last row, the mouse tracking will be stopped and the status will be changed from Image 5 to Image 6 automatically. While the mouse button is clicked on a window, its information is stored and displayed in the window information grid view. By this feature, you can open a popup window by clicking the left or right mouse button first, move the mouse on it and then click a mouse button again to retrieve and display the information of the popup window.

Image 7 Stops mouse tracking.

Image 8 Clears the mouse action grid view but keeps the rows.

Image 9 Inserts a new row.

Image 10 Removes selected rows.

Image 11 Simulates mouse actions based on recorded information in the mouse action grid view. You can change a mouse action by choosing another option in the combo box in a cell.

Image 12 Hides the window information grid view.

Image 13 Displays the window information grid view.

F1 Button - Suspends the window under the cursor. You can suspend a popup window then use Microsoft Spy++ to retrieve its information and compare it with the results retrieved by Mouse Tracker. This function will be disabled when Image 14 is clicked and will be enabled when Image 15 is clicked.

F2 Button - Resumes all suspended windows.

Right-click Pop-up Menu - Copies content of a cell, selected rows or all rows to the clipboard.

Following the Following the steps as below, you can do a simple simulation of mouse actions and open two popup windows at the end:

  • Run MouseTrack.exe and Notepad.exe;
  • Click Image 16 to start mouse tracking;
  • Move the mouse on Notepad, click the right mouse button to popup the content menu;
  • Move the mouse on the row, "Insert Unicode Control characters ”, to open another popup window;
  • Move the mouse on the new popup window and click any row;
  • Click Image 17 to stop mouse tracking;
  • In Mouse action grid view of Mouse Tracker, select the last two rows and click Image 18 to remove them;
  • Click the Mouse Action column of the current last row, choose “Move” in the combo box;
  • Now, the contents of mouse action grid view are like the contents in the screenshot at the top of this page;
  • Click Image 19 to start simulation. Mouse Tracker will open two popup windows and move the mouse on the second window.

You may create other interesting simulations of mouse actions on other windows.

Technical Specifications

GUI

The GUI of Mouse Tracker includes Windows Form, SplitContainer, DataGridView and ContextMenuStrip primarily.

Retrieval of Window Information

This part is responsible for retrieving the window, style and member information from a window. Kernel32.ReadProcessMemory and Kernel32.WriteProcessMemory are the functions that allow access to memory in another process. Mouse Tracker not only allocates a large enough local buffer for both receiving data in Kernel32.ReadProcessMemory and sending data in Kernel32.WriteProcessMemory, but also casts the pointer of the local buffer to the pointers of different target data structures to avoid transferring data between the local buffer and a data structure.

C#
/// <summary>
///
/// </summary>
/// <param name="hWnd"></param>
/// <param name="pt"></param>
/// <returns></returns>
public Window getWindow(IntPtr hWnd, POINT pt)
{
    Window win = null;
    try
    {
        // Get the window under the cursor
        if (hWnd == IntPtr.Zero) hWnd = User32.WindowFromPoint(pt);
        if (hWnd == IntPtr.Zero) return null;

        win = new Window();
        win.hWnd = hWnd;

        // Get the rect of window
        bool b = User32.GetWindowRect(hWnd, out win.rect);

        StringBuilder sb = new StringBuilder(128);
        // Get the class name
        User32.GetClassName(hWnd, sb, sb.Capacity);
        win.className = sb.ToString();
        //Console.Write("cls: " + win.clsName);

        // Get the text length
        int n = (int)User32.SendMessage(hWnd, WM.GETTEXTLENGTH, 0, 0);
        if (n > 0)
        {
            // Get the text of window
            n = (int)User32.SendMessage(hWnd, WM.GETTEXT, (uint)sb.Capacity, sb);
            win.text = sb.ToString();
            //Console.WriteLine("; text: " + win.text);
        }

        // Get the window styles
        n = (int)User32.GetWindowLongPtr(hWnd, (int)User32.GWL.GWL_STYLE);
        if (n != 0)
        {
            if ((n & (uint)User32.WindowStyleFlags.WS_POPUP) != 0)
            	win.styles += ", WS_POPUP";
            if ((n & (int)User32.WindowStyleFlags.WS_CHILD) != 0)
            	win.styles += ", WS_CHILD";
            if ((n & (int)User32.WindowStyleFlags.WS_MINIMIZE) != 0)
            	win.styles += ", WS_MINIMIZE";
            if ((n & (int)User32.WindowStyleFlags.WS_VISIBLE) != 0)
            	win.styles += ", WS_VISIBLE";
            if ((n & (int)User32.WindowStyleFlags.WS_DISABLED) != 0)
            	win.styles += ", WS_DISABLED";
            if ((n & (int)User32.WindowStyleFlags.WS_CLIPSIBLINGS) != 0)
            	win.styles += ", WS_CLIPSIBLINGS";
            if ((n & (int)User32.WindowStyleFlags.WS_CLIPCHILDREN) != 0)
            	win.styles += ", WS_CLIPCHILDREN";
            if ((n & (int)User32.WindowStyleFlags.WS_MAXIMIZE) != 0)
            	win.styles += ", WS_MAXIMIZE";
            if ((n & (int)User32.WindowStyleFlags.WS_BORDER) != 0)
            	win.styles += ", WS_BORDER";
            if ((n & (int)User32.WindowStyleFlags.WS_DLGFRAME) != 0)
            	win.styles += ", WS_DLGFRAME";
            if ((n & (int)User32.WindowStyleFlags.WS_VSCROLL) != 0)
            	win.styles += ", WS_VSCROLL";
            if ((n & (int)User32.WindowStyleFlags.WS_HSCROLL) != 0)
            	win.styles += ", WS_HSCROLL";
            if ((n & (int)User32.WindowStyleFlags.WS_SYSMENU) != 0)
            	win.styles += ", WS_SYSMENU";
            if ((n & (int)User32.WindowStyleFlags.WS_THICKFRAME) != 0)
            	win.styles += ", WS_THICKFRAME";
            if ((n & (int)User32.WindowStyleFlags.WS_GROUP) != 0)
            	win.styles += ", WS_GROUP";
            if ((n & (int)User32.WindowStyleFlags.WS_TABSTOP) != 0)
            	win.styles += ", WS_TABSTOP";
            if ((n & (int)User32.WindowStyleFlags.WS_MINIMIZEBOX) != 0)
            	win.styles += ", WS_MINIMIZEBOX";
            if ((n & (int)User32.WindowStyleFlags.WS_MAXIMIZEBOX) != 0)
            	win.styles += ", WS_MAXIMIZEBOX";
            win.styles += ", WS_OVERLAPPED";
            win.styles = win.styles.Remove(0, 2);
        }
        else win.styles = string.Empty;

        // get the extended window style
        n = (int)User32.GetWindowLongPtr(hWnd, (int)User32.GWL.GWL_EXSTYLE);
        if (n != 0)
        {
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_DLGMODALFRAME)
            	!= 0) win.extendedStyles += ", WS_EX_DLGMODALFRAME";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_NOPARENTNOTIFY)
            	!= 0) win.extendedStyles += ", WS_EX_NOPARENTNOTIFY";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_TOPMOST)
            	!= 0) 	win.extendedStyles += ", WS_EX_TOPMOST";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_ACCEPTFILES)
            	!= 0) win.extendedStyles += ", WS_EX_ACCEPTFILES";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_TRANSPARENT)
            	!= 0) win.extendedStyles += ", WS_EX_TRANSPARENT";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_MDICHILD)
            	!= 0) win.extendedStyles += ", WS_EX_MDICHILD";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_TOOLWINDOW)
            	!= 0) win.extendedStyles += ", WS_EX_TOOLWINDOW";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_WINDOWEDGE)
            	!= 0) win.extendedStyles += ", WS_EX_WINDOWEDGE";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_CLIENTEDGE)
            	!= 0) win.extendedStyles += ", WS_EX_CLIENTEDGE";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_CONTEXTHELP)
            	!= 0) win.extendedStyles += ", WS_EX_CONTEXTHELP";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_RIGHT)
            	!= 0) win.extendedStyles += ", WS_EX_RIGHT";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_LEFT)
            	!= 0) win.extendedStyles += ", WS_EX_LEFT";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_RTLREADING)
            	!= 0) win.extendedStyles += ", WS_EX_RTLREADING";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_LTRREADING)
            	!= 0) win.extendedStyles += ", WS_EX_LTRREADING";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_LEFTSCROLLBAR)
            	!= 0) win.extendedStyles += ", WS_EX_LEFTSCROLLBAR";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_RIGHTSCROLLBAR)
            	!= 0) win.extendedStyles += ", WS_EX_RIGHTSCROLLBAR";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_CONTROLPARENT)
            	!= 0) win.extendedStyles += ", WS_EX_CONTROLPARENT";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_STATICEDGE)
            	!= 0) win.extendedStyles += ", WS_EX_STATICEDGE";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_APPWINDOW)
            	!= 0) win.extendedStyles += ", WS_EX_APPWINDOW";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_LAYERED)
            	!= 0) win.extendedStyles += ", WS_EX_LAYERED";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_NOINHERITLAYOUT)
            	!= 0) win.extendedStyles += ", WS_EX_NOINHERITLAYOUT";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_LAYOUTRTL)
            	!= 0) win.extendedStyles += ", WS_EX_LAYOUTRTL";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_COMPOSITED)
            	!= 0) win.extendedStyles += ", WS_EX_COMPOSITED";
            if ((n & (uint)User32.ExtendedWindowStyleFlags.WS_EX_NOACTIVATE)
            	!= 0) win.extendedStyles += ", WS_EX_NOACTIVATE";
            win.extendedStyles = win.extendedStyles.Remove(0, 2);
        }
        else win.extendedStyles = string.Empty;

        // Get parent, child, owner, next, previous
        win.parent = User32.GetParent(hWnd);
        win.owner = User32.GetAncestor(hWnd, 3);
        win.child = User32.GetWindow(hWnd, GW.CHILD);
        win.previous = User32.GetWindow(hWnd, GW.HWNDPREV);
        win.next = User32.GetWindow(hWnd, GW.HWNDNEXT);

        // Get Buttons
        int count = (int)User32.SendMessage(hWnd, TB.BUTTONCOUNT, 0, 0);
        //Console.WriteLine(count.ToString());
        if (count > 0)
        {
            win.itemType = "Button";
            win.itemStrings = new string[count];
            win.itemStyles = new string[count];

            unsafe
            {
                UInt32 processId = 0;
                UInt32 threadId = User32.GetWindowThreadProcessId(hWnd, out processId);
                IntPtr hProcess = Kernel32.OpenProcess
                	(ProcessRights.ALL_ACCESS, false, processId);

                // Create the local buffer of one page
                const int BUFFER_SIZE = 0x1000;
                byte* localBuffer = stackalloc byte[BUFFER_SIZE];
                IntPtr ipLocalBuffer = new IntPtr(localBuffer);
                Int32 bytesRead = 0;
                IntPtr ipBytesRead = new IntPtr(bytesRead);

                // Create the remote buffer of one page
                IntPtr ipRemoteBuffer = Kernel32.VirtualAllocEx(
                    hProcess,
                    IntPtr.Zero,
                    new UIntPtr(BUFFER_SIZE),
                    MemAllocationType.COMMIT,
                    MemoryProtection.PAGE_READWRITE);

                // Cast the local buffer pointer to the pointer of target data structures
                TBBUTTON* tb = (TBBUTTON*)localBuffer;
                TBBUTTONINFO* tbf = (TBBUTTONINFO*)localBuffer;
                int idCommand = 0;

                for (int i = 0; i < count; i++)
                {
                    // Get the button idCommand
                    n = (int)User32.SendMessage
                    	(hWnd, TB.GETBUTTON, (IntPtr)i, ipRemoteBuffer);
                    if (n == 0) continue;

                    // Read data from the remote buffer into the local buffer
                    b = Kernel32.ReadProcessMemory(
                        hProcess,
                        ipRemoteBuffer,
                        ipLocalBuffer,
                        (UInt32)sizeof(TBBUTTON),
                        ipBytesRead);
                    idCommand = tb->idCommand;

                    // Get the button text
                    n = (int)User32.SendMessage
                    	(hWnd, TB.GETBUTTONTEXTW, (IntPtr)idCommand, ipRemoteBuffer);
                    if (n != -1)
                    {
                        b = Kernel32.ReadProcessMemory(
                            hProcess,
                            ipRemoteBuffer,
                            ipLocalBuffer,
                            BUFFER_SIZE,
                            ipBytesRead);
                        win.itemStrings[i] =
			Marshal.PtrToStringUni((IntPtr)localBuffer, n);
                    }

                    // get button info
                    for (n = 0; n < BUFFER_SIZE; n++) localBuffer[n] = 0;

                    tbf->cbSize = (uint)Marshal.SizeOf(typeof(TBBUTTONINFO));
                    tbf->dwMask = 0x08 | 0x40;
                    //Const TBIF_STYLE = $00000008
                    //Const TBIF_LPARAM = $00000010
                    //Const TBIF_COMMAND = $00000020
                    //Const TBIF_SIZE = $00000040

                    // Write data from the local buffer into the remote buffer.
                    b = Kernel32.WriteProcessMemory(
                        hProcess,
                        ipRemoteBuffer,
                        ipLocalBuffer,
                        (uint)sizeof(TBBUTTONINFO),
                        out n);

                    // Get the button styles.
                    n = (int)User32.SendMessage
                    	(hWnd, TB.GETBUTTONINFO, (IntPtr)idCommand, ipRemoteBuffer);
                    if (n != -1)
                    {
                        b = Kernel32.ReadProcessMemory(
                            hProcess,
                            ipRemoteBuffer,
                            ipLocalBuffer,
                            (UInt32)sizeof(TBBUTTONINFO),
                            ipBytesRead);

                        string style = string.Empty;

                        //style += " -- (BTNS_BUTTON";
                        if ((tbf->fsStyle & BTNS.SEP)
                        	!= 0x00) style += ", BTNS_SEP";
                        if ((tbf->fsStyle & BTNS.DROPDOWN)
                        	!= 0x00) style += ", BTNS_DROPDOWN";
                        if ((tbf->fsStyle & BTNS.AUTOSIZE)
                        	!= 0x00) style += ", BTNS_AUTOSIZE";
                        if ((tbf->fsStyle & BTNS.WHOLEDROPDOWN)
                        	!= 0x00) style += ", BTNS_WHOLEDROPDOWN";

                        if (style.Length > 2) win.itemStyles[i] = style.Substring(2);
                        //Console.WriteLine(win.itemStyles[i]);
                    }
                }

                // Free remote buffer
                Kernel32.VirtualFreeEx(
                    hProcess,
                    ipRemoteBuffer,
                    UIntPtr.Zero,
                    MemAllocationType.RELEASE);

                // Release process handle
                Kernel32.CloseHandle(hProcess);
            }
        }
        else
        {
            // Get menu
            const uint MF_BYPOSITION = 0x400;
            const uint MN_GETHMENU = 0x1E1;
            //IntPtr hMenu = User32.GetMenu(hWnd); //not work
            IntPtr hMenu = User32.SendMessage
            	(hWnd, MN_GETHMENU, IntPtr.Zero, IntPtr.Zero);

            if (User32.IsMenu(hMenu))
            {
                win.itemType = "Menu Item";
                win.hMenu = hMenu;

                n = (int)User32.GetMenuItemCount(hMenu);
                win.itemStrings = new string[n];

                // the menu item info is always 0, so comment out.
                //User32.MENUITEMINFO mif = new User32.MENUITEMINFO();
                //mif.cbSize = (uint)Marshal.SizeOf(typeof(User32.MENUITEMINFO));
                for (uint i = 0; i < n; i++)
                {
                    //uint id = User32.GetMenuItemID(hMenu, i);
                    // Get menu the text of items
                    User32.GetMenuString(hMenu, i, sb, sb.Capacity, MF_BYPOSITION);
                    win.itemStrings[i] = sb.ToString();

                    //mif.fMask = 0x100; //MIIM_FTYPE
                    //b = User32.GetMenuItemInfo(hMenu, i, true, ref mif);
                    //Console.WriteLine
                    //("fType {0}: {1}\t{2}", b, mif.fType, sb.ToString());
                    // mif.fType
                }
            }
        }
    }
    catch (Exception e)
    {
        Console.WriteLine("Error in public Window getWindow(POINT pt): " + e.Message);
    }
    return win;
}

Global Low Level Keyboard and Mouse Hooks

This part creates the global low level keyboard and mouse hooks by calling function SetWindowsHookEx with parameter idHook value of WH_MOUSE_LL and WH_KEYBOARD_LL. The big problem here is the error, CallbackOnCollectedDelegate, which happens time to time. The whole error message is as below:

"CallbackOnCollectedDelegate was detected.

Message: A callback was made on a garbage collected delegate of type 'Mouse Tracker!Mouse Tracker.Form1+LowLevelProc::Invoke'. This may cause the application to crash, file corruption and data loss. When passing delegates to an unmanaged code, they must be kept alive by the managed application until it is guaranteed that they will never be called."

I searched the internet and found a lot of articles and talks about this error message but not one of them is actually helpful. The major hindrance to solving this error is that we do not know when the system will call GC.Collect() and we cannot control the system garbage collection. One suggestion from Microsoft here is to create a static callback, but it does not work with Mouse Tracker because of its complexity. After many tests and failures, I found the following practical solutions:

  1. To reduce the CPU time and resource occupied by the mouse event callback, the callback does not process the second and rest events within 200 milliseconds if they are mouse move events.
  2. To reduce the CPU time and resource occupied by the mouse event callback, the callback raises a local defined event to handle the complicated work of retrieving and displaying window information. The callback itself returns as soon as possible.
  3. To avoid system calling GC.Collect() at a bad time for Mouse Tracker, when every time Mouse Tracker completes processing a mouse (or keyboard) event, Mouse Tracker exits the global low level mouse (or keyboard) hook, calls GC.Collect()actively, and then reenters a new hook. Thus the system will not call GC.Collect() again. This approach is most important and effective for solving the error.

I also found out while a button on Mouse Tracker or a cell in a data grid view is clicked, the mouse hook is broken and the callback function will not be called again, so SetHook(WH_MOUSE_LL) is recalled in the codes.

C#
/// <summary>
///  Set global low-level keyboard and mouse event hook
/// </summary>
/// <param name="idHook"></param>
public void SetHook(int idHook)
{
    using (Process curProcess = Process.GetCurrentProcess())
    using (ProcessModule curModule = curProcess.MainModule)
    {
        switch (idHook)
        {
            case WH_MOUSE_LL:
                if (mouseHookID != IntPtr.Zero) User32.UnhookWindowsHookEx(mouseHookID);
                // Call GC.Coolect() to avoid system calling it in hook
                GC.Collect();
                // Mouse event hook
                mouseHookID = SetWindowsHookEx(idHook, MouseHookProc, 
			Kernel32.GetModuleHandle(curModule.ModuleName), 0);
                break;
                
            case WH_KEYBOARD_LL:
                if (keyboardHookID != IntPtr.Zero) 
			User32.UnhookWindowsHookEx(keyboardHookID);
                // // Call GC.Coolect() to avoid system calling it in hook
                GC.Collect();
                // Keyboard event hook
                keyboardHookID = SetWindowsHookEx(idHook, KeyboardHookProc, 
			Kernel32.GetModuleHandle(curModule.ModuleName), 0);
                break;
        }
    }
}

/// <summary>
///  Mouse event hook callback
/// </summary>
/// <param name="nCode"></param>
/// <param name="wParam"></param>
/// <param name="lParam"></param>
/// <returns></returns>
public IntPtr Mouse_HookCallback(int nCode, IntPtr wParam, IntPtr lParam)
{
    try
    {
        if (nCode < 0) return User32.CallNextHookEx(mouseHookID, nCode, wParam, lParam);
        
        POINT point = ((MSLLHOOKSTRUCT)Marshal.PtrToStructure
			(lParam, typeof(MSLLHOOKSTRUCT))).pt;
        
        MouseMessages mouseMessage = 0;
        switch ((MouseMessages)wParam)
        {
            case MouseMessages.WM_MOUSEMOVE:
            case MouseMessages.WM_LBUTTONDOWN:
            case MouseMessages.WM_RBUTTONDOWN:
            case MouseMessages.WM_LBUTTONUP:
            case MouseMessages.WM_RBUTTONUP:
                mouseMessage = (MouseMessages)wParam;
                break;
                
            default:
                // Do not process other events.
                return User32.CallNextHookEx(mouseHookID, nCode, wParam, lParam);
        }
        
        IntPtr hWnd = IntPtr.Zero;
        if (mouseMessage != MouseMessages.WM_MOUSEMOVE)
        { // over form1
            IntPtr hOwner = tool.getRootOwner(point, out hWnd);
            if (hOwner == Handle)
            {
                // The event happened on Form1, treat it as a mouse move event.
                mouseMessage = MouseMessages.WM_MOUSEMOVE;
                // Allow buttons on Form1 to receive clicking immediately.
                this.Activate();
            }
        }
        
        int nowMs = DateTime.Now.Millisecond;
        if (mouseMessage == MouseMessages.WM_MOUSEMOVE &&
            nowMs - refTimeMs < 200 && nowMs > refTimeMs)
        {
            // Filter out the mouse move events which happened in 200 ms.
            return User32.CallNextHookEx(mouseHookID, nCode, 
				(IntPtr) mouseMessage, lParam);
        }
        
        // Prepare parameters and raise a local mouse event
        MouseActionEventArgs e = new MouseActionEventArgs(mouseMessage, point, hWnd);
        OnMouseAction(e);
        
        // Change reference time
        refTimeMs = DateTime.Now.Millisecond;
    }
    catch (Exception e)
    {
        Debug.WriteLine("Error in private IntPtr Mouse_HookCallback
		(int nCode, IntPtr wParam, IntPtr lParam): " + e.Message);
    }
    return User32.CallNextHookEx(mouseHookID, nCode, wParam, lParam);
}
C#
// Local mouse action event handler
public void MouseAction(object sender, MouseActionEventArgs e)
{
    Form1 form = (Form1)sender;
    DataGridViewRowCollection rows = form.dataGridView1.Rows;
    
    rows[form.index].Cells[1].Value = e.point.x;
    rows[form.index].Cells[2].Value = e.point.y;
    
    switch (e.mouseMessage)
    {
        case MouseMessages.WM_MOUSEMOVE:
            if (form.isNewRow)
            {
                rows[form.index].Cells[0].Value = "Move";
                form.isNewRow = false;
            }
            
            // Exits the global low level mouse hook, calls GC.Collect() actively, 
            // and then reenters a new hook
            form.SetHook(Form1.WH_MOUSE_LL);
            return;
            
        case MouseMessages.WM_LBUTTONDOWN:
            rows[form.index].Cells[0].Value = "Left Button Down";
            break;
            
        case MouseMessages.WM_RBUTTONDOWN:
            rows[form.index].Cells[0].Value = "Right Button Down";
            break;
            
        case MouseMessages.WM_LBUTTONUP:
            rows[form.index].Cells[0].Value = "Left Button Up";
            break;
            
        case MouseMessages.WM_RBUTTONUP:
            rows[form.index].Cells[0].Value = "Right Button Up";
            break;
    }
    
    Window win = new Tool().getWindow(e.hWnd, e.point);
    if (win != null)
    {
        //high light window's fame
        switch (e.mouseMessage)
        {
            case MouseMessages.WM_LBUTTONDOWN:
            case MouseMessages.WM_RBUTTONDOWN:
                drawFrame(win);
                lastWin = win;
                break;
                
            case MouseMessages.WM_LBUTTONUP:
            case MouseMessages.WM_RBUTTONUP:
                drawFrame(lastWin);
                
                if (lastWin.hWnd != win.hWnd)
                {
                    drawFrame(win);
                    Thread thd = new Thread(drawFrame);
                    thd.Name = "delay";
                    thd.Start(win);
                }
                lastWin = null;
                
                break;
        }
        
        // Exits the global low level mouse hook, calls GC.Collect() actively, 
        // and then reenters a new hook
        form.SetHook(Form1.WH_MOUSE_LL);
        
        if (!form.windows.ContainsKey(win.hWnd)) form.windows.Add(win.hWnd, win);
        rows[form.index].Cells[3].Value = win.hWnd.ToString();
        
        form.dataGridView1.ClearSelection();
        rows[form.index++].Cells[3].Selected = true;
        form.isNewRow = true;
        if (form.index >= form.dataGridView1.Rows.Count) 
			form.StopTracking_Click(null, null);
        
        if (!form.splitContainer1.Panel2Collapsed) form.showWindowProperties(win.hWnd);
    }
}

Suspending and Resuming Windows

Suspending and resuming windows are to keep a popup window opening in the screen after the mouse button being released. The called functions are Kernal32.SuspendThread and Kernal32.ResumeThread. If Kernal32.SuspendThread is called more times, Kernal32.ResumeThread has to be called the same times to resume the window. So Mouse Tracker has to ensure calling Kernal32.SuspendThread only once on each window. Another demand is to resume all suspended windows before the application shutdown.

C#
/// <summary>
/// The list keeps suspended windows
/// </summary>
List<IntPtr> suspendHWnds = new List<IntPtr>();
/// <summary>
/// Keyboard event hook callback
/// </summary>
/// <param name="nCode"></param>
/// <param name="wParam"></param>
/// <param name="lParam"></param>
/// <returns></returns>
public IntPtr KeyBoard_HookCallback(int nCode, IntPtr wParam, IntPtr lParam)
{
    if (nCode >= 0 && wParam == (IntPtr)WM_KEYDOWN)
    {
        int vkCode = Marshal.ReadInt32(lParam);
        switch ((Keys)vkCode)
        {
            case Keys.F1:
            // Suspend the window under the cursor
                if (!StartTracking.Enabled) break;
                
                POINT point;
                point.x = System.Windows.Forms.Cursor.Position.X;
                point.y = System.Windows.Forms.Cursor.Position.Y;
                IntPtr hWnd = new Tool().getOwner(point);
                if (hWnd == this.Handle) return new IntPtr(1);
                
                uint thrdId = User32.GetWindowThreadProcessId(hWnd, IntPtr.Zero);
                IntPtr hThrd = Kernel32.OpenThread
		(Kernel32.ThreadAccess.SUSPEND_RESUME, true, thrdId);
                
                //  Ensure a window to be  suspended only once.
                if (!suspendHWnds.Contains(hWnd)) Kernel32.SuspendThread(hThrd);
                bool b = Kernel32.CloseHandle(hThrd);
                suspendHWnds.Add(hWnd);
                
        	       // Exits the global low level keyboard hook,
        	       // calls GC.Collect() actively, and then reenters a new hook
                SetHook(WH_KEYBOARD_LL);
                return new IntPtr(1);
                
            case Keys.F2:
            // Resume all resumed windows
                ResumeWindows();
                
            // Exits the global low level keyboard hook,
        	   // calls GC.Collect() actively, and then reenters a new hook
                SetHook(WH_KEYBOARD_LL);
                return new IntPtr(1);
        }
    }
    return User32.CallNextHookEx(keyboardHookID, nCode, wParam, lParam);
}

/// <summary>
/// Resume all the suspended windows
/// </summary>
void ResumeWindows()
{
    foreach (IntPtr hWnd in suspendHWnds)
    {
        uint thrdId = User32.GetWindowThreadProcessId((IntPtr)hWnd, IntPtr.Zero);
        IntPtr hThrd = Kernel32.OpenThread
		(Kernel32.ThreadAccess.SUSPEND_RESUME, true, thrdId);
        while (Kernel32.ResumeThread(hThrd) > 0);
        bool b = Kernel32.CloseHandle(hThrd);
    }
    suspendHWnds.Clear();
}

Simulation of Mouse Actions

Simulating mouse actions calls User32.SetCursorPos to move the mouse, and User32.mouse_event to simulate mouse button clicks.

C#
/// <summary>
/// Simulate mouse actions
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
private void Simulate_Click(object sender, EventArgs e)
{
    //object rows = (dataGridView1.SelectedRows.Count > 0) ?  
		dataGridView1.SelectedRows : dataGridView1.Rows;
    POINT point;
    point.x = point.y = 0;
    IntPtr hWnd = IntPtr.Zero;
    foreach (DataGridViewRow row in dataGridView1.Rows)
    {
        Thread.Sleep(40);
        try
        {
            int x = int.Parse(row.Cells[1].Value.ToString());
            int y = int.Parse(row.Cells[2].Value.ToString());
            point.x = x;
            point.y = y;
            hWnd = windows[(IntPtr)int.Parse(row.Cells[3].Value.ToString())].owner;
            
            MouseEventFlags flag = MouseEventFlags.MOVE;
            switch (row.Cells[0].Value.ToString())
            {
                case "Move":
                    User32.SetCursorPos(x, y);
                    continue;
                    
                case "Left Button Down":
                    flag = MouseEventFlags.LEFTDOWN;
                    break;
                    
                case "Right Button Down":
                    flag = MouseEventFlags.RIGHTDOWN;
                    break;
                    
                case "Left Button Up":
                    flag = MouseEventFlags.LEFTUP;
                    break;
                    
                case "Right Button Up":
                    flag = MouseEventFlags.RIGHTUP;
                    break;
            }
            
            // Move the cursor
            User32.SetCursorPos(x, y);
            // Click a mouse button
            User32.mouse_event(flag, 0, 0, 0, 0);
            
        }
        catch
        {
            continue;
        }
    }
}

Highlighting the Frame of Window

During mouse tracking, Mouse Tracker will highlight the frame of a window on which a mouse button is pressed down or released up; and then Mouse Tracker will unhighlight the frame when the next mouse button event raises or after 200 milliseconds. Mouse Tracker calls function DrawReversibleFrame to implement both highlight and unhighlight.

C#
void drawFrame(object win)
{
    if (Thread.CurrentThread.Name != null) Thread.Sleep(200);
    
    System.Windows.Forms.ControlPaint.DrawReversibleFrame(
        new System.Drawing.Rectangle(((Window)win).rect.X, 
	((Window)win).rect.Y, ((Window)win).rect.Width, ((Window)win).rect.Height),
        System.Drawing.Color.Black,
        System.Windows.Forms.FrameStyle.Thick);
}

Conclusion

I have introduced a practical usage of a number of complex functions in the article and the application, Mouse Tracker. Mouse tracker is not a simple demo that shows simple results when calling separate functions, but rather a practical utility application to provide some useful information. I have put huge effort into finding the correct usages of low level keyboard and mouse hooks, User32 and Kernel32 APIs, and their related functions; and making them work together. Hopefully, both my effort and experience will make your C# development easier and more successful.

I'd appreciate your vote; it is a powerful motivating force for me.

Thanks James, my son, for revising the writing of the article.

Reference

I have used the free Fwdw_icons in Mouse Tracker.

History

  • 25th November, 2009: Update for running on 64-bit computers
  • 20th November, 2009: Update for running on computers with a small amount of RAM
  • 11th November, 2009: Initial version

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
United States United States
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

 
GeneralMy vote of 5 Pin
conrad Braam2-Dec-19 3:25
conrad Braam2-Dec-19 3:25 
Generalapp to record Test case and record mouse movement and keyboard clicks Pin
Member 117448536-Jun-15 20:31
Member 117448536-Jun-15 20:31 
QuestionUnable to capture message of messagebox. Pin
Rajeev from Patna29-Jul-12 19:17
Rajeev from Patna29-Jul-12 19:17 
Generalis crashing.. Pin
Jochen Baier13-Nov-09 11:21
Jochen Baier13-Nov-09 11:21 
AnswerRe: is crashing.. Pin
Dianyang Wu15-Nov-09 7:39
Dianyang Wu15-Nov-09 7:39 
GeneralRe: is crashing.. Pin
Jochen Baier15-Nov-09 10:38
Jochen Baier15-Nov-09 10:38 
GeneralRe: is crashing.. Pin
Dianyang Wu15-Nov-09 12:58
Dianyang Wu15-Nov-09 12:58 
GeneralRe: is crashing.. Pin
Shine100116-Nov-09 8:49
Shine100116-Nov-09 8:49 
GeneralRe: is crashing.. Pin
Dianyang Wu20-Nov-09 15:47
Dianyang Wu20-Nov-09 15:47 
Generalnew version still crashing...(NT) Pin
Jochen Baier20-Nov-09 9:21
Jochen Baier20-Nov-09 9:21 
GeneralRe: new version still crashing...(NT) Pin
Dianyang Wu20-Nov-09 15:22
Dianyang Wu20-Nov-09 15:22 
GeneralRe: new version still crashing...(NT) Pin
Member 63821-Nov-09 22:03
Member 63821-Nov-09 22:03 
GeneralRe: new version still crashing...(NT) Pin
Dianyang Wu26-Nov-09 5:55
Dianyang Wu26-Nov-09 5:55 
GeneralRe: new version still crashing...(NT) Pin
Dianyang Wu26-Nov-09 5:42
Dianyang Wu26-Nov-09 5:42 
GeneralNice Pin
dequadin13-Nov-09 8:35
dequadin13-Nov-09 8:35 
GeneralRe: Nice Pin
Dianyang Wu15-Nov-09 8:10
Dianyang Wu15-Nov-09 8:10 
GeneralRe: Nice Pin
Stephen Swensen20-Nov-09 7:20
Stephen Swensen20-Nov-09 7:20 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.