Handling Touch, Pen, or Mouse Digitizer input in your .NET Application

• Download demo project - 235.5 KB
• Download sample files - 16.4 KB

Introduction

Using the included Tablet Technology that is built into Windows Vista, you can enable your application to perform unique actions for different input devices such as pen, mouse and touch input - even on a desktop PC.

Background

With Windows Vista, the software features of the Tablet PC are built into the core OS. This means that you can add support for Ink, handwriting recognition and in your Vista application without having to think about re-distributing dll's or worrying about whether your user is on a Tablet PC or not. In Windows XP, you had to have a Tablet PC in order to even think about using a pen as input using the API's in the OS.

What can I do with this kind of feature?

Digitizers have been used for years by graphics artists and the Tablet PC hardware integrated that input right into the display. Now, touch (resistive and capacitive) digitizers and electro-magnetic digitizers can be found on Mobile PC's as well as desktop LCD displays. Some PC's even have dual-mode digitizers integrated into them so you can handwrite on the screen if the pen is in range and then will automatically switch to touch mode when you use your finger on the screen. I think this is a great opportunity to add new functionality to applications to make use of this.

Think about this scenario: You have a photo application that allows you to handwrite keywords right on the picture using a Pen (that's cool in itself) and then when you reach out and touch the display with your finger, the application switches to panning mode so you can move the photo around. Pretty cool.

Using the code

I used the existing RealTimeStylus sample application that is in the Windows SDK as the base for this sample. The RealTimeStylus object (RTS) allows you to collect data from an attached digitizer or mouse device and gives you the ability to manipulate that data on the fly. You can add, remove & re-order different plug-ins with RealTimeStylus, which means you can change the way the input is rendered dynamically as well.

Since the sample application has most of the functionality I need, I want to add the ability to determine the input device that the current user input is coming from. RealTimeStylus is part of the Tablet PC features, which implies the use of ink, but that doesn't have to be the case. I can just use the input from RTS for whatever I want.

Let's start with making sure we receive the correct events by subscribing to the DataInterestMask. I want to make sure we get notification of when the input starts, when the application is receiving data, and of any errors that might occur:

public DataInterestMask DataInterest
{
    get
    {
        return DataInterestMask.StylusDown | DataInterestMask.Packets | 
            DataInterestMask.Error;
    }
}

Next, I need to make sure I add handlers for the above notifications. The steps for determining where the input originated, and what to do with it are straightforward:

1. In the StylusDown event, query for the originating input device type.
2. In the Packets event, choose to do something with the inbound data. In my case, I'll draw different colored ellipses depending on which input device from which the packets are coming.

The StylusDown event is very similar to a MouseDown event - it notifies you that some input is about to happen. This gives us opportunity to query which input device that with which the user is interacting. The Tablet object here describes all the capabilities of the current input device, including the type, or DeviceKind enumeration.

public void StylusDown(RealTimeStylus sender, StylusDownData data) 
{
    Tablet currentTablet = 
        sender.GetTabletFromTabletContextId(data.Stylus.TabletContextId);

    if(currentTablet != null)
    {
        // Keep track of the current input device type
        tabletKind = currentTablet.DeviceKind;
    }        
}

Now that you know which input device is about to create data, a handler for the Packets event needs to be added. This simply switches on the current TabletDeviceKind that we kept track of in the previous handler and performs some different action for each device type. I'm just going to draw different colored ellipses for each input device:

public void Packets(RealTimeStylus sender,  PacketsData data)
{           
    // For each new packet received, extract the x,y data
    // and draw a small circle around the result.
    for (int i = 0; i < data.Count; i += data.PacketPropertyCount)
    {
        // Packet data always has x followed by y followed by the rest
        Point point = new Point(data[i], data[i+1]);

        // Since the packet data is in Ink Space coordinates, convert to 
        // Pixels
        point.X = (int)Math.Round((float)point.X * 
            (float)myGraphics.DpiX/2540.0F);
        point.Y = (int)Math.Round((float)point.Y * 
            (float)myGraphics.DpiY/2540.0F);

        // Draw a circle corresponding to the packet
        switch (this.tabletKind)
        {
            case TabletDeviceKind.Pen:
                // Make the packets from the stylus smaller and green
                myGraphics.DrawEllipse(Pens.Green, point.X - 2, point.Y - 2, 
                    4, 4);
                break;
            case TabletDeviceKind.Mouse:
                // Make the packets from the mouse/pointing device mid-sized 
                // and red
                myGraphics.DrawEllipse(Pens.Red, point.X - 2, point.Y - 2, 
                    10, 10);
                break;
            case TabletDeviceKind.Touch:
                // Make the packets from a finger/touch digitizer larger and 
                // blue
                myGraphics.DrawEllipse(Pens.Blue, point.X - 2, point.Y - 2, 
                   20, 20);
                break;
        }
    }
}

Points of Interest

The sample above is inspecting only the X and Y coordinates of the packets coming in from the Mouse, Touch or Pen devices. However, digitizers can have additional packet data types as well. Electro-magnetic digitizers (either built into Tablet PC's or USB external digitizers) can also have pressure data available. Adding this support would allow this sample application to alter the size of each ellipse drawn based on how hard the user was pressing on the digitizer. Perhaps that will be an update to this article.

These API's are new to Windows Vista and are available in .NET 2.0, C++ COM, and in .NET 3.0's Windows Presentation Foundation (WPF).

History

6th of March, 2007 - Created article