Download src_vd - 1.01 MB

List of contents
1. Introduction
1.1 Why to process requests in user mode?
1.2 Article structure
1.3 Technologies used
1.4 Project structure
1.5 Project directory structure
2. Windows and disks
3. Implementation
3.1 Stage 1. Initialization
3.2 Stage 2. Disk mounting
3.3 Stage 3. Processing requests
3.4 Stage 4. Unmounting
4. How to build this Solution
4.1 Testing
4.2 Supported Windows Versions
5. Conclusion

Introduction

This article is oriented to Windows-developers and considers how to create virtual disk in Windows system.

In Windows virtual disks are implemented by means of kernel mode drivers. Some necessary information concerning disk drivers for Windows is given in the section “Windows and disks”.

The implementation of virtual disk for Windows can also be found in the widely known project with open sources FileDisk (http://www.acc.umu.se/~bosse/ ). The main difference of the solution, which is given in this article, is that FileDisk processes requests to the virtual disk in kernel mode while the proposed solution processes them in user mode.

Why to process requests in user mode?

This approach can be useful when there is already developed code of user mode that provides access to the data source (there can be any data source, e.g. disk image in memory, remote disk, or cash register) and it’s hard to port this code to the kernel mode; or there is no source code at all, for example when performing network access or using specific encryption library.
As an example of such user mode library I used SparseFile (file-container with some data; as the data is accumulated it will size up to the maximal size).

Article structure

In section «Windows and disks» we will discuss the way the Windows interacts with the disks; possible variants of virtual disk creation; necessary information for virtual disk driver development.

In section «Implementation» the solution architecture is considered as well as key implementation aspects and main stages of our disk’s life cycle.

The section «How to build this Solution» contains a couple of phrases about how to build and test the project.

Technologies used

C++. The whole project with both kernel mode and user mode parts is developed in C++ with the utilizing of the exceptions. There are a lot of contradictory opinions concerning the utilizing of C++ for driver development. Here is the good article on this topic http://www.osronline.com/article.cfm?article=490.
CppLib (http://www.acc.umu.se/~bosse/). This beautiful library made it possible to develop drivers in C++.
STLPort (http://www.stlport.org/). A standard library.
BOOST. (http://www.boost.org/). Really useful library.

Project structure

Solution includes several projects:

CoreMntTest (user mode, executable) – creates disk image and mounts it using the code from CoreMnt_user.
CoreMnt_user (user mode, library) – receives requests to virtual disks from the CoreMnt driver and serves them.
UsrUtils (user mode, library) – contains auxiliary code of the interaction with drivers using DeviceIoControl.
CoreMnt (kernel mode, executable) – implements OS requirements for the disks; performs request transformation; sends them to CoreMnt_user for serving.
drvUtils (kernel mode, headers only library) – auxiliary code of kernel mode, for example synchronization tools.

The scheme below represents the projects’ relations.

virtual-disk-tutorial/pic2.png

Project directory structure

.\bin - folder with binary files
.\lib - folder with library files
.\obj - folder with object files
.\src - folder with source files
|
|-> .\CoreMnt       - Kernel mode driver.
|-> .\CoreMnt_user - User mode mount library.
|-> .\CoreMntTest   - User mode mount test.
|-> .\drvCppLib     - Kernel Library to develop driver in C++.
|-> .\drvUtils      - Kernel Library with utils for kernel mode projects.
|-> .\mnt_cmn       - Files that are shared between projects.
|-> .\STLPort       - Directory with STLPort 4.6 ported for utilizing in windows drivers.
|-> .\usrUtils      - Win32 Library with utils for user mode projects.

Windows and disks

If you are acquainted with driver development for Windows and have developed at least one even the simplest driver then you can skip this section.

I’m glad to appease the rest – everything is very simple here. Windows sends a request “write” or “read” to the disk. The disk returns read data or error code. And that’s all.

Surely there are some nuances, how it can be without them.

Let’s consider a simplified scheme of processing of requests to the disk. So, what is going on after an application calls, for example, ReadFile function. First the read file request is received by the file system driver (for example ntfs.sys). The scheme illustrates this process:

virtual-disk-tutorial/pic5.png

File system driver detects where the requested file is exactly situated on the disk (with what offset) and forms the read disk request. A file can be divided into parts, which can be located at different places on the disk. In this situation several requests will be formed. Just those very requests will be received by our virtual disk driver from the file system driver. And by the way, virtual disk can be also implemented at the file system level, see details in the article fs-filter-driver-tutorial.aspx .

Terms listed below will be used in this article.

Implementation

The solution itself is a driver (CoreMnt.sys) and application (CoreMntTest.exe). The general scheme is as follows.

virtual-disk-tutorial/pic1.png

Driver provides the service of disk mounting. Application creates data source and mounts it as a disk using the service. Driver receives IRP, process them in user mode and returns the result. General scheme of driver work is represented on the figure.

virtual-disk-tutorial/pic3.png

Application (CoreMntTest.exe) serves the requests from OS to the virtual disks. Structural scheme is represented on the figure.

virtual-disk-tutorial/pic4.png

Now let’s consider how it looks in the source code, stage by stage.

Stage 1. Initialization

On this stage we start CoreMnt driver by means of the command:

c:\>sc start CoreMnt

We should create the management device in DriverEntry as an access point for CoreMntTest from user mode:

NTSTATUS DriverEntry(IN PDRIVER_OBJECT DriverObject,
                     IN PUNICODE_STRING RegistryPath)
{
...
    NTSTATUS status;
    status = IoCreateDevice(DriverObject,     // pointer on DriverObject
                            0,                // additional size of memory
                            &gDeviceName,     // pointer to UNICODE_STRING
                            FILE_DEVICE_NULL, // Device type
                            0,                // Device characteristic
                            FALSE,            // "Exclusive" device
                            &gDeviceObject);  // pointer do device object
    if (status != STATUS_SUCCESS)
        return STATUS_FAILED_DRIVER_ENTRY;

    status = IoCreateSymbolicLink(&gSymbolicLinkName,&gDeviceName);
    if (status != STATUS_SUCCESS)
        return STATUS_FAILED_DRIVER_ENTRY;

Next step is the register the driver request handler. We will use a single handler for all request types:

    for (size_t i = 0; i <= IRP_MJ_MAXIMUM_FUNCTION; ++i) 
        DriverObject->MajorFunction[i] = IrpHandler;

And finally let’s create MountManager:

    gMountManager = new MountManager(DriverObject);
    return STATUS_SUCCESS;
}

Stage 2. Disk mounting

On this stage we start application CoreMntTest.exe. It sends management message CORE_MNT_MOUNT_IOCTL to the driver:

    CORE_MNT_MOUNT_REQUEST request;
    request.totalLength = totalSize;
    request.mountPojnt = mountPoint;
    DWORD bytesWritten = 0;
    CORE_MNT_MOUNT_RESPONSE response;
    if(!m_coreControl.DeviceIoGet(CORE_MNT_MOUNT_IOCTL, 
                                  &request, 
                                  sizeof(request),
                                  &response,
                                  sizeof(response),
                                  &bytesWritten))
    {
        throw std::exception(__FUNCTION__" DeviceIoGet failed.&);
    }

Function DispatchMount deserialize request parameter and calls MountManager::Mount:

        if(inputBufferLength < sizeof(CORE_MNT_MOUNT_REQUEST) || 
            outputBufferLength < sizeof(CORE_MNT_MOUNT_RESPONSE) )
        {
            throw std::exception(__FUNCTION__" buffer size mismatch");
        }
        DISK_PROPERTIES diskProperties;
        CORE_MNT_MOUNT_REQUEST * request = (CORE_MNT_MOUNT_REQUEST *)buffer;
        diskProperties.totalLength.QuadPart = request->totalLength;
        CORE_MNT_MOUNT_RESPONSE * response = 
            (CORE_MNT_MOUNT_RESPONSE *)buffer;
  response->deviceId = gMountManager->Mount(&diskProperties);

In MountManager::Mount we create an object of the class MountedDisk and save it. MountedDisk includes LogicIrpDispatcher. Its constructor creates disk device. OS will sends requests to this device:

LogicIrpDispatcher::LogicIrpDispatcher(PDISK_PROPERTIES diskProperties, 
                                       PDRIVER_OBJECT   DriverObject,
                                       MountManager* mountManager)
{
...
    //create device
    status = IoCreateDevice(DriverObject,sizeof(InThreadDeviceExtension),
        &deviceName,FILE_DEVICE_DISK,
        0,
        FALSE,&deviceObject_);
    if (!NT_SUCCESS(status))
        throw std::exception(__FUNCTION__" can't create device.");

After the device is created we have to initialize DeviceExtension. We want to use it for storing device identifier. Thus when we get IRP we will easily find the corresponding MountedDisk:

    InThreadDeviceExtension* devExt = 
        (InThreadDeviceExtension*)deviceObject_->DeviceExtension;
    memset(devExt, 0, sizeof(InThreadDeviceExtension));

    devExt->mountManager = mountManager;
    devExt->deviceId = diskProperties->deviceId;

So at the moment MountManager has created an instance of MountedDisk and saved it to the container. Initialization stage is finishing in the user mode. A thread is creates for each disk and all its requests are served there. A thread sends IOCTL RequestExchange to the driver and proceeds to the request awaiting mode:

    while(true)
    {
        int type = 0;
        int size = 0;
        __int64 offset = 0;
        drvCtrl->RequestExchange(deviceId, lastType, lastStatus, lastSize, &dataBuf[0], dataBuf.size(),
                                 &type, &size, &offset);

        //do requested operation
        DispatchImageOperation(image, type, size, offset, &dataBuf[0], dataBuf.size(), &lastStatus);
        lastType = type;
        lastSize = size;
    }

Performance note: serving requests in one thread is surely the «bottle neck». There will be certainly a thread pool in the real project.

Stage 3. Processing requests

So, our virtual disk is ready to process requests. Let’s follow the complete sequence in request processing. It all begins with the function IrpHandler, which is registered by our driver as the procedure of IRP processing. Here we get the device identifier from DeviceExtension (we saved it there on initialization stage) and transmit IRP to MountManager:

NTSTATUS IrpHandler( IN PDEVICE_OBJECT fdo, IN PIRP pIrp )
{
...
        InThreadDeviceExtension* devExt = 
            (InThreadDeviceExtension*)fdo->DeviceExtension;
        return gMountManager->DispatchIrp(devExt->deviceId, pIrp);

MountManager receives IRP, finds the corresponding MountedDisk by device identifier, and redirects IRP to it. The code below decides if it is possible to process this request at once or it should be processed in the user mode:

NTSTATUS MountedDisk::DispatchIrp(PIRP irp)
{
    IrpParam irpParam(0,0,0,0);
    irpDispatcher_.getIrpParam(irp, &irpParam);
    if(irpParam.type == directOperationEmpty)
    {
...
            irpDispatcher_.dispatch(irp);
...
        NTSTATUS status = irp->IoStatus.Status;
        IoCompleteRequest(irp, IO_NO_INCREMENT);
        return status;
    }
    IoMarkIrpPending( irp );
    irpQueue_.push(irp);
    return STATUS_PENDING;
}

Decision making is simple: if it is IRP_MJ_READ or IRP_MJ_WRITE then it should be processed in user mode. Driver can process all other requests itself. For example IOCTL_DISK_GET_LENGTH_INFO: our driver knows the disk size, and it knows also that the disk size cannot be changed. The complete list of requests, which Windows can send to the disk, can be found in LogicIrpDispatcher::dispatchIoctl.

The thread that serves this disk selects requests from the list:

void MountedDisk::RequestExchange(UINT32 lastType, UINT32 lastStatus, UINT32 lastSize, char* buf, UINT32 bufSize,
                     UINT32 * type, UINT32 * length, UINT64 * offset)
{
...
    NTSTATUS status = KeWaitForMultipleObjects(sizeof(eventsArray)/sizeof(PVOID), eventsArray, WaitAny, 
        Executive, KernelMode, FALSE, NULL, 0);
...
            IrpParam irpParam(0,0,0,0);
            irpDispatcher_.getIrpParam(lastIrp_, &irpParam);
            *type = irpParam.type;
            *length = irpParam.size;
            *offset = irpParam.offset;

If it is IRP_MJ_WRITE, data to write will be copied to the buffer. Then this buffer will be passed to the user mode code:

        if(*type != directOperationEmpty 
            && opType2DirType(directOperationTypes(*type)) == directOperationWrite)
        {
            IrpParam irpParam(0,0,0,0);
            irpDispatcher_.getIrpParam(lastIrp_, &irpParam);

            if(irpParam.buffer)
                memcpy(buf, irpParam.buffer, *length);

After returning from RequestExchange function we will get to the cycle of request processing (DispatchImage) again:

    while(true)
    {
        int type = 0;
        int size = 0;
        __int64 offset = 0;
        drvCtrl->RequestExchange(deviceId, lastType, lastStatus, lastSize, &dataBuf[0], dataBuf.size(),
                                 &type, &size, &offset);

        //do requested operation
        DispatchImageOperation(image, type, size, offset, &dataBuf[0], dataBuf.size(), &lastStatus);
        lastType = type;
        lastSize = size;
    }

The variables type, size, offset now contain the new request to be processed. It’s a task for DispatchImageOperation function:

void DispatchImageOperation(IImage * image, 
                            int type, int size, __int64 in_offset, char* buf,
                            int bufsize,
                            int* status)
{
    switch(type)
    {
...
    case directOperationRead:
        {
           image->Read((char*)buf, in_offset, size);
           *status = 0;
            break;
        }
    case directOperationWrite:
        {    
            image->Write((const char*)buf, in_offset, size);
            *status = 0;
            break;
        }

After the request is served the function RequestExchange will be called again and the thread will proceed to the new request awaiting mode.

Stage 4. Unmounting

This stage starts in user mode with the call of UnmountImage function. Code below checks if the disk is being used at the moment:

void UnmountImage(int devId, wchar_t mountPoint, DriverControl * drvCtrl)
{
...
        if (!DeviceIoControl(hVolume,FSCTL_LOCK_VOLUME,NULL,
                             0,NULL,0,&BytesReturned,NULL))
        {
            throw std::exception("Unable to lock logical drive");
        }
        else if (!DeviceIoControl(hVolume,FSCTL_DISMOUNT_VOLUME,
                                  NULL,0,NULL,0,&BytesReturned,NULL))
        {
            throw std::exception("Unable to dismount logical drive");
        }
        else if (!DeviceIoControl(hVolume,FSCTL_UNLOCK_VOLUME,NULL,
                                  0,NULL,0,&BytesReturned,NULL))
        {
            throw std::exception("Unable to unlock logical drive");
        }

Then we destroy the connection of mounting point with our device:

    if (UndefineLogicDrive(mountPoint))
        throw std::exception("Unable to undefine logical drive");

And then we send a message for all those components that store the disk list in the system (for example explorer.exe or any other file manager):

    SHChangeNotify(SHCNE_DRIVEREMOVED, SHCNF_PATH, root, NULL);

And finally we notify our driver that the device can be deleted:

    drvCtrl->Unmount(devId);
  }

MountManager::Unmount simply deletes the corresponding MountedDisk form the container that causes its destructor call:

MountedDisk::~MountedDisk()
{

We set the stop event for the thread of processing requests for this disk:

    stopEvent_.set();

And we terminate all IRP, which were not served and are in a queue at the moment:

    if(lastIrp_)
        CompleteLastIrp(STATUS_DEVICE_NOT_READY, 0);

    while(irpQueue_.pop(lastIrp_))
        CompleteLastIrp(STATUS_DEVICE_NOT_READY, 0);
}

The thread of request processing, which has been in the MountedDisk::RequestExchange in the awaiting state, reacts on stopEvent_ set and throws an exception:

    NTSTATUS status = KeWaitForMultipleObjects(sizeof(eventsArray)/sizeof(PVOID), eventsArray, WaitAny, 
        Executive, KernelMode, FALSE, NULL, 0);
    if(status != STATUS_SUCCESS)
    {
        throw std::exception("MountedDisk::RequestExchange - mount stop.");
    }

We will get the thrown exception in catch block of DispatchException function and return STATUS_UNSUCCESSFUL to the user mode:

NTSTATUS DispatchExchange(PVOID buffer, ULONG inputBufferLength, ULONG outputBufferLength)
{
    try
    {
...
        gMountManager->RequestExchange(request->deviceId, 
                                       request->lastType, 
                                       request->lastStatus, 
                                       request->lastSize, 
                                       request->data, 
                                       request->dataSize, 
                                       &response.type,
                                       &response.size, 
                                       &response.offset);
...
    }
    catch(const std::exception & ex)
    {
        KdPrint((__FUNCTION__" %s\n", ex.what()));
        return STATUS_UNSUCCESSFUL;
    }
}

Returned error state will be then processed by user mode code in DriverControl::RequestExchange function, and will also throw an exception in its turn:

void DriverControl::RequestExchange(int deviceId, 
                                    int lastType, 
                                    int lastStatus, 
                                    int lastSize, 
                                    char * data, 
                                    int dataSize,
                                    int *type, 
                                    int *size, 
                                    __int64 * offset)
{
...
    if(!m_coreControl.DeviceIoGet(CORE_MNT_EXCHANGE_IOCTL, 
                                  &request, 
                                  sizeof(request), 
                                  &response, 
                                  sizeof(response), 
                                  &bytesWritten))
    {
        throw std::exception(__FUNCTION__" DeviceIoGet failed.");
    }
...
}

This exception in its turn will be caught by catch block in SyncMountmanager::mountDispatchThread:

void SyncMountManager::mountDispatchThread(void* pContext)
{
...
    try
    {
        DispatchImage(dispContext->devId, 
                      image->GetMountPoint(),
                      dispContext->image, 
                      dispContext->mountManager->GetDriverControl());
    }
    catch(const std::exception& ex)
    {
        dispContext->mountManager->OnUnmount(dispContext->image, ex.what());
    }
...
}

It will cause the request processing thread termination and also the IImage destructor call.

How to build this Solution

Install Windows Driver Developer Kit 2003. (http://www.microsoft.com/whdc/devtools/ddk/default.mspx)
Set global environment variable "BASEDIR" to path of installed DDK.

Computer -> Properties -> Advanced -> Environment variables ->System Variables -> New
Like this: BASEDIR -> c:\winddk\3790

Download and install boost (tested with 1.40 version) (http://www.boost.org/users/download/ )
Set global environment variable "BOOST" to path of installed boost. (You have to restart your computer after this.)
Use Visual Studio 2008 to build the solution.

Testing

Build the solution using the instructions above.
Copy CoreMnt.sys to %windir%\system32\drivers.

virtual-disk-tutorial/pic6.png

Register the driver in the system by the command:

sc create CoreMnt type=  kernel binPath= system32\drivers\CoreMnt.sys

virtual-disk-tutorial/pic7.png

Start driver by the command:

sc start CoreMnt

Start CoreMntTest.exe.

If everything was ok then CoreMntTest.exe will display the message:

virtual-disk-tutorial/pic8.png

  Image was mounted. Press  any key for unmount.

Disk Z will appear in the system.

virtual-disk-tutorial/pic9.png

Now we can format it.

virtual-disk-tutorial/pic10.png

A file «tst_img» will appear in the directory along with CoreMntTest.exe.

Supported Windows Versions

Windows XP SP2

Conclusion

In this article we considered only one from the possible approaches to the virtual disk creation. Described approach can be called logical virtual disk. The following approaches can be also mentioned:

Virtual disk of file system. Driver receives requests CreateFile, ReadFile, WriteFile and so on.
Physical virtual disk. This approach is semantically similar to the logical one but it has some differences at driver level. The requests to the virtual disk have other format, disk device should be PNP.

This article on the Apriorit website

How to develop a virtual disk for Windows