Using Memory Mapped Files and JNI to communicate between Java and C++ programs

• Download source files - 68.8 Kb

Introduction

Sometimes your Java applications have to talk to each other, and sometimes your Java applications have to communicate with your C++ programs.

Consider the following scenario:

You have a Java program, let's call it JServer. When some interesting events happen, JServer will notify the clients. The clients can be other Java programs or C++ programs. This may look like this:

The JServer has a text area. After the user typed something, the JServer will tell the Java and C++ clients that the data is ready, please get it.

There are many ways to solve these kinds of problems. Here I would like to use Memory Mapped Files and JNI to do the job. Why Memory Mapped Files, because it is efficient. Why JNI, because I do not want to restrict my code to Microsoft Java Virtual Machine.

Warning: Whenever you use JNI, your code is not 100% pure Java anymore. But that's fine with me. :)

Bridge: Named Memory Mapped Files

You can use window messages like WM_COPYDATA, pipe, sockets (just name a few) to share data between different processes on the same machine. But the most efficient way is to use memory mapped files. Because all the mechanisms mentioned above are using the memory mapped files internally, to do the dirty work. The only "problem" with memory mapped files is that all the involving processes must use exactly the same name for the file mapping kernel object. But that is fine with me too. :)

Implementing Memory Mapped Files with Java

It is good that JDK 1.4 provides some memory mapping facilities, but that's not enough. Here I will introduce a very simple Java class MemMapFile. You can easily extend it to do much more complicated work. It has the following fields and native methods:

public static final int PAGE_READONLY = 0x02;
public static final int PAGE_READWRITE = 0x04;
public static final int PAGE_WRITECOPY = 0x08;

public static final int FILE_MAP_COPY = 0x0001;
public static final int FILE_MAP_WRITE = 0x0002;
public static final int FILE_MAP_READ = 0x0004;

public static native int createFileMapping(int lProtect, 
         int dwMaximumSizeHigh, int dwMaximumSizeLow, String name);
public static native int openFileMapping(int dwDesiredAccess, 
         boolean bInheritHandle, String name);
public static native int mapViewOfFile(int hFileMappingObj, 
         int dwDesiredAccess, int dwFileOffsetHigh, 
         int dwFileOffsetLow, int dwNumberOfBytesToMap);
public static native boolean unmapViewOfFile(int lpBaseAddress);
public static native void writeToMem(int lpBaseAddress, String content);
public static native String readFromMem(int lpBaseAddress);
public static native boolean closeHandle(int hObject);

public static native void broadcast();

These sound familiar to you. Yes, MemMapFile is nothing but a Java wrapper to the Win32 APIs about Memory Mapped Files. Also you will notice that I do not have the corresponding LPSECURITY_ATTRIBUTES parameter, that is because I want to make things simple, or I have to write another wrapper class. In the native side, I will just pass NULL (which is acceptable for most of the cases) for this parameter like the following:

JNIEXPORT jint JNICALL Java_com_stanley_memmap_MemMapFile_createFileMapping
    (JNIEnv * pEnv, jclass, jint lProtect, jint dwMaximumSizeHigh, 
    jint dwMaximumSizeLow, jstring name) {

    HANDLE hFile = INVALID_HANDLE_VALUE;
    HANDLE hMapFile = NULL;

    LPCSTR lpName = pEnv->GetStringUTFChars(name, NULL);
    __try {
        hMapFile = CreateFileMapping(hFile, NULL, lProtect, 
                dwMaximumSizeHigh, dwMaximumSizeLow, lpName);
        if(hMapFile == NULL) {
            ErrorHandler(_T("Can not create file mapping object"));
            __leave;
        }
        if(GetLastError() ==  ERROR_ALREADY_EXISTS)
            { ErrorHandler(_T("File mapping object already
            exists"));
            CloseHandle(hMapFile);
            __leave;
        }
    } __finally{
    }
    pEnv->ReleaseStringUTFChars(name, lpName);
    // if hMapFile is NULL, just return NULL, or return the handle
    return reinterpret_cast<<code>jint>(hMapFile);
}

When you get the handle of the file mapping object, you need to cast it to jint type and cached in your Java side. When you need it later, you can cast it back to HANDLE. The following is the implementation for the mapViewOfFile, you will see that I am using the HANDLE hMapFile returned from createFileMapping.

JNIEXPORT jint JNICALL Java_com_stanley_memmap_MemMapFile_mapViewOfFile
    (JNIEnv *, jclass, jint hMapFile, jint dwDesiredAccess, 
    jint dwFileOffsetHigh, 
    jint dwFileOffsetLow, jint dwNumberOfBytesToMap) {

    PVOID pView = NULL;
    pView = MapViewOfFile(reinterpret_cast<<code>HANDLE>(hMapFile), 
                  dwDesiredAccess, 
                  dwFileOffsetHigh, dwFileOffsetLow, dwNumberOfBytesToMap);
    if(pView == NULL) ErrorHandler(_T("Can not map view of file"));
    return reinterpret_cast<jint>(pView);
}

The pView pointer is a flat pointer, you can do whatever you want at that address. My writeToMem() and readFromMem() will simply write some string into that memory and read the string from the memory.

When something interesting happens, you will notice your clients. You have many ways to do that. I am lazy, I just put a broadcast method into the MemMapFile, it will broadcast a message telling that the data is ready.

JNIEXPORT void JNICALL Java_com_stanley_memmap_MemMapFile_broadcast
    (JNIEnv *, jclass) {
    SendMessage(HWND_BROADCAST, UWM_DATA_READY, 0, 0);
}

UWM_DATA_READY is a user defined message. User defined message is frequently used to cooperate different processes. If you are not familiar with it, you can go to Dr. Newcomer's homepage, he has an excellent article about Windows message management.

Before you can use user defined message, you have to register it first. Your native implementation DLL entry point function is a good place to register your own message. So you will have something like:

#define UWM_DATA_READY_MSG _T
       ("UWM_DATA_READY_MSG-{7FDB2CB4-5510-4d30-99A9-CD7752E0D680}")
UINT UWM_DATA_READY;
BOOL APIENTRY DllMain(HINSTANCE hinstDll, 
        DWORD dwReasion, LPVOID lpReserved) {
    if(dwReasion == DLL_PROCESS_ATTACH) 
        UWM_DATA_READY = RegisterWindowMessage(UWM_DATA_READY_MSG);
    return TRUE; 
}

All right, with MemMapFile, now we can share memory among processes, provided these processes know the File-Mapping kernel object name.

Building JServer

It is trivial to make a C++ server that creates the shared memory. It is more interesting to make a Java server. Let's still call it JServer. In your JServer, you need someway similar to the following, to cache the raw pointers return from the native code.

private int mapFilePtr;
private int viewPtr;

Then you can initialize them like the following:

mapFilePtr = MemMapFile.createFileMapping(MemMapFile.PAGE_READWRITE, 0,
                                    dwMemFileSize, fileMappingObjName);
if(mapFilePtr != 0) {
  viewPtr = MemMapFile.mapViewOfFile(mapFilePtr, MemMapFile.FILE_MAP_READ |
                                    MemMapFile.FILE_MAP_WRITE, 0, 0, 0);
}

When you want to notify your clients, you can post your just registered message. In my example, after I type something in the text area, I will click the button "Write and Broadcast". The button behavior is the following:

public void actionPerformed(ActionEvent e) {
     if(viewPtr != 0) {
          MemMapFile.writeToMem(viewPtr, textArea.getText());
          MemMapFile.broadcast();
     }
}

The content in the text area will be written to the shared memory and the data ready message is posted.

When you want to close the server, do not forget calls to unmapViewOfFile() and CloseHandle() to release the resource and unmap the shared memory from your process's address space.

Building Java client

Building a Java client is not very straight forward. The problem is that how the client gets the message that the data ready. Still there are many ways to do it. In this example, I will use a hidden window and JNI callback to deal with the message.

Step 1: Define an interface MemMapFileObserver.

I will explain why I am using an interface in Step 2.

public interface MemMapFileObserver {
  public void onDataReady();
}

Step 2: Define a proxy MemMapProxy.

MemMapProxy will process the data ready message. Of course I am using JNI again.

public class MemMapProxy {
  static {
    System.loadLibrary("MemMapProxyLib");
  }
  private MemMapFileObserver observer;
  public MemMapProxy(MemMapFileObserver observer) {
    this.observer = observer;
    init();
  }
  public void fireDataReadyEvent() {
    observer.onDataReady();
  }
  private native boolean init();
  public native void destroy();
}

Now you must know why I am using an Observer interface. I just want to make the code more generic. Any client that is interested in the data ready message can just implement this interface. In my proxy class, I just have one observer. If you like, you can use an EventListenerList.

The MemMapProxy class looks very simple, while the tricky part is hidden in the init() native method.

JNIEXPORT jboolean JNICALL Java_com_stanley_memmap_MemMapProxy_init
    (JNIEnv * pEnv, jobject jobj) {
    HANDLE hThread;
    hThread = (HANDLE)_beginthreadex(NULL, 0, 
          &CreateWndThread, NULL, 0, &uThreadId);
    if(!hThread) {
        MessageBox(NULL, _T("Fail creating thread"), NULL, MB_OK);
        return false;
    }
    g_jobj = pEnv->NewGlobalRef(jobj);
    return true;
}

unsigned WINAPI CreateWndThread(LPVOID pThreadParam) {
    HANDLE hWnd = CreateWindow(_T("dummy window"), 
                      NULL, WS_OVERLAPPEDWINDOW, 
                      CW_USEDEFAULT, CW_USEDEFAULT, CW_USEDEFAULT, 
                      CW_USEDEFAULT, NULL, NULL, hInstance, NULL);
    if(hWnd == NULL) {
        MessageBox(NULL, _T("Failed create dummy window"), 
                                  NULL, MB_OK|MB_ICONERROR);
        return 0;
    }
    jint nSize = 1;
    jint nVms;
    jint nStatus = JNI_GetCreatedJavaVMs(&g_pJvm, nSize, &nVms);
    if(nStatus == 0) {
        nStatus = g_pJvm->AttachCurrentThread
                (reinterpret_cast<void**>(&g_pEnv), NULL);
        if(nStatus != 0) ErrorHandler(_T("Can not attach thread"));
    }
    else {
        ErrorHandler(_T("Can not get the jvm"));
    }
    MSG Msg;
    while(GetMessage(&Msg, 0, 0, 0)) {
        TranslateMessage(&Msg);
        DispatchMessage(&Msg);
    }
    return Msg.wParam;
} 

// cache the JNIEnv* pointer
JNIEnv* g_pEnv = NULL;
// cache the JavaVM* pointer
JavaVM* g_pJvm = NULL;
// cache the jobject(MemMapProxy) pointer
jobject g_jobj = NULL;

As you have seen the init() will start a "daemon" thread. This thread will create a hidden window and initialize some global variables, then it will enter the message loop. The daemon thread will be alive until it gets the WM_QUIT message. Why am I doing this way? The reason is that I need something that can be always running to monitor the window's message after the init() returns.

Before creating the hidden window, I need to do two things. First I have to register a window class. In my example this class is named "dummy window". Second, I have to register the same UWM_DATA_READY message as the one in the MemMapFile. I register my window class and message in my DLL entry point function just like what I did before.

The window function of my hidden window is simple:

LRESULT CALLBACK WndProc(HWND hWnd, UINT Msg, 
                   WPARAM wParam, LPARAM lParam) {
    if(Msg == UWM_DATA_READY) {
        Callback();
    }
    else return DefWindowProc(hWnd, Msg, wParam, lParam);
    return 0;
}


void Callback() {
    if(g_pEnv == NULL || g_jobj == NULL) return;

    jclass cls = g_pEnv->GetObjectClass(g_jobj);
    jmethodID mid = g_pEnv->GetMethodID(cls, "fireDataReadyEvent", "()V");
    g_pEnv->CallVoidMethod(g_jobj, mid, NULL);
}

When it gets UWM_DATA_READY message, it will call my Callback(). Callback() is a callback function, it will call fireDataReadyEvent() in my Java side. fireDataReadyEvent() will in turn call the observer's method onDataReady().

JNI callback is very similar to the IDispatch in COM. You have to get the method ID first, based on the method name, then you can invoke the method. However, things will get more interesting if you have multithreads involved. First, JNIEnv* pointer is only valid in the current thread. Second you can not pass local reference to another thread. Our callback happens in our daemon thread, so we have to find a way to get the JNIEnv* pointer and get the reference to our proxy object.

You must have noticed that I have several global variables in my DLL, g_jobj, g_pEnv and g_pJvm. In my init(), I initialized g_jobj by creating a new global reference to the proxy object. Then I can pass it to my daemon thread. In my CreateWndThread(), I can get a pointer to the JVM by JNI_GetCreatedJavaVMs(), then by attaching my daemon thread to the JVM, I can get my JNIEnv* pointer. All right, so far so good. :)

Step 3: Creating a Java client

It is simple to create a Java client. What I need to do is to implement MemMapFileObserver interface.

  public void onDataReady() {
    int mapFilePtr = MemMapFile.openFileMapping(MemMapFile.FILE_MAP_READ, 
                       false, fileMappingObjName);
    if(mapFilePtr != 0) {
      int viewPtr = MemMapFile.mapViewOfFile(mapFilePtr,
               MemMapFile.FILE_MAP_READ, 0, 0, 0);
      if(viewPtr != 0) {
        String content = MemMapFile.readFromMem(viewPtr);
        textArea.setText(content);
        MemMapFile.unmapViewOfFile(viewPtr);
      }
      MemMapFile.closeHandle(mapFilePtr);
    }
  }

Building C++ client

Building a C++ is simple. In my example, I am using a MFC Dialog. It has a CEdit control. It will register the UWM_DATA_READY message too. When I get this message, I will read the shared memory and copy the content to my edit control. The following is my onDataReady():

LRESULT CMemMapCppClientDlg::OnDataReady(WPARAM, LPARAM) {
    HANDLE hMapFile = NULL;
    PVOID pView = NULL;
    hMapFile = OpenFileMapping(FILE_MAP_READ, FALSE, m_pszMemMapFileName);
    if(hMapFile == NULL) {
        MessageBox("Can not open file mapping");
        return 0;
    }
    pView = MapViewOfFile(hMapFile, FILE_MAP_READ, 0, 0, 0);
    if(pView == NULL) {
        MessageBox("Can map view of file");
        CloseHandle(hMapFile);
        return 0;
    }
    LPSTR szContent = reinterpret_cast<<code>LPSTR>(pView);
    int nLen = strlen(szContent);
    CString strContent;
    while(nLen > 0) {
        strContent += *szContent++;
        --nLen;
    }
    strContent += '\0';
    strContent.Replace("\n", "\r\n");
    m_edit.SetWindowText(strContent);
    if(pView) UnmapViewOfFile(pView);
    if(hMapFile) CloseHandle(hMapFile);
    return 0;
}

One thing I want to explain is that you need to replace the new line character('\n') with a return character ('\r') followed by a new line character to make the content compatible.

Memory synchronization

There are no major synchronization issues with my simple example. However it will be your big concern if your case is more complicated, or you will be in trouble.

You can not use critical section only to protect your resource, because critical section can only synchronize the threads contained within the same process. However it is not that hard to use Mutex and Semaphore kernel objects to guard your data. Or you can use critical section and kernel objects together if you are more concerned about efficiency. It will be more interesting if you add more native functions to the MemMapFile, like the Wait functions and CreateMutex()...that wrapper the corresponding Win32API. This is a good excise for you. :)

Conclusion

In this article, I am using a simple example to illustrate how to communicate between Java and Java, Java and C++ programs by using shared memory and JNI. I did not use Microsoft Visual J++ and com.ms.xxx.xxxx stuff because I want to use Sun JVM. You can easily extend my example and apply to more complicated cases.