A heap or shared memory based mutex pool manager
An article on a mutex pool manager based on heap or shared memory.
Introduction
This article develops a mutex pool manager. The manager is used to manage a mutex pool with a huge mutex number. The mutex pool is allocated from the heap or shared memory based on the manager name parameter specified. An unnamed manager allocates the pool from the heap. Mutexes with an unnamed manager can synchronize threads within a single process. On the other hand, a named manager allocates the pool from the shared memory. Mutexes with a named manager can synchronize threads between processes. The manager can grow the mutex pool automatically based on runtime demand.
The mutex in the pool is implemented using the Windows API InterlockedExchange; the Windows event object is used to synchronize the threads only if there is contention.
If you have a page based memory buffer, the page number is huge, and each page needs a mutex; or if you have an in-memory hash table, the hash bucket number is huge, and each bucket needs a mutex, and the mutex pool manager may serve you very well.
How to Use the Mutex Pool Manager
The first step is to create a mutex pool manager object. The constructor of the mutex manager class is as follows:
CMutexManager::CMutexManager(DWORD dwInitCount, DWORD dwIncCount, const wchar_t *pszName)
dwInitCount specifies the initial mutex count created by the manager. dwIncCount specifies the memory allocation granularity for extending the mutex pool. As the pool grows, memory is allocated in units of dwIncCount mutexes. pszName specifies the manager name, it is limited to 128 characters. Name comparison is case sensitive; if the name matches the name of an existing named mutex manager, dwInitCount and dwIncCount are ignored because they have already been set by the process that originally created the manager. If pszName is NULL, the mutex manager is created without a name. The mutexes with a named manager can synchronize threads between processes, the mutexes with an unnamed manager can synchronize threads within a single process.
The mutex manager exposes three public methods for the applications to call, they are:
bool CMutexManager::CreateMutex(DWORD& dwIndex)
This method creates a mutex from the mutex pool, sets the mutex reference count to one, and returns the index of the mutex being created. A thread must call CloseMutex for each time that it creates the mutex.
bool CMutexManager::CreateMutex(DWORD& dwIndex) { //Check if the manager is initialized if (m_bInitialized == false) { return false; } //Lock the manager CAutoLock<CSysMutex> lockAuto(m_pSysMutex); //Get the shared manager header MutexSharedManagerHeader *pSharedHeader = m_sManagerHeader.pSharedHeader; //Check if there is free mutex if (pSharedHeader->dwMutexNextFree == INVALID_MUTEX_ID) { //Check if we can add more mutex if (pSharedHeader->dwMutexCount == MAX_MUTEX_COUNT) { return false; } //Determine how many new mutex to add DWORD dwCount = MAX_MUTEX_COUNT - pSharedHeader->dwMutexCount; dwCount = (dwCount >= pSharedHeader->dwIncCount) ? pSharedHeader->dwIncCount : dwCount; //Create one more mutex block if (!CreateMutexBlock(dwCount, m_strName.c_str())) { return false; } } //Get the free mutex ID dwIndex = pSharedHeader->dwMutexNextFree; //Get the mutex address Mutex *pMutex = NULL; while (!GetMutex(dwIndex, &pMutex)) { //Create more mutex block if (!CreateMutexBlock(pSharedHeader->dwIncCount, m_strName.c_str())) { return false; } } //Set the next free mutex index pSharedHeader->dwMutexNextFree = pMutex->dwMutexNextFree; //Decrease the free mutex count --pSharedHeader->dwMutexFree; //Increase the inuse mutex count ++pSharedHeader->dwMutexInUse; //Update the new max inuse mutex count if (pSharedHeader->dwMutexInUse > pSharedHeader->dwMutexInUseMax) { pSharedHeader->dwMutexInUseMax = pSharedHeader->dwMutexInUse; } //Set 0 to the mutex created memset(pMutex, 0, sizeof(pMutex)); //Assign the index pMutex->dwIndex = dwIndex; //Increase the mutex reference count pMutex->dwMutexRefCount++; return true; }
bool CMutexManager::OpenMutex(DWORD dwIndex)
This method opens a mutex that has been created before, and increases the mutex reference count by one. A thread must call CloseMutex once for each time that it opens the mutex.
bool CMutexManager::OpenMutex(DWORD dwIndex) { //Check if the manager is initialized if (m_bInitialized == false) { return false; } //Lock the manager CAutoLock<CSysMutex> lockAuto(m_pSysMutex); //Get the shared manager header MutexSharedManagerHeader *pSharedHeader = m_sManagerHeader.pSharedHeader; //Check if the mutex index is within the range if (dwIndex >= pSharedHeader->dwMutexCount) { return false; } Mutex *pMutex = NULL; //Get the mutex address while (!GetMutex(dwIndex, &pMutex)) { //Create more mutex block if (!CreateMutexBlock(pSharedHeader->dwIncCount, m_strName.c_str())) { return false; } } //Check if the mutex has been created if (pMutex->dwMutexRefCount == 0) { return false; } //Increase the mutex reference count pMutex->dwMutexRefCount++; return true; }
bool CMutexManager::CloseMutex(DWORD dwIndex)
For a mutex being created or opened before, this method decreases the mutex reference count by one. If the reference count reaches 0, the mutex manager will mark the mutex free and release it back to the mutex pool. A thread must call CloseMutex once for each time that it creates or opens the mutex.
bool CMutexManager::CloseMutex(DWORD dwIndex) { //Check if the manager is initialized if (m_bInitialized == false) { return false; } //Lock the manager CAutoLock<CSysMutex> lockAuto(m_pSysMutex); //Get the shared manager header MutexSharedManagerHeader *pSharedHeader = m_sManagerHeader.pSharedHeader; //Check if the mutex index is within the range if (dwIndex >= pSharedHeader->dwMutexCount) { return false; } Mutex *pMutex = NULL; //Get the mutex address while (!GetMutex(dwIndex, &pMutex)) { //Create more mutex block if (!CreateMutexBlock(pSharedHeader->dwIncCount, m_strName.c_str())) { return false; } } //Decrease the mutex reference count pMutex->dwMutexRefCount--; //If the reference count is 0, put the mutex back to the free list if (pMutex->dwMutexRefCount == 0) { pMutex->dwMutexNextFree = pSharedHeader->dwMutexNextFree; pSharedHeader->dwMutexNextFree = pMutex->dwIndex; //Increase the free mutex count ++pSharedHeader->dwMutexFree; //Decrease the in use mutex count --pSharedHeader->dwMutexInUse; } return true; }
The helper class CMutex is provided to lock and unlock a mutex. The constructor of the class is as follows:
CMutex::CMutex(CMutexManager *pManager, DWORD dwIndex)
pManager specifies the mutex pool manager, dwIndex specifies the mutex index.
The class exposes two public methods for the applications to call, they are Lock() and Unlock().
bool CMutex::Lock()
This method waits for the ownership of the specified mutex object. The method returns when the calling thread is granted ownership. After a thread has ownership of a mutex object, it can make additional calls to Lock() without blocking its execution. This prevents a thread from deadlocking itself while waiting for a mutex that it already owns. A thread must call Unlock() once for each time that it calls Lock().
The implementation of Lock():
bool CMutex::Lock() { if ((m_pMutex == NULL) || (m_pManager == NULL)) { return false; } //Get the process and thread IDs DWORD dwProcessId = GetCurrentProcessId(); DWORD dwThreadId = GetCurrentThreadId(); //The event object CSysEvent *pEvent = NULL; while (true) { //Enter the critical section EnterCriticalSection(); //Check if the mutex is locked if (m_pMutex->bLocked == false) { //Set the lock flag m_pMutex->bLocked = true; //Increase the lock reference count m_pMutex->dwLockRefCount++; //Assign the process and thread IDs m_pMutex->dwProcessId = dwProcessId; m_pMutex->dwThreadId = dwThreadId; //Free the event delete pEvent; //Leave the critical section LeaveCriticalSection(); return true; } //Check if the same thread calls the lock again if ((m_pMutex->dwProcessId == dwProcessId) && (m_pMutex->dwThreadId == dwThreadId)) { //Increase the lock reference count m_pMutex->dwLockRefCount++; //Free the event delete pEvent; //Leave the critical section LeaveCriticalSection(); return true; } //Increase the waiter count m_pMutex->dwThreadsWaiting++; //Create the event if it's not created before, then wait on the event if (pEvent == NULL) { //Get the event name std::wstring strName; if (!GetEventName(strName)) { //Leave the critical section LeaveCriticalSection(); return false; } //Create the event pEvent = new (std::nothrow) CSysEvent(false, false, strName.c_str(), NULL); if (pEvent == NULL) { //Leave the critical section LeaveCriticalSection(); return false; } } //Leave the critical section LeaveCriticalSection(); //Wait on the event if (!pEvent->Lock(INFINITE)) { delete pEvent; return false; } } //Free the event delete pEvent; //Leave the critical section LeaveCriticalSection(); return true; }
bool CMutex::Unlock()
This method releases the ownership of the specified mutex object. A thread uses the Lock method to acquire ownership of a mutex object. To release its ownership, the thread must call Unlock() once for each time that it calls the Lock() method.
The implementation of Unlock():
bool CMutex::Unlock() { if ((m_pMutex == NULL) || (m_pManager == NULL)) { return false; } //Enter the critical section EnterCriticalSection(); //Get the process and thread IDs DWORD dwProcessId = GetCurrentProcessId(); DWORD dwThreadId = GetCurrentThreadId(); //Check if the thread is the owner if ((m_pMutex->dwProcessId != dwProcessId) || (m_pMutex->dwThreadId != dwThreadId)) { LeaveCriticalSection(); return false; } //Check if the mutex is locked if (m_pMutex->bLocked == false) { LeaveCriticalSection(); return false; } //Decrease the lock reference count m_pMutex->dwLockRefCount--; //Check if the lock reference count is not 0 if (m_pMutex->dwLockRefCount > 0) { LeaveCriticalSection(); return true; } //Release the lock m_pMutex->bLocked = false; //Reset the process and thread IDs m_pMutex->dwProcessId = 0; m_pMutex->dwThreadId = 0; //Check if there is a thread waitting on the lock if (m_pMutex->dwThreadsWaiting > 0) { //Get the event name std::wstring strName; if (!GetEventName(strName)) { LeaveCriticalSection(); return false; } //Create the event CSysEvent *pEvent = new (std::nothrow)CSysEvent(false, false, strName.c_str(), NULL); if (pEvent == NULL) { LeaveCriticalSection(); return false; } //Set the event state to signaled if (!pEvent->SetEvent ()) { LeaveCriticalSection(); delete pEvent; return false; } //Free the event delete pEvent; //Decrease the waiter count m_pMutex->dwThreadsWaiting--; } //Leave the critical section LeaveCriticalSection(); return true; }
The spin lock methods used by Lock() and Unlock() are:
bool CMutex::EnterCriticalSection()
bool CMutex::EnterCriticalSection() { if (m_pMutex == 0) { return false; } //Spin and get the spin lock. Each thread only owns the spin lock //for a very short of period while (InterlockedExchange(&(m_pMutex->lSpinLock), 1) != 0) { //Yield execution to another thread that is ready to run on //the current processor. SwitchToThread(); } return true; }
bool CMutex::LeaveCriticalSection()
bool CMutex::LeaveCriticalSection() { if (m_pMutex == NULL) { return false; } //Rlease the spin lock InterlockedExchange(&(m_pMutex->lSpinLock), 0); return true; }
Bibliography
- Dan Chou. "A Quick and Versatile Synchronization Object" (MSDN library, Technical articles).