A scalable client/server using select() socket function
4.23/5 (19 votes)
An article on using select() function to create scalable client/server applications
Introduction
This article is a continuation of my article on IOCP. In that article I had demonstrated the use of I/O Completion ports to create scalable client/server applications. In this article I will demonstrate how to use the select() function to create scalable client/server applications. In this implementation client and server send and display simple string messages.
The select() function works with synchronous sockets and doesn't require threads to be created.
The client I have provided here clientselect.exe is the same as clientiocp.exe in my earlier article. I have just renamed it. The code for this client can be found in my earlier article.
Using the code
The select() function will allow a developer to allocate the sockets in three different sets and it will monitor the sockets for state changes. We can process the socket based on its status. The three sets that are created for sockets are:
- Read Set – Check the sockets belonging to this group for readability. A socket will be considered readable when:
- A connection is pending on the listening socket
- Data is received on the socket
- Connection is closed or terminated
- Write Set – Check the sockets belonging to this group for writability. A socket will be considered writable when data can be sent on the socket
- Exception Set – Check the sockets belonging to this group for errors.
The sets are implemented using an fd_set structure. The definition of fd_set can be found in winsock2.h.
typedef struct fd_set {
u_int fd_count; /* how many are SET? */
SOCKET fd_array[FD_SETSIZE]; /* an array of SOCKETs */
} fd_set;
The following are macros that can be used to manipulate the sets. The source code of these macros can be found in winsock2.h.
FD_CLR– Removes a socket from the setFD_ISSET– Helps in identifying if a socket belongs to a specified setFD_SET– Assigns a socket to a specified setFD_ZERO– Resets the set
For this implementation the client information will be stored in a singly linked list of CClientContext structure.
class CClientContext //To store and manage client related information
{
private:
int m_nTotalBytes;
int m_nSentBytes;
SOCKET m_Socket; //accepted socket
char m_szBuffer[MAX_BUFFER_LEN];
CClientContext *m_pNext; //this will be a singly linked list
public:
//Get/Set calls
void SetTotalBytes(int n)
{
m_nTotalBytes = n;
}
int GetTotalBytes()
{
return m_nTotalBytes;
}
void SetSentBytes(int n)
{
m_nSentBytes = n;
}
void IncrSentBytes(int n)
{
m_nSentBytes += n;
}
int GetSentBytes()
{
return m_nSentBytes;
}
void SetSocket(SOCKET s)
{
m_Socket = s;
}
SOCKET GetSocket()
{
return m_Socket;
}
void SetBuffer(char *szBuffer)
{
strcpy(m_szBuffer, szBuffer);
}
void GetBuffer(char *szBuffer)
{
strcpy(szBuffer, m_szBuffer);
}
char* GetBuffer()
{
return m_szBuffer;
}
void ZeroBuffer()
{
ZeroMemory(m_szBuffer, MAX_BUFFER_LEN);
}
CClientContext* GetNext()
{
return m_pNext;
}
void SetNext(CClientContext *pNext)
{
m_pNext = pNext;
}
//Constructor
CClientContext()
{
m_Socket = SOCKET_ERROR;
ZeroMemory(m_szBuffer, MAX_BUFFER_LEN);
m_nTotalBytes = 0;
m_nSentBytes = 0;
m_pNext = NULL;
}
//destructor
~CClientContext()
{
closesocket(m_Socket);
}
};
Following is InitSets() function. This function will initialize the sets. Assign the sockets to appropriate sets. This function will be called before calling the select() function.
//Initialize the Sets
void InitSets(SOCKET ListenSocket)
{
//Initialize
FD_ZERO(&g_ReadSet);
FD_ZERO(&g_WriteSet);
FD_ZERO(&g_ExceptSet);
//Assign the ListenSocket to Sets
FD_SET(ListenSocket, &g_ReadSet);
FD_SET(ListenSocket, &g_ExceptSet);
//Iterate the client context list and assign the sockets to Sets
CClientContext *pClientContext = GetClientContextHead();
while(pClientContext)
{
if(pClientContext->GetSentBytes() < pClientContext->GetTotalBytes())
{
//We have data to send
FD_SET(pClientContext->GetSocket(), &g_WriteSet);
}
else
{
//We can read on this socket
FD_SET(pClientContext->GetSocket(), &g_ReadSet);
}
//Add it to Exception Set
FD_SET(pClientContext->GetSocket(), &g_ExceptSet);
//Move to next node on the list
pClientContext = pClientContext->GetNext();
}
}
Following is AcceptConnections() function. This function will monitor the sockets using select(). It will also process the sockets according to their status.
//This function will loop on while it will manage multiple clients
//using select()
void AcceptConnections(SOCKET ListenSocket)
{
while (true)
{
InitSets(ListenSocket);
if (select(0, &g_ReadSet, &g_WriteSet, &g_ExceptSet, 0) > 0)
{
//One of the socket changed state, let's process it.
//ListenSocket? Accept the new connection
if (FD_ISSET(ListenSocket, &g_ReadSet))
{
sockaddr_in ClientAddress;
int nClientLength = sizeof(ClientAddress);
//Accept remote connection attempt from the client
SOCKET Socket = accept(ListenSocket,
(sockaddr*)&ClientAddress, &nClientLength);
if (INVALID_SOCKET == Socket)
{
printf("\nError occurred while accepting socket:
%ld.", GetSocketSpecificError(ListenSocket));
}
//Display Client's IP
printf("\nClient connected from: %s", inet_ntoa
(ClientAddress.sin_addr));
//Making it a non blocking socket
u_long nNoBlock = 1;
ioctlsocket(Socket, FIONBIO, &nNoBlock);
CClientContext *pClientContext = new CClientContext;
pClientContext->SetSocket(Socket);
//Add the client context to the list
AddClientContextToList(pClientContext);
}
//Error occurred for ListenSocket?
if (FD_ISSET(ListenSocket, &g_ExceptSet))
{
printf("\nError occurred while accepting socket: %ld.",
GetSocketSpecificError(ListenSocket));
continue;
}
//Iterate the client context list to see if
//any of the socket there has changed its state
CClientContext *pClientContext = GetClientContextHead();
while (pClientContext)
{
//Check in Read Set
if (FD_ISSET(pClientContext->GetSocket(), &g_ReadSet))
{
int nBytes = recv(pClientContext->GetSocket(),
pClientContext->GetBuffer(), MAX_BUFFER_LEN, 0);
if ((0 == nBytes) || (SOCKET_ERROR == nBytes))
{
if (0 != nBytes) //Some error occurred,
//client didn't close the connection
{
printf("\nError occurred while
receiving on the socket: %d.",
GetSocketSpecificError
(pClientContext->GetSocket()));
}
//In either case remove the client from list
pClientContext = DeleteClientContext
(pClientContext);
continue;
}
//Set the buffer
pClientContext->SetTotalBytes(nBytes);
pClientContext->SetSentBytes(0);
printf("\nThe following message was received: %s",
pClientContext->GetBuffer());
}
//Check in Write Set
if (FD_ISSET(pClientContext->GetSocket(), &g_WriteSet))
{
int nBytes = 0;
if (0 < (pClientContext->GetTotalBytes() -
pClientContext->GetSentBytes()))
{
nBytes = send(pClientContext->GetSocket(),
(pClientContext->GetBuffer() +
pClientContext->GetSentBytes()),
(pClientContext->GetTotalBytes() -
pClientContext->GetSentBytes()), 0);
if (SOCKET_ERROR == nBytes)
{
printf("\nError occurred while
sending on the socket: %d.",
GetSocketSpecificError
(pClientContext->GetSocket()));
pClientContext = DeleteClientContext
(pClientContext);
continue;
}
if (nBytes ==
(pClientContext->GetTotalBytes() -
pClientContext->GetSentBytes()))
{
//We are done sending the data,
//reset Buffer Size
pClientContext->SetTotalBytes(0);
pClientContext->SetSentBytes(0);
}
else
{
pClientContext->IncrSentBytes(nBytes);
}
}
}
//Check in Exception Set
if (FD_ISSET(pClientContext->GetSocket(), &g_ExceptSet))
{
printf("\nError occurred on the socket: %d.",
GetSocketSpecificError(pClientContext->GetSocket()));
pClientContext = DeleteClientContext(pClientContext);
continue;
}
//Move to next node on the list
pClientContext = pClientContext->GetNext();
}//while
}
else //select
{
printf("\nError occurred while executing select(): %ld.",
WSAGetLastError());
return; //Get out of this function
}
}
}
I have created a function GetSocketSpecificError() to get the error on a specific socket. When working with select() we can't rely on WSAGetLastError() because multiple sockets may have errors and we require socket specific errors.
//When using select() multiple sockets may have errors
//This function will give us the socket specific error
//WSAGetLastError() can't be relied upon
int GetSocketSpecificError(SOCKET Socket)
{
int nOptionValue;
int nOptionValueLength = sizeof(nOptionValue);
//Get error code specific to this socket
getsockopt(Socket, SOL_SOCKET, SO_ERROR, (char*)&nOptionValue,
&nOptionValueLength);
return nOptionValue;
}
The following are linked list manipulation functions. These will be used to work with CClientContext singly linked list.
//Get the head node pointer
CClientContext* GetClientContextHead()
{
return g_pClientContextHead;
}
//Add a new client context to the list
void AddClientContextToList(CClientContext *pClientContext)
{
//Add the new client context right at the head
pClientContext->SetNext(g_pClientContextHead);
g_pClientContextHead = pClientContext;
}
//This function will delete the node and will return the next node of the list
CClientContext * DeleteClientContext(CClientContext *pClientContext)
{
//See if we have to delete the head node
if (pClientContext == g_pClientContextHead)
{
CClientContext *pTemp = g_pClientContextHead;
g_pClientContextHead = g_pClientContextHead->GetNext();
delete pTemp;
return g_pClientContextHead;
}
//Iterate the list and delete the appropriate node
CClientContext *pPrev = g_pClientContextHead;
CClientContext *pCurr = g_pClientContextHead->GetNext();
while (pCurr)
{
if (pCurr == pClientContext)
{
CClientContext *pTemp = pCurr->GetNext();
pPrev->SetNext(pTemp);
delete pCurr;
return pTemp;
}
pPrev = pCurr;
pCurr = pCurr->GetNext();
}
return NULL;
}
Screen Shots
History
- August 7, 2007: Initial Release