Screenshot - Client1.jpg

Introduction

This article is a continuation of my article on IOCP. In that article I had demonstrated the use of I/O Completion ports to create scalable client/server applications. In this article I will demonstrate how to use the select() function to create scalable client/server applications. In this implementation client and server send and display simple string messages.

The select() function works with synchronous sockets and doesn't require threads to be created.

The client I have provided here clientselect.exe is the same as clientiocp.exe in my earlier article. I have just renamed it. The code for this client can be found in my earlier article.

Using the code

The select() function will allow a developer to allocate the sockets in three different sets and it will monitor the sockets for state changes. We can process the socket based on its status. The three sets that are created for sockets are:

Read Set – Check the sockets belonging to this group for readability. A socket will be considered readable when:

A connection is pending on the listening socket
Data is received on the socket
Connection is closed or terminated

Write Set – Check the sockets belonging to this group for writability. A socket will be considered writable when data can be sent on the socket
Exception Set – Check the sockets belonging to this group for errors.

The sets are implemented using an fd_set structure. The definition of fd_set can be found in winsock2.h.

typedef struct fd_set {
        u_int fd_count;               /* how many are SET? */
        SOCKET  fd_array[FD_SETSIZE];   /* an array of SOCKETs */
} fd_set;

The following are macros that can be used to manipulate the sets. The source code of these macros can be found in winsock2.h.

FD_CLR – Removes a socket from the set
FD_ISSET – Helps in identifying if a socket belongs to a specified set
FD_SET – Assigns a socket to a specified set
FD_ZERO – Resets the set

For this implementation the client information will be stored in a singly linked list of CClientContext structure.

class CClientContext  //To store and manage client related information

{
private:
     int               m_nTotalBytes;
     int               m_nSentBytes;
     SOCKET            m_Socket;  //accepted socket

     char              m_szBuffer[MAX_BUFFER_LEN];
     CClientContext   *m_pNext; //this will be a singly linked list


public:
     //Get/Set calls

     void SetTotalBytes(int n)
     {
          m_nTotalBytes = n;
     }

     int GetTotalBytes()
     {
          return m_nTotalBytes;
     }

     void SetSentBytes(int n)
     {
          m_nSentBytes = n;
     }

     void IncrSentBytes(int n)
     {
          m_nSentBytes += n;
     }

     int GetSentBytes()
     {
          return m_nSentBytes;
     }

     void SetSocket(SOCKET s)
     {
          m_Socket = s;
     }

     SOCKET GetSocket()
     {
          return m_Socket;
     }

     void SetBuffer(char *szBuffer)
     {
          strcpy(m_szBuffer, szBuffer);
     }

     void GetBuffer(char *szBuffer)
     {
          strcpy(szBuffer, m_szBuffer);
     }

     char* GetBuffer()
     {
          return m_szBuffer;
     }

     void ZeroBuffer()
     {
          ZeroMemory(m_szBuffer, MAX_BUFFER_LEN);
     }

     CClientContext* GetNext()
     {
          return m_pNext;
     }

     void SetNext(CClientContext *pNext)
     {
          m_pNext = pNext;
     }

     //Constructor

     CClientContext()
     {
          m_Socket =  SOCKET_ERROR;
          ZeroMemory(m_szBuffer, MAX_BUFFER_LEN);
          m_nTotalBytes = 0;
          m_nSentBytes = 0;
          m_pNext = NULL;
     }

     //destructor

     ~CClientContext()
     {
          closesocket(m_Socket);
     }
};

Following is InitSets() function. This function will initialize the sets. Assign the sockets to appropriate sets. This function will be called before calling the select() function.

//Initialize the Sets

void InitSets(SOCKET ListenSocket) 
{
     //Initialize

     FD_ZERO(&g_ReadSet);
     FD_ZERO(&g_WriteSet);
     FD_ZERO(&g_ExceptSet);

     //Assign the ListenSocket to Sets

     FD_SET(ListenSocket, &g_ReadSet);
     FD_SET(ListenSocket, &g_ExceptSet);

     //Iterate the client context list and assign the sockets to Sets

     CClientContext   *pClientContext  = GetClientContextHead();
     while(pClientContext)
     {
          if(pClientContext->GetSentBytes() < pClientContext->GetTotalBytes())
          {
               //We have data to send

               FD_SET(pClientContext->GetSocket(), &g_WriteSet);
          }
          else
          {
               //We can read on this socket

               FD_SET(pClientContext->GetSocket(), &g_ReadSet);
          }

          //Add it to Exception Set

          FD_SET(pClientContext->GetSocket(), &g_ExceptSet); 

          //Move to next node on the list

          pClientContext = pClientContext->GetNext();
     }
}

Following is AcceptConnections() function. This function will monitor the sockets using select(). It will also process the sockets according to their status.

//This function will loop on while it will manage multiple clients 

//using select()

void AcceptConnections(SOCKET ListenSocket)
{
     while (true)
     {
          InitSets(ListenSocket);
          if (select(0, &g_ReadSet, &g_WriteSet, &g_ExceptSet, 0) > 0) 
          {
               //One of the socket changed state, let's process it.


               //ListenSocket?  Accept the new connection

               if (FD_ISSET(ListenSocket, &g_ReadSet)) 
               {
                    sockaddr_in ClientAddress;
                    int nClientLength = sizeof(ClientAddress);

                    //Accept remote connection attempt from the client

                    SOCKET Socket = accept(ListenSocket, 
                    (sockaddr*)&ClientAddress, &nClientLength);
                    if (INVALID_SOCKET == Socket)
                    {
                         printf("\nError occurred while accepting socket: 
                        %ld.", GetSocketSpecificError(ListenSocket));
                    }

                    //Display Client's IP

                    printf("\nClient connected from: %s", inet_ntoa
                        (ClientAddress.sin_addr)); 

                    //Making it a non blocking socket

                    u_long nNoBlock = 1;
                    ioctlsocket(Socket, FIONBIO, &nNoBlock);
                    CClientContext   *pClientContext  = new CClientContext;
                    pClientContext->SetSocket(Socket);

                    //Add the client context to the list

                    AddClientContextToList(pClientContext);
               }

               //Error occurred for ListenSocket?

               if (FD_ISSET(ListenSocket, &g_ExceptSet)) 
               {
                    printf("\nError occurred while accepting socket: %ld.", 
                    GetSocketSpecificError(ListenSocket));
                    continue;
               }

               //Iterate the client context list to see if 

               //any of the socket there has changed its state

               CClientContext   *pClientContext  = GetClientContextHead();
               while (pClientContext)
               {
                    //Check in Read Set

                    if (FD_ISSET(pClientContext->GetSocket(), &g_ReadSet))
                    {
                         int nBytes = recv(pClientContext->GetSocket(), 
                           pClientContext->GetBuffer(), MAX_BUFFER_LEN, 0);
                         if ((0 == nBytes) || (SOCKET_ERROR == nBytes))
                         {
                              if (0 != nBytes) //Some error occurred, 

                              //client didn't close the connection

                              {
                                   printf("\nError occurred while 
                                receiving on the socket: %d.", 
                                GetSocketSpecificError
                                (pClientContext->GetSocket()));
                              }

                              //In either case remove the client from list

                              pClientContext = DeleteClientContext
                                (pClientContext);
                              continue;
                         }

                         //Set the buffer

                         pClientContext->SetTotalBytes(nBytes);
                         pClientContext->SetSentBytes(0);

                         printf("\nThe following message was received: %s", 
                            pClientContext->GetBuffer());
                    }

                    //Check in Write Set

                    if (FD_ISSET(pClientContext->GetSocket(), &g_WriteSet))
                    {
                         int nBytes = 0;
                         if (0 < (pClientContext->GetTotalBytes() - 
                            pClientContext->GetSentBytes()))
                         {
                              nBytes = send(pClientContext->GetSocket(), 
                            (pClientContext->GetBuffer() + 
                            pClientContext->GetSentBytes()), 
                            (pClientContext->GetTotalBytes() - 
                            pClientContext->GetSentBytes()), 0);
                              if (SOCKET_ERROR == nBytes)
                              {
                                   printf("\nError occurred while 
                                sending on the socket: %d.", 
                                GetSocketSpecificError
                                (pClientContext->GetSocket()));
                                   pClientContext = DeleteClientContext
                                    (pClientContext);
                                   continue;
                              }

                              if (nBytes == 
                                (pClientContext->GetTotalBytes() - 
                                pClientContext->GetSentBytes()))
                              {
                                   //We are done sending the data, 

                              //reset Buffer Size

                                   pClientContext->SetTotalBytes(0);
                                   pClientContext->SetSentBytes(0);
                              }
                              else
                              {
                                   pClientContext->IncrSentBytes(nBytes);
                              }
                         }
                    }

                    //Check in Exception Set

                    if (FD_ISSET(pClientContext->GetSocket(), &g_ExceptSet))
                    {
                         printf("\nError occurred on the socket: %d.", 
                           GetSocketSpecificError(pClientContext->GetSocket()));
                         pClientContext = DeleteClientContext(pClientContext);
                         continue;
                    }

                    //Move to next node on the list

                    pClientContext = pClientContext->GetNext();
               }//while

          }
          else //select

          {
               printf("\nError occurred while executing select(): %ld.", 
                            WSAGetLastError());
               return; //Get out of this function

          }
     }
}

I have created a function GetSocketSpecificError() to get the error on a specific socket. When working with select() we can't rely on WSAGetLastError() because multiple sockets may have errors and we require socket specific errors.

//When using select() multiple sockets may have errors

//This function will give us the socket specific error

//WSAGetLastError() can't be relied upon

int GetSocketSpecificError(SOCKET Socket)
{
     int nOptionValue;
     int nOptionValueLength = sizeof(nOptionValue);

     //Get error code specific to this socket

     getsockopt(Socket, SOL_SOCKET, SO_ERROR, (char*)&nOptionValue, 
                                            &nOptionValueLength);

     return nOptionValue;
}

The following are linked list manipulation functions. These will be used to work with CClientContext singly linked list.

//Get the head node pointer

CClientContext* GetClientContextHead()
{
     return g_pClientContextHead;
}

//Add a new client context to the list

void AddClientContextToList(CClientContext *pClientContext)
{
     //Add the new client context right at the head

     pClientContext->SetNext(g_pClientContextHead);
     g_pClientContextHead = pClientContext;
}

//This function will delete the node and will return the next node of the list

CClientContext * DeleteClientContext(CClientContext *pClientContext)
{
     //See if we have to delete the head node

     if (pClientContext == g_pClientContextHead) 
     {
          CClientContext *pTemp = g_pClientContextHead;
          g_pClientContextHead = g_pClientContextHead->GetNext();
          delete pTemp;
          return g_pClientContextHead;
     }

     //Iterate the list and delete the appropriate node

     CClientContext *pPrev = g_pClientContextHead;
     CClientContext *pCurr = g_pClientContextHead->GetNext();

     while (pCurr)
     {
          if (pCurr == pClientContext)
          {
               CClientContext *pTemp = pCurr->GetNext();
               pPrev->SetNext(pTemp);
               delete pCurr;
               return pTemp;
          }

          pPrev = pCurr;
          pCurr = pCurr->GetNext();
     }

     return NULL;
}

Screen Shots

Screenshot - Server1.jpg

Screenshot - Client1.jpg

Screenshot - Server2.jpg

Screenshot - Client2.jpg

History

August 7, 2007: Initial Release

A scalable client/server using select() socket function