Search This Blog

Tuesday 22 February 2011

Semaphore_in_UNIX_Notes


15.8. Semaphores

A semaphore isn't a form of IPC similar to the others that we've described (pipes, FIFOs, and message queues). A semaphore is a counter used to provide access to a shared data object for multiple processes.
The Single UNIX Specification includes an alternate set of semaphore interfaces in the semaphore option of its real-time extensions. We do not discuss these interfaces in this text.
To obtain a shared resource, a process needs to do the following:
1.      Test the semaphore that controls the resource.
2.      If the value of the semaphore is positive, the process can use the resource. In this case, the process decrements the semaphore value by 1, indicating that it has used one unit of the resource.
3.      Otherwise, if the value of the semaphore is 0, the process goes to sleep until the semaphore value is greater than 0. When the process wakes up, it returns to step 1.
When a process is done with a shared resource that is controlled by a semaphore, the semaphore value is incremented by 1. If any other processes are asleep, waiting for the semaphore, they are awakened.
To implement semaphores correctly, the test of a semaphore's value and the decrementing of this value must be an atomic operation. For this reason, semaphores are normally implemented inside the kernel.
A common form of semaphore is called a binary semaphore. It controls a single resource, and its value is initialized to 1. In general, however, a semaphore can be initialized to any positive value, with the value indicating how many units of the shared resource are available for sharing.
XSI semaphores are, unfortunately, more complicated than this. Three features contribute to this unnecessary complication.
1.      A semaphore is not simply a single non-negative value. Instead, we have to define a semaphore as a set of one or more semaphore values. When we create a semaphore, we specify the number of values in the set.
2.      The creation of a semaphore (semget) is independent of its initialization (semctl). This is a fatal flaw, since we cannot atomically create a new semaphore set and initialize all the values in the set.
3.      Since all forms of XSI IPC remain in existence even when no process is using them, we have to worry about a program that terminates without releasing the semaphores it has been allocated. The undo feature that we describe later is supposed to handle this.
The kernel maintains a semid_ds structure for each semaphore set:
   struct semid_ds {
     struct ipc_perm  sem_perm;  /* see Section 15.6.2 */
     unsigned short   sem_nsems; /* # of semaphores in set */
     time_t           sem_otime; /* last-semop() time */
     time_t           sem_ctime; /* last-change time */
     .
     .
     .
   };

The Single UNIX Specification defines the fields shown, but implementations can define additional members in the semid_ds structure.
Each semaphore is represented by an anonymous structure containing at least the following members:
   struct {
     unsigned short  semval;   /* semaphore value, always >= 0 */
     pid_t           sempid;   /* pid for last operation */
     unsigned short  semncnt;  /* # processes awaiting semval>curval */
     unsigned short  semzcnt;  /* # processes awaiting semval==0 */
     .
     .
     .
   };

Figure 15.28 lists the system limits (Section 15.6.3) that affect semaphore sets.
Figure 15.28. System limits that affect semaphores

Description
Typical values

FreeBSD 5.2.1
Linux 2.4.22
Mac OS X 10.3
Solaris 9

The maximum value of any semaphore
32,767
32,767
32,767
32,767

The maximum value of any semaphore's adjust-on-exit value
16,384
32,767
16,384
16,384

The maximum number of semaphore sets, systemwide
10
128
87,381
10

The maximum number of semaphores, systemwide
60
32,000
87,381
60

The maximum number of semaphores per semaphore set
60
250
87,381
25

The maximum number of undo structures, systemwide
30
32,000
87,381
30

The maximum number of undo entries per undo structures
10
32
10
10

The maximum number of operations per semop call
100
32
100
10


The first function to call is semget to obtain a semaphore ID.
#include <sys/sem.h>
 
int semget(key_t key, int nsems, int flag);

Returns: semaphore ID if OK, 1 on error

In Section 15.6.1, we described the rules for converting the key into an identifier and discussed whether a new set is created or an existing set is referenced. When a new set is created, the following members of the semid_ds structure are initialized.
·         The ipc_perm structure is initialized as described in Section 15.6.2. The mode member of this structure is set to the corresponding permission bits of flag. These permissions are specified with the values from Figure 15.24.
·         sem_otime is set to 0.
·         sem_ctime is set to the current time.
·         sem_nsems is set to nsems.
The number of semaphores in the set is nsems. If a new set is being created (typically in the server), we must specify nsems. If we are referencing an existing set (a client), we can specify nsems as 0.
The semctl function is the catchall for various semaphore operations.
#include <sys/sem.h>
 
int semctl(int semid, int semnum, int  cmd,
           ... /* union semun arg */);

Returns: (see following)

The fourth argument is optional, depending on the command requested, and if present, is of type semun, a union of various command-specific arguments:
   union semun {
     int              val;    /* for SETVAL */
     struct semid_ds *buf;    /* for IPC_STAT and IPC_SET */
     unsigned short  *array;  /* for GETALL and SETALL */
   };

Note that the optional argument is the actual union, not a pointer to the union.
The cmd argument specifies one of the following ten commands to be performed on the set specified by semid. The five commands that refer to one particular semaphore value use semnum to specify one member of the set. The value of semnum is between 0 and nsems-1, inclusive.
IPC_STAT
Fetch the semid_ds structure for this set, storing it in the structure pointed to by arg.buf.
IPC_SET
Set the sem_perm.uid, sem_perm.gid, and sem_perm.mode fields from the structure pointed to by arg.buf in the semid_ds structure associated with this set. This command can be executed only by a process whose effective user ID equals sem_perm.cuid or sem_perm.uid or by a process with superuser privileges.
IPC_RMID
Remove the semaphore set from the system. This removal is immediate. Any other process still using the semaphore will get an error of EIDRM on its next attempted operation on the semaphore. This command can be executed only by a process whose effective user ID equals sem_perm.cuid or sem_perm.uid or by a process with superuser privileges.
GETVAL
Return the value of semval for the member semnum.
SETVAL
Set the value of semval for the member semnum. The value is specified by arg.val.
GETPID
Return the value of sempid for the member semnum.
GETNCNT
Return the value of semncnt for the member semnum.
GETZCNT
Return the value of semzcnt for the member semnum.
GETALL
Fetch all the semaphore values in the set. These values are stored in the array pointed to by arg.array.
SETALL
Set all the semaphore values in the set to the values pointed to by arg.array.

For all the GET commands other than GETALL, the function returns the corresponding value. For the remaining commands, the return value is 0.
The function semop atomically performs an array of operations on a semaphore set.
[View full width]
#include <sys/sem.h>
 
int semop(int semid, struct sembuf semoparray[],
 size_t nops);

Returns: 0 if OK, 1 on error

The semoparray argument is a pointer to an array of semaphore operations, represented by sembuf structures:
 struct sembuf {
   unsigned short  sem_num;  /* member # in set (0, 1, ..., nsems-1) */
   short           sem_op;   /* operation (negative, 0, or positive) */
   short           sem_flg;  /* IPC_NOWAIT, SEM_UNDO */
 };

The nops argument specifies the number of operations (elements) in the array.
The operation on each member of the set is specified by the corresponding sem_op value. This value can be negative, 0, or positive. (In the following discussion, we refer to the "undo" flag for a semaphore. This flag corresponds to the SEM_UNDO bit in the corresponding sem_flg member.)
1.      The easiest case is when sem_op is positive. This case corresponds to the returning of resources by the process. The value of sem_op is added to the semaphore's value. If the undo flag is specified, sem_op is also subtracted from the semaphore's adjustment value for this process.
2.      If sem_op is negative, we want to obtain resources that the semaphore controls.
If the semaphore's value is greater than or equal to the absolute value of sem_op (the resources are available), the absolute value of sem_op is subtracted from the semaphore's value. This guarantees that the resulting value for the semaphore is greater than or equal to 0. If the undo flag is specified, the absolute value of sem_op is also added to the semaphore's adjustment value for this process.
If the semaphore's value is less than the absolute value of sem_op (the resources are not available), the following conditions apply.
a.       If IPC_NOWAIT is specified, semop returns with an error of EAGAIN.
b.      If IPC_NOWAIT is not specified, the semncnt value for this semaphore is incremented (since the caller is about to go to sleep), and the calling process is suspended until one of the following occurs.
                                                         i.            The semaphore's value becomes greater than or equal to the absolute value of sem_op (i.e., some other process has released some resources). The value of semncnt for this semaphore is decremented (since the calling process is done waiting), and the absolute value of sem_op is subtracted from the semaphore's value. If the undo flag is specified, the absolute value of sem_op is also added to the semaphore's adjustment value for this process.
                                                       ii.            The semaphore is removed from the system. In this case, the function returns an error of EIDRM.
                                                      iii.            A signal is caught by the process, and the signal handler returns. In this case, the value of semncnt for this semaphore is decremented (since the calling process is no longer waiting), and the function returns an error of EINTR.
3.      If sem_op is 0, this means that the calling process wants to wait until the semaphore's value becomes 0.
If the semaphore's value is currently 0, the function returns immediately.
If the semaphore's value is nonzero, the following conditions apply.
a.       If IPC_NOWAIT is specified, return is made with an error of EAGAIN.
b.      If IPC_NOWAIT is not specified, the semzcnt value for this semaphore is incremented (since the caller is about to go to sleep), and the calling process is suspended until one of the following occurs.
                                                         i.            The semaphore's value becomes 0. The value of semzcnt for this semaphore is decremented (since the calling process is done waiting).
                                                       ii.            The semaphore is removed from the system. In this case, the function returns an error of EIDRM.
                                                      iii.            A signal is caught by the process, and the signal handler returns. In this case, the value of semzcnt for this semaphore is decremented (since the calling process is no longer waiting), and the function returns an error of EINTR.
The semop function operates atomically; it does either all the operations in the array or none of them.

Semaphore Adjustment on exit

As we mentioned earlier, it is a problem if a process terminates while it has resources allocated through a semaphore. Whenever we specify the SEM_UNDO flag for a semaphore operation and we allocate resources (a sem_op value less than 0), the kernel remembers how many resources we allocated from that particular semaphore (the absolute value of sem_op). When the process terminates, either voluntarily or involuntarily, the kernel checks whether the process has any outstanding semaphore adjustments and, if so, applies the adjustment to the corresponding semaphore.
If we set the value of a semaphore using semctl, with either the SETVAL or SETALL commands, the adjustment value for that semaphore in all processes is set to 0.
ExampleTiming Comparison of Semaphores versus Record Locking
If we are sharing a single resource among multiple processes, we can use either a semaphore or record locking. It's interesting to compare the timing differences between the two techniques.
With a semaphore, we create a semaphore set consisting of a single member and initialize the semaphore's value to 1. To allocate the resource, we call semop with a sem_op of -1; to release the resource, we perform a sem_op of +1. We also specify SEM_UNDO with each operation, to handle the case of a process that terminates without releasing its resource.
With record locking, we create an empty file and use the first byte of the file (which need not exist) as the lock byte. To allocate the resource, we obtain a write lock on the byte; to release it, we unlock the byte. The properties of record locking guarantee that if a process terminates while holding a lock, then the lock is automatically released by the kernel.
Figure 15.29 shows the time required to perform these two locking techniques on Linux. In each case, the resource was allocated and then released 100,000 times. This was done simultaneously by three different processes. The times in Figure 15.29 are the totals in seconds for all three processes.
On Linux, there is about a 60 percent penalty in the elapsed time for record locking compared to semaphore locking.
Even though record locking is slower than semaphore locking, if we're locking a single resource (such as a shared memory segment) and don't need all the fancy features of XSI semaphores, record locking is preferred. The reasons are that it is much simpler to use, and the system takes care of any lingering locks when a process terminates.
Figure 15.29. Timing comparison of locking alternatives on Linux
Operation
User
System
Clock
semaphores with undo
0.38
0.48
0.86
advisory record locking
0.41
0.95
1.36

Semaphore in UNIX - An Overview

Insight about Semaphore: 

First let us begin our topic by giving an insight to what is actually a semaphore. A semaphore is nothing but a term used in UNIX for a variable which acts as a counter. So the next question that comes in mind is what for we need this variable. It’s so simple. The reason is explained below. For instance there may be times when two processes try to access the same file simultaneously. In this event we must control the access of the file when the other process is accessing. This is done by assigning value to semaphore.
The value of the semaphore is initialized by the first process when the file is in access by it. When the second process try to access the file it checks the value of the semaphore and if it finds the value as initialized it does not access the file. After the first process is completed it reinitializes the semaphore value and now the second process uses it. The above example is for two processes but a semaphore can be used even when numbers of processes try to access the same file. Thus semaphores are used to coordinate access to a resource by different processes.

Where a Semaphore gets stored

We have seen that semaphore can be used when numbers of processes try to access the same file. In this case we must make the semaphore available accessible to all processes so that they can read and check the value and also initialize and reinitialize the value of semaphore appropriately. For this reason only the semaphore is stored in kernel so that it can be accessed by all processes.

Details of Semaphore

The command
$ ipcs –s
will give the list of existing semaphores.

Function to create a simple semaphore

The function that can be used to create semaphores is semget( ). This function takes 3 arguments as input namely:
Name of the semaphore with which we are going to create the semaphore. This is technically called as key of the semaphore.
The number of sub semaphores to be created. The minimum required number is 1.
The semaphore mode is specified. There are 2 modes in semaphore namely read and rewrite.
The return integer value from this function indicates the semaphore id with which the semaphore is associated and if the return integer is -1 it indicates that an error has occurred in the creation of semaphore.

How Kernel maintains internally the semaphores:

For every semaphore created a structure is created inside by the kernel byname semid_ds and this structure is found in header file sys/ipc.h.

How to set user defined values for semaphore:

In the create semaphore function we saw that the semaphore created returns a semaphore id which is system generated. By we could realize in real usage scenarios it would prove helpful only when the user would be able to define the value of the semaphore. This is because if the user would be able to define the value of the semaphore then they could design easily stating the numbers they have given for semaphores .For instance number one may be used to mark that semaphore is free and say number two to mark the semaphore is in use by another process and so on. This can be done by using the following functions and values namely:
semctl( )
GETPID
SETVAL
along with semget( ) which we used before for creating semaphores.
Let us see in brief how to do this:
·The first step is to create semaphore as we discussed before by using semget( ) function which returns the semaphore id.
·Before seeing how to do this first let us see what the semctl( ) function takes as arguments. It takes 4 parameters as arguments namely:
·The semaphore id created by semget( ) function
·Sub semaphore id. For the first time it would be zero since it gets created that time.
·GETPID. This returns the Process id of the process.
·value to be placed for the above process id. This value only will be returned by the function semctl( ).
·After this by using SETVAL in semctl( ) function with all the arguments the same except instead of GETPID we place SETVAL and give the value to be placed in this which is returned by the semctl( ) function.
This is how we assign user defined values to semaphores which give ease of maintenance and use of semaphores by users.

Aspects of Semaphore:

·Semaphores are identified by semaphore id which is unique for each semaphore.
·The semaphores can be deleted by suing the function semdelete(semaphore)
·The semaphore value can be incremented or decremented by using functions wait (sem) and signal (sem) respectively. wait (sem) decrements the semaphore value and if in the process of decrementing the value of semaphore reaches negative value then the process is suspended and placed in queue for waiting. signal (sem) increments the value of the semaphore and it is opposite in action to wait (sem).In other words it causes the first process in queue to get executed.
The value of semaphore represents thus the number of threads which are nothing but processes. In other words we found that if the value is positive then we have threads to decrement and proceed for execution without suspending. If the value of semaphore is negative then it represents that the number of threads or process is blocked and kept in suspended state. If the value of semaphore is zero then it means that there are no threads or processes in waiting state.

So from the above the important fact in semaphores is when we create semaphores we found that semaphores can be assigned with any value. Also we found from above that after creation of semaphores we can modify that is either increment or decrement the value of semaphores. But at any situation we cannot read the current value of semaphore.

=======================================================

Semaphores in System V UNIX

The System V specification of the UNIX operating system introduces semaphores among the other facilities for Inter-Process Communication (IPC). This implementation is a (rather messy) generalization of the above described scheme. Three features contribute to this unnecessary complication:
  • A semaphore counter is not just a single value. It's rather a set of one or more values, and the exact number of values is specified upon semaphore creation.
  • The creation of a semaphore is independent of its initialization. This is a fatal flaw, since we cannot atomically create a new semaphore set and initialize all the values in the set: a competing process may sneak in and access the semaphore after creation but before initialization, thus finding semagarbage.
  • A semaphore must be explicitly destroyed when the shared resource is no longer necessary, since the systemwide total number of semaphore sets is limited.
Moreover, semaphore counters are always nonnegative, a zero causing the waiting process to be put to sleepgif
A semaphore is created by the semget(3) system call, whose synopsis is:
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/sem.h>
 
int semget(key_t key, int nsems, int flag);
This system call returns an integer id for the semaphore (analogous to a file descriptor), or -1 in case of error.
The key argument is analogous to a file name, and is used to identify exclusively a semaphore set (and other IPC objects as well) all over the system, while the semaphore id is - like a file descriptor - known only by one process and by its siblings that inherit it through variables after a fork(). Cooperating processes wishing to access the semaphore set must use the same key, much like a file name is used to store on disk shared data by cooperating processes. A key is implemented as an integer (whose size is specified by the key_t type), and a library function called ftok() is provided for the purpose of generating meaningful keys (see its manpage).
The value of nsems specifies the number of semaphores in newly created semaphore set.
The value of flag is a bit mask obtained ORing constants that specify:
  • The access permissions for the semaphore set, in the same way as for a filegif;
  • The creation mode, which can be:
    • IPC_PRIVATE: if this is specified, the value of the key is ignored, and a private semaphore set is created, that can be used by a process and its siblings through semaphore id inheritance. Since there's no key, other unrelated processes have no way to access the semaphore set.
    • IPC_CREAT: if this is specified, and a semaphore with the given key does not exist, it is created, otherwise the call returns with -1, setting the appropriate errno value.
For example, to create one semaphore with key 333, readable an writable by an user and his group, you'd use a code fragment like:
int sid;
...
sid=semget((key_t)333, 1, SEM_R|SEM_W|(SEM_R>>3)|(SEM_W>>3)|IPC_CREAT);
if (sid<0) 
{
    perror(``Semaphore 333'');
    exit(1);
}
...
Once a semaphore has been created, a process accesses it either directly via its id, if its a private semaphore and the id has been inherited from a creating ancestor, or by calling semget() to get its id in exchange for the right key, much like opening an already existing file. In this case the IPC_PRIVATE and IPC_CREAT bits are not set.
Semaphore initialization and flagging are performed respectively via the semctl() and semop() system calls. Refer to their manpages for details on their usagegif

Unix semaphore example

/* semabinit.c - initialize a semaphore for use by programs sema and semb */
 
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/sem.h>
#include <stdio.h>
 
/* The semaphore key is an arbitrary long integer which serves as an
   external identifier by which the semaphore is known to any program
   that wishes to use it. */
 
#define KEY (1492)
 
void main()
{
   int id; /* Number by which the semaphore is known within a program */
 
   /* The next thing is an argument to the semctl() function. Semctl() 
      does various things to the semaphore depending on which arguments
      are passed. We will use it to make sure that the value of the 
      semaphore is initially 0. */
 
      union semun {
            int val;
            struct semid_ds *buf;
            ushort * array;
      } argument;
 
   argument.val = 0;
 
   /* Create the semaphore with external key KEY if it doesn't already 
      exists. Give permissions to the world. */
 
   id = semget(KEY, 1, 0666 | IPC_CREAT);
 
   /* Always check system returns. */
 
   if(id < 0)
   {
      fprintf(stderr, "Unable to obtain semaphore.\n");
      exit(0);
   }
 
   /* What we actually get is an array of semaphores. The second 
      argument to semget() was the array dimension - in our case
      1. */
 
   /* Set the value of the number 0 semaphore in semaphore array
      # id to the value 0. */
 
   if( semctl(id, 0, SETVAL, argument) < 0)
   {
      fprintf( stderr, "Cannot set semaphore value.\n");
   }
   else
   {
      fprintf(stderr, "Semaphore %d initialized.\n", KEY);
   }
}
 
/* Semaphore example program a (sema.c) */
/* We have two programs, sema and semb. Semb may be initiated at any 
  time, but will be forced to wait until sema is executed. Sema and
  semb do not have to be executed by the same user! */
 
#include <stdio.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/sem.h>
 
#define KEY (1492)
/* This is the external name by which the semaphore is known to any
   program that wishes to access it. */
 
void main()
{
   int id;  /* Internal identifier of the semaphore. */
   struct sembuf operations[1];
   /* An "array" of one operation to perform on the semaphore. */
 
   int retval; /* Return value from semop() */
 
   /* Get the index for the semaphore with external name KEY. */
   id = semget(KEY, 1, 0666);
   if(id < 0)
   /* Semaphore does not exist. */
   {
      fprintf(stderr, "Program sema cannot find semaphore, exiting.\n");
      exit(0);
   }
 
   /* Do a semaphore V-operation. */
   printf("Program sema about to do a V-operation. \n");
 
   /* Set up the sembuf structure. */
   /* Which semaphore in the semaphore array : */
    operations[0].sem_num = 0;
    /* Which operation? Add 1 to semaphore value : */
    operations[0].sem_op = 1;
    /* Set the flag so we will wait : */   
    operations[0].sem_flg = 0;
 
    /* So do the operation! */
    retval = semop(id, operations, 1);
 
    if(retval == 0)
    {
       printf("Successful V-operation by program sema.\n");
    }
    else
    {
       printf("sema: V-operation did not succeed.\n");
      perror("REASON");
    }
}
 
/* Think carefully about what the V-operation does. If sema is executed 
   twice, then semb can execute twice. */
 
/* Semaphore example program b (semb.c) */
/* We have two programs, sema and semb. Semb may be initiated at any 
  time, but will be forced to wait until sema is executed. Sema and
  semb do not have to be executed by the same user! */
 
/* HOW TO TEST:
   Execute semb &
   The & is important - otherwise you would have have to move to
   a different terminal to execute sema.
 
   Then execute sema.
*/
 
#include <stdio.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/sem.h>
 
#define KEY (1492)
/* This is the external name by which the semaphore is known to any
   program that wishes to access it. */
 
void main()
{
   int id;  /* Internal identifier of the semaphore. */
   struct sembuf operations[1];
   /* An "array" of one operation to perform on the semaphore. */
 
   int retval; /* Return value from semop() */
 
   /* Get the index for the semaphore with external name KEY. */
   id = semget(KEY, 1, 0666);
   if(id < 0)
   /* Semaphore does not exist. */
   {
      fprintf(stderr, "Program semb cannot find semaphore, exiting.\n");
      exit(0);
   }
 
   /* Do a semaphore P-operation. */
   printf("Program semb about to do a P-operation. \n");
   printf("Process id is %d\n", getpid());
 
   /* Set up the sembuf structure. */
   /* Which semaphore in the semaphore array : */
    operations[0].sem_num = 0;
    /* Which operation? Subtract 1 from semaphore value : */
    operations[0].sem_op = -1;
    /* Set the flag so we will wait : */   
    operations[0].sem_flg = 0;
 
    /* So do the operation! */
    retval = semop(id, operations, 1);
 
    if(retval == 0)
    {
       printf("Successful P-operation by program semb.\n");
       printf("Process id is %d\n", getpid());
    }
    else
    {
       printf("semb: P-operation did not succeed.\n");
    }
}
 
/* Think carefully about what the V-operation does. If sema is executed 
   twice, then semb can execute twice. */

Synchronization: Semaphores

Goal

The goal of this tutorial is explain how semaphores can be used to solved synchronization problems, which arise through cooperation between processes. The tutorial will start with the basics on creating and setting-up semaphores, then tackle the most basic use of semaphores, to protect critical sections of code. Finally, the Bounded Buffer Problem, described in the tutorial on Cooperation, will be tackled.

The Critical Section Problem

I assume you have read the tutorial on cooperating processes, and Interprocess Communication (IPC) facilities provided by the Operating System for process cooperation. The most synchronization problem confronting cooperating processes, is controling access to shared resource. Suppose two processes share access to a file, or shared memory segment (or when we discuss threads in a couple of weeks, we'll see threads share the same memory, so they must synchronize their actions), and at least one of these processes can modify the data in this shared area of memory. That part of the code of each program, where one process is reading from or writing to a shared memory area, is a critical section of code, because we must ensure that only one process execute a critical section of code at a time. The Critical Section Problem is to design a protocol that the processes can use to coordinate their activities when one wants to enter its critical section of code.
Suppose we have two processes that share a memory segment of four bytes which stores an integer value. Let this value be named by the variable V. Process 1 (P1) and Process 2 (P2) have a section of code with the following lines:

       if (0 < V)
           V--;

These lines of code will be part of a critical section of code, because it is important that each process be allowed to execute all of them without interference from the other process. Here is an example of how the processes can interfere with each other:
  1. V has the value 1
  2. P1 tests 0 < V, which is true
  3. P1 is removed from the CPU and replaced by P2
  4. P2 tests 0 < V, which is true
  5. P2 executes V--, so V has the value 0
  6. P1 given back the CPU and starts where it last left off: executing V--, so V has the value -1
By interfering with each other during the execution of the lines of code, the two processes caused the value of the common variable V to drop below 0, which they were trying to prevent from happening.
The protocol developed for solving the Critical Section Problem involved three steps:
  1. Entry section: Before entering the critical section, a process must request permission to enter
  2. Critical section: After permission is granted, a process may execute the code in the critical section. Other processes respect the request, and keep out of their critical sections.
  3. Exit section: The process acknowledges it has left its critical section.
The problem that remains is how to effectively implement this protocol. How can the processes communicate their requests and grant their permissions so that only one process at a time is in a critical section.

The Solution

The Dutch scientist E.W. Dijkstra showed how to solve the Critical Section Problem in the mid-sixties, and introduced the concept of a semaphore to control synchronization. A semaphore is an integer variable which is accessed through through two special operations, called wait and signal. Why we need special operations will be discussed shortly. Originally, Dijkstra called the two special operations P (for proberen, the dutch word for test) and V (for verhogen, the dutch word to increment). Let S be an integer variable, our semaphore, then the classical definition of wait is given by

       wait(S) {
          while(S ≤ 0);
          S--;
       }

and signal is given by

       signal(S) {
          S--;
       }

We can use the semaphore to synchronize our processes:

       // Entry Section
       wait(S);

       // Critical Section
       if (0 < V)
           V--;

       // Exit Section
       signal(S);
  1. Process P calls wait(S)
  2. While S is 0, P waits.
  3. When S is 1, P is allowed to enter its critical section
  4. While P is in its critical section, the value of S is zero, blocking other processes from entering their critical section.
  5. When P is finished and ready to leave its critical section, it executes signal resetting S to 1 and allowing another process to enter.
A semaphore which can take the value zero or one is called a binary semaphore, or mutex, for mutually exclusive.
There is a glaring weakness with this implementation: the code for wait is no better than the code we were trying to protect: For the semaphore to work, S must be shared, so it is tested and changed in wait, and this requires synchronization between the processes; so how does this resolve the original problem?

Atomic Operations

The problem we have is that we have some lines of code which need to be executed without interference from other processes; this is no problem as long as a process has control of the CPU, but if it is taken-off the CPU by the Operating System before it can finish executing its section of code, another process may have an opportunity to trash its work. The code must be run atomically: in one uninterruptible unit. So, for semaphores to solve the problem, the implementation of the two functions wait and signal must be atomic: there can be no interruptions while these operations are being implemented.
Operating systems, like Unix, which have semaphores, guarantee that the operations wait and signal are run atomically. How can they do this? It is the Operating System that determines when to switch a process off the CPU; while the Operating System cannot decide when it must leave a process on the CPU, it can govern its own CPU use. So, the Operating System keeps track of the semaphore and its value: when processes call operations on the semaphore such as wait and signal they are actually making system calls which request the Operating System to check the semaphore and change its value. The Operating System then steps in and performs the operation, without interruption.

System V Semaphores

Unix System V introduced semaphores to Unix in the mid-eighties. Since the Operating System maintains semaphores and operates on them, the actual code required is rather complicated. To add to this confusion, a Unix semaphore is not really a single integer value, but an array of integer values. We'll see later why we may want to use more than one semaphore. I will refer to Unix semaphores as semaphore structures, since they include (possibly) multiple semaphores.
Semaphores are to be shared by processes which are unrelated to each other. This means that they need to have some means of coordinating their access to the same semaphore. The way this is done is for all processes which will use the semaphore to exchange a shared key value. This key value is defined the same way in each process. The value of the key can be any integer, although if there already exists a semaphore structure with that key, that semaphore will be used. One way to be sure that your key is unique is to see what semaphore structures the system already has set-up. Typing ipcs -sem on the command line will list all semaphore structures. Unix only allows a few semaphore structures at a time (a maximum of 10 in the whole system), so most any key you choose is likely to be unique.
There are five basic steps in allocating a semaphore structure:
  1. Request a semaphore structure, using the common key
  2. Initialize the semaphore structure by setting the values of each semaphore
  3. Define the basic operations we want to perform on the semaphore structure
  4. Use the basic operations in our program
  5. Remove the semaphore structure when we are done with it.
The first step is a request of a semaphore structure.
semget

Purpose
Request a semaphore structure
Include
#include<sys/types.h>
#include<sys/ipc.h>
#include<sys/sem.h>
Useage
int  semget(key_t key, int numsems, int flag)
Arguments
key:    key value
numsems:    number of semaphores
flag:    IPC_CREAT or IPC_EXCL
Returns
-1 on error
semaphore ID if successful
Errors
ENOMEM:  No memory available
EEXIST:  structure exists and IPC_EXCL set

There are three ways you may want to call semget

       //Create the structure if it doesn't exist
       //or use an existing structure with the same key
       semaid = semget(key, numsems, IPC_CREAT);

       //Require a new structure is created:
       //Throws an error if a structure exists with the key
       semaid = semget(key, numsems, IPC_CREAT | IPC_EXCL);

       //Create the structure if it doesn't exist using a private key
       //The return value semaid can only be shared with related processes
       semaid = semget(IPC_PRIVATE, numsems, IPC_CREAT);

The return value semaid will be used to refer to the semaphore structure in all further functions.
Once you have the semaid, you initialize the structure, by setting the value of every semaphore in the set. The function for doing this is semctl which is a catch-all for alot of different functions. You will want to see the manpages: man 2 semctl for all the details.
semctl

Purpose
Mainly to remove a semaphore structure from the system, and to set the values of each semaphore in the structure.
Include
#include<sys/types.h>
#include<sys/ipc.h>
#include<sys/sem.h>
Useage
int semctl(int semaid, int semnum, int cmd, union semun arg);
Arguments
semaid:    semaphore ID
semnum:    between 0 and numsems-1
cmd:    SETVAL or SETALL
arg:    see below
Returns
0, for the two commands above
Errors
NONE
Using this structure is a little complicated, so lets start with the definition of union semun. This union structure is not actually defined in any of the header files, so you will have to define the structure in your program before you call main:

union semun {
      int val; /* used for SETVAL only */
      struct semid_ds *buf; /* for IPC_STAT and IPC_SET, not discussed here */
      ushort *array; /* used for GETALL and SETALL */
};

To make the discussion concrete, I will consider how to set the value of the semaphore structure in two cases: where we want to set one particular semaphore, and when we want to set the value for all semaphores at once (the case of two semaphores in the structure). Here is the code to set one semaphore in a set

       union semun    arg;
       arg.val = 1;

       //This sets the first semaphore to the value 1
       //(numbering starts at 0)
       semctl(semaid, 0, SETVAL, arg);

If we want to set all the values of the semaphore structure, we need an array of integers, and to set each value in the array:

       union semun    arg;
       int    arr[2];
       arr[0] = 1;
       arr[1] = 5;
       arg.array = arr;

       //Second argument is not used
       semctl(semaid, 0, SETALL, arg);
We use a special structure type, struct sembuf, to define the basic operations we want to define on the semaphore set. Here is the structure:

struct sembuf {
      int sem_num; /* member # in {0, . . . , numsems - 1} */
      short sem_op; /* operation: negative, zero, positive */
      short sem_flag; /* IPC_NOWAIT or SEM_UNDO */
};

Each argument is important:
  • sem_num:   This refers to the particular semaphore we are going to adjust in the semaphore structure. The number of semaphores, numsems was determined when the semaphores structure was created, using semget. Counting starts at 0.
  • sem_op:   The operation we will perform on the semaphore.
    • positive: the value is added to the semaphore's current value
    • negative: the value is added to the semaphore's current value, provided the current value is at least zero. If the value would be less than zero, the process will wait for the value to become large enough so that the sum is at least zero. If the flag IPC_NOWAIT is set, process does not wait.
    • zero: the process waits for the value of the semaphore to reach zero, unless IPC_NOWAIT is specified.
  • sem_flag One of two possible flags:
    • IPC_NOWAIT: do not wait on the semaphore, return immediately if the operation cannot be carried-out. In this case, the error EAGAIN is returned.
    • SEM_UNDO: if the process should terminate before it can restore the count on the semaphore, then the count is restored by the Operating System. Suppose your process decrements a semaphore, taking its value to zero, but terminates before it can reset the semaphore back to one. Any process waiting for the semaphore to be one again would be stuck. The Operating System takes care of this if this flag is set.


Lets look at how we would set-up the semaphore operations wait and signal. Suppose we have created a semaphore structure having one semaphore, with semaphore ID semaid, and this semaphore has been set to an initial value one. The two operations which I will label WAIT and SIGNAL are defined as follows:

       struct sembuf    WAIT[1], SIGNAL[1];

       //Defining WAIT
       WAIT[0].sem_num = 0;
       WAIT[0].sem_op = -1;
       WAIT[0].sem_flag = SEM_UNDO;

       //Defining SIGNAL
       SIGNAL[0].sem_num = 0;
       SIGNAL[0].sem_op = 1;
       SIGNAL[0].sem_flag = SEM_UNDO;
Using our semaphore actions is actually easy, we call the operation semop
semop

Purpose
Implement a predefined action on a semaphore set
Include
#include<sys/types.h>
#include<sys/ipc.h>
#include<sys/sem.h>
Useage
int semop(int semaid, struct sembuf semoparray[], size_t nops);
Arguments
semaid:    semaphore ID
semopsarray:    operations to be performed on the semaphores
nops:    length of semopsarray
Returns
-1 on error
SUCCESS
Errors
EAGAIN:  Process would wait and IPC_NOWAIT set

Here is how our code protecting the critical section now looks

       // Entry Section
       semop(semaid, WAIT, 1);

       // Critical Section
       if (0 < V)
           V--;

       // Exit Section
       semop(semaid, SIGNAL, 1);
Removing a semaphore structure can only be done by the user ID of the creator of the semaphore structure. This can be done on the command line using ipcrm semaid, or you can explicitly remove the semaphore structure within the program using semctl

int semctl (semid, 0, IPC_RMID, 0)

You will find more details at FAQ.

 

The Bounded Buffer Problem

You will want to refresh your memory Bounded Buffer Problem. We now add an additional feature to our Critical Section Problem: the shared resource is stored in a buffer, BUFFER (a storage facility for data) of limited size. We have two types of processes, Producers who try to add to BUFFER and Consumers who try to remove from BUFFER. One possible solution to this problem is to treat use a single binary semaphore just as in the Critical Section Problem together with a variable COUNTER, which is shared in common (say as a shared memory segment.) Here is code for the Consumer:

       // Entry Section
       semop(semaid, WAIT, 1);

       // Critical Section
       if (0 < COUNT)
           COUNT--;
           //OK to take from BUFFER
           . . . BUFFER . . .

       // Exit Section
       semop(semaid, SIGNAL, 1);

and for the producer

       // Entry Section
       semop(semaid, WAIT, 1);

       // Critical Section
       if (COUNT < BUFFER_MAX )
           COUNT++;
           //OK to add to BUFFER
           . . . BUFFER . . .

       // Exit Section
       semop(semaid, SIGNAL, 1);
This presents one solution to the Bounded Buffer Problem, and in many circumstances it may be perfectly fine. But there is a problem which you should be aware of, that may make its performance unsuitable
  • A Consumer must constantly poll COUNTER to test if it is positive. Similarly, a Producer must constantly poll COUNTER to test if it is below BUFFER_MAX.
Suppose a Consumer must have the data in the BUFFER, but BUFFER is empty. The only way to determine whether data has been added to the buffer is to continually enter the Critical Section and check COUNTER. This creates what is called a spinlock, where the process enters a useless cycle waiting for a resource. If the resource will be available quickly, this is not a problem; but if the resource may take some time, this spinlock position wastes CPU time that could be more productively spent. A solution to this problem is to use a semaphore to count as well. A semaphore which is used as a counter is called a counting semaphore.
It is pretty clear how we can use a counting semaphore to restrict the Consumer's behavior: set a semaphore to the value of BUFFER_MAX, and have each Consumer decrement its value, each Producer increment its value. If the count goes to zero, Consumers are forced to wait. Unfortunately, Unix System V semaphores effectively have no upper limit, so we cannot use just one counter to control both Consumers and Producers. Here is how we can implement a counter using semaphores. A Producer creates data and fills BUFFER, a Consumer takes data and empties BUFFER. We use two semaphores to implement our counter: EMPTY and FULL. We need two actions on the counter INCREMENT and DECREMENT:
  • INCREMENT: Increment FULL and Decrement EMPTY
    The idea is that a Producer fills one more spot in BUFFER and removes one more empty spot
  • DECREMENT: Decrement FULL and Increment EMPTY
    The idea is that a Consumer empties one more spot in BUFFER and removes one more full spot
By combining these two additional semaphores to keep count, with our binary semaphore we can solve the Bounded Buffer Problem. I'll show you how to define the action of INCREMENT for the Producer process (the action SIGNAL is the same as we saw before--reseting the binary semaphore):

struct sembuf    CONSUMER[3},PRODUCER[3], SIGNAL[1];

       //Defining PRODUCER
       //Code for the binary semaphore
       PRODUCER[0].sem_num = 0;
       PRODUCER[0].sem_op = -1;
       PRODuCER[0].sem_flag = SEM_UNDO;

       //Defining INCREMENT
       //Our FULL semaphore
       PRODUCER[1].sem_num = 0;
       PRODUCER[1].sem_op = 1;
       PRODUCER[1].sem_flag = SEM_UNDO;

       //Defining INCREMENT
       //Our EMPTY semaphore
       PRODUCER[2].sem_num = 0;
       PRODUCER[2].sem_op = -1;
       PRODUCER[2].sem_flag = SEM_UNDO;

I'll leave the defining of the action CONSUMER to you. Here is the new implementation for the Bounded Buffer Problem for the Consumer and Producer:

       // Entry Section
       semop(semaid, CONSUMER, 3);

       // Critical Section
       //OK to take from BUFFER
       . . . BUFFER . . .

       // Exit Section
       semop(semaid, SIGNAL, 1);

and code for the Producer

       // Entry Section
       semop(semaid, PRODUCER, 3);

       // Critical Section
       //OK to add to BUFFER
       . . . BUFFER . . .

       // Exit Section
       semop(semaid, SIGNAL, 1);

No comments: