☑ Sharing pthreads locks between processes

31 Jan 2013 at 1:02PM in Software
 |  | 

How to share pthreads primitives across processes.

share tomatoes

The POSIX threads library has some useful primitives for locking between multiple threads, primarily mutexes and condition variables.

Typically these are only effective to lock between threads within the same process. However, pthreads defines a PTHREAD_PROCESS_SHARED attribute for both of these primitives which can be used to specify that they should also be used between processes.

This attribute simply indicates that the primitives be used in a way which is compatible with shared access, however—application code must still arrange to store them in shared memory. This is fairly easily arranged with an anonymous mmap(), or shmget() and shmat() if your processes don’t have a parent/child relationship.

The use of shared memory makes things a little more complicated, which is disappointing, but it still seems to me that using these primitives to synchronise processes is probably still a little more elegant than using pipes or something similar (even if that’s probably a little more portable).

I’ve added a code example as a GitHub Gist which illustrates this. I’ve used an anonymous mmap() for shared memory—previously I’d used System V shared memory, but since this isn’t cleaned up automatically on application exit then the mmap() approach is safer.

2025 UPDATE: I wrote this article back in 2013 when my blog didn’t format code snippets very well, but since then the link above to the code snippet became broken. I’ve updated it to point to a Gist, which should be quite persistent, but since this blog also formats code better now, here’s the same code inline.

If you want to see the difference that the mutex makes, just comment out pthread_mutex_lock() and pthread_mutex_unlock() in child_loop() and compare the final counter value to when they’re included.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
#include <errno.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include <sys/wait.h>
#include <unistd.h>

struct shared_data
{
    unsigned int counter;
    pthread_mutex_t mutex;
};

void child_loop(struct shared_data *shared)
{
    for (int i = 0; i < 100; ++i) {
        pthread_mutex_lock(&shared->mutex);
        unsigned int old_counter_value = shared->counter;
        // Sleep up to 100ms
        usleep(rand() % (100 * 1000));
        shared->counter = old_counter_value + 1;
        pthread_mutex_unlock(&shared->mutex);
    }
}

int main()
{
    // Create shared data section using anonymous mmap.
    struct shared_data *shared = (struct shared_data*)mmap(
            NULL,
            sizeof(struct shared_data),
            PROT_READ | PROT_WRITE,
            MAP_SHARED | MAP_ANONYMOUS,
            -1,
            0);
    if (shared == MAP_FAILED) {
        printf("mmap failed\n");
        return 1;
    }

    // Initialise the shared data, and its mutex.
    shared->counter = 0;
    pthread_mutexattr_t mutexattr;
    if (pthread_mutexattr_init(&mutexattr) != 0) {
        printf("pthread_mutexattr_init failed\n");
        return 1;
    }
    if (pthread_mutexattr_setpshared(&mutexattr, PTHREAD_PROCESS_SHARED) != 0) {
        printf("pthread_mutexattr_setpshared failed\n");
        return 1;
    }
    if (pthread_mutex_init(&shared->mutex, &mutexattr) != 0) {
        printf("pthread_mutex_init failed\n");
        return 1;
    }

    // Create five child processes using the shared data.
    for (int i = 0; i < 5; ++i) {
        pid_t ret = fork();
        if (ret < 0) {
            printf("fork failed\n");
        } else if (ret == 0) {
            printf("running child...\n");
            child_loop(shared);
            printf("child done\n");
            // Important to terminate here, not exit the loop in children.
            return 0;
        } else {
            printf("created child %d\n", ret);
        }
    }

    // Wait until there are no more children.
    printf("parent waiting for children...\n");
    while (1) {
        pid_t ret = wait(0);
        if (ret < 0) {
            if (errno != ECHILD) {
                printf("unexpected error from wait: %s\n", strerror(errno));
            }
            break;
        }
        printf("child %d terminated\n", ret);
    }

    // Finally, print the final counter result.
    printf("final value of counter: %d\n", shared->counter);

    return 0;
}
31 Jan 2013 at 1:02PM in Software
 |  |