Skip to contentSkip to conversation

'call_rcu' unstable?

RFC PATCH New Mathieu Desnoyers 1 participant 5 messages Dec 12, 2012 – Dec 26, 2012 5 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw
Quoted 17 lines · * zs ([email protected]) wrote:

thanks Mathieu Desnoyers for the patience.

Quoted 3 lines

- Did you ensure that you issue rcu_register_thread() at thread start of each of your threads ? And rcu_register_thread before returning from each thread ?

Sure.

Quoted 1 line

- How is the list g_sslvpnctxlist[h] initialized ?

Here, is the way:
I alloc and init 'g_sslvpnctxlist' in one process, and exec rcu_read_lock/unlock/call_rcu in another process(which create many rx/tx threads).

        if (sslvpn_shm_alloc(&shm) == -1) {
                syslog(LOG_ERR, "alloc share stats memory failed %s\n", strerror(errno));
                exit(-1);
        }
        g_sslvpnctxlist = (void *)shm.addr;
        for (i = 0; i < sslvpn_max_users; i++)
                 CDS_INIT_LIST_HEAD(&g_sslvpnctxlist[i]);
Quoted 3 lines

It looks strange, that someone uses share-memory(created by mmap) to hold the 'g_sslvpnctxlist' (+_+);
Later I will re-coding this part, not use share-memory.

A couple of things to keep in mind:

- the shm lists need to be initialized before being passed to the other process adding to them and deleting from them,
- you need to be aware that a synchronize_rcu() (or call_rcu()) executing in the context of one process will not see the RCU read-side locks of another process. So while it should theoretically be OK to do cds_list_add_rcu() from one process and read data from another process, both data read and use of call_rcu/synchronize_rcu need to be performed within the same process, which might make reclaim more complicated. But I understand that you allocate list node, add, remove, and iterate all from the same process, and only initialize the list head from a different process, so this should be good.

Quoted 5 lines

Are there concurrent

Quoted 2 lines

modifications of this list ? If yes, are they synchronized with a mutex ?

I do not use mutex, because all the add/del are executing in one thread.

Fair enough, as long as your list head init has been performed before your first "add".

Quoted 10 lines
Quoted 6 lines

- When your application is a state where the call_rcu worker thread busy-waits on RCU read-side critical sections, it would be interesting to know on what read-side critical section it is waiting. In order to do so, from gdb attached to your process when it is hung:
- we'

I did attach the process and check each thread, it says:
six threads block in 'msgrcv' IO, and one thread hangs in 'update_counter_and_wait:uruc.c:247'

Hrm, interesting! I wonder what messages they are waiting on. This might be the root cause of the issue: a blocking msgrcv that waits for data that is not coming, and at least one of them would be holding the RCU read-side lock while waiting. This would in turn also stop progress from the call_rcu worker thread.

Quoted 4 lines

I do not have the chance to check the rcu_reader TLS values,
because my customer will not allow the problem happlen again(I have replaced rcu_lock by pthread_mutex).

I understand.

Quoted 3 lines

I am trying to reproduce the problem in my testing workplace, if happens, I will give more.

OK, that would be helpful! But maybe finding out what messages those threads are waiting on might be enough. Is it expected by design that those threads wait for a long time on msgrcv ? Is the rcu read-side lock held across those msgrcv calls ?

Thanks,

Mathieu

Quoted 158 lines

thanks .

------------------ Original ------------------

From:  "Mathieu Desnoyers"<[email protected]>;
Date:  Wed, Dec 12, 2012 10:41 AM
To:  "zs"<[email protected]>; 

Cc: "lttng-dev"<[email protected]>;
Subject: Re: [lttng-dev] 'call_rcu' unstable?

Quoted 8 lines · * zs ([email protected]) wrote:

Thanks , BUT ...
I really check my code:

zs# find .|xargs grep rcu_read

./sslvpn_ctx.c: rcu_read_lock();
./sslvpn_ctx.c: rcu_read_unlock();
./sslvpn_ctx.c: rcu_read_unlock();

OK, in that case, continuing on the debugging checklist:

- Did you ensure that you issue rcu_register_thread() at thread start of each of your threads ? And rcu_register_thread before returning from each thread ?
- How is the list g_sslvpnctxlist[h] initialized ? Are there concurrent modifications of this list ? If yes, are they synchronized with a mutex ?
- When your application is a state where the call_rcu worker thread busy-waits on RCU read-side critical sections, it would be interesting to know on what read-side critical section it is waiting. In order to do so, from gdb attached to your process when it is hung:
- we'd need to look at the urcu.c "registry" list. We'd need to figure out which list entries are keeping the busy-loop waiting.
- then, we should look at each thread's "rcu_reader" TLS variable, to see its address and content.
By comparing the content of the list and each active thread's rcu_reader TLS variable, we should be able to figure out what is keeping grace period to complete. If you can provide these dumps, it would let me help you digging further into your issue.

Thanks,

Mathieu

Quoted 102 lines

AND in sslvpn_ctx.c:
void *sslvpn_lookup_ssl(unsigned long iip)

{
        struct sslvpn_ctx *ctx;
        int h;

        h = get_hash(iip, 0);

        rcu_read_lock();
        cds_list_for_each_entry_rcu(ctx, &g_sslvpnctxlist[h], cdlist) {
                if ((ctx->flags & SSL_CTX_ESTABLISHED) && ctx->iip && ctx->iip == iip) {

                        uatomic_add(&ctx->ssl_use, 1);
                        rcu_read_unlock();
                        return ctx;
                }
        }

        rcu_read_unlock();
        return NULL;
}

By the way, *sslvpn_lookup_ssl* called by 6 threads for TX. the 7th thread only will call *call_rcu*:

int sslvpn_del_ctx(struct sslvpn_ctx *pctx)

{
        ...
        cds_list_del_rcu(&ctx->cdlist);
        ctx->flags |= SSL_CTX_DYING;
        call_rcu(&ctx->rcu, func);
        ...
}

------------------ Original ------------------

From:  "Mathieu Desnoyers"<[email protected]>;
Date:  Wed, Dec 12, 2012 01:55 AM
To:  "zs"<[email protected]>; 

Cc: "lttng-dev"<[email protected]>;
Subject: Re: [lttng-dev] 'call_rcu' unstable?

Quoted 9 lines · * zs ([email protected]) wrote:

Hi list,

I found a big problem in my product, that use urcu 0.7.5. My program cost too mutch CPU in the funtion 'update_counter_and_wait:uruc.c:247', and I use gdb to see to *wait_loops*, it says -167777734. The CPU usage grows up from 1% to 100% in one day!

Here is the sample code to show how I use urcu library:

#include <urcu.h>

Quoted 8 lines

thread ()
{
rcu_register_thread();

for (;;) {
rcu_read_lock();
xxx
rcu_read_unlock();

Please triple-check that all your rcu_read_lock() and rcu_read_unlock() are balanced (no double-unlock, nor missing unlock for each lock taken).

The type of problem you get would happen in such a case.

Thanks,

Mathieu

Quoted 20 lines

}
}

main()
{
rcu_init();
pthread_create(, , , , thread);

rcu_register_thread();
for (;;)
{
if (xxx)
call_rcu();
}

}
_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Signature · 4 lines
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
Signature · 4 lines
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
Signature · 4 lines
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com

A message that is not in the archive (sent off-list or not yet received): 20121212145848.GA29236@Krystal

#3 lore raw
Quoted 69 lines · * Mathieu Desnoyers ([email protected]) wrote:
Quoted 17 lines · * zs ([email protected]) wrote:

thanks Mathieu Desnoyers for the patience.

Quoted 3 lines

- Did you ensure that you issue rcu_register_thread() at thread start of each of your threads ? And rcu_register_thread before returning from each thread ?

Sure.

Quoted 1 line

- How is the list g_sslvpnctxlist[h] initialized ?

Here, is the way:
I alloc and init 'g_sslvpnctxlist' in one process, and exec rcu_read_lock/unlock/call_rcu in another process(which create many rx/tx threads).

        if (sslvpn_shm_alloc(&shm) == -1) {
                syslog(LOG_ERR, "alloc share stats memory failed %s\n", strerror(errno));
                exit(-1);
        }
        g_sslvpnctxlist = (void *)shm.addr;
        for (i = 0; i < sslvpn_max_users; i++)
                 CDS_INIT_LIST_HEAD(&g_sslvpnctxlist[i]);
Quoted 3 lines

It looks strange, that someone uses share-memory(created by mmap) to hold the 'g_sslvpnctxlist' (+_+);
Later I will re-coding this part, not use share-memory.

A couple of things to keep in mind:

- the shm lists need to be initialized before being passed to the other process adding to them and deleting from them,
- you need to be aware that a synchronize_rcu() (or call_rcu()) executing in the context of one process will not see the RCU read-side locks of another process. So while it should theoretically be OK to do cds_list_add_rcu() from one process and read data from another process, both data read and use of call_rcu/synchronize_rcu need to be performed within the same process, which might make reclaim more complicated. But I understand that you allocate list node, add, remove, and iterate all from the same process, and only initialize the list head from a different process, so this should be good.

Quoted 5 lines

Are there concurrent

Quoted 2 lines

modifications of this list ? If yes, are they synchronized with a mutex ?

I do not use mutex, because all the add/del are executing in one thread.

Fair enough, as long as your list head init has been performed before your first "add".

Quoted 10 lines
Quoted 6 lines

- When your application is a state where the call_rcu worker thread busy-waits on RCU read-side critical sections, it would be interesting to know on what read-side critical section it is waiting. In order to do so, from gdb attached to your process when it is hung:
- we'

I did attach the process and check each thread, it says:
six threads block in 'msgrcv' IO, and one thread hangs in 'update_counter_and_wait:uruc.c:247'

Hrm, interesting! I wonder what messages they are waiting on. This might be the root cause of the issue: a blocking msgrcv that waits for data that is not coming, and at least one of them would be holding the RCU read-side lock while waiting. This would in turn also stop progress from the call_rcu worker thread.

Quoted 4 lines

I do not have the chance to check the rcu_reader TLS values,
because my customer will not allow the problem happlen again(I have replaced rcu_lock by pthread_mutex).

I understand.

Quoted 1 line
Quoted 7 lines
Quoted 2 lines

I am trying to reproduce the problem in my testing workplace, if happens, I will give more.

OK, that would be helpful! But maybe finding out what messages those threads are waiting on might be enough. Is it expected by design that those threads wait for a long time on msgrcv ? Is the rcu read-side lock held across those msgrcv calls ?

One more question about the nature of your application: is it using a "fork()" (or clone()) not followed by exec() ?

This design pattern is quite common in daemons using multi-process worker threads, and it's very likely in your case since you are using shared memory between processes. This has important impacts on the way threads are handled (see the pthread_atfork(3) manpage). Given that URCU keeps track of all reader threads, if any of those vanish at fork(), due to Linux implementation limitations, they need to be cleared from the urcu registry before a fork() not followed by exec().

For more info, please refer to the Userspace RCU README file, under section "Interaction with fork()".

Especially the part:

"Most liburcu implementations require that all registrations (as reader, defer_rcu and call_rcu threads) should be released before a fork() is performed, except for the rather common scenario where fork() is immediately followed by exec() in the child process."

So one possible explanation for the scenario you are observing is that the parent process still has RCU readers registered while performing fork(), and that the child process is unable to complete grace periods due to this stale list entry.

One thing to keep in mind is that this commit:

commit 765f3eadad5647e6fa853414fc652670f9e00966

Author: Mathieu Desnoyers <[email protected]>
Date:   Sat Sep 10 22:02:58 2011 -0700

call_rcu: register work threads as rcu readers

makes all call_rcu worker threads register as RCU readers. Since the default call_rcu thread is never cleaned up, it becomes hard for the application to ensure fork() correctness.

Am I on the right track ?

Thanks,

Mathieu

Quoted 173 lines

Thanks,

Mathieu

Quoted 158 lines

thanks .

------------------ Original ------------------

From:  "Mathieu Desnoyers"<[email protected]>;
Date:  Wed, Dec 12, 2012 10:41 AM
To:  "zs"<[email protected]>; 

Cc: "lttng-dev"<[email protected]>;
Subject: Re: [lttng-dev] 'call_rcu' unstable?

Quoted 8 lines · * zs ([email protected]) wrote:

Thanks , BUT ...
I really check my code:

zs# find .|xargs grep rcu_read

./sslvpn_ctx.c: rcu_read_lock();
./sslvpn_ctx.c: rcu_read_unlock();
./sslvpn_ctx.c: rcu_read_unlock();

OK, in that case, continuing on the debugging checklist:

- Did you ensure that you issue rcu_register_thread() at thread start of each of your threads ? And rcu_register_thread before returning from each thread ?
- How is the list g_sslvpnctxlist[h] initialized ? Are there concurrent modifications of this list ? If yes, are they synchronized with a mutex ?
- When your application is a state where the call_rcu worker thread busy-waits on RCU read-side critical sections, it would be interesting to know on what read-side critical section it is waiting. In order to do so, from gdb attached to your process when it is hung:
- we'd need to look at the urcu.c "registry" list. We'd need to figure out which list entries are keeping the busy-loop waiting.
- then, we should look at each thread's "rcu_reader" TLS variable, to see its address and content.
By comparing the content of the list and each active thread's rcu_reader TLS variable, we should be able to figure out what is keeping grace period to complete. If you can provide these dumps, it would let me help you digging further into your issue.

Thanks,

Mathieu

Quoted 102 lines

AND in sslvpn_ctx.c:
void *sslvpn_lookup_ssl(unsigned long iip)

{
        struct sslvpn_ctx *ctx;
        int h;

        h = get_hash(iip, 0);

        rcu_read_lock();
        cds_list_for_each_entry_rcu(ctx, &g_sslvpnctxlist[h], cdlist) {
                if ((ctx->flags & SSL_CTX_ESTABLISHED) && ctx->iip && ctx->iip == iip) {

                        uatomic_add(&ctx->ssl_use, 1);
                        rcu_read_unlock();
                        return ctx;
                }
        }

        rcu_read_unlock();
        return NULL;
}

By the way, *sslvpn_lookup_ssl* called by 6 threads for TX. the 7th thread only will call *call_rcu*:

int sslvpn_del_ctx(struct sslvpn_ctx *pctx)

{
        ...
        cds_list_del_rcu(&ctx->cdlist);
        ctx->flags |= SSL_CTX_DYING;
        call_rcu(&ctx->rcu, func);
        ...
}

------------------ Original ------------------

From:  "Mathieu Desnoyers"<[email protected]>;
Date:  Wed, Dec 12, 2012 01:55 AM
To:  "zs"<[email protected]>; 

Cc: "lttng-dev"<[email protected]>;
Subject: Re: [lttng-dev] 'call_rcu' unstable?

Quoted 9 lines · * zs ([email protected]) wrote:

Hi list,

I found a big problem in my product, that use urcu 0.7.5. My program cost too mutch CPU in the funtion 'update_counter_and_wait:uruc.c:247', and I use gdb to see to *wait_loops*, it says -167777734. The CPU usage grows up from 1% to 100% in one day!

Here is the sample code to show how I use urcu library:

#include <urcu.h>

Quoted 8 lines

thread ()
{
rcu_register_thread();

for (;;) {
rcu_read_lock();
xxx
rcu_read_unlock();

Please triple-check that all your rcu_read_lock() and rcu_read_unlock() are balanced (no double-unlock, nor missing unlock for each lock taken).

The type of problem you get would happen in such a case.

Thanks,

Mathieu

Quoted 20 lines

}
}

main()
{
rcu_init();
pthread_create(, , , , thread);

rcu_register_thread();
for (;;)
{
if (xxx)
call_rcu();
}

}
_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Signature · 4 lines
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
Signature · 4 lines
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
Signature · 9 lines
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
Signature · 4 lines
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com

A message that is not in the archive (sent off-list or not yet received): 20121213103537.GB8637@Krystal

#5 lore raw

* Mathieu Desnoyers ([email protected]) wrote:
[...]

Quoted 33 lines

One more question about the nature of your application: is it using a "fork()" (or clone()) not followed by exec() ?

This design pattern is quite common in daemons using multi-process worker threads, and it's very likely in your case since you are using shared memory between processes. This has important impacts on the way threads are handled (see the pthread_atfork(3) manpage). Given that URCU keeps track of all reader threads, if any of those vanish at fork(), due to Linux implementation limitations, they need to be cleared from the urcu registry before a fork() not followed by exec().

For more info, please refer to the Userspace RCU README file, under section "Interaction with fork()".

Especially the part:

"Most liburcu implementations require that all registrations (as reader, defer_rcu and call_rcu threads) should be released before a fork() is performed, except for the rather common scenario where fork() is immediately followed by exec() in the child process."

So one possible explanation for the scenario you are observing is that the parent process still has RCU readers registered while performing fork(), and that the child process is unable to complete grace periods due to this stale list entry.

One thing to keep in mind is that this commit:

commit 765f3eadad5647e6fa853414fc652670f9e00966

Author: Mathieu Desnoyers <[email protected]>
Date:   Sat Sep 10 22:02:58 2011 -0700
Quoted 7 lines

call_rcu: register work threads as rcu readers

makes all call_rcu worker threads register as RCU readers. Since the default call_rcu thread is never cleaned up, it becomes hard for the application to ensure fork() correctness.

Am I on the right track ?

I created a small patch that reproduces your faulty behavior by simply using fork() in a program (without following exec()). The child process gets stucked in wait_for_readers() eating 100% of one CPU. More to come soon.

Comments are welcome,

Thanks,

Mathieu

---

tests/Makefile.am +3 -1
index f8b4c67..0e15a4c 100644
@@ -20,7 +20,8 @@ noinst_PROGRAMS = test_urcu test_urcu_dynamic_link test_urcu_timing \
 	test_urcu_wfcq_dynlink \
 	test_urcu_lfq_dynlink test_urcu_lfs_dynlink test_urcu_hash \
 	test_urcu_lfs_rcu_dynlink \
-	test_urcu_multiflavor test_urcu_multiflavor_dynlink
+	test_urcu_multiflavor test_urcu_multiflavor_dynlink \
+	test_urcu_fork
 noinst_HEADERS = rcutorture.h test_urcu_multiflavor.h
 
 if COMPAT_ARCH
@@ -85,6 +86,7 @@ test_urcu_signal_timing_CFLAGS= -DRCU_SIGNAL $(AM_CFLAGS)
 test_urcu_signal_yield_SOURCES = test_urcu.c $(URCU_SIGNAL)
 test_urcu_signal_yield_CFLAGS = -DRCU_SIGNAL -DDEBUG_YIELD $(AM_CFLAGS)
 
+test_urcu_fork_SOURCES = test_urcu_fork.c $(URCU)
 
 test_rwlock_timing_SOURCES = test_rwlock_timing.c $(URCU_SIGNAL)
 
tests/test_urcu_fork.c new file +141 -0

141 lines not on this page · raw message

Signature · 4 lines
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com

A message that is not in the archive (sent off-list or not yet received): 20121218042214.GA16410@Krystal

#7 lore raw

RFC PATCH 1/1 Fix call_rcu fork handling

Fix call_rcu fork handling by putting all call_rcu threads in a quiescent state before fork (paused state), and unpausing them when the parent returns from fork.

On the child, everything will run fine as long as we don't issue fork() from a call_rcu callback.

Side-note: pthread_atfork is not appropriate when playing with multithread and malloc/free. The glibc malloc implementation sadly expects that all malloc/free are executed from the context of a single thread while pthread atfork handlers are running, which leads to interesting hang in glibc.

---

tests/test_urcu_fork.c +6 -0
index 07c521a..6e454b5 100644
@@ -85,6 +85,8 @@ int main(int argc, char **argv)
 	pid_t pid;
 	int ret;
 
+#if 0
+	/* pthread_atfork does not work with malloc/free in callbacks */
 	ret = pthread_atfork(call_rcu_before_fork,
 		call_rcu_after_fork_parent,
 		call_rcu_after_fork_child);
@@ -93,6 +95,7 @@ int main(int argc, char **argv)
 		perror("pthread_atfork");
 		exit(EXIT_FAILURE);
 	}
+#endif
 
 	test_rcu();
 
@@ -101,10 +104,12 @@ int main(int argc, char **argv)
 	fprintf(stderr, "%s parent pid: %d, before fork\n",
 		argv[0], (int) getpid());
 
+	call_rcu_before_fork();
 	pid = fork();
 
 	if (pid == 0) {
 		/* child */
+		call_rcu_after_fork_child();
 		fprintf(stderr, "%s child pid: %d, after fork\n",
 			argv[0], (int) getpid());
 		test_rcu();
@@ -114,6 +119,7 @@ int main(int argc, char **argv)
 		int status;
 
 		/* parent */
+		call_rcu_after_fork_parent();
 		fprintf(stderr, "%s parent pid: %d, after fork\n",
 			argv[0], (int) getpid());
 		test_rcu();
urcu-call-rcu-impl.h +41 -4

80 lines not on this page · raw message

urcu-call-rcu.h +6 -4
index 1dad0e2..997bb2f 100644
@@ -44,10 +44,12 @@ struct call_rcu_data;
 
 /* Flag values. */
 
-#define URCU_CALL_RCU_RT	0x1
-#define URCU_CALL_RCU_RUNNING	0x2
-#define URCU_CALL_RCU_STOP	0x4
-#define URCU_CALL_RCU_STOPPED	0x8
+#define URCU_CALL_RCU_RT	(1U << 0)
+#define URCU_CALL_RCU_RUNNING	(1U << 1)
+#define URCU_CALL_RCU_STOP	(1U << 2)
+#define URCU_CALL_RCU_STOPPED	(1U << 3)
+#define URCU_CALL_RCU_PAUSE	(1U << 4)
+#define URCU_CALL_RCU_PAUSED	(1U << 5)
 
 /*
  * The rcu_head data structure is placed in the structure to be freed
Signature · 4 lines
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com

A message that is not in the archive (sent off-list or not yet received): 20121218054205.GA17469@Krystal

#9 lore raw

Re: [RFC PATCH] Fix call_rcu fork handling

Quoted 4 lines · * Mathieu Desnoyers ([email protected]) wrote:

Fix call_rcu fork handling by putting all call_rcu threads in a quiescent state before fork (paused state), and unpausing them when the parent returns from fork.

Quoted 8 lines

On the child, everything will run fine as long as we don't issue fork() from a call_rcu callback.

Side-note: pthread_atfork is not appropriate when playing with multithread and malloc/free. The glibc malloc implementation sadly expects that all malloc/free are executed from the context of a single thread while pthread atfork handlers are running, which leads to interesting hang in glibc.

I'll rework the patch so the modification to tests/test_urcu_fork.c is moved to a separate patch, and merge the fix into the master branch, since it solves a reproduceable problem.

Thanks,

Mathieu

Quoted 165 lines

---

tests/test_urcu_fork.c +6 -0
index 07c521a..6e454b5 100644
@@ -85,6 +85,8 @@ int main(int argc, char **argv)
 	pid_t pid;
 	int ret;
 
+#if 0
+	/* pthread_atfork does not work with malloc/free in callbacks */
 	ret = pthread_atfork(call_rcu_before_fork,
 		call_rcu_after_fork_parent,
 		call_rcu_after_fork_child);
@@ -93,6 +95,7 @@ int main(int argc, char **argv)
 		perror("pthread_atfork");
 		exit(EXIT_FAILURE);
 	}
+#endif
 
 	test_rcu();
 
@@ -101,10 +104,12 @@ int main(int argc, char **argv)
 	fprintf(stderr, "%s parent pid: %d, before fork\n",
 		argv[0], (int) getpid());
 
+	call_rcu_before_fork();
 	pid = fork();
 
 	if (pid == 0) {
 		/* child */
+		call_rcu_after_fork_child();
 		fprintf(stderr, "%s child pid: %d, after fork\n",
 			argv[0], (int) getpid());
 		test_rcu();
@@ -114,6 +119,7 @@ int main(int argc, char **argv)
 		int status;
 
 		/* parent */
+		call_rcu_after_fork_parent();
 		fprintf(stderr, "%s parent pid: %d, after fork\n",
 			argv[0], (int) getpid());
 		test_rcu();
urcu-call-rcu-impl.h +41 -4
index 6580397..f6625ba 100644
@@ -75,8 +75,9 @@ static CDS_LIST_HEAD(call_rcu_data_list);
 
 static DEFINE_URCU_TLS(struct call_rcu_data *, thread_call_rcu_data);
 
-/* Guard call_rcu thread creation. */
-
+/*
+ * Guard call_rcu thread creation and atfork handlers.
+ */
 static pthread_mutex_t call_rcu_mutex = PTHREAD_MUTEX_INITIALIZER;
 
 /* If a given thread does not have its own call_rcu thread, this is default. */
@@ -254,6 +255,21 @@ static void *call_rcu_thread(void *arg)
 		struct cds_wfcq_node *cbs, *cbs_tmp_n;
 		enum cds_wfcq_ret splice_ret;
 
+		if (uatomic_read(&crdp->flags) & URCU_CALL_RCU_PAUSE) {
+			/*
+			 * Pause requested. Become quiescent: remove
+			 * ourself from all global lists, and don't
+			 * process any callback. The callback lists may
+			 * still be non-empty though.
+			 */
+			rcu_unregister_thread();
+			cmm_smp_mb__before_uatomic_or();
+			uatomic_or(&crdp->flags, URCU_CALL_RCU_PAUSED);
+			while ((uatomic_read(&crdp->flags) & URCU_CALL_RCU_PAUSE) != 0)
+				poll(NULL, 0, 1);
+			rcu_register_thread();
+		}
+
 		cds_wfcq_init(&cbs_tmp_head, &cbs_tmp_tail);
 		splice_ret = __cds_wfcq_splice_blocking(&cbs_tmp_head,
 			&cbs_tmp_tail, &crdp->cbs_head, &crdp->cbs_tail);
@@ -705,12 +721,25 @@ void free_all_cpu_call_rcu_data(void)
 
 /*
  * Acquire the call_rcu_mutex in order to ensure that the child sees
- * all of the call_rcu() data structures in a consistent state.
+ * all of the call_rcu() data structures in a consistent state. Ensure
+ * that all call_rcu threads are in a quiescent state across fork.
  * Suitable for pthread_atfork() and friends.
  */
 void call_rcu_before_fork(void)
 {
+	struct call_rcu_data *crdp;
+
 	call_rcu_lock(&call_rcu_mutex);
+
+	cds_list_for_each_entry(crdp, &call_rcu_data_list, list) {
+		uatomic_or(&crdp->flags, URCU_CALL_RCU_PAUSE);
+		cmm_smp_mb__after_uatomic_or();
+		wake_call_rcu_thread(crdp);
+	}
+	cds_list_for_each_entry(crdp, &call_rcu_data_list, list) {
+		while ((uatomic_read(&crdp->flags) & URCU_CALL_RCU_PAUSED) == 0)
+			poll(NULL, 0, 1);
+	}
 }
 
 /*
@@ -720,6 +749,10 @@ void call_rcu_before_fork(void)
  */
 void call_rcu_after_fork_parent(void)
 {
+	struct call_rcu_data *crdp;
+
+	cds_list_for_each_entry(crdp, &call_rcu_data_list, list)
+		uatomic_and(&crdp->flags, ~URCU_CALL_RCU_PAUSE);
 	call_rcu_unlock(&call_rcu_mutex);
 }
 
@@ -752,7 +785,11 @@ void call_rcu_after_fork_child(void)
 	rcu_set_pointer(&per_cpu_call_rcu_data, NULL);
 	URCU_TLS(thread_call_rcu_data) = NULL;
 
-	/* Dispose of all of the rest of the call_rcu_data structures. */
+	/*
+	 * Dispose of all of the rest of the call_rcu_data structures.
+	 * Leftover call_rcu callbacks will be merged into the new
+	 * default call_rcu thread queue.
+	 */
 	cds_list_for_each_entry_safe(crdp, next, &call_rcu_data_list, list) {
 		if (crdp == default_call_rcu_data)
 			continue;
urcu-call-rcu.h +6 -4
index 1dad0e2..997bb2f 100644
@@ -44,10 +44,12 @@ struct call_rcu_data;
 
 /* Flag values. */
 
-#define URCU_CALL_RCU_RT	0x1
-#define URCU_CALL_RCU_RUNNING	0x2
-#define URCU_CALL_RCU_STOP	0x4
-#define URCU_CALL_RCU_STOPPED	0x8
+#define URCU_CALL_RCU_RT	(1U << 0)
+#define URCU_CALL_RCU_RUNNING	(1U << 1)
+#define URCU_CALL_RCU_STOP	(1U << 2)
+#define URCU_CALL_RCU_STOPPED	(1U << 3)
+#define URCU_CALL_RCU_PAUSE	(1U << 4)
+#define URCU_CALL_RCU_PAUSED	(1U << 5)
 
 /*
  * The rcu_head data structure is placed in the structure to be freed
Signature · 9 lines
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
Signature · 4 lines
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com