Skip to contentSkip to conversation

URCU accessing rcu_head->func

DISCUSSION Mathieu Desnoyers 2 participants 3 messages Feb 3, 2014 3 messages not in the archive

lore
Share & tools
lore
#1 lore raw
In reply to a message not in the archive

----- Original Message -----

Quoted 20 lines

From: "Florian Lohoff" <[email protected]>
To: [email protected]
Sent: Friday, January 31, 2014 3:16:32 PM
Subject: [lttng-dev] URCU accessing rcu_head->func

Hi,

i am having multiple timer which can issue a call_rcu on an object using an rcu_head.

It can happen than an object is already on the call_rcu list for destruction and another timer fires.

So i am doing something like this.

	if (!rs->rcuhead.func) {
		call_rcu(&rs->rcuhead, free_obj);
	}
Quoted 6 lines

I have a feeling in my gut that this didnt blow up just by accident and i am doing something utterly stupid.

I am in the read side critical section so my structure cant appear under my feet but the above can be racy at best.

I guess you meant "disappear" here ?

Quoted 6 lines

What would be the best way to achieve something like:

"queue for call_rcu if not already queued"

Another flag with an atomic inc or something?

Indeed, the scheme you describe above seems to be racy, since there could be two or more read-side critical sections accessing the same object concurrently, and doing two call_rcu() enqueuing the same object (same linked list node) twice into the call_rcu lists will simply corrupt those lists.

Let me first rephrase your intent here just to make sure I understand: you use RCU read-side to provide existence guarantee on the "rs" object, and you want to enqueue the object rcuhead with call_rcu() for deletion (free_obj) after a grace period has elapsed.

Assuming you care about keeping this code path wait-free or lock-free (at least), using uatomic_xchg() would allow you to set an atomic flag and know, from the returned value, whether you "own" the object reclaim or not. You can then decide whether or not call_rcu() needs to be called based on the value read by uatomic_xchg().

You could do something similar with mutexes, but then you would lose the wait-freedom characteristic of this code path, which I am guessing is one key reason why you use RCU read-side lock and call_rcu() in the first place.

Does it make sense ?

Thanks,

Mathieu

Quoted 10 lines

Flo
--
Florian Lohoff [email protected]

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#3 lore raw
In reply to a message not in the archive

Hi, Mathieu,

Quoted 6 lines · Mathieu Desnoyers
Quoted 4 lines

I am in the read side critical section so my structure cant appear under my feet but the above can be racy at best.

I guess you meant "disappear" here ?

Yep

Quoted 22 lines
Quoted 5 lines

What would be the best way to achieve something like:

"queue for call_rcu if not already queued"

Another flag with an atomic inc or something?

Indeed, the scheme you describe above seems to be racy, since there could be two or more read-side critical sections accessing the same object concurrently, and doing two call_rcu() enqueuing the same object (same linked list node) twice into the call_rcu lists will simply corrupt those lists.

Let me first rephrase your intent here just to make sure I understand: you use RCU read-side to provide existence guarantee on the "rs" object, and you want to enqueue the object rcuhead with call_rcu() for deletion (free_obj) after a grace period has elapsed.

Assuming you care about keeping this code path wait-free or lock-free (at least), using uatomic_xchg() would allow you to set an atomic flag and know, from the returned value, whether you "own" the object reclaim or not. You can then decide whether or not call_rcu() needs to be called based on the value read by uatomic_xchg().

Quoted 5 lines

You could do something similar with mutexes, but then you would lose the wait-freedom characteristic of this code path, which I am guessing is one key reason why you use RCU read-side lock and call_rcu() in the first place.

Does it make sense ?

I solved it a little different. I am using a RCU hash for keeping the data structures and i simply took the reasult of cds_lfht_del or cds_lfht_add_replace.

If i successfully remove my structure from the hash i queue for later destruction. As the removal should only happen successfully once i should be done.

Its just a little bit more tricky to get correct.

Flo

Signature · 1 line
Florian Lohoff                                                 [email protected]

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

#5 lore raw
In reply to a message not in the archive

----- Original Message -----

Quoted 61 lines

From: "Florian Lohoff" <[email protected]>
To: "Mathieu Desnoyers" <[email protected]>
Cc: [email protected], "Paul E. McKenney" <[email protected]>
Sent: Monday, February 3, 2014 2:52:59 AM
Subject: Re: [lttng-dev] URCU accessing rcu_head->func

Hi, Mathieu,

Quoted 6 lines · Mathieu Desnoyers
Quoted 4 lines

I am in the read side critical section so my structure cant appear under my feet but the above can be racy at best.

I guess you meant "disappear" here ?

Yep

Quoted 31 lines
Quoted 5 lines

What would be the best way to achieve something like:

"queue for call_rcu if not already queued"

Another flag with an atomic inc or something?

Indeed, the scheme you describe above seems to be racy, since there could be
two or more read-side critical sections accessing the same object concurrently,
and doing two call_rcu() enqueuing the same object (same linked list node) twice
into the call_rcu lists will simply corrupt those lists.

Let me first rephrase your intent here just to make sure I understand: you use
RCU read-side to provide existence guarantee on the "rs" object, and you want
to enqueue the object rcuhead with call_rcu() for deletion (free_obj) after a
grace period has elapsed.

Assuming you care about keeping this code path wait-free or lock-free (at least),
using uatomic_xchg() would allow you to set an atomic flag and know, from the
returned value, whether you "own" the object reclaim or not. You can then decide
whether or not call_rcu() needs to be called based on the value read by uatomic_xchg().

Quoted 6 lines

You could do something similar with mutexes, but then you would lose the wait-freedom characteristic of this code path, which I am guessing is one key reason why you use RCU read-side lock and call_rcu() in the first place.

Does it make sense ?

I solved it a little different. I am using a RCU hash for keeping the data structures and i simply took the reasult of cds_lfht_del or cds_lfht_add_replace.

Quoted 5 lines

If i successfully remove my structure from the hash i queue for later destruction. As the removal should only happen successfully once i should be done.

Its just a little bit more tricky to get correct.

Indeed! We have the return value to del/add replace in place for this kind of use-case, and it does get you the same result as using the flag technique.

Thanks,

Mathieu

Quoted 5 lines

Flo
--
Florian Lohoff [email protected]

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com