Skip to contentSkip to conversation

rcu: local_irq_disable() also delimits RCU_SCHED read-site critical sections

PATCH In review Lai Jiangshan 3 participants 5 messages Mar 16, 2010 – Mar 17, 2010 RCU

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 rcu: local_irq_disable() also delimits RCU_SCHED read-site critical sections

It is documented that local_irq_disable() also delimits RCU_SCHED read-site critical sections.
See the document of synchronize_sched() or Documentation/RCU/whatisRCU.txt.

So we have to test irqs_disabled() in rcu_read_lock_sched_held(). Otherwise rcu-lockdep brings incorrect complaint.

---

include/linux/rcupdate.h +2 -2
index 3024050..2ce5674 100644
@@ -160,7 +160,7 @@ static inline int rcu_read_lock_sched_held(void)
 		return 1;
 	if (debug_locks)
 		lockdep_opinion = lock_is_held(&rcu_sched_lock_map);
-	return lockdep_opinion || preempt_count() != 0;
+	return lockdep_opinion || preempt_count() != 0 || irqs_disabled();
 }
 #else /* #ifdef CONFIG_PREEMPT */
 static inline int rcu_read_lock_sched_held(void)
@@ -191,7 +191,7 @@ static inline int rcu_read_lock_bh_held(void)
 #ifdef CONFIG_PREEMPT
 static inline int rcu_read_lock_sched_held(void)
 {
-	return !rcu_scheduler_active || preempt_count() != 0;
+	return !rcu_scheduler_active || preempt_count() != 0 || irqs_disabled();
 }
 #else /* #ifdef CONFIG_PREEMPT */
 static inline int rcu_read_lock_sched_held(void)
#1 lore raw
Quoted 7 lines · Lai Jiangshan

It is documented that local_irq_disable() also delimits RCU_SCHED read-site critical sections.
See the document of synchronize_sched() or Documentation/RCU/whatisRCU.txt.

So we have to test irqs_disabled() in rcu_read_lock_sched_held(). Otherwise rcu-lockdep brings incorrect complaint.

It would be useful to include the warning in question in the changelog - so that others who might be affected by it can see the fix and can track its progress. (and dont start a parallel effort debugging/reporting/fixing it)

Thanks,

Ingo

#2 lore raw
In reply to Lai Jiangshan
Quoted 7 lines · Lai Jiangshan

It is documented that local_irq_disable() also delimits RCU_SCHED read-site critical sections.
See the document of synchronize_sched() or Documentation/RCU/whatisRCU.txt.

So we have to test irqs_disabled() in rcu_read_lock_sched_held(). Otherwise rcu-lockdep brings incorrect complaint.

Interesting -- I was under the impression that preempt_count() covered this as well, due to the following in include/linux/hardirq.h:

	#define PREEMPT_MASK    (__IRQ_MASK(PREEMPT_BITS) << PREEMPT_SHIFT)
	#define SOFTIRQ_MASK    (__IRQ_MASK(SOFTIRQ_BITS) << SOFTIRQ_SHIFT)
	#define HARDIRQ_MASK    (__IRQ_MASK(HARDIRQ_BITS) << HARDIRQ_SHIFT)
	#define NMI_MASK        (__IRQ_MASK(NMI_BITS)     << NMI_SHIFT)

But irqs_disabled() does look to sample the actual interrupt hardware.

So, if there are cases where RCU is used where the interrupt hardware is disabled, but preempt_count() has not been updated, this patch is the right thing to do.

Thanx, Paul

Quoted 25 lines

---

include/linux/rcupdate.h +2 -2
index 3024050..2ce5674 100644
@@ -160,7 +160,7 @@ static inline int rcu_read_lock_sched_held(void)
 		return 1;
 	if (debug_locks)
 		lockdep_opinion = lock_is_held(&rcu_sched_lock_map);
-	return lockdep_opinion || preempt_count() != 0;
+	return lockdep_opinion || preempt_count() != 0 || irqs_disabled();
 }
 #else /* #ifdef CONFIG_PREEMPT */
 static inline int rcu_read_lock_sched_held(void)
@@ -191,7 +191,7 @@ static inline int rcu_read_lock_bh_held(void)
 #ifdef CONFIG_PREEMPT
 static inline int rcu_read_lock_sched_held(void)
 {
-	return !rcu_scheduler_active || preempt_count() != 0;
+	return !rcu_scheduler_active || preempt_count() != 0 || irqs_disabled();
 }
 #else /* #ifdef CONFIG_PREEMPT */
 static inline int rcu_read_lock_sched_held(void)
#3 lore raw
Quoted 17 lines · Paul E. McKenney
Quoted 7 lines · Lai Jiangshan

It is documented that local_irq_disable() also delimits RCU_SCHED read-site critical sections.
See the document of synchronize_sched() or Documentation/RCU/whatisRCU.txt.

So we have to test irqs_disabled() in rcu_read_lock_sched_held(). Otherwise rcu-lockdep brings incorrect complaint.

Interesting -- I was under the impression that preempt_count() covered this as well, due to the following in include/linux/hardirq.h:

	#define PREEMPT_MASK    (__IRQ_MASK(PREEMPT_BITS) << PREEMPT_SHIFT)
	#define SOFTIRQ_MASK    (__IRQ_MASK(SOFTIRQ_BITS) << SOFTIRQ_SHIFT)
	#define HARDIRQ_MASK    (__IRQ_MASK(HARDIRQ_BITS) << HARDIRQ_SHIFT)
	#define NMI_MASK        (__IRQ_MASK(NMI_BITS)     << NMI_SHIFT)
Quoted 8 lines

But irqs_disabled() does look to sample the actual interrupt hardware.

So, if there are cases where RCU is used where the interrupt hardware is disabled, but preempt_count() has not been updated, this patch is the right thing to do.

Thanx, Paul

local_irq_disable() does not touch preempt_count, it touchs a register of current CPU.

The following quick test module is a case where RCU_SCHED is used where the interrupt hardware is disabled, but preempt_count() has not been updated, and it raises rcu-lockdep complaint.

#include <linux/module.h>
#include <linux/hardirq.h>
#include <linux/rcupdate.h>

void *test = &test;

int test_init(void)

{
	local_irq_disable();
	printk(KERN_INFO "%p\n", rcu_dereference_sched(test));
	local_irq_enable();

	return 0;
}

void test_exit(void) {}

module_init(test_init);
module_exit(test_exit);
MODULE_LICENSE("GPL");

#4 lore raw
In reply to Paul E. McKenney
Quoted 17 lines · Paul E. McKenney
Quoted 7 lines · Lai Jiangshan

It is documented that local_irq_disable() also delimits RCU_SCHED read-site critical sections.
See the document of synchronize_sched() or Documentation/RCU/whatisRCU.txt.

So we have to test irqs_disabled() in rcu_read_lock_sched_held(). Otherwise rcu-lockdep brings incorrect complaint.

Interesting -- I was under the impression that preempt_count() covered this as well, due to the following in include/linux/hardirq.h:

	#define PREEMPT_MASK    (__IRQ_MASK(PREEMPT_BITS) << PREEMPT_SHIFT)
	#define SOFTIRQ_MASK    (__IRQ_MASK(SOFTIRQ_BITS) << SOFTIRQ_SHIFT)
	#define HARDIRQ_MASK    (__IRQ_MASK(HARDIRQ_BITS) << HARDIRQ_SHIFT)
	#define NMI_MASK        (__IRQ_MASK(NMI_BITS)     << NMI_SHIFT)
Quoted 5 lines

But irqs_disabled() does look to sample the actual interrupt hardware.

So, if there are cases where RCU is used where the interrupt hardware is disabled, but preempt_count() has not been updated, this patch is the right thing to do.

In light of later emails in this thread, first a big "thank you!!!" to Lai, and I will pull this one in. Good stuff!!!

Thanx, Paul

Quoted 30 lines
Quoted 17 lines

---

include/linux/rcupdate.h +1 -1
index 3024050..2ce5674 100644
@@ -160,7 +160,7 @@ static inline int rcu_read_lock_sched_held(void)
 		return 1;
 	if (debug_locks)
 		lockdep_opinion = lock_is_held(&rcu_sched_lock_map);
-	return lockdep_opinion || preempt_count() != 0;
+	return lockdep_opinion || preempt_count() != 0 || irqs_disabled();
 }
 #else /* #ifdef CONFIG_PREEMPT */
 static inline int rcu_read_lock_sched_held(void)

include/linux/rcupdate.h static inline int rcu_read_lock_bh_held(void) · line 197

Quoted 8 lines
include/linux/rcupdate.h +1 -1
@@ -191,7 +191,7 @@ static inline int rcu_read_lock_bh_held(void)
 #ifdef CONFIG_PREEMPT
 static inline int rcu_read_lock_sched_held(void)
 {
-	return !rcu_scheduler_active || preempt_count() != 0;
+	return !rcu_scheduler_active || preempt_count() != 0 || irqs_disabled();
 }
 #else /* #ifdef CONFIG_PREEMPT */
 static inline int rcu_read_lock_sched_held(void)

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/