Skip to contentSkip to conversation

cgroup: Use irqsave in cgroup_rstat_flush_locked().

PATCH Applied Sebastian Andrzej Siewior 2 participants 5 messages Mar 1, 2022 – Mar 2, 2022 2 messages not in the archive

Subsystems (4)
lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [PATCH 0/2] Correct locking assumption on PREEMPT_RT

A message that is not in the archive (sent off-list or not yet received): [email protected]

#2 lore raw

PATCH 1/2 cgroup: Use irqsave in cgroup_rstat_flush_locked().

All callers of cgroup_rstat_flush_locked() acquire cgroup_rstat_lock either with spin_lock_irq() or spin_lock_irqsave(). cgroup_rstat_flush_locked() itself acquires cgroup_rstat_cpu_lock which is a raw_spin_lock. This lock is also acquired in cgroup_rstat_updated() in IRQ context and therefore requires _irqsave() locking suffix in cgroup_rstat_flush_locked().
Since there is no difference between spin_lock_t and raw_spin_lock_t on !RT lockdep does not complain here. On RT lockdep complains because the interrupts were not disabled here and a deadlock is possible.

Acquire the raw_spin_lock_t with disabled interrupts.

---
 kernel/cgroup/rstat.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
kernel/cgroup/rstat.c +3 -2
index 9d331ba44870a..53b771c20ee50 100644
@@ -153,8 +153,9 @@ static void cgroup_rstat_flush_locked(struct cgroup *cgrp, bool may_sleep)
 		raw_spinlock_t *cpu_lock = per_cpu_ptr(&cgroup_rstat_cpu_lock,
 						       cpu);
 		struct cgroup *pos = NULL;
+		unsigned long flags;
 
-		raw_spin_lock(cpu_lock);
+		raw_spin_lock_irqsave(cpu_lock, flags);
 		while ((pos = cgroup_rstat_cpu_pop_updated(pos, cgrp, cpu))) {
 			struct cgroup_subsys_state *css;
 
@@ -166,7 +167,7 @@ static void cgroup_rstat_flush_locked(struct cgroup *cgrp, bool may_sleep)
 				css->ss->css_rstat_flush(css, cpu);
 			rcu_read_unlock();
 		}
-		raw_spin_unlock(cpu_lock);
+		raw_spin_unlock_irqrestore(cpu_lock, flags);
 
 		/* if @may_sleep, play nice and yield if necessary */
 		if (may_sleep && (need_resched() ||
Signature · 1 line
2.35.1
maintainer
#3 lore raw

Hello,

Quoted 10 lines · Sebastian Andrzej Siewior

All callers of cgroup_rstat_flush_locked() acquire cgroup_rstat_lock either with spin_lock_irq() or spin_lock_irqsave(). cgroup_rstat_flush_locked() itself acquires cgroup_rstat_cpu_lock which is a raw_spin_lock. This lock is also acquired in cgroup_rstat_updated() in IRQ context and therefore requires _irqsave() locking suffix in cgroup_rstat_flush_locked().
Since there is no difference between spin_lock_t and raw_spin_lock_t on !RT lockdep does not complain here. On RT lockdep complains because the interrupts were not disabled here and a deadlock is possible.

Quoted 3 lines

Acquire the raw_spin_lock_t with disabled interrupts.

Can you please add a comment explaining why irqsave is being used? As it stands, it just looks spurious.

Thanks.

Signature · 1 line
tejun
#4 lore raw

PATCH 1/1 cgroup: Add a comment to cgroup_rstat_flush_locked().

Add a comment why spin_lock_irq() -> raw_spin_lock_irqsave() is needed.

---

Quoted 1 line · Tejun Heo

Hello,

Hello Tejun,

Quoted 2 lines

Can you please add a comment explaining why irqsave is being used? As it stands, it just looks spurious.

Something like this?

 kernel/cgroup/rstat.c | 8 ++++++++
 1 file changed, 8 insertions(+)
kernel/cgroup/rstat.c +8 -0
index 53b771c20ee50..ba7a660184e41 100644
@@ -155,6 +155,14 @@ static void cgroup_rstat_flush_locked(struct cgroup *cgrp, bool may_sleep)
 		struct cgroup *pos = NULL;
 		unsigned long flags;
 
+		/*
+		 * The _irqsave() is needed because cgroup_rstat_lock is
+		 * spinlock_t which is a sleeping lock on PREEMPT_RT. Acquiring
+		 * this lock with the _irq() suffix only disables interrupts on
+		 * a non-PREEMPT_RT kernel. The raw_spinlock_t below disables
+		 * interrupts on both configurations. The _irqsave() ensures
+		 * that interrupts are always disabled and later restored.
+		 */
 		raw_spin_lock_irqsave(cpu_lock, flags);
 		while ((pos = cgroup_rstat_cpu_pop_updated(pos, cgrp, cpu))) {
 			struct cgroup_subsys_state *css;
Signature · 1 line
2.35.1
maintainer Acked-by on v1 1/1
#5 lore raw

Re: [PATCH] cgroup: Add a comment to cgroup_rstat_flush_locked().

Quoted 5 lines · Sebastian Andrzej Siewior

Add a comment why spin_lock_irq() -> raw_spin_lock_irqsave() is needed.

---
On 2022-03-01 20:38:51 [-1000], Tejun Heo wrote:

Quoted 8 lines
Quoted 1 line

Hello,

Hello Tejun,

Quoted 2 lines

Can you please add a comment explaining why irqsave is being used? As it stands, it just looks spurious.

Something like this?

Yeah, looks good to me.

Thanks.

Signature · 1 line
tejun
#6 lore raw

PATCH 2/2 mm: workingset: Replace IRQ-off check with a lockdep assert.

In reply to a message not in the archive

Commit
68d48e6a2df57 ("mm: workingset: add vmstat counter for shadow nodes")

introduced an IRQ-off check to ensure that a lock is held which also disabled interrupts. This does not work the same way on PREEMPT_RT because none of the locks, that are held, disable interrupts.

Replace this check with a lockdep assert which ensures that the lock is held.

---
 mm/workingset.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
mm/workingset.c +4 -1
index 2e4fd7c3296fe..8a3828acc0bfd 100644
@@ -434,6 +434,8 @@ struct list_lru shadow_nodes;
 
 void workingset_update_node(struct xa_node *node)
 {
+	struct address_space *mapping;
+
 	/*
 	 * Track non-empty nodes that contain only shadow entries;
 	 * unlink those that contain pages or are being freed.
@@ -442,7 +444,8 @@ void workingset_update_node(struct xa_node *node)
 	 * already where they should be. The list_empty() test is safe
 	 * as node->private_list is protected by the i_pages lock.
 	 */
-	VM_WARN_ON_ONCE(!irqs_disabled());  /* For __inc_lruvec_page_state */
+	mapping = container_of(node->array, struct address_space, i_pages);
+	lockdep_assert_held(&mapping->i_pages.xa_lock);
 
 	if (node->count && node->count == node->nr_values) {
 		if (list_empty(&node->private_list)) {
Signature · 1 line
2.35.1