Skip to contentSkip to conversation

sched_ext: sync disable_irq_work in bpf_scx_unreg()

PATCH Superseded Richard Cheng 4 participants 6 messages Apr 22 – Apr 24

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 sched_ext: sync disable_irq_work in bpf_scx_unreg()

When unregistered my self-written scx scheduler, the following panic occurs [1].

The root cause is that the JIT page backing ops->quiescent() is freed before all callers of that function have stopped.

The expected ordering during teardown is:

    bitmap_zero(sch->has_op) + synchronize_rcu()
        -> guarantees no CPU will ever call sch->ops.* again
    -> only THEN free the BPF struct_ops JIT page

bpf_scx_unreg() is supposed to enforce the order, but after commit f4a6c506d118 ("sched_ext: Always bounce scx_disable() through irq_work"), disable_work is no longer queued directly, causing kthread_flush_work() to be a noop. Thus, the caller drops the struct_ops map too early and poisoned with AARCH64_BREAK_FAULT before disable_workfn ever execute.

So the subsequent dequeue_task() still sees SCX_HAS_OP(sch, quiescent) as true and calls ops.quiescent, which hit on the poisoned page and BRK panic.

Fix it by syncing disable_irq_work first, so disable_work is guaranteed to be queued before waiting for it.

Fixes f4a6c506d118 ("sched_ext: Always bounce scx_disable() through irq_work")

---
[1]:

[  188.572805] sched_ext: BPF scheduler "invariant_0.1.0_aarch64_unknown_linux_gnu_debug" enabled
[  229.923133] Kernel text patching generated an invalid instruction at 0xffff80009bc2c1f8!
[  229.923146] Internal error: Oops - BRK: 00000000f2000100 [#1]  SMP
[  230.077871] CPU: 48 UID: 0 PID: 1760 Comm: kworker/u583:7 Not tainted 7.0.0+ #3 PREEMPT(full)
[  230.086677] Hardware name: NVIDIA GB200 NVL/P3809-BMC, BIOS 02.05.12 20251107
[  230.093972] Workqueue: events_unbound bpf_map_free_deferred
[  230.099675] Sched_ext: invariant_0.1.0_aarch64_unknown_linux_gnu_debug (disabling), task: runnable_at=-174ms
[  230.116843] pc : 0xffff80009bc2c1f8
[  230.120406] lr : dequeue_task_scx+0x270/0x2d0
[  230.217749] Call trace:
[  230.228515]  0xffff80009bc2c1f8 (P)
[  230.232077]  dequeue_task+0x84/0x188
[  230.235728]  sched_change_begin+0x1dc/0x250
[  230.240000]  __set_cpus_allowed_ptr_locked+0x17c/0x240
[  230.245250]  __set_cpus_allowed_ptr+0x74/0xf0
[  230.249701]  ___migrate_enable+0x4c/0xa0
[  230.253707]  bpf_map_free_deferred+0x1a4/0x1b0
[  230.258246]  process_one_work+0x184/0x540
[  230.262342]  worker_thread+0x19c/0x348
[  230.266170]  kthread+0x13c/0x150
[  230.269465]  ret_from_fork+0x10/0x20
[  230.281393] Code: d4202000 d4202000 d4202000 d4202000 (d4202000)
[  230.287621] ---[ end trace 0000000000000000 ]---
[  231.160046] Kernel panic - not syncing: Oops - BRK: Fatal exception in interrupt

Best regards,
Richard Cheng.

---
 kernel/sched/ext.c | 6 ++++++
 1 file changed, 6 insertions(+)
kernel/sched/ext.c +6 -0
index 012ca8bd70fb..065660382a0c 100644
@@ -7349,6 +7349,12 @@ static void bpf_scx_unreg(void *kdata, struct bpf_link *link)
 	struct scx_sched *sch = rcu_dereference_protected(ops->priv, true);
 
 	scx_disable(sch, SCX_EXIT_UNREG);
+	/*
+	 * sch->disable_work might still not queued, causing kthread_flush_work()
+	 * as a noop. Syncing the irq_work first is required to guarantee the
+	 * kthread work has been queued before waiting for it.
+	 */
+	irq_work_sync(&sch->disable_irq_work);
 	kthread_flush_work(&sch->disable_work);
 	RCU_INIT_POINTER(ops->priv, NULL);
 	kobject_put(&sch->kobj);
Signature · 1 line
2.43.0
reviewer Reviewed-by on v1 1/1
#1 lore raw

Hi Richard,

Quoted 65 lines · Richard Cheng

When unregistered my self-written scx scheduler, the following panic occurs [1].

The root cause is that the JIT page backing ops->quiescent() is freed before all callers of that function have stopped.

The expected ordering during teardown is:

    bitmap_zero(sch->has_op) + synchronize_rcu()
        -> guarantees no CPU will ever call sch->ops.* again
    -> only THEN free the BPF struct_ops JIT page

bpf_scx_unreg() is supposed to enforce the order, but after commit f4a6c506d118 ("sched_ext: Always bounce scx_disable() through irq_work"), disable_work is no longer queued directly, causing kthread_flush_work() to be a noop. Thus, the caller drops the struct_ops map too early and poisoned with AARCH64_BREAK_FAULT before disable_workfn ever execute.

So the subsequent dequeue_task() still sees SCX_HAS_OP(sch, quiescent) as true and calls ops.quiescent, which hit on the poisoned page and BRK panic.

Fix it by syncing disable_irq_work first, so disable_work is guaranteed to be queued before waiting for it.

Fixes f4a6c506d118 ("sched_ext: Always bounce scx_disable() through irq_work")

---
[1]:

[  188.572805] sched_ext: BPF scheduler "invariant_0.1.0_aarch64_unknown_linux_gnu_debug" enabled
[  229.923133] Kernel text patching generated an invalid instruction at 0xffff80009bc2c1f8!
[  229.923146] Internal error: Oops - BRK: 00000000f2000100 [#1]  SMP
[  230.077871] CPU: 48 UID: 0 PID: 1760 Comm: kworker/u583:7 Not tainted 7.0.0+ #3 PREEMPT(full)
[  230.086677] Hardware name: NVIDIA GB200 NVL/P3809-BMC, BIOS 02.05.12 20251107
[  230.093972] Workqueue: events_unbound bpf_map_free_deferred
[  230.099675] Sched_ext: invariant_0.1.0_aarch64_unknown_linux_gnu_debug (disabling), task: runnable_at=-174ms
[  230.116843] pc : 0xffff80009bc2c1f8
[  230.120406] lr : dequeue_task_scx+0x270/0x2d0
[  230.217749] Call trace:
[  230.228515]  0xffff80009bc2c1f8 (P)
[  230.232077]  dequeue_task+0x84/0x188
[  230.235728]  sched_change_begin+0x1dc/0x250
[  230.240000]  __set_cpus_allowed_ptr_locked+0x17c/0x240
[  230.245250]  __set_cpus_allowed_ptr+0x74/0xf0
[  230.249701]  ___migrate_enable+0x4c/0xa0
[  230.253707]  bpf_map_free_deferred+0x1a4/0x1b0
[  230.258246]  process_one_work+0x184/0x540
[  230.262342]  worker_thread+0x19c/0x348
[  230.266170]  kthread+0x13c/0x150
[  230.269465]  ret_from_fork+0x10/0x20
[  230.281393] Code: d4202000 d4202000 d4202000 d4202000 (d4202000)
[  230.287621] ---[ end trace 0000000000000000 ]---
[  231.160046] Kernel panic - not syncing: Oops - BRK: Fatal exception in interrupt

Best regards,
Richard Cheng.

---
 kernel/sched/ext.c | 6 ++++++
 1 file changed, 6 insertions(+)
kernel/sched/ext.c +0 -0
index 012ca8bd70fb..065660382a0c 100644

kernel/sched/ext.c static void bpf_scx_unreg(void *kdata, struct bpf_link *link) · line 7354

Quoted 6 lines
kernel/sched/ext.c +3 -0
@@ -7349,6 +7349,12 @@ static void bpf_scx_unreg(void *kdata, struct bpf_link *link)
 	struct scx_sched *sch = rcu_dereference_protected(ops->priv, true);
 
 	scx_disable(sch, SCX_EXIT_UNREG);
+	/*
+	 * sch->disable_work might still not queued, causing kthread_flush_work()
+	 * as a noop. Syncing the irq_work first is required to guarantee the

nit, maybe rephrase: sch->disable_work might not have been queued yet, causing kthread_flush_work() to be a no-op.

Quoted 9 lines
kernel/sched/ext.c +3 -0
+	 * kthread work has been queued before waiting for it.
+	 */
+	irq_work_sync(&sch->disable_irq_work);
 	kthread_flush_work(&sch->disable_work);
 	RCU_INIT_POINTER(ops->priv, NULL);
 	kobject_put(&sch->kobj);
Signature · 1 line
2.43.0

I can't reproduce it locally, but from a logical perspective it makes sense. Nice catch!

Thanks,
-Andrea

Reviewed-by on v1 1/1
#2 lore raw
In reply to Richard Cheng

Hi Richard,

Quoted 2 lines · Richard Cheng

When unregistered my self-written scx scheduler, the following panic occurs [1].

Nit: you've placed the panic log [1] below the --- separator. Content below this line will not be preserved in the comment msg. Could you please move it into the commit msg body and send a v2 patch?

Quoted 20 lines

The root cause is that the JIT page backing ops->quiescent() is freed before all callers of that function have stopped.

The expected ordering during teardown is:

    bitmap_zero(sch->has_op) + synchronize_rcu()
        -> guarantees no CPU will ever call sch->ops.* again
    -> only THEN free the BPF struct_ops JIT page

bpf_scx_unreg() is supposed to enforce the order, but after commit f4a6c506d118 ("sched_ext: Always bounce scx_disable() through irq_work"), disable_work is no longer queued directly, causing kthread_flush_work() to be a noop. Thus, the caller drops the struct_ops map too early and poisoned with AARCH64_BREAK_FAULT before disable_workfn ever execute.

So the subsequent dequeue_task() still sees SCX_HAS_OP(sch, quiescent) as true and calls ops.quiescent, which hit on the poisoned page and BRK panic.

Quoted 5 lines

Fix it by syncing disable_irq_work first, so disable_work is guaranteed to be queued before waiting for it.

Fixes f4a6c506d118 ("sched_ext: Always bounce scx_disable() through irq_work")

Thanks for the fix, and the logic looks correct to me.

Also, scx_root_enable_workfn() has the same pattern in its error path:

	scx_error(sch, "scx_root_enable() failed (%d)", ret);
	kthread_flush_work(&sch->disable_work);

The comment above indicates that this flush is meant to "ensure that error is reported before init completion". However, because scx_error() goes through scx_vexit() -> irq_work_queue(), the flush can be a no-op here as well. The same applies to the sub-scheduler enable error path.

Should those be fixed in the same patch? Tejun, Andrea, wdyt? Thanks

[...]

kernel/sched/ext.c static void bpf_scx_unreg(void *kdata, struct bpf_link *link) · line 7354

Quoted 11 lines
kernel/sched/ext.c +3 -0
index 012ca8bd70fb..065660382a0c 100644
@@ -7349,6 +7349,12 @@ static void bpf_scx_unreg(void *kdata, struct bpf_link *link)
 	struct scx_sched *sch = rcu_dereference_protected(ops->priv, true);
 
 	scx_disable(sch, SCX_EXIT_UNREG);
+	/*
+	 * sch->disable_work might still not queued, causing kthread_flush_work()
+	 * as a noop. Syncing the irq_work first is required to guarantee the

Perhaps s/noop/no-op/? Though it's just a matter of taste. ^_^

Quoted 10 lines
kernel/sched/ext.c +3 -0
+	 * kthread work has been queued before waiting for it.
+	 */
+	irq_work_sync(&sch->disable_irq_work);
 	kthread_flush_work(&sch->disable_work);
 	RCU_INIT_POINTER(ops->priv, NULL);
 	kobject_put(&sch->kobj);
Signature · 1 line
2.43.0
Signature · 2 lines
Cheers,
Cheng-Yang
reviewer
#3 lore raw

Hello,

Quoted 4 lines · Cheng-Yang Chou

Also, scx_root_enable_workfn() has the same pattern in its error path:

	scx_error(sch, "scx_root_enable() failed (%d)", ret);
	kthread_flush_work(&sch->disable_work);

Yeah, can you please add a helper - e.g. flush_disable_work() - to package sync and flush and use that in both places?

Quoted 5 lines

kernel/sched/ext.c static void bpf_scx_unreg(void *kdata, struct bpf_link *link) · line 7354

Quoted 3 lines
kernel/sched/ext.c +3 -0
+	/*
+	 * sch->disable_work might still not queued, causing kthread_flush_work()
+	 * as a noop. Syncing the irq_work first is required to guarantee the

Perhaps s/noop/no-op/? Though it's just a matter of taste. ^_^

noop is used widely in the kernel. In general, I don't think we need this level of language policing.

Thanks.

Signature · 1 line
tejun
#4 lore raw
Quoted 10 lines · Tejun Heo

Hello,

Quoted 4 lines · Cheng-Yang Chou

Also, scx_root_enable_workfn() has the same pattern in its error path:

	scx_error(sch, "scx_root_enable() failed (%d)", ret);
	kthread_flush_work(&sch->disable_work);

Yeah, can you please add a helper - e.g. flush_disable_work() - to package sync and flush and use that in both places?

Hello Tejun,

No problem I'll add this part in v2.

Quoted 7 lines
Quoted 6 lines

kernel/sched/ext.c static void bpf_scx_unreg(void *kdata, struct bpf_link *link) · line 7354

Quoted 4 lines
kernel/sched/ext.c +3 -0
+	/*
+	 * sch->disable_work might still not queued, causing kthread_flush_work()
+	 * as a noop. Syncing the irq_work first is required to guarantee the

Perhaps s/noop/no-op/? Though it's just a matter of taste. ^_^

Quoted 7 lines

noop is used widely in the kernel. In general, I don't think we need this level of language policing.

Thanks.

Signature · 1 line
tejun

Thanks.

- Richard

#5 lore raw
In reply to Cheng-Yang Chou
Quoted 10 lines · Cheng-Yang Chou

Hi Richard,

Quoted 2 lines · Richard Cheng

When unregistered my self-written scx scheduler, the following panic occurs [1].

Nit: you've placed the panic log [1] below the --- separator. Content below this line will not be preserved in the comment msg. Could you please move it into the commit msg body and send a v2 patch?

Hi Cheng-Yang,

Sure thing, thanks.

Quoted 35 lines
Quoted 20 lines

The root cause is that the JIT page backing ops->quiescent() is freed before all callers of that function have stopped.

The expected ordering during teardown is:

    bitmap_zero(sch->has_op) + synchronize_rcu()
        -> guarantees no CPU will ever call sch->ops.* again
    -> only THEN free the BPF struct_ops JIT page

bpf_scx_unreg() is supposed to enforce the order, but after commit f4a6c506d118 ("sched_ext: Always bounce scx_disable() through irq_work"), disable_work is no longer queued directly, causing kthread_flush_work() to be a noop. Thus, the caller drops the struct_ops map too early and poisoned with AARCH64_BREAK_FAULT before disable_workfn ever execute.

So the subsequent dequeue_task() still sees SCX_HAS_OP(sch, quiescent) as true and calls ops.quiescent, which hit on the poisoned page and BRK panic.

Quoted 5 lines

Fix it by syncing disable_irq_work first, so disable_work is guaranteed to be queued before waiting for it.

Fixes f4a6c506d118 ("sched_ext: Always bounce scx_disable() through irq_work")

Thanks for the fix, and the logic looks correct to me.

Also, scx_root_enable_workfn() has the same pattern in its error path:

	scx_error(sch, "scx_root_enable() failed (%d)", ret);
	kthread_flush_work(&sch->disable_work);
Quoted 7 lines

The comment above indicates that this flush is meant to "ensure that error is reported before init completion". However, because scx_error() goes through scx_vexit() -> irq_work_queue(), the flush can be a no-op here as well. The same applies to the sub-scheduler enable error path.

Should those be fixed in the same patch? Tejun, Andrea, wdyt? Thanks

Sure I'll amend that part in v2.

Quoted 15 lines

[...]

kernel/sched/ext.c static void bpf_scx_unreg(void *kdata, struct bpf_link *link) · line 7354

Quoted 11 lines
kernel/sched/ext.c +3 -0
index 012ca8bd70fb..065660382a0c 100644
@@ -7349,6 +7349,12 @@ static void bpf_scx_unreg(void *kdata, struct bpf_link *link)
 	struct scx_sched *sch = rcu_dereference_protected(ops->priv, true);
 
 	scx_disable(sch, SCX_EXIT_UNREG);
+	/*
+	 * sch->disable_work might still not queued, causing kthread_flush_work()
+	 * as a noop. Syncing the irq_work first is required to guarantee the

Perhaps s/noop/no-op/? Though it's just a matter of taste. ^_^

Quoted 14 lines
Quoted 10 lines
kernel/sched/ext.c +3 -0
+	 * kthread work has been queued before waiting for it.
+	 */
+	irq_work_sync(&sch->disable_irq_work);
 	kthread_flush_work(&sch->disable_work);
 	RCU_INIT_POINTER(ops->priv, NULL);
 	kobject_put(&sch->kobj);
Signature · 1 line
2.43.0
Signature · 2 lines
Cheers,
Cheng-Yang

- Richard