mm, vmstat: kernel BUG at mm/vmstat.c:1408!
Subsystems (4)
Hi all,
I've started seeing the following in the latest -next kernel.
[ 531.127489] kernel BUG at mm/vmstat.c:1408!
[ 531.128157] invalid opcode: 0000 [#1] PREEMPT SMP KASAN
[ 531.128872] Modules linked in:
[ 531.129324] CPU: 6 PID: 407 Comm: kworker/6:1 Not tainted 4.4.0-rc5-next-20151218-sasha-00021-gaba8d84-dirty #2750
[ 531.130939] Workqueue: vmstat vmstat_update
[ 531.131741] task: ffff880204070000 ti: ffff880204078000 task.ti: ffff880204078000
[ 531.133189] RIP: vmstat_update (mm/vmstat.c:1408)
[ 531.134466] RSP: 0018:ffff88020407fbf8 EFLAGS: 00010293
[ 531.135132] RAX: 0000000000000006 RBX: ffff8800418e2fd8 RCX: 0000000000000000
[ 531.135995] RDX: 0000000000000007 RSI: ffffffff8c0982a0 RDI: ffffffff9b8bd6e4
[ 531.137475] RBP: ffff88020407fc18 R08: 0000000000000000 R09: ffff880204070230
[ 531.138304] R10: ffffffff8c0982a0 R11: 00000000e272b4f2 R12: ffff880204c1bf60
[ 531.139329] R13: ffff880204ab09c8 R14: ffff880204ab09b8 R15: ffff880204ab09b0
[ 531.140261] FS: 0000000000000000(0000) GS:ffff880204c00000(0000) knlGS:0000000000000000
[ 531.141218] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 531.142036] CR2: 00007f039a8c1944 CR3: 000000000ea28000 CR4: 00000000000006a0
[ 531.142752] Stack:
[ 531.142963] ffff880204c21000 ffff880204c21000 ffff880204c1bf60 ffff880204ab09c8
[ 531.144095] ffff88020407fd40 ffffffff813a8fea 0000000041b58ab3 ffffffff8e667cdb
[ 531.145258] ffff880204ab09f8 ffff880204c1bf68 ffff880204ab09c0 ffff880200000000
[ 531.146475] Call Trace:
[ 531.147037] process_one_work (./arch/x86/include/asm/preempt.h:22 kernel/workqueue.c:2045)
[ 531.150790] worker_thread (include/linux/compiler.h:218 include/linux/list.h:206 kernel/workqueue.c:2171)
[ 531.155176] kthread (kernel/kthread.c:209)
[ 531.158941] ret_from_fork (arch/x86/entry/entry_64.S:469)
[ 531.160654] Code: 75 1e be 79 00 00 00 48 c7 c7 80 0f 10 8c 89 45 e4 e8 cd 92 cd ff 8b 45 e4 c6 05 e1 c4 13 1a 01 89 c0 f0 48 0f ab 03 72 02 eb 0e <0f> 0b 48 c7 c7 c0 f1 47 90 e8 3d 03 ae 01 48 83 c4 08 5b 41 5c
All code
========
0: 75 1e jne 0x20
2: be 79 00 00 00 mov $0x79,%esi
7: 48 c7 c7 80 0f 10 8c mov $0xffffffff8c100f80,%rdi
e: 89 45 e4 mov %eax,-0x1c(%rbp)
11: e8 cd 92 cd ff callq 0xffffffffffcd92e3
16: 8b 45 e4 mov -0x1c(%rbp),%eax
19: c6 05 e1 c4 13 1a 01 movb $0x1,0x1a13c4e1(%rip) # 0x1a13c501
20: 89 c0 mov %eax,%eax
22: f0 48 0f ab 03 lock bts %rax,(%rbx)
27: 72 02 jb 0x2b
29: eb 0e jmp 0x39
2b:* 0f 0b ud2 <-- trapping instruction
2d: 48 c7 c7 c0 f1 47 90 mov $0xffffffff9047f1c0,%rdi
34: e8 3d 03 ae 01 callq 0x1ae0376
39: 48 83 c4 08 add $0x8,%rsp
3d: 5b pop %rbx
3e: 41 5c pop %r12
...Code starting with the faulting instruction
===========================================
0: 0f 0b ud2
2: 48 c7 c7 c0 f1 47 90 mov $0xffffffff9047f1c0,%rdi
9: e8 3d 03 ae 01 callq 0x1ae034b
e: 48 83 c4 08 add $0x8,%rsp
12: 5b pop %rbx
13: 41 5c pop %r12
...
[ 531.164630] RIP vmstat_update (mm/vmstat.c:1408)
[ 531.165523] RSP <ffff88020407fbf8>Thanks,
Sasha
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 1 line · Sasha Levin
[ 531.164630] RIP vmstat_update (mm/vmstat.c:1408)
Hmmm.. Yes we need to fold the diffs first before disabling the timer otherwise the shepherd task may intervene.
Does this patch fix it?
Subject: quiet_vmstat: Avoid race with shepherd by folding counters first
We need to fold the counters first otherwise the shepherd task may remotely reactivate the vmstat worker.
This also avoids the strange loop. Nothing can really increase the counters at that point since we are in the cpu idle loop. So folding the counters once is enough. Cancelling work that does not exist is fine too so just avoid the branches completely.
mm/vmstat.c +3 -5
@@ -1419,11 +1419,9 @@ void quiet_vmstat(void)if (system_state !=SYSTEM_RUNNING )return ; -do { -if (!cpumask_test_and_set_cpu (smp_processor_id (), cpu_stat_off)) -cancel_delayed_work (this_cpu_ptr (&vmstat_work)); - - }while (refresh_cpu_vm_stats (false)); +refresh_cpu_vm_stats (false); +cancel_delayed_work (this_cpu_ptr (&vmstat_work)); +cpumask_set_cpu (smp_processor_id (), cpu_stat_off); }/*
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 7 lines · Christoph Lameter
Quoted 1 line · Sasha Levin
Quoted 1 line
[ 531.164630] RIP vmstat_update (mm/vmstat.c:1408)Hmmm.. Yes we need to fold the diffs first before disabling the timer otherwise the shepherd task may intervene.
Does this patch fix it?
It didn't. With the patch I'm still seeing:
[ 1997.674112] kernel BUG at mm/vmstat.c:1408! [ 1997.674930] invalid opcode: 0000 [#1] PREEMPT SMP KASAN [ 1997.676252] Modules linked in: [ 1997.676880] CPU: 4 PID: 14713 Comm: kworker/4:0 Not tainted 4.4.0-rc5-next-20151221-sasha-00020-g840272e-dirty #2753 [ 1997.679262] Workqueue: vmstat vmstat_update [ 1997.680109] task: ffff88015bca0000 ti: ffff88015bcb8000 task.ti: ffff88015bcb8000 [ 1997.681279] RIP: 0010:[<ffffffffa2683c48>] [<ffffffffa2683c48>] vmstat_update+0x178/0x1a0 [ 1997.682810] RSP: 0018:ffff88015bcbfc00 EFLAGS: 00010297 [ 1997.683611] RAX: 0000000000000004 RBX: ffff8803d2801a18 RCX: 0000000000000000 [ 1997.684689] RDX: 0000000000000007 RSI: ffffffffad098220 RDI: ffffffffbc8be724 [ 1997.685750] RBP: ffff88015bcbfc20 R08: 0000000000000000 R09: ffff88015bca0230 [ 1997.686812] R10: ffffffffad098220 R11: ffff880180a1be78 R12: ffff880180a1be60 [ 1997.688087] R13: ffff880157b27908 R14: ffff880180a21000 R15: ffff880157b27900 [ 1997.689120] FS: 0000000000000000(0000) GS:ffff880180a00000(0000) knlGS:0000000000000000 [ 1997.690290] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1997.691141] CR2: 00007f05e5690000 CR3: 0000000158699000 CR4: 00000000000006a0 [ 1997.692189] Stack: [ 1997.692496] ffff880180a21000 ffff880157b27918 ffff880180a1be60 ffff880157b27908 [ 1997.693689] ffff88015bcbfd40 ffffffffa23aa40f ffffffffaf66686b ffff880157b27948 [ 1997.694851] ffff880180a1be68 ffff880100000000 ffff880157b27910 ffff880157b27920 [ 1997.696013] Call Trace: [ 1997.696542] [<ffffffffa23aa40f>] process_one_work+0xacf/0x12a0 [ 1997.697516] [<ffffffffa23a9940>] ? cancel_delayed_work_sync+0x10/0x10 [ 1997.698889] [<fffffffface0df27>] ? __schedule+0x1127/0x14c0 [ 1997.699738] [<ffffffffa23e11bd>] ? get_parent_ip+0xd/0x40 [ 1997.700547] [<ffffffffa23e12d9>] ? preempt_count_add+0xe9/0x140 [ 1997.701426] [<ffffffffa23ab898>] worker_thread+0xcb8/0x1090 [ 1997.702259] [<ffffffffa23d2ad0>] ? load_mm_ldt+0x1f0/0x1f0 [ 1997.703084] [<ffffffffa23d5b53>] ? update_rq_clock+0x123/0x2e0 [ 1997.703962] [<ffffffffa23aabe0>] ? process_one_work+0x12a0/0x12a0 [ 1997.704896] [<ffffffffa23aabe0>] ? process_one_work+0x12a0/0x12a0 [ 1997.705804] [<ffffffffa23bf8ce>] kthread+0x31e/0x330 [ 1997.706551] [<ffffffffa23d3195>] ? finish_task_switch+0x6c5/0x970 [ 1997.707481] [<ffffffffa23bf5b0>] ? kthread_worker_fn+0x680/0x680 [ 1997.708374] [<ffffffffa23bf5b0>] ? kthread_worker_fn+0x680/0x680 [ 1997.709269] [<fffffffface1a50f>] ret_from_fork+0x3f/0x70 [ 1997.710067] [<ffffffffa23bf5b0>] ? kthread_worker_fn+0x680/0x680 [ 1997.710964] Code: 75 1e be 79 00 00 00 48 c7 c7 40 16 10 ad 89 45 e4 e8 2d 88 cd ff 8b 45 e4 c6 05 a0 a9 13 1a 01 89 c0 f0 48 0f ab 03 72 02 eb 0e <0f> 0b 48 c7 c7 c0 21 48 b1 e8 45 d6 ad 01 48 83 c4 08 5b 41 5c [ 1997.714961] RIP [<ffffffffa2683c48>] vmstat_update+0x178/0x1a0 [ 1997.715852] RSP <ffff88015bcbfc00>
Thanks,
Sasha
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 9 lines · Sasha Levin
Quoted 7 lines · Christoph Lameter
Quoted 7 lines · Sasha Levin
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
[ 531.164630] RIP vmstat_update (mm/vmstat.c:1408)Hmmm.. Yes we need to fold the diffs first before disabling the timer otherwise the shepherd task may intervene.
Does this patch fix it?
It didn't. With the patch I'm still seeing:
I've also noticed a new warning from the workqueue code which my scripts didn't pick up before:
[ 3462.380681] BUG: workqueue lockup - pool cpus=2 node=2 flags=0x4 nice=0 stuck for 54s! [ 3462.522041] workqueue vmstat: flags=0xc [ 3462.527795] pwq 4: cpus=2 node=2 flags=0x0 nice=0 active=1/256 [ 3462.554836] pending: vmstat_update
Thanks,
Sasha
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 7 lines · Sasha Levin
I've also noticed a new warning from the workqueue code which my scripts didn't pick up before:
[ 3462.380681] BUG: workqueue lockup - pool cpus=2 node=2 flags=0x4 nice=0 stuck for 54s! [ 3462.522041] workqueue vmstat: flags=0xc [ 3462.527795] pwq 4: cpus=2 node=2 flags=0x0 nice=0 active=1/256 [ 3462.554836] pending: vmstat_update
Does that mean that vmstat_update locks up or something that schedules it?
Also what workload triggers the BUG()?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Ran this here but no errors. Need config etc to reproduce.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 10 lines · Christoph Lameter
Quoted 7 lines · Sasha Levin
Quoted 7 lines
I've also noticed a new warning from the workqueue code which my scripts didn't pick up before:
[ 3462.380681] BUG: workqueue lockup - pool cpus=2 node=2 flags=0x4 nice=0 stuck for 54s! [ 3462.522041] workqueue vmstat: flags=0xc [ 3462.527795] pwq 4: cpus=2 node=2 flags=0x0 nice=0 active=1/256 [ 3462.554836] pending: vmstat_updateDoes that mean that
vmstat_updatelocks up or something that schedules it?
I think it means that vmstat_update didn't finish running (working).
Quoted 1 line
Also what workload triggers the
BUG()?
Fuzzing with trinity inside a KVM guest. I've attached my config.
Thanks,
Sasha
Quoted 5 lines · Sasha Levin
Quoted 3 lines
Quoted 2 lines
Quoted 2 lines
[ 3462.527795] pwq 4: cpus=2 node=2 flags=0x0 nice=0 active=1/256 [ 3462.554836] pending: vmstat_updateDoes that mean that
vmstat_updatelocks up or something that schedules it?I think it means that
vmstat_updatedidn't finish running (working).
There is nothing in there that blocks.
Quoted 3 lines
Quoted 1 line
Also what workload triggers the
BUG()?Fuzzing with trinity inside a KVM guest. I've attached my config.
Ok will have a look.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Builds with your kernel config fail with
CHK include/generated/bounds.h
CHK include/generated/asm-offsets.h CALL scripts/checksyscalls.sh CHK include/generated/compile.h
make[1]: *** No rule to make target 'signing_key.pem', needed by 'certs/signing_key.x509'. Stop.
Makefile:967: recipe for target 'certs' failed
make: *** [certs] Error 2
make: *** Waiting for unfinished jobs....
CHK kernel/config_data.h
???
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 3 lines · Sasha Levin
Quoted 1 line
Also what workload triggers the
BUG()?Fuzzing with trinity inside a KVM guest. I've attached my config.
Ok build and bootup works fine after fix from Tetsuo to config. Does not like my initrd it seems. Is there a root with the tools available somehow?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 8 lines · Christoph Lameter
Quoted 3 lines · Sasha Levin
Quoted 1 line
Also what workload triggers the
BUG()?Fuzzing with trinity inside a KVM guest. I've attached my config.
Ok build and bootup works fine after fix from Tetsuo to config. Does not like my initrd it seems. Is there a root with the tools available somehow?
Will is hosting a stand-alone version here: git://git.kernel.org/pub/scm/linux/kernel/git/will/kvmtool.git
Thanks,
Sasha
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 6 lines · Christoph Lameter
Index:
linux/mm/vmstat.c
===================================================================mm/vmstat.c +0
-0@@ -1419,11 +1419,9 @@ void quiet_vmstat(void)if (system_state !=SYSTEM_RUNNING )
mm/vmstat.c void quiet_vmstat(void) · line 1424
Quoted 10 lines
mm/vmstat.c +2
-5return ; -do { -if (!cpumask_test_and_set_cpu (smp_processor_id (), cpu_stat_off)) -cancel_delayed_work (this_cpu_ptr (&vmstat_work)); - - }while (refresh_cpu_vm_stats (false)); +refresh_cpu_vm_stats (false); +cancel_delayed_work (this_cpu_ptr (&vmstat_work));
shouldn't this be cancel_delayed_work_sync ?
mm/vmstat.c void quiet_vmstat(void) · line 1426
Quoted 2 lines
mm/vmstat.c +1
-0+cpumask_set_cpu (smp_processor_id (), cpu_stat_off);
else ongoing vmstat_update can again encounter cpu_stat_off set ?
Quoted 2 lines
mm/vmstat.c +0
-0}
Signature · 8 lines
regards Shiraz Hashim -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 5 lines · Shiraz Hashim
Review
Quoted 3 lines
(quoted diff) +2
-0+ refresh_cpu_vm_stats(false); + cancel_delayed_work(this_cpu_ptr(&vmstat_work));shouldn't this be
cancel_delayed_work_sync?
Hmmm... This is executed with preemption off and the work is on the same cpu. If it would be able to run concurrently then we would need this.
Ok but it could run from the timer interrupt if that is still on and occuring shortly before we go idle. Guess this needs to be similar to the code we execute on cpu down in the vmstat notifiers (see vmstat_cpuup_callback).
Does this fix it? I have not been able to reproduce the issue so far.
Patch against -next.
Subject: vmstat: Use delayed work_sync and avoid loop.
mm/vmstat.c +3 -5
@@ -1419,11 +1419,9 @@ void quiet_vmstat(void)if (system_state !=SYSTEM_RUNNING )return ; -do { -if (!cpumask_test_and_set_cpu (smp_processor_id (), cpu_stat_off)) -cancel_delayed_work (this_cpu_ptr (&vmstat_work)); - - }while (refresh_cpu_vm_stats (false)); +refresh_cpu_vm_stats (false); +cancel_delayed_work_sync (this_cpu_ptr (&vmstat_work)); +cpumask_set_cpu (smp_processor_id (), cpu_stat_off); }/*
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 1 line · Christoph Lameter
Does this fix it? I have not been able to reproduce the issue so far.
I too am not able to reproduce. Was just thinking what else can go wrong in Sasha's setup.
Signature · 8 lines
regards Shiraz Hashim -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Hi Sasha,
Quoted 35 lines · Christoph Lameter
Quoted 5 lines · Shiraz Hashim
Review
Quoted 3 lines
(quoted diff) +2
-0+ refresh_cpu_vm_stats(false); + cancel_delayed_work(this_cpu_ptr(&vmstat_work));shouldn't this be
cancel_delayed_work_sync?Hmmm... This is executed with preemption off and the work is on the same cpu. If it would be able to run concurrently then we would need this.
Ok but it could run from the timer interrupt if that is still on and occuring shortly before we go idle. Guess this needs to be similar to the code we execute on cpu down in the vmstat notifiers (see
vmstat_cpuup_callback).Does this fix it? I have not been able to reproduce the issue so far.
Patch against -next.
Subject: vmstat: Use delayed
work_syncand avoid loop.Signed-off-by Christoph Lameter <[email protected]>Index:
linux/mm/vmstat.c
===================================================================mm/vmstat.c +0
-1@@ -1419,11 +1419,9 @@ void quiet_vmstat(void)if (system_state !=SYSTEM_RUNNING )return ; - do {
mm/vmstat.c void quiet_vmstat(void) · line 1427
Quoted 10 lines
mm/vmstat.c +3
-4-if (!cpumask_test_and_set_cpu (smp_processor_id (), cpu_stat_off)) -cancel_delayed_work (this_cpu_ptr (&vmstat_work)); - - }while (refresh_cpu_vm_stats (false)); +refresh_cpu_vm_stats (false); +cancel_delayed_work_sync (this_cpu_ptr (&vmstat_work)); +cpumask_set_cpu (smp_processor_id (), cpu_stat_off); }/*
Can you please give it a try, seems it is reproducing easily at your end.
Signature · 8 lines
regards Shiraz Hashim -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 3 lines · Shiraz Hashim
Can you please give it a try, seems it is reproducing easily at your end.
Ahh!, it cannot be done as it is called with preemption disabled.
[ 8.131226] Process swapper/4 (pid: 0, stack limit = 0xffffffc06fc18058) [ 8.137908] Call trace: [ 8.140344] [<ffffffc0000c19cc>] __might_sleep+0x15c/0x16c [ 8.145812] [<ffffffc0000b3d64>] flush_work+0x3c/0x190 [ 8.150933] [<ffffffc0000b4450>] __cancel_work_timer+0x138/0x1e4 [ 8.156922] [<ffffffc0000b45a8>] cancel_delayed_work_sync+0xc/0x18 [ 8.163087] [<ffffffc0001782dc>] quiet_vmstat+0x3c/0x60 [ 8.168294] [<ffffffc0000dd2f0>] cpu_startup_entry+0x2c/0x330 [ 8.174024] [<ffffffc000090b24>] secondary_start_kernel+0x124/0x134
Signature · 8 lines
regards Shiraz Hashim -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 47 lines · Shiraz Hashim
Hi Sasha,
Quoted 44 lines · Christoph Lameter
Quoted 5 lines · Shiraz Hashim
Review
Quoted 3 lines
(quoted diff) +2
-0+ refresh_cpu_vm_stats(false); + cancel_delayed_work(this_cpu_ptr(&vmstat_work));shouldn't this be
cancel_delayed_work_sync?Hmmm... This is executed with preemption off and the work is on the same cpu. If it would be able to run concurrently then we would need this.
Ok but it could run from the timer interrupt if that is still on and occuring shortly before we go idle. Guess this needs to be similar to the code we execute on cpu down in the vmstat notifiers (see
vmstat_cpuup_callback).Does this fix it? I have not been able to reproduce the issue so far.
Patch against -next.
Subject: vmstat: Use delayed
work_syncand avoid loop.Signed-off-by Christoph Lameter <[email protected]>Index:
linux/mm/vmstat.c
===================================================================mm/vmstat.c +3
-5@@ -1419,11 +1419,9 @@ void quiet_vmstat(void)if (system_state !=SYSTEM_RUNNING )return ; -do { -if (!cpumask_test_and_set_cpu (smp_processor_id (), cpu_stat_off)) -cancel_delayed_work (this_cpu_ptr (&vmstat_work)); - - }while (refresh_cpu_vm_stats (false)); +refresh_cpu_vm_stats (false); +cancel_delayed_work_sync (this_cpu_ptr (&vmstat_work)); +cpumask_set_cpu (smp_processor_id (), cpu_stat_off); }
Quoted 4 lines
mm/vmstat.c void quiet_vmstat(void) · line 1427
Quoted 1 line
mm/vmstat.c +0
-0/* Can you please give it a try, seems it is reproducing easily at your end.
I'm seeing:
[3637853.902081] BUG: sleeping function called from invalid context at kernel/workqueue.c:2725 [3637853.904291] in_atomic(): 1, irqs_disabled(): 0, pid: 0, name: swapper/0 [3637853.905488] no locks held by swapper/0/0. [3637853.906387] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.4.0-next-20160114-sasha-00021-gf1273d1-dirty #2797 [3637853.908369] 1ffffffff5dc0f42 0f76d753b1b5b625 ffffffffaee07a90 ffffffffa2433eee [3637853.909561] 0000000041b58ab3 ffffffffae960c30 ffffffffa2433e26 ffffffffaee49bc0 [3637853.911601] 0000000000000000 0000000000000000 0000000000000000 ffffffffaee07a78 [3637853.913281] Call Trace: [3637853.913775] [<ffffffffa2433eee>] dump_stack+0xc8/0x12a [3637853.914649] [<ffffffffa2433e26>] ? _atomic_dec_and_lock+0x106/0x106 [3637853.915824] [<ffffffffa0440125>] ___might_sleep+0x2c5/0x480 [3637853.917056] [<ffffffffa044033b>] __might_sleep+0x5b/0x260 [3637853.918310] [<ffffffffa040a890>] flush_work+0xe0/0x820 [3637853.919270] [<ffffffffa040a7b0>] ? __queue_delayed_work+0x460/0x460 [3637853.921561] [<ffffffffa040a7b0>] ? __queue_delayed_work+0x460/0x460 [3637853.922898] [<ffffffffa054d367>] ? del_timer+0x107/0x170 [3637853.924395] [<ffffffffa054d260>] ? lock_timer_base+0x220/0x220 [3637853.925827] [<ffffffffa0411445>] ? try_to_grab_pending+0x115/0x680 [3637853.926924] [<ffffffffa0411b87>] ? __cancel_work_timer+0x1d7/0x5f0 [3637853.928215] [<ffffffffa04d4b58>] ? mark_held_locks+0x1a8/0x220 [3637853.929075] [<ffffffffa0411bdb>] __cancel_work_timer+0x22b/0x5f0 [3637853.930765] [<ffffffffa04119b0>] ? try_to_grab_pending+0x680/0x680 [3637853.931642] [<ffffffffa075fb60>] ? fill_contig_page_info+0x2e0/0x2e0 [3637853.932774] [<ffffffffa0411fda>] cancel_delayed_work_sync+0x1a/0x20 [3637853.933897] [<ffffffffa0766754>] quiet_vmstat+0x74/0x140 [3637853.934991] [<ffffffffa04c20eb>] cpu_startup_entry+0x12b/0x6d0 [3637853.936174] [<ffffffffa04c1fc0>] ? call_cpuidle+0x160/0x160 [3637853.937420] [<ffffffffabfe3816>] rest_init+0x1d6/0x1e0 [3637853.938799] [<ffffffffbb7e4c2a>] start_kernel+0x66c/0x6a6 [3637853.940742] [<ffffffffbb7e45be>] ? thread_info_cache_init+0xb/0xb [3637853.942416] [<ffffffffbb9b0ae7>] ? memblock_reserve+0x59/0x5e [3637853.943880] [<ffffffffbb7e3120>] ? early_idt_handler_array+0x120/0x120 [3637853.945161] [<ffffffffbb7e33c4>] x86_64_start_reservations+0x2a/0x2c [3637853.946005] [<ffffffffbb7e351d>] x86_64_start_kernel+0x157/0x17a
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
[CCing Andrew]
I am just reading through this old discussion again because "vmstat:
make vmstat_updater deferrable again and shut down on idle" which seems to be the culprit AFAIU has been merged as 0eb77e988032 and I do not see any follow up fix merged to linus tree
Quoted 66 lines · Sasha Levin
Hi all,
I've started seeing the following in the latest -next kernel.
[ 531.127489] kernel BUG at mm/vmstat.c:1408! [ 531.128157] invalid opcode: 0000 [#1] PREEMPT SMP KASAN [ 531.128872] Modules linked in: [ 531.129324] CPU: 6 PID: 407 Comm: kworker/6:1 Not tainted 4.4.0-rc5-next-20151218-sasha-00021-gaba8d84-dirty #2750 [ 531.130939] Workqueue: vmstat vmstat_update [ 531.131741] task: ffff880204070000 ti: ffff880204078000 task.ti: ffff880204078000 [ 531.133189] RIP: vmstat_update (mm/vmstat.c:1408) [ 531.134466] RSP: 0018:ffff88020407fbf8 EFLAGS: 00010293 [ 531.135132] RAX: 0000000000000006 RBX: ffff8800418e2fd8 RCX: 0000000000000000 [ 531.135995] RDX: 0000000000000007 RSI: ffffffff8c0982a0 RDI: ffffffff9b8bd6e4 [ 531.137475] RBP: ffff88020407fc18 R08: 0000000000000000 R09: ffff880204070230 [ 531.138304] R10: ffffffff8c0982a0 R11: 00000000e272b4f2 R12: ffff880204c1bf60 [ 531.139329] R13: ffff880204ab09c8 R14: ffff880204ab09b8 R15: ffff880204ab09b0 [ 531.140261] FS: 0000000000000000(0000) GS:ffff880204c00000(0000) knlGS:0000000000000000 [ 531.141218] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 531.142036] CR2: 00007f039a8c1944 CR3: 000000000ea28000 CR4: 00000000000006a0 [ 531.142752] Stack: [ 531.142963] ffff880204c21000 ffff880204c21000 ffff880204c1bf60 ffff880204ab09c8 [ 531.144095] ffff88020407fd40 ffffffff813a8fea 0000000041b58ab3 ffffffff8e667cdb [ 531.145258] ffff880204ab09f8 ffff880204c1bf68 ffff880204ab09c0 ffff880200000000 [ 531.146475] Call Trace: [ 531.147037] process_one_work (./arch/x86/include/asm/preempt.h:22 kernel/workqueue.c:2045) [ 531.150790] worker_thread (include/linux/compiler.h:218 include/linux/list.h:206 kernel/workqueue.c:2171) [ 531.155176] kthread (kernel/kthread.c:209) [ 531.158941] ret_from_fork (arch/x86/entry/entry_64.S:469) [ 531.160654] Code: 75 1e be 79 00 00 00 48 c7 c7 80 0f 10 8c 89 45 e4 e8 cd 92 cd ff 8b 45 e4 c6 05 e1 c4 13 1a 01 89 c0 f0 48 0f ab 03 72 02 eb 0e <0f> 0b 48 c7 c7 c0 f1 47 90 e8 3d 03 ae 01 48 83 c4 08 5b 41 5c All code ======== 0: 75 1e jne 0x20 2: be 79 00 00 00 mov $0x79,%esi 7: 48 c7 c7 80 0f 10 8c mov $0xffffffff8c100f80,%rdi e: 89 45 e4 mov %eax,-0x1c(%rbp) 11: e8 cd 92 cd ff callq 0xffffffffffcd92e3 16: 8b 45 e4 mov -0x1c(%rbp),%eax 19: c6 05 e1 c4 13 1a 01 movb $0x1,0x1a13c4e1(%rip) # 0x1a13c501 20: 89 c0 mov %eax,%eax 22: f0 48 0f ab 03 lock bts %rax,(%rbx) 27: 72 02 jb 0x2b 29: eb 0e jmp 0x39 2b:* 0f 0b ud2 <-- trapping instruction 2d: 48 c7 c7 c0 f1 47 90 mov $0xffffffff9047f1c0,%rdi 34: e8 3d 03 ae 01 callq 0x1ae0376 39: 48 83 c4 08 add $0x8,%rsp 3d: 5b pop %rbx 3e: 41 5c pop %r12 ...Code starting with the faulting instruction
===========================================0: 0f 0b ud2 2: 48 c7 c7 c0 f1 47 90 mov $0xffffffff9047f1c0,%rdi 9: e8 3d 03 ae 01 callq 0x1ae034b e: 48 83 c4 08 add $0x8,%rsp 12: 5b pop %rbx 13: 41 5c pop %r12 ... [ 531.164630] RIP vmstat_update (mm/vmstat.c:1408) [ 531.165523] RSP <ffff88020407fbf8>Thanks,
Sasha
Signature · 8 lines
Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 4 lines · Michal Hocko
I am just reading through this old discussion again because "vmstat:
makevmstat_updaterdeferrable again and shut down on idle" which seems to be the culprit AFAIU has been merged as0eb77e988032and I do not see any follow up fix merged to linus tree
So this isn't an "old" discussion - the bug is very much there and I can hit it easily. As a workaround I've "disabled" vmstat.
Thanks,
Sasha
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 8 lines · Sasha Levin
Quoted 4 lines · Michal Hocko
I am just reading through this old discussion again because "vmstat:
makevmstat_updaterdeferrable again and shut down on idle" which seems to be the culprit AFAIU has been merged as0eb77e988032and I do not see any follow up fix merged to linus treeSo this isn't an "old" discussion - the bug is very much there and I can hit it easily. As a workaround I've "disabled" vmstat.
Well the report is since 18th Dec which is over month old. Should we revert 0eb77e988032 as a pre caution and make sure this is done properly in -mm tree. AFAIR none of the proposed fix worked without other fallouts?
Signature · 8 lines
Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 6 lines · Michal Hocko
[CCing Andrew]
I am just reading through this old discussion again because "vmstat:
makevmstat_updaterdeferrable again and shut down on idle" which seems to be the culprit AFAIU has been merged as0eb77e988032and I do not see any follow up fix merged to linus tree
Is there any way to reproce this issue? This is running through trinity right? Can we please get the exact syscall that causes this to occur?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 11 lines · Christoph Lameter
Quoted 6 lines · Michal Hocko
[CCing Andrew]
I am just reading through this old discussion again because "vmstat:
makevmstat_updaterdeferrable again and shut down on idle" which seems to be the culprit AFAIU has been merged as0eb77e988032and I do not see any follow up fix merged to linus treeIs there any way to reproce this issue? This is running through trinity right? Can we please get the exact syscall that causes this to occur?
As per the backtrace in the initial report this seems to be time dependent as the crash happens from _kthread_ context. So it doesn't seem to be directly related to any particular syscall.
Signature · 8 lines
Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 7 lines · Michal Hocko
Quoted 6 lines · Sasha Levin
Quoted 4 lines · Michal Hocko
I am just reading through this old discussion again because "vmstat:
makevmstat_updaterdeferrable again and shut down on idle" which seems to be the culprit AFAIU has been merged as0eb77e988032and I do not see any follow up fix merged to linus tree
Quoted 7 lines
Quoted 2 lines
So this isn't an "old" discussion - the bug is very much there and I can hit it easily. As a workaround I've "disabled" vmstat.
Well the report is since 18th Dec which is over month old. Should we revert
0eb77e988032as a pre caution and make sure this is done properly in -mm tree. AFAIR none of the proposed fix worked without other fallouts?
Seems that we are unable to get enough information to reproduce the issue?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 9 lines · Christoph Lameter
Quoted 7 lines · Michal Hocko
Quoted 7 lines · Sasha Levin
Quoted 6 lines
Quoted 6 lines · Michal Hocko
Quoted 4 lines
Quoted 4 lines
I am just reading through this old discussion again because "vmstat:
makevmstat_updaterdeferrable again and shut down on idle" which seems to be the culprit AFAIU has been merged as0eb77e988032and I do not see any follow up fix merged to linus tree
Quoted 8 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
So this isn't an "old" discussion - the bug is very much there and I can hit it easily. As a workaround I've "disabled" vmstat.
Well the report is since 18th Dec which is over month old. Should we revert
0eb77e988032as a pre caution and make sure this is done properly in -mm tree. AFAIR none of the proposed fix worked without other fallouts?Seems that we are unable to get enough information to reproduce the issue?
As I've mentioned - this reproduces frequently. I'd be happy to add in debug information into the kernel that might help you reproduce it, but as it seems like a timing issue, I can't provide a simple reproducer.
Thanks,
Sasha
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 4 lines · Sasha Levin
As I've mentioned - this reproduces frequently. I'd be happy to add in debug information into the kernel that might help you reproduce it, but as it seems like a timing issue, I can't provide a simple reproducer.
This isnt really important I think. Lets remove it.
Subject: vmstat: Remove BUG_ON from vmstat_update
If we detect that there is nothing to do just set the flag and do not check if it was already set before. Races really do not matter. If the flag is set by any code then the shepherd will start dealing with the situation and reenable the vmstat workers when necessary again.
Concurrent actions could be onlining and offlining of processors or be a result of concurrency issues when updating the cpumask from multiple processors.
mm/vmstat.c +1 -11
@@ -1408,17 +1408,7 @@ static void vmstat_update(struct work_st* Defer the checking for differentials to the * shepherd thread on a different processor. */ -int r; -/* -* Shepherd work thread does not race since it never -* changes the bit if its zero but the cpu -* online / off line code may race if -* worker threads are still allowed during -* shutdown / startup. -*/ - r =cpumask_test_and_set_cpu (smp_processor_id (), - cpu_stat_off); -VM_BUG_ON (r); +cpumask_set_cpu (smp_processor_id (), cpu_stat_off); } }
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
On Wed 20-01-16 09:55:22, Christoph Lameter wrote:
[...]
Quoted 10 lines
Subject: vmstat: Remove
BUG_ONfromvmstat_updateIf we detect that there is nothing to do just set the flag and do not check if it was already set before. Races really do not matter. If the flag is set by any code then the shepherd will start dealing with the situation and reenable the vmstat workers when necessary again.
Concurrent actions could be onlining and offlining of processors or be a result of concurrency issues when updating the cpumask from multiple processors.
Now that 7e988032 ("vmstat: make vmstat_updater deferrable again and shut down on idle) is merged the VM_BUG_ON is simply bogus because vmstat_update might "race" with quiet_vmstat. The changelog should reflect that. What about the following wording?
"
Since 0eb77e988032 ("vmstat: make vmstat_updater deferrable again and shut down on idle") quiet_vmstat might update cpu_stat_off and mark a particular cpu to be handled by vmstat_shepherd. This might trigger a VM_BUG_ON in vmstat_update because the work item might have been sleeping during the idle period and see the cpu_stat_off updated after the wake up. The VM_BUG_ON is therefore misleading and no more appropriate. Moreover it doesn't really suite any protection from real bugs because vmstat_shepherd will simply reschedule the vmstat_work anytime it sees a particular cpu set or vmstat_update would do the same from the worker context directly. Even when the two would race the result wouldn't be incorrect as the counters update is fully idempotent.
shut down on idle")
CC: stable # 4.4+
"
Quoted 24 lines
Signed-off-by Christoph Lameter <[email protected]>Index:
linux/mm/vmstat.c
===================================================================mm/vmstat.c +1
-11@@ -1408,17 +1408,7 @@ static void vmstat_update(struct work_st* Defer the checking for differentials to the * shepherd thread on a different processor. */ -int r; -/* -* Shepherd work thread does not race since it never -* changes the bit if its zero but the cpu -* online / off line code may race if -* worker threads are still allowed during -* shutdown / startup. -*/ - r =cpumask_test_and_set_cpu (smp_processor_id (), - cpu_stat_off); -VM_BUG_ON (r); +cpumask_set_cpu (smp_processor_id (), cpu_stat_off); } }
Signature · 8 lines
Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 9 lines · Michal Hocko
On Wed 20-01-16 09:55:22, Christoph Lameter wrote:
[...]Quoted 7 lines
Subject: vmstat: Remove
BUG_ONfromvmstat_updateIf we detect that there is nothing to do just set the flag and do not check if it was already set before. Races really do not matter. If the flag is set by any code then the shepherd will start dealing with the situation and reenable the vmstat workers when necessary again.
Quoted 8 lines
Quoted 3 lines
Concurrent actions could be onlining and offlining of processors or be a result of concurrency issues when updating the cpumask from multiple processors.
Now that
7e988032("vmstat: makevmstat_updaterdeferrable again and shut down on idle) is merged theVM_BUG_ONis simply bogus becausevmstat_updatemight "race" withquiet_vmstat. The changelog should reflect that. What about the following wording?
How can it race if preemption is off?
Quoted 11 lines
Since
0eb77e988032("vmstat: makevmstat_updaterdeferrable again and shut down on idle")quiet_vmstatmight updatecpu_stat_offand mark a particular cpu to be handled byvmstat_shepherd. This might trigger aVM_BUG_ONinvmstat_updatebecause the work item might have been sleeping during the idle period and see thecpu_stat_offupdated after the wake up. TheVM_BUG_ONis therefore misleading and no more appropriate. Moreover it doesn't really suite any protection from real bugs becausevmstat_shepherdwill simply reschedule thevmstat_workanytime it sees a particular cpu set orvmstat_updatewould do the same from the worker context directly. Even when the two would race the result wouldn't be incorrect as the counters update is fully idempotent.
Hmmm... the vmstat_update can be interrupted while running and the cpu put into idle mode? If vmstat_update is running then the cpu is not idle but running code. If this is really going on then there is other stuff wrong with the idling logic.
Quoted 3 lines
Fixes 0eb77e988032 ("vmstat: make vmstat_updater deferrable again andshut down on idle")
CC: stable # 4.4+
?? There has not been an upstream release with this yet.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 15 lines · Christoph Lameter
Quoted 13 lines · Michal Hocko
On Wed 20-01-16 09:55:22, Christoph Lameter wrote:
[...]Quoted 10 lines
Subject: vmstat: Remove
BUG_ONfromvmstat_updateIf we detect that there is nothing to do just set the flag and do not check if it was already set before. Races really do not matter. If the flag is set by any code then the shepherd will start dealing with the situation and reenable the vmstat workers when necessary again.
Concurrent actions could be onlining and offlining of processors or be a result of concurrency issues when updating the cpumask from multiple processors.
Quoted 6 lines
Quoted 4 lines
Now that
7e988032("vmstat: makevmstat_updaterdeferrable again and shut down on idle) is merged theVM_BUG_ONis simply bogus becausevmstat_updatemight "race" withquiet_vmstat. The changelog should reflect that. What about the following wording?How can it race if preemption is off?
See below.
Quoted 13 lines
Quoted 11 lines
Since
0eb77e988032("vmstat: makevmstat_updaterdeferrable again and shut down on idle")quiet_vmstatmight updatecpu_stat_offand mark a particular cpu to be handled byvmstat_shepherd. This might trigger aVM_BUG_ONinvmstat_updatebecause the work item might have been sleeping during the idle period and see thecpu_stat_offupdated after the wake up. TheVM_BUG_ONis therefore misleading and no more appropriate. Moreover it doesn't really suite any protection from real bugs becausevmstat_shepherdwill simply reschedule thevmstat_workanytime it sees a particular cpu set orvmstat_updatewould do the same from the worker context directly. Even when the two would race the result wouldn't be incorrect as the counters update is fully idempotent.
Quoted 4 lines
Hmmm... the
vmstat_updatecan be interrupted while running and the cpu put into idle mode? Ifvmstat_updateis running then the cpu is not idle but running code. If this is really going on then there is other stuff wrong with the idling logic.
The vmstat update might be still waiting for its timer, idle mode started and kick vmstat_update which might cpumask_test_and_set_cpu. Once the idle terminates and the originally schedule vmstate_update executes it sees the bit set and BUG_ON.
Quoted 5 lines
Quoted 3 lines
Fixes 0eb77e988032 ("vmstat: make vmstat_updater deferrable again andshut down on idle")
CC: stable # 4.4+?? There has not been an upstream release with this yet.
Ohh, I thought it made it into 4.4 but you are right it is post 4.4 so no CC: stable required.
Signature · 8 lines
Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 18 lines · Michal Hocko
Quoted 13 lines
Quoted 11 lines
Since
0eb77e988032("vmstat: makevmstat_updaterdeferrable again and shut down on idle")quiet_vmstatmight updatecpu_stat_offand mark a particular cpu to be handled byvmstat_shepherd. This might trigger aVM_BUG_ONinvmstat_updatebecause the work item might have been sleeping during the idle period and see thecpu_stat_offupdated after the wake up. TheVM_BUG_ONis therefore misleading and no more appropriate. Moreover it doesn't really suite any protection from real bugs becausevmstat_shepherdwill simply reschedule thevmstat_workanytime it sees a particular cpu set orvmstat_updatewould do the same from the worker context directly. Even when the two would race the result wouldn't be incorrect as the counters update is fully idempotent.Quoted 4 lines
Hmmm... the
vmstat_updatecan be interrupted while running and the cpu put into idle mode? Ifvmstat_updateis running then the cpu is not idle but running code. If this is really going on then there is other stuff wrong with the idling logic.
Quoted 4 lines
The vmstat update might be still waiting for its timer, idle mode started and kick
vmstat_updatewhich mightcpumask_test_and_set_cpu. Once the idle terminates and the originally schedulevmstate_updateexecutes it sees the bit set andBUG_ON.
Ok so we are going into idle mode and the vmstat_update timer is pending. Then the timer will not fire since going idle switches preemption off. quiet_vmstat will run without the chance of running vmstat_update
We could be going idle and not have disabled preemption yet. Then vmstat_update will run. On return to the idling operation preemption will be disabled and quiet_vmstat() will be run.
I do not see how these two things could race.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 1 line · Christoph Lameter
On Thu, 21 Jan 2016, Michal Hocko wrote:
[...]
Quoted 9 lines
Quoted 4 lines
The vmstat update might be still waiting for its timer, idle mode started and kick
vmstat_updatewhich mightcpumask_test_and_set_cpu. Once the idle terminates and the originally schedulevmstate_updateexecutes it sees the bit set andBUG_ON.Ok so we are going into idle mode and the
vmstat_updatetimer is pending. Then the timer will not fire since going idle switches preemption off.quiet_vmstatwill run without the chance of runningvmstat_update
Quoted 5 lines
We could be going idle and not have disabled preemption yet. Then
vmstat_updatewill run. On return to the idling operation preemption will be disabled andquiet_vmstat()will be run.I do not see how these two things could race.
It goes like this:
CPU0: CPU1vmstat_update
cpumask_test_and_set_cpu (0->1)
[...]
vmstat_shepherd
<enter idle> cpumask_test_and_clear_cpu(CPU0) (1->0)quiet_vmstat
cpumask_test_and_set_cpu (0->1)
queue_delayed_work_on(CPU0)refresh_cpu_vm_stats()
[...]vmstat_update
nothing_to_do
cpumask_test_and_set_cpu (1->1)
VM_BUG_ON
Or am I missing something?
Signature · 8 lines
Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 17 lines · Michal Hocko
It goes like this:
CPU0: CPU1vmstat_update
cpumask_test_and_set_cpu(0->1)
[...]
vmstat_shepherd
<enter idle>cpumask_test_and_clear_cpu(CPU0)(1->0)quiet_vmstat
cpumask_test_and_set_cpu(0->1)
queue_delayed_work_on(CPU0)refresh_cpu_vm_stats()
[...]vmstat_update
nothing_to_do
cpumask_test_and_set_cpu(1->1)
VM_BUG_ON
Quoted 1 line
Or am I missing something?
Ok then the following should fix it:
Subject: vmstat: Queue work before clearing cpu_stat_off
There is a race between vmstat_shepherd and quiet_vmstat() because the responsibility for checking for counter updates changes depending on the state of teh bit in cpu_stat_off. So queue the work before changing state of the bit in vmstat_shepherd. That way quiet_vmstat is guaranteed to remove the work request when clearing the bit and the bug in vmstat_update wont trigger anymore.
mm/vmstat.c +4 -2
@@ -1480,12 +1480,14 @@ static void vmstat_shepherd(struct work_get_online_cpus ();/* Check processors whose vmstat worker threads have been disabled */ for_each_cpu (cpu, cpu_stat_off) -if (need_update (cpu) && -cpumask_test_and_clear_cpu (cpu, cpu_stat_off)) +if (need_update (cpu)) {queue_delayed_work_on (cpu, vmstat_wq, &per_cpu (vmstat_work, cpu),0 ); +cpumask_clear_cpu (smp_processor_id (), cpu_stat_off); + } +put_online_cpus (); schedule_delayed_work(&shepherd,
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 22 lines · Christoph Lameter
Subject: vmstat: Queue work before clearing
cpu_stat_offThere is a race between
vmstat_shepherdandquiet_vmstat()because the responsibility for checking for counter updates changes depending on the state of teh bit incpu_stat_off. So queue the work before changing state of the bit invmstat_shepherd. That wayquiet_vmstatis guaranteed to remove the work request when clearing the bit and the bug invmstat_updatewont trigger anymore.Signed-off-by Christoph Lameter <[email protected]>Index:
linux/mm/vmstat.c
===================================================================mm/vmstat.c +1
-2@@ -1480,12 +1480,14 @@ static void vmstat_shepherd(struct work_get_online_cpus ();/* Check processors whose vmstat worker threads have been disabled */ for_each_cpu (cpu, cpu_stat_off) -if (need_update (cpu) && -cpumask_test_and_clear_cpu (cpu, cpu_stat_off)) +if (need_update (cpu)) {
mm/vmstat.c static void vmstat_shepherd(struct work_ · line 1493
Quoted 10 lines
mm/vmstat.c +3
-0queue_delayed_work_on (cpu, vmstat_wq, &per_cpu (vmstat_work, cpu),0 ); +cpumask_clear_cpu (smp_processor_id (), cpu_stat_off); + } +put_online_cpus (); schedule_delayed_work(&shepherd,
This can alternatively lead to following where vmstat may not be scheduled for cpu when it is back from idle.
CPU0: CPU1:
vmstat_shepherd
<enter idle> queue_delayed_work_on(CPU0)quiet_vmstat
cancel_delayed_work
cpumask_test_and_set_cpu (0->1)
cpumask_clear_cpu(CPU0) (1->0)
Signature · 8 lines
regards Shiraz Hashim -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 19 lines · Christoph Lameter
Quoted 17 lines · Michal Hocko
It goes like this:
CPU0: CPU1vmstat_update
cpumask_test_and_set_cpu(0->1)
[...]
vmstat_shepherd
<enter idle>cpumask_test_and_clear_cpu(CPU0)(1->0)quiet_vmstat
cpumask_test_and_set_cpu(0->1)
queue_delayed_work_on(CPU0)refresh_cpu_vm_stats()
[...]vmstat_update
nothing_to_do
cpumask_test_and_set_cpu(1->1)
VM_BUG_ON
Quoted 3 lines
Quoted 1 line
Or am I missing something?
Ok then the following should fix it:
Wouldn't it be much more easier and simply get rid of the VM_BUG_ON? What is the point of keeping it in the first place. The code can perfectly cope with the race.
Signature · 8 lines
Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 3 lines · Michal Hocko
Wouldn't it be much more easier and simply get rid of the
VM_BUG_ON? What is the point of keeping it in the first place. The code can perfectly cope with the race.
Ok then lets do that.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 7 lines · Christoph Lameter
Quoted 3 lines · Michal Hocko
Wouldn't it be much more easier and simply get rid of the
VM_BUG_ON? What is the point of keeping it in the first place. The code can perfectly cope with the race.Ok then lets do that.
Could you repost the patch with the updated description?
Signature · 8 lines
Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 1 line · Michal Hocko
Could you repost the patch with the updated description?
Subject: vmstat: Remove BUG_ON from vmstat_update
If we detect that there is nothing to do just set the flag and do not check if it was already set before. Races really do not matter. If the flag is set by any code then the shepherd will start dealing with the situation and reenable the vmstat workers when necessary again.
Since 0eb77e988032 ("vmstat: make vmstat_updater deferrable again and shut down on idle") quiet_vmstat might update cpu_stat_off and mark a particular cpu to be handled by vmstat_shepherd. This might trigger a VM_BUG_ON in vmstat_update because the work item might have been sleeping during the idle period and see the cpu_stat_off updated after the wake up. The VM_BUG_ON is therefore misleading and no more appropriate. Moreover it doesn't really suite any protection from real bugs because vmstat_shepherd will simply reschedule the vmstat_work anytime it sees a particular cpu set or vmstat_update would do the same from the worker context directly. Even when the two would race the result wouldn't be incorrect as the counters update is fully idempotent.
mm/vmstat.c +1 -11
@@ -1408,17 +1408,7 @@ static void vmstat_update(struct work_st* Defer the checking for differentials to the * shepherd thread on a different processor. */ -int r; -/* -* Shepherd work thread does not race since it never -* changes the bit if its zero but the cpu -* online / off line code may race if -* worker threads are still allowed during -* shutdown / startup. -*/ - r =cpumask_test_and_set_cpu (smp_processor_id (), - cpu_stat_off); -VM_BUG_ON (r); +cpumask_set_cpu (smp_processor_id (), cpu_stat_off); } }
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Michal Hocko
Acked-byQuoted 11 lines · Christoph Lameter
Quoted 1 line · Michal Hocko
Could you repost the patch with the updated description?
Subject: vmstat: Remove
BUG_ONfromvmstat_updateIf we detect that there is nothing to do just set the flag and do not check if it was already set before. Races really do not matter. If the flag is set by any code then the shepherd will start dealing with the situation and reenable the vmstat workers when necessary again.
Quoted 12 lines
Since
0eb77e988032("vmstat: makevmstat_updaterdeferrable again and shut down on idle")quiet_vmstatmight updatecpu_stat_offand mark a particular cpu to be handled byvmstat_shepherd. This might trigger aVM_BUG_ONinvmstat_updatebecause the work item might have been sleeping during the idle period and see thecpu_stat_offupdated after the wake up. TheVM_BUG_ONis therefore misleading and no more appropriate. Moreover it doesn't really suite any protection from real bugs becausevmstat_shepherdwill simply reschedule thevmstat_workanytime it sees a particular cpu set orvmstat_updatewould do the same from the worker context directly. Even when the two would race the result wouldn't be incorrect as the counters update is fully idempotent.
Would be appropriate IMO
Quoted 1 line
Signed-off-by Christoph Lameter <[email protected]>
Thanks!
Quoted 23 lines
Index:
linux/mm/vmstat.c
===================================================================mm/vmstat.c +1
-11@@ -1408,17 +1408,7 @@ static void vmstat_update(struct work_st* Defer the checking for differentials to the * shepherd thread on a different processor. */ -int r; -/* -* Shepherd work thread does not race since it never -* changes the bit if its zero but the cpu -* online / off line code may race if -* worker threads are still allowed during -* shutdown / startup. -*/ - r =cpumask_test_and_set_cpu (smp_processor_id (), - cpu_stat_off); -VM_BUG_ON (r); +cpumask_set_cpu (smp_processor_id (), cpu_stat_off); } }
Signature · 8 lines
Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
In reply to Christoph LameterHi Christoph,
While you're fixing that commit up, can you perhaps find a better home for quiet_vmstat()? It not only munches cycles when switching cross -core mightily, for -rt it injects a sleeping lock into the idle task.
12.89% [kernel] [k] refresh_cpu_vm_stats.isra.12
4.75% [kernel] [k] __schedule
4.70% [kernel] [k] mutex_unlock
3.14% [kernel] [k] __switch_to -Mike
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
Quoted 3 lines · Mike Galbraith
While you're fixing that commit up, can you perhaps find a better home for
quiet_vmstat()? It not only munches cycles when switching cross -core mightily, for -rt it injects a sleeping lock into the idle task.
Not sure what you are talking about. No sleeping locks are used in quiet_vmstat() nor does it switch across cores. It would be broken if it would do so.
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
Quoted 9 lines · Christoph Lameter
Quoted 3 lines · Mike Galbraith
While you're fixing that commit up, can you perhaps find a better home for
quiet_vmstat()? It not only munches cycles when switching cross -core mightily, for -rt it injects a sleeping lock into the idle task.Not sure what you are talking about. No sleeping locks are used in
quiet_vmstat()nor does it switch across cores. It would be broken if it would do so.
By switching cross-core, I'm referring to scheduling of communicating tasks.
The perf top snippet...
12.89% [kernel] [k] refresh_cpu_vm_stats.isra.12
4.75% [kernel] [k] __schedule
4.70% [kernel] [k] mutex_unlock
3.14% [kernel] [k] __switch_to... was pipe-test, an unrealistic microbench, but the same will happen at lower frequency in the real world.
Here's the sleeping lock for -rt:
[ 2.279582] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.5.0-rt3 #7 [ 2.280444] Hardware name: MEDION MS-7848/MS-7848, BIOS M7848W08.20C 09/23/2013 [ 2.281316] ffff88040b00d640 ffff88040b01fe10 ffffffff812d20e2 0000000000000000 [ 2.282202] ffff88040b01fe30 ffffffff81081095 ffff88041ec4cee0 ffff88041ec501e0 [ 2.283073] ffff88040b01fe48 ffffffff815ff910 ffff88041ec4cee0 ffff88040b01fe88 [ 2.283941] Call Trace: [ 2.284797] [<ffffffff812d20e2>] dump_stack+0x49/0x67 [ 2.285658] [<ffffffff81081095>] ___might_sleep+0xf5/0x180 [ 2.286521] [<ffffffff815ff910>] rt_spin_lock+0x20/0x50 [ 2.287382] [<ffffffff81075919>] try_to_grab_pending+0x69/0x240 [ 2.288239] [<ffffffff81075b16>] cancel_delayed_work+0x26/0xe0 [ 2.289094] [<ffffffff8115ec05>] quiet_vmstat+0x75/0xa0 [ 2.289949] [<ffffffff8109ab38>] cpu_idle_loop+0x38/0x3e0 [ 2.290800] [<ffffffff8109aef3>] cpu_startup_entry+0x13/0x20 [ 2.291647] [<ffffffff81036164>] start_secondary+0x114/0x140
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
Quoted 2 lines · Mike Galbraith
By switching cross-core, I'm referring to scheduling of communicating tasks.
??? Its cancelling a work request. That is a "communicating task"?
Quoted 13 lines
Here's the sleeping lock for -rt:
[ 2.279582] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.5.0-rt3 #7 [ 2.280444] Hardware name: MEDION MS-7848/MS-7848, BIOS M7848W08.20C 09/23/2013 [ 2.281316] ffff88040b00d640 ffff88040b01fe10 ffffffff812d20e2 0000000000000000 [ 2.282202] ffff88040b01fe30 ffffffff81081095 ffff88041ec4cee0 ffff88041ec501e0 [ 2.283073] ffff88040b01fe48 ffffffff815ff910 ffff88041ec4cee0 ffff88040b01fe88 [ 2.283941] Call Trace: [ 2.284797] [<ffffffff812d20e2>] dump_stack+0x49/0x67 [ 2.285658] [<ffffffff81081095>] ___might_sleep+0xf5/0x180 [ 2.286521] [<ffffffff815ff910>] rt_spin_lock+0x20/0x50 [ 2.287382] [<ffffffff81075919>] try_to_grab_pending+0x69/0x240 [ 2.288239] [<ffffffff81075b16>] cancel_delayed_work+0x26/0xe0
OMG cancelling a work request causes a sleeping lock to be taken?
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
Quoted 6 lines · Christoph Lameter
Quoted 2 lines · Mike Galbraith
By switching cross-core, I'm referring to scheduling of communicating tasks.
??? Its cancelling a work request. That is a "communicating task"?
No no no, pipe-test is two tasks playing ping-pong via a pipe. They are about as synchronous as it gets, so each task goes idle at high frequency. Idle is fastpath.
Quoted 14 lines
Quoted 13 lines
Here's the sleeping lock for -rt:
[ 2.279582] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.5.0-rt3 #7 [ 2.280444] Hardware name: MEDION MS-7848/MS-7848, BIOS M7848W08.20C 09/23/2013 [ 2.281316] ffff88040b00d640 ffff88040b01fe10 ffffffff812d20e2 0000000000000000 [ 2.282202] ffff88040b01fe30 ffffffff81081095 ffff88041ec4cee0 ffff88041ec501e0 [ 2.283073] ffff88040b01fe48 ffffffff815ff910 ffff88041ec4cee0 ffff88040b01fe88 [ 2.283941] Call Trace: [ 2.284797] [] dump_stack+0x49/0x67 [ 2.285658] [] ___might_sleep+0xf5/0x180 [ 2.286521] [] rt_spin_lock+0x20/0x50 [ 2.287382] [] try_to_grab_pending+0x69/0x240 [ 2.288239] [] cancel_delayed_work+0x26/0xe0
Quoted 1 line
OMG cancelling a work request causes a sleeping lock to be taken?
Yup. In -rt, spinlocks that are not raw are transformed into rtmutex.
-Mike
Quoted 5 lines · Christoph Lameter
Subject: vmstat: Remove
BUG_ONfromvmstat_updateIf we detect that there is nothing to do just set the flag and do not check if it was already set before. [..]
Ok, I am assuming this is in Andrew's queue already, but this bug hit my machine overnight, so I'm applying it directly..
Linus
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
In reply to Mike GalbraithQuoted 10 lines · Mike Galbraith
Hi Christoph,
While you're fixing that commit up, can you perhaps find a better home for
quiet_vmstat()? It not only munches cycles when switching cross -core mightily, for -rt it injects a sleeping lock into the idle task.12.89% [kernel] [k] refresh_cpu_vm_stats.isra.12 4.75% [kernel] [k] __schedule 4.70% [kernel] [k] mutex_unlock 3.14% [kernel] [k] __switch_to
Hmm, I wouldn't have expected that refresh_cpu_vm_stats could have such a large footprint. I guess this would be just an expensive noop because we have to check all the zones*counters and do an expensive this_cpu_xchg. Is the whole deferred thing worth this overhead?
0eb77e988032 ("vmstat: make vmstat_updater deferrable again and shut down on idle") doesn't talk about any numbers and neither does 39bf6270f524 ("VM statistics: Make timer deferrable").
But even when refresh_cpu_vm_stats is made more effective there would still be the problem for RT and the work canceling, though. We would need much more changes to make this RT ready (both timer and WQ code would need some changes AFAIU).
Unless there is a clear and huge win from doing the vmstat update deferrable then I think a revert is more appropriate IMHO.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
Quoted 12 lines · Michal Hocko
Quoted 10 lines · Mike Galbraith
Hi Christoph,
While you're fixing that commit up, can you perhaps find a better home for
quiet_vmstat()? It not only munches cycles when switching cross -core mightily, for -rt it injects a sleeping lock into the idle task.12.89% [kernel] [k] refresh_cpu_vm_stats.isra.12 4.75% [kernel] [k] __schedule 4.70% [kernel] [k] mutex_unlock 3.14% [kernel] [k] __switch_to
Quoted 4 lines
Hmm, I wouldn't have expected that
refresh_cpu_vm_statscould have such a large footprint. I guess this would be just an expensive noop because we have to check all the zones*counters and do an expensivethis_cpu_xchg. Is the whole deferred thing worth this overhead?
Why would the deferring cause this overhead?
Also there is no cross core activity from quiet_vmstat(). It simply disables the local vmstat updates.
Quoted 2 lines
Unless there is a clear and huge win from doing the vmstat update deferrable then I think a revert is more appropriate IMHO.
It reduces the OS events that the application experiences by folding it into the tick events. If its not deferrable then a timer event will be generated in addition to the tick. We do not want that.
Workqueues are used in many places. If RT can sleep within workqueue management functions then spinlocks cannot be taken anymore and there may be issues with preemption.
The regression that I know of (independent of "RT") is due as far as I know due to the switch of the parameters of some vmstat functions to 64 bit instead of 32 bit.
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
Quoted 14 lines · Christoph Lameter
Quoted 12 lines · Michal Hocko
Quoted 10 lines · Mike Galbraith
Hi Christoph,
While you're fixing that commit up, can you perhaps find a better home for
quiet_vmstat()? It not only munches cycles when switching cross -core mightily, for -rt it injects a sleeping lock into the idle task.12.89% [kernel] [k] refresh_cpu_vm_stats.isra.12 4.75% [kernel] [k] __schedule 4.70% [kernel] [k] mutex_unlock 3.14% [kernel] [k] __switch_to
Quoted 6 lines
Quoted 4 lines
Hmm, I wouldn't have expected that
refresh_cpu_vm_statscould have such a large footprint. I guess this would be just an expensive noop because we have to check all the zones*counters and do an expensivethis_cpu_xchg. Is the whole deferred thing worth this overhead?Why would the deferring cause this overhead?
I guess the profile speaks for itself, doesn't it?
Quoted 2 lines
Also there is no cross core activity from
quiet_vmstat(). It simply disables the local vmstat updates.
It doesn't go cross core but it still does nr_zones * counters atomic ops.
Quoted 6 lines
Quoted 2 lines
Unless there is a clear and huge win from doing the vmstat update deferrable then I think a revert is more appropriate IMHO.
It reduces the OS events that the application experiences by folding it into the tick events. If its not deferrable then a timer event will be generated in addition to the tick. We do not want that.
Yes this is what I have read in the changelog. But "how much" part is really missing. Is this even quantifiable?
Quoted 3 lines
Workqueues are used in many places. If RT can sleep within workqueue management functions then spinlocks cannot be taken anymore and there may be issues with preemption.
RT can sleep in _any_ spinlock except for raw spin locks. Even though the !RT kernel is not sleeping doesn't really matter much because cancel_delayed_work is quite a heavy function which shouldn't be called from the idle context AFAIU. Sure most of the time it will boil down to del_timer but it can hit the slowpath as well if the timer got migrated to a different CPU and we have to race with the WQ pool management IIUC.
Maybe this overhead can be reduced by outsourcing the functionality to vmstat_shepherd which can check idle CPUs, cancel the timer for them update the differentials and put them to cpu_stat_off?
Quoted 3 lines
The regression that I know of (independent of "RT") is due as far as I know due to the switch of the parameters of some vmstat functions to 64 bit instead of 32 bit.
I am not sure I am following.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
In reply to Christoph LameterQuoted 14 lines · Christoph Lameter
Quoted 12 lines · Michal Hocko
Quoted 10 lines · Mike Galbraith
Hi Christoph,
While you're fixing that commit up, can you perhaps find a better home for
quiet_vmstat()? It not only munches cycles when switching cross -core mightily, for -rt it injects a sleeping lock into the idle task.12.89% [kernel] [k] refresh_cpu_vm_stats.isra.12 4.75% [kernel] [k] __schedule 4.70% [kernel] [k] mutex_unlock 3.14% [kernel] [k] __switch_to
Quoted 6 lines
Quoted 4 lines
Hmm, I wouldn't have expected that
refresh_cpu_vm_statscould have such a large footprint. I guess this would be just an expensive noop because we have to check all the zones*counters and do an expensivethis_cpu_xchg. Is the whole deferred thing worth this overhead?Why would the deferring cause this overhead?
Because we schedule to idle cores aggressively, thus we may pop in and out of idle at high frequency.
Quoted 2 lines
Also there is no cross core activity from
quiet_vmstat(). It simply disables the local vmstat updates.
Again, the cross core activity is not due to quiet_vmstat(), it is due to pipe-test threads running on two cores, and meeting quiet_vmstat() at high frequency.
Quoted 6 lines
Quoted 2 lines
Unless there is a clear and huge win from doing the vmstat update deferrable then I think a revert is more appropriate IMHO.
It reduces the OS events that the application experiences by folding it into the tick events. If its not deferrable then a timer event will be generated in addition to the tick. We do not want that.
Perf and RT say we don't want quiet_vmstat() in the idle loop either.
-Mike
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
Quoted 1 line · Mike Galbraith
Perf and RT say we don't want
quiet_vmstat()in the idle loop either.
BTW, the perf numbers were not from an RT kernel, they were from my PREEMPT_VOLUNTARY desktop kernel.
-Mike
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
In reply to Michal HockoQuoted 3 lines · Michal Hocko
Quoted 1 line
Why would the deferring cause this overhead?
I guess the profile speaks for itself, doesn't it?
But the system is going idle? Why would this impact performance?
Quoted 5 lines
Quoted 2 lines
Also there is no cross core activity from
quiet_vmstat(). It simply disables the local vmstat updates.It doesn't go cross core but it still does
nr_zones* counters atomic ops.
If there are updates then yes.
Quoted 6 lines
Quoted 3 lines
It reduces the OS events that the application experiences by folding it into the tick events. If its not deferrable then a timer event will be generated in addition to the tick. We do not want that.
Yes this is what I have read in the changelog. But "how much" part is really missing. Is this even quantifiable?
Oh yes. If you want to have low latency responses yes. And event like that causes a couple of microseconds delay which will cause a moneytary impact if you have to react to stock trading events.
Quoted 3 lines
Maybe this overhead can be reduced by outsourcing the functionality to
vmstat_shepherdwhich can check idle CPUs, cancel the timer for them update the differentials and put them tocpu_stat_off?
Remote updating of differentials is problematic due to the counters being per cpu values that are expected to only be updated from the cpu that "owns" them.
Quoted 5 lines
Quoted 3 lines
The regression that I know of (independent of "RT") is due as far as I know due to the switch of the parameters of some vmstat functions to 64 bit instead of 32 bit.
I am not sure I am following.
An additional patch was merged for 4.5 that increases the arguments to the counter operations to 64 bit which is known for regressions.
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
In reply to Mike GalbraithQuoted 4 lines · Mike Galbraith
Quoted 1 line
Why would the deferring cause this overhead?
Because we schedule to idle cores aggressively, thus we may pop in and out of idle at high frequency.
Whats the point of going idle if you have things to do soon?
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
In reply to Mike GalbraithQuoted 6 lines · Mike Galbraith
Quoted 1 line · Mike Galbraith
Perf and RT say we don't want
quiet_vmstat()in the idle loop either.BTW, the perf numbers were not from an RT kernel, they were from my
PREEMPT_VOLUNTARYdesktop kernel.
Can we move quiet_vmstat() elsewhere after we have checked that really nothing else is going on soon?
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
In reply to Christoph LameterQuoted 8 lines · Christoph Lameter
Quoted 4 lines · Mike Galbraith
Quoted 1 line
Why would the deferring cause this overhead?
Because we schedule to idle cores aggressively, thus we may pop in and out of idle at high frequency.
Whats the point of going idle if you have things to do soon?
When a task schedules off, how do you know it'll be back at all, much less soon?
-Mike
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
In reply to Christoph LameterQuoted 4 lines · Christoph Lameter
Quoted 2 lines · Mike Galbraith
On Tue, 2016-01-26 at 03:14 +0100, Mike Galbraith wrote:
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Perf and RT say we don't want
quiet_vmstat()in the idle loop either.BTW, the perf numbers were not from an RT kernel, they were from my
PREEMPT_VOLUNTARYdesktop kernel.Can we move
quiet_vmstat()elsewhere after we have checked that really nothing else is going on soon?
How would you check? Precognition doesn't work for mortals.
-Mike
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
Quoted 8 lines · Mike Galbraith
Quoted 7 lines · Christoph Lameter
Quoted 5 lines · Mike Galbraith
Quoted 2 lines · Mike Galbraith
Perf and RT say we don't want
quiet_vmstat()in the idle loop either.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
BTW, the perf numbers were not from an RT kernel, they were from my
PREEMPT_VOLUNTARYdesktop kernel.Can we move
quiet_vmstat()elsewhere after we have checked that really nothing else is going on soon?How would you check? Precognition doesn't work for mortals.
Dont we have some decision mechanism to go into higher levels of power savings when the system is idle for longer times?
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
In reply to Mike GalbraithQuoted 5 lines · Mike Galbraith
Quoted 4 lines · Christoph Lameter
Quoted 2 lines · Mike Galbraith
Quoted 1 line
Why would the deferring cause this overhead?
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Because we schedule to idle cores aggressively, thus we may pop in and out of idle at high frequency.
Whats the point of going idle if you have things to do soon?
When a task schedules off, how do you know it'll be back at all, much less soon?
Ok so you are running an artificial benchmark that always gets the system running again when it decides to go idle?
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
In reply to Christoph LameterQuoted 7 lines · Christoph Lameter
Quoted 3 lines · Michal Hocko
Quoted 1 line
Why would the deferring cause this overhead?
I guess the profile speaks for itself, doesn't it?
But the system is going idle? Why would this impact performance?
We enter/exit idle a lot.
Your reluctance to move it seem to suggest that 99.99% of CPUs on the planet chewing up cycles (measured) doing what for most is useless work on every micro-idle is a perfectly fine price to pay to ensure that .01% (or whatever tiny minority) get what they want.
I disagree. You're burning electrons for no benefit at all to me on my box. You want to do high speed trading, that's fine, but I expect my box to be able to pop in and out of idle without having to pay a toll to the high speed trading bandits of the world, thank you very much.
This specialty thing does not belong in the generic fast path.
-Mike
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
Quoted 6 lines · Mike Galbraith
I disagree. You're burning electrons for no benefit at all to me on my box. You want to do high speed trading, that's fine, but I expect my box to be able to pop in and out of idle without having to pay a toll to the high speed trading bandits of the world, thank you very much.
This specialty thing does not belong in the generic fast path.
The system going idle is a fastpath. Mind boogling.
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
Quoted 10 lines · Christoph Lameter
Quoted 6 lines · Mike Galbraith
I disagree. You're burning electrons for no benefit at all to me on my box. You want to do high speed trading, that's fine, but I expect my box to be able to pop in and out of idle without having to pay a toll to the high speed trading bandits of the world, thank you very much.
This specialty thing does not belong in the generic fast path.
The system going idle is a fastpath. Mind boogling.
Hohum, noted. Now what about those cycles, and the sleeping lock you injected for -rt?
-Mike
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
In reply to Christoph LameterQuoted 11 lines · Christoph Lameter
Quoted 9 lines · Mike Galbraith
Quoted 8 lines · Christoph Lameter
Quoted 5 lines · Mike Galbraith
Quoted 1 line
Why would the deferring cause this overhead?
Because we schedule to idle cores aggressively, thus we may pop in and
out of idle at high frequency.
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
Whats the point of going idle if you have things to do soon?
When a task schedules off, how do you know it'll be back at all, much
less soon?Ok so you are running an artificial benchmark that always gets the system running again when it decides to go idle?
The benchmark does not alter the cycle expenditure per event. The real world will pay the toll less frequently than the artificial benchmark, yes, but it will pay nonetheless, and for most, needlessly. Or?
-Mike
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
In reply to Mike GalbraithQuoted 4 lines · Mike Galbraith
Quoted 1 line
The system going idle is a fastpath. Mind boogling.
Hohum, noted. Now what about those cycles, and the sleeping lock you injected for -rt?
Since we (the NOHZ people) care mostly about NOHZ then lets restrict that to the NOHZ mode. Then it should not affect your load.
Subject: Move quiet_vmstat() to NOHZ code
quiet_vmstat() seems to cause regressions for some load because the cpu going idle is a "fastpath". Mind boogling. Strange claim. If the system goes idle then it has nothing to do after all
But anyways if we shift the quiet_vmstat() into the NOHZ logic when it stops the tick then it will only affect those cores that are setup for NOHZ mode. That is where we want this processing after all to ensure that the OS keeps itself off those cores.
kernel/sched/idle.c +0 -1
@@ -213,7 +213,6 @@ static void cpu_idle_loop(void)*/ __current_set_polling (); -quiet_vmstat ();tick_nohz_idle_enter ();while (!need_resched ()) {
kernel/time/tick-sched.c +1 -0
@@ -811,6 +811,7 @@ static void __tick_nohz_idle_enter(struc ts->idle_calls++; expires =tick_nohz_stop_sched_tick (ts, now, cpu); +quiet_vmstat ();if (expires.tv64 >0LL ) { ts->idle_sleeps++; ts->idle_expires = expires;
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
In reply to Mike GalbraithQuoted 6 lines · Mike Galbraith
Quoted 2 lines
Ok so you are running an artificial benchmark that always gets the system running again when it decides to go idle?
The benchmark does not alter the cycle expenditure per event. The real world will pay the toll less frequently than the artificial benchmark, yes, but it will pay nonetheless, and for most, needlessly. Or?
Well if it does not pay it there then it is going to pay for it later when the vmstat_updater needs to cause a wakeup to fold the data. Also not folding the vmstat updates increases the time that minor updates are deferred and thus impacts the accuracy of the global vm counters. That seems to have been important recently.
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
In reply to Christoph LameterGood morning,
Quoted 9 lines · Christoph Lameter
Quoted 4 lines · Mike Galbraith
Quoted 1 line
The system going idle is a fastpath. Mind boogling.
Hohum, noted. Now what about those cycles, and the sleeping lock you injected for -rt?
Since we (the NOHZ people) care mostly about NOHZ then lets restrict that to the NOHZ mode. Then it should not affect your load.
Tons of folks do have NO_HZ enabled (including me). Isn't there a spot somewhere in NO_HZ_FULL code where it can take up residence? (one with a tad lower maximum call frequency would be good, a nohz_full cpu isn't necessarily being used for pure compute exclusively)
Quoted 38 lines
Subject: Move
quiet_vmstat()to NOHZ code
quiet_vmstat()seems to cause regressions for some load because the cpu going idle is a "fastpath". Mind boogling. Strange claim. If the system goes idle then it has nothing to do after allBut anyways if we shift the
quiet_vmstat()into the NOHZ logic when it stops the tick then it will only affect those cores that are setup for NOHZ mode. That is where we want this processing after all to ensure that the OS keeps itself off those cores.Signed-off-by Christoph Lameter <[email protected]>Index:
linux/kernel/sched/idle.c
===================================================================kernel/sched/idle.c +0
-1@@ -213,7 +213,6 @@ static void cpu_idle_loop(void)*/ __current_set_polling (); -quiet_vmstat ();tick_nohz_idle_enter ();while (!need_resched ()) {Index:
linux/kernel/time/tick-sched.c
===================================================================kernel/time/tick-sched.c +1
-0@@ -811,6 +811,7 @@ static void __tick_nohz_idle_enter(struc ts->idle_calls++; expires =tick_nohz_stop_sched_tick (ts, now, cpu); +quiet_vmstat ();if (expires.tv64 >0LL ) { ts->idle_sleeps++; ts->idle_expires = expires;
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
Quoted 10 lines · Mike Galbraith
Good morning,
Quoted 7 lines · Christoph Lameter
Quoted 4 lines · Mike Galbraith
Quoted 1 line
The system going idle is a fastpath. Mind boogling.
Hohum, noted. Now what about those cycles, and the sleeping lock you injected for -rt?
Quoted 7 lines
Quoted 2 lines
Since we (the NOHZ people) care mostly about NOHZ then lets restrict that to the NOHZ mode. Then it should not affect your load.
Tons of folks do have
NO_HZenabled (including me). Isn't there a spot somewhere inNO_HZ_FULLcode where it can take up residence? (one with a tad lower maximum call frequency would be good, anohz_fullcpu isn't necessarily being used for pure compute exclusively)
I forgot to mention that the spot you picked is called with irqs disabled.
Quoted 5 lines
Quoted 5 lines
Subject: Move
quiet_vmstat()to NOHZ code
quiet_vmstat()seems to cause regressions for some load because the cpu going idle is a "fastpath". Mind boogling. Strange claim. If the system goes idle then it has nothing to do after all
Hm, seems I also forgot to say "Hohum, noted..." again.
-Mike
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
In reply to Mike GalbraithQuoted 7 lines · Mike Galbraith
Quoted 2 lines
Since we (the NOHZ people) care mostly about NOHZ then lets restrict that to the NOHZ mode. Then it should not affect your load.
Tons of folks do have
NO_HZenabled (including me). Isn't there a spot somewhere inNO_HZ_FULLcode where it can take up residence? (one with a tad lower maximum call frequency would be good, anohz_fullcpu isn't necessarily being used for pure compute exclusively)
Frederic, any idea on where to put quiet_vmstat()?
Quoted 39 lines
Quoted 38 lines
Subject: Move
quiet_vmstat()to NOHZ code
quiet_vmstat()seems to cause regressions for some load because the cpu going idle is a "fastpath". Mind boogling. Strange claim. If the system goes idle then it has nothing to do after allBut anyways if we shift the
quiet_vmstat()into the NOHZ logic when it stops the tick then it will only affect those cores that are setup for NOHZ mode. That is where we want this processing after all to ensure that the OS keeps itself off those cores.Signed-off-by Christoph Lameter <[email protected]>Index:
linux/kernel/sched/idle.c
===================================================================kernel/sched/idle.c +0
-1@@ -213,7 +213,6 @@ static void cpu_idle_loop(void)*/ __current_set_polling (); -quiet_vmstat ();tick_nohz_idle_enter ();while (!need_resched ()) {Index:
linux/kernel/time/tick-sched.c
===================================================================kernel/time/tick-sched.c +1
-0@@ -811,6 +811,7 @@ static void __tick_nohz_idle_enter(struc ts->idle_calls++; expires =tick_nohz_stop_sched_tick (ts, now, cpu); +quiet_vmstat ();if (expires.tv64 >0LL ) { ts->idle_sleeps++; ts->idle_expires = expires;
PATCH 1/1 mm, vmstat: make quiet_vmstat lighter (was: Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable) again and shut down on idle)
In reply to Mike GalbraithQuoted 10 lines · Mike Galbraith
Hi Christoph,
While you're fixing that commit up, can you perhaps find a better home for
quiet_vmstat()? It not only munches cycles when switching cross -core mightily, for -rt it injects a sleeping lock into the idle task.12.89% [kernel] [k] refresh_cpu_vm_stats.isra.12 4.75% [kernel] [k] __schedule 4.70% [kernel] [k] mutex_unlock 3.14% [kernel] [k] __switch_to
What about the following fix?
---
Quoted 1 line
From
c74a04c4fdfe1fa67933bb1ac83d3de0532aaab2Mon Sep 17 00:00:00 2001
From: Michal Hocko <[email protected]>
Date: Wed, 27 Jan 2016 17:24:22 +0100
Subject: [PATCH] mm, vmstat: make quiet_vmstat lighter
Mike has reported a considerable overhead of refresh_cpu_vm_stats from the idle entry during pipe test:
12.89% [kernel] [k] refresh_cpu_vm_stats.isra.12
4.75% [kernel] [k] __schedule
4.70% [kernel] [k] mutex_unlock
3.14% [kernel] [k] __switch_toThis is caused by 0eb77e988032 ("vmstat: make vmstat_updater deferrable again and shut down on idle") which has placed quiet_vmstat into cpu_idle_loop. The main reason here seems to be that the idle entry has to get over all zones and perform atomic operations for each vmstat entry even though there might be no per cpu diffs. This is a pointless overhead for _each_ idle entry.
Make sure that quiet_vmstat is as light as possible.
First of all it doesn't make any sense to do any local sync if the
current cpu is already set in oncpu_stat_off because vmstat_update puts itself there only if there is nothing to do.
Then we can check need_update which should be a cheap way to check for potential per-cpu diffs and only then do refresh_cpu_vm_stats.
The original patch also did cancel_delayed_work which we are not doing here. There are two reasons for that. Firstly cancel_delayed_work from idle context will blow up on RT kernels (reported by Mike):
[ 2.279582] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.5.0-rt3 #7 [ 2.280444] Hardware name: MEDION MS-7848/MS-7848, BIOS M7848W08.20C 09/23/2013 [ 2.281316] ffff88040b00d640 ffff88040b01fe10 ffffffff812d20e2 0000000000000000 [ 2.282202] ffff88040b01fe30 ffffffff81081095 ffff88041ec4cee0 ffff88041ec501e0 [ 2.283073] ffff88040b01fe48 ffffffff815ff910 ffff88041ec4cee0 ffff88040b01fe88 [ 2.283941] Call Trace: [ 2.284797] [<ffffffff812d20e2>] dump_stack+0x49/0x67 [ 2.285658] [<ffffffff81081095>] ___might_sleep+0xf5/0x180 [ 2.286521] [<ffffffff815ff910>] rt_spin_lock+0x20/0x50 [ 2.287382] [<ffffffff81075919>] try_to_grab_pending+0x69/0x240 [ 2.288239] [<ffffffff81075b16>] cancel_delayed_work+0x26/0xe0 [ 2.289094] [<ffffffff8115ec05>] quiet_vmstat+0x75/0xa0 [ 2.289949] [<ffffffff8109ab38>] cpu_idle_loop+0x38/0x3e0 [ 2.290800] [<ffffffff8109aef3>] cpu_startup_entry+0x13/0x20 [ 2.291647] [<ffffffff81036164>] start_secondary+0x114/0x140
And secondly, even on !RT kernels it might add some non trivial overhead which is not necessary. Even if the vmstat worker wakes up and preempts idle then it will be most likely a single shot noop because the stats were already synced and so it would end up on the oncpu_stat_off anyway. We just need to teach both vmstat_shepherd and vmstat_update to stop scheduling the worker if there is nothing to do.
--- mm/vmstat.c | 60 ++++++++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 40 insertions(+), 20 deletions(-)
mm/vmstat.c +40 -20
@@ -1396,10 +1396,15 @@ static void vmstat_update(struct work_struct *w)* Counters were updated so we expect more updates * to occur in the future. Keep on running the * update worker thread. +* If we were marked on cpu_stat_off clear the flag +* so that vmstat_shepherd doesn't schedule us again. */ -queue_delayed_work_on (smp_processor_id (), vmstat_wq, -this_cpu_ptr (&vmstat_work), -round_jiffies_relative (sysctl_stat_interval)); +if (!cpumask_test_and_clear_cpu (smp_processor_id (), + cpu_stat_off)) { +queue_delayed_work_on (smp_processor_id (), vmstat_wq, +this_cpu_ptr (&vmstat_work), +round_jiffies_relative (sysctl_stat_interval)); + } }else {/* * We did not update any counters so the app may be in
@@ -1417,18 +1422,6 @@ static void vmstat_update(struct work_struct *w)* until the diffs stay at zero. The function is used by NOHZ and can only be * invoked when tick processing is not active. */ -void quiet_vmstat (void ) -{ -if (system_state !=SYSTEM_RUNNING ) -return ; - -do { -if (!cpumask_test_and_set_cpu (smp_processor_id (), cpu_stat_off)) -cancel_delayed_work (this_cpu_ptr (&vmstat_work)); - - }while (refresh_cpu_vm_stats (false)); -} -/* * Check if the diffs for a certain cpu indicate that * an update is needed.
@@ -1452,6 +1445,30 @@ static bool need_update(int cpu)return false; } +void quiet_vmstat (void ) +{ +if (system_state !=SYSTEM_RUNNING ) +return ; + +/* +* If we are already in hands of the shepherd then there +* is nothing for us to do here. +*/ +if (cpumask_test_and_set_cpu (smp_processor_id (), cpu_stat_off)) +return ; + +if (!need_update (smp_processor_id ())) +return ; + +/* +* Just refresh counters and do not care about the pending delayed +* vmstat_update. It doesn't fire that often to matter and canceling +* it would be too expensive from this path. +* vmstat_shepherd will take care about that for us. +*/ +refresh_cpu_vm_stats (false); +} +/* * Shepherd worker thread that checks the
@@ -1470,11 +1487,14 @@ static void vmstat_shepherd(struct work_struct *w)get_online_cpus ();/* Check processors whose vmstat worker threads have been disabled */ for_each_cpu (cpu, cpu_stat_off) -if (need_update (cpu) && -cpumask_test_and_clear_cpu (cpu, cpu_stat_off)) - -queue_delayed_work_on (cpu, vmstat_wq, - &per_cpu (vmstat_work, cpu),0 ); +if (need_update (cpu)) { +if (cpumask_test_and_clear_cpu (cpu, cpu_stat_off)) +queue_delayed_work_on (cpu, vmstat_wq, + &per_cpu (vmstat_work, cpu),0 ); + }else { +cpumask_set_cpu (cpu, cpu_stat_off); +cancel_delayed_work (this_cpu_ptr (&vmstat_work)); + }put_online_cpus ();
Signature · 5 lines
2.7.0.rc3 -- Michal Hocko SUSE Labs
Re: [PATCH] mm, vmstat: make quiet_vmstat lighter (was: Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable) again and shut down on idle)
Quoted 10 lines · Michal Hocko
Quoted 9 lines · Mike Galbraith
Hi Christoph,
While you're fixing that commit up, can you perhaps find a better home
forquiet_vmstat()? It not only munches cycles when switching cross
-core mightily, for -rt it injects a sleeping lock into the idle task.
Quoted 6 lines
Quoted 4 lines
12.89% [kernel] [k] refresh_cpu_vm_stats.isra.12 4.75% [kernel] [k] __schedule 4.70% [kernel] [k] mutex_unlock 3.14% [kernel] [k] __switch_toWhat about the following fix?
I haven't done _extensive_ testing, but 4.5-rc1-rt (with NO_HZ_FULL enabling hacks restored, as it's otherwise a dead .config item) is gripe free whether nohz_full CPUs are in use or not, and overhead went away in it and desktop configured master kernel (PREEMPT_VOLUNTARY).
Quoted 194 lines
---
Fromc74a04c4fdfe1fa67933bb1ac83d3de0532aaab2Mon Sep 17 00:00:00 2001
From: Michal Hocko <[email protected]>
Date: Wed, 27 Jan 2016 17:24:22 +0100
Subject: [PATCH] mm, vmstat: makequiet_vmstatlighterMike has reported a considerable overhead of
refresh_cpu_vm_statsfrom
the idle entry during pipe test:12.89% [kernel] [k] refresh_cpu_vm_stats.isra.12 4.75% [kernel] [k] __schedule 4.70% [kernel] [k] mutex_unlock 3.14% [kernel] [k] __switch_toThis is caused by
0eb77e988032("vmstat: makevmstat_updaterdeferrable
again and shut down on idle") which has placedquiet_vmstatintocpu_idle_loop. The main reason here seems to be that the idle entry has
to get over all zones and perform atomic operations for each vmstat entry even though there might be no per cpu diffs. This is a pointless
overhead for _each_ idle entry.Make sure that
quiet_vmstatis as light as possible.First of all it doesn't make any sense to do any local sync if the
current cpu is already set inoncpu_stat_offbecausevmstat_updateputs
itself there only if there is nothing to do.Then we can check
need_updatewhich should be a cheap way to check for
potential per-cpu diffs and only then dorefresh_cpu_vm_stats.The original patch also did
cancel_delayed_workwhich we are not doing
here. There are two reasons for that. Firstlycancel_delayed_workfrom
idle context will blow up on RT kernels (reported by Mike):[ 2.279582] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.5.0-rt3 #7 [ 2.280444] Hardware name: MEDION MS-7848/MS-7848, BIOS M7848W08.20C 09/23/2013 [ 2.281316] ffff88040b00d640 ffff88040b01fe10 ffffffff812d20e2 0000000000000000 [ 2.282202] ffff88040b01fe30 ffffffff81081095 ffff88041ec4cee0 ffff88041ec501e0 [ 2.283073] ffff88040b01fe48 ffffffff815ff910 ffff88041ec4cee0 ffff88040b01fe88 [ 2.283941] Call Trace: [ 2.284797] [<ffffffff812d20e2>] dump_stack+0x49/0x67 [ 2.285658] [<ffffffff81081095>] ___might_sleep+0xf5/0x180 [ 2.286521] [<ffffffff815ff910>] rt_spin_lock+0x20/0x50 [ 2.287382] [<ffffffff81075919>] try_to_grab_pending+0x69/0x240 [ 2.288239] [<ffffffff81075b16>] cancel_delayed_work+0x26/0xe0 [ 2.289094] [<ffffffff8115ec05>] quiet_vmstat+0x75/0xa0 [ 2.289949] [<ffffffff8109ab38>] cpu_idle_loop+0x38/0x3e0 [ 2.290800] [<ffffffff8109aef3>] cpu_startup_entry+0x13/0x20 [ 2.291647] [<ffffffff81036164>] start_secondary+0x114/0x140And secondly, even on !RT kernels it might add some non trivial overhead
which is not necessary. Even if the vmstat worker wakes up and preempts
idle then it will be most likely a single shot noop because the stats were already synced and so it would end up on theoncpu_stat_offanyway.
We just need to teach bothvmstat_shepherdandvmstat_updateto stop scheduling the worker if there is nothing to do.Reported-by Mike Galbraith <[email protected]>Signed-off-by Michal Hocko <[email protected]>--- mm/vmstat.c | 60 ++++++++++++++++++++++++++++++++++++++++----------- --------- 1 file changed, 40 insertions(+), 20 deletions(-)mm/vmstat.c +2
-0@@ -1396,10 +1396,15 @@ static void vmstat_update(struct work_struct *w) * Counters were updated so we expect more updates * to occur in the future. Keep on running the * update worker thread. + * If we were marked on cpu_stat_off clear the flag + * so that vmstat_shepherd doesn't schedule usagain.
mm/vmstat.c +0
-3*/ -queue_delayed_work_on (smp_processor_id (), vmstat_wq, -this_cpu_ptr (&vmstat_work), - round_jiffies_relative(sysctl_stat_interval));
mm/vmstat.c +3
-0+if (!cpumask_test_and_clear_cpu (smp_processor_id (), + cpu_stat_off)) { + queue_delayed_work_on(smp_processor_id(),
vmstat_wq,mm/vmstat.c +2
-0+this_cpu_ptr (&vmstat_work), + round_jiffies_relative(sysctl_stat_interval));
mm/vmstat.c +1
-0+ } }else {/* * We did not update any counters so the app may be in
mm/vmstat.c +0
-0@@ -1417,18 +1422,6 @@ static void vmstat_update(struct work_struct *w) * until the diffs stay at zero. The function is used by NOHZ andcan only be
* invoked when tick processing is not active.
*/mm/vmstat.c +0
-7-void quiet_vmstat(void) -{ - if (system_state != SYSTEM_RUNNING) - return; - - do { - if (!cpumask_test_and_set_cpu(smp_processor_id(),
cpu_stat_off))mm/vmstat.c +0
-1- cancel_delayed_work(this_cpu_ptr(&vmstat_work));
mm/vmstat.c +17
-4- - }while (refresh_cpu_vm_stats (false)); -} -/* * Check if the diffs for a certain cpu indicate that * an update is needed. @@ -1452,6 +1445,30 @@ static bool need_update(int cpu)return false; } +void quiet_vmstat (void ) +{ +if (system_state !=SYSTEM_RUNNING ) +return ; + +/* +* If we are already in hands of the shepherd then there +* is nothing for us to do here. +*/ +if (cpumask_test_and_set_cpu (smp_processor_id (), cpu_stat_off)) +return ; + +if (!need_update (smp_processor_id ())) +return ; + +/* +* Just refresh counters and do not care about the pending delayed
mm/vmstat.c +1
-0+* vmstat_update. It doesn't fire that often to matter and canceling
mm/vmstat.c +6
-0+* it would be too expensive from this path. +* vmstat_shepherd will take care about that for us. +*/ +refresh_cpu_vm_stats (false); +} +/* * Shepherd worker thread that checks the @@ -1470,11 +1487,14 @@ static void vmstat_shepherd(struct work_struct *w)get_online_cpus ();/* Check processors whose vmstat worker threads have been disabled */
mm/vmstat.c +0
-2for_each_cpu (cpu, cpu_stat_off) -if (need_update (cpu) && - cpumask_test_and_clear_cpu(cpu,
cpu_stat_off))mm/vmstat.c +2
-3- -queue_delayed_work_on (cpu, vmstat_wq, - &per_cpu (vmstat_work, cpu),0 ); +if (need_update (cpu)) { + if (cpumask_test_and_clear_cpu(cpu,
cpu_stat_off))mm/vmstat.c +1
-0+ queue_delayed_work_on(cpu,
vmstat_wq,mm/vmstat.c +1
-0+ &per_cpu(vmstat_work, cpu),0);
mm/vmstat.c +3
-0+ }else { +cpumask_set_cpu (cpu, cpu_stat_off); + cancel_delayed_work(this_cpu_ptr(&vmstat_work));
mm/vmstat.c +1
-0+ }put_online_cpus ();Signature · 2 lines
-- 2.7.0.rc3
Re: [PATCH] mm, vmstat: make quiet_vmstat lighter (was: Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable) again and shut down on idle)
In reply to Michal HockoQuoted 7 lines · Michal Hocko
This is caused by
0eb77e988032("vmstat: makevmstat_updaterdeferrable again and shut down on idle") which has placedquiet_vmstatintocpu_idle_loop. The main reason here seems to be that the idle entry has to get over all zones and perform atomic operations for each vmstat entry even though there might be no per cpu diffs. This is a pointless overhead for _each_ idle entry.
Quoted 5 lines
Make sure that
quiet_vmstatis as light as possible.First of all it doesn't make any sense to do any local sync if the
current cpu is already set inoncpu_stat_offbecausevmstat_updateputs itself there only if there is nothing to do.
But there might be something to do that happened in the meantime because counters were incremented. It needs to be checked. The shepherd can do that but it will delay the folding of diffs for awhile.
Quoted 13 lines
mm/vmstat.c +13
-0+void quiet_vmstat (void ) +{ +if (system_state !=SYSTEM_RUNNING ) +return ; + +/* +* If we are already in hands of the shepherd then there +* is nothing for us to do here. +*/ +if (cpumask_test_and_set_cpu (smp_processor_id (), cpu_stat_off)) +return ; + +if (!need_update (smp_processor_id ()))
mm/vmstat.c static bool need_update(int cpu) · line 1470
Quoted 10 lines
mm/vmstat.c +10
-0+return ; + +/* +* Just refresh counters and do not care about the pending delayed +* vmstat_update. It doesn't fire that often to matter and canceling +* it would be too expensive from this path. +* vmstat_shepherd will take care about that for us. +*/ +refresh_cpu_vm_stats (false); +}
The problem here is that there will be an additional tick generated on idle. This is an issue for power because now the processor has to needlessly wake up again, do tick processing etc just to effectively do a cancel_delayed_work().
The cancelling of the work request is required to avoid this additonal tick.
Quoted 5 lines
mm/vmstat.c +0
-1@@ -1470,11 +1487,14 @@ static void vmstat_shepherd(struct work_struct *w)get_online_cpus ();/* Check processors whose vmstat worker threads have been disabled */ for_each_cpu (cpu, cpu_stat_off) - if (need_update(cpu) &&
mm/vmstat.c static void vmstat_shepherd(struct work_struct *w) · line 1495
Quoted 10 lines
mm/vmstat.c +6
-4- cpumask_test_and_clear_cpu(cpu, cpu_stat_off)) - -queue_delayed_work_on (cpu, vmstat_wq, - &per_cpu (vmstat_work, cpu),0 ); +if (need_update (cpu)) { +if (cpumask_test_and_clear_cpu (cpu, cpu_stat_off)) +queue_delayed_work_on (cpu, vmstat_wq, + &per_cpu (vmstat_work, cpu),0 ); + }else { +cpumask_set_cpu (cpu, cpu_stat_off);
Umm the flag is already set right? This just causes a bouncing cacheline since we are accessing a global set of cpus,. Drop this line?
mm/vmstat.c static void vmstat_shepherd(struct work_struct *w) · line 1496
Quoted 1 line
mm/vmstat.c +1
-0+cancel_delayed_work (this_cpu_ptr (&vmstat_work));
Ok so this is to move the cancel into the shepherd.
Aside from the two issues mentioned this looks good.
Re: [PATCH] mm, vmstat: make quiet_vmstat lighter (was: Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable) again and shut down on idle)
Quoted 11 lines · Christoph Lameter
Quoted 9 lines · Michal Hocko
This is caused by
0eb77e988032("vmstat: makevmstat_updaterdeferrable again and shut down on idle") which has placedquiet_vmstatintocpu_idle_loop. The main reason here seems to be that the idle entry has to get over all zones and perform atomic operations for each vmstat entry even though there might be no per cpu diffs. This is a pointless overhead for _each_ idle entry.Make sure that
quiet_vmstatis as light as possible.
Quoted 7 lines
Quoted 3 lines
First of all it doesn't make any sense to do any local sync if the
current cpu is already set inoncpu_stat_offbecausevmstat_updateputs itself there only if there is nothing to do.But there might be something to do that happened in the meantime because counters were incremented. It needs to be checked. The shepherd can do that but it will delay the folding of diffs for awhile.
shepherd ticks with the same period so it should run relatively soon. What is important here the test_and_set is the fast path which will trigger a lot of the current CPU is mostly running in the userspace.
Quoted 29 lines
Quoted 13 lines
(quoted diff) +13
-0+void quiet_vmstat(void) +{ + if (system_state != SYSTEM_RUNNING) + return; + + /* + * If we are already in hands of the shepherd then there + * is nothing for us to do here. + */ + if (cpumask_test_and_set_cpu(smp_processor_id(), cpu_stat_off)) + return; + + if (!need_update(smp_processor_id()))Review
Quoted 10 lines
(quoted diff) +10
-0+ return; + + /* + * Just refresh counters and do not care about the pending delayed + * vmstat_update. It doesn't fire that often to matter and canceling + * it would be too expensive from this path. + * vmstat_shepherd will take care about that for us. + */ + refresh_cpu_vm_stats(false); +}The problem here is that there will be an additional tick generated on idle. This is an issue for power because now the processor has to needlessly wake up again, do tick processing etc just to effectively do a
cancel_delayed_work().
Quoted 2 lines
The cancelling of the work request is required to avoid this additonal tick.
Yes but as mentioned in the changelog calling cancel_delayed_work is quite expensive to be run on each idle entry slow path. In addition it is RT unfriendly. So considering a single tick vs. other reasons I think this is a reasonable compromise.
Quoted 10 lines
Quoted 10 lines
(quoted diff) +1
-5@@ -1470,11 +1487,14 @@ static void vmstat_shepherd(struct work_struct *w) get_online_cpus(); /* Check processors whose vmstat worker threads have been disabled */ for_each_cpu(cpu, cpu_stat_off) - if (need_update(cpu) && - cpumask_test_and_clear_cpu(cpu, cpu_stat_off)) - - queue_delayed_work_on(cpu, vmstat_wq, - &per_cpu(vmstat_work, cpu), 0); + if (need_update(cpu)) {
Quoted 8 lines
Review static void vmstat_shepherd(struct work_struct *w) · line 1495
Quoted 5 lines
(quoted diff) +5
-0+ if (cpumask_test_and_clear_cpu(cpu, cpu_stat_off)) + queue_delayed_work_on(cpu, vmstat_wq, + &per_cpu(vmstat_work, cpu), 0); + } else { + cpumask_set_cpu(cpu, cpu_stat_off);Umm the flag is already set right? This just causes a bouncing cacheline since we are accessing a global set of cpus,. Drop this line?
Ohh, right you are. I will remove this.
Quoted 4 lines
Review static void vmstat_shepherd(struct work_struct *w) · line 1496
Quoted 2 lines
(quoted diff) +1
-0+ cancel_delayed_work(this_cpu_ptr(&vmstat_work));Ok so this is to move the cancel into the shepherd.
Yeah, I have added a comment to explain that.
Quoted 1 line
Aside from the two issues mentioned this looks good.
Updated version:
---
Quoted 1 line
From
1d0febff044382b88ff44d245b5afe1a6e402c62Mon Sep 17 00:00:00 2001
From: Michal Hocko <[email protected]>
Date: Wed, 27 Jan 2016 17:24:22 +0100
Subject: [PATCH] mm, vmstat: make quiet_vmstat lighter
Mike has reported a considerable overhead of refresh_cpu_vm_stats from the idle entry during pipe test:
12.89% [kernel] [k] refresh_cpu_vm_stats.isra.12
4.75% [kernel] [k] __schedule
4.70% [kernel] [k] mutex_unlock
3.14% [kernel] [k] __switch_toThis is caused by 0eb77e988032 ("vmstat: make vmstat_updater deferrable again and shut down on idle") which has placed quiet_vmstat into cpu_idle_loop. The main reason here seems to be that the idle entry has to get over all zones and perform atomic operations for each vmstat entry even though there might be no per cpu diffs. This is a pointless overhead for _each_ idle entry.
Make sure that quiet_vmstat is as light as possible.
First of all it doesn't make any sense to do any local sync if the
current cpu is already set in oncpu_stat_off because vmstat_update puts itself there only if there is nothing to do.
Then we can check need_update which should be a cheap way to check for potential per-cpu diffs and only then do refresh_cpu_vm_stats.
The original patch also did cancel_delayed_work which we are not doing here. There are two reasons for that. Firstly cancel_delayed_work from idle context will blow up on RT kernels (reported by Mike):
[ 2.279582] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.5.0-rt3 #7 [ 2.280444] Hardware name: MEDION MS-7848/MS-7848, BIOS M7848W08.20C 09/23/2013 [ 2.281316] ffff88040b00d640 ffff88040b01fe10 ffffffff812d20e2 0000000000000000 [ 2.282202] ffff88040b01fe30 ffffffff81081095 ffff88041ec4cee0 ffff88041ec501e0 [ 2.283073] ffff88040b01fe48 ffffffff815ff910 ffff88041ec4cee0 ffff88040b01fe88 [ 2.283941] Call Trace: [ 2.284797] [<ffffffff812d20e2>] dump_stack+0x49/0x67 [ 2.285658] [<ffffffff81081095>] ___might_sleep+0xf5/0x180 [ 2.286521] [<ffffffff815ff910>] rt_spin_lock+0x20/0x50 [ 2.287382] [<ffffffff81075919>] try_to_grab_pending+0x69/0x240 [ 2.288239] [<ffffffff81075b16>] cancel_delayed_work+0x26/0xe0 [ 2.289094] [<ffffffff8115ec05>] quiet_vmstat+0x75/0xa0 [ 2.289949] [<ffffffff8109ab38>] cpu_idle_loop+0x38/0x3e0 [ 2.290800] [<ffffffff8109aef3>] cpu_startup_entry+0x13/0x20 [ 2.291647] [<ffffffff81036164>] start_secondary+0x114/0x140
And secondly, even on !RT kernels it might add some non trivial overhead which is not necessary. Even if the vmstat worker wakes up and preempts idle then it will be most likely a single shot noop because the stats were already synced and so it would end up on the oncpu_stat_off anyway. We just need to teach both vmstat_shepherd and vmstat_update to stop scheduling the worker if there is nothing to do.
--- mm/vmstat.c | 64 ++++++++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 44 insertions(+), 20 deletions(-)
mm/vmstat.c +44 -20
@@ -1396,10 +1396,15 @@ static void vmstat_update(struct work_struct *w)* Counters were updated so we expect more updates * to occur in the future. Keep on running the * update worker thread. +* If we were marked on cpu_stat_off clear the flag +* so that vmstat_shepherd doesn't schedule us again. */ -queue_delayed_work_on (smp_processor_id (), vmstat_wq, -this_cpu_ptr (&vmstat_work), -round_jiffies_relative (sysctl_stat_interval)); +if (!cpumask_test_and_clear_cpu (smp_processor_id (), + cpu_stat_off)) { +queue_delayed_work_on (smp_processor_id (), vmstat_wq, +this_cpu_ptr (&vmstat_work), +round_jiffies_relative (sysctl_stat_interval)); + } }else {/* * We did not update any counters so the app may be in
@@ -1417,18 +1422,6 @@ static void vmstat_update(struct work_struct *w)* until the diffs stay at zero. The function is used by NOHZ and can only be * invoked when tick processing is not active. */ -void quiet_vmstat (void ) -{ -if (system_state !=SYSTEM_RUNNING ) -return ; - -do { -if (!cpumask_test_and_set_cpu (smp_processor_id (), cpu_stat_off)) -cancel_delayed_work (this_cpu_ptr (&vmstat_work)); - - }while (refresh_cpu_vm_stats (false)); -} -/* * Check if the diffs for a certain cpu indicate that * an update is needed.
@@ -1452,6 +1445,30 @@ static bool need_update(int cpu)return false; } +void quiet_vmstat (void ) +{ +if (system_state !=SYSTEM_RUNNING ) +return ; + +/* +* If we are already in hands of the shepherd then there +* is nothing for us to do here. +*/ +if (cpumask_test_and_set_cpu (smp_processor_id (), cpu_stat_off)) +return ; + +if (!need_update (smp_processor_id ())) +return ; + +/* +* Just refresh counters and do not care about the pending delayed +* vmstat_update. It doesn't fire that often to matter and canceling +* it would be too expensive from this path. +* vmstat_shepherd will take care about that for us. +*/ +refresh_cpu_vm_stats (false); +} +/* * Shepherd worker thread that checks the
@@ -1470,11 +1487,18 @@ static void vmstat_shepherd(struct work_struct *w)get_online_cpus ();/* Check processors whose vmstat worker threads have been disabled */ for_each_cpu (cpu, cpu_stat_off) -if (need_update (cpu) && -cpumask_test_and_clear_cpu (cpu, cpu_stat_off)) - -queue_delayed_work_on (cpu, vmstat_wq, - &per_cpu (vmstat_work, cpu),0 ); +if (need_update (cpu)) { +if (cpumask_test_and_clear_cpu (cpu, cpu_stat_off)) +queue_delayed_work_on (cpu, vmstat_wq, + &per_cpu (vmstat_work, cpu),0 ); + }else { +/* +* Cancel the work if quiet_vmstat has put this +* cpu on cpu_stat_off because the work item might +* be still scheduled +*/ +cancel_delayed_work (this_cpu_ptr (&vmstat_work)); + }put_online_cpus ();
Signature · 6 lines
2.7.0.rc3 -- Michal Hocko SUSE Labs
Re: [PATCH] mm, vmstat: make quiet_vmstat lighter (was: Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable) again and shut down on idle)
In reply to Christoph LameterQuoted 1 line · Christoph Lameter
On Wed, 27 Jan 2016, Michal Hocko wrote:
[...]
Quoted 20 lines
Quoted 20 lines
(quoted diff) +20
-0+void quiet_vmstat(void) +{ + if (system_state != SYSTEM_RUNNING) + return; + + /* + * If we are already in hands of the shepherd then there + * is nothing for us to do here. + */ + if (cpumask_test_and_set_cpu(smp_processor_id(), cpu_stat_off)) + return; + + if (!need_update(smp_processor_id())) + return; + + /* + * Just refresh counters and do not care about the pending delayed + * vmstat_update. It doesn't fire that often to matter and canceling + * it would be too expensive from this path. + * vmstat_shepherd will take care about that for us.
Quoted 8 lines
Review
Quoted 3 lines
(quoted diff) +3
-0+ */ + refresh_cpu_vm_stats(false); +}The problem here is that there will be an additional tick generated on idle. This is an issue for power because now the processor has to needlessly wake up again, do tick processing etc just to effectively do a
cancel_delayed_work().
Thinking about it some more, making vmstat_update deferrable should help to not interrupt idle no?
---
mm/vmstat.c +1 -1
@@ -1512,7 +1512,7 @@ static void __init start_shepherd_timer(void)int cpu;for_each_possible_cpu (cpu) -INIT_DELAYED_WORK (per_cpu_ptr (&vmstat_work, cpu), +INIT_DEFERRABLE_WORK (per_cpu_ptr (&vmstat_work, cpu), vmstat_update);if (!alloc_cpumask_var (&cpu_stat_off,GFP_KERNEL ))
Signature · 2 lines
Michal Hocko SUSE Labs
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
In reply to Christoph LameterQuoted 11 lines · Christoph Lameter
Quoted 7 lines · Mike Galbraith
Quoted 2 lines
Since we (the NOHZ people) care mostly about NOHZ then lets restrict that to the NOHZ mode. Then it should not affect your load.
Tons of folks do have
NO_HZenabled (including me). Isn't there a spot somewhere inNO_HZ_FULLcode where it can take up residence? (one with a tad lower maximum call frequency would be good, anohz_fullcpu isn't necessarily being used for pure compute exclusively)Frederic, any idea on where to put
quiet_vmstat()?
I guess those who want this also want task isolation, right? And Chris integrates it in his patchset.
PATCH 1/1 vmstat: make vmstat_update deferrable (was: Re: [PATCH] mm, vmstat: make quiet_vmstat lighter) path cycle muncher (vmstat: make vmstat_updater deferrable) again and shut down on idle)
In reply to Michal HockoQuoted 32 lines · Michal Hocko
Quoted 1 line · Christoph Lameter
On Wed, 27 Jan 2016, Michal Hocko wrote:
[...]
Quoted 20 lines
Quoted 20 lines
(quoted diff) +20
-0+void quiet_vmstat(void) +{ + if (system_state != SYSTEM_RUNNING) + return; + + /* + * If we are already in hands of the shepherd then there + * is nothing for us to do here. + */ + if (cpumask_test_and_set_cpu(smp_processor_id(), cpu_stat_off)) + return; + + if (!need_update(smp_processor_id())) + return; + + /* + * Just refresh counters and do not care about the pending delayed + * vmstat_update. It doesn't fire that often to matter and canceling + * it would be too expensive from this path. + * vmstat_shepherd will take care about that for us.Quoted 8 lines
Review
Quoted 3 lines
(quoted diff) +3
-0+ */ + refresh_cpu_vm_stats(false); +}The problem here is that there will be an additional tick generated on idle. This is an issue for power because now the processor has to needlessly wake up again, do tick processing etc just to effectively do a
cancel_delayed_work().
Quoted 2 lines
Thinking about it some more, making
vmstat_updatedeferrable should help to not interrupt idle no?
With the full changelog
Quoted 1 line
From
89858bf594c020f065b4f415a12d75bee64a7954Mon Sep 17 00:00:00 2001
From: Michal Hocko <[email protected]>
Date: Thu, 28 Jan 2016 16:31:43 +0100
Subject: [PATCH] vmstat: make vmstat_update deferrable
0eb77e988032 ("vmstat: make vmstat_updater deferrable again and shut down on idle") made vmstat_shepherd deferrable. vmstat_update itself is still using standard timer which might interrupt idle task. This is possible because "mm, vmstat: make quiet_vmstat lighter" removed cancel_delayed_work from the quiet_vmstat. Change vmstat_work to use DEFERRABLE_WORK to prevent from pointless wakeups from the idle context.
--- mm/vmstat.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/vmstat.c +1 -1
@@ -1512,7 +1512,7 @@ static void __init start_shepherd_timer(void)int cpu;for_each_possible_cpu (cpu) -INIT_DELAYED_WORK (per_cpu_ptr (&vmstat_work, cpu), +INIT_DEFERRABLE_WORK (per_cpu_ptr (&vmstat_work, cpu), vmstat_update);if (!alloc_cpumask_var (&cpu_stat_off,GFP_KERNEL ))
Signature · 6 lines
2.7.0.rc3 -- Michal Hocko SUSE Labs
Re: [PATCH] mm, vmstat: make quiet_vmstat lighter (was: Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable) again and shut down on idle)
In reply to Michal HockoQuoted 5 lines · Michal Hocko
mm/vmstat.c +5
-0+void quiet_vmstat (void ) +{ +if (system_state !=SYSTEM_RUNNING ) +return ; +
mm/vmstat.c static bool need_update(int cpu) · line 1462
Quoted 10 lines
mm/vmstat.c +10
-0+/* +* If we are already in hands of the shepherd then there +* is nothing for us to do here. +*/ +if (cpumask_test_and_set_cpu (smp_processor_id (), cpu_stat_off)) +return ; + +if (!need_update (smp_processor_id ())) +return ; +
You can drop the need_update check. refresh_cpu_vm_stats() does the same checks in a more efficient way. If you keep this the checks will performed twice.
Quoted 10 lines
mm/vmstat.c +9
-0+/* +* Just refresh counters and do not care about the pending delayed +* vmstat_update. It doesn't fire that often to matter and canceling +* it would be too expensive from this path. +* vmstat_shepherd will take care about that for us. +*/ +refresh_cpu_vm_stats (false); +} +
Re: [PATCH] mm, vmstat: make quiet_vmstat lighter (was: Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable) again and shut down on idle)
In reply to Michal HockoQuoted 2 lines · Michal Hocko
Thinking about it some more, making
vmstat_updatedeferrable should help to not interrupt idle no?
It will align with the tick then. Yes indeed this should be done.
Quoted 15 lines
---
mm/vmstat.c +1
-1@@ -1512,7 +1512,7 @@ static void __init start_shepherd_timer(void)int cpu;for_each_possible_cpu (cpu) -INIT_DELAYED_WORK (per_cpu_ptr (&vmstat_work, cpu), +INIT_DEFERRABLE_WORK (per_cpu_ptr (&vmstat_work, cpu), vmstat_update);if (!alloc_cpumask_var (&cpu_stat_off,GFP_KERNEL ))
Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable again and shut down on idle)
In reply to Frederic WeisbeckerQuoted 4 lines · Frederic Weisbecker
Quoted 1 line
Frederic, any idea on where to put
quiet_vmstat()?I guess those who want this also want task isolation, right? And Chris integrates it in his patchset.
Ok.
Re: [PATCH] vmstat: make vmstat_update deferrable (was: Re: [PATCH] mm, vmstat: make quiet_vmstat lighter) path cycle muncher (vmstat: make vmstat_updater deferrable) again and shut down on idle)
In reply to Michal HockoQuoted 6 lines · Michal Hocko
down on idle") made
vmstat_shepherddeferrable.vmstat_updateitself is still using standard timer which might interrupt idle task. This is possible because "mm, vmstat: makequiet_vmstatlighter" removedcancel_delayed_workfrom thequiet_vmstat. Changevmstat_workto useDEFERRABLE_WORKto prevent from pointless wakeups from the idle context.
Re: [PATCH] mm, vmstat: make quiet_vmstat lighter (was: Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable) again and shut down on idle)
In reply to Christoph LameterQuoted 13 lines · Christoph Lameter
Quoted 11 lines · Michal Hocko
mm/vmstat.c +11
-0+void quiet_vmstat (void ) +{ +if (system_state !=SYSTEM_RUNNING ) +return ; + +/* +* If we are already in hands of the shepherd then there +* is nothing for us to do here. +*/ +if (cpumask_test_and_set_cpu (smp_processor_id (), cpu_stat_off)) +return ;
Quoted 8 lines
mm/vmstat.c static bool need_update(int cpu) · line 1462
Quoted 4 lines
mm/vmstat.c +4
-0+ +if (!need_update (smp_processor_id ())) +return ; +You can drop the
need_updatecheck.refresh_cpu_vm_stats()does the same checks in a more efficient way. If you keep this the checks will performed twice.
refresh_cpu_vm_stats does this_cpu_xchg for each counter. Is it really more effective than memchr_inv?
Signature · 2 lines
Michal Hocko SUSE Labs
Christoph Lameter
Acked-byRe: [PATCH] mm, vmstat: make quiet_vmstat lighter (was: Re: fast path cycle muncher (vmstat: make vmstat_updater deferrable) again and shut down on idle)
Quoted 6 lines · Michal Hocko
Quoted 3 lines
You can drop the
need_updatecheck.refresh_cpu_vm_stats()does the same checks in a more efficient way. If you keep this the checks will performed twice.
refresh_cpu_vm_statsdoesthis_cpu_xchgfor each counter. Is it really more effective thanmemchr_inv?
Ok makes sense.