Skip to contentSkip to conversation

kswapd using __this_cpu_add() in preemptible code

DISCUSSION Sergey Senozhatsky 4 participants 5 messages Mar 18, 2014 – Mar 20, 2014

Subsystems (3)
Memory management VFS MM core
lore
Share & tools
lore

Sergey Senozhatsky

author
#0 lore raw

Hello gentlemen,

Commit 589a606f9539663f162e4a110d117527833b58a4 ("percpu: add preemption checks to __this_cpu ops") added preempt check to used in __count_vm_events() __this_cpu ops, causing the following kswapd warning:

 BUG: using __this_cpu_add() in preemptible [00000000] code: kswapd0/56
 caller is __this_cpu_preempt_check+0x2b/0x2d
 Call Trace:
 [<ffffffff813b8d4d>] dump_stack+0x4e/0x7a
 [<ffffffff8121366f>] check_preemption_disabled+0xce/0xdd
 [<ffffffff812136bb>] __this_cpu_preempt_check+0x2b/0x2d
 [<ffffffff810f622e>] inode_lru_isolate+0xed/0x197
 [<ffffffff810be43c>] list_lru_walk_node+0x7b/0x14c
 [<ffffffff810f6141>] ? iput+0x131/0x131
 [<ffffffff810f681f>] prune_icache_sb+0x35/0x4c
 [<ffffffff810e3951>] super_cache_scan+0xe3/0x143
 [<ffffffff810b1301>] shrink_slab_node+0x103/0x16f
 [<ffffffff810b19fd>] shrink_slab+0x75/0xe4
 [<ffffffff810b3f3d>] balance_pgdat+0x2fa/0x47f
 [<ffffffff810b4395>] kswapd+0x2d3/0x2fd
 [<ffffffff81068049>] ? __wake_up_sync+0xd/0xd
 [<ffffffff810b40c2>] ? balance_pgdat+0x47f/0x47f
 [<ffffffff81051e75>] kthread+0xd6/0xde
 [<ffffffff81051d9f>] ? kthread_create_on_node+0x162/0x162
 [<ffffffff813be5bc>] ret_from_fork+0x7c/0xb0
 [<ffffffff81051d9f>] ? kthread_create_on_node+0x162/0x162

list_lru_walk_node() seems to be the only place where __count_vm_events() called with preemption enabled. remaining __count_vm_events() and __count_vm_event() calls are done with preemption disabled (unless I overlooked something).

	-ss

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

maintainer
#1 lore raw
Quoted 29 lines · On Tue, 18 Mar 2014 21:53:30 +0300 Sergey Senozhatsky…

Hello gentlemen,

Commit 589a606f9539663f162e4a110d117527833b58a4 ("percpu: add preemption checks to __this_cpu ops") added preempt check to used in __count_vm_events() __this_cpu ops, causing the following kswapd warning:

 BUG: using __this_cpu_add() in preemptible [00000000] code: kswapd0/56
 caller is __this_cpu_preempt_check+0x2b/0x2d
 Call Trace:
 [<ffffffff813b8d4d>] dump_stack+0x4e/0x7a
 [<ffffffff8121366f>] check_preemption_disabled+0xce/0xdd
 [<ffffffff812136bb>] __this_cpu_preempt_check+0x2b/0x2d
 [<ffffffff810f622e>] inode_lru_isolate+0xed/0x197
 [<ffffffff810be43c>] list_lru_walk_node+0x7b/0x14c
 [<ffffffff810f6141>] ? iput+0x131/0x131
 [<ffffffff810f681f>] prune_icache_sb+0x35/0x4c
 [<ffffffff810e3951>] super_cache_scan+0xe3/0x143
 [<ffffffff810b1301>] shrink_slab_node+0x103/0x16f
 [<ffffffff810b19fd>] shrink_slab+0x75/0xe4
 [<ffffffff810b3f3d>] balance_pgdat+0x2fa/0x47f
 [<ffffffff810b4395>] kswapd+0x2d3/0x2fd
 [<ffffffff81068049>] ? __wake_up_sync+0xd/0xd
 [<ffffffff810b40c2>] ? balance_pgdat+0x47f/0x47f
 [<ffffffff81051e75>] kthread+0xd6/0xde
 [<ffffffff81051d9f>] ? kthread_create_on_node+0x162/0x162
 [<ffffffff813be5bc>] ret_from_fork+0x7c/0xb0
 [<ffffffff81051d9f>] ? kthread_create_on_node+0x162/0x162
Quoted 4 lines

list_lru_walk_node() seems to be the only place where __count_vm_events() called with preemption enabled. remaining __count_vm_events() and __count_vm_event() calls are done with preemption disabled (unless I overlooked something).

Christoph caught one. How does this look?

From: Andrew Morton <[email protected]>
Subject: fs/inode.c:inode_lru_isolate(): use atomic count_vm_events()

"percpu: add preemption checks to __this_cpu ops" added preempt check to used in __count_vm_events() __this_cpu ops, causing the following kswapd warning:

 BUG: using __this_cpu_add() in preemptible [00000000] code: kswapd0/56
 caller is __this_cpu_preempt_check+0x2b/0x2d
 Call Trace:
 [<ffffffff813b8d4d>] dump_stack+0x4e/0x7a
 [<ffffffff8121366f>] check_preemption_disabled+0xce/0xdd
 [<ffffffff812136bb>] __this_cpu_preempt_check+0x2b/0x2d
 [<ffffffff810f622e>] inode_lru_isolate+0xed/0x197
 [<ffffffff810be43c>] list_lru_walk_node+0x7b/0x14c
 [<ffffffff810f6141>] ? iput+0x131/0x131
 [<ffffffff810f681f>] prune_icache_sb+0x35/0x4c

Switch from __count_vm_events() to the preempt-safe count_vm_events().

Reported-by Sergey Senozhatsky <[email protected]>
Cc Christoph Lameter <[email protected]>
Signed-off-by Andrew Morton <[email protected]>

---

 fs/inode.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
fs/inode.c renamed from fs/inode.c~fs-inodec-inode_lru_isolate-use-atomic-count_vm_events +2 -2
diff -puN fs/inode.c~fs-inodec-inode_lru_isolate-use-atomic-count_vm_events fs/inode.c
@@ -722,9 +722,9 @@ inode_lru_isolate(struct list_head *item
 			unsigned long reap;
 			reap = invalidate_mapping_pages(&inode->i_data, 0, -1);
 			if (current_is_kswapd())
-				__count_vm_events(KSWAPD_INODESTEAL, reap);
+				count_vm_events(KSWAPD_INODESTEAL, reap);
 			else
-				__count_vm_events(PGINODESTEAL, reap);
+				count_vm_events(PGINODESTEAL, reap);
 			if (current->reclaim_state)
 				current->reclaim_state->reclaimed_slab += reap;
 		}

_

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author Tested-by
#2 lore raw
Quoted 31 lines · Andrew Morton
Quoted 29 lines · On Tue, 18 Mar 2014 21:53:30 +0300 Sergey Senozhatsky…

Hello gentlemen,

Commit 589a606f9539663f162e4a110d117527833b58a4 ("percpu: add preemption checks to __this_cpu ops") added preempt check to used in __count_vm_events() __this_cpu ops, causing the following kswapd warning:

 BUG: using __this_cpu_add() in preemptible [00000000] code: kswapd0/56
 caller is __this_cpu_preempt_check+0x2b/0x2d
 Call Trace:
 [<ffffffff813b8d4d>] dump_stack+0x4e/0x7a
 [<ffffffff8121366f>] check_preemption_disabled+0xce/0xdd
 [<ffffffff812136bb>] __this_cpu_preempt_check+0x2b/0x2d
 [<ffffffff810f622e>] inode_lru_isolate+0xed/0x197
 [<ffffffff810be43c>] list_lru_walk_node+0x7b/0x14c
 [<ffffffff810f6141>] ? iput+0x131/0x131
 [<ffffffff810f681f>] prune_icache_sb+0x35/0x4c
 [<ffffffff810e3951>] super_cache_scan+0xe3/0x143
 [<ffffffff810b1301>] shrink_slab_node+0x103/0x16f
 [<ffffffff810b19fd>] shrink_slab+0x75/0xe4
 [<ffffffff810b3f3d>] balance_pgdat+0x2fa/0x47f
 [<ffffffff810b4395>] kswapd+0x2d3/0x2fd
 [<ffffffff81068049>] ? __wake_up_sync+0xd/0xd
 [<ffffffff810b40c2>] ? balance_pgdat+0x47f/0x47f
 [<ffffffff81051e75>] kthread+0xd6/0xde
 [<ffffffff81051d9f>] ? kthread_create_on_node+0x162/0x162
 [<ffffffff813be5bc>] ret_from_fork+0x7c/0xb0
 [<ffffffff81051d9f>] ? kthread_create_on_node+0x162/0x162
Quoted 7 lines
Quoted 4 lines

list_lru_walk_node() seems to be the only place where __count_vm_events() called with preemption enabled. remaining __count_vm_events() and __count_vm_event() calls are done with preemption disabled (unless I overlooked something).

Christoph caught one. How does this look?

Thank you.

Tested-by Sergey Senozhatsky <[email protected]>
	-ss
Quoted 45 lines

From: Andrew Morton <[email protected]>
Subject: fs/inode.c:inode_lru_isolate(): use atomic count_vm_events()

"percpu: add preemption checks to __this_cpu ops" added preempt check to used in __count_vm_events() __this_cpu ops, causing the following kswapd warning:

 BUG: using __this_cpu_add() in preemptible [00000000] code: kswapd0/56
 caller is __this_cpu_preempt_check+0x2b/0x2d
 Call Trace:
 [<ffffffff813b8d4d>] dump_stack+0x4e/0x7a
 [<ffffffff8121366f>] check_preemption_disabled+0xce/0xdd
 [<ffffffff812136bb>] __this_cpu_preempt_check+0x2b/0x2d
 [<ffffffff810f622e>] inode_lru_isolate+0xed/0x197
 [<ffffffff810be43c>] list_lru_walk_node+0x7b/0x14c
 [<ffffffff810f6141>] ? iput+0x131/0x131
 [<ffffffff810f681f>] prune_icache_sb+0x35/0x4c

Switch from __count_vm_events() to the preempt-safe count_vm_events().

Reported-by Sergey Senozhatsky <[email protected]>
Cc Christoph Lameter <[email protected]>
Signed-off-by Andrew Morton <[email protected]>

---

 fs/inode.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff -puN fs/inode.c~fs-inodec-inode_lru_isolate-use-atomic-count_vm_events fs/inode.c

fs/inode.c renamed from fs/inode.c~fs-inodec-inode_lru_isolate-use-atomic-count_vm_events +2 -2
@@ -722,9 +722,9 @@ inode_lru_isolate(struct list_head *item
 			unsigned long reap;
 			reap = invalidate_mapping_pages(&inode->i_data, 0, -1);
 			if (current_is_kswapd())
-				__count_vm_events(KSWAPD_INODESTEAL, reap);
+				count_vm_events(KSWAPD_INODESTEAL, reap);
 			else
-				__count_vm_events(PGINODESTEAL, reap);
+				count_vm_events(PGINODESTEAL, reap);
 			if (current->reclaim_state)
 				current->reclaim_state->reclaimed_slab += reap;
 		}
Quoted 2 lines

_

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Acked-by
#3 lore raw
In reply to Andrew Morton
Quoted 73 lines · Andrew Morton
Quoted 29 lines · On Tue, 18 Mar 2014 21:53:30 +0300 Sergey Senozhatsky…

Hello gentlemen,

Commit 589a606f9539663f162e4a110d117527833b58a4 ("percpu: add preemption checks to __this_cpu ops") added preempt check to used in __count_vm_events() __this_cpu ops, causing the following kswapd warning:

 BUG: using __this_cpu_add() in preemptible [00000000] code: kswapd0/56
 caller is __this_cpu_preempt_check+0x2b/0x2d
 Call Trace:
 [<ffffffff813b8d4d>] dump_stack+0x4e/0x7a
 [<ffffffff8121366f>] check_preemption_disabled+0xce/0xdd
 [<ffffffff812136bb>] __this_cpu_preempt_check+0x2b/0x2d
 [<ffffffff810f622e>] inode_lru_isolate+0xed/0x197
 [<ffffffff810be43c>] list_lru_walk_node+0x7b/0x14c
 [<ffffffff810f6141>] ? iput+0x131/0x131
 [<ffffffff810f681f>] prune_icache_sb+0x35/0x4c
 [<ffffffff810e3951>] super_cache_scan+0xe3/0x143
 [<ffffffff810b1301>] shrink_slab_node+0x103/0x16f
 [<ffffffff810b19fd>] shrink_slab+0x75/0xe4
 [<ffffffff810b3f3d>] balance_pgdat+0x2fa/0x47f
 [<ffffffff810b4395>] kswapd+0x2d3/0x2fd
 [<ffffffff81068049>] ? __wake_up_sync+0xd/0xd
 [<ffffffff810b40c2>] ? balance_pgdat+0x47f/0x47f
 [<ffffffff81051e75>] kthread+0xd6/0xde
 [<ffffffff81051d9f>] ? kthread_create_on_node+0x162/0x162
 [<ffffffff813be5bc>] ret_from_fork+0x7c/0xb0
 [<ffffffff81051d9f>] ? kthread_create_on_node+0x162/0x162
Quoted 4 lines

list_lru_walk_node() seems to be the only place where __count_vm_events() called with preemption enabled. remaining __count_vm_events() and __count_vm_event() calls are done with preemption disabled (unless I overlooked something).

Christoph caught one. How does this look?

From: Andrew Morton <[email protected]>
Subject: fs/inode.c:inode_lru_isolate(): use atomic count_vm_events()

"percpu: add preemption checks to __this_cpu ops" added preempt check to used in __count_vm_events() __this_cpu ops, causing the following kswapd warning:

 BUG: using __this_cpu_add() in preemptible [00000000] code: kswapd0/56
 caller is __this_cpu_preempt_check+0x2b/0x2d
 Call Trace:
 [<ffffffff813b8d4d>] dump_stack+0x4e/0x7a
 [<ffffffff8121366f>] check_preemption_disabled+0xce/0xdd
 [<ffffffff812136bb>] __this_cpu_preempt_check+0x2b/0x2d
 [<ffffffff810f622e>] inode_lru_isolate+0xed/0x197
 [<ffffffff810be43c>] list_lru_walk_node+0x7b/0x14c
 [<ffffffff810f6141>] ? iput+0x131/0x131
 [<ffffffff810f681f>] prune_icache_sb+0x35/0x4c

Switch from __count_vm_events() to the preempt-safe count_vm_events().

Reported-by Sergey Senozhatsky <[email protected]>
Cc Christoph Lameter <[email protected]>
Signed-off-by Andrew Morton <[email protected]>

---

 fs/inode.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff -puN fs/inode.c~fs-inodec-inode_lru_isolate-use-atomic-count_vm_events fs/inode.c

fs/inode.c renamed from fs/inode.c~fs-inodec-inode_lru_isolate-use-atomic-count_vm_events +0 -0
@@ -722,9 +722,9 @@ inode_lru_isolate(struct list_head *item
 			unsigned long reap;

fs/inode.c inode_lru_isolate(struct list_head *item · line 730

Quoted 10 lines
fs/inode.c renamed from fs/inode.c~fs-inodec-inode_lru_isolate-use-atomic-count_vm_events +2 -2
 			reap = invalidate_mapping_pages(&inode->i_data, 0, -1);
 			if (current_is_kswapd())
-				__count_vm_events(KSWAPD_INODESTEAL, reap);
+				count_vm_events(KSWAPD_INODESTEAL, reap);
 			else
-				__count_vm_events(PGINODESTEAL, reap);
+				count_vm_events(PGINODESTEAL, reap);
 			if (current->reclaim_state)
 				current->reclaim_state->reclaimed_slab += reap;
 		}
Acked-by Dave Chinner <[email protected]>

Cheers,

Dave.

Signature · 8 lines
Dave Chinner
[email protected]

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#4 lore raw
In reply to Andrew Morton
Quoted 1 line · Andrew Morton

Christoph caught one. How does this look?

The fundamental decision to be made here is if we want the counter overhead coming on platforms that do not have lockless percpu atomics and therefore would require an irq on/off sequence for safe counter increments.

So far we have said that we do allow the counters to be racy for performance sake. Your patch would remove the races.

If we want to keep the races and the performance than we need to change __count_vm_events to use raw_cpu_add instead of __this_cpu_add.

Subject: vmstat: Use raw_cpu_ops to avoid false positives on preemption checks

vm counters are allowed to be racy. Use raw_cpu_ops to avoid preemption checks.

Signed-off-by Christoph Lameter <[email protected]>
include/linux/vmstat.h +2 -2
Index: linux/include/linux/vmstat.h
===================================================================
@@ -29,7 +29,7 @@ DECLARE_PER_CPU(struct vm_event_state, v
 
 static inline void __count_vm_event(enum vm_event_item item)
 {
-	__this_cpu_inc(vm_event_states.event[item]);
+	raw_cpu_inc(vm_event_states.event[item]);
 }
 
 static inline void count_vm_event(enum vm_event_item item)

@@ -39,7 +39,7 @@ static inline void count_vm_event(enum v
 
 static inline void __count_vm_events(enum vm_event_item item, long delta)
 {
-	__this_cpu_add(vm_event_states.event[item], delta);
+	raw_cpu_add(vm_event_states.event[item], delta);
 }
 
 static inline void count_vm_events(enum vm_event_item item, long delta)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>