Skip to contentSkip to conversation

tracing/mm: Don't trace kmem_cache_free on offline cpus

PATCH Superseded Shreyas B. Prabhu 3 participants 17 messages Apr 28, 2015 – May 8, 2015 Tracing

lore
Share & tools
lore
#0 lore raw

PATCH 1/3 tracing/mm: Don't trace kmem_cache_free on offline cpus

Since tracepoints use RCU for protection, they must not be called on offline cpus. trace_kmem_cache_free can be called on an offline cpu in this scenario caught by LOCKDEP:

===============================
[ INFO: suspicious RCU usage. ]
4.1.0-rc1+ #9 Not tainted
-------------------------------
include/trace/events/kmem.h:148 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

RCU used illegally from offline CPU!
rcu_scheduler_active = 1, debug_locks = 1
no locks held by swapper/1/0.

stack backtrace:
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.1.0-rc1+ #9
Call Trace:
[c000001fed2f78f0] [c0000000009dee8c] .dump_stack+0x98/0xd4 (unreliable)
[c000001fed2f7970] [c000000000128d88] .lockdep_rcu_suspicious+0x108/0x170
[c000001fed2f7a00] [c00000000026f924] .kmem_cache_free+0x344/0x4b0
[c000001fed2f7ab0] [c0000000000bd1cc] .__mmdrop+0x4c/0x160
[c000001fed2f7b40] [c0000000001068e0] .idle_task_exit+0xf0/0x100
[c000001fed2f7bc0] [c000000000066948] .pnv_smp_cpu_kill_self+0x58/0x2c0
[c000001fed2f7ca0] [c00000000003ce34] .cpu_die+0x34/0x50
[c000001fed2f7d10] [c0000000000176d0] .arch_cpu_idle_dead+0x20/0x40
[c000001fed2f7d80] [c00000000011f9a8] .cpu_startup_entry+0x708/0x7a0
[c000001fed2f7ec0] [c00000000003cb6c] .start_secondary+0x36c/0x3a0
[c000001fed2f7f90] [c000000000008b6c] start_secondary_prolog+0x10/0x14

Fix this by converting kmem_cache_free trace point into TRACE_EVENT_CONDITION where condition is cpu_online(smp_processor_id())

---
 include/trace/events/kmem.h | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
include/trace/events/kmem.h +4 -2
index 81ea598..dd9e612 100644
@@ -140,11 +140,13 @@ DEFINE_EVENT(kmem_free, kfree,
 	TP_ARGS(call_site, ptr)
 );
 
-DEFINE_EVENT(kmem_free, kmem_cache_free,
+DEFINE_EVENT_CONDITION(kmem_free, kmem_cache_free,
 
 	TP_PROTO(unsigned long call_site, const void *ptr),
 
-	TP_ARGS(call_site, ptr)
+	TP_ARGS(call_site, ptr),
+
+	TP_CONDITION(cpu_online(smp_processor_id()))
 );
 
 TRACE_EVENT(mm_page_free,
Signature · 1 line
1.9.3
#1 lore raw

PATCH 2/3 tracing/mm: Don't trace mm_page_free on offline cpus

Since tracepoints use RCU for protection, they must not be called on offline cpus. trace_mm_page_free can be called on an offline cpu in this scenario caught by LOCKDEP:

===============================
[ INFO: suspicious RCU usage. ]
4.1.0-rc1+ #9 Not tainted
-------------------------------
include/trace/events/kmem.h:170 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

RCU used illegally from offline CPU!
rcu_scheduler_active = 1, debug_locks = 1
no locks held by swapper/1/0.

stack backtrace:
 CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.1.0-rc1+ #9
 Call Trace:
 [c000001fed2f7790] [c0000000009dee8c] .dump_stack+0x98/0xd4 (unreliable)
 [c000001fed2f7810] [c000000000128d88] .lockdep_rcu_suspicious+0x108/0x170
 [c000001fed2f78a0] [c000000000203bc4] .free_pages_prepare+0x494/0x680
 [c000001fed2f7980] [c000000000207fd0] .free_hot_cold_page+0x50/0x280
 [c000001fed2f7a30] [c00000000004d000] .destroy_context+0x90/0xd0
 [c000001fed2f7ab0] [c0000000000bd1d8] .__mmdrop+0x58/0x160
 [c000001fed2f7b40] [c0000000001068e0] .idle_task_exit+0xf0/0x100
 [c000001fed2f7bc0] [c000000000066948] .pnv_smp_cpu_kill_self+0x58/0x2c0
 [c000001fed2f7ca0] [c00000000003ce34] .cpu_die+0x34/0x50
 [c000001fed2f7d10] [c0000000000176d0] .arch_cpu_idle_dead+0x20/0x40
 [c000001fed2f7d80] [c00000000011f9a8] .cpu_startup_entry+0x708/0x7a0
 [c000001fed2f7ec0] [c00000000003cb6c] .start_secondary+0x36c/0x3a0
 [c000001fed2f7f90] [c000000000008b6c] start_secondary_prolog+0x10/0x14

Fix this by converting mm_page_free trace point into TRACE_EVENT_CONDITION where condition is cpu_online(smp_processor_id())

---
 include/trace/events/kmem.h | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
include/trace/events/kmem.h +3 -1
index dd9e612..4abda92 100644
@@ -149,12 +149,14 @@ DEFINE_EVENT_CONDITION(kmem_free, kmem_cache_free,
 	TP_CONDITION(cpu_online(smp_processor_id()))
 );
 
-TRACE_EVENT(mm_page_free,
+TRACE_EVENT_CONDITION(mm_page_free,
 
 	TP_PROTO(struct page *page, unsigned int order),
 
 	TP_ARGS(page, order),
 
+	TP_CONDITION(cpu_online(smp_processor_id())),
+
 	TP_STRUCT__entry(
 		__field(	unsigned long,	pfn		)
 		__field(	unsigned int,	order		)
Signature · 1 line
1.9.3
Reviewed-by Preeti U Murthy on v1 2/3
#2 lore raw

Re: [PATCH 2/3] tracing/mm: Don't trace mm_page_free on offline cpus

Ccing Paul,

Quoted 52 lines · Shreyas B. Prabhu

Since tracepoints use RCU for protection, they must not be called on offline cpus. trace_mm_page_free can be called on an offline cpu in this scenario caught by LOCKDEP:

===============================
[ INFO: suspicious RCU usage. ]
4.1.0-rc1+ #9 Not tainted
-------------------------------
include/trace/events/kmem.h:170 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

RCU used illegally from offline CPU!
rcu_scheduler_active = 1, debug_locks = 1
no locks held by swapper/1/0.

stack backtrace:
 CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.1.0-rc1+ #9
 Call Trace:
 [c000001fed2f7790] [c0000000009dee8c] .dump_stack+0x98/0xd4 (unreliable)
 [c000001fed2f7810] [c000000000128d88] .lockdep_rcu_suspicious+0x108/0x170
 [c000001fed2f78a0] [c000000000203bc4] .free_pages_prepare+0x494/0x680
 [c000001fed2f7980] [c000000000207fd0] .free_hot_cold_page+0x50/0x280
 [c000001fed2f7a30] [c00000000004d000] .destroy_context+0x90/0xd0
 [c000001fed2f7ab0] [c0000000000bd1d8] .__mmdrop+0x58/0x160
 [c000001fed2f7b40] [c0000000001068e0] .idle_task_exit+0xf0/0x100
 [c000001fed2f7bc0] [c000000000066948] .pnv_smp_cpu_kill_self+0x58/0x2c0
 [c000001fed2f7ca0] [c00000000003ce34] .cpu_die+0x34/0x50
 [c000001fed2f7d10] [c0000000000176d0] .arch_cpu_idle_dead+0x20/0x40
 [c000001fed2f7d80] [c00000000011f9a8] .cpu_startup_entry+0x708/0x7a0
 [c000001fed2f7ec0] [c00000000003cb6c] .start_secondary+0x36c/0x3a0
 [c000001fed2f7f90] [c000000000008b6c] start_secondary_prolog+0x10/0x14

Fix this by converting mm_page_free trace point into TRACE_EVENT_CONDITION where condition is cpu_online(smp_processor_id())

---
 include/trace/events/kmem.h | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
include/trace/events/kmem.h +1 -1
index dd9e612..4abda92 100644
@@ -149,12 +149,14 @@ DEFINE_EVENT_CONDITION(kmem_free, kmem_cache_free,
        TP_CONDITION(cpu_online(smp_processor_id()))
 );
 
-TRACE_EVENT(mm_page_free,
+TRACE_EVENT_CONDITION(mm_page_free,
 

include/trace/events/kmem.h DEFINE_EVENT_CONDITION(kmem_free, kmem_cache_free,

Quoted 10 lines
include/trace/events/kmem.h +2 -1
        TP_PROTO(struct page *page, unsigned int order),
 
        TP_ARGS(page, order),
 
+       TP_CONDITION(cpu_online(smp_processor_id())),
+
        TP_STRUCT__entry(
                __field(        unsigned long,  pfn             )
                __field(        unsigned int,   order           )
--

Regards
Preeti U Murthy

Quoted 7 lines

1.9.3

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#3 lore raw

Re: [PATCH 2/3] tracing/mm: Don't trace mm_page_free on offline cpus

Quoted 79 lines · Preeti Murthy

Ccing Paul,

Quoted 52 lines · Shreyas B. Prabhu

Since tracepoints use RCU for protection, they must not be called on offline cpus. trace_mm_page_free can be called on an offline cpu in this scenario caught by LOCKDEP:

===============================
[ INFO: suspicious RCU usage. ]
4.1.0-rc1+ #9 Not tainted
-------------------------------
include/trace/events/kmem.h:170 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

RCU used illegally from offline CPU!
rcu_scheduler_active = 1, debug_locks = 1
no locks held by swapper/1/0.

stack backtrace:
 CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.1.0-rc1+ #9
 Call Trace:
 [c000001fed2f7790] [c0000000009dee8c] .dump_stack+0x98/0xd4 (unreliable)
 [c000001fed2f7810] [c000000000128d88] .lockdep_rcu_suspicious+0x108/0x170
 [c000001fed2f78a0] [c000000000203bc4] .free_pages_prepare+0x494/0x680
 [c000001fed2f7980] [c000000000207fd0] .free_hot_cold_page+0x50/0x280
 [c000001fed2f7a30] [c00000000004d000] .destroy_context+0x90/0xd0
 [c000001fed2f7ab0] [c0000000000bd1d8] .__mmdrop+0x58/0x160
 [c000001fed2f7b40] [c0000000001068e0] .idle_task_exit+0xf0/0x100
 [c000001fed2f7bc0] [c000000000066948] .pnv_smp_cpu_kill_self+0x58/0x2c0
 [c000001fed2f7ca0] [c00000000003ce34] .cpu_die+0x34/0x50
 [c000001fed2f7d10] [c0000000000176d0] .arch_cpu_idle_dead+0x20/0x40
 [c000001fed2f7d80] [c00000000011f9a8] .cpu_startup_entry+0x708/0x7a0
 [c000001fed2f7ec0] [c00000000003cb6c] .start_secondary+0x36c/0x3a0
 [c000001fed2f7f90] [c000000000008b6c] start_secondary_prolog+0x10/0x14

Fix this by converting mm_page_free trace point into TRACE_EVENT_CONDITION where condition is cpu_online(smp_processor_id())

---
 include/trace/events/kmem.h | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
include/trace/events/kmem.h +1 -1
index dd9e612..4abda92 100644
@@ -149,12 +149,14 @@ DEFINE_EVENT_CONDITION(kmem_free, kmem_cache_free,
        TP_CONDITION(cpu_online(smp_processor_id()))
 );
 
-TRACE_EVENT(mm_page_free,
+TRACE_EVENT_CONDITION(mm_page_free,
 

include/trace/events/kmem.h DEFINE_EVENT_CONDITION(kmem_free, kmem_cache_free,

Quoted 10 lines
include/trace/events/kmem.h +2 -1
        TP_PROTO(struct page *page, unsigned int order),
 
        TP_ARGS(page, order),
 
+       TP_CONDITION(cpu_online(smp_processor_id())),
+
        TP_STRUCT__entry(
                __field(        unsigned long,  pfn             )
                __field(        unsigned int,   order           )
--

Regards
Preeti U Murthy

Quoted 7 lines

1.9.3

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#4 lore raw

PATCH 3/3 tracing/mm: Don't trace mm_page_pcpu_drain on offline cpus

In reply to Shreyas B. Prabhu

Since tracepoints use RCU for protection, they must not be called on offline cpus. trace_mm_page_pcpu_drain can be called on an offline cpu in this scenario caught by LOCKDEP:

===============================
[ INFO: suspicious RCU usage. ]
4.1.0-rc1+ #9 Not tainted
-------------------------------
include/trace/events/kmem.h:265 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

RCU used illegally from offline CPU!
rcu_scheduler_active = 1, debug_locks = 1
1 lock held by swapper/5/0:
#0: (&(&zone->lock)->rlock){..-...}, at: [<c0000000002073b0>] .free_pcppages_bulk+0x70/0x920

stack backtrace:
 CPU: 5 PID: 0 Comm: swapper/5 Not tainted 4.1.0-rc1+ #9
 Call Trace:
 [c000001fed2e7720] [c0000000009dee8c] .dump_stack+0x98/0xd4 (unreliable)
 [c000001fed2e77a0] [c000000000128d88] .lockdep_rcu_suspicious+0x108/0x170
 [c000001fed2e7830] [c00000000020794c] .free_pcppages_bulk+0x60c/0x920
 [c000001fed2e7980] [c000000000208188] .free_hot_cold_page+0x208/0x280
 [c000001fed2e7a30] [c00000000004d000] .destroy_context+0x90/0xd0
 [c000001fed2e7ab0] [c0000000000bd1d8] .__mmdrop+0x58/0x160
 [c000001fed2e7b40] [c0000000001068e0] .idle_task_exit+0xf0/0x100
 [c000001fed2e7bc0] [c000000000066948] .pnv_smp_cpu_kill_self+0x58/0x2c0
 [c000001fed2e7ca0] [c00000000003ce34] .cpu_die+0x34/0x50
 [c000001fed2e7d10] [c0000000000176d0] .arch_cpu_idle_dead+0x20/0x40
 [c000001fed2e7d80] [c00000000011f9a8] .cpu_startup_entry+0x708/0x7a0
 [c000001fed2e7ec0] [c00000000003cb6c] .start_secondary+0x36c/0x3a0
 [c000001fed2e7f90] [c000000000008b6c] start_secondary_prolog+0x10/0x14

Fix this by converting mm_page_pcpu_drain trace point into TRACE_EVENT_CONDITION where condition is cpu_online(smp_processor_id())

---
 include/trace/events/kmem.h | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)
include/trace/events/kmem.h +15 -1
index 4abda92..6cd975f 100644
@@ -257,12 +257,26 @@ DEFINE_EVENT(mm_page, mm_page_alloc_zone_locked,
 	TP_ARGS(page, order, migratetype)
 );
 
-DEFINE_EVENT_PRINT(mm_page, mm_page_pcpu_drain,
+TRACE_EVENT_CONDITION(mm_page_pcpu_drain,
 
 	TP_PROTO(struct page *page, unsigned int order, int migratetype),
 
 	TP_ARGS(page, order, migratetype),
 
+	TP_CONDITION(cpu_online(smp_processor_id())),
+
+	TP_STRUCT__entry(
+		__field(	unsigned long,	pfn		)
+		__field(	unsigned int,	order		)
+		__field(	int,		migratetype	)
+	),
+
+	TP_fast_assign(
+		__entry->pfn		= page ? page_to_pfn(page) : -1UL;
+		__entry->order		= order;
+		__entry->migratetype	= migratetype;
+	),
+
 	TP_printk("page=%p pfn=%lu order=%d migratetype=%d",
 		pfn_to_page(__entry->pfn), __entry->pfn,
 		__entry->order, __entry->migratetype)
Signature · 1 line
1.9.3
#5 lore raw

Re: [PATCH 3/3] tracing/mm: Don't trace mm_page_pcpu_drain on offline cpus

Ccing Paul,

Quoted 61 lines · Shreyas B. Prabhu

Since tracepoints use RCU for protection, they must not be called on offline cpus. trace_mm_page_pcpu_drain can be called on an offline cpu in this scenario caught by LOCKDEP:

===============================
[ INFO: suspicious RCU usage. ]
4.1.0-rc1+ #9 Not tainted
-------------------------------
include/trace/events/kmem.h:265 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

RCU used illegally from offline CPU!
rcu_scheduler_active = 1, debug_locks = 1
1 lock held by swapper/5/0:
#0: (&(&zone->lock)->rlock){..-...}, at: [<c0000000002073b0>] .free_pcppages_bulk+0x70/0x920

stack backtrace:
 CPU: 5 PID: 0 Comm: swapper/5 Not tainted 4.1.0-rc1+ #9
 Call Trace:
 [c000001fed2e7720] [c0000000009dee8c] .dump_stack+0x98/0xd4 (unreliable)
 [c000001fed2e77a0] [c000000000128d88] .lockdep_rcu_suspicious+0x108/0x170
 [c000001fed2e7830] [c00000000020794c] .free_pcppages_bulk+0x60c/0x920
 [c000001fed2e7980] [c000000000208188] .free_hot_cold_page+0x208/0x280
 [c000001fed2e7a30] [c00000000004d000] .destroy_context+0x90/0xd0
 [c000001fed2e7ab0] [c0000000000bd1d8] .__mmdrop+0x58/0x160
 [c000001fed2e7b40] [c0000000001068e0] .idle_task_exit+0xf0/0x100
 [c000001fed2e7bc0] [c000000000066948] .pnv_smp_cpu_kill_self+0x58/0x2c0
 [c000001fed2e7ca0] [c00000000003ce34] .cpu_die+0x34/0x50
 [c000001fed2e7d10] [c0000000000176d0] .arch_cpu_idle_dead+0x20/0x40
 [c000001fed2e7d80] [c00000000011f9a8] .cpu_startup_entry+0x708/0x7a0
 [c000001fed2e7ec0] [c00000000003cb6c] .start_secondary+0x36c/0x3a0
 [c000001fed2e7f90] [c000000000008b6c] start_secondary_prolog+0x10/0x14

Fix this by converting mm_page_pcpu_drain trace point into TRACE_EVENT_CONDITION where condition is cpu_online(smp_processor_id())

---
 include/trace/events/kmem.h | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)
include/trace/events/kmem.h +5 -1
index 4abda92..6cd975f 100644
@@ -257,12 +257,26 @@ DEFINE_EVENT(mm_page, mm_page_alloc_zone_locked,
        TP_ARGS(page, order, migratetype)
 );
 
-DEFINE_EVENT_PRINT(mm_page, mm_page_pcpu_drain,
+TRACE_EVENT_CONDITION(mm_page_pcpu_drain,
 
        TP_PROTO(struct page *page, unsigned int order, int migratetype),
 
        TP_ARGS(page, order, migratetype),
 
+       TP_CONDITION(cpu_online(smp_processor_id())),
+
+       TP_STRUCT__entry(
+               __field(        unsigned long,  pfn             )

include/trace/events/kmem.h DEFINE_EVENT(mm_page, mm_page_alloc_zone_locked, · line 279

Quoted 10 lines
include/trace/events/kmem.h +10 -0
+               __field(        unsigned int,   order           )
+               __field(        int,            migratetype     )
+       ),
+
+       TP_fast_assign(
+               __entry->pfn            = page ? page_to_pfn(page) : -1UL;
+               __entry->order          = order;
+               __entry->migratetype    = migratetype;
+       ),
+

What was the need to do the above changes besides adding TP_CONDITION ?

Regards
Preeti U Murthy

#6 lore raw

Re: [PATCH 3/3] tracing/mm: Don't trace mm_page_pcpu_drain on offline cpus

Quoted 8 lines
Quoted 8 lines
include/trace/events/kmem.h +1 -1
-DEFINE_EVENT_PRINT(mm_page, mm_page_pcpu_drain,
+TRACE_EVENT_CONDITION(mm_page_pcpu_drain,
 
        TP_PROTO(struct page *page, unsigned int order, int migratetype),
 
        TP_ARGS(page, order, migratetype),
Quoted 17 lines
Quoted 4 lines
include/trace/events/kmem.h +4 -0
+       TP_CONDITION(cpu_online(smp_processor_id())),
+
+       TP_STRUCT__entry(
+               __field(        unsigned long,  pfn             )

include/trace/events/kmem.h DEFINE_EVENT(mm_page, mm_page_alloc_zone_locked, · line 279

Quoted 10 lines
include/trace/events/kmem.h +10 -0
+               __field(        unsigned int,   order           )
+               __field(        int,            migratetype     )
+       ),
+
+       TP_fast_assign(
+               __entry->pfn            = page ? page_to_pfn(page) : -1UL;
+               __entry->order          = order;
+               __entry->migratetype    = migratetype;
+       ),
+

What was the need to do the above changes besides adding TP_CONDITION ?

IIUC there is no existing macro which can both add a condition and override printk format, hence the fall back to TRACE_EVENT_CONDITION.

Thanks,
Shreyas

maintainer
#7 lore raw

Re: [PATCH 3/3] tracing/mm: Don't trace mm_page_pcpu_drain on offline cpus

Quoted 2 lines · On Wed, 29 Apr 2015 20:19:28 +0530 Shreyas B Prabhu…

IIUC there is no existing macro which can both add a condition and override printk format, hence the fall back to TRACE_EVENT_CONDITION.

Hmm, want me to send you a patch that changes that?

-- Steve

#8 lore raw

Re: [PATCH 3/3] tracing/mm: Don't trace mm_page_pcpu_drain on offline cpus

Quoted 8 lines · Steven Rostedt
Quoted 2 lines · On Wed, 29 Apr 2015 20:19:28 +0530 Shreyas B Prabhu…

IIUC there is no existing macro which can both add a condition and override printk format, hence the fall back to TRACE_EVENT_CONDITION.

Hmm, want me to send you a patch that changes that?

I am not sure if its worth the effort now. It doesn't look like any other trace point apart from the above use case will benefit from it. Only smbus_write and smbus_reply seem to come close. But even they need separate TP_fast_assign.

Thanks,
Shreyas

maintainer
#9 lore raw

Re: [PATCH 3/3] tracing/mm: Don't trace mm_page_pcpu_drain on offline cpus

Quoted 9 lines · On Wed, 29 Apr 2015 21:28:38 +0530 Shreyas B Prabhu…
Quoted 6 lines · Steven Rostedt
Quoted 2 lines · On Wed, 29 Apr 2015 20:19:28 +0530 Shreyas B Prabhu…

IIUC there is no existing macro which can both add a condition and override printk format, hence the fall back to TRACE_EVENT_CONDITION.

Quoted 6 lines
Quoted 2 lines

Hmm, want me to send you a patch that changes that?

I am not sure if its worth the effort now. It doesn't look like any other trace point apart from the above use case will benefit from it. Only smbus_write and smbus_reply seem to come close. But even they need separate TP_fast_assign.

It shouldn't be a problem to implement. But I'm currently cleaning up those files, and any changes will cause nasty conflicts.

Lets do this. Push the current changes as is, and when I get around to adding a DEFINE_EVENT_PRINT_CONDITION(), we can modify that code to use it.

-- Steve

#10 lore raw

Re: [PATCH 3/3] tracing/mm: Don't trace mm_page_pcpu_drain on offline cpus

Quoted 5 lines · Steven Rostedt
Quoted 4 lines

I am not sure if its worth the effort now. It doesn't look like any other trace point apart from the above use case will benefit from it. Only smbus_write and smbus_reply seem to come close. But even they need separate TP_fast_assign.

Quoted 7 lines

It shouldn't be a problem to implement. But I'm currently cleaning up those files, and any changes will cause nasty conflicts.

Lets do this. Push the current changes as is, and when I get around to adding a DEFINE_EVENT_PRINT_CONDITION(), we can modify that code to use it.

Okay, sure.

Thanks,
Shreyas

Reviewed-by Preeti U Murthy
#11 lore raw

Re: [PATCH 3/3] tracing/mm: Don't trace mm_page_pcpu_drain on offline cpus

Quoted 8 lines · Shreyas B Prabhu
Quoted 5 lines · Steven Rostedt
Quoted 4 lines

I am not sure if its worth the effort now. It doesn't look like any other trace point apart from the above use case will benefit from it. Only smbus_write and smbus_reply seem to come close. But even they need separate TP_fast_assign.

Quoted 8 lines
Quoted 7 lines

It shouldn't be a problem to implement. But I'm currently cleaning up those files, and any changes will cause nasty conflicts.

Lets do this. Push the current changes as is, and when I get around to adding a DEFINE_EVENT_PRINT_CONDITION(), we can modify that code to use it.

Okay, sure.

Looks good then.

Quoted 4 lines

Thanks,
Shreyas

#12 lore raw

Re: [PATCH 3/3] tracing/mm: Don't trace mm_page_pcpu_drain on offline cpus

Quoted 17 lines · Preeti Murthy
Quoted 15 lines · Shreyas B Prabhu
Quoted 12 lines · Steven Rostedt
Quoted 4 lines

I am not sure if its worth the effort now. It doesn't look like any other trace point apart from the above use case will benefit from it. Only smbus_write and smbus_reply seem to come close. But even they need separate TP_fast_assign.

It shouldn't be a problem to implement. But I'm currently cleaning up those files, and any changes will cause nasty conflicts.

Lets do this. Push the current changes as is, and when I get around to adding a DEFINE_EVENT_PRINT_CONDITION(), we can modify that code to use it.

Quoted 5 lines
Quoted 1 line

Okay, sure.

Looks good then.

Thanks a lot!

#13 lore raw

Re: [PATCH 3/3] tracing/mm: Don't trace mm_page_pcpu_drain on offline cpus

In reply to Steven Rostedt
Quoted 19 lines · Steven Rostedt
Quoted 9 lines · On Wed, 29 Apr 2015 21:28:38 +0530 Shreyas B Prabhu…
Quoted 6 lines · Steven Rostedt
Quoted 2 lines · On Wed, 29 Apr 2015 20:19:28 +0530 Shreyas B Prabhu…

IIUC there is no existing macro which can both add a condition and override printk format, hence the fall back to TRACE_EVENT_CONDITION.

Quoted 6 lines
Quoted 2 lines

Hmm, want me to send you a patch that changes that?

I am not sure if its worth the effort now. It doesn't look like any other trace point apart from the above use case will benefit from it. Only smbus_write and smbus_reply seem to come close. But even they need separate TP_fast_assign.

Quoted 7 lines

It shouldn't be a problem to implement. But I'm currently cleaning up those files, and any changes will cause nasty conflicts.

Lets do this. Push the current changes as is, and when I get around to adding a DEFINE_EVENT_PRINT_CONDITION(), we can modify that code to use it.

Hi Steve,
Do you have any other suggestions for this patchset or will you take them as is?

Thanks,
Shreyas

maintainer
#14 lore raw

Re: [PATCH 3/3] tracing/mm: Don't trace mm_page_pcpu_drain on offline cpus

Quoted 3 lines · On Fri, 08 May 2015 10:07:15 +0530 Shreyas B Prabhu…

Hi Steve,
Do you have any other suggestions for this patchset or will you take them as is?

I'm fine with them (add my Acked-by), but I don't usually pull in changes for include/trace/events/*.h files. Those are usually maintained by the creators of the files (subsystems they represent).

But I could pull them in if need be.

-- Steve

#15 lore raw

Re: [PATCH 3/3] tracing/mm: Don't trace mm_page_pcpu_drain on offline cpus

Quoted 4 lines · Steven Rostedt

On Fri, 08 May 2015 10:07:15 +0530
Shreyas B Prabhu <[email protected]> wrote:

Quoted 8 lines
Quoted 3 lines

Hi Steve,
Do you have any other suggestions for this patchset or will you take them as is?

I'm fine with them (add my Acked-by), but I don't usually pull in changes for include/trace/events/*.h files. Those are usually maintained by the creators of the files (subsystems they represent).

Oh I didn't know that. Let me resend the patches with your Acked-by, this time cc'ing people who have added/edited the events.

Thanks,
Shreyas

Reviewed-by Preeti U Murthy on v1 1/3
#16 lore raw
In reply to Shreyas B. Prabhu

Ccing Paul,

Quoted 50 lines · Shreyas B. Prabhu

Since tracepoints use RCU for protection, they must not be called on offline cpus. trace_kmem_cache_free can be called on an offline cpu in this scenario caught by LOCKDEP:

===============================
[ INFO: suspicious RCU usage. ]
4.1.0-rc1+ #9 Not tainted
-------------------------------
include/trace/events/kmem.h:148 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

RCU used illegally from offline CPU!
rcu_scheduler_active = 1, debug_locks = 1
no locks held by swapper/1/0.

stack backtrace:
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.1.0-rc1+ #9
Call Trace:
[c000001fed2f78f0] [c0000000009dee8c] .dump_stack+0x98/0xd4 (unreliable)
[c000001fed2f7970] [c000000000128d88] .lockdep_rcu_suspicious+0x108/0x170
[c000001fed2f7a00] [c00000000026f924] .kmem_cache_free+0x344/0x4b0
[c000001fed2f7ab0] [c0000000000bd1cc] .__mmdrop+0x4c/0x160
[c000001fed2f7b40] [c0000000001068e0] .idle_task_exit+0xf0/0x100
[c000001fed2f7bc0] [c000000000066948] .pnv_smp_cpu_kill_self+0x58/0x2c0
[c000001fed2f7ca0] [c00000000003ce34] .cpu_die+0x34/0x50
[c000001fed2f7d10] [c0000000000176d0] .arch_cpu_idle_dead+0x20/0x40
[c000001fed2f7d80] [c00000000011f9a8] .cpu_startup_entry+0x708/0x7a0
[c000001fed2f7ec0] [c00000000003cb6c] .start_secondary+0x36c/0x3a0
[c000001fed2f7f90] [c000000000008b6c] start_secondary_prolog+0x10/0x14

Fix this by converting kmem_cache_free trace point into TRACE_EVENT_CONDITION where condition is cpu_online(smp_processor_id())

---
 include/trace/events/kmem.h | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
include/trace/events/kmem.h +1 -1
index 81ea598..dd9e612 100644
@@ -140,11 +140,13 @@ DEFINE_EVENT(kmem_free, kfree,
        TP_ARGS(call_site, ptr)
 );
 
-DEFINE_EVENT(kmem_free, kmem_cache_free,
+DEFINE_EVENT_CONDITION(kmem_free, kmem_cache_free,

include/trace/events/kmem.h DEFINE_EVENT(kmem_free, kfree, · line 152

Quoted 10 lines
include/trace/events/kmem.h +3 -1
 
        TP_PROTO(unsigned long call_site, const void *ptr),
 
-       TP_ARGS(call_site, ptr)
+       TP_ARGS(call_site, ptr),
+
+       TP_CONDITION(cpu_online(smp_processor_id()))
 );
 
 TRACE_EVENT(mm_page_free,

Regards
Preeti U Murthy

Quoted 8 lines

--
1.9.3

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/