Skip to contentSkip to conversation

module: ppc64 module CRC relocation fix causes perf issues

PATCH In review Anton Blanchard 6 participants 15 messages Jul 15, 2013 – Jul 26, 2013 PowerPC

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 module: ppc64 module CRC relocation fix causes perf issues

Module CRCs are implemented as absolute symbols that get resolved by a linker script. We build an intermediate .o that contains an unresolved symbol for each CRC. genksysms parses this .o, calculates the CRCs and writes a linker script that "resolves" the symbols to the calculated CRC.

Unfortunately the ppc64 relocatable kernel sees these CRCs as symbols that need relocating and relocates them at boot. Commit d4703aef (module: handle ppc64 relocating kcrctabs when CONFIG_RELOCATABLE=y) added a hook to reverse the bogus relocations. Part of this patch created a symbol at 0x0:

# head -2 /proc/kallsyms 
0000000000000000 T reloc_start
c000000000000000 T .__start

This reloc_start symbol is causing lots of confusion to perf. It thinks reloc_start is a massive function that stretches from 0x0 to 0xc000000000000000 and we get various cryptic errors out of perf, including:

problem incrementing symbol count, skipping event

This patch removes the reloc_start linker script label and instead defines it as PHYSICAL_START. We also need to wrap it with CONFIG_PPC64 because the ppc32 kernel can set a non zero PHYSICAL_START at compile time and we wouldn't want to subtract it from the CRCs in that case.

---

This bug was originally reported on Fedora 19 (3.9.x), so I've marked it for stable.

arch/powerpc/include/asm/module.h +2 -3
Index: b/arch/powerpc/include/asm/module.h
===================================================================
@@ -82,10 +82,9 @@ struct exception_table_entry;
 void sort_ex_table(struct exception_table_entry *start,
 		   struct exception_table_entry *finish);
 
-#ifdef CONFIG_MODVERSIONS
+#if defined(CONFIG_MODVERSIONS) && defined(CONFIG_PPC64)
 #define ARCH_RELOCATES_KCRCTAB
-
-extern const unsigned long reloc_start[];
+#define reloc_start PHYSICAL_START
 #endif
 #endif /* __KERNEL__ */
 #endif	/* _ASM_POWERPC_MODULE_H */
arch/powerpc/kernel/vmlinux.lds.S +0 -3
Index: b/arch/powerpc/kernel/vmlinux.lds.S
===================================================================
@@ -38,9 +38,6 @@ jiffies = jiffies_64 + 4;
 #endif
 SECTIONS
 {
-	. = 0;
-	reloc_start = .;
-
 	. = KERNELBASE;
 
 /*
Acked-by on v1 1/1
#1 lore raw
Quoted 23 lines · Anton Blanchard

Module CRCs are implemented as absolute symbols that get resolved by a linker script. We build an intermediate .o that contains an unresolved symbol for each CRC. genksysms parses this .o, calculates the CRCs and writes a linker script that "resolves" the symbols to the calculated CRC.

Unfortunately the ppc64 relocatable kernel sees these CRCs as symbols that need relocating and relocates them at boot. Commit d4703aef (module: handle ppc64 relocating kcrctabs when CONFIG_RELOCATABLE=y) added a hook to reverse the bogus relocations. Part of this patch created a symbol at 0x0:

# head -2 /proc/kallsyms 
0000000000000000 T reloc_start
c000000000000000 T .__start

This reloc_start symbol is causing lots of confusion to perf. It thinks reloc_start is a massive function that stretches from 0x0 to 0xc000000000000000 and we get various cryptic errors out of perf, including:

problem incrementing symbol count, skipping event

Quoted 8 lines

This patch removes the reloc_start linker script label and instead defines it as PHYSICAL_START. We also need to wrap it with CONFIG_PPC64 because the ppc32 kernel can set a non zero PHYSICAL_START at compile time and we wouldn't want to subtract it from the CRCs in that case.

Ben?

Cheers,
Rusty.

#2 lore raw
In reply to Anton Blanchard
Quoted 5 lines · Anton Blanchard

Module CRCs are implemented as absolute symbols that get resolved by a linker script. We build an intermediate .o that contains an unresolved symbol for each CRC. genksysms parses this .o, calculates the CRCs and writes a linker script that "resolves" the symbols to the calc

Scott, can somebody from FSL test that on 32-bit and Ack it ?

Thanks !

Cheers,
Ben.

Quoted 61 lines

Unfortunately the ppc64 relocatable kernel sees these CRCs as symbols that need relocating and relocates them at boot. Commit d4703aef (module: handle ppc64 relocating kcrctabs when CONFIG_RELOCATABLE=y) added a hook to reverse the bogus relocations. Part of this patch created a symbol at 0x0:

# head -2 /proc/kallsyms 
0000000000000000 T reloc_start
c000000000000000 T .__start

This reloc_start symbol is causing lots of confusion to perf. It thinks reloc_start is a massive function that stretches from 0x0 to 0xc000000000000000 and we get various cryptic errors out of perf, including:

problem incrementing symbol count, skipping event

This patch removes the reloc_start linker script label and instead defines it as PHYSICAL_START. We also need to wrap it with CONFIG_PPC64 because the ppc32 kernel can set a non zero PHYSICAL_START at compile time and we wouldn't want to subtract it from the CRCs in that case.

---

This bug was originally reported on Fedora 19 (3.9.x), so I've marked it for stable.

Index: b/arch/powerpc/include/asm/module.h
===================================================================

arch/powerpc/include/asm/module.h +2 -3
@@ -82,10 +82,9 @@ struct exception_table_entry;
 void sort_ex_table(struct exception_table_entry *start,
 		   struct exception_table_entry *finish);
 
-#ifdef CONFIG_MODVERSIONS
+#if defined(CONFIG_MODVERSIONS) && defined(CONFIG_PPC64)
 #define ARCH_RELOCATES_KCRCTAB
-
-extern const unsigned long reloc_start[];
+#define reloc_start PHYSICAL_START
 #endif
 #endif /* __KERNEL__ */
 #endif	/* _ASM_POWERPC_MODULE_H */

Index: b/arch/powerpc/kernel/vmlinux.lds.S
===================================================================

arch/powerpc/kernel/vmlinux.lds.S +0 -3
@@ -38,9 +38,6 @@ jiffies = jiffies_64 + 4;
 #endif
 SECTIONS
 {
-	. = 0;
-	reloc_start = .;
-
 	. = KERNELBASE;
 
 /*
#3 lore raw
Quoted 7 lines · Benjamin Herrenschmidt
Quoted 5 lines · Anton Blanchard

Module CRCs are implemented as absolute symbols that get resolved by a linker script. We build an intermediate .o that contains an unresolved symbol for each CRC. genksysms parses this .o, calculates the CRCs and writes a linker script that "resolves" the symbols to the calc

Quoted 6 lines

Scott, can somebody from FSL test that on 32-bit and Ack it ?

Thanks !

Cheers,
Ben.

It boots for me on e500mc and I can insert modules.

-Scott

#4 lore raw
Quoted 8 lines · Scott Wood
Quoted 7 lines · Benjamin Herrenschmidt
Quoted 5 lines · Anton Blanchard

Module CRCs are implemented as absolute symbols that get resolved by a linker script. We build an intermediate .o that contains an unresolved symbol for each CRC. genksysms parses this .o, calculates the CRCs and writes a linker script that "resolves" the symbols to the calc

Quoted 8 lines
Quoted 6 lines

Scott, can somebody from FSL test that on 32-bit and Ack it ?

Thanks !

Cheers,
Ben.

It boots for me on e500mc and I can insert modules.

But does perf work ? :-)

Cheers,
Ben.

#5 lore raw
Quoted 14 lines · Benjamin Herrenschmidt
Quoted 3 lines · Scott Wood
Quoted 2 lines · Benjamin Herrenschmidt
Quoted 1 line · Anton Blanchard

Module CRCs are implemented as absolute symbols that get

resolved by

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

a linker script. We build an intermediate .o that contains an unresolved symbol for each CRC. genksysms parses this .o,

calculates

Quoted 1 line
Quoted 1 line
Quoted 1 line

the CRCs and writes a linker script that "resolves" the symbols

to

Quoted 4 lines
Quoted 4 lines
Quoted 1 line

the calc

Scott, can somebody from FSL test that on 32-bit and Ack it ?

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

Thanks !

Cheers,
Ben.

It boots for me on e500mc and I can insert modules.

But does perf work ? :-)

What specifically should I do to test it?

-Scott

#6 lore raw

Hi Scott,

Quoted 1 line

What specifically should I do to test it?

Could you double check perf annotate works? I'm 99% sure it will but that is what was failing on ppc64.

Anton

#7 lore raw
Quoted 7 lines · Anton Blanchard

Hi Scott,

Quoted 1 line

What specifically should I do to test it?

Could you double check perf annotate works? I'm 99% sure it will but that is what was failing on ppc64.

I'm not really sure what it's supposed to look like when "perf annotate" works. It spits a bunch of unreadable[1] dark-blue-on-black assembly code at me, all with "0.00 :" in the left column.

Oh, wait -- some lines have "100.00 : " on the left, in even-more-unreadable dark-red-on-black.

Apart from the annoying colors, is there anything specific I should be looking for? Some sort of error message, or output that actually makes sense?

I've attached the output from "perf annotate" and "perf report". perf.data was generated by "perf record find /usr > /dev/null" on an NFS root (which took a few seconds to complete), so the large amount of __alloc_skb makes some sense, but the way perf annotate shows 100% on one instruction in each function seems odd.

-Scott

[1] ...unless I crank the brightness up on my monitor to the point where whites are blinding, or redirect the output to a file so the colors go away.

Percent | Source code & Disassembly of vmlinux
------------------------------------------------

         :
         :
         :
         :	Disassembly of section .text:
         :
         :	c0097510 <perf_event_comm>:
    0.00 :	c0097510:       94 21 ff a0     stwu    r1,-96(r1)
    0.00 :	c0097514:       7c 08 02 a6     mflr    r0
    0.00 :	c0097518:       bf 01 00 40     stmw    r24,64(r1)
    0.00 :	c009751c:       7c 7e 1b 78     mr      r30,r3
    0.00 :	c0097520:       90 01 00 64     stw     r0,100(r1)
    0.00 :	c0097524:       3b 80 00 00     li      r28,0
    0.00 :	c0097528:       3b 63 04 68     addi    r27,r3,1128
    0.00 :	c009752c:       3b 40 00 00     li      r26,0
    0.00 :	c0097530:       87 bb 00 04     lwzu    r29,4(r27)
    0.00 :	c0097534:       2f 9d 00 00     cmpwi   cr7,r29,0
    0.00 :	c0097538:       41 9e 00 1c     beq-    cr7,c0097554 <perf_event_comm+0x44>
    0.00 :	c009753c:       7f 00 00 a6     mfmsr   r24
    0.00 :	c0097540:       7c 00 01 46     .long 0x7c000146
    0.00 :	c0097544:       80 1d 00 3c     lwz     r0,60(r29)
    0.00 :	c0097548:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c009754c:       40 9e 00 c4     bne-    cr7,c0097610 <perf_event_comm+0x100>
    0.00 :	c0097550:       7f 00 01 06     .long 0x7f000106
  100.00 :	c0097554:       2f 9c 00 01     cmpwi   cr7,r28,1
    0.00 :	c0097558:       3b 9c 00 01     addi    r28,r28,1
    0.00 :	c009755c:       40 be ff d4     bne-    cr7,c0097530 <perf_event_comm+0x20>
    0.00 :	c0097560:       3d 20 c0 72     lis     r9,-16270
    0.00 :	c0097564:       80 09 99 50     lwz     r0,-26288(r9)
    0.00 :	c0097568:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c009756c:       41 be 00 88     beq+    cr7,c00975f4 <perf_event_comm+0xe4>
    0.00 :	c0097570:       3b e1 00 08     addi    r31,r1,8
    0.00 :	c0097574:       38 00 00 00     li      r0,0
    0.00 :	c0097578:       39 20 00 03     li      r9,3
    0.00 :	c009757c:       38 9e 01 d8     addi    r4,r30,472
    0.00 :	c0097580:       38 a0 00 10     li      r5,16
    0.00 :	c0097584:       7f e3 fb 78     mr      r3,r31
    0.00 :	c0097588:       90 01 00 1c     stw     r0,28(r1)
    0.00 :	c009758c:       90 01 00 20     stw     r0,32(r1)
    0.00 :	c0097590:       90 01 00 28     stw     r0,40(r1)
    0.00 :	c0097594:       90 01 00 2c     stw     r0,44(r1)
    0.00 :	c0097598:       90 01 00 30     stw     r0,48(r1)
    0.00 :	c009759c:       91 21 00 24     stw     r9,36(r1)
    0.00 :	c00975a0:       90 01 00 08     stw     r0,8(r1)
    0.00 :	c00975a4:       90 01 00 0c     stw     r0,12(r1)
    0.00 :	c00975a8:       90 01 00 10     stw     r0,16(r1)
    0.00 :	c00975ac:       90 01 00 14     stw     r0,20(r1)
    0.00 :	c00975b0:       93 c1 00 18     stw     r30,24(r1)
    0.00 :	c00975b4:       48 1a 05 8d     bl      c0237b40 <strlcpy>
    0.00 :	c00975b8:       7f e3 fb 78     mr      r3,r31
    0.00 :	c00975bc:       4b f8 0e 89     bl      c0018444 <strlen>
    0.00 :	c00975c0:       3c 80 c0 09     lis     r4,-16375
    0.00 :	c00975c4:       39 23 00 08     addi    r9,r3,8
    0.00 :	c00975c8:       93 e1 00 1c     stw     r31,28(r1)
    0.00 :	c00975cc:       55 29 00 38     rlwinm  r9,r9,0,0,28
    0.00 :	c00975d0:       3c 60 c0 09     lis     r3,-16375
    0.00 :	c00975d4:       38 09 00 10     addi    r0,r9,16
    0.00 :	c00975d8:       91 21 00 20     stw     r9,32(r1)
    0.00 :	c00975dc:       38 63 fd 80     addi    r3,r3,-640
    0.00 :	c00975e0:       38 84 65 50     addi    r4,r4,25936
    0.00 :	c00975e4:       38 a1 00 18     addi    r5,r1,24
    0.00 :	c00975e8:       b0 01 00 2a     sth     r0,42(r1)
    0.00 :	c00975ec:       38 c0 00 00     li      r6,0
    0.00 :	c00975f0:       4b ff 86 61     bl      c008fc50 <perf_event_aux>
    0.00 :	c00975f4:       80 01 00 64     lwz     r0,100(r1)
    0.00 :	c00975f8:       bb 01 00 40     lmw     r24,64(r1)
    0.00 :	c00975fc:       38 21 00 60     addi    r1,r1,96
    0.00 :	c0097600:       7c 08 03 a6     mtlr    r0
    0.00 :	c0097604:       4e 80 00 20     blr
    0.00 :	c0097608:       60 00 00 00     nop
    0.00 :	c009760c:       60 00 00 00     nop
    0.00 :	c0097610:       38 7d 00 08     addi    r3,r29,8
    0.00 :	c0097614:       7f b9 eb 78     mr      r25,r29
    0.00 :	c0097618:       48 48 98 a9     bl      c0520ec0 <_raw_spin_lock>
    0.00 :	c009761c:       7f a3 eb 78     mr      r3,r29
    0.00 :	c0097620:       4b ff ce 81     bl      c00944a0 <task_ctx_sched_out>
    0.00 :	c0097624:       87 f9 00 34     lwzu    r31,52(r25)
    0.00 :	c0097628:       7f 99 f8 00     cmpw    cr7,r25,r31
    0.00 :	c009762c:       3b ff ff f8     addi    r31,r31,-8
    0.00 :	c0097630:       41 9e 00 60     beq-    cr7,c0097690 <perf_event_comm+0x180>
    0.00 :	c0097634:       39 60 00 00     li      r11,0
    0.00 :	c0097638:       60 00 00 00     nop
    0.00 :	c009763c:       60 00 00 00     nop
    0.00 :	c0097640:       80 1f 00 a0     lwz     r0,160(r31)
    0.00 :	c0097644:       7c 09 03 78     mr      r9,r0
    0.00 :	c0097648:       74 0a 00 08     andis.  r10,r0,8
    0.00 :	c009764c:       53 49 9b 18     rlwimi  r9,r26,19,12,12
    0.00 :	c0097650:       41 82 00 14     beq-    c0097664 <perf_event_comm+0x154>
    0.00 :	c0097654:       80 1f 00 30     lwz     r0,48(r31)
    0.00 :	c0097658:       91 3f 00 a0     stw     r9,160(r31)
    0.00 :	c009765c:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c0097660:       41 9c 00 50     blt-    cr7,c00976b0 <perf_event_comm+0x1a0>
    0.00 :	c0097664:       83 ff 00 08     lwz     r31,8(r31)
    0.00 :	c0097668:       7f 99 f8 00     cmpw    cr7,r25,r31
    0.00 :	c009766c:       3b ff ff f8     addi    r31,r31,-8
    0.00 :	c0097670:       40 9e ff d0     bne+    cr7,c0097640 <perf_event_comm+0x130>
    0.00 :	c0097674:       2f 8b 00 00     cmpwi   cr7,r11,0
    0.00 :	c0097678:       41 9e 00 18     beq-    cr7,c0097690 <perf_event_comm+0x180>
    0.00 :	c009767c:       80 7d 00 70     lwz     r3,112(r29)
    0.00 :	c0097680:       2f 83 00 00     cmpwi   cr7,r3,0
    0.00 :	c0097684:       41 9e 00 0c     beq-    cr7,c0097690 <perf_event_comm+0x180>
    0.00 :	c0097688:       4b ff a6 69     bl      c0091cf0 <put_ctx>
    0.00 :	c009768c:       93 5d 00 70     stw     r26,112(r29)
    0.00 :	c0097690:       7c 00 04 ac     sync    
    0.00 :	c0097694:       93 5d 00 08     stw     r26,8(r29)
    0.00 :	c0097698:       7f a3 eb 78     mr      r3,r29
    0.00 :	c009769c:       4b ff e1 75     bl      c0095810 <perf_event_context_sched_in.clone.38>
    0.00 :	c00976a0:       4b ff fe b0     b       c0097550 <perf_event_comm+0x40>
    0.00 :	c00976a4:       60 00 00 00     nop
    0.00 :	c00976a8:       60 00 00 00     nop
    0.00 :	c00976ac:       60 00 00 00     nop
    0.00 :	c00976b0:       7f e3 fb 78     mr      r3,r31
    0.00 :	c00976b4:       4b ff 83 cd     bl      c008fa80 <__perf_event_mark_enabled>
    0.00 :	c00976b8:       39 60 00 01     li      r11,1
    0.00 :	c00976bc:       4b ff ff a8     b       c0097664 <perf_event_comm+0x154>
 Percent |	Source code & Disassembly of vmlinux
------------------------------------------------
         :
         :
         :
         :	Disassembly of section .text:
         :
         :	c00368c0 <mod_timer>:
  100.00 :	c00368c0:       94 21 ff d0     stwu    r1,-48(r1)
    0.00 :	c00368c4:       7c 08 02 a6     mflr    r0
    0.00 :	c00368c8:       bf 61 00 1c     stmw    r27,28(r1)
    0.00 :	c00368cc:       7c 7f 1b 78     mr      r31,r3
    0.00 :	c00368d0:       90 01 00 34     stw     r0,52(r1)
    0.00 :	c00368d4:       7c 9e 23 78     mr      r30,r4
    0.00 :	c00368d8:       83 a3 00 18     lwz     r29,24(r3)
    0.00 :	c00368dc:       2f 9d 00 00     cmpwi   cr7,r29,0
    0.00 :	c00368e0:       41 9c 01 70     blt-    cr7,c0036a50 <mod_timer+0x190>
    0.00 :	c00368e4:       7f bd f2 14     add     r29,r29,r30
    0.00 :	c00368e8:       7f a0 f2 78     xor     r0,r29,r30
    0.00 :	c00368ec:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c00368f0:       90 01 00 08     stw     r0,8(r1)
    0.00 :	c00368f4:       40 9e 01 8c     bne-    cr7,c0036a80 <mod_timer+0x1c0>
    0.00 :	c00368f8:       80 1f 00 00     lwz     r0,0(r31)
    0.00 :	c00368fc:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c0036900:       41 9e 01 ac     beq-    cr7,c0036aac <mod_timer+0x1ec>
    0.00 :	c0036904:       80 1f 00 08     lwz     r0,8(r31)
    0.00 :	c0036908:       3b 80 00 01     li      r28,1
    0.00 :	c003690c:       7f 80 f0 00     cmpw    cr7,r0,r30
    0.00 :	c0036910:       41 9e 00 f0     beq-    cr7,c0036a00 <mod_timer+0x140>
    0.00 :	c0036914:       80 1f 00 10     lwz     r0,16(r31)
    0.00 :	c0036918:       7c 00 00 34     cntlzw  r0,r0
    0.00 :	c003691c:       54 00 d9 7e     rlwinm  r0,r0,27,5,31
    0.00 :	c0036920:       0f 00 00 00     twnei   r0,0
    0.00 :	c0036924:       38 7f 00 0c     addi    r3,r31,12
    0.00 :	c0036928:       38 81 00 08     addi    r4,r1,8
    0.00 :	c003692c:       4b ff f9 75     bl      c00362a0 <lock_timer_base.clone.24>
    0.00 :	c0036930:       81 3f 00 00     lwz     r9,0(r31)
    0.00 :	c0036934:       7c 7b 1b 78     mr      r27,r3
    0.00 :	c0036938:       3b 80 00 00     li      r28,0
    0.00 :	c003693c:       2f 89 00 00     cmpwi   cr7,r9,0
    0.00 :	c0036940:       41 9e 00 2c     beq-    cr7,c003696c <mod_timer+0xac>
    0.00 :	c0036944:       81 7f 00 04     lwz     r11,4(r31)
    0.00 :	c0036948:       3c 00 00 20     lis     r0,32
    0.00 :	c003694c:       60 00 02 00     ori     r0,r0,512
    0.00 :	c0036950:       3b 80 00 01     li      r28,1
    0.00 :	c0036954:       91 69 00 04     stw     r11,4(r9)
    0.00 :	c0036958:       91 2b 00 00     stw     r9,0(r11)
    0.00 :	c003695c:       90 1f 00 04     stw     r0,4(r31)
    0.00 :	c0036960:       80 1f 00 0c     lwz     r0,12(r31)
    0.00 :	c0036964:       70 09 00 01     andi.   r9,r0,1
    0.00 :	c0036968:       41 82 00 b8     beq-    c0036a20 <mod_timer+0x160>
    0.00 :	c003696c:       3d 20 c0 72     lis     r9,-16270
    0.00 :	c0036970:       80 09 98 ec     lwz     r0,-26388(r9)
    0.00 :	c0036974:       54 29 00 24     rlwinm  r9,r1,0,0,18
    0.00 :	c0036978:       83 a9 00 08     lwz     r29,8(r9)
    0.00 :	c003697c:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c0036980:       40 9e 01 40     bne-    cr7,c0036ac0 <mod_timer+0x200>
    0.00 :	c0036984:       3d 20 c0 72     lis     r9,-16270
    0.00 :	c0036988:       57 bd 10 3a     rlwinm  r29,r29,2,0,29
    0.00 :	c003698c:       39 29 9b 40     addi    r9,r9,-25792
    0.00 :	c0036990:       7d 69 e8 2e     lwzx    r11,r9,r29
    0.00 :	c0036994:       3d 20 c0 6d     lis     r9,-16275
    0.00 :	c0036998:       39 29 04 70     addi    r9,r9,1136
    0.00 :	c003699c:       7f ab 48 2e     lwzx    r29,r11,r9
    0.00 :	c00369a0:       7f 9b e8 00     cmpw    cr7,r27,r29
    0.00 :	c00369a4:       41 9e 00 40     beq-    cr7,c00369e4 <mod_timer+0x124>
    0.00 :	c00369a8:       80 1b 00 04     lwz     r0,4(r27)
    0.00 :	c00369ac:       7f 9f 00 00     cmpw    cr7,r31,r0
    0.00 :	c00369b0:       41 9e 01 2c     beq-    cr7,c0036adc <mod_timer+0x21c>
    0.00 :	c00369b4:       80 1f 00 0c     lwz     r0,12(r31)
    0.00 :	c00369b8:       54 00 07 be     clrlwi  r0,r0,30
    0.00 :	c00369bc:       90 1f 00 0c     stw     r0,12(r31)
    0.00 :	c00369c0:       7c 00 04 ac     sync    
    0.00 :	c00369c4:       38 00 00 00     li      r0,0
    0.00 :	c00369c8:       90 1b 00 00     stw     r0,0(r27)
    0.00 :	c00369cc:       7f a3 eb 78     mr      r3,r29
    0.00 :	c00369d0:       48 4e a4 f1     bl      c0520ec0 <_raw_spin_lock>
    0.00 :	c00369d4:       80 1f 00 0c     lwz     r0,12(r31)
    0.00 :	c00369d8:       54 00 07 be     clrlwi  r0,r0,30
    0.00 :	c00369dc:       7f a0 03 78     or      r0,r29,r0
    0.00 :	c00369e0:       90 1f 00 0c     stw     r0,12(r31)
    0.00 :	c00369e4:       93 df 00 08     stw     r30,8(r31)
    0.00 :	c00369e8:       7f e4 fb 78     mr      r4,r31
    0.00 :	c00369ec:       7f a3 eb 78     mr      r3,r29
    0.00 :	c00369f0:       4b ff f1 d1     bl      c0035bc0 <internal_add_timer>
    0.00 :	c00369f4:       80 81 00 08     lwz     r4,8(r1)
    0.00 :	c00369f8:       7f a3 eb 78     mr      r3,r29
    0.00 :	c00369fc:       48 4e 9e 65     bl      c0520860 <_raw_spin_unlock_irqrestore>
    0.00 :	c0036a00:       80 01 00 34     lwz     r0,52(r1)
    0.00 :	c0036a04:       7f 83 e3 78     mr      r3,r28
    0.00 :	c0036a08:       bb 61 00 1c     lmw     r27,28(r1)
    0.00 :	c0036a0c:       38 21 00 30     addi    r1,r1,48
    0.00 :	c0036a10:       7c 08 03 a6     mtlr    r0
    0.00 :	c0036a14:       4e 80 00 20     blr
    0.00 :	c0036a18:       60 00 00 00     nop
    0.00 :	c0036a1c:       60 00 00 00     nop
    0.00 :	c0036a20:       81 23 00 10     lwz     r9,16(r3)
    0.00 :	c0036a24:       80 03 00 0c     lwz     r0,12(r3)
    0.00 :	c0036a28:       39 29 ff ff     addi    r9,r9,-1
    0.00 :	c0036a2c:       91 23 00 10     stw     r9,16(r3)
    0.00 :	c0036a30:       81 3f 00 08     lwz     r9,8(r31)
    0.00 :	c0036a34:       7f 89 00 00     cmpw    cr7,r9,r0
    0.00 :	c0036a38:       40 9e ff 34     bne+    cr7,c003696c <mod_timer+0xac>
    0.00 :	c0036a3c:       80 03 00 08     lwz     r0,8(r3)
    0.00 :	c0036a40:       90 03 00 0c     stw     r0,12(r3)
    0.00 :	c0036a44:       4b ff ff 28     b       c003696c <mod_timer+0xac>
    0.00 :	c0036a48:       60 00 00 00     nop
    0.00 :	c0036a4c:       60 00 00 00     nop
    0.00 :	c0036a50:       3d 20 c0 71     lis     r9,-16271
    0.00 :	c0036a54:       83 a9 30 c4     lwz     r29,12484(r9)
    0.00 :	c0036a58:       7f bd 20 50     subf    r29,r29,r4
    0.00 :	c0036a5c:       2f 9d 00 ff     cmpwi   cr7,r29,255
    0.00 :	c0036a60:       40 bd fe 98     ble-    cr7,c00368f8 <mod_timer+0x38>
    0.00 :	c0036a64:       7f bd 46 70     srawi   r29,r29,8
    0.00 :	c0036a68:       7f bd f2 14     add     r29,r29,r30
    0.00 :	c0036a6c:       7f a0 f2 78     xor     r0,r29,r30
    0.00 :	c0036a70:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c0036a74:       90 01 00 08     stw     r0,8(r1)
    0.00 :	c0036a78:       41 9e fe 80     beq+    cr7,c00368f8 <mod_timer+0x38>
    0.00 :	c0036a7c:       60 00 00 00     nop
    0.00 :	c0036a80:       38 61 00 08     addi    r3,r1,8
    0.00 :	c0036a84:       38 80 00 20     li      r4,32
    0.00 :	c0036a88:       48 20 94 39     bl      c023fec0 <find_last_bit>
    0.00 :	c0036a8c:       80 1f 00 00     lwz     r0,0(r31)
    0.00 :	c0036a90:       3b c0 00 01     li      r30,1
    0.00 :	c0036a94:       7f de 18 30     slw     r30,r30,r3
    0.00 :	c0036a98:       3b de ff ff     addi    r30,r30,-1
    0.00 :	c0036a9c:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c0036aa0:       93 c1 00 08     stw     r30,8(r1)
    0.00 :	c0036aa4:       7f be f0 78     andc    r30,r29,r30
    0.00 :	c0036aa8:       40 9e fe 5c     bne+    cr7,c0036904 <mod_timer+0x44>
    0.00 :	c0036aac:       81 3f 00 10     lwz     r9,16(r31)
    0.00 :	c0036ab0:       2f 89 00 00     cmpwi   cr7,r9,0
    0.00 :	c0036ab4:       39 20 00 01     li      r9,1
    0.00 :	c0036ab8:       7c 09 07 9e     .long 0x7c09079e
    0.00 :	c0036abc:       4b ff fe 64     b       c0036920 <mod_timer+0x60>
    0.00 :	c0036ac0:       7f a3 eb 78     mr      r3,r29
    0.00 :	c0036ac4:       48 02 29 bd     bl      c0059480 <idle_cpu>
    0.00 :	c0036ac8:       2f 83 00 00     cmpwi   cr7,r3,0
    0.00 :	c0036acc:       41 9e fe b8     beq+    cr7,c0036984 <mod_timer+0xc4>
    0.00 :	c0036ad0:       48 02 2c d1     bl      c00597a0 <get_nohz_timer_target>
    0.00 :	c0036ad4:       7c 7d 1b 78     mr      r29,r3
    0.00 :	c0036ad8:       4b ff fe ac     b       c0036984 <mod_timer+0xc4>
    0.00 :	c0036adc:       7f 7d db 78     mr      r29,r27
    0.00 :	c0036ae0:       4b ff ff 04     b       c00369e4 <mod_timer+0x124>
 Percent |	Source code & Disassembly of vmlinux
------------------------------------------------
         :
         :
         :
         :	Disassembly of section .text:
         :
         :	c0096550 <perf_event_comm_output>:
  100.00 :	c0096550:       94 21 ff 30     stwu    r1,-208(r1)
    0.00 :	c0096554:       7c 08 02 a6     mflr    r0
    0.00 :	c0096558:       bf 21 00 b4     stmw    r25,180(r1)
    0.00 :	c009655c:       7c 7b 1b 78     mr      r27,r3
    0.00 :	c0096560:       90 01 00 d4     stw     r0,212(r1)
    0.00 :	c0096564:       3b e4 00 0c     addi    r31,r4,12
    0.00 :	c0096568:       7c 9c 23 78     mr      r28,r4
    0.00 :	c009656c:       7f e3 fb 78     mr      r3,r31
    0.00 :	c0096570:       38 81 00 20     addi    r4,r1,32
    0.00 :	c0096574:       7f 65 db 78     mr      r5,r27
    0.00 :	c0096578:       a3 3c 00 12     lhz     r25,18(r28)
    0.00 :	c009657c:       4b ff e9 c5     bl      c0094f40 <perf_event_header__init_id>
    0.00 :	c0096580:       a0 bc 00 12     lhz     r5,18(r28)
    0.00 :	c0096584:       38 61 00 08     addi    r3,r1,8
    0.00 :	c0096588:       7f 64 db 78     mr      r4,r27
    0.00 :	c009658c:       48 00 37 e5     bl      c0099d70 <perf_output_begin>
    0.00 :	c0096590:       2f 83 00 00     cmpwi   cr7,r3,0
    0.00 :	c0096594:       41 9e 00 1c     beq-    cr7,c00965b0 <perf_event_comm_output+0x60>
    0.00 :	c0096598:       b3 3c 00 12     sth     r25,18(r28)
    0.00 :	c009659c:       80 01 00 d4     lwz     r0,212(r1)
    0.00 :	c00965a0:       bb 21 00 b4     lmw     r25,180(r1)
    0.00 :	c00965a4:       38 21 00 d0     addi    r1,r1,208
    0.00 :	c00965a8:       7c 08 03 a6     mtlr    r0
    0.00 :	c00965ac:       4e 80 00 20     blr
    0.00 :	c00965b0:       80 9c 00 00     lwz     r4,0(r28)
    0.00 :	c00965b4:       7f 63 db 78     mr      r3,r27
    0.00 :	c00965b8:       3b 40 10 00     li      r26,4096
    0.00 :	c00965bc:       4b ff b9 d5     bl      c0091f90 <perf_event_pid>
    0.00 :	c00965c0:       80 9c 00 00     lwz     r4,0(r28)
    0.00 :	c00965c4:       90 7c 00 14     stw     r3,20(r28)
    0.00 :	c00965c8:       7f 63 db 78     mr      r3,r27
    0.00 :	c00965cc:       4b ff ba 05     bl      c0091fd0 <perf_event_tid>
    0.00 :	c00965d0:       7f e4 fb 78     mr      r4,r31
    0.00 :	c00965d4:       90 7c 00 18     stw     r3,24(r28)
    0.00 :	c00965d8:       38 a0 00 10     li      r5,16
    0.00 :	c00965dc:       38 61 00 08     addi    r3,r1,8
    0.00 :	c00965e0:       48 00 36 d1     bl      c0099cb0 <perf_output_copy>
    0.00 :	c00965e4:       83 dc 00 08     lwz     r30,8(r28)
    0.00 :	c00965e8:       83 bc 00 04     lwz     r29,4(r28)
    0.00 :	c00965ec:       60 00 00 00     nop
    0.00 :	c00965f0:       83 e1 00 14     lwz     r31,20(r1)
    0.00 :	c00965f4:       7f a4 eb 78     mr      r4,r29
    0.00 :	c00965f8:       80 61 00 18     lwz     r3,24(r1)
    0.00 :	c00965fc:       7f 9e f8 40     cmplw   cr7,r30,r31
    0.00 :	c0096600:       7f fe ff 1e     .long 0x7ffeff1e
    0.00 :	c0096604:       7f e5 fb 78     mr      r5,r31
    0.00 :	c0096608:       7f df f0 50     subf    r30,r31,r30
    0.00 :	c009660c:       4b f8 24 4d     bl      c0018a58 <memcpy>
    0.00 :	c0096610:       80 01 00 14     lwz     r0,20(r1)
    0.00 :	c0096614:       81 21 00 18     lwz     r9,24(r1)
    0.00 :	c0096618:       2f 9e 00 00     cmpwi   cr7,r30,0
    0.00 :	c009661c:       7f bd fa 14     add     r29,r29,r31
    0.00 :	c0096620:       7c 1f 00 50     subf    r0,r31,r0
    0.00 :	c0096624:       7d 29 fa 14     add     r9,r9,r31
    0.00 :	c0096628:       2f 00 00 00     cmpwi   cr6,r0,0
    0.00 :	c009662c:       91 21 00 18     stw     r9,24(r1)
    0.00 :	c0096630:       90 01 00 14     stw     r0,20(r1)
    0.00 :	c0096634:       40 9a 00 3c     bne-    cr6,c0096670 <perf_event_comm_output+0x120>
    0.00 :	c0096638:       81 61 00 1c     lwz     r11,28(r1)
    0.00 :	c009663c:       81 21 00 0c     lwz     r9,12(r1)
    0.00 :	c0096640:       38 0b 00 01     addi    r0,r11,1
    0.00 :	c0096644:       90 01 00 1c     stw     r0,28(r1)
    0.00 :	c0096648:       81 69 00 0c     lwz     r11,12(r9)
    0.00 :	c009664c:       39 6b ff ff     addi    r11,r11,-1
    0.00 :	c0096650:       7d 6b 00 38     and     r11,r11,r0
    0.00 :	c0096654:       91 61 00 1c     stw     r11,28(r1)
    0.00 :	c0096658:       39 6b 00 10     addi    r11,r11,16
    0.00 :	c009665c:       55 6b 10 3a     rlwinm  r11,r11,2,0,29
    0.00 :	c0096660:       7d 29 5a 14     add     r9,r9,r11
    0.00 :	c0096664:       80 09 00 0c     lwz     r0,12(r9)
    0.00 :	c0096668:       93 41 00 14     stw     r26,20(r1)
    0.00 :	c009666c:       90 01 00 18     stw     r0,24(r1)
    0.00 :	c0096670:       40 9e ff 80     bne+    cr7,c00965f0 <perf_event_comm_output+0xa0>
    0.00 :	c0096674:       7f 63 db 78     mr      r3,r27
    0.00 :	c0096678:       38 81 00 08     addi    r4,r1,8
    0.00 :	c009667c:       38 a1 00 20     addi    r5,r1,32
    0.00 :	c0096680:       4b ff e8 e1     bl      c0094f60 <perf_event__output_id_sample>
    0.00 :	c0096684:       38 61 00 08     addi    r3,r1,8
    0.00 :	c0096688:       48 00 39 c9     bl      c009a050 <perf_output_end>
    0.00 :	c009668c:       b3 3c 00 12     sth     r25,18(r28)
    0.00 :	c0096690:       80 01 00 d4     lwz     r0,212(r1)
    0.00 :	c0096694:       bb 21 00 b4     lmw     r25,180(r1)
    0.00 :	c0096698:       38 21 00 d0     addi    r1,r1,208
    0.00 :	c009669c:       7c 08 03 a6     mtlr    r0
    0.00 :	c00966a0:       4e 80 00 20     blr
 Percent |	Source code & Disassembly of vmlinux
------------------------------------------------
         :
         :
         :
         :	Disassembly of section .text:
         :
         :	c0235580 <__rb_insert_augmented>:
    0.00 :	c0235580:       94 21 ff e0     stwu    r1,-32(r1)
    0.00 :	c0235584:       7c 08 02 a6     mflr    r0
    0.00 :	c0235588:       bf 81 00 10     stmw    r28,16(r1)
    0.00 :	c023558c:       7c 9f 23 78     mr      r31,r4
    0.00 :	c0235590:       90 01 00 24     stw     r0,36(r1)
    0.00 :	c0235594:       90 a1 00 08     stw     r5,8(r1)
    0.00 :	c0235598:       81 23 00 00     lwz     r9,0(r3)
    0.00 :	c023559c:       2f 89 00 00     cmpwi   cr7,r9,0
    0.00 :	c02355a0:       41 9e 01 9c     beq-    cr7,c023573c <__rb_insert_augmented+0x1bc>
    0.00 :	c02355a4:       83 c9 00 00     lwz     r30,0(r9)
    0.00 :	c02355a8:       73 c0 00 01     andi.   r0,r30,1
    0.00 :	c02355ac:       40 82 00 8c     bne-    c0235638 <__rb_insert_augmented+0xb8>
    0.00 :	c02355b0:       81 7e 00 04     lwz     r11,4(r30)
    0.00 :	c02355b4:       63 c0 00 01     ori     r0,r30,1
    0.00 :	c02355b8:       7f c8 f3 78     mr      r8,r30
    0.00 :	c02355bc:       7f 89 58 00     cmpw    cr7,r9,r11
  100.00 :	c02355c0:       2f 0b 00 00     cmpwi   cr6,r11,0
    0.00 :	c02355c4:       41 9e 00 8c     beq-    cr7,c0235650 <__rb_insert_augmented+0xd0>
    0.00 :	c02355c8:       41 9a 00 10     beq-    cr6,c02355d8 <__rb_insert_augmented+0x58>
    0.00 :	c02355cc:       81 4b 00 00     lwz     r10,0(r11)
    0.00 :	c02355d0:       71 47 00 01     andi.   r7,r10,1
    0.00 :	c02355d4:       41 82 00 9c     beq-    c0235670 <__rb_insert_augmented+0xf0>
    0.00 :	c02355d8:       83 a9 00 04     lwz     r29,4(r9)
    0.00 :	c02355dc:       7d 3c 4b 78     mr      r28,r9
    0.00 :	c02355e0:       7f 9d 18 00     cmpw    cr7,r29,r3
    0.00 :	c02355e4:       41 9e 00 fc     beq-    cr7,c02356e0 <__rb_insert_augmented+0x160>
    0.00 :	c02355e8:       2f 9d 00 00     cmpwi   cr7,r29,0
    0.00 :	c02355ec:       93 be 00 08     stw     r29,8(r30)
    0.00 :	c02355f0:       93 c9 00 04     stw     r30,4(r9)
    0.00 :	c02355f4:       41 9e 00 0c     beq-    cr7,c0235600 <__rb_insert_augmented+0x80>
    0.00 :	c02355f8:       63 c0 00 01     ori     r0,r30,1
    0.00 :	c02355fc:       90 1d 00 00     stw     r0,0(r29)
    0.00 :	c0235600:       80 1e 00 00     lwz     r0,0(r30)
    0.00 :	c0235604:       54 0b 00 3b     rlwinm. r11,r0,0,0,29
    0.00 :	c0235608:       90 09 00 00     stw     r0,0(r9)
    0.00 :	c023560c:       93 9e 00 00     stw     r28,0(r30)
    0.00 :	c0235610:       41 82 00 c0     beq-    c02356d0 <__rb_insert_augmented+0x150>
    0.00 :	c0235614:       80 0b 00 08     lwz     r0,8(r11)
    0.00 :	c0235618:       7f 80 f0 00     cmpw    cr7,r0,r30
    0.00 :	c023561c:       41 9e 01 14     beq-    cr7,c0235730 <__rb_insert_augmented+0x1b0>
    0.00 :	c0235620:       91 2b 00 04     stw     r9,4(r11)
    0.00 :	c0235624:       80 01 00 08     lwz     r0,8(r1)
    0.00 :	c0235628:       7f c3 f3 78     mr      r3,r30
    0.00 :	c023562c:       7d 24 4b 78     mr      r4,r9
    0.00 :	c0235630:       7c 09 03 a6     mtctr   r0
    0.00 :	c0235634:       4e 80 04 21     bctrl
    0.00 :	c0235638:       80 01 00 24     lwz     r0,36(r1)
    0.00 :	c023563c:       bb 81 00 10     lmw     r28,16(r1)
    0.00 :	c0235640:       38 21 00 20     addi    r1,r1,32
    0.00 :	c0235644:       7c 08 03 a6     mtlr    r0
    0.00 :	c0235648:       4e 80 00 20     blr
    0.00 :	c023564c:       60 00 00 00     nop
    0.00 :	c0235650:       81 7e 00 08     lwz     r11,8(r30)
    0.00 :	c0235654:       63 c0 00 01     ori     r0,r30,1
    0.00 :	c0235658:       2f 8b 00 00     cmpwi   cr7,r11,0
    0.00 :	c023565c:       41 9e 00 44     beq-    cr7,c02356a0 <__rb_insert_augmented+0x120>
    0.00 :	c0235660:       81 4b 00 00     lwz     r10,0(r11)
    0.00 :	c0235664:       71 47 00 01     andi.   r7,r10,1
    0.00 :	c0235668:       40 82 00 38     bne-    c02356a0 <__rb_insert_augmented+0x120>
    0.00 :	c023566c:       60 00 00 00     nop
    0.00 :	c0235670:       90 0b 00 00     stw     r0,0(r11)
    0.00 :	c0235674:       90 09 00 00     stw     r0,0(r9)
    0.00 :	c0235678:       81 3e 00 00     lwz     r9,0(r30)
    0.00 :	c023567c:       55 29 00 3a     rlwinm  r9,r9,0,0,29
    0.00 :	c0235680:       2f 89 00 00     cmpwi   cr7,r9,0
    0.00 :	c0235684:       91 3e 00 00     stw     r9,0(r30)
    0.00 :	c0235688:       41 9e 00 b0     beq-    cr7,c0235738 <__rb_insert_augmented+0x1b8>
    0.00 :	c023568c:       83 c9 00 00     lwz     r30,0(r9)
    0.00 :	c0235690:       7d 03 43 78     mr      r3,r8
    0.00 :	c0235694:       73 c7 00 01     andi.   r7,r30,1
    0.00 :	c0235698:       41 82 ff 18     beq+    c02355b0 <__rb_insert_augmented+0x30>
    0.00 :	c023569c:       4b ff ff 9c     b       c0235638 <__rb_insert_augmented+0xb8>
    0.00 :	c02356a0:       83 a9 00 08     lwz     r29,8(r9)
    0.00 :	c02356a4:       7d 3c 4b 78     mr      r28,r9
    0.00 :	c02356a8:       7f 9d 18 00     cmpw    cr7,r29,r3
    0.00 :	c02356ac:       41 9e 00 a4     beq-    cr7,c0235750 <__rb_insert_augmented+0x1d0>
    0.00 :	c02356b0:       2f 9d 00 00     cmpwi   cr7,r29,0
    0.00 :	c02356b4:       93 be 00 04     stw     r29,4(r30)
    0.00 :	c02356b8:       93 c9 00 08     stw     r30,8(r9)
    0.00 :	c02356bc:       40 9e ff 3c     bne+    cr7,c02355f8 <__rb_insert_augmented+0x78>
    0.00 :	c02356c0:       4b ff ff 40     b       c0235600 <__rb_insert_augmented+0x80>
    0.00 :	c02356c4:       60 00 00 00     nop
    0.00 :	c02356c8:       60 00 00 00     nop
    0.00 :	c02356cc:       60 00 00 00     nop
    0.00 :	c02356d0:       91 3f 00 00     stw     r9,0(r31)
    0.00 :	c02356d4:       4b ff ff 50     b       c0235624 <__rb_insert_augmented+0xa4>
    0.00 :	c02356d8:       60 00 00 00     nop
    0.00 :	c02356dc:       60 00 00 00     nop
    0.00 :	c02356e0:       81 63 00 08     lwz     r11,8(r3)
    0.00 :	c02356e4:       2f 8b 00 00     cmpwi   cr7,r11,0
    0.00 :	c02356e8:       91 69 00 04     stw     r11,4(r9)
    0.00 :	c02356ec:       91 23 00 08     stw     r9,8(r3)
    0.00 :	c02356f0:       41 9e 00 0c     beq-    cr7,c02356fc <__rb_insert_augmented+0x17c>
    0.00 :	c02356f4:       61 20 00 01     ori     r0,r9,1
    0.00 :	c02356f8:       90 0b 00 00     stw     r0,0(r11)
    0.00 :	c02356fc:       80 01 00 08     lwz     r0,8(r1)
    0.00 :	c0235700:       7d 23 4b 78     mr      r3,r9
    0.00 :	c0235704:       93 a9 00 00     stw     r29,0(r9)
    0.00 :	c0235708:       7f a4 eb 78     mr      r4,r29
    0.00 :	c023570c:       7f bc eb 78     mr      r28,r29
    0.00 :	c0235710:       7c 09 03 a6     mtctr   r0
    0.00 :	c0235714:       4e 80 04 21     bctrl
    0.00 :	c0235718:       7f a9 eb 78     mr      r9,r29
    0.00 :	c023571c:       83 bd 00 04     lwz     r29,4(r29)
    0.00 :	c0235720:       4b ff fe c8     b       c02355e8 <__rb_insert_augmented+0x68>
    0.00 :	c0235724:       60 00 00 00     nop
    0.00 :	c0235728:       60 00 00 00     nop
    0.00 :	c023572c:       60 00 00 00     nop
    0.00 :	c0235730:       91 2b 00 08     stw     r9,8(r11)
    0.00 :	c0235734:       4b ff fe f0     b       c0235624 <__rb_insert_augmented+0xa4>
    0.00 :	c0235738:       7f c3 f3 78     mr      r3,r30
    0.00 :	c023573c:       38 00 00 01     li      r0,1
    0.00 :	c0235740:       90 03 00 00     stw     r0,0(r3)
    0.00 :	c0235744:       4b ff fe f4     b       c0235638 <__rb_insert_augmented+0xb8>
    0.00 :	c0235748:       60 00 00 00     nop
    0.00 :	c023574c:       60 00 00 00     nop
    0.00 :	c0235750:       81 63 00 04     lwz     r11,4(r3)
    0.00 :	c0235754:       2f 8b 00 00     cmpwi   cr7,r11,0
    0.00 :	c0235758:       91 69 00 08     stw     r11,8(r9)
    0.00 :	c023575c:       91 23 00 04     stw     r9,4(r3)
    0.00 :	c0235760:       41 9e 00 0c     beq-    cr7,c023576c <__rb_insert_augmented+0x1ec>
    0.00 :	c0235764:       61 20 00 01     ori     r0,r9,1
    0.00 :	c0235768:       90 0b 00 00     stw     r0,0(r11)
    0.00 :	c023576c:       80 01 00 08     lwz     r0,8(r1)
    0.00 :	c0235770:       7d 23 4b 78     mr      r3,r9
    0.00 :	c0235774:       93 a9 00 00     stw     r29,0(r9)
    0.00 :	c0235778:       7f a4 eb 78     mr      r4,r29
    0.00 :	c023577c:       7f bc eb 78     mr      r28,r29
    0.00 :	c0235780:       7c 09 03 a6     mtctr   r0
    0.00 :	c0235784:       4e 80 04 21     bctrl
    0.00 :	c0235788:       7f a9 eb 78     mr      r9,r29
    0.00 :	c023578c:       83 bd 00 08     lwz     r29,8(r29)
    0.00 :	c0235790:       4b ff ff 20     b       c02356b0 <__rb_insert_augmented+0x130>
 Percent |	Source code & Disassembly of vmlinux
------------------------------------------------
         :
         :
         :
         :	Disassembly of section .text:
         :
         :	c040b270 <__alloc_skb>:
  100.00 :	c040b270:       94 21 ff d0     stwu    r1,-48(r1)
    0.00 :	c040b274:       7c 08 02 a6     mflr    r0
    0.00 :	c040b278:       7d 80 00 26     mfcr    r12
    0.00 :	c040b27c:       90 01 00 34     stw     r0,52(r1)
    0.00 :	c040b280:       54 a0 07 fe     clrlwi  r0,r5,31
    0.00 :	c040b284:       2e 00 00 00     cmpwi   cr4,r0,0
    0.00 :	c040b288:       bf 81 00 20     stmw    r28,32(r1)
    0.00 :	c040b28c:       91 81 00 1c     stw     r12,28(r1)
    0.00 :	c040b290:       7c 7c 1b 78     mr      r28,r3
    0.00 :	c040b294:       7c 9e 23 78     mr      r30,r4
    0.00 :	c040b298:       41 92 01 18     beq-    cr4,c040b3b0 <__alloc_skb+0x140>
    0.00 :	c040b29c:       3d 20 c0 72     lis     r9,-16270
    0.00 :	c040b2a0:       83 a9 9e 50     lwz     r29,-25008(r9)
    0.00 :	c040b2a4:       3d 20 c0 72     lis     r9,-16270
    0.00 :	c040b2a8:       80 09 c4 3c     lwz     r0,-15300(r9)
    0.00 :	c040b2ac:       2f 80 00 00     cmpwi   cr7,r0,0
    0.00 :	c040b2b0:       40 9e 01 24     bne-    cr7,c040b3d4 <__alloc_skb+0x164>
    0.00 :	c040b2b4:       7f a3 eb 78     mr      r3,r29
    0.00 :	c040b2b8:       57 c4 00 3c     rlwinm  r4,r30,0,0,30
    0.00 :	c040b2bc:       4b cd 04 25     bl      c00db6e0 <kmem_cache_alloc>
    0.00 :	c040b2c0:       7c 7f 1b 79     mr.     r31,r3
    0.00 :	c040b2c4:       41 82 00 c4     beq-    c040b388 <__alloc_skb+0x118>
    0.00 :	c040b2c8:       7c 00 f9 ec     dcbtst  r0,r31
    0.00 :	c040b2cc:       3b 9c 00 3f     addi    r28,r28,63
    0.00 :	c040b2d0:       7f c4 f3 78     mr      r4,r30
    0.00 :	c040b2d4:       57 83 00 32     rlwinm  r3,r28,0,0,25
    0.00 :	c040b2d8:       38 a1 00 08     addi    r5,r1,8
    0.00 :	c040b2dc:       38 63 00 c0     addi    r3,r3,192
    0.00 :	c040b2e0:       4b ff ff 01     bl      c040b1e0 <__kmalloc_reserve.clone.56>
    0.00 :	c040b2e4:       7c 7e 1b 79     mr.     r30,r3
    0.00 :	c040b2e8:       41 82 00 d8     beq-    c040b3c0 <__alloc_skb+0x150>
    0.00 :	c040b2ec:       4b cc ec 45     bl      c00d9f30 <ksize>
    0.00 :	c040b2f0:       3b 83 ff 40     addi    r28,r3,-192
    0.00 :	c040b2f4:       7f be e2 15     add.    r29,r30,r28
    0.00 :	c040b2f8:       41 82 00 08     beq-    c040b300 <__alloc_skb+0x90>
    0.00 :	c040b2fc:       7c 00 e9 ec     dcbtst  r0,r29
    0.00 :	c040b300:       7f e3 fb 78     mr      r3,r31
    0.00 :	c040b304:       38 80 00 00     li      r4,0
    0.00 :	c040b308:       38 a0 00 98     li      r5,152
    0.00 :	c040b30c:       3b 9c 01 80     addi    r28,r28,384
    0.00 :	c040b310:       4b c0 d5 9d     bl      c00188ac <memset>
    0.00 :	c040b314:       89 21 00 08     lbz     r9,8(r1)
    0.00 :	c040b318:       38 00 00 00     li      r0,0
    0.00 :	c040b31c:       93 9f 00 a8     stw     r28,168(r31)
    0.00 :	c040b320:       3b 80 00 01     li      r28,1
    0.00 :	c040b324:       51 20 6c a4     rlwimi  r0,r9,13,18,18
    0.00 :	c040b328:       90 1f 00 78     stw     r0,120(r31)
    0.00 :	c040b32c:       93 9f 00 ac     stw     r28,172(r31)
    0.00 :	c040b330:       93 df 00 a0     stw     r30,160(r31)
    0.00 :	c040b334:       7f a3 eb 78     mr      r3,r29
    0.00 :	c040b338:       93 df 00 a4     stw     r30,164(r31)
    0.00 :	c040b33c:       38 80 00 00     li      r4,0
    0.00 :	c040b340:       93 df 00 98     stw     r30,152(r31)
    0.00 :	c040b344:       38 a0 00 24     li      r5,36
    0.00 :	c040b348:       93 bf 00 9c     stw     r29,156(r31)
    0.00 :	c040b34c:       4b c0 d5 61     bl      c00188ac <memset>
    0.00 :	c040b350:       93 9d 00 24     stw     r28,36(r29)
    0.00 :	c040b354:       41 92 00 34     beq-    cr4,c040b388 <__alloc_skb+0x118>
    0.00 :	c040b358:       80 1f 00 64     lwz     r0,100(r31)
    0.00 :	c040b35c:       53 80 9a d8     rlwimi  r0,r28,19,11,12
    0.00 :	c040b360:       90 1f 00 64     stw     r0,100(r31)
    0.00 :	c040b364:       93 9f 01 60     stw     r28,352(r31)
    0.00 :	c040b368:       81 7f 01 14     lwz     r11,276(r31)
    0.00 :	c040b36c:       39 40 00 00     li      r10,0
    0.00 :	c040b370:       80 1f 01 28     lwz     r0,296(r31)
    0.00 :	c040b374:       51 4b 9a d8     rlwimi  r11,r10,19,11,12
    0.00 :	c040b378:       91 7f 01 14     stw     r11,276(r31)
    0.00 :	c040b37c:       89 61 00 08     lbz     r11,8(r1)
    0.00 :	c040b380:       51 60 6c a4     rlwimi  r0,r11,13,18,18
    0.00 :	c040b384:       90 1f 01 28     stw     r0,296(r31)
    0.00 :	c040b388:       80 01 00 34     lwz     r0,52(r1)
    0.00 :	c040b38c:       7f e3 fb 78     mr      r3,r31
    0.00 :	c040b390:       81 81 00 1c     lwz     r12,28(r1)
    0.00 :	c040b394:       bb 81 00 20     lmw     r28,32(r1)
    0.00 :	c040b398:       38 21 00 30     addi    r1,r1,48
    0.00 :	c040b39c:       7c 08 03 a6     mtlr    r0
    0.00 :	c040b3a0:       7d 80 81 20     mtcrf   8,r12
    0.00 :	c040b3a4:       4e 80 00 20     blr
    0.00 :	c040b3a8:       60 00 00 00     nop
    0.00 :	c040b3ac:       60 00 00 00     nop
    0.00 :	c040b3b0:       3d 20 c0 72     lis     r9,-16270
    0.00 :	c040b3b4:       83 a9 9e 4c     lwz     r29,-25012(r9)
    0.00 :	c040b3b8:       4b ff fe ec     b       c040b2a4 <__alloc_skb+0x34>
    0.00 :	c040b3bc:       60 00 00 00     nop
    0.00 :	c040b3c0:       7f e4 fb 78     mr      r4,r31
    0.00 :	c040b3c4:       7f a3 eb 78     mr      r3,r29
    0.00 :	c040b3c8:       4b cc f0 c9     bl      c00da490 <kmem_cache_free>
    0.00 :	c040b3cc:       3b e0 00 00     li      r31,0
    0.00 :	c040b3d0:       4b ff ff b8     b       c040b388 <__alloc_skb+0x118>
    0.00 :	c040b3d4:       70 a0 00 02     andi.   r0,r5,2
    0.00 :	c040b3d8:       63 c0 20 00     ori     r0,r30,8192
    0.00 :	c040b3dc:       7f de 00 9e     .long 0x7fde009e
    0.00 :	c040b3e0:       4b ff fe d4     b       c040b2b4 <__alloc_skb+0x44>
# Events: 8  cycles
#
# Overhead  Command      Shared Object                  Symbol
# ........  .......  .................  ......................
#
    97.04%     find  [kernel.kallsyms]  [k] __alloc_skb
     2.32%     find  [kernel.kallsyms]  [k] mod_timer
     0.62%     find  [kernel.kallsyms]  [k] __rb_insert_augmented
     0.01%     find  [kernel.kallsyms]  [k] perf_event_comm_output
     0.00%     find  [kernel.kallsyms]  [k] perf_event_comm

#
# (For a higher level overview, try: perf report --sort comm,dso) #

#8 lore raw
Quoted 14 lines · Scott Wood
Quoted 7 lines · Anton Blanchard

Hi Scott,

Quoted 1 line

What specifically should I do to test it?

Could you double check perf annotate works? I'm 99% sure it will but that is what was failing on ppc64.

I'm not really sure what it's supposed to look like when "perf annotate" works. It spits a bunch of unreadable[1] dark-blue-on-black assembly code at me, all with "0.00 :" in the left column.

Quoted 6 lines

Oh, wait -- some lines have "100.00 : " on the left, in even-more-unreadable dark-red-on-black.

Apart from the annoying colors, is there anything specific I should be looking for? Some sort of error message, or output that actually makes sense?

The colours look fine on my terminal, so I don't know what you've done there. If you care you can use "--stdio" to use the plainer interface, though it still uses colours.

That output looks fine in terms of the bug Anton was chasing. As far as only ever hitting one instruction that does look weird.

cheers

#9 lore raw
Quoted 19 lines · Michael Ellerman
Quoted 7 lines · Scott Wood
Quoted 6 lines · Anton Blanchard

Hi Scott,

Quoted 1 line

What specifically should I do to test it?

Could you double check perf annotate works? I'm 99% sure it will

but

Quoted 10 lines
Quoted 1 line

that is what was failing on ppc64.

I'm not really sure what it's supposed to look like when "perf annotate" works. It spits a bunch of unreadable[1] dark-blue-on-black assembly code at me, all with "0.00 :" in the left column.

Oh, wait -- some lines have "100.00 : " on the left, in even-more-unreadable dark-red-on-black.

Quoted 6 lines
Quoted 3 lines

Apart from the annoying colors, is there anything specific I should be looking for? Some sort of error message, or output that actually makes sense?

The colours look fine on my terminal, so I don't know what you've done there.

It probably looks better if the terminal is configured to have a light background (which of course makes some other programs look worse), or (as I noted) if you've got your monitor set to be very bright. I now see that xfce4-terminal lets me redefine the standard colors, though, so that should help.

Quoted 5 lines

If you care you can use "--stdio" to use the plainer interface, though it still uses colours.

That output looks fine in terms of the bug Anton was chasing. As far as only ever hitting one instruction that does look weird.

OK. I'll add "investigate weird e500 perf annotate results" to the TODO list...

-Scott

#10 lore raw
In reply to Scott Wood

Hi Scott,

Quoted 11 lines

I'm not really sure what it's supposed to look like when "perf annotate" works. It spits a bunch of unreadable[1] dark-blue-on-black assembly code at me, all with "0.00 :" in the left column.

Oh, wait -- some lines have "100.00 : " on the left, in even-more-unreadable dark-red-on-black.

Apart from the annoying colors, is there anything specific I should be looking for? Some sort of error message, or output that actually makes sense?

Thanks for testing! Ben, I think the patch is good to go.

Anton

#11 lore raw
Quoted 5 lines · Anton Blanchard
Quoted 3 lines

Apart from the annoying colors, is there anything specific I should be looking for? Some sort of error message, or output that actually makes sense?

Thanks for testing! Ben, I think the patch is good to go.

Sent it yesterday to Linus, it's upstream already :-)

Cheers,
Ben.

#12 lore raw
Quoted 5 lines · Benjamin Herrenschmidt
Quoted 4 lines · Anton Blanchard
Quoted 3 lines

Apart from the annoying colors, is there anything specific I should be looking for? Some sort of error message, or output that actually makes sense?

Quoted 7 lines
Quoted 1 line

Thanks for testing! Ben, I think the patch is good to go.

Sent it yesterday to Linus, it's upstream already :-)

Cheers,
Ben.

Sorry I'm a bit late to the thread, I've ben swamped. Has someone tested this with kexec/kdump? Thats why the origional patch was created, because when kexec loads the kernel at a different physical address, the relocations messed with the module crc's, and modules couldn't load during the kexec boot. Assuming that kernaddr_start gets set appropriately during boot, using PHYSICAL_START should be fine, but I wanted to check, and don't currently have access to a powerpc system to do so.
Neil

Quoted 3 lines
#13 lore raw

Hi Neil,

Quoted 8 lines

Sorry I'm a bit late to the thread, I've ben swamped. Has someone tested this with kexec/kdump? Thats why the origional patch was created, because when kexec loads the kernel at a different physical address, the relocations messed with the module crc's, and modules couldn't load during the kexec boot. Assuming that kernaddr_start gets set appropriately during boot, using PHYSICAL_START should be fine, but I wanted to check, and don't currently have access to a powerpc system to do so. Neil

I tested a relocatable kernel forced to run at a non zero physical address (ie basically kdump). I verified CRCs were bad with your original patch backed out, and were good with this patch applied.

Anton

Acked-by
#14 lore raw
Quoted 12 lines · Anton Blanchard

Hi Neil,

Quoted 8 lines

Sorry I'm a bit late to the thread, I've ben swamped. Has someone tested this with kexec/kdump? Thats why the origional patch was created, because when kexec loads the kernel at a different physical address, the relocations messed with the module crc's, and modules couldn't load during the kexec boot. Assuming that kernaddr_start gets set appropriately during boot, using PHYSICAL_START should be fine, but I wanted to check, and don't currently have access to a powerpc system to do so. Neil

Quoted 6 lines

I tested a relocatable kernel forced to run at a non zero physical address (ie basically kdump). I verified CRCs were bad with your original patch backed out, and were good with this patch applied.

Anton

Perfect, sounds like a sufficient test to me.