Skip to contentSkip to conversation

sysrq-m oops

BUG john stultz 4 participants 7 messages Jun 7, 2007 – Jun 8, 2007

lore
Share & tools
lore
#0 lore raw

Hey All,
With 2.6.21 and the current -git, we're seeing the following oops when
we try sysrq-m:

...
Node 1 Normal: 85*4kB 34*8kB 20*16kB 4*32kB 3*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 953*4096kB =
3906020kB
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap = 2040212kB
Total swap = 2040212kB

Unable to handle kernel paging request at 0000000003480000 RIP: 
 [<ffffffff8102a16d>] show_mem+0xdd/0x1e0
PGD 2052e9067 PUD 2052ea067 PMD 0 
Oops: 0000 [1] PREEMPT SMP 
CPU 3 
Pid: 0, comm: swapper Not tainted 2.6.22-rc3-git7john #6
RIP: 0010:[<ffffffff8102a16d>]  [<ffffffff8102a16d>] show_mem+0xdd/0x1e0
RSP: 0018:ffff810211f8bdf8  EFLAGS: 00010006
RAX: 0000000000780000 RBX: 00000000000f0000 RCX: 0000000003480000
RDX: 0000000003480000 RSI: ffff81000210a0f0 RDI: 0000000000112000
RBP: 00000000000f0000 R08: 00000000ffffffff R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff810000012189
R13: 0000000000007179 R14: 0000000000000000 R15: 0000000000000000
FS:  00002ba3b8bdbf40(0000) GS:ffff810211f629c0(0000)

knlGS:0000000000000000
CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 0000000003480000 CR3: 00000002052e8000 CR4: 00000000000006e0 Process swapper (pid: 0, threadinfo ffff8101118be000, task ffff810211f70080)

Stack:  0000000000000000 ffffffff81a90460 0000000000000001
000000000000006d
 ffff810110bf6000 0000000000000006 0000000000000002 ffffffff812e0046
 ffff81011584fbc0 ffffffff820cc590 0000000000000061 000000000000006d
Call Trace:
 <IRQ>  [<ffffffff812e0046>] __handle_sysrq+0x86/0x140
 [<ffffffff8134309c>] receive_chars+0x27c/0x300
 [<ffffffff8105a600>] hrtimer_wakeup+0x0/0x30
 [<ffffffff8105c857>] clocksource_get_next+0x47/0x60
 [<ffffffff813444d2>] serial8250_interrupt+0x142/0x160
 [<ffffffff81076c94>] handle_IRQ_event+0x34/0x70
 [<ffffffff8107838a>] handle_edge_irq+0xca/0x150
 [<ffffffff8100d6cd>] do_IRQ+0xbd/0x1b0
 [<ffffffff81008ef0>] default_idle+0x0/0x40
 [<ffffffff8100a971>] ret_from_intr+0x0/0xa
 <EOI>  [<ffffffff816ab340>] unix_poll+0x0/0xb0
 [<ffffffff81008f19>] default_idle+0x29/0x40
 [<ffffffff81008f9f>] cpu_idle+0x6f/0xe0


Code: 8b 02 f6 c4 04 75 92 8b 02 66 85 c0 79 7a 48 83 c5 01 49 83 
RIP  [<ffffffff8102a16d>] show_mem+0xdd/0x1e0
 RSP <ffff810211f8bdf8>
CR2: 0000000003480000
Kernel panic - not syncing: Aiee, killing interrupt handler!

I'll keep digging but I wanted to see if anyone had any quick thoughts or suggestions.

thanks
-john

#1 lore raw
Quoted 4 lines · john stultz

Hey All,
With 2.6.21 and the current -git, we're seeing the following oops when
we try sysrq-m:

It's here in arch/x86_64/mm/init.c::show_mem():

        for_each_online_pgdat(pgdat) {
               for (i = 0; i < pgdat->node_spanned_pages; ++i) {
                        page = pfn_to_page(pgdat->node_start_pfn + i);
                        total++;
 ======>                if (PageReserved(page))
                                reserved++;
                        else if (PageSwapCache(page))
                                cached++;
                        else if (page_count(page))
                                shared += page_count(page) - 1;
               }
        }

page is completely bogus (it's 0x0000000003480000)

#2 lore raw

Chuck Ebbert wrote: [Thu Jun 07 2007, 11:42:38AM EDT]

Quoted 10 lines · john stultz
Quoted 4 lines

Hey All,
With 2.6.21 and the current -git, we're seeing the following oops when
we try sysrq-m:

It's here in arch/x86_64/mm/init.c::show_mem():

        for_each_online_pgdat(pgdat) {
               for (i = 0; i < pgdat->node_spanned_pages; ++i) {

This is probably with sparsemem? I'm working with [email protected] to test a patch. Basically you need to validate the pfn because it could be in a hole. Most arches which support sparsemem perform this check.

			if (!pfn_valid(pgdat->node_start_pfn + i))
				continue;

bob

Quoted 17 lines
                        page = pfn_to_page(pgdat->node_start_pfn + i);
                        total++;
 ======>                if (PageReserved(page))
                                reserved++;
                        else if (PageSwapCache(page))
                                cached++;
                        else if (page_count(page))
                                shared += page_count(page) - 1;
               }
        }

page is completely bogus (it's 0x0000000003480000)
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#3 lore raw
Quoted 11 lines · Bob Picco

Chuck Ebbert wrote: [Thu Jun 07 2007, 11:42:38AM EDT]

Quoted 10 lines · john stultz
Quoted 4 lines

Hey All,
With 2.6.21 and the current -git, we're seeing the following oops when
we try sysrq-m:

It's here in arch/x86_64/mm/init.c::show_mem():

        for_each_online_pgdat(pgdat) {
               for (i = 0; i < pgdat->node_spanned_pages; ++i) {
Quoted 7 lines

This is probably with sparsemem? I'm working with [email protected] to test a patch. Basically you need to validate the pfn because it could be in a hole. Most arches which support sparsemem perform this check.

			if (!pfn_valid(pgdat->node_start_pfn + i))
				continue;

Yep. That catch avoids the issue! Sounds like a 2.6.21.x candidate.

thanks so much!
-john

#4 lore raw

john stultz wrote: [Thu Jun 07 2007, 03:54:41PM EDT]
[snip]
john you are welcome.

We aren't sampling for holes in memory. Thus we encounter a section hole with empty section map pointer for SPARSEMEM and OOPs for show_mem. This issue has been seen in 2.6.21, current git and current mm. The patch below is for mainline and mm. It was boot tested for SPARSEMEM, current VMEMMAP of Andy's in mm ml and DISCONTIGMEM. A slightly different patch will be posted to stable for 2.6.21.

Previous to commit f0a5a58aa812b31fd9f197c4ba48245942364eae memory_present was called for node_start_pfn to node_end_pfn. This would cover the hole(s) with reserved pages and valid sections. Most SPARSEMEM supported arches do a pfn_valid check in show_mem before computing the page structure address.

This issue was brought to my attention on IRC by Arnaldo Carvalho de Melo at [email protected]. Thanks to Arnaldo for testing.

 arch/x86_64/mm/init.c |    2 ++
 1 file changed, 2 insertions(+)
arch/x86_64/mm/init.c +2 -0
Index: linux-2.6.22-rc4-mm1/arch/x86_64/mm/init.c
===================================================================
@@ -79,6 +79,8 @@ void show_mem(void)
 			if (unlikely(i % MAX_ORDER_NR_PAGES == 0)) {
 				touch_nmi_watchdog();
 			}
+			if (!pfn_valid(pgdat->node_start_pfn + i))
+				continue;
 			page = pfn_to_page(pgdat->node_start_pfn + i);
 			total++;
 			if (PageReserved(page))
#5 lore raw
Quoted 29 lines · Bob Picco

john stultz wrote: [Thu Jun 07 2007, 03:54:41PM EDT]
[snip]
john you are welcome.

We aren't sampling for holes in memory. Thus we encounter a section hole with empty section map pointer for SPARSEMEM and OOPs for show_mem. This issue has been seen in 2.6.21, current git and current mm. The patch below is for mainline and mm. It was boot tested for SPARSEMEM, current VMEMMAP of Andy's in mm ml and DISCONTIGMEM. A slightly different patch will be posted to stable for 2.6.21.

Previous to commit f0a5a58aa812b31fd9f197c4ba48245942364eae memory_present was called for node_start_pfn to node_end_pfn. This would cover the hole(s) with reserved pages and valid sections. Most SPARSEMEM supported arches do a pfn_valid check in show_mem before computing the page structure address.

This issue was brought to my attention on IRC by Arnaldo Carvalho de Melo at [email protected]. Thanks to Arnaldo for testing.

 arch/x86_64/mm/init.c |    2 ++
 1 file changed, 2 insertions(+)

Index: linux-2.6.22-rc4-mm1/arch/x86_64/mm/init.c
===================================================================

arch/x86_64/mm/init.c void show_mem(void) · line 86

Quoted 8 lines
arch/x86_64/mm/init.c +2 -0
@@ -79,6 +79,8 @@ void show_mem(void)
 			if (unlikely(i % MAX_ORDER_NR_PAGES == 0)) {
 				touch_nmi_watchdog();
 			}
+			if (!pfn_valid(pgdat->node_start_pfn + i))
+				continue;
 			page = pfn_to_page(pgdat->node_start_pfn + i);
 			total++;
 			if (PageReserved(page))

Yes this is required with SPARSEMEM. Good spot.

-apw

#6 lore raw
In reply to Bob Picco
Quoted 10 lines · Bob Picco

Chuck Ebbert wrote: [Thu Jun 07 2007, 11:42:38AM EDT]

Quoted 9 lines · john stultz
Quoted 4 lines

Hey All,
With 2.6.21 and the current -git, we're seeing the following oops when
we try sysrq-m:

It's here in arch/x86_64/mm/init.c::show_mem():

        for_each_online_pgdat(pgdat) {
               for (i = 0; i < pgdat->node_spanned_pages; ++i) {
Quoted 8 lines

This is probably with sparsemem? I'm working with [email protected] to test a patch. Basically you need to validate the pfn because it could be in a hole. Most arches which support sparsemem perform this check.

			if (!pfn_valid(pgdat->node_start_pfn + i))
				continue;

bob

Yep you would want one of those for sure with sparsemem.

-apw

Quoted 17 lines
Quoted 17 lines
                        page = pfn_to_page(pgdat->node_start_pfn + i);
                        total++;
 ======>                if (PageReserved(page))
                                reserved++;
                        else if (PageSwapCache(page))
                                cached++;
                        else if (page_count(page))
                                shared += page_count(page) - 1;
               }
        }

page is completely bogus (it's 0x0000000003480000)
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/