Skip to contentSkip to conversation

2.6.21-rc4-rt0 BUG: at kernel/fork.c:1033 copy_process()

PATCH In review Michal Piotrowski 4 participants 12 messages Mar 21, 2007 – Mar 23, 2007

lore
Share & tools
lore

Michal Piotrowski

author
#0 lore raw

Hi Ingo,

It might be lockdep related

-#ifdef CONFIG_TRACE_IRQFLAGS
+#if defined(CONFIG_TRACE_IRQFLAGS) && defined(CONFIG_LOCKDEP)

BUG: at kernel/fork.c:1033 copy_process()
[<c0105474>] dump_trace+0x78/0x21a
[<c010564b>] show_trace_log_lvl+0x35/0x54
[<c0105ddc>] show_trace+0x2c/0x2e
[<c0105ea3>] dump_stack+0x29/0x2b
[<c01234e0>] copy_process+0x1d1/0x1440
[<c01249fc>] do_fork+0xa8/0x18f
[<c010239e>] kernel_thread+0x93/0x99
[<c01393ec>] keventd_create_kthread+0x2f/0x7c
[<c01394ae>] kthread_create+0x75/0xbb
[<c011feba>] migration_call+0x5c/0x3cf
[<c04ddb54>] migration_init+0x2b/0x62
 [<c04d14c5>] init+0x56/0x361
[<c0104ffb>] kernel_thread_helper+0x7/0x10
=======================
---------------------------

| preempt count: 00000000 ]
| 0-level deep critical section nesting:
----------------------------------------

http://www.stardust.webpages.pl/files/tbf/bitis-gabonica/2.6.21-rc4-rt0/rt-config
http://www.stardust.webpages.pl/files/tbf/bitis-gabonica/2.6.21-rc4-rt0/rt-dmesg

Regards,
Michal

Signature · 5 lines
Michal K. K. Piotrowski
LTG - Linux Testers Group (PL)
(http://www.stardust.webpages.pl/ltg/)
LTG - Linux Testers Group (EN)
(http://www.stardust.webpages.pl/linux_testers_group_en/)
#1 lore raw

PATCH 1/1 setup_boot_APIC_clock() irq-enable fix

Quoted 5 lines · Michal Piotrowski

Hi Ingo,

2.6.21-rc4-rt0

BUG: at kernel/fork.c:1033 copy_process()

thanks Michal - this is a real bug that affects upstream too. Find the fix below - i've test-booted it and it fixes the warning.

Linus, Andrew, this is a must-have for v2.6.21.

Ingo

--------------->
Subject: [patch] setup_boot_APIC_clock() irq-enable fix
From: Ingo Molnar <[email protected]>

latest -git triggers an irqtrace/lockdep warning of a leaked irqs-off condition:

  BUG: at kernel/fork.c:1033 copy_process()

after some debugging it turns out that commit ca1b940c accidentally left interrupts disabled - which trickled down all the way to the first time we fork a kernel thread and triggered the warning.

the fix is to re-enable interrupts in the 'else' branch of setup_boot_APIC_clock()'s pmtimers calibration path.

Reported-by Michal Piotrowski <[email protected]>
Signed-off-by Ingo Molnar <[email protected]>
---
 arch/i386/kernel/apic.c |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/i386/kernel/apic.c +2 -1
Index: linux/arch/i386/kernel/apic.c
===================================================================
@@ -507,7 +507,8 @@ void __init setup_boot_APIC_clock(void)
 			apic_printk(APIC_VERBOSE, "... jiffies result ok\n");
 		else
 			local_apic_timer_verify_ok = 0;
-	}
+	} else
+		local_irq_enable();
 
 	if (!local_apic_timer_verify_ok) {
 		printk(KERN_WARNING
Acked-by on v1 1/1
#2 lore raw

Re: [patch] setup_boot_APIC_clock() irq-enable fix

Quoted 12 lines · Ingo Molnar

Subject: [patch] setup_boot_APIC_clock() irq-enable fix
From: Ingo Molnar <[email protected]>

latest -git triggers an irqtrace/lockdep warning of a leaked irqs-off condition:

  BUG: at kernel/fork.c:1033 copy_process()

after some debugging it turns out that commit ca1b940c accidentally left interrupts disabled - which trickled down all the way to the first time we fork a kernel thread and triggered the warning.

Quoted 5 lines

the fix is to re-enable interrupts in the 'else' branch of setup_boot_APIC_clock()'s pmtimers calibration path.

Reported-by Michal Piotrowski <[email protected]>
Signed-off-by Ingo Molnar <[email protected]>
Acked-by Thomas Gleixner <[email protected]>
author
#3 lore raw

Re: [patch] setup_boot_APIC_clock() irq-enable fix

In reply to Ingo Molnar
Quoted 11 lines · Ingo Molnar
Quoted 5 lines · Michal Piotrowski

Hi Ingo,

2.6.21-rc4-rt0

BUG: at kernel/fork.c:1033 copy_process()

thanks Michal - this is a real bug that affects upstream too. Find the fix below - i've test-booted it and it fixes the warning.

Problem is fixed, thanks.

Regards,
Michal

Signature · 5 lines
Michal K. K. Piotrowski
LTG - Linux Testers Group (PL)
(http://www.stardust.webpages.pl/ltg/)
LTG - Linux Testers Group (EN)
(http://www.stardust.webpages.pl/linux_testers_group_en/)
author
#4 lore raw

2.6.21-rc4-rt0-kdump (was: Re: [patch] setup_boot_APIC_clock() irq-enable fix)

Michal Piotrowski napisał(a):

Quoted 6 lines · Ingo Molnar
Quoted 5 lines · Michal Piotrowski
Quoted 2 lines

Hi Ingo,

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

2.6.21-rc4-rt0

BUG: at kernel/fork.c:1033 copy_process()

thanks Michal - this is a real bug that affects upstream too. Find the fix below - i've test-booted it and it fixes the warning.

Problem is fixed, thanks.

BTW. It seems that nobody uses -rt as a crash dump kernel ;)

BUG: unable to handle kernel paging request at virtual address f7ebf8c4
 printing eip:
c1610192
*pde = 00000000
stopped custom tracer.
Oops: 0000 [#1]
PREEMPT 
Modules linked in:
CPU:    0
EIP:    0060:[<c1610192>]    Not tainted VLI
EFLAGS: 00010206   (2.6.21-rc4-rt0-kdump #3)

EIP is at copy_oldmem_page+0x4a/0xd0
eax: 000008c4 ebx: f7ebf000 ecx: 00000100 edx: 00000246
esi: f7ebf8c4 edi: c4c520fc ebp: c4d54e30 esp: c4d54e18
ds: 007b es: 007b fs: 00d8 gs: 0000 ss: 0068 preempt:00000001
Process swapper (pid: 1, ti=c4d54000 task=c4d52c20 task.ti=c4d54000)

Stack: c17ab7e0 c183f982 c1969658 00000400 00000400 00037ebf c4d54e5c c16af187 
       00037ebf c4c520fc 00000400 000008c4 00000000 00000000 c4c696e0 00000400 
       c4c520fc c4d54f94 c19a9cfd c4c520fc 00000400 c4d54f78 00000000 c1840996 
Call Trace:
 [<c16af187>] read_from_oldmem+0x73/0x98
 [<c19a9cfd>] vmcore_init+0x26c/0xab7
 [<c199979b>] init+0xaa/0x287
 [<c16044eb>] kernel_thread_helper+0x7/0x10
 =======================

l *copy_oldmem_page+0x4a/0xd0
0xc1610148 is in copy_oldmem_page (arch/i386/kernel/crash_dump.c:35).

30       * copying the data to a pre-allocated kernel page and then copying to user
31       * space in non-atomic context.
32       */

33 ssize_t copy_oldmem_page(unsigned long pfn, char *buf,
34 size_t csize, unsigned long offset, int userbuf)
35 {
36 void *vaddr;
37
38 if (!csize)
39 return 0;

---------------------------
| preempt count: 00000001 ]
| 1-level deep critical section nesting:
----------------------------------------
.. [<c184045a>] .... __spin_lock_irqsave+0x23/0x65
.....[<c1604f8c>] .. ( <= die+0x44/0x24d)

l *0xc184045a
0xc184045a is in __spin_lock_irqsave (kernel/spinlock.c:122).
117 {
118 unsigned long flags;
119

120             local_irq_save(flags);
121             preempt_disable();
122             spin_acquire(&lock->dep_map, 0, 0, _RET_IP_);
123             /*
124              * On lockdep we dont want the hand-coded irq-enable of
125              * _raw_spin_lock_flags() code, because lockdep assumes
126              * that interrupts are not re-enabled during lock-acquire:

l *0xc1604f8c
0xc1604f8c is in die (arch/i386/kernel/traps.c:477). 472
473 oops_enter();
474
475 if (die.lock_owner != raw_smp_processor_id()) {

476                     console_verbose();
477                     spin_lock_irqsave(&die.lock, flags);
478                     die.lock_owner = smp_processor_id();
479                     die.lock_owner_depth = 0;
480                     bust_spinlocks(1);

481 }

Code: 10 05 00 c1 e3 05 03 1d 60 8e d6 c1 89 1c 24 e8 fc 33 00 00 89 c3 83 7d 18 00 75 2a 8b 4d 10 c1 e9 02 8b 45 14 8d 34 03 8b 7d 0c <f3> a5 8b 4d 10 83 e1 03 74 02 f3 a4 e8 cb 10 05 00 89 1c 24 e8 
EIP: [<c1610192>] copy_oldmem_page+0x4a/0xd0 SS:ESP 0068:c4d54e18
Kernel panic - not syncing: Attempted to kill init!
 [<c160496d>] dump_trace+0x78/0x21a
 [<c1604b44>] show_trace_log_lvl+0x35/0x54
 [<c16052c4>] show_trace+0x2c/0x2e
 [<c160538b>] dump_stack+0x29/0x2b
 [<c1618b30>] panic+0x68/0x130
 [<c161b67c>] do_exit+0xa1/0x7e3
 [<c160516a>] die+0x222/0x24d
 [<c1612a3f>] do_page_fault+0x4a1/0x586
 [<c1841044>] error_code+0x74/0x7c
 [<c1610192>] copy_oldmem_page+0x4a/0xd0
 [<c16af187>] read_from_oldmem+0x73/0x98
 [<c19a9cfd>] vmcore_init+0x26c/0xab7
 [<c199979b>] init+0xaa/0x287
 [<c16044eb>] kernel_thread_helper+0x7/0x10
 =======================
---------------------------

| preempt count: 00000001 ]
| 1-level deep critical section nesting:
----------------------------------------
.. [<c1618ae6>] .... panic+0x1e/0x130
.....[<c161b67c>] .. ( <= do_exit+0xa1/0x7e3)

l *0xc1618ae6
0xc1618ae6 is in panic (kernel/panic.c:85).

80               * have preempt disabled. Some functions called from here want
81               * preempt to be disabled. No point enabling it later though...
82               */
83              preempt_disable();
84
85              bust_spinlocks(1);
86              va_start(args, fmt);
87              vsnprintf(buf, sizeof(buf), fmt, args);
88              va_end(args);
89              printk(KERN_EMERG "Kernel panic - not syncing: %s\n",buf);

l *0xc161b67c
0xc161b67c is in do_exit (include/linux/pid_namespace.h:42).
37 kref_put(&ns->kref, free_pid_ns);
38 }
39
40 static inline struct task_struct *child_reaper(struct task_struct *tsk)
41 {
42 return init_pid_ns.child_reaper;
43 }
44
45 #endif /* _LINUX_PID_NS_H */

http://www.stardust.webpages.pl/files/tbf/bitis-gabonica/2.6.21-rc4-rt0/kdump-console.log
http://www.stardust.webpages.pl/files/tbf/bitis-gabonica/2.6.21-rc4-rt0/kdump-config

NOHZ: local_softirq_pending 02 on CPU#1
NOHZ: local_softirq_pending 02 on CPU#0
NOHZ: local_softirq_pending 08 on CPU#0
NOHZ: local_softirq_pending 02 on CPU#1
NOHZ: local_softirq_pending 10 on CPU#0
NOHZ: local_softirq_pending 02 on CPU#0
NOHZ: local_softirq_pending 02 on CPU#1
NOHZ: local_softirq_pending 08 on CPU#0

CPU0 CPU1

  0:        304          0   IO-APIC-edge      timer
  1:       2319          0   IO-APIC-edge      i8042
  7:          0          0   IO-APIC-edge      parport0
  8:          1          0   IO-APIC-edge      rtc
  9:          1          0   IO-APIC-fasteoi   acpi
 12:          3          0   IO-APIC-edge      i8042
 14:        738          0   IO-APIC-edge      ide0
 15:       3050          0   IO-APIC-edge      ide1
 16:      23802          0   IO-APIC-fasteoi   uhci_hcd:usb2, uhci_hcd:usb5
 17:      33123          0   IO-APIC-fasteoi   eth1
 19:      29280          0   IO-APIC-fasteoi   libata, uhci_hcd:usb4
 20:          2          0   IO-APIC-fasteoi   ehci_hcd:usb1
 21:          0          0   IO-APIC-fasteoi   uhci_hcd:usb3
 22:      38572          0   IO-APIC-fasteoi   Intel ICH5
NMI:          0          0
LOC:     351305     268386
ERR:          0
MIS:          0

Hibernation is still broken.

http://www.stardust.webpages.pl/files/tbf/bitis-gabonica/2.6.21-rc4-rt0/console.log
http://www.stardust.webpages.pl/files/tbf/bitis-gabonica/2.6.21-rc4-rt0/rt-config

Regards,
Michal

Signature · 5 lines
Michal K. K. Piotrowski
LTG - Linux Testers Group (PL)
(http://www.stardust.webpages.pl/ltg/)
LTG - Linux Testers Group (EN)
(http://www.stardust.webpages.pl/linux_testers_group_en/)
#5 lore raw

Re: 2.6.21-rc4-rt0-kdump (was: Re: [patch] setup_boot_APIC_clock() irq-enable fix)

Quoted 49 lines · Michal Piotrowski

Michal Piotrowski napisał(a):

Quoted 6 lines · Ingo Molnar
Quoted 5 lines · Michal Piotrowski
Quoted 2 lines

Hi Ingo,

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

2.6.21-rc4-rt0

BUG: at kernel/fork.c:1033 copy_process()

thanks Michal - this is a real bug that affects upstream too. Find the fix below - i've test-booted it and it fixes the warning.

Problem is fixed, thanks.

BTW. It seems that nobody uses -rt as a crash dump kernel ;)

BUG: unable to handle kernel paging request at virtual address f7ebf8c4
 printing eip:
c1610192
*pde = 00000000
stopped custom tracer.
Oops: 0000 [#1]
PREEMPT 
Modules linked in:
CPU:    0
EIP:    0060:[<c1610192>]    Not tainted VLI
EFLAGS: 00010206   (2.6.21-rc4-rt0-kdump #3)

EIP is at copy_oldmem_page+0x4a/0xd0
eax: 000008c4 ebx: f7ebf000 ecx: 00000100 edx: 00000246
esi: f7ebf8c4 edi: c4c520fc ebp: c4d54e30 esp: c4d54e18
ds: 007b es: 007b fs: 00d8 gs: 0000 ss: 0068 preempt:00000001
Process swapper (pid: 1, ti=c4d54000 task=c4d52c20 task.ti=c4d54000)

Stack: c17ab7e0 c183f982 c1969658 00000400 00000400 00037ebf c4d54e5c c16af187 
       00037ebf c4c520fc 00000400 000008c4 00000000 00000000 c4c696e0 00000400 
       c4c520fc c4d54f94 c19a9cfd c4c520fc 00000400 c4d54f78 00000000 c1840996 
Call Trace:
 [<c16af187>] read_from_oldmem+0x73/0x98
 [<c19a9cfd>] vmcore_init+0x26c/0xab7
 [<c199979b>] init+0xaa/0x287
 [<c16044eb>] kernel_thread_helper+0x7/0x10
 =======================

l *copy_oldmem_page+0x4a/0xd0
0xc1610148 is in copy_oldmem_page (arch/i386/kernel/crash_dump.c:35).

30       * copying the data to a pre-allocated kernel page and then copying to user
31       * space in non-atomic context.
32       */
Quoted 8 lines

33 ssize_t copy_oldmem_page(unsigned long pfn, char *buf,
34 size_t csize, unsigned long offset, int userbuf)
35 {
36 void *vaddr;
37
38 if (!csize)
39 return 0;

Can you please paste the disassembly of copy_oldmem_page() on your system. Not sure from where this faulting address 0xf7ebf8c4 is coming. We are still in vmcore_init(), so we should be copying the data to kernel buffers only. This looks like a valid kernel address.

Can you also put some printk() here to find out from where 0xf7ebf8c4 has come? It does not look like a fixed kernel virutual address returned by kmap_atomic_pfn(). Then is it passed by kernel as a parameter to copy_oldmem_page()?

Thanks
Vivek

author
#6 lore raw

Re: 2.6.21-rc4-rt0-kdump

Vivek Goyal napisał(a):

Quoted 53 lines · Michal Piotrowski
Quoted 43 lines

Michal Piotrowski napisał(a):

Quoted 9 lines · Ingo Molnar
Quoted 7 lines · Michal Piotrowski
Quoted 4 lines

Hi Ingo,
2.6.21-rc4-rt0

BUG: at kernel/fork.c:1033 copy_process()

thanks Michal - this is a real bug that affects upstream too. Find the fix below - i've test-booted it and it fixes the warning.

Problem is fixed, thanks.

BTW. It seems that nobody uses -rt as a crash dump kernel ;)

BUG: unable to handle kernel paging request at virtual address f7ebf8c4
 printing eip:
c1610192
*pde = 00000000
stopped custom tracer.
Oops: 0000 [#1]
PREEMPT 
Modules linked in:
CPU:    0
EIP:    0060:[<c1610192>]    Not tainted VLI
EFLAGS: 00010206   (2.6.21-rc4-rt0-kdump #3)

EIP is at copy_oldmem_page+0x4a/0xd0
eax: 000008c4 ebx: f7ebf000 ecx: 00000100 edx: 00000246
esi: f7ebf8c4 edi: c4c520fc ebp: c4d54e30 esp: c4d54e18
ds: 007b es: 007b fs: 00d8 gs: 0000 ss: 0068 preempt:00000001
Process swapper (pid: 1, ti=c4d54000 task=c4d52c20 task.ti=c4d54000)

Stack: c17ab7e0 c183f982 c1969658 00000400 00000400 00037ebf c4d54e5c c16af187 
       00037ebf c4c520fc 00000400 000008c4 00000000 00000000 c4c696e0 00000400 
       c4c520fc c4d54f94 c19a9cfd c4c520fc 00000400 c4d54f78 00000000 c1840996 
Call Trace:
 [<c16af187>] read_from_oldmem+0x73/0x98
 [<c19a9cfd>] vmcore_init+0x26c/0xab7
 [<c199979b>] init+0xaa/0x287
 [<c16044eb>] kernel_thread_helper+0x7/0x10
 =======================

l *copy_oldmem_page+0x4a/0xd0
0xc1610148 is in copy_oldmem_page (arch/i386/kernel/crash_dump.c:35).

30       * copying the data to a pre-allocated kernel page and then copying to user
31       * space in non-atomic context.
32       */
Quoted 8 lines

33 ssize_t copy_oldmem_page(unsigned long pfn, char *buf,
34 size_t csize, unsigned long offset, int userbuf)
35 {
36 void *vaddr;
37
38 if (!csize)
39 return 0;

Quoted 1 line

Can you please paste the disassembly of copy_oldmem_page() on your system.

disassemble *copy_oldmem_page
Dump of assembler code for function copy_oldmem_page:

0xc1610148 <copy_oldmem_page+0>:        push   %ebp
0xc1610149 <copy_oldmem_page+1>:        mov    %esp,%ebp
0xc161014b <copy_oldmem_page+3>:        push   %edi
0xc161014c <copy_oldmem_page+4>:        push   %esi
0xc161014d <copy_oldmem_page+5>:        push   %ebx
0xc161014e <copy_oldmem_page+6>:        sub    $0xc,%esp
0xc1610151 <copy_oldmem_page+9>:        call   0xc160f5c4 <mcount>
0xc1610156 <copy_oldmem_page+14>:       mov    0x8(%ebp),%ebx
0xc1610159 <copy_oldmem_page+17>:       xor    %edx,%edx
0xc161015b <copy_oldmem_page+19>:       cmpl   $0x0,0x10(%ebp)
0xc161015f <copy_oldmem_page+23>:       je     0xc161020d <copy_oldmem_page+197>
0xc1610165 <copy_oldmem_page+29>:       call   0xc1661258 <pagefault_disable>
0xc161016a <copy_oldmem_page+34>:       shl    $0x5,%ebx
0xc161016d <copy_oldmem_page+37>:       add    0xc1d68e60,%ebx
0xc1610173 <copy_oldmem_page+43>:       mov    %ebx,(%esp)
0xc1610176 <copy_oldmem_page+46>:       call   0xc1613577 <kmap>
0xc161017b <copy_oldmem_page+51>:       mov    %eax,%ebx
0xc161017d <copy_oldmem_page+53>:       cmpl   $0x0,0x18(%ebp)
0xc1610181 <copy_oldmem_page+57>:       jne    0xc16101ad <copy_oldmem_page+101>
0xc1610183 <copy_oldmem_page+59>:       mov    0x10(%ebp),%ecx
0xc1610186 <copy_oldmem_page+62>:       shr    $0x2,%ecx
0xc1610189 <copy_oldmem_page+65>:       mov    0x14(%ebp),%eax
0xc161018c <copy_oldmem_page+68>:       lea    (%ebx,%eax,1),%esi
0xc161018f <copy_oldmem_page+71>:       mov    0xc(%ebp),%edi
0xc1610192 <copy_oldmem_page+74>:       rep movsl %ds:(%esi),%es:(%edi)
0xc1610194 <copy_oldmem_page+76>:       mov    0x10(%ebp),%ecx
0xc1610197 <copy_oldmem_page+79>:       and    $0x3,%ecx
0xc161019a <copy_oldmem_page+82>:       je     0xc161019e <copy_oldmem_page+86>
0xc161019c <copy_oldmem_page+84>:       rep movsb %ds:(%esi),%es:(%edi)
0xc161019e <copy_oldmem_page+86>:       call   0xc166126e <pagefault_enable>
0xc16101a3 <copy_oldmem_page+91>:       mov    %ebx,(%esp)
0xc16101a6 <copy_oldmem_page+94>:       call   0xc1613533 <kunmap_virt>
0xc16101ab <copy_oldmem_page+99>:       jmp    0xc161020a <copy_oldmem_page+194>
0xc16101ad <copy_oldmem_page+101>:      mov    0xc19d4004,%edi
0xc16101b3 <copy_oldmem_page+107>:      test   %edi,%edi
0xc16101b5 <copy_oldmem_page+109>:      jne    0xc16101ca <copy_oldmem_page+130>
0xc16101b7 <copy_oldmem_page+111>:      movl   $0xc18bbc93,(%esp)
0xc16101be <copy_oldmem_page+118>:      call   0xc1619671 <printk>
0xc16101c3 <copy_oldmem_page+123>:      mov    $0xfffffff2,%edx
0xc16101c8 <copy_oldmem_page+128>:      jmp    0xc161020d <copy_oldmem_page+197>
0xc16101ca <copy_oldmem_page+130>:      mov    $0x400,%ecx
0xc16101cf <copy_oldmem_page+135>:      mov    %eax,%esi
0xc16101d1 <copy_oldmem_page+137>:      rep movsl %ds:(%esi),%es:(%edi)
0xc16101d3 <copy_oldmem_page+139>:      call   0xc166126e <pagefault_enable>
0xc16101d8 <copy_oldmem_page+144>:      mov    %ebx,(%esp)
0xc16101db <copy_oldmem_page+147>:      call   0xc1613533 <kunmap_virt>
0xc16101e0 <copy_oldmem_page+152>:      mov    0x10(%ebp),%eax
0xc16101e3 <copy_oldmem_page+155>:      mov    %eax,0x8(%esp)
0xc16101e7 <copy_oldmem_page+159>:      mov    0xc19d4004,%eax
0xc16101ec <copy_oldmem_page+164>:      add    %eax,0x14(%ebp)
0xc16101ef <copy_oldmem_page+167>:      mov    0x14(%ebp),%eax
0xc16101f2 <copy_oldmem_page+170>:      mov    %eax,0x4(%esp)
0xc16101f6 <copy_oldmem_page+174>:      mov    0xc(%ebp),%eax
0xc16101f9 <copy_oldmem_page+177>:      mov    %eax,(%esp)
0xc16101fc <copy_oldmem_page+180>:      call   0xc1700e98 <copy_to_user>
0xc1610201 <copy_oldmem_page+185>:      mov    $0xfffffff2,%edx
0xc1610206 <copy_oldmem_page+190>:      test   %eax,%eax
0xc1610208 <copy_oldmem_page+192>:      jne    0xc161020d <copy_oldmem_page+197>
0xc161020a <copy_oldmem_page+194>:      mov    0x10(%ebp),%edx
0xc161020d <copy_oldmem_page+197>:      mov    %edx,%eax
0xc161020f <copy_oldmem_page+199>:      add    $0xc,%esp
0xc1610212 <copy_oldmem_page+202>:      pop    %ebx
0xc1610213 <copy_oldmem_page+203>:      pop    %esi
0xc1610214 <copy_oldmem_page+204>:      pop    %edi
0xc1610215 <copy_oldmem_page+205>:      pop    %ebp
0xc1610216 <copy_oldmem_page+206>:      ret

End of assembler dump.

Quoted 4 lines

Not sure from where this faulting address 0xf7ebf8c4 is coming. We are still in vmcore_init(), so we should be copying the data to kernel buffers only. This looks like a valid kernel address.

Quoted 8 lines

Can you also put some printk() here to find out from where 0xf7ebf8c4 has come? It does not look like a fixed kernel virutual address returned by kmap_atomic_pfn(). Then is it passed by kernel as a parameter to copy_oldmem_page()?

Thanks
Vivek

Regards,
Michal

Signature · 5 lines
Michal K. K. Piotrowski
LTG - Linux Testers Group (PL)
(http://www.stardust.webpages.pl/ltg/)
LTG - Linux Testers Group (EN)
(http://www.stardust.webpages.pl/linux_testers_group_en/)
author
#7 lore raw

Re: 2.6.21-rc4-rt0-kdump

In reply to Vivek Goyal

Vivek Goyal napisał(a):

Quoted 58 lines · Michal Piotrowski
Quoted 43 lines

Michal Piotrowski napisał(a):

Quoted 9 lines · Ingo Molnar
Quoted 7 lines · Michal Piotrowski
Quoted 4 lines

Hi Ingo,
2.6.21-rc4-rt0

BUG: at kernel/fork.c:1033 copy_process()

thanks Michal - this is a real bug that affects upstream too. Find the fix below - i've test-booted it and it fixes the warning.

Problem is fixed, thanks.

BTW. It seems that nobody uses -rt as a crash dump kernel ;)

BUG: unable to handle kernel paging request at virtual address f7ebf8c4
 printing eip:
c1610192
*pde = 00000000
stopped custom tracer.
Oops: 0000 [#1]
PREEMPT 
Modules linked in:
CPU:    0
EIP:    0060:[<c1610192>]    Not tainted VLI
EFLAGS: 00010206   (2.6.21-rc4-rt0-kdump #3)

EIP is at copy_oldmem_page+0x4a/0xd0
eax: 000008c4 ebx: f7ebf000 ecx: 00000100 edx: 00000246
esi: f7ebf8c4 edi: c4c520fc ebp: c4d54e30 esp: c4d54e18
ds: 007b es: 007b fs: 00d8 gs: 0000 ss: 0068 preempt:00000001
Process swapper (pid: 1, ti=c4d54000 task=c4d52c20 task.ti=c4d54000)

Stack: c17ab7e0 c183f982 c1969658 00000400 00000400 00037ebf c4d54e5c c16af187 
       00037ebf c4c520fc 00000400 000008c4 00000000 00000000 c4c696e0 00000400 
       c4c520fc c4d54f94 c19a9cfd c4c520fc 00000400 c4d54f78 00000000 c1840996 
Call Trace:
 [<c16af187>] read_from_oldmem+0x73/0x98
 [<c19a9cfd>] vmcore_init+0x26c/0xab7
 [<c199979b>] init+0xaa/0x287
 [<c16044eb>] kernel_thread_helper+0x7/0x10
 =======================

l *copy_oldmem_page+0x4a/0xd0
0xc1610148 is in copy_oldmem_page (arch/i386/kernel/crash_dump.c:35).

30       * copying the data to a pre-allocated kernel page and then copying to user
31       * space in non-atomic context.
32       */
Quoted 8 lines

33 ssize_t copy_oldmem_page(unsigned long pfn, char *buf,
34 size_t csize, unsigned long offset, int userbuf)
35 {
36 void *vaddr;
37
38 if (!csize)
39 return 0;

Can you please paste the disassembly of copy_oldmem_page() on your system. Not sure from where this faulting address 0xf7ebf8c4 is coming. We are still in vmcore_init(), so we should be copying the data to kernel buffers only. This looks like a valid kernel address.

Quoted 4 lines

Can you also put some printk() here to find out from where 0xf7ebf8c4 has come? It does not look like a fixed kernel virutual address returned by kmap_atomic_pfn(). Then is it passed by kernel as a parameter to copy_oldmem_page()?

I added
printk(KERN_WARNING "copy_oldmem_page() pfn=%lu , buf=%s , nr_bytes=%d , offset=%lu , userbuf=%d\n", pfn, buf, nr_bytes, offset, userbuf);

before
tmp = copy_oldmem_page(pfn, buf, nr_bytes, offset, userbuf);

result is here
http://www.stardust.webpages.pl/files/tbf/bitis-gabonica/2.6.21-rc4-rt0/kdump-console2.log

'buf' might be broken.

Quoted 4 lines

Thanks
Vivek

Regards,
Michal

Signature · 5 lines
Michal K. K. Piotrowski
LTG - Linux Testers Group (PL)
(http://www.stardust.webpages.pl/ltg/)
LTG - Linux Testers Group (EN)
(http://www.stardust.webpages.pl/linux_testers_group_en/)
#8 lore raw

Re: 2.6.21-rc4-rt0-kdump (was: Re: [patch] setup_boot_APIC_clock() irq-enable fix)

In reply to Michal Piotrowski
Quoted 8 lines · Michal Piotrowski
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
BUG: at kernel/fork.c:1033 copy_process()

thanks Michal - this is a real bug that affects upstream too. Find the fix below - i've test-booted it and it fixes the warning.

Problem is fixed, thanks.

BTW. It seems that nobody uses -rt as a crash dump kernel ;)

it's been tested with v2.6.20-rt8, and it should work as long as you enable CONFIG_RELOCATABLE. But i'm not using it myself, and v2.6.21-rc4-rt0 isnt a particularly encouraging version string for people to try ;)

Quoted 4 lines

Hibernation is still broken.

http://www.stardust.webpages.pl/files/tbf/bitis-gabonica/2.6.21-rc4-rt0/console.log
http://www.stardust.webpages.pl/files/tbf/bitis-gabonica/2.6.21-rc4-rt0/rt-config

what's the failure mode besides the lockdep + other debug messages - it doesnt resume? Your log seems to have at least one sequence of resume related messages - those seem to have worked fine.

Ingo

author
#9 lore raw

Re: 2.6.21-rc4-rt0-kdump (was: Re: [patch] setup_boot_APIC_clock() irq-enable fix)

Quoted 17 lines · Ingo Molnar
Quoted 8 lines · Michal Piotrowski
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
BUG: at kernel/fork.c:1033 copy_process()

thanks Michal - this is a real bug that affects upstream too. Find the fix below - i've test-booted it and it fixes the warning.

Problem is fixed, thanks.

BTW. It seems that nobody uses -rt as a crash dump kernel ;)

it's been tested with v2.6.20-rt8, and it should work as long as you enable CONFIG_RELOCATABLE. But i'm not using it myself, and v2.6.21-rc4-rt0 isnt a particularly encouraging version string for people to try ;)

Quoted 8 lines
Quoted 4 lines

Hibernation is still broken.

http://www.stardust.webpages.pl/files/tbf/bitis-gabonica/2.6.21-rc4-rt0/console.log
http://www.stardust.webpages.pl/files/tbf/bitis-gabonica/2.6.21-rc4-rt0/rt-config

what's the failure mode besides the lockdep + other debug messages - it doesnt resume? Your log seems to have at least one sequence of resume related messages - those seem to have worked fine.

Kernel has crashed after

Read 497936 kbytes in 23.09 seconds (21.56 MB/s)
swsusp: Reading resume file was successful
PM: Preparing devices for restore. Suspending console(s)

Quoted 3 lines

Ingo

Regards,
Michal

Signature · 5 lines
Michal K. K. Piotrowski
LTG - Linux Testers Group (PL)
(http://www.stardust.webpages.pl/ltg/)
LTG - Linux Testers Group (EN)
(http://www.stardust.webpages.pl/linux_testers_group_en/)
#10 lore raw

Re: 2.6.21-rc4-rt0-kdump (was: Re: [patch] setup_boot_APIC_clock() irq-enable fix)

Quoted 5 lines · Michal Piotrowski
Quoted 5 lines
Quoted 3 lines

Hibernation is still broken.

http://www.stardust.webpages.pl/files/tbf/bitis-gabonica/2.6.21-rc4-rt0/console.log

Quoted 1 line
Quoted 7 lines
Quoted 5 lines

http://www.stardust.webpages.pl/files/tbf/bitis-gabonica/2.6.21-rc4-rt0/rt-config

what's the failure mode besides the lockdep + other debug messages - it doesnt resume? Your log seems to have at least one sequence of resume related messages - those seem to have worked fine.

Kernel has crashed after

crashed == 'hung hard' or 'spontaneous reboot' or 'other'?

Quoted 4 lines

Read 497936 kbytes in 23.09 seconds (21.56 MB/s)
swsusp: Reading resume file was successful
PM: Preparing devices for restore.
Suspending console(s)

i havent used sw-suspend for a while, so here's a stupid question: why does it try to suspend the consoles in the resume path? I assume the messages above mean that we are already in the resume path?

Ingo

author
#11 lore raw

Re: 2.6.21-rc4-rt0-kdump (was: Re: [patch] setup_boot_APIC_clock() irq-enable fix)

Quoted 10 lines · Ingo Molnar
Quoted 7 lines · Michal Piotrowski
Quoted 7 lines
Quoted 3 lines

Hibernation is still broken.

http://www.stardust.webpages.pl/files/tbf/bitis-gabonica/2.6.21-rc4-rt0/console.log

Quoted 1 line

http://www.stardust.webpages.pl/files/tbf/bitis-gabonica/2.6.21-rc4-rt0/rt-config

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

what's the failure mode besides the lockdep + other debug messages - it doesnt resume? Your log seems to have at least one sequence of resume related messages - those seem to have worked fine.

Kernel has crashed after

crashed == 'hung hard' or 'spontaneous reboot' or 'other'?

It means 'nothing has happened for a five minutes'.

Quoted 9 lines
Quoted 5 lines

Read 497936 kbytes in 23.09 seconds (21.56 MB/s)
swsusp: Reading resume file was successful
PM: Preparing devices for restore.
Suspending console(s)

i havent used sw-suspend for a while, so here's a stupid question: why does it try to suspend the consoles in the resume path? I assume the messages above mean that we are already in the resume path?

Yes. (I'm not sure, but "Suspending console(s)" might come from PM_DEBUG.)

Quoted 3 lines

Ingo

Regards,
Michal

Signature · 5 lines
Michal K. K. Piotrowski
LTG - Linux Testers Group (PL)
(http://www.stardust.webpages.pl/ltg/)
LTG - Linux Testers Group (EN)
(http://www.stardust.webpages.pl/linux_testers_group_en/)