Reading /proc/kcore causes a BUG()
Subsystems (2)
Is a fix for this going to make 2.6.31?
To replicate
cat /proc/kcore >/dev/null
See also
http://bugzilla.kernel.org/show_bug.cgi?id=13850
To get
BUG: unable to handle kernel paging request at eda08000 IP: [<c01a0e98>] read_kcore+0x27f/0x341 *pdpt = 0000000071809027 Oops: 0000 [#1] SMP
last sysfs file: /sys/class/net/lo/operstate
Pid: 2361, comm: cat Not tainted (2.6.31.git-x1 #1)
EIP: 0061:[<c01a0e98>] EFLAGS: 00010286 CPU: 1
EIP is at read_kcore+0x27f/0x341
EAX: c04404fc EBX: 00001000 ECX: 00000400 EDX: eda09000
ESI: eda08000 EDI: ec129000 EBP: ec60bf48 ESP: ec60bf10
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0069
Process cat (pid: 2361, ti=ec60a000 task=ecf800b0 task.ti=ec60a000)
Stack: 00001000 08051000 00000000 00001000 00000003 eda08000 ec129000 c04404fc <0> eda09000 00000000 00001000 ecd39380 fffffffb c01a0c19 ec60bf6c c019a907 <0> ec60bf98 00001000 08051000 eccbab00 eccbab00 c019a8af 00001000 ec60bf8c Call Trace: [<c01a0c19>] ? read_kcore+0x0/0x341 [<c019a907>] ? proc_reg_read+0x58/0x6c [<c019a8af>] ? proc_reg_read+0x0/0x6c [<c016905e>] ? vfs_read+0x87/0x110 [<c0169180>] ? sys_read+0x3b/0x60 [<c0106a4d>] ? syscall_call+0x7/0xb Code: d3 29 f3 89 f8 29 f0 39 d7 0f 46 d8 8d 0c 33 89 4d e8 29 5d ec 8b 45 e4 f6 40 0c 01 75 18 89 d9 c1 e9 02 89 f7 2b 7d dc 03 7d e0 <f3> a5 89 d9 83 e1 03 74 02 f3 a4 8b 55 e4 8b 12 89 55 e4 83 7d EIP: [<c01a0e98>] read_kcore+0x27f/0x341 SS:ESP 0069:ec60bf10 CR2: 00000000eda08000 ---[ end trace 4387f828fd1590eb ]---
I tried this on the latest git checkout (as of 2009-09-03 11:00 GMT) under Xen as a domU. The bugzilla report states it happens on non xen machines also.
I know reading /proc/kcore isn't such a good idea, but badly written backup scripts are triggering this on our customer's servers :-(
Signature · 1 line
Nick Craig-Wood <[email protected]> -- http://www.craig-wood.com/nick
Quoted 41 lines · Nick Craig-Wood
Is a fix for this going to make 2.6.31?
To replicate
cat
/proc/kcore>/dev/nullSee also
http://bugzilla.kernel.org/show_bug.cgi?id=13850
To get
BUG: unable to handle kernel paging request at eda08000 IP: [<c01a0e98>] read_kcore+0x27f/0x341 *pdpt = 0000000071809027 Oops: 0000 [#1] SMPlast sysfs file:
/sys/class/net/lo/operstatePid: 2361, comm: cat Not tainted (2.6.31.git-x1 #1)
EIP: 0061:[<c01a0e98>] EFLAGS: 00010286 CPU: 1
EIP is atread_kcore+0x27f/0x341
EAX:c04404fcEBX: 00001000 ECX: 00000400 EDX:eda09000
ESI:eda08000EDI:ec129000EBP:ec60bf48ESP:ec60bf10
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0069
Process cat (pid: 2361, ti=ec60a000 task=ecf800b0 task.ti=ec60a000)Stack: 00001000 08051000 00000000 00001000 00000003 eda08000 ec129000 c04404fc <0> eda09000 00000000 00001000 ecd39380 fffffffb c01a0c19 ec60bf6c c019a907 <0> ec60bf98 00001000 08051000 eccbab00 eccbab00 c019a8af 00001000 ec60bf8c Call Trace: [<c01a0c19>] ? read_kcore+0x0/0x341 [<c019a907>] ? proc_reg_read+0x58/0x6c [<c019a8af>] ? proc_reg_read+0x0/0x6c [<c016905e>] ? vfs_read+0x87/0x110 [<c0169180>] ? sys_read+0x3b/0x60 [<c0106a4d>] ? syscall_call+0x7/0xb Code: d3 29 f3 89 f8 29 f0 39 d7 0f 46 d8 8d 0c 33 89 4d e8 29 5d ec 8b 45 e4 f6 40 0c 01 75 18 89 d9 c1 e9 02 89 f7 2b 7d dc 03 7d e0 <f3> a5 89 d9 83 e1 03 74 02 f3 a4 8b 55 e4 8b 12 89 55 e4 83 7d EIP: [<c01a0e98>] read_kcore+0x27f/0x341 SS:ESP 0069:ec60bf10 CR2: 00000000eda08000 ---[ end trace 4387f828fd1590eb ]---
Quoted 6 lines
I tried this on the latest git checkout (as of 2009-09-03 11:00 GMT) under Xen as a domU. The bugzilla report states it happens on non xen machines also.
I know reading
/proc/kcoreisn't such a good idea, but badly written backup scripts are triggering this on our customer's servers :-(
AFAICT the bug was fixed but I can't seem to find the patches in Linus' git either. Lets CC Andrew and Hiroyuki-san.
Pekka
Quoted 46 lines · On Thu, 3 Sep 2009 15:14:55 +0300 Pekka Enberg <[email protected]> wrote:
Quoted 45 lines · Nick Craig-Wood
Is a fix for this going to make 2.6.31?
To replicate
cat
/proc/kcore>/dev/nullSee also
http://bugzilla.kernel.org/show_bug.cgi?id=13850
To get
BUG: unable to handle kernel paging request at eda08000 IP: [<c01a0e98>] read_kcore+0x27f/0x341 *pdpt = 0000000071809027 Oops: 0000 [#1] SMPlast sysfs file:
/sys/class/net/lo/operstatePid: 2361, comm: cat Not tainted (2.6.31.git-x1 #1)
EIP: 0061:[<c01a0e98>] EFLAGS: 00010286 CPU: 1
EIP is atread_kcore+0x27f/0x341
EAX:c04404fcEBX: 00001000 ECX: 00000400 EDX:eda09000
ESI:eda08000EDI:ec129000EBP:ec60bf48ESP:ec60bf10
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0069
Process cat (pid: 2361, ti=ec60a000 task=ecf800b0 task.ti=ec60a000)Stack: 00001000 08051000 00000000 00001000 00000003 eda08000 ec129000 c04404fc <0> eda09000 00000000 00001000 ecd39380 fffffffb c01a0c19 ec60bf6c c019a907 <0> ec60bf98 00001000 08051000 eccbab00 eccbab00 c019a8af 00001000 ec60bf8c Call Trace: [<c01a0c19>] ? read_kcore+0x0/0x341 [<c019a907>] ? proc_reg_read+0x58/0x6c [<c019a8af>] ? proc_reg_read+0x0/0x6c [<c016905e>] ? vfs_read+0x87/0x110 [<c0169180>] ? sys_read+0x3b/0x60 [<c0106a4d>] ? syscall_call+0x7/0xb Code: d3 29 f3 89 f8 29 f0 39 d7 0f 46 d8 8d 0c 33 89 4d e8 29 5d ec 8b 45 e4 f6 40 0c 01 75 18 89 d9 c1 e9 02 89 f7 2b 7d dc 03 7d e0 <f3> a5 89 d9 83 e1 03 74 02 f3 a4 8b 55 e4 8b 12 89 55 e4 83 7d EIP: [<c01a0e98>] read_kcore+0x27f/0x341 SS:ESP 0069:ec60bf10 CR2: 00000000eda08000 ---[ end trace 4387f828fd1590eb ]---I tried this on the latest git checkout (as of 2009-09-03 11:00 GMT) under Xen as a domU. The bugzilla report states it happens on non xen machines also.
Quoted 6 lines
Quoted 2 lines
I know reading
/proc/kcoreisn't such a good idea, but badly written backup scripts are triggering this on our customer's servers :-(AFAICT the bug was fixed but I can't seem to find the patches in Linus' git either. Lets CC Andrew and Hiroyuki-san.
Ah, it's now tested under mmotm. please wait.
Thanks,
-Kame
Quoted 12 lines · KAMEZAWA Hiroyuki
Quoted 10 lines · On Thu, 3 Sep 2009 15:14:55 +0300 Pekka Enberg <[email protected]> wrote:
Quoted 9 lines · Nick Craig-Wood
Is a fix for this going to make 2.6.31?
To replicate
cat
/proc/kcore>/dev/nullSee also
[snip]
Quoted 11 lines
Quoted 9 lines
Quoted 6 lines
I tried this on the latest git checkout (as of 2009-09-03 11:00 GMT) under Xen as a domU. The bugzilla report states it happens on non xen machines also.
I know reading
/proc/kcoreisn't such a good idea, but badly written backup scripts are triggering this on our customer's servers :-(AFAICT the bug was fixed but I can't seem to find the patches in Linus' git either. Lets CC Andrew and Hiroyuki-san.
Ah, it's now tested under mmotm. please wait.
I tried mmotm but I couldn't get it to boot under Xen :-(
If you send me a patch against latest git I'm willing to test it (I tried to extract the relevant patch from mmotm but failed dismally)
Thanks
Nick
Signature · 1 line
Nick Craig-Wood <[email protected]> -- http://www.craig-wood.com/nick
Quoted 24 lines · On Fri, 4 Sep 2009 10:00:39 +0100 Nick Craig-Wood <[email protected]> wrote:
Quoted 12 lines · KAMEZAWA Hiroyuki
Quoted 10 lines · On Thu, 3 Sep 2009 15:14:55 +0300 Pekka Enberg <[email protected]> wrote:
Quoted 9 lines · Nick Craig-Wood
Is a fix for this going to make 2.6.31?
To replicate
cat
/proc/kcore>/dev/nullSee also
[snip]
Quoted 10 lines
Quoted 9 lines
Quoted 6 lines
I tried this on the latest git checkout (as of 2009-09-03 11:00 GMT) under Xen as a domU. The bugzilla report states it happens on non xen machines also.
I know reading
/proc/kcoreisn't such a good idea, but badly written backup scripts are triggering this on our customer's servers :-(AFAICT the bug was fixed but I can't seem to find the patches in Linus' git either. Lets CC Andrew and Hiroyuki-san.
Quoted 9 lines
Quoted 1 line
Ah, it's now tested under mmotm. please wait.
I tried mmotm but I couldn't get it to boot under Xen :-(
If you send me a patch against latest git I'm willing to test it (I tried to extract the relevant patch from mmotm but failed dismally)
Thanks
If you already downloaded mmotm, use these patches.
vmalloc-unmap-vmalloc-area-after-hiding-it.patch kcore-fix-vread-vwrite-to-be-aware-of-holes.patch kcore-fix-vread-vwrite-to-be-aware-of-holes-update.patch kcore-proc-kcore-should-use-vread.patch
All I tested was x86-32/x86-64. then more tests are welcomed.
-Kame
Quoted 4 lines
Nick
Signature · 1 line
Nick Craig-Wood <[email protected]> -- http://www.craig-wood.com/nick
Quoted 34 lines · KAMEZAWA Hiroyuki
Quoted 24 lines · On Fri, 4 Sep 2009 10:00:39 +0100 Nick Craig-Wood <[email protected]> wrote:
Quoted 12 lines · KAMEZAWA Hiroyuki
Quoted 10 lines · On Thu, 3 Sep 2009 15:14:55 +0300 Pekka Enberg <[email protected]> wrote:
Quoted 9 lines · Nick Craig-Wood
Is a fix for this going to make 2.6.31?
To replicate
cat
/proc/kcore>/dev/nullSee also
[snip]
Quoted 10 lines
Quoted 9 lines
Quoted 6 lines
I tried this on the latest git checkout (as of 2009-09-03 11:00 GMT) under Xen as a domU. The bugzilla report states it happens on non xen machines also.
I know reading
/proc/kcoreisn't such a good idea, but badly written backup scripts are triggering this on our customer's servers :-(AFAICT the bug was fixed but I can't seem to find the patches in Linus' git either. Lets CC Andrew and Hiroyuki-san.
Quoted 6 lines
Quoted 1 line
Ah, it's now tested under mmotm. please wait.
I tried mmotm but I couldn't get it to boot under Xen :-(
If you send me a patch against latest git I'm willing to test it (I tried to extract the relevant patch from mmotm but failed dismally)
Quoted 8 lines
If you already downloaded mmotm, use these patches.
vmalloc-unmap-vmalloc-area-after-hiding-it.patch kcore-fix-vread-vwrite-to-be-aware-of-holes.patch kcore-fix-vread-vwrite-to-be-aware-of-holes-update.patch kcore-proc-kcore-should-use-vread.patch
All I tested was x86-32/x86-64. then more tests are welcomed.
I can confirm that these patches fix the problem on x86-32 running under Xen DomU - thank you very much.
Signature · 1 line
Nick Craig-Wood <[email protected]> -- http://www.craig-wood.com/nick
Quoted 13 lines · On Fri, 4 Sep 2009 18:16:45 +0900 KAMEZAWA Hiroyuki…
Quoted 10 lines · On Fri, 4 Sep 2009 10:00:39 +0100 Nick Craig-Wood <[email protected]> wrote:
Quoted 9 lines · KAMEZAWA Hiroyuki
Quoted 7 lines · On Thu, 3 Sep 2009 15:14:55 +0300 Pekka Enberg <[email protected]> wrote:
Quoted 6 lines · Nick Craig-Wood
Is a fix for this going to make 2.6.31?
To replicate
__cat/proc/kcore>/dev/null
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
See also
[snip]
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
I tried this on the latest git checkout (as of 2009-09-03 11:00 GMT) under Xen as a domU.
__Thebugzilla report states it happens on non xen machines also.
Is this a regression? I've lost track..
Quoted 16 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
I know reading
/proc/kcoreisn't such a good idea, but badly written backup scripts are triggering this on our customer's servers :-(AFAICT the bug was fixed but I can't seem to find the patches in Linus' git either. Lets CC Andrew and Hiroyuki-san.
Quoted 9 lines
Quoted 1 line
Ah, it's now tested under mmotm. please wait.
I tried mmotm but I couldn't get it to boot under Xen :-(
If you send me a patch against latest git I'm willing to test it (I tried to extract the relevant patch from mmotm but failed dismally)
Thanks
Quoted 8 lines
If you already downloaded mmotm, use these patches.
vmalloc-unmap-vmalloc-area-after-hiding-it.patch kcore-fix-vread-vwrite-to-be-aware-of-holes.patch kcore-fix-vread-vwrite-to-be-aware-of-holes-update.patch kcore-proc-kcore-should-use-vread.patch
All I tested was x86-32/x86-64. then more tests are welcomed.
That's a lot of stuff for 2.6.31. Is there some simple quickfix we can do?
Quoted 17 lines · On Fri, 4 Sep 2009 10:31:56 -0700 Andrew Morton <[email protected]>…
Quoted 15 lines · On Fri, 4 Sep 2009 18:16:45 +0900 KAMEZAWA Hiroyuki…
Quoted 12 lines · On Fri, 4 Sep 2009 10:00:39 +0100 Nick Craig-Wood <[email protected]> wrote:
Quoted 11 lines · KAMEZAWA Hiroyuki
Quoted 9 lines · On Thu, 3 Sep 2009 15:14:55 +0300 Pekka Enberg <[email protected]> wrote:
Quoted 8 lines · Nick Craig-Wood
Is a fix for this going to make 2.6.31?
To replicate
__cat/proc/kcore>/dev/nullSee also
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
[snip]
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
I tried this on the latest git checkout (as of 2009-09-03 11:00 GMT) under Xen as a domU.
__Thebugzilla report states it happens on non xen machines also.Is this a regression? I've lost track..
regression. (this comes from new per-cpu area implemantation, which uses vmalloc area with memory holes.)
Quoted 18 lines
Quoted 18 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
I know reading
/proc/kcoreisn't such a good idea, but badly written backup scripts are triggering this on our customer's servers :-(AFAICT the bug was fixed but I can't seem to find the patches in Linus' git either. Lets CC Andrew and Hiroyuki-san.
Quoted 9 lines
Quoted 1 line
Ah, it's now tested under mmotm. please wait.
I tried mmotm but I couldn't get it to boot under Xen :-(
If you send me a patch against latest git I'm willing to test it (I tried to extract the relevant patch from mmotm but failed dismally)
Thanks
If you already downloaded mmotm, use these patches.
Quoted 9 lines
Quoted 6 lines
vmalloc-unmap-vmalloc-area-after-hiding-it.patch kcore-fix-vread-vwrite-to-be-aware-of-holes.patch kcore-fix-vread-vwrite-to-be-aware-of-holes-update.patch kcore-proc-kcore-should-use-vread.patch
All I tested was x86-32/x86-64. then more tests are welcomed.
That's a lot of stuff for 2.6.31. Is there some simple quickfix we can do?
I don't like this patch. But here, my 1st version which wasn't sent out. -Kame
==/proc/kcore may access in memory holes in vmalloc area. skip it by using copy_from_user().
--- fs/proc/kcore.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
fs/proc/kcore.c +3 -1
@@ -361,7 +361,9 @@ read_kcore(struct file *file, char __use/* don't dump ioremap'd stuff! (TA) */ if (m->flags &VM_IOREMAP )continue ; -memcpy (elf_buf + (vmstart - start), +/* we may access memory holes */ +__copy_from_user_inatomic ( + elf_buf + (vmstart - start), (char *)vmstart, vmsize); }read_unlock (&vmlist_lock);
Quoted 4 lines · On Mon, 7 Sep 2009 09:38:56 +0900 KAMEZAWA Hiroyuki…
Quoted 2 lines
That's a lot of stuff for 2.6.31. Is there some simple quickfix we can do?
I don't like this patch. But here, my 1st version which wasn't sent out. -Kame
Fixed warnings. (I confirmed this patch works well on x86-64.) ==
To access vmalloc area which may have memory holes, copy_from_user is useful. By this,
# cat /proc/kcore > /dev/null will not panic.
--- fs/proc/kcore.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
fs/proc/kcore.c +7 -1
@@ -361,7 +361,13 @@ read_kcore(struct file *file, char __use/* don't dump ioremap'd stuff! (TA) */ if (m->flags &VM_IOREMAP )continue ; -memcpy (elf_buf + (vmstart - start), +/* +* we may access memory holes, then use +* ex_table. checking return value just for +* avoid warnings. +*/ + vmsize =__copy_from_user_inatomic ( + elf_buf + (vmstart - start), (char *)vmstart, vmsize); }read_unlock (&vmlist_lock);
Quoted 28 lines · KAMEZAWA Hiroyuki
Quoted 4 lines · On Mon, 7 Sep 2009 09:38:56 +0900 KAMEZAWA Hiroyuki…
Quoted 2 lines
That's a lot of stuff for 2.6.31. Is there some simple quickfix we can do?
I don't like this patch. But here, my 1st version which wasn't sent out. -Kame
Fixed warnings. (I confirmed this patch works well on x86-64.) ==
To access vmalloc area which may have memory holes,copy_from_useris useful. By this,# cat /proc/kcore > /dev/null will not panic.Signed-off-by KAMEZAWA Hiroyuki <[email protected]>--- fs/proc/kcore.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)Index:
linux-2.6.31-rc8/fs/proc/kcore.c
===================================================================fs/proc/kcore.c +0
-1@@ -361,7 +361,13 @@ read_kcore(struct file *file, char __use/* don't dump ioremap'd stuff! (TA) */ if (m->flags &VM_IOREMAP )continue ; - memcpy(elf_buf + (vmstart - start),
fs/proc/kcore.c read_kcore(struct file *file, char __use · line 373
Quoted 10 lines
fs/proc/kcore.c +7
-0+/* +* we may access memory holes, then use +* ex_table. checking return value just for +* avoid warnings. +*/ + vmsize =__copy_from_user_inatomic ( + elf_buf + (vmstart - start), (char *)vmstart, vmsize); }read_unlock (&vmlist_lock);
I gave this patch a thrashing and it works fine for me too (Xen domU x86)
It would be great if we could have this quick fix for 2.6.31.
Perhaps it should also go out to 2.6.30-stable also?
I can confirm that this patch applies to 2.6.30.5 and fixes the problem there too.
Signature · 1 line
Nick Craig-Wood <[email protected]> -- http://www.craig-wood.com/nick