Crash (ext3 ) during 2.6.29-rc6 boot
2.6.29-rc6 bootup on a powerpc box failed with
Unable to handle kernel paging request for data at address 0xc00000003f380000
Faulting instruction address: 0xc000000000039574
cpu 0x1: Vector: 300 (Data Access) at [c00000003baf3020]
pc: c000000000039574: .memcpy+0x74/0x244
lr: d00000000244916c: .ext3_xattr_get+0x288/0x2f4 [ext3]
sp: c00000003baf32a0
msr: 8000000000009032
dar: c00000003f380000
dsisr: 40000000
current = 0xc00000003e54b010
paca = 0xc000000000a53680
pid = 1840, comm = readahead
enter ? for help
[link register ] d00000000244916c .ext3_xattr_get+0x288/0x2f4 [ext3]
[c00000003baf32a0] d000000002449104 .ext3_xattr_get+0x220/0x2f4 [ext3] (unreliab
le)
[c00000003baf3390] d00000000244a6e8 .ext3_xattr_security_get+0x40/0x5c [ext3] [c00000003baf3400] c000000000148154 .generic_getxattr+0x74/0x9c [c00000003baf34a0] c000000000333400 .inode_doinit_with_dentry+0x1c4/0x678 [c00000003baf3560] c00000000032c6b0 .security_d_instantiate+0x50/0x68 [c00000003baf35e0] c00000000013c818 .d_instantiate+0x78/0x9c [c00000003baf3680] c00000000013ced0 .d_splice_alias+0xf0/0x120 [c00000003baf3720] d00000000243e05c .ext3_lookup+0xec/0x134 [ext3] [c00000003baf37c0] c000000000131e74 .do_lookup+0x110/0x260 [c00000003baf3880] c000000000134ed0 .__link_path_walk+0xa98/0x1010 [c00000003baf3970] c0000000001354a0 .path_walk+0x58/0xc4 [c00000003baf3a20] c000000000135720 .do_path_lookup+0x138/0x1e4 [c00000003baf3ad0] c00000000013645c .path_lookup_open+0x6c/0xc8 [c00000003baf3b70] c000000000136780 .do_filp_open+0xcc/0x874 [c00000003baf3d10] c0000000001251e0 .do_sys_open+0x80/0x140 [c00000003baf3dc0] c00000000016aaec .compat_sys_open+0x24/0x38 [c00000003baf3e30] c00000000000855c syscall_exit+0x0/0x40
--- Exception: c01 (System Call) at 000000000ff0ef18
SP (ffc6f4b0) is in userspace
1:mon>
Following EXT3 related options were enabled in the config.
CONFIG_EXT3_FS=mCONFIG_EXT3_FS_XATTR=yCONFIG_EXT3_FS_POSIX_ACL=yCONFIG_EXT3_FS_SECURITY=y
Thanks
-Sachin
Signature · 6 lines
--------------------------------- Sachin Sant IBM Linux Technology Center India Systems and Technology Labs Bangalore, India ---------------------------------
Quoted 39 lines · On Mon, 23 Feb 2009 15:16:05 +0530 "Sachin P. Sant" <[email protected]> wrote:
2.6.29-rc6 bootup on a powerpc box failed with
Unable to handle kernel paging request for data at address 0xc00000003f380000 Faulting instruction address: 0xc000000000039574 cpu 0x1: Vector: 300 (Data Access) at [c00000003baf3020] pc: c000000000039574: .memcpy+0x74/0x244 lr: d00000000244916c: .ext3_xattr_get+0x288/0x2f4 [ext3] sp: c00000003baf32a0msr: 8000000000009032
dar:c00000003f380000
dsisr: 40000000
current =0xc00000003e54b010
paca =0xc000000000a53680
pid = 1840, comm = readahead
enter ? for help
[link register ]d00000000244916c.ext3_xattr_get+0x288/0x2f4 [ext3]
[c00000003baf32a0]d000000002449104.ext3_xattr_get+0x220/0x2f4 [ext3] (unreliab
le)
[c00000003baf3390]d00000000244a6e8.ext3_xattr_security_get+0x40/0x5c [ext3] [c00000003baf3400]c000000000148154.generic_getxattr+0x74/0x9c [c00000003baf34a0]c000000000333400.inode_doinit_with_dentry+0x1c4/0x678 [c00000003baf3560]c00000000032c6b0.security_d_instantiate+0x50/0x68 [c00000003baf35e0]c00000000013c818.d_instantiate+0x78/0x9c [c00000003baf3680]c00000000013ced0.d_splice_alias+0xf0/0x120 [c00000003baf3720]d00000000243e05c.ext3_lookup+0xec/0x134 [ext3] [c00000003baf37c0]c000000000131e74.do_lookup+0x110/0x260 [c00000003baf3880]c000000000134ed0.__link_path_walk+0xa98/0x1010 [c00000003baf3970]c0000000001354a0.path_walk+0x58/0xc4 [c00000003baf3a20]c000000000135720.do_path_lookup+0x138/0x1e4 [c00000003baf3ad0]c00000000013645c.path_lookup_open+0x6c/0xc8 [c00000003baf3b70]c000000000136780.do_filp_open+0xcc/0x874 [c00000003baf3d10]c0000000001251e0.do_sys_open+0x80/0x140 [c00000003baf3dc0]c00000000016aaec.compat_sys_open+0x24/0x38 [c00000003baf3e30]c00000000000855csyscall_exit+0x0/0x40
--- Exception: c01 (System Call) at000000000ff0ef18
SP (ffc6f4b0) is in userspace
1:mon>
Quoted 7 lines
Following EXT3 related options were enabled in the config.
CONFIG_EXT3_FS=mCONFIG_EXT3_FS_XATTR=yCONFIG_EXT3_FS_POSIX_ACL=yCONFIG_EXT3_FS_SECURITY=y
hm, I wonder what could have caused that - we haven't altered fs/ext3/xattr.c in ages.
What is the most recent kernel version you know of which didn't do this? Bear in mind that this crash might be triggered by the current contents of the filesystem, so if possible, please test some other kernel versions on that disk.
It looks like we died in ext3_xattr_block_get():
memcpy(buffer, bh->b_data + le16_to_cpu(entry->e_value_offs), size);
Perhaps entry->e_value_offs is no good. I wonder if the filesystem is corrupted and this snuck through the defenses.
I also wonder if there is enough info in that trace for a ppc person to be able to determine whether the faulting address is in the source or destination of the memcpy() (please)?
Quoted 11 lines · Andrew Morton
It looks like we died in
ext3_xattr_block_get():memcpy(buffer, bh->b_data + le16_to_cpu(entry->e_value_offs), size);Perhaps
entry->e_value_offsis no good. I wonder if the filesystem is corrupted and this snuck through the defenses.I also wonder if there is enough info in that trace for a ppc person to be able to determine whether the faulting address is in the source or destination of the
memcpy()(please)?
It appears to have faulted on a load, implicating the source. The address being referenced (0xc00000003f380000) doesn't look outlandish. I wonder if this kernel has CONFIG_DEBUG_PAGEALLOC turned on, and what page size is selected?
Paul.
Quoted 8 lines · Andrew Morton
hm, I wonder what could have caused that - we haven't altered
fs/ext3/xattr.cin ages.What is the most recent kernel version you know of which didn't do this? Bear in mind that this crash might be triggered by the current contents of the filesystem, so if possible, please test some other kernel versions on that disk.
I am trying to boot a vanilla kernel on this machine for the first time. Haven't tried any other kernels. Will give it a try.
Quoted 5 lines
It looks like we died in
ext3_xattr_block_get():memcpy(buffer, bh->b_data + le16_to_cpu(entry->e_value_offs), size);
Quoted 7 lines
Perhaps
entry->e_value_offsis no good. I wonder if the filesystem is corrupted and this snuck through the defenses.I also wonder if there is enough info in that trace for a ppc person to be able to determine whether the faulting address is in the source or destination of the
memcpy()(please)?
Some more information if this could be of any help.
0:mon> di 0xc000000000039574
c000000000039574 e9240008 ld r9,8(r4) c000000000039578 409d0010 ble cr7,c000000000039588 # .memcpy+0x88/0x244 c00000000003957c 79290002 rotldi r9,r9,32 c000000000039580 91230000 stw r9,0(r3) c000000000039584 38630004 addi r3,r3,4 c000000000039588 409e0010 bne cr7,c000000000039598 # .memcpy+0x98/0x244 c00000000003958c 79298000 rotldi r9,r9,16 c000000000039590 b1230000 sth r9,0(r3) c000000000039594 38630002 addi r3,r3,2 c000000000039598 409f000c bns cr7,c0000000000395a4 # .memcpy+0xa4/0x244 c00000000003959c 79294000 rotldi r9,r9,8 c0000000000395a0 99230000 stb r9,0(r3) c0000000000395a4 e8610030 ld r3,48(r1) c0000000000395a8 4e800020 blr c0000000000395ac 78a6e8c2 rldicl r6,r5,61,3 c0000000000395b0 38a5fff0 addi r5,r5,-16 0:mon> r R00 = 000000000000e40f R16 = 00000000100edbc8 R01 = c00000003e59b3e0 R17 = 00000000100b0000 R02 = c0000000009c2110 R18 = 0000000000000005 R03 = c000000044bc90e0 R19 = 00000000fff0d7a8 R04 = c000000039cffff4 R20 = 00000000fff0d708 R05 = 0000000000000003 R21 = 00000000000000ff R06 = 0000000000000000 R22 = 0000000000000006 R07 = 0000000000000001 R23 = c00000000079ab49 R08 = 723a7573725f743a R24 = c0000000372fe2a8 R09 = 3a6f626a6563745f R25 = c000000044bc90c8 R10 = c00000003b250968 R26 = c0000000372fe240 R11 = c000000000039500 R27 = c0000000372fe3b0 R12 = d00000000244c590 R28 = c0000000372c5280 R13 = c000000000a53480 R29 = 000000000000001b R14 = 00000000100d0000 R30 = d0000000024654d0 R15 = 0000000000000000 R31 = ffffffffffffffde pc = c000000000039574 .memcpy+0x74/0x244 lr = d00000000244916c .ext3_xattr_get+0x288/0x2f4 [ext3] msr = 8000000000009032 cr = 4400844b ctr = 0000000000000000 xer = 0000000000000001 trap = 300 dar = c000000039d00000 dsisr = 40000000
0:mon>
So the other thing i noticed was that this machine was running a kernel with selinux enabled. I turned off selinux and there were no issues during bootup. It was a clean boot.
Thanks
-Sachin
Signature · 6 lines
--------------------------------- Sachin Sant IBM Linux Technology Center India Systems and Technology Labs Bangalore, India ---------------------------------
Quoted 4 lines · Paul Mackerras
It appears to have faulted on a load, implicating the source. The address being referenced (
0xc00000003f380000) doesn't look outlandish. I wonder if this kernel hasCONFIG_DEBUG_PAGEALLOCturned on, and what page size is selected?
Yes CONFIG_DEBUG_PAGEALLOC is enabled and the page size is 64K.
CONFIG_DEBUG_PAGEALLOC=yCONFIG_PPC_64K_PAGES=y
Thanks
-Sachin
Signature · 6 lines
--------------------------------- Sachin Sant IBM Linux Technology Center India Systems and Technology Labs Bangalore, India ---------------------------------
Quoted 10 lines
Quoted 8 lines · Andrew Morton
It looks like we died in
ext3_xattr_block_get():memcpy(buffer, bh->b_data + le16_to_cpu(entry->e_value_offs), size);Perhaps
entry->e_value_offsis no good. I wonder if the filesystem is corrupted and this snuck through the defenses.
Quoted 8 lines
Quoted 3 lines
I also wonder if there is enough info in that trace for a ppc person to be able to determine whether the faulting address is in the source or destination of the
memcpy()(please)?It appears to have faulted on a load, implicating the source. The address being referenced (
0xc00000003f380000) doesn't look outlandish. I wonder if this kernel hasCONFIG_DEBUG_PAGEALLOCturned on, and what page size is selected?
Hmm, OK. But then I'm not sure how that can happen. Obviously, memcpy
somehow got beyond end of the page referenced by bh->b_data. So it means that le16_to_cpu(entry->e_value_offs) + size > page_size. But ext3_xattr_find_entry() calls ext3_xattr_check_entry() which in particular checks whether e_value_offs + e_value_size isn't greater than bh->b_size. So I see no way how memcpy can get beyond end of the page.
Sachin, is the problem reproducible? If yes, can you send us contents
of the page just before the faulting address (i.e., for current fault it would be 0xc00000003f370000-0xc00000003f37ffff). As far as I can remember powerpc monitor could dump it.
BTW, I suppose you use 4KB blocksize on the filesystem, right?
Honza
Signature · 2 lines
Jan Kara <[email protected]> SuSE CR Labs
Quoted 8 lines · Jan Kara
Hmm, OK. But then I'm not sure how that can happen. Obviously, memcpy
somehow got beyond end of the page referenced bybh->b_data. So it means thatle16_to_cpu(entry->e_value_offs)+ size >page_size. Butext3_xattr_find_entry()callsext3_xattr_check_entry()which in particular checks whethere_value_offs+e_value_sizeisn't greater thanbh->b_size. So I see no way how memcpy can get beyond end of the page.
Sachin, is the problem reproducible? If yes, can you send us contents
Yes, i am able to recreate this problem easily. As i had mentioned if the earlier kernel is booted with selinux enabled and then 2.6.29-rc6 is booted i get this crash. But if i specify selinux=0 at command line, 2.6.29-rc6 boots without any problem.
Quoted 4 lines
of the page just before the faulting address (i.e., for current fault it would be 0xc00000003f370000-0xc00000003f37ffff). As far as I can remember powerpc monitor could dump it.
Here is the page dump. This time it crashed while accessing address 0xc00000002d670000.
Unable to handle kernel paging request for data at address 0xc0000
0002d670000
Faulting instruction address: 0xc000000000039574 cpu 0x1: Vector: 300 (Data Access) at [c00000004288b0b0]
pc: c000000000039574: .memcpy+0x74/0x244
lr: c0000000001b497c: .ext3_xattr_get+0x288/0x2f4
sp: c00000004288b330
msr: 8000000000009032
1:mon> d 0xc00000002d660000
............................... <SNIP> ...............................
c00000002d66efd0 0000000000000000 0000000000000000 |................| c00000002d66efe0 0000000000000000 0000000000000000 |................| c00000002d66eff0 0000000000000000 0000000000000000 |................| c00000002d66f000 000002ea00040000 01000000e200d20a |................| c00000002d66f010 0000000000000000 0000000000000000 |................| c00000002d66f020 0706e40f00000000 1b000000e200d20a |................| c00000002d66f030 73656c696e757800 0000000000000000 |selinux.........| c00000002d66f040 0000000000000000 0000000000000000 |................| c00000002d66f050 0000000000000000 0000000000000000 |................| c00000002d66f060 0000000000000000 0000000000000000 |................|
............................... <SNIP> ...............................
c00000002d66ff60 0000000000000000 0000000000000000 |................| c00000002d66ff70 0000000000000000 0000000000000000 |................| c00000002d66ff80 0000000000000000 0000000000000000 |................| c00000002d66ff90 0000000000000000 0000000000000000 |................| c00000002d66ffa0 0000000000000000 0000000000000000 |................| c00000002d66ffb0 0000000000000000 0000000000000000 |................| c00000002d66ffc0 0000000000000000 0000000000000000 |................| c00000002d66ffd0 0000000000000000 0000000000000000 |................| c00000002d66ffe0 0000000073797374 656d5f753a6f626a |....system_u:obj| c00000002d66fff0 6563745f723a7573 725f743a73300000 |ect_r:usr_t:s0..| c00000002d670000 **************** **************** | | 1:mon> r R00 = 000000000000e40f R16 = 000000000000005d R01 = c00000004288b330 R17 = 0000000000000000 R02 = c0000000009f59b8 R18 = 00000000fffbfe9e R03 = c000000044aa34a0 R19 = 0000000010042638 R04 = c00000002d66fff4 R20 = 0000000010041610 R05 = 0000000000000003 R21 = 00000000000000ff R06 = 0000000000000000 R22 = 0000000000000006 R07 = 0000000000000001 R23 = c0000000007d27c1 R08 = 723a7573725f743a R24 = c00000002c0cd758 R09 = 3a6f626a6563745f R25 = c000000044aa3488 R10 = c00000000017b43c R26 = c00000002c0cd6f0 R11 = c00000002d66f020 R27 = c00000002c0cd860 R12 = d0000000023c14b0 R28 = c00000002c0b0840 R13 = c000000000a93680 R29 = 000000000000001b R14 = 00000000000041ed R30 = c0000000009880b0 R15 = 0000000010040000 R31 = ffffffffffffffde pc = c000000000039574 .memcpy+0x74/0x244 lr = c0000000001b497c .ext3_xattr_get+0x288/0x2f4 msr = 8000000000009032 cr = 4400044b ctr = 0000000000000000 xer = 0000000020000001 trap = 300 dar = c00000002d670000 dsisr = 40000000
1:mon> zr
Quoted 2 lines
BTW, I suppose you use 4KB blocksize on the filesystem, right?
Yes.
dumpe2fs /dev/sda3 | grep -i "block size" dumpe2fs 1.39 (29-May-2006)
Block size: 4096
Thanks
-Sachin
Signature · 6 lines
--------------------------------- Sachin Sant IBM Linux Technology Center India Systems and Technology Labs Bangalore, India ---------------------------------
Hello,
Quoted 14 lines · Sachin P. Sant
Quoted 8 lines · Jan Kara
Hmm, OK. But then I'm not sure how that can happen. Obviously, memcpy
somehow got beyond end of the page referenced bybh->b_data. So it means thatle16_to_cpu(entry->e_value_offs)+ size >page_size. Butext3_xattr_find_entry()callsext3_xattr_check_entry()which in particular checks whethere_value_offs+e_value_sizeisn't greater thanbh->b_size. So I see no way how memcpy can get beyond end of the page.
Sachin, is the problem reproducible? If yes, can you send us contentsYes, i am able to recreate this problem easily. As i had mentioned if the earlier kernel is booted with selinux enabled and then 2.6.29-rc6 is booted i get this crash. But if i specify selinux=0 at command line, 2.6.29-rc6 boots without any problem.
Quoted 6 lines
Quoted 4 lines
of the page just before the faulting address (i.e., for current fault it would be 0xc00000003f370000-0xc00000003f37ffff). As far as I can remember powerpc monitor could dump it.
Here is the page dump. This time it crashed while accessing address
0xc00000002d670000.
Thanks for the dump.
Quoted 58 lines
Unable to handle kernel paging request for data at address 0xc0000
0002d670000
Faulting instruction address:0xc000000000039574cpu0x1: Vector: 300 (Data Access) at [c00000004288b0b0]
pc:c000000000039574: .memcpy+0x74/0x244
lr:c0000000001b497c: .ext3_xattr_get+0x288/0x2f4
sp:c00000004288b330
msr: 80000000000090321:mon> d
0xc00000002d660000
............................... <SNIP> ...............................c00000002d66efd0 0000000000000000 0000000000000000 |................| c00000002d66efe0 0000000000000000 0000000000000000 |................| c00000002d66eff0 0000000000000000 0000000000000000 |................| c00000002d66f000 000002ea00040000 01000000e200d20a |................| c00000002d66f010 0000000000000000 0000000000000000 |................| c00000002d66f020 0706e40f00000000 1b000000e200d20a |................| c00000002d66f030 73656c696e757800 0000000000000000 |selinux.........| c00000002d66f040 0000000000000000 0000000000000000 |................| c00000002d66f050 0000000000000000 0000000000000000 |................| c00000002d66f060 0000000000000000 0000000000000000 |................|............................... <SNIP> ...............................
c00000002d66ff60 0000000000000000 0000000000000000 |................| c00000002d66ff70 0000000000000000 0000000000000000 |................| c00000002d66ff80 0000000000000000 0000000000000000 |................| c00000002d66ff90 0000000000000000 0000000000000000 |................| c00000002d66ffa0 0000000000000000 0000000000000000 |................| c00000002d66ffb0 0000000000000000 0000000000000000 |................| c00000002d66ffc0 0000000000000000 0000000000000000 |................| c00000002d66ffd0 0000000000000000 0000000000000000 |................| c00000002d66ffe0 0000000073797374 656d5f753a6f626a |....system_u:obj| c00000002d66fff0 6563745f723a7573 725f743a73300000 |ect_r:usr_t:s0..| c00000002d670000 **************** **************** | | 1:mon> r R00 = 000000000000e40f R16 = 000000000000005d R01 = c00000004288b330 R17 = 0000000000000000 R02 = c0000000009f59b8 R18 = 00000000fffbfe9e R03 = c000000044aa34a0 R19 = 0000000010042638 R04 = c00000002d66fff4 R20 = 0000000010041610 R05 = 0000000000000003 R21 = 00000000000000ff R06 = 0000000000000000 R22 = 0000000000000006 R07 = 0000000000000001 R23 = c0000000007d27c1 R08 = 723a7573725f743a R24 = c00000002c0cd758 R09 = 3a6f626a6563745f R25 = c000000044aa3488 R10 = c00000000017b43c R26 = c00000002c0cd6f0 R11 = c00000002d66f020 R27 = c00000002c0cd860 R12 = d0000000023c14b0 R28 = c00000002c0b0840 R13 = c000000000a93680 R29 = 000000000000001b R14 = 00000000000041ed R30 = c0000000009880b0 R15 = 0000000010040000 R31 = ffffffffffffffde pc = c000000000039574 .memcpy+0x74/0x244 lr = c0000000001b497c .ext3_xattr_get+0x288/0x2f4 msr = 8000000000009032 cr = 4400044b ctr = 0000000000000000 xer = 0000000020000001 trap = 300 dar = c00000002d670000 dsisr = 40000000
Quoted 8 lines
1:mon> zr
Quoted 2 lines
BTW, I suppose you use 4KB blocksize on the filesystem, right?
Yes.
dumpe2fs
/dev/sda3| grep -i "block size" dumpe2fs 1.39 (29-May-2006)
Block size: 4096
OK. The xattr block causing oops is completely correct. To me it seems
more like some problem in powerpc memcpy() (I saw there went some changes into in in the end of December) - we call it to copy 27 bytes from address 0xc00000002d66ffe4 (which is one byte before end of the page). Could some of the powerpc guys have a look whether this could be the case? I'm not quite fluent in the powerpc assembly so it would take me ages ;).
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Quoted 68 lines
Quoted 8 lines · Andrew Morton
hm, I wonder what could have caused that - we haven't altered
fs/ext3/xattr.cin ages.What is the most recent kernel version you know of which didn't do this? Bear in mind that this crash might be triggered by the current contents of the filesystem, so if possible, please test some other kernel versions on that disk.
I am trying to boot a vanilla kernel on this machine for the first time. Haven't tried any other kernels. Will give it a try.
Quoted 5 lines
It looks like we died in
ext3_xattr_block_get():memcpy(buffer, bh->b_data + le16_to_cpu(entry->e_value_offs), size);Quoted 7 lines
Perhaps
entry->e_value_offsis no good. I wonder if the filesystem is corrupted and this snuck through the defenses.I also wonder if there is enough info in that trace for a ppc person to be able to determine whether the faulting address is in the source or destination of the
memcpy()(please)?Some more information if this could be of any help.
0:mon> di
0xc000000000039574c000000000039574 e9240008 ld r9,8(r4) c000000000039578 409d0010 ble cr7,c000000000039588 # .memcpy+0x88/0x244 c00000000003957c 79290002 rotldi r9,r9,32 c000000000039580 91230000 stw r9,0(r3) c000000000039584 38630004 addi r3,r3,4 c000000000039588 409e0010 bne cr7,c000000000039598 # .memcpy+0x98/0x244 c00000000003958c 79298000 rotldi r9,r9,16 c000000000039590 b1230000 sth r9,0(r3) c000000000039594 38630002 addi r3,r3,2 c000000000039598 409f000c bns cr7,c0000000000395a4 # .memcpy+0xa4/0x244 c00000000003959c 79294000 rotldi r9,r9,8 c0000000000395a0 99230000 stb r9,0(r3) c0000000000395a4 e8610030 ld r3,48(r1) c0000000000395a8 4e800020 blr c0000000000395ac 78a6e8c2 rldicl r6,r5,61,3 c0000000000395b0 38a5fff0 addi r5,r5,-16 0:mon> r R00 = 000000000000e40f R16 = 00000000100edbc8 R01 = c00000003e59b3e0 R17 = 00000000100b0000 R02 = c0000000009c2110 R18 = 0000000000000005 R03 = c000000044bc90e0 R19 = 00000000fff0d7a8 R04 = c000000039cffff4 R20 = 00000000fff0d708 R05 = 0000000000000003 R21 = 00000000000000ff R06 = 0000000000000000 R22 = 0000000000000006 R07 = 0000000000000001 R23 = c00000000079ab49 R08 = 723a7573725f743a R24 = c0000000372fe2a8 R09 = 3a6f626a6563745f R25 = c000000044bc90c8 R10 = c00000003b250968 R26 = c0000000372fe240 R11 = c000000000039500 R27 = c0000000372fe3b0 R12 = d00000000244c590 R28 = c0000000372c5280 R13 = c000000000a53480 R29 = 000000000000001b R14 = 00000000100d0000 R30 = d0000000024654d0 R15 = 0000000000000000 R31 = ffffffffffffffde pc = c000000000039574 .memcpy+0x74/0x244 lr = d00000000244916c .ext3_xattr_get+0x288/0x2f4 [ext3] msr = 8000000000009032 cr = 4400844b ctr = 0000000000000000 xer = 0000000000000001 trap = 300 dar = c000000039d00000 dsisr = 40000000
Quoted 1 line
0:mon>
Yes, this makes me even more suspitious that memcpy() on powerpc could
be at fault. The instruction (ld r9,8(r4)) is loading last 8 bytes to copy, but in fact it should load only 3 bytes in our case because remaining 5 bytes are not in the range we specified and thus larger load can cause page fault...
Honza
Signature · 2 lines
Jan Kara <[email protected]> SuSE CR Labs
Quoted 9 lines · Paul Mackerras
Quoted 8 lines · Andrew Morton
It looks like we died in
ext3_xattr_block_get():memcpy(buffer, bh->b_data + le16_to_cpu(entry->e_value_offs), size);Perhaps
entry->e_value_offsis no good. I wonder if the filesystem is corrupted and this snuck through the defenses.
Quoted 8 lines
Quoted 3 lines
I also wonder if there is enough info in that trace for a ppc person to be able to determine whether the faulting address is in the source or destination of the
memcpy()(please)?It appears to have faulted on a load, implicating the source. The address being referenced (
0xc00000003f380000) doesn't look outlandish. I wonder if this kernel hasCONFIG_DEBUG_PAGEALLOCturned on, and what page size is selected?
I'm seeing a similar thing on PS3, but not in ext3. During early userspace setup (udevd), it crashes accessing a 0xc00* address in:
| NIP setup+0x20/0x130 | LR copy_user_page+0x18/0x6c | Call trace: | do_wp_page+0x5b4/0x89c | do_page_fault+0x3a8/0x58c | handle_page_fault+0x20/0x5c
I have CONFIG_DEBUG_PAGEALLOC=y. If I disable it, the system boots fine.
If needed, I can probably bisect this tomorrow. It definitely didn't happen in 2.6.29-rc5.
With kind regards,
Geert Uytterhoeven
Software Architect
Sony Techsoft Centre Europe
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium
Phone: +32 (0)2 700 8453 Fax: +32 (0)2 700 8622 E-mail: [email protected] Internet: http://www.sony-europe.com/
A division of Sony Europe (Belgium) N.V. VAT BE 0413.825.160 · RPR Brussels Fortis · BIC GEBABEBB · IBAN BE41293037680010
Quoted 89 lines · Jan Kara
Hello,
Quoted 14 lines · Sachin P. Sant
Quoted 8 lines · Jan Kara
Hmm, OK. But then I'm not sure how that can happen. Obviously, memcpy
somehow got beyond end of the page referenced bybh->b_data. So it means thatle16_to_cpu(entry->e_value_offs)+ size >page_size. Butext3_xattr_find_entry()callsext3_xattr_check_entry()which in particular checks whethere_value_offs+e_value_sizeisn't greater thanbh->b_size. So I see no way how memcpy can get beyond end of the page.
Sachin, is the problem reproducible? If yes, can you send us contentsYes, i am able to recreate this problem easily. As i had mentioned if the earlier kernel is booted with selinux enabled and then 2.6.29-rc6 is booted i get this crash. But if i specify selinux=0 at command line, 2.6.29-rc6 boots without any problem.
Quoted 6 lines
Quoted 4 lines
of the page just before the faulting address (i.e., for current fault it would be 0xc00000003f370000-0xc00000003f37ffff). As far as I can remember powerpc monitor could dump it.
Here is the page dump. This time it crashed while accessing address
0xc00000002d670000.Thanks for the dump.
Quoted 64 lines
Unable to handle kernel paging request for data at address 0xc0000
0002d670000
Faulting instruction address:0xc000000000039574cpu0x1: Vector: 300 (Data Access) at [c00000004288b0b0]
pc:c000000000039574: .memcpy+0x74/0x244
lr:c0000000001b497c: .ext3_xattr_get+0x288/0x2f4
sp:c00000004288b330
msr: 80000000000090321:mon> d
0xc00000002d660000
............................... <SNIP> ...............................c00000002d66efd0 0000000000000000 0000000000000000 |................| c00000002d66efe0 0000000000000000 0000000000000000 |................| c00000002d66eff0 0000000000000000 0000000000000000 |................| c00000002d66f000 000002ea00040000 01000000e200d20a |................| c00000002d66f010 0000000000000000 0000000000000000 |................| c00000002d66f020 0706e40f00000000 1b000000e200d20a |................| c00000002d66f030 73656c696e757800 0000000000000000 |selinux.........| c00000002d66f040 0000000000000000 0000000000000000 |................| c00000002d66f050 0000000000000000 0000000000000000 |................| c00000002d66f060 0000000000000000 0000000000000000 |................|............................... <SNIP> ...............................
c00000002d66ff60 0000000000000000 0000000000000000 |................| c00000002d66ff70 0000000000000000 0000000000000000 |................| c00000002d66ff80 0000000000000000 0000000000000000 |................| c00000002d66ff90 0000000000000000 0000000000000000 |................| c00000002d66ffa0 0000000000000000 0000000000000000 |................| c00000002d66ffb0 0000000000000000 0000000000000000 |................| c00000002d66ffc0 0000000000000000 0000000000000000 |................| c00000002d66ffd0 0000000000000000 0000000000000000 |................| c00000002d66ffe0 0000000073797374 656d5f753a6f626a |....system_u:obj| c00000002d66fff0 6563745f723a7573 725f743a73300000 |ect_r:usr_t:s0..| c00000002d670000 **************** **************** | | 1:mon> r R00 = 000000000000e40f R16 = 000000000000005d R01 = c00000004288b330 R17 = 0000000000000000 R02 = c0000000009f59b8 R18 = 00000000fffbfe9e R03 = c000000044aa34a0 R19 = 0000000010042638 R04 = c00000002d66fff4 R20 = 0000000010041610 R05 = 0000000000000003 R21 = 00000000000000ff R06 = 0000000000000000 R22 = 0000000000000006 R07 = 0000000000000001 R23 = c0000000007d27c1 R08 = 723a7573725f743a R24 = c00000002c0cd758 R09 = 3a6f626a6563745f R25 = c000000044aa3488 R10 = c00000000017b43c R26 = c00000002c0cd6f0 R11 = c00000002d66f020 R27 = c00000002c0cd860 R12 = d0000000023c14b0 R28 = c00000002c0b0840 R13 = c000000000a93680 R29 = 000000000000001b R14 = 00000000000041ed R30 = c0000000009880b0 R15 = 0000000010040000 R31 = ffffffffffffffde pc = c000000000039574 .memcpy+0x74/0x244 lr = c0000000001b497c .ext3_xattr_get+0x288/0x2f4 msr = 8000000000009032 cr = 4400044b ctr = 0000000000000000 xer = 0000000020000001 trap = 300 dar = c00000002d670000 dsisr = 400000001:mon> zr
Quoted 2 lines
BTW, I suppose you use 4KB blocksize on the filesystem, right?
Yes.
Quoted 8 lines
Quoted 2 lines
dumpe2fs
/dev/sda3| grep -i "block size" dumpe2fs 1.39 (29-May-2006)
Block size: 4096OK. The xattr block causing oops is completely correct. To me it seems
more like some problem in powerpcmemcpy()(I saw there went some changes into in in the end of December) - we call it to copy 27 bytes from address0xc00000002d66ffe4(which is one byte before end of the page). Could some of the powerpc guys have a look whether this could be the case? I'm not quite fluent in the powerpc assembly so it would take me ages ;).
You're right - it's a problem with the 64bit powerpc memcpy(). And the brown paper bag is all mine (commit 25d6e2d7c58ddc4a3b614fc5381591c0cfe66556). On Power6 and Cell we're doing a load double that goes beyond the source size we were given to copy. I'll see if I can find a nice way of fixing this up, if not then I'll ask Ben to revert.
Sorry about the goose chase!
Mark
Quoted 29 lines · Geert Uytterhoeven
Quoted 9 lines · Paul Mackerras
Quoted 8 lines · Andrew Morton
It looks like we died in
ext3_xattr_block_get():memcpy(buffer, bh->b_data + le16_to_cpu(entry->e_value_offs), size);Perhaps
entry->e_value_offsis no good. I wonder if the filesystem is corrupted and this snuck through the defenses.Quoted 8 lines
Quoted 3 lines
I also wonder if there is enough info in that trace for a ppc person to be able to determine whether the faulting address is in the source or destination of the
memcpy()(please)?It appears to have faulted on a load, implicating the source. The address being referenced (
0xc00000003f380000) doesn't look outlandish. I wonder if this kernel hasCONFIG_DEBUG_PAGEALLOCturned on, and what page size is selected?I'm seeing a similar thing on PS3, but not in ext3. During early userspace setup (udevd), it crashes accessing a 0xc00* address in:
| NIP setup+0x20/0x130 | LR copy_user_page+0x18/0x6c | Call trace: | do_wp_page+0x5b4/0x89c | do_page_fault+0x3a8/0x58c | handle_page_fault+0x20/0x5c
Quoted 4 lines
I have
CONFIG_DEBUG_PAGEALLOC=y. If I disable it, the system boots fine.If needed, I can probably bisect this tomorrow. It definitely didn't happen in 2.6.29-rc5.
No need to bisect - it was 25d6e2d7c58ddc4a3b614fc5381591c0cfe66556, my commit that "optimised" 64bit memcpy() for Power6 and Cell.
The bug was in -rc1, but if your copies were 8-byte aligned with respect to the source the problem wouldn't have been seen... Could this have been why you didn't see it in -rc5?
I'll work on a fix now.
Thanks!
Mark
Quoted 9 lines · Sachin P. Sant
Quoted 8 lines · Jan Kara
Hmm, OK. But then I'm not sure how that can happen. Obviously, memcpy
somehow got beyond end of the page referenced bybh->b_data. So it means thatle16_to_cpu(entry->e_value_offs)+ size >page_size. Butext3_xattr_find_entry()callsext3_xattr_check_entry()which in particular checks whethere_value_offs+e_value_sizeisn't greater thanbh->b_size. So I see no way how memcpy can get beyond end of the page.
Sachin, is the problem reproducible? If yes, can you send us contents
Quoted 4 lines
Yes, i am able to recreate this problem easily. As i had mentioned if the earlier kernel is booted with selinux enabled and then 2.6.29-rc6 is booted i get this crash. But if i specify selinux=0 at command line, 2.6.29-rc6 boots without any problem.
Hi Sanchin and Geert,
Does the patch below fix the problems you're seeing? If it does I'll send a properly written up and formatted patch to linuxppc-dev (as well as another one to fix the same problem in copy_tofrom_user()).
Thanks and sorry again!
Mark
--- arch/powerpc/lib/memcpy_64.S | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-)
arch/powerpc/lib/memcpy_64.S +20 -6
@@ -53,18 +53,19 @@ END_FTR_SECTION_IFCLR(CPU_FTR_UNALIGNED_ 3: std r8,8(r3) beq 3f addi r3,r3,16 - ld r9,8(r4) .Ldo_tail: bf cr7*4+1,1f - rotldi r9,r9,32 + lwz r9,8(r4) + addi r4,r4,4 stw r9,0(r3) addi r3,r3,4 1: bf cr7*4+2,2f - rotldi r9,r9,16 + lhz r9,8(r4) + addi r4,r4,2 sth r9,0(r3) addi r3,r3,2 2: bf cr7*4+3,3f - rotldi r9,r9,8 + lbz r9,8(r4) stb r9,0(r3) 3: ld r3,48(r1) /* return dest pointer */ blr
@@ -133,11 +134,24 @@ END_FTR_SECTION_IFCLR(CPU_FTR_UNALIGNED_ cmpwi cr1,r5,8 addi r3,r3,32 sld r9,r9,r10 - ble cr1,.Ldo_tail + ble cr1,6f ld r0,8(r4) srd r7,r0,r11 or r9,r7,r9 - b .Ldo_tail +6: + bf cr7*4+1,1f + rotldi r9,r9,32 + stw r9,0(r3) + addi r3,r3,4 +1: bf cr7*4+2,2f + rotldi r9,r9,16 + sth r9,0(r3) + addi r3,r3,2 +2: bf cr7*4+3,3f + rotldi r9,r9,8 + stb r9,0(r3) +3: ld r3,48(r1) /* return dest pointer */ + blr .Ldst_unaligned: PPC_MTOCRF 0x01,r6 # put #bytes to 8B bdry into cr7
Quoted 15 lines · Mark Nelson
Quoted 9 lines · Sachin P. Sant
Quoted 8 lines · Jan Kara
Hmm, OK. But then I'm not sure how that can happen. Obviously, memcpy
somehow got beyond end of the page referenced bybh->b_data. So it means thatle16_to_cpu(entry->e_value_offs)+ size >page_size. Butext3_xattr_find_entry()callsext3_xattr_check_entry()which in particular checks whethere_value_offs+e_value_sizeisn't greater thanbh->b_size. So I see no way how memcpy can get beyond end of the page.
Sachin, is the problem reproducible? If yes, can you send us contentsQuoted 4 lines
Yes, i am able to recreate this problem easily. As i had mentioned if the earlier kernel is booted with selinux enabled and then 2.6.29-rc6 is booted i get this crash. But if i specify selinux=0 at command line, 2.6.29-rc6 boots without any problem.
Quoted 5 lines
Hi Sanchin and Geert,
Does the patch below fix the problems you're seeing? If it does I'll send a properly written up and formatted patch to linuxppc-dev (as well as another one to fix the same problem in
copy_tofrom_user()).
Unfortunately not, now it crashes while accessing the memory pointed to by GPR16, in
NIP: copy_page_range+x0608/0x628
LR: dup_mm+0x2e4/0x428
Trace: debug_table+0xcc70/0x1afe0 (unreliable)
dup_mm+0x2e4/0x428 copy_process+0x86c/0xf9c do_fork+0x188/0x39c sys_clone+0x58/0x70 ppc_clone+0x8/0xc
However, after reverting 25d6e2d7c58ddc4a3b614fc5381591c0cfe66556, I still see similar problems as above (crash in copy_page_range()).
Which makes me think that
1. Your new patch fixes the problem introduced by 25d6e2d7,
2. There's still another issue than the one introduced by 25d6e2d7.
With kind regards,
Geert Uytterhoeven
Software Architect
Sony Techsoft Centre Europe
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium
Phone: +32 (0)2 700 8453 Fax: +32 (0)2 700 8622 E-mail: [email protected] Internet: http://www.sony-europe.com/
A division of Sony Europe (Belgium) N.V.
VAT BE 0413.825.160 · RPR Brussels
Fortis · BIC GEBABEBB · IBAN BE41293037680010
Quoted 35 lines · Mark Nelson
Quoted 29 lines · Geert Uytterhoeven
Quoted 9 lines · Paul Mackerras
Quoted 8 lines · Andrew Morton
It looks like we died in
ext3_xattr_block_get():memcpy(buffer, bh->b_data + le16_to_cpu(entry->e_value_offs), size);Perhaps
entry->e_value_offsis no good. I wonder if the filesystem is corrupted and this snuck through the defenses.Quoted 8 lines
Quoted 3 lines
I also wonder if there is enough info in that trace for a ppc person to be able to determine whether the faulting address is in the source or destination of the
memcpy()(please)?It appears to have faulted on a load, implicating the source. The address being referenced (
0xc00000003f380000) doesn't look outlandish. I wonder if this kernel hasCONFIG_DEBUG_PAGEALLOCturned on, and what page size is selected?I'm seeing a similar thing on PS3, but not in ext3. During early userspace setup (udevd), it crashes accessing a 0xc00* address in:
| NIP setup+0x20/0x130 | LR copy_user_page+0x18/0x6c | Call trace: | do_wp_page+0x5b4/0x89c | do_page_fault+0x3a8/0x58c | handle_page_fault+0x20/0x5cQuoted 4 lines
I have
CONFIG_DEBUG_PAGEALLOC=y. If I disable it, the system boots fine.If needed, I can probably bisect this tomorrow. It definitely didn't happen in 2.6.29-rc5.
Quoted 6 lines
No need to bisect - it was
25d6e2d7c58ddc4a3b614fc5381591c0cfe66556, my commit that "optimised" 64bitmemcpy()for Power6 and Cell.The bug was in -rc1, but if your copies were 8-byte aligned with respect to the source the problem wouldn't have been seen... Could this have been why you didn't see it in -rc5?
Hmm... I just started seeing it on older kernels (-rc5+), too...
With kind regards,
Geert Uytterhoeven
Software Architect
Sony Techsoft Centre Europe
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium
Phone: +32 (0)2 700 8453 Fax: +32 (0)2 700 8622 E-mail: [email protected] Internet: http://www.sony-europe.com/
A division of Sony Europe (Belgium) N.V. VAT BE 0413.825.160 · RPR Brussels Fortis · BIC GEBABEBB · IBAN BE41293037680010
Quoted 6 lines · Mark Nelson
Hi Sanchin and Geert,
Does the patch below fix the problems you're seeing? If it does I'll send a properly written up and formatted patch to linuxppc-dev (as well as another one to fix the same problem in
copy_tofrom_user()).
This patch fixes the issue at my side. I tried booting the system few times and every single time it came up clean.
Thanks
-Sachin
Signature · 6 lines
--------------------------------- Sachin Sant IBM Linux Technology Center India Systems and Technology Labs Bangalore, India ---------------------------------
Quoted 34 lines · Geert Uytterhoeven
Quoted 15 lines · Mark Nelson
Quoted 9 lines · Sachin P. Sant
Quoted 8 lines · Jan Kara
Hmm, OK. But then I'm not sure how that can happen. Obviously, memcpy
somehow got beyond end of the page referenced bybh->b_data. So it means thatle16_to_cpu(entry->e_value_offs)+ size >page_size. Butext3_xattr_find_entry()callsext3_xattr_check_entry()which in particular checks whethere_value_offs+e_value_sizeisn't greater thanbh->b_size. So I see no way how memcpy can get beyond end of the page.
Sachin, is the problem reproducible? If yes, can you send us contentsQuoted 4 lines
Yes, i am able to recreate this problem easily. As i had mentioned if the earlier kernel is booted with selinux enabled and then 2.6.29-rc6 is booted i get this crash. But if i specify selinux=0 at command line, 2.6.29-rc6 boots without any problem.
Quoted 5 lines
Hi Sanchin and Geert,
Does the patch below fix the problems you're seeing? If it does I'll send a properly written up and formatted patch to linuxppc-dev (as well as another one to fix the same problem in
copy_tofrom_user()).Unfortunately not, now it crashes while accessing the memory pointed to by GPR16, in
NIP: copy_page_range+x0608/0x628
LR:dup_mm+0x2e4/0x428
Trace:debug_table+0xcc70/0x1afe0(unreliable)dup_mm+0x2e4/0x428 copy_process+0x86c/0xf9c do_fork+0x188/0x39c sys_clone+0x58/0x70 ppc_clone+0x8/0xc
Quoted 5 lines
However, after reverting
25d6e2d7c58ddc4a3b614fc5381591c0cfe66556, I still see similar problems as above (crash incopy_page_range()).
Which makes me think that
1. Your new patch fixes the problem introduced by25d6e2d7,
2. There's still another issue than the one introduced by25d6e2d7.
Does the following patch fix the errors you're seeing? (it applies the same fix as the previous patch but this time to copy_tofrom_user, which I updated in a4e22f02f5b6518c1484faea1f88d81802b9feac)
Thanks!
Mark
--- arch/powerpc/lib/copyuser_64.S | 38 +++++++++++++++++++++++++++++++------- 1 file changed, 31 insertions(+), 7 deletions(-)
arch/powerpc/lib/copyuser_64.S +31 -7
@@ -62,18 +62,19 @@ END_FTR_SECTION_IFCLR(CPU_FTR_UNALIGNED_ 72: std r8,8(r3) beq+ 3f addi r3,r3,16 -23: ld r9,8(r4) .Ldo_tail: bf cr7*4+1,1f - rotldi r9,r9,32 +23: lwz r9,8(r4) + addi r4,r4,4 73: stw r9,0(r3) addi r3,r3,4 1: bf cr7*4+2,2f - rotldi r9,r9,16 +44: lhz r9,8(r4) + addi r4,r4,2 74: sth r9,0(r3) addi r3,r3,2 2: bf cr7*4+3,3f - rotldi r9,r9,8 +45: lbz r9,8(r4) 75: stb r9,0(r3) 3: li r3,0 blr
@@ -141,11 +142,24 @@ END_FTR_SECTION_IFCLR(CPU_FTR_UNALIGNED_ 6: cmpwi cr1,r5,8 addi r3,r3,32 sld r9,r9,r10 - ble cr1,.Ldo_tail + ble cr1,7f 34: ld r0,8(r4) srd r7,r0,r11 or r9,r7,r9 - b .Ldo_tail +7: + bf cr7*4+1,1f + rotldi r9,r9,32 +94: stw r9,0(r3) + addi r3,r3,4 +1: bf cr7*4+2,2f + rotldi r9,r9,16 +95: sth r9,0(r3) + addi r3,r3,2 +2: bf cr7*4+3,3f + rotldi r9,r9,8 +96: stb r9,0(r3) +3: li r3,0 + blr .Ldst_unaligned: PPC_MTOCRF 0x01,r6 /* put #bytes to 8B bdry into cr7 */
@@ -218,7 +232,6 @@ END_FTR_SECTION_IFCLR(CPU_FTR_UNALIGNED_ 121: 132: addi r3,r3,8 -123: 134: 135: 138:
@@ -226,6 +239,9 @@ END_FTR_SECTION_IFCLR(CPU_FTR_UNALIGNED_ 140: 141: 142: +123: +144: +145: /* * here we have had a fault on a load and r3 points to the first
@@ -309,6 +325,9 @@ END_FTR_SECTION_IFCLR(CPU_FTR_UNALIGNED_ 187: 188: 189: +194: +195: +196: 1: ld r6,-24(r1) ld r5,-8(r1)
@@ -329,7 +348,9 @@ END_FTR_SECTION_IFCLR(CPU_FTR_UNALIGNED_ .llong 72b,172b .llong 23b,123b .llong 73b,173b + .llong 44b,144b .llong 74b,174b + .llong 45b,145b .llong 75b,175b .llong 24b,124b .llong 25b,125b
@@ -347,6 +368,9 @@ END_FTR_SECTION_IFCLR(CPU_FTR_UNALIGNED_ .llong 79b,179b .llong 80b,180b .llong 34b,134b + .llong 94b,194b + .llong 95b,195b + .llong 96b,196b .llong 35b,135b .llong 81b,181b .llong 36b,136b
Quoted 9 lines · Sachin P. Sant
Quoted 6 lines · Mark Nelson
Hi Sanchin and Geert,
Does the patch below fix the problems you're seeing? If it does I'll send a properly written up and formatted patch to linuxppc-dev (as well as another one to fix the same problem in
copy_tofrom_user()).This patch fixes the issue at my side. I tried booting the system few times and every single time it came up clean.
Good to hear. Thanks for testing Sanchin!
Mark
Quoted 41 lines · Mark Nelson
Quoted 34 lines · Geert Uytterhoeven
Quoted 15 lines · Mark Nelson
Quoted 9 lines · Sachin P. Sant
Quoted 8 lines · Jan Kara
Hmm, OK. But then I'm not sure how that can happen. Obviously, memcpy
somehow got beyond end of the page referenced bybh->b_data. So it means thatle16_to_cpu(entry->e_value_offs)+ size >page_size. Butext3_xattr_find_entry()callsext3_xattr_check_entry()which in particular checks whethere_value_offs+e_value_sizeisn't greater thanbh->b_size. So I see no way how memcpy can get beyond end of the page.
Sachin, is the problem reproducible? If yes, can you send us contentsQuoted 4 lines
Yes, i am able to recreate this problem easily. As i had mentioned if the earlier kernel is booted with selinux enabled and then 2.6.29-rc6 is booted i get this crash. But if i specify selinux=0 at command line, 2.6.29-rc6 boots without any problem.
Quoted 5 lines
Hi Sanchin and Geert,
Does the patch below fix the problems you're seeing? If it does I'll send a properly written up and formatted patch to linuxppc-dev (as well as another one to fix the same problem in
copy_tofrom_user()).Unfortunately not, now it crashes while accessing the memory pointed to by GPR16, in
NIP: copy_page_range+x0608/0x628
LR:dup_mm+0x2e4/0x428
Trace:debug_table+0xcc70/0x1afe0(unreliable)dup_mm+0x2e4/0x428 copy_process+0x86c/0xf9c do_fork+0x188/0x39c sys_clone+0x58/0x70 ppc_clone+0x8/0xcQuoted 5 lines
However, after reverting
25d6e2d7c58ddc4a3b614fc5381591c0cfe66556, I still see similar problems as above (crash incopy_page_range()).
Which makes me think that
1. Your new patch fixes the problem introduced by25d6e2d7,
2. There's still another issue than the one introduced by25d6e2d7.
Quoted 3 lines
Does the following patch fix the errors you're seeing? (it applies the same fix as the previous patch but this time to
copy_tofrom_user, which I updated ina4e22f02f5b6518c1484faea1f88d81802b9feac)
Thanks, but I still get crashes in copy_page_range().
With kind regards,
Geert Uytterhoeven
Software Architect
Sony Techsoft Centre Europe
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium
Phone: +32 (0)2 700 8453 Fax: +32 (0)2 700 8622 E-mail: [email protected] Internet: http://www.sony-europe.com/
A division of Sony Europe (Belgium) N.V. VAT BE 0413.825.160 · RPR Brussels Fortis · BIC GEBABEBB · IBAN BE41293037680010
Quoted 42 lines · Geert Uytterhoeven
Quoted 41 lines · Mark Nelson
Quoted 39 lines · Geert Uytterhoeven
Quoted 15 lines · Mark Nelson
Quoted 9 lines · Sachin P. Sant
Quoted 8 lines · Jan Kara
Hmm, OK. But then I'm not sure how that can happen. Obviously, memcpy
somehow got beyond end of the page referenced bybh->b_data. So it means thatle16_to_cpu(entry->e_value_offs)+ size >page_size. Butext3_xattr_find_entry()callsext3_xattr_check_entry()which in particular checks whethere_value_offs+e_value_sizeisn't greater thanbh->b_size. So I see no way how memcpy can get beyond end of the page.
Sachin, is the problem reproducible? If yes, can you send us contentsQuoted 4 lines
Yes, i am able to recreate this problem easily. As i had mentioned if the earlier kernel is booted with selinux enabled and then 2.6.29-rc6 is booted i get this crash. But if i specify selinux=0 at command line, 2.6.29-rc6 boots without any problem.
Quoted 5 lines
Hi Sanchin and Geert,
Does the patch below fix the problems you're seeing? If it does I'll send a properly written up and formatted patch to linuxppc-dev (as well as another one to fix the same problem in
copy_tofrom_user()).Unfortunately not, now it crashes while accessing the memory pointed to by GPR16, in
NIP: copy_page_range+x0608/0x628
LR:dup_mm+0x2e4/0x428
Trace:debug_table+0xcc70/0x1afe0(unreliable)dup_mm+0x2e4/0x428 copy_process+0x86c/0xf9c do_fork+0x188/0x39c sys_clone+0x58/0x70 ppc_clone+0x8/0xcHowever, after reverting
25d6e2d7c58ddc4a3b614fc5381591c0cfe66556, I still see similar problems as above (crash incopy_page_range()).
Which makes me think that
1. Your new patch fixes the problem introduced by25d6e2d7,
2. There's still another issue than the one introduced by25d6e2d7.
Quoted 6 lines
Quoted 3 lines
Does the following patch fix the errors you're seeing? (it applies the same fix as the previous patch but this time to
copy_tofrom_user, which I updated ina4e22f02f5b6518c1484faea1f88d81802b9feac)Thanks, but I still get crashes in
copy_page_range().
Hmmm... I'm out of ideas for the moment, but thanks for testing anyway!
Mark
Quoted 50 lines · Mark Nelson
Quoted 42 lines · Geert Uytterhoeven
Quoted 41 lines · Mark Nelson
Quoted 39 lines · Geert Uytterhoeven
Quoted 15 lines · Mark Nelson
Quoted 9 lines · Sachin P. Sant
Quoted 8 lines · Jan Kara
Hmm, OK. But then I'm not sure how that can happen. Obviously, memcpy
somehow got beyond end of the page referenced bybh->b_data. So it means thatle16_to_cpu(entry->e_value_offs)+ size >page_size. Butext3_xattr_find_entry()callsext3_xattr_check_entry()which in particular checks whethere_value_offs+e_value_sizeisn't greater thanbh->b_size. So I see no way how memcpy can get beyond end of the page.
Sachin, is the problem reproducible? If yes, can you send us contentsQuoted 4 lines
Yes, i am able to recreate this problem easily. As i had mentioned if the earlier kernel is booted with selinux enabled and then 2.6.29-rc6 is booted i get this crash. But if i specify selinux=0 at command line, 2.6.29-rc6 boots without any problem.
Quoted 5 lines
Hi Sanchin and Geert,
Does the patch below fix the problems you're seeing? If it does I'll send a properly written up and formatted patch to linuxppc-dev (as well as another one to fix the same problem in
copy_tofrom_user()).Unfortunately not, now it crashes while accessing the memory pointed to by GPR16, in
NIP: copy_page_range+x0608/0x628
LR:dup_mm+0x2e4/0x428
Trace:debug_table+0xcc70/0x1afe0(unreliable)dup_mm+0x2e4/0x428 copy_process+0x86c/0xf9c do_fork+0x188/0x39c sys_clone+0x58/0x70 ppc_clone+0x8/0xcHowever, after reverting
25d6e2d7c58ddc4a3b614fc5381591c0cfe66556, I still see similar problems as above (crash incopy_page_range()).
Which makes me think that
1. Your new patch fixes the problem introduced by25d6e2d7,
2. There's still another issue than the one introduced by25d6e2d7.Quoted 6 lines
Quoted 3 lines
Does the following patch fix the errors you're seeing? (it applies the same fix as the previous patch but this time to
copy_tofrom_user, which I updated ina4e22f02f5b6518c1484faea1f88d81802b9feac)Thanks, but I still get crashes in
copy_page_range().
Quoted 8 lines
Hmmm... I'm out of ideas for the moment, but thanks for testing anyway!
Mark
_______________________________________________
Linuxppc-dev mailing list
[email protected]
https://ozlabs.org/mailman/listinfo/linuxppc-dev
If you revert both 25d6e2d7c58ddc4a3b614fc5381591c0cfe66556 and a4e22f02f5b6518c1484faea1f88d81802b9feac, does it help? You could also try to revert 57dda6ef5bd5b9e60410477ad29e654097e2cca1 just in case I need to keep wearing the brown paper bag for a bit longer :)
Thanks!
Mark
PATCH 1/1 powerpc: Fix 64bit memcpy() regression
In reply to Mark NelsonThis fixes a regression introduced by commit 25d6e2d7c58ddc4a3b614fc5381591c0cfe66556 ("powerpc: Update 64bit memcpy() using CPU_FTR_UNALIGNED_LD_STD").
This commit allowed CPUs that have the CPU_FTR_UNALIGNED_LD_STD CPU feature bit present to do the memcpy() with unaligned load doubles. But, along with this came a bug where our final load double would read bytes beyond a page boundary and into the next (unmapped) page. This was caught by enabling CONFIG_DEBUG_PAGEALLOC,
The fix was to read only the number of bytes that we need to store rather than reading a full 8-byte doubleword and storing only a portion of that.
In order to minimise the amount of existing code touched we use the original do_tail for the src_unaligned case.
Below is an example of the regression, as reported by Sachin Sant:
Unable to handle kernel paging request for data at address 0xc00000003f380000
Faulting instruction address: 0xc000000000039574
cpu 0x1: Vector: 300 (Data Access) at [c00000003baf3020]
pc: c000000000039574: .memcpy+0x74/0x244
lr: d00000000244916c: .ext3_xattr_get+0x288/0x2f4 [ext3]
sp: c00000003baf32a0 msr: 8000000000009032
dar: c00000003f380000
dsisr: 40000000
current = 0xc00000003e54b010
paca = 0xc000000000a53680
pid = 1840, comm = readahead
enter ? for help
[link register ] d00000000244916c .ext3_xattr_get+0x288/0x2f4 [ext3]
[c00000003baf32a0] d000000002449104 .ext3_xattr_get+0x220/0x2f4 [ext3] (unreliab
le)
[c00000003baf3390] d00000000244a6e8 .ext3_xattr_security_get+0x40/0x5c [ext3] [c00000003baf3400] c000000000148154 .generic_getxattr+0x74/0x9c [c00000003baf34a0] c000000000333400 .inode_doinit_with_dentry+0x1c4/0x678 [c00000003baf3560] c00000000032c6b0 .security_d_instantiate+0x50/0x68 [c00000003baf35e0] c00000000013c818 .d_instantiate+0x78/0x9c [c00000003baf3680] c00000000013ced0 .d_splice_alias+0xf0/0x120 [c00000003baf3720] d00000000243e05c .ext3_lookup+0xec/0x134 [ext3] [c00000003baf37c0] c000000000131e74 .do_lookup+0x110/0x260 [c00000003baf3880] c000000000134ed0 .__link_path_walk+0xa98/0x1010 [c00000003baf3970] c0000000001354a0 .path_walk+0x58/0xc4 [c00000003baf3a20] c000000000135720 .do_path_lookup+0x138/0x1e4 [c00000003baf3ad0] c00000000013645c .path_lookup_open+0x6c/0xc8 [c00000003baf3b70] c000000000136780 .do_filp_open+0xcc/0x874 [c00000003baf3d10] c0000000001251e0 .do_sys_open+0x80/0x140 [c00000003baf3dc0] c00000000016aaec .compat_sys_open+0x24/0x38 [c00000003baf3e30] c00000000000855c syscall_exit+0x0/0x40
--- Exception: c01 (System Call) at 000000000ff0ef18
SP (ffc6f4b0) is in userspace
1:mon>
--- arch/powerpc/lib/memcpy_64.S | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-)
arch/powerpc/lib/memcpy_64.S +20 -6
@@ -53,18 +53,19 @@ END_FTR_SECTION_IFCLR(CPU_FTR_UNALIGNED_ 3: std r8,8(r3) beq 3f addi r3,r3,16 - ld r9,8(r4) .Ldo_tail: bf cr7*4+1,1f - rotldi r9,r9,32 + lwz r9,8(r4) + addi r4,r4,4 stw r9,0(r3) addi r3,r3,4 1: bf cr7*4+2,2f - rotldi r9,r9,16 + lhz r9,8(r4) + addi r4,r4,2 sth r9,0(r3) addi r3,r3,2 2: bf cr7*4+3,3f - rotldi r9,r9,8 + lbz r9,8(r4) stb r9,0(r3) 3: ld r3,48(r1) /* return dest pointer */ blr
@@ -133,11 +134,24 @@ END_FTR_SECTION_IFCLR(CPU_FTR_UNALIGNED_ cmpwi cr1,r5,8 addi r3,r3,32 sld r9,r9,r10 - ble cr1,.Ldo_tail + ble cr1,6f ld r0,8(r4) srd r7,r0,r11 or r9,r7,r9 - b .Ldo_tail +6: + bf cr7*4+1,1f + rotldi r9,r9,32 + stw r9,0(r3) + addi r3,r3,4 +1: bf cr7*4+2,2f + rotldi r9,r9,16 + sth r9,0(r3) + addi r3,r3,2 +2: bf cr7*4+3,3f + rotldi r9,r9,8 + stb r9,0(r3) +3: ld r3,48(r1) /* return dest pointer */ + blr .Ldst_unaligned: PPC_MTOCRF 0x01,r6 # put #bytes to 8B bdry into cr7
PATCH 1/1 powerpc: Fix 64bit __copy_tofrom_user() regression
This fixes a regression introduced by commit a4e22f02f5b6518c1484faea1f88d81802b9feac ("powerpc: Update 64bit __copy_tofrom_user() using CPU_FTR_UNALIGNED_LD_STD").
The same bug that existed in the 64bit memcpy() also exists here so fix it here too. The fix is the same as that applied to memcpy() with the addition of fixes for the exception handling code required for __copy_tofrom_user().
This stops us reading beyond the end of the source region we were told to copy.
--- arch/powerpc/lib/copyuser_64.S | 38 +++++++++++++++++++++++++++++++------- 1 file changed, 31 insertions(+), 7 deletions(-)
arch/powerpc/lib/copyuser_64.S +31 -7
@@ -62,18 +62,19 @@ END_FTR_SECTION_IFCLR(CPU_FTR_UNALIGNED_ 72: std r8,8(r3) beq+ 3f addi r3,r3,16 -23: ld r9,8(r4) .Ldo_tail: bf cr7*4+1,1f - rotldi r9,r9,32 +23: lwz r9,8(r4) + addi r4,r4,4 73: stw r9,0(r3) addi r3,r3,4 1: bf cr7*4+2,2f - rotldi r9,r9,16 +44: lhz r9,8(r4) + addi r4,r4,2 74: sth r9,0(r3) addi r3,r3,2 2: bf cr7*4+3,3f - rotldi r9,r9,8 +45: lbz r9,8(r4) 75: stb r9,0(r3) 3: li r3,0 blr
@@ -141,11 +142,24 @@ END_FTR_SECTION_IFCLR(CPU_FTR_UNALIGNED_ 6: cmpwi cr1,r5,8 addi r3,r3,32 sld r9,r9,r10 - ble cr1,.Ldo_tail + ble cr1,7f 34: ld r0,8(r4) srd r7,r0,r11 or r9,r7,r9 - b .Ldo_tail +7: + bf cr7*4+1,1f + rotldi r9,r9,32 +94: stw r9,0(r3) + addi r3,r3,4 +1: bf cr7*4+2,2f + rotldi r9,r9,16 +95: sth r9,0(r3) + addi r3,r3,2 +2: bf cr7*4+3,3f + rotldi r9,r9,8 +96: stb r9,0(r3) +3: li r3,0 + blr .Ldst_unaligned: PPC_MTOCRF 0x01,r6 /* put #bytes to 8B bdry into cr7 */
@@ -218,7 +232,6 @@ END_FTR_SECTION_IFCLR(CPU_FTR_UNALIGNED_ 121: 132: addi r3,r3,8 -123: 134: 135: 138:
@@ -226,6 +239,9 @@ END_FTR_SECTION_IFCLR(CPU_FTR_UNALIGNED_ 140: 141: 142: +123: +144: +145: /* * here we have had a fault on a load and r3 points to the first
@@ -309,6 +325,9 @@ END_FTR_SECTION_IFCLR(CPU_FTR_UNALIGNED_ 187: 188: 189: +194: +195: +196: 1: ld r6,-24(r1) ld r5,-8(r1)
@@ -329,7 +348,9 @@ END_FTR_SECTION_IFCLR(CPU_FTR_UNALIGNED_ .llong 72b,172b .llong 23b,123b .llong 73b,173b + .llong 44b,144b .llong 74b,174b + .llong 45b,145b .llong 75b,175b .llong 24b,124b .llong 25b,125b
@@ -347,6 +368,9 @@ END_FTR_SECTION_IFCLR(CPU_FTR_UNALIGNED_ .llong 79b,179b .llong 80b,180b .llong 34b,134b + .llong 94b,194b + .llong 95b,195b + .llong 96b,196b .llong 35b,135b .llong 81b,181b .llong 36b,136b
Quoted 7 lines · Mark Nelson
Quoted 6 lines · Mark Nelson
Quoted 5 lines · Geert Uytterhoeven
Quoted 3 lines · Mark Nelson
Does the following patch fix the errors you're seeing? (it applies the same fix as the previous patch but this time to
copy_tofrom_user, which I updated ina4e22f02f5b6518c1484faea1f88d81802b9feac)
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
Thanks, but I still get crashes in
copy_page_range().Hmmm... I'm out of ideas for the moment, but thanks for testing anyway!
If you revert both
25d6e2d7c58ddc4a3b614fc5381591c0cfe66556anda4e22f02f5b6518c1484faea1f88d81802b9feac, does it help? You could also try to revert57dda6ef5bd5b9e60410477ad29e654097e2cca1just in case I need to keep wearing the brown paper bag for a bit longer :)
Still doesn't help.
However, I noticed I never enabled CONFIG_DEBUG_PAGEALLOC before 2.6.29-rc5. So far I tried 2.6.2[5-8], and they all crash with CONFIG_DEBUG_PAGEALLOC. I guess it never actually worked on PS3.
With kind regards,
Geert Uytterhoeven
Software Architect
Sony Techsoft Centre Europe
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium
Phone: +32 (0)2 700 8453 Fax: +32 (0)2 700 8622 E-mail: [email protected] Internet: http://www.sony-europe.com/
A division of Sony Europe (Belgium) N.V.
VAT BE 0413.825.160 · RPR Brussels
Fortis · BIC GEBABEBB · IBAN BE41293037680010