2.6.9-mm1 oops
100% reproducible with elevator=cfq
Unable to handle kernel NULL pointer dereference at virtual address 00000000 printing eip: c013f2c7 *pde = 00000000 Oops: 0000 [#1] PREEMPT Modules linked in: ipv6 dm_mod emu10k1 sound soundcore ac97_codec unix CPU: 0 EIP: 0060:[put_page+7/144] Not tainted VLI EFLAGS: 00010282 (2.6.9-mm1)
EIP is at put_page+0x7/0x90
eax: 00000000 ebx: df047400 ecx: 00000000 edx: 00000000
esi: 00000000 edi: 00000000 ebp: df047400 esp: cadd0c80
ds: 007b es: 007b ss: 0068
Process diotest4 (pid: 4498, threadinfo=cadd0000 task=de712a40)
Stack: df047400 c0178d50 00000000 00000000 c0179c91 df047400 00000001 c01b391b
c86b6b00 cb0c9b80 cadd0d1c cb0c9c38 c019d31e cee0b0b0 00000000 00000000
00003000 00000000 df047400 00000000 08051000 0000000c c017a182 00000001
Call Trace:
[dio_cleanup+48/64] dio_cleanup+0x30/0x40
[direct_io_worker+817/1568] direct_io_worker+0x331/0x620
[journal_put_journal_head+59/192] journal_put_journal_head+0x3b/0xc0
[ext3_do_update_inode+366/944] ext3_do_update_inode+0x16e/0x3b0
[__blockdev_direct_IO+514/752] __blockdev_direct_IO+0x202/0x2f0
[ext3_direct_io_get_blocks+0/240] ext3_direct_io_get_blocks+0x0/0xf0
[ext3_direct_IO+201/576] ext3_direct_IO+0xc9/0x240
[ext3_direct_io_get_blocks+0/240] ext3_direct_io_get_blocks+0x0/0xf0
[generic_file_direct_IO+121/160] generic_file_direct_IO+0x79/0xa0
[generic_file_direct_write+134/416] generic_file_direct_write+0x86/0x1a0
[generic_file_aio_write_nolock+689/1200] generic_file_aio_write_nolock+0x2b1/0x4b0
[generic_file_aio_write+131/256] generic_file_aio_write+0x83/0x100
[ext3_file_write+68/224] ext3_file_write+0x44/0xe0
[do_sync_write+190/240] do_sync_write+0xbe/0xf0
[autoremove_wake_function+0/96] autoremove_wake_function+0x0/0x60
[dnotify_parent+162/224] dnotify_parent+0xa2/0xe0
[do_sync_write+0/240] do_sync_write+0x0/0xf0
[vfs_write+184/304] vfs_write+0xb8/0x130
[sys_write+81/128] sys_write+0x51/0x80
[syscall_call+7/11] syscall_call+0x7/0xb
Code: 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 83 ec 04 8b 54 24 08 <8b> 02 a9 00 80 00 00 75 44 8b 02 f6 c4 08 75 1f 8b 02 89 d1 a9
<1>Unable to handle kernel paging request at virtual address 4e6c9214
printing eip:
c0178939
*pde = 00000000
Oops: 0000 [#2]
PREEMPT
Modules linked in: ipv6 dm_mod emu10k1 sound soundcore ac97_codec unix
CPU: 0
EIP: 0060:[dio_get_page+25/96] Not tainted VLI
EFLAGS: 00010216 (2.6.9-mm1) EIP is at dio_get_page+0x19/0x60
eax: 62615a59 ebx: c4e72800 ecx: 00000000 edx: 00000000
esi: 00000000 edi: 00000000 ebp: c4e72800 esp: de868c7c
ds: 007b es: 007b ss: 0068
Process diotest1 (pid: 4493, threadinfo=de868000 task=d1c435d0)
Stack: dffe0520 c4e72800 c0178d48 c4e72800 00000000 c0179c91 c4e72800 00000001
c01b391b c86b6380 c08349fc de868d1c c0834ab4 c019d31e cee0b0b0 00000000
00000000 00040000 00000000 c4e72800 00000000 08050000 0000000c c017a182
Call Trace:
[dio_cleanup+40/64] dio_cleanup+0x28/0x40
[direct_io_worker+817/1568] direct_io_worker+0x331/0x620
[journal_put_journal_head+59/192] journal_put_journal_head+0x3b/0xc0
[ext3_do_update_inode+366/944] ext3_do_update_inode+0x16e/0x3b0
[__blockdev_direct_IO+514/752] __blockdev_direct_IO+0x202/0x2f0
[ext3_direct_io_get_blocks+0/240] ext3_direct_io_get_blocks+0x0/0xf0
[ext3_direct_IO+201/576] ext3_direct_IO+0xc9/0x240
[ext3_direct_io_get_blocks+0/240] ext3_direct_io_get_blocks+0x0/0xf0
[generic_file_direct_IO+121/160] generic_file_direct_IO+0x79/0xa0
[generic_file_direct_write+134/416] generic_file_direct_write+0x86/0x1a0
[generic_file_aio_write_nolock+689/1200] generic_file_aio_write_nolock+0x2b1/0x4b0
[generic_file_aio_write+131/256] generic_file_aio_write+0x83/0x100
[ext3_file_write+68/224] ext3_file_write+0x44/0xe0
[do_sync_write+190/240] do_sync_write+0xbe/0xf0
[autoremove_wake_function+0/96] autoremove_wake_function+0x0/0x60
[dnotify_parent+162/224] dnotify_parent+0xa2/0xe0
[do_sync_write+0/240] do_sync_write+0x0/0xf0
[vfs_write+184/304] vfs_write+0xb8/0x130
[schedule+770/1376] schedule+0x302/0x560
[sys_write+81/128] sys_write+0x51/0x80
[syscall_call+7/11] syscall_call+0x7/0xb
Code: 00 89 87 b0 00 00 00 eb a5 89 f6 8d bc 27 00 00 00 00 83 ec 08 89 5c 24 04 8b 5c 24 0c 8b 83 b0 01 00 00 39 83 b4 01 00 00 74 18 <8b> 94 83 b0 00 00 00 40 89 83 b0 01 00 00 89 d0 8b 5c 24 04 83 Signature · 1 line
I route therefore you are
Quoted 2 lines · Lorenzo Allegrucci
100% reproducible with elevator=cfq
but not with the other io schedulers?
Quoted 31 lines
Unable to handle kernel NULL pointer dereference at virtual address 00000000 printing eip: c013f2c7 *pde = 00000000 Oops: 0000 [#1] PREEMPT Modules linked in: ipv6 dm_mod emu10k1 sound soundcore ac97_codec unix CPU: 0 EIP: 0060:[put_page+7/144] Not tainted VLI EFLAGS: 00010282 (2.6.9-mm1)EIP is at
put_page+0x7/0x90
eax: 00000000 ebx:df047400ecx: 00000000 edx: 00000000
esi: 00000000 edi: 00000000 ebp:df047400esp:cadd0c80
ds: 007b es: 007b ss: 0068
Process diotest4 (pid: 4498, threadinfo=cadd0000 task=de712a40)Stack: df047400 c0178d50 00000000 00000000 c0179c91 df047400 00000001 c01b391b c86b6b00 cb0c9b80 cadd0d1c cb0c9c38 c019d31e cee0b0b0 00000000 00000000 00003000 00000000 df047400 00000000 08051000 0000000c c017a182 00000001 Call Trace: [dio_cleanup+48/64] dio_cleanup+0x30/0x40 [direct_io_worker+817/1568] direct_io_worker+0x331/0x620 [journal_put_journal_head+59/192] journal_put_journal_head+0x3b/0xc0 [ext3_do_update_inode+366/944] ext3_do_update_inode+0x16e/0x3b0 [__blockdev_direct_IO+514/752] __blockdev_direct_IO+0x202/0x2f0 [ext3_direct_io_get_blocks+0/240] ext3_direct_io_get_blocks+0x0/0xf0 [ext3_direct_IO+201/576] ext3_direct_IO+0xc9/0x240 [ext3_direct_io_get_blocks+0/240] ext3_direct_io_get_blocks+0x0/0xf0 [generic_file_direct_IO+121/160] generic_file_direct_IO+0x79/0xa0 [generic_file_direct_write+134/416] generic_file_direct_write+0x86/0x1a0 [generic_file_aio_write_nolock+689/1200] generic_file_aio_write_nolock+0x2b1/0x4b0 [generic_file_aio_write+131/256] generic_file_aio_write+0x83/0x100
Quoted 8 lines
[ext3_file_write+68/224] ext3_file_write+0x44/0xe0 [do_sync_write+190/240] do_sync_write+0xbe/0xf0 [autoremove_wake_function+0/96] autoremove_wake_function+0x0/0x60 [dnotify_parent+162/224] dnotify_parent+0xa2/0xe0 [do_sync_write+0/240] do_sync_write+0x0/0xf0 [vfs_write+184/304] vfs_write+0xb8/0x130 [sys_write+81/128] sys_write+0x51/0x80 [syscall_call+7/11] syscall_call+0x7/0xb
Please provide a test case, or at least a description of the problem.
Signature · 1 line
Jens Axboe
Quoted 5 lines · Jens Axboe
Quoted 2 lines · Lorenzo Allegrucci
100% reproducible with elevator=cfq
but not with the other io schedulers?
No, now I can reproduce it with the anticipatory scheduler too.
Quoted 40 lines
Quoted 31 lines
Unable to handle kernel NULL pointer dereference at virtual address 00000000 printing eip: c013f2c7 *pde = 00000000 Oops: 0000 [#1] PREEMPT Modules linked in: ipv6 dm_mod emu10k1 sound soundcore ac97_codec unix CPU: 0 EIP: 0060:[put_page+7/144] Not tainted VLI EFLAGS: 00010282 (2.6.9-mm1)EIP is at
put_page+0x7/0x90
eax: 00000000 ebx:df047400ecx: 00000000 edx: 00000000
esi: 00000000 edi: 00000000 ebp:df047400esp:cadd0c80
ds: 007b es: 007b ss: 0068
Process diotest4 (pid: 4498, threadinfo=cadd0000 task=de712a40)Stack: df047400 c0178d50 00000000 00000000 c0179c91 df047400 00000001 c01b391b c86b6b00 cb0c9b80 cadd0d1c cb0c9c38 c019d31e cee0b0b0 00000000 00000000 00003000 00000000 df047400 00000000 08051000 0000000c c017a182 00000001 Call Trace: [dio_cleanup+48/64] dio_cleanup+0x30/0x40 [direct_io_worker+817/1568] direct_io_worker+0x331/0x620 [journal_put_journal_head+59/192] journal_put_journal_head+0x3b/0xc0 [ext3_do_update_inode+366/944] ext3_do_update_inode+0x16e/0x3b0 [__blockdev_direct_IO+514/752] __blockdev_direct_IO+0x202/0x2f0 [ext3_direct_io_get_blocks+0/240] ext3_direct_io_get_blocks+0x0/0xf0 [ext3_direct_IO+201/576] ext3_direct_IO+0xc9/0x240 [ext3_direct_io_get_blocks+0/240] ext3_direct_io_get_blocks+0x0/0xf0 [generic_file_direct_IO+121/160] generic_file_direct_IO+0x79/0xa0 [generic_file_direct_write+134/416] generic_file_direct_write+0x86/0x1a0 [generic_file_aio_write_nolock+689/1200] generic_file_aio_write_nolock+0x2b1/0x4b0 [generic_file_aio_write+131/256] generic_file_aio_write+0x83/0x100Quoted 8 lines
[ext3_file_write+68/224] ext3_file_write+0x44/0xe0 [do_sync_write+190/240] do_sync_write+0xbe/0xf0 [autoremove_wake_function+0/96] autoremove_wake_function+0x0/0x60 [dnotify_parent+162/224] dnotify_parent+0xa2/0xe0 [do_sync_write+0/240] do_sync_write+0x0/0xf0 [vfs_write+184/304] vfs_write+0xb8/0x130 [sys_write+81/128] sys_write+0x51/0x80 [syscall_call+7/11] syscall_call+0x7/0xb
Quoted 1 line
Please provide a test case, or at least a description of the problem.
I can 100% reproduce it running "runalltests.sh -x 100" of ltp-full-20040804.
The test cases seem to be diotest1 and diotest4 because they always appear in the oops, but I couldn't reproduce it running diotest1 and diotest4 alone.
Signature · 1 line
I route therefore you are
Quoted 8 lines · Lorenzo Allegrucci
Quoted 5 lines · Jens Axboe
Quoted 2 lines · Lorenzo Allegrucci
100% reproducible with elevator=cfq
but not with the other io schedulers?
No, now I can reproduce it with the anticipatory scheduler too.
I've just got the same oops using XFS instead of ext3.
Unable to handle kernel NULL pointer dereference at virtual address 0000007a printing eip: c013f2c7 *pde = 00000000 Oops: 0000 [#1] PREEMPT Modules linked in: xfs ipv6 dm_mod emu10k1 sound soundcore ac97_codec unix CPU: 0 EIP: 0060:[put_page+7/144] Not tainted VLI EFLAGS: 00010296 (2.6.9-mm1)
EIP is at put_page+0x7/0x90
eax: 0000007a ebx: de3f3400 ecx: 00000000 edx: 0000007a
esi: 00000000 edi: 00000000 ebp: de3f3400 esp: dd93ac5c
ds: 007b es: 007b ss: 0068
Process diotest1 (pid: 4795, threadinfo=dd93a000 task=dd923a00)
Stack: de3f3400 c0178d50 0000007a 00000000 c0179c91 de3f3400 00000001 d59075ec
00000001 00000008 e0a96eb5 d8e63610 d8e636dc 00000000 00000000 00000000
00000000 00000000 de3f3400 00000000 08050000 0000000c c017a182 00000001
Call Trace:
[dio_cleanup+48/64] dio_cleanup+0x30/0x40
[direct_io_worker+817/1568] direct_io_worker+0x331/0x620
[pg0+543870645/1069630464] xfs_ilock_map_shared+0x25/0x40 [xfs]
[__blockdev_direct_IO+514/752] __blockdev_direct_IO+0x202/0x2f0
[pg0+544038528/1069630464] linvfs_get_blocks_direct+0x0/0x50 [xfs]
[pg0+544032512/1069630464] linvfs_unwritten_convert_direct+0x0/0x70 [xfs]
[pg0+544038830/1069630464] linvfs_direct_IO+0xde/0xe0 [xfs]
[pg0+544038528/1069630464] linvfs_get_blocks_direct+0x0/0x50 [xfs]
[pg0+544032512/1069630464] linvfs_unwritten_convert_direct+0x0/0x70 [xfs]
[generic_file_direct_IO+121/160] generic_file_direct_IO+0x79/0xa0
[generic_file_direct_write+134/416] generic_file_direct_write+0x86/0x1a0
[pg0+544067824/1069630464] xfs_write+0x3b0/0xc00 [xfs]
[pg0+544049612/1069630464] linvfs_write+0x8c/0xa0 [xfs]
[do_sync_write+190/240] do_sync_write+0xbe/0xf0
[autoremove_wake_function+0/96] autoremove_wake_function+0x0/0x60
[do_sync_write+0/240] do_sync_write+0x0/0xf0
[vfs_write+184/304] vfs_write+0xb8/0x130
[sys_write+81/128] sys_write+0x51/0x80
[syscall_call+7/11] syscall_call+0x7/0xb
Code: 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 83 ec 04 8b 54 24 08 <8b> 02 a9 00 80 00 00 75 44 8b 02 f6 c4 08 75 1f 8b 02 89 d1 a9
<1>Unable to handle kernel paging request at virtual address 5a3f30b0
printing eip:
c0178939
*pde = 00000000
Oops: 0000 [#2]
PREEMPT
Modules linked in: xfs ipv6 dm_mod emu10k1 sound soundcore ac97_codec unix
CPU: 0
EIP: 0060:[dio_get_page+25/96] Not tainted VLI
EFLAGS: 00010a02 (2.6.9-mm1) EIP is at dio_get_page+0x19/0x60
eax: 5f000000 ebx: de3f3000 ecx: 00000000 edx: 00000000
esi: 00000000 edi: 00000000 ebp: de3f3000 esp: dc930c58
ds: 007b es: 007b ss: 0068
Process diotest4 (pid: 4947, threadinfo=dc930000 task=dcde9aa0)
Stack: 0000000e de3f3000 c0178d48 de3f3000 00000000 c0179c91 de3f3000 00000001
0000000e 0000003f 00000008 e0a96eb5 d14b65d0 d14b669c 00000000 00000000
00000000 0000f000 00000000 de3f3000 00000000 08051000 0000000c c017a182
Call Trace:
[dio_cleanup+40/64] dio_cleanup+0x28/0x40
[direct_io_worker+817/1568] direct_io_worker+0x331/0x620
[pg0+543870645/1069630464] xfs_ilock_map_shared+0x25/0x40 [xfs]
[__blockdev_direct_IO+514/752] __blockdev_direct_IO+0x202/0x2f0
[pg0+544038528/1069630464] linvfs_get_blocks_direct+0x0/0x50 [xfs]
[pg0+544032512/1069630464] linvfs_unwritten_convert_direct+0x0/0x70 [xfs]
[pg0+544038830/1069630464] linvfs_direct_IO+0xde/0xe0 [xfs]
[pg0+544038528/1069630464] linvfs_get_blocks_direct+0x0/0x50 [xfs]
[pg0+544032512/1069630464] linvfs_unwritten_convert_direct+0x0/0x70 [xfs]
[filemap_fdatawait+115/128] filemap_fdatawait+0x73/0x80
[generic_file_direct_IO+121/160] generic_file_direct_IO+0x79/0xa0
[generic_file_direct_write+134/416] generic_file_direct_write+0x86/0x1a0
[pg0+544067824/1069630464] xfs_write+0x3b0/0xc00 [xfs]
[pg0+544049612/1069630464] linvfs_write+0x8c/0xa0 [xfs]
[do_sync_write+190/240] do_sync_write+0xbe/0xf0
[autoremove_wake_function+0/96] autoremove_wake_function+0x0/0x60
[dnotify_parent+162/224] dnotify_parent+0xa2/0xe0
[do_sync_write+0/240] do_sync_write+0x0/0xf0
[vfs_write+184/304] vfs_write+0xb8/0x130
[sys_write+81/128] sys_write+0x51/0x80
[syscall_call+7/11] syscall_call+0x7/0xb
Code: 00 89 87 b0 00 00 00 eb a5 89 f6 8d bc 27 00 00 00 00 83 ec 08 89 5c 24 04 8b 5c 24 0c 8b 83 b0 01 00 00 39 83 b4 01 00 00 74 18 <8b> 94 83 b0 00 00 00 40 89 83 b0 01 00 00 89 d0 8b 5c 24 04 83 Signature · 1 line
I route therefore you are
Quoted 11 lines · Lorenzo Allegrucci
Quoted 8 lines · Lorenzo Allegrucci
Quoted 5 lines · Jens Axboe
Quoted 2 lines · Lorenzo Allegrucci
100% reproducible with elevator=cfq
but not with the other io schedulers?
No, now I can reproduce it with the anticipatory scheduler too.
I've just got the same oops using XFS instead of ext3.
And disabling CONFIG_PREEMPT_BKL doesn't help.
Signature · 1 line
I route therefore you are
Quoted 7 lines · Lorenzo Allegrucci
Quoted 6 lines · Lorenzo Allegrucci
Quoted 5 lines · Lorenzo Allegrucci
Quoted 4 lines · Jens Axboe
Quoted 2 lines · Lorenzo Allegrucci
100% reproducible with elevator=cfq
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
but not with the other io schedulers?
No, now I can reproduce it with the anticipatory scheduler too.
I've just got the same oops using XFS instead of ext3.
And disabling
CONFIG_PREEMPT_BKLdoesn't help.
I'll try to reproduce it here now.
Signature · 1 line
Jens Axboe
Quoted 10 lines · Jens Axboe
Quoted 9 lines · Lorenzo Allegrucci
Quoted 8 lines · Lorenzo Allegrucci
Quoted 7 lines · Lorenzo Allegrucci
Quoted 5 lines · Jens Axboe
Quoted 2 lines · Lorenzo Allegrucci
100% reproducible with elevator=cfq
but not with the other io schedulers?
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
No, now I can reproduce it with the anticipatory scheduler too.
I've just got the same oops using XFS instead of ext3.
And disabling
CONFIG_PREEMPT_BKLdoesn't help.I'll try to reproduce it here now.
Something fishy is going on here - it seems to happen in the if (ret) cleanup in direct_io_worker(), last error was -EFAULT for this case. And reverting the dio eof patch 'fixes' the bug, I don't know why yet.
Signature · 1 line
Jens Axboe
Quoted 15 lines · Jens Axboe
Quoted 14 lines · Jens Axboe
Quoted 13 lines · Lorenzo Allegrucci
Quoted 11 lines · Lorenzo Allegrucci
Quoted 8 lines · Lorenzo Allegrucci
Quoted 5 lines · Jens Axboe
Quoted 2 lines · Lorenzo Allegrucci
100% reproducible with elevator=cfq
but not with the other io schedulers?
No, now I can reproduce it with the anticipatory scheduler too.
I've just got the same oops using XFS instead of ext3.
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
And disabling
CONFIG_PREEMPT_BKLdoesn't help.I'll try to reproduce it here now.
Something fishy is going on here - it seems to happen in the if (ret) cleanup in
direct_io_worker(), last error was-EFAULTfor this case. And reverting the dio eof patch 'fixes' the bug, I don't know why yet.
->head and ->tail were not initialized in the cleanup path, I'm guessing this happens if we adjust the read to zero. Seems best to simply check for that condition and bail early, instead of initing ->head and tail earlier and go through the whole path.
fs/direct-io.c renamed from opt/kernel/BK/linux-2.6/fs/direct-io.c +2 -0
@@ -987,6 +987,8 @@ isize =i_size_read (inode);if (bytes_todo > (isize - offset)) bytes_todo = isize - offset; +if (!bytes_todo) +return 0 ;for (seg =0 ; seg < nr_segs && bytes_todo; seg++) { user_addr = (unsigned long )iov[seg].iov_base;
Signature · 1 line
Jens Axboe