Ext4 stack trace with savedwrite patches
Hi,
Quoted 3 lines · Aneesh Kumar K.V
I am hitting this while running stress test with the saved write patch series. I guess we are missing a set page dirty some where. I will continue to debug this, but if you have any suggestion let me know.
<snip>
So this warning can happen when page got dirtied but ->page_mkwrite() was not called. I don't know details of how autonuma works but a quick look suggests that autonuma can also do numa hinting faults for file pages. So the following seems to be possible:
Autonuma decides to check for accesses to a mapped shared file page that is dirty. pte_present gets cleared, pte_write stays set (due to logic introduced in commit b191f9b106 "mm: numa: preserve PTE write permissions across a NUMA hinting fault"). Then page writeback happens, page_mkclean() is called to write-protect the page. However page_check_address() returns NULL for the PTE (__page_check_address() returns NULL for !pte_present PTEs) so we don't clear pte_write bit in page_mkclean_one(). Sometime later a process looks at the page through mmap, takes NUMA fault and do_numa_page() reestablishes a writeable mapping of the page although the filesystem does not expect there to be one and funny things happen afterwards...
I'll defer to more mm-savvy people to decide how this should be fixed. My naive understanding is that page_mkclean_one() should clear the pte_write bit even for pages that are undergoing NUMA probation but I'm not sure about a preferred way to achieve that...
Honza
Quoted 34 lines
[ 3177.528954] ------------[ cut here ]------------
[ 3177.528968] WARNING: CPU: 155 PID: 84480 at fs/ext4/inode.c:3711 ext4_set_page_dirty+0x9c/0xe0
[ 3177.528969] Modules linked in: powernv_op_panel
[ 3177.528977] CPU: 155 PID: 84480 Comm: stress-ng-mmap Not tainted 4.10.0-rc8-00038-g528b408-dirty #6
[ 3177.528979] task: c000000bbbda7d00 task.stack: c000001d777c0000
[ 3177.528981] NIP: c00000000043322c LR: c00000000027f460 CTR: c000000000433190
[ 3177.528983] REGS: c000001d777c3850 TRAP: 0700 Not tainted (4.10.0-rc8-00038-g528b408-dirty)
[ 3177.528984] MSR: 9000000000029033 <SF,HV,EE,ME,IR,DR,RI,LE>
[ 3177.528994] CR: 22082442 XER: 00000000
[ 3177.528995] CFAR: c0000000004331dc SOFTE: 1
GPR00: c00000000027f460 c000001d777c3ad0 c000000000fb9c00 f0000000063ac880
GPR04: 0000000000000010 00000000c0000018 0000000000000000 f0000000073dbf20
GPR08: c000000000f39c00 0000000000000001 c000000000f39c00 0000000000000018
GPR12: c000000000433190 c00000000fb9c080 c0000018eb220386 0008000000000000
GPR16: f0000000063ac880 c000001e44c76f90 c000001e180c6ca0 00007fffa20f0000
GPR20: c000001bfc672d48 0000000000000000 c000001d777c3b40 00007fffa20e0000
GPR24: 0000000000000000 c1ffffffffffe7ff ffffffffffffffff 860322eb180000c0
GPR28: c000001e180c6900 c000001d777c3cc0 00007fffa20f0000 f0000000063ac880
[ 3177.529032] NIP [c00000000043322c] ext4_set_page_dirty+0x9c/0xe0
[ 3177.529035] LR [c00000000027f460] set_page_dirty+0xb0/0x190
[ 3177.529036] Call Trace:
[ 3177.529039] [c000001d777c3ad0] [00007fffa20f0000] 0x7fffa20f0000 (unreliable)
[ 3177.529043] [c000001d777c3af0] [c00000000027f460] set_page_dirty+0xb0/0x190
[ 3177.529047] [c000001d777c3b20] [c0000000002c0abc] unmap_page_range+0xf1c/0x1040
[ 3177.529050] [c000001d777c3c50] [c0000000002c10f4] unmap_vmas+0x84/0x120
[ 3177.529053] [c000001d777c3ca0] [c0000000002cbe80] unmap_region+0xd0/0x1a0
[ 3177.529057] [c000001d777c3d80] [c0000000002ce7cc] do_munmap+0x2dc/0x4a0
[ 3177.529061] [c000001d777c3df0] [c0000000002cea94] SyS_munmap+0x64/0xb0
[ 3177.529065] [c000001d777c3e30] [c00000000000b96c] system_call+0x38/0xfc
[ 3177.529066] Instruction dump:
[ 3177.529068] 7c0803a6 4e800020 60000000 60000000 60420000 3d42fff8 892a6416 2f890000
[ 3177.529076] 409effc4 39200001 3d02fff8 99286416 <0fe00000> 4bffffb0 60000000 60000000
[ 3177.529083] ---[ end trace 50350faad3b7b385 ]---Signature · 8 lines
Jan Kara <[email protected]> SUSE Labs, CR -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 13 lines · Jan Kara
Hi,
Quoted 3 lines · Aneesh Kumar K.V
I am hitting this while running stress test with the saved write patch series. I guess we are missing a set page dirty some where. I will continue to debug this, but if you have any suggestion let me know.
<snip>
So this warning can happen when page got dirtied but ->page_mkwrite() was not called. I don't know details of how autonuma works but a quick look suggests that autonuma can also do numa hinting faults for file pages. So the following seems to be possible:
Quoted 7 lines
Autonuma decides to check for accesses to a mapped shared file page that is dirty.
pte_presentgets cleared,pte_writestays set (due to logic introduced in commitb191f9b106"mm: numa: preserve PTE write permissions across a NUMA hinting fault"). Then page writeback happens,page_mkclean()is called to write-protect the page. Howeverpage_check_address()returns NULL for the PTE (__page_check_address()returns NULL for !pte_present PTEs) so we don't clearpte_writebit inpage_mkclean_one().
Even though we cleared _PAGE_PRESENT a pte_present() check return true for numa fault pte. The problem with savedwrite patch series that i quoted in the original mail was that pte_write() was checking on _PAGE_WRITE where as numa fault stashed the write bit as savedwrite bit. Hence page_mkclean was skipping those ptes.
Quoted 6 lines
Sometime later
a process looks at the page through mmap, takes NUMA fault anddo_numa_page()reestablishes a writeable mapping of the page although the filesystem does not expect there to be one and funny things happen afterwards...
Quoted 6 lines
I'll defer to more mm-savvy people to decide how this should be fixed. My naive understanding is that
page_mkclean_one()should clear thepte_writebit even for pages that are undergoing NUMA probation but I'm not sure about a preferred way to achieve that...
Yes found that and finally decided that instead of fixing all those code path, we can update pte_write to handle autonuma preserved write bit.
https://lkml.kernel.org/r/1488203787-17849-2-git-send-email-aneesh.kumar@linux.vnet.ibm.com
-aneesh
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>