bad rss-counter message in 3.14rc5

I just saw this on my box that's been running trinity..

[48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted)

There's nothing else, no trace, nothing. Any ideas where to begin with this?

Quoted 5 lines · Dave Jones

I just saw this on my box that's been running trinity..

[48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted)

There's nothing else, no trace, nothing. Any ideas where to begin with this?

ah, on the serial console there was also this truncated warning..

[48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted)
[48924.133273] ------------[ cut here ]------------
[48924.133391] kernel BUG at include/linux/swapops.h:131!

Dave

124 static inline struct page *migration_entry_to_page(swp_entry_t entry) 125 {
126 struct page *p = pfn_to_page(swp_offset(entry));
127 /*

128          * Any use of migration entries may only occur while the
129          * corresponding page is locked
130          */

131 BUG_ON(!PageLocked(p));
132 return p;
133 }

Quoted 19 lines · Dave Jones

Quoted 5 lines · Dave Jones

I just saw this on my box that's been running trinity..

[48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted)

There's nothing else, no trace, nothing. Any ideas where to begin with this?

ah, on the serial console there was also this truncated warning..

[48825.517189] BUG: Bad rss-counter state mm:ffff880177921d40 idx:0 val:1 (Not tainted)
[48924.133273] ------------[ cut here ]------------
[48924.133391] kernel BUG at include/linux/swapops.h:131!

Dave

124 static inline struct page *migration_entry_to_page(swp_entry_t entry) 125 {
126 struct page *p = pfn_to_page(swp_offset(entry));
127 /*

Quoted 6 lines

128          * Any use of migration entries may only occur while the
129          * corresponding page is locked
130          */

131 BUG_ON(!PageLocked(p));
132 return p;
133 }

I hit this again, this time a full trace made it over the serial console. This time there was no bad rss-counter message though.

kernel BUG at include/linux/swapops.h:131! invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC

Modules linked in: snd_seq_dummy fuse hidp tun bnep rfcomm llc2 af_key ipt_ULOG can_raw nfnetlink scsi_transport_iscsi nfc caif_socket caif af_802154 phonet af_rxrpc can_bcm can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 cfg80211 xfs libcrc32c coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode pcspkr serio_raw btusb bluetooth 6lowpan_iphc rfkill usb_debug shpchp snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp snd_timer snd pps_core soundcore
CPU: 2 PID: 10002 Comm: trinity-c36 Not tainted 3.14.0-rc5+ #131 
task: ffff880108966750 ti: ffff88018911a000 task.ti: ffff88018911a000
RIP: 0010:[<ffffffff9d72d129>]  [<ffffffff9d72d129>] migration_entry_to_page.part.47+0x4/0x6
RSP: 0000:ffff88018911bae8  EFLAGS: 00010246
RAX: ffffea00048a8980 RBX: ffff8801a08ae020 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 3c00000000122a26
RBP: ffff88018911bae8 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: fffffffffffffffe R12: 0000000024544c3c
R13: ffff88018911bc18 R14: 0000000040c00000 R15: 0000000040a04000
FS:  0000000000000000(0000) GS:ffff88024d080000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000001 CR3: 00000001e3c27000 CR4: 00000000001407e0
DR0: 0000000000ab5000 DR1: 0000000001008000 DR2: 0000000002230000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
Stack:
 ffff88018911bbc8 ffffffff9d17ec1e 0000000040d65fff 0000000040d65fff
 ffff8801e3c27000 0000000040d66000 ffff88011c72e008 0000000040d66000
 0000000040d65fff 0000000000000001 0000000040d66000 ffff88018911bb98
Call Trace:
 [<ffffffff9d17ec1e>] unmap_single_vma+0x89e/0x8a0
 [<ffffffff9d17fd49>] unmap_vmas+0x49/0x90
 [<ffffffff9d1890f5>] exit_mmap+0xe5/0x1a0
 [<ffffffff9d068d13>] mmput+0x73/0x110
 [<ffffffff9d06d022>] do_exit+0x2a2/0xb50
 [<ffffffff9d07bb63>] ? __sigqueue_free.part.11+0x33/0x40
 [<ffffffff9d07c39c>] ? __dequeue_signal+0x13c/0x220
 [<ffffffff9d06e8cc>] do_group_exit+0x4c/0xc0
 [<ffffffff9d07fd41>] get_signal_to_deliver+0x2d1/0x6d0
 [<ffffffff9d0024c7>] do_signal+0x57/0x9d0

 [<ffffffff9d11003e>] ? __acct_update_integrals+0x8e/0x120
 [<ffffffff9d73d66b>] ? preempt_count_sub+0x6b/0xf0
 [<ffffffff9d738ec1>] ? _raw_spin_unlock+0x31/0x50
 [<ffffffff9d0aa0b1>] ? vtime_account_user+0x91/0xa0
 [<ffffffff9d15215b>] ? context_tracking_user_exit+0x9b/0x100
 [<ffffffff9d002eb1>] do_notify_resume+0x71/0xc0
 [<ffffffff9d739c06>] retint_signal+0x46/0x90
Code: df 48 c1 ff 06 49 01 fc 4c 89 e7 e8 79 ff ff ff 85 c0 74 0c 4c 89 e0 48 c1 e0 06 48 29 d8 eb 02 31 c0 5b 41 5c 5d c3 55 48 89 e5 <0f> 0b 55 48 89 e5 0f 0b 55 48 89 e5 0f 0b 55 31 f6 48 89 e5 e8

...

Quoted 46 lines

Quoted 10 lines

124 static inline struct page *migration_entry_to_page(swp_entry_t entry) 125 {
126 struct page *p = pfn_to_page(swp_offset(entry));
127 /*

128          * Any use of migration entries may only occur while the
129          * corresponding page is locked
130          */

131 BUG_ON(!PageLocked(p));
132 return p;
133 }

I hit this again, this time a full trace made it over the serial console. This time there was no bad rss-counter message though.

kernel BUG at include/linux/swapops.h:131! invalid opcode: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC

Modules linked in: snd_seq_dummy fuse hidp tun bnep rfcomm llc2 af_key ipt_ULOG can_raw nfnetlink scsi_transport_iscsi nfc caif_socket caif af_802154 phonet af_rxrpc can_bcm can pppoe pppox ppp_generic slhc irda crc_ccitt rds rose x25 atm netrom appletalk ipx p8023 psnap p8022 llc ax25 cfg80211 xfs libcrc32c coretemp hwmon x86_pkg_temp_thermal kvm_intel kvm crct10dif_pclmul crc32c_intel ghash_clmulni_intel snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic microcode pcspkr serio_raw btusb bluetooth 6lowpan_iphc rfkill usb_debug shpchp snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm e1000e ptp snd_timer snd pps_core soundcore
CPU: 2 PID: 10002 Comm: trinity-c36 Not tainted 3.14.0-rc5+ #131 
task: ffff880108966750 ti: ffff88018911a000 task.ti: ffff88018911a000
RIP: 0010:[<ffffffff9d72d129>]  [<ffffffff9d72d129>] migration_entry_to_page.part.47+0x4/0x6
RSP: 0000:ffff88018911bae8  EFLAGS: 00010246
RAX: ffffea00048a8980 RBX: ffff8801a08ae020 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 3c00000000122a26
RBP: ffff88018911bae8 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: fffffffffffffffe R12: 0000000024544c3c
R13: ffff88018911bc18 R14: 0000000040c00000 R15: 0000000040a04000
FS:  0000000000000000(0000) GS:ffff88024d080000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000001 CR3: 00000001e3c27000 CR4: 00000000001407e0
DR0: 0000000000ab5000 DR1: 0000000001008000 DR2: 0000000002230000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
Stack:
 ffff88018911bbc8 ffffffff9d17ec1e 0000000040d65fff 0000000040d65fff
 ffff8801e3c27000 0000000040d66000 ffff88011c72e008 0000000040d66000
 0000000040d65fff 0000000000000001 0000000040d66000 ffff88018911bb98
Call Trace:
 [<ffffffff9d17ec1e>] unmap_single_vma+0x89e/0x8a0
 [<ffffffff9d17fd49>] unmap_vmas+0x49/0x90
 [<ffffffff9d1890f5>] exit_mmap+0xe5/0x1a0
 [<ffffffff9d068d13>] mmput+0x73/0x110
 [<ffffffff9d06d022>] do_exit+0x2a2/0xb50
 [<ffffffff9d07bb63>] ? __sigqueue_free.part.11+0x33/0x40
 [<ffffffff9d07c39c>] ? __dequeue_signal+0x13c/0x220
 [<ffffffff9d06e8cc>] do_group_exit+0x4c/0xc0
 [<ffffffff9d07fd41>] get_signal_to_deliver+0x2d1/0x6d0
 [<ffffffff9d0024c7>] do_signal+0x57/0x9d0

Quoted 8 lines

 [<ffffffff9d11003e>] ? __acct_update_integrals+0x8e/0x120
 [<ffffffff9d73d66b>] ? preempt_count_sub+0x6b/0xf0
 [<ffffffff9d738ec1>] ? _raw_spin_unlock+0x31/0x50
 [<ffffffff9d0aa0b1>] ? vtime_account_user+0x91/0xa0
 [<ffffffff9d15215b>] ? context_tracking_user_exit+0x9b/0x100
 [<ffffffff9d002eb1>] do_notify_resume+0x71/0xc0
 [<ffffffff9d739c06>] retint_signal+0x46/0x90
Code: df 48 c1 ff 06 49 01 fc 4c 89 e7 e8 79 ff ff ff 85 c0 74 0c 4c 89 e0 48 c1 e0 06 48 29 d8 eb 02 31 c0 5b 41 5c 5d c3 55 48 89 e5 <0f> 0b 55 48 89 e5 0f 0b 55 48 89 e5 0f 0b 55 31 f6 48 89 e5 e8

Anyone ? I'm hitting this trace on an almost daily basis, which is a pain while trying to reproduce a different bug..

Quoted 2 lines

Anyone ? I'm hitting this trace on an almost daily basis, which is a pain while trying to reproduce a different bug..

Damn, I thought we'd fixed that but it seems not. Cc's added.

Guys, what stops the migration target page from coming unlocked in parallel with zap_pte_range()'s call to migration_entry_to_page()?

Quoted 5 lines · On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton <[email protected]>…

Quoted 2 lines

Anyone ? I'm hitting this trace on an almost daily basis, which is a pain while trying to reproduce a different bug..

Damn, I thought we'd fixed that but it seems not. Cc's added.

Quoted 2 lines

Guys, what stops the migration target page from coming unlocked in parallel with zap_pte_range()'s call to migration_entry_to_page()?

page_table_lock, sort-of. At least, transitions of is_migration_entry() and page_locked() happen under ptl.

I don't see any holes in regular migration. Do you know if this is reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?

Quoted 5 lines · On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton <[email protected]>…

Quoted 2 lines

Anyone ? I'm hitting this trace on an almost daily basis, which is a pain while trying to reproduce a different bug..

Damn, I thought we'd fixed that but it seems not. Cc's added.

Quoted 2 lines

Guys, what stops the migration target page from coming unlocked in parallel with zap_pte_range()'s call to migration_entry_to_page()?

page_table_lock, sort-of. At least, transitions of is_migration_entry() and page_locked() happen under ptl.

I don't see any holes in regular migration. Do you know if this is reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?

Quoted 10 lines · Andrew Morton

Quoted 7 lines · On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton <[email protected]>…

Quoted 2 lines

Anyone ? I'm hitting this trace on an almost daily basis, which is a pain while trying to reproduce a different bug..

Damn, I thought we'd fixed that but it seems not. Cc's added.

Guys, what stops the migration target page from coming unlocked in parallel with zap_pte_range()'s call to migration_entry_to_page()?

Quoted 5 lines

page_table_lock, sort-of. At least, transitions of is_migration_entry() and page_locked() happen under ptl.

I don't see any holes in regular migration. Do you know if this is reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?

CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run.

Quoted 14 lines · On Tue, 11 Mar 2014 00:51:09 -0400 Dave Jones <[email protected]> wrote:

Quoted 13 lines · Andrew Morton

Quoted 7 lines · On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton <[email protected]>…

Quoted 2 lines

Anyone ? I'm hitting this trace on an almost daily basis, which is a pain while trying to reproduce a different bug..

Damn, I thought we'd fixed that but it seems not. Cc's added.

Guys, what stops the migration target page from coming unlocked in parallel with zap_pte_range()'s call to migration_entry_to_page()?

page_table_lock, sort-of. At least, transitions of is_migration_entry() and page_locked() happen under ptl.

Quoted 4 lines

Quoted 2 lines

I don't see any holes in regular migration. Do you know if this is reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?

CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run.

There probably isn't much point unless trinity is using sys_move_pages(). Is it?

If so it would be interesting to disable
trinity's move_pages calls and see if it still fails.

Grasping at straws here, trying to reduce the amount of code to look at :(

Quoted 14 lines · On Tue, 11 Mar 2014 00:51:09 -0400 Dave Jones <[email protected]> wrote:

Quoted 13 lines · Andrew Morton

Quoted 7 lines · On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton <[email protected]>…

Quoted 2 lines

Anyone ? I'm hitting this trace on an almost daily basis, which is a pain while trying to reproduce a different bug..

Damn, I thought we'd fixed that but it seems not. Cc's added.

Guys, what stops the migration target page from coming unlocked in parallel with zap_pte_range()'s call to migration_entry_to_page()?

page_table_lock, sort-of. At least, transitions of is_migration_entry() and page_locked() happen under ptl.

Quoted 4 lines

Quoted 2 lines

I don't see any holes in regular migration. Do you know if this is reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?

CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run.

There probably isn't much point unless trinity is using sys_move_pages(). Is it? If so it would be interesting to disable trinity's move_pages calls and see if it still fails.

Quoted 19 lines · Andrew Morton

Quoted 17 lines · On Tue, 11 Mar 2014 00:51:09 -0400 Dave Jones <[email protected]> wrote:

Quoted 15 lines · Andrew Morton

Quoted 7 lines · On Mon, 10 Mar 2014 20:13:40 -0700 Andrew Morton <[email protected]>…

Quoted 2 lines

Anyone ? I'm hitting this trace on an almost daily basis, which is a pain while trying to reproduce a different bug..

Damn, I thought we'd fixed that but it seems not. Cc's added.

Guys, what stops the migration target page from coming unlocked in parallel with zap_pte_range()'s call to migration_entry_to_page()?

page_table_lock, sort-of. At least, transitions of is_migration_entry() and page_locked() happen under ptl.

I don't see any holes in regular migration. Do you know if this is reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?

Quoted 5 lines

Quoted 1 line

CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run.

There probably isn't much point unless trinity is using sys_move_pages(). Is it? If so it would be interesting to disable trinity's move_pages calls and see if it still fails.

Ok, with move_pages excluded it still oopses.

Quoted 9 lines

Quoted 5 lines

Quoted 3 lines

I don't see any holes in regular migration. Do you know if this is reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?

CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run.

There probably isn't much point unless trinity is using sys_move_pages(). Is it? If so it would be interesting to disable trinity's move_pages calls and see if it still fails.

Ok, with move_pages excluded it still oopses.

Quoted 6 lines · Dave Jones

Quoted 6 lines

Quoted 5 lines

Quoted 3 lines

I don't see any holes in regular migration. Do you know if this is reproducible with CONFIG_NUMA_BALANCING=n or CONFIG_NUMA=n?

CONFIG_NUMA_BALANCING was n already btw, so I'll do a NUMA=n run.

Quoted 5 lines

Quoted 3 lines

There probably isn't much point unless trinity is using sys_move_pages(). Is it? If so it would be interesting to disable trinity's move_pages calls and see if it still fails.

Ok, with move_pages excluded it still oopses.

Dave, is it possible to somehow figure out was someone reading pagemap file at moment of the bug triggering?

Quoted 5 lines

Quoted 2 lines

Ok, with move_pages excluded it still oopses.

Dave, is it possible to somehow figure out was someone reading pagemap file at moment of the bug triggering?

We can sprinkle printk()s wherever might be useful, might not be 100% accurate but should be close enough to confirm/deny the theory.

Quoted 3 lines · Sasha Levin

Quoted 3 lines

Quoted 2 lines

Ok, with move_pages excluded it still oopses.

Quoted 5 lines

Quoted 2 lines

Dave, is it possible to somehow figure out was someone reading pagemap file at moment of the bug triggering?

We can sprinkle printk()s wherever might be useful, might not be 100% accurate but should be close enough to confirm/deny the theory.

After reading some more, I suppose the idea I had is wrong, investigating. Will ping if I find something.

Quoted 7 lines · Cyrill Gorcunov

Quoted 6 lines · Sasha Levin

Quoted 5 lines

Quoted 2 lines

Ok, with move_pages excluded it still oopses.

Dave, is it possible to somehow figure out was someone reading pagemap file at moment of the bug triggering?

Quoted 5 lines

Quoted 2 lines

We can sprinkle printk()s wherever might be useful, might not be 100% accurate but should be close enough to confirm/deny the theory.

After reading some more, I suppose the idea I had is wrong, investigating. Will ping if I find something.

I can rule it out anyway, I can reproduce this by telling trinity to do nothing
other than mmap()'s. I'll try and narrow down the exact parameters.

Quoted 11 lines · Dave Jones

Quoted 10 lines · Cyrill Gorcunov

Quoted 8 lines · Sasha Levin

Quoted 5 lines

Quoted 2 lines

Ok, with move_pages excluded it still oopses.

Dave, is it possible to somehow figure out was someone reading pagemap file at moment of the bug triggering?

We can sprinkle printk()s wherever might be useful, might not be 100% accurate but should be close enough to confirm/deny the theory.

Quoted 5 lines

Quoted 2 lines

After reading some more, I suppose the idea I had is wrong, investigating. Will ping if I find something.

I can rule it out anyway, I can reproduce this by telling trinity to do nothing
other than mmap()'s. I'll try and narrow down the exact parameters.

Dave, iirc trinity can write log file pointing which exactly syscall sequence was passed, right? Share it too please.

Quoted 5 lines

Quoted 2 lines

After reading some more, I suppose the idea I had is wrong, investigating. Will ping if I find something.

I can rule it out anyway, I can reproduce this by telling trinity to do nothing
other than mmap()'s. I'll try and narrow down the exact parameters.

Dave, iirc trinity can write log file pointing which exactly syscall sequence was passed, right? Share it too please.

Quoted 3 lines

Dave, iirc trinity can write log file pointing which exactly syscall sequence was passed, right? Share it too please.

Hm, I may have been mistaken, and the damage was done by a previous run. I went from being able to reproduce it almost instantly to now not being able to reproduce it at all. Will keep trying.

Quoted 7 lines · Dave Jones

Quoted 3 lines

Dave, iirc trinity can write log file pointing which exactly syscall sequence was passed, right? Share it too please.

Hm, I may have been mistaken, and the damage was done by a previous run. I went from being able to reproduce it almost instantly to now not being able to reproduce it at all. Will keep trying.

Sasha already gave a link to the syscalls sequence, so no rush.

Quoted 2 lines

Sasha already gave a link to the syscalls sequence, so no rush.

It'd be nice to get a more concise reproducer, his list had a little of everything in there.

Quoted 5 lines · Cyrill Gorcunov

Quoted 4 lines · Dave Jones

Quoted 3 lines

Dave, iirc trinity can write log file pointing which exactly syscall sequence was passed, right? Share it too please.

Quoted 5 lines

Quoted 3 lines

Hm, I may have been mistaken, and the damage was done by a previous run. I went from being able to reproduce it almost instantly to now not being able to reproduce it at all. Will keep trying.

Sasha already gave a link to the syscalls sequence, so no rush.

It'd be nice to get a more concise reproducer, his list had a little of everything in there.

And yes, it is possible (though very unusual) to find an anon page or swap entry in a VM_SHARED nonlinear mapping: coming from that horrid get_user_pages(write, force) case which COWs even in a shared mapping.

Quoted 4 lines · Hugh Dickins

And yes, it is possible (though very unusual) to find an anon page or swap entry in a VM_SHARED nonlinear mapping: coming from that horrid get_user_pages(write, force) case which COWs even in a shared mapping.

Hmm. Maybe we could just disallow that forced case.

It *used* to be a trivial "we can just do a COW", but that was back when the VM was much simpler and we had no rmap's etc. So "that horrid case" used to be a simple hack that wasn't painful. But I suspect we could very easily just fail it instead of forcing a COW, if that would make it simpler for the VM code.

I'd love that, if we can get away with it now: depends very much on whether we then turn out to break userspace or not.

If I remember correctly, it's been that way since early days, in case ptrace were used to put a breakpoint into a MAP_SHARED mapping of an executable: to prevent that modification from reaching the file, if the file happened to be opened O_RDWR. Usually it's not open for writing, and mapped MAP_PRIVATE anyway.

Quoted 3 lines · Hugh Dickins

I'd love that, if we can get away with it now: depends very much on whether we then turn out to break userspace or not.

Right. I suspect we can, though, but it's one of those "we can try it and see". Remind me early in the 3.15 merge window, and we can just turn the "force" case into an error case and see if anybody hollers.

Quoted 6 lines

If I remember correctly, it's been that way since early days, in case ptrace were used to put a breakpoint into a MAP_SHARED mapping of an executable: to prevent that modification from reaching the file, if the file happened to be opened O_RDWR. Usually it's not open for writing, and mapped MAP_PRIVATE anyway.

Yes, it's been that way since the very beginning, I think it goes back pretty much as far as MAP_SHARED does.

We used to play lots of games wrt MAP_SHARED - in fact I think we used to silently turn a MAP_SHARED RO mapping into MAP_PRIVATE because for the longest time there was no "true" writable MAP_SHARED at all, but we did have a coherent MAP_PRIVATE and something like the indexer for nntpd wanted a read-only shared mapping of the nntp spool or something like that. I forget the details, it's a _loong_ time ago.

So the whole "force turns a MAP_SHARED page into MAP_PRIVATE" all used to make a lot more sense in that kind of situation, when MAP_SHARED vs MAP_PRIVATE was much less of a black-and-white thing.

I really suspect nobody cares wrt ptrace, especially since presumably other systems haven't had those kinds of games (although who knows - HP-UX in particular had some of the shittiest mmap() implementations on the planet - it made even the original Linux mmap hacks look like a thing of pure beauty in comparison).

For 3.15, and probably 3.16 too, we should keep in place whatever partial accommodations we have for the case (such as allowing for anon and swap in fremap's zap_pte), in case we do need to revert; but clean those away later on. (Not many, I think: it was mainly a guilty secret that VM accounting didn't really hold together.)

:) That fits with what I heard of HP-UX mmap,
but I never had the pleasure of dealing with it.

Quoted 8 lines · Linus Torvalds

Quoted 3 lines · Hugh Dickins

I'd love that, if we can get away with it now: depends very much on whether we then turn out to break userspace or not.

Right. I suspect we can, though, but it's one of those "we can try it and see". Remind me early in the 3.15 merge window, and we can just turn the "force" case into an error case and see if anybody hollers.

Super, I'll do that, thanks.

For 3.15, and probably 3.16 too, we should keep in place whatever partial accommodations we have for the case (such as allowing for anon and swap in fremap's zap_pte), in case we do need to revert; but clean those away later on. (Not many, I think: it was mainly a guilty secret that VM accounting didn't really hold together.)

Quoted 6 lines · Dave Jones

Quoted 5 lines · Cyrill Gorcunov

Quoted 4 lines · Dave Jones

Quoted 3 lines

Dave, iirc trinity can write log file pointing which exactly syscall sequence was passed, right? Share it too please.

Quoted 7 lines

Quoted 5 lines

Quoted 3 lines

Hm, I may have been mistaken, and the damage was done by a previous run. I went from being able to reproduce it almost instantly to now not being able to reproduce it at all. Will keep trying.

Sasha already gave a link to the syscalls sequence, so no rush.

It'd be nice to get a more concise reproducer, his list had a little of everything in there.

I've so far failed to find any explanation for your swapops.h BUG; but believe I have identified one cause for "Bad rss-counter"s.

My hunch is that the swapops.h BUG is "nearby", but I just cannot fit it together (the swapops.h BUG comes when rmap cannot find all all the migration entries it inserted earlier: it's a very useful BUG for validating rmap).

Untested patch below: I can't quite say Reported-by, because it may not even be one that you and Sasha have been seeing; but I'm hopeful, remap_file_pages is in the list.

Please give this a try, preferably on 3.14-rc or earlier: I've never seen "Bad rss-counter"s there myself (trinity uses remap_file_pages a lot more than most of us); but have seen them on mmotm/next, so some other trigger is coming up there, I'll worry about that once it reaches 3.15-rc.

Quoted 4 lines

Untested patch below: I can't quite say Reported-by, because it may not even be one that you and Sasha have been seeing; but I'm hopeful, remap_file_pages is in the list.

Quoted 5 lines

Please give this a try, preferably on 3.14-rc or earlier: I've never seen "Bad rss-counter"s there myself (trinity uses remap_file_pages a lot more than most of us); but have seen them on mmotm/next, so some other trigger is coming up there, I'll worry about that once it reaches 3.15-rc.

The patch fixed the "Bad rss-counter" errors I've been seeing both in 3.14-rc7 and -next.

Quoted 10 lines · Sasha Levin

Quoted 9 lines

Untested patch below: I can't quite say Reported-by, because it may not even be one that you and Sasha have been seeing; but I'm hopeful, remap_file_pages is in the list.

Please give this a try, preferably on 3.14-rc or earlier: I've never seen "Bad rss-counter"s there myself (trinity uses remap_file_pages a lot more than most of us); but have seen them on mmotm/next, so some other trigger is coming up there, I'll worry about that once it reaches 3.15-rc.

Quoted 2 lines

The patch fixed the "Bad rss-counter" errors I've been seeing both in 3.14-rc7 and -next.

It's looking good here too so far. I'll leave it running overnight to be sure.

Quoted 12 lines · Dave Jones

Quoted 10 lines · Sasha Levin

Quoted 9 lines

Untested patch below: I can't quite say Reported-by, because it may not even be one that you and Sasha have been seeing; but I'm hopeful, remap_file_pages is in the list.

Please give this a try, preferably on 3.14-rc or earlier: I've never seen "Bad rss-counter"s there myself (trinity uses remap_file_pages a lot more than most of us); but have seen them on mmotm/next, so some other trigger is coming up there, I'll worry about that once it reaches 3.15-rc.

Quoted 4 lines

Quoted 2 lines

The patch fixed the "Bad rss-counter" errors I've been seeing both in 3.14-rc7 and -next.

It's looking good here too so far. I'll leave it running overnight to be sure.

Of course, that isn't going to happen. Immediately after posting this, I hit the swapops bug. Patch does seem to have cured the bad rss counters though.

Another positive on the rss counters, great, thanks Dave. That encourages me to think again on the swapops BUG, but no promises.

Quoted 2 lines · Hugh Dickins

Another positive on the rss counters, great, thanks Dave. That encourages me to think again on the swapops BUG, but no promises.

So while I slept I ran a test kernel with that swapops BUG replaced with a printk. I'm not sure of the validity of this, given the state of the kernel afterwards is somewhat suspect, but I did see in the logs this morning..

[18728.075153] migration_entry_to_page BUG hit
[18728.200705] BUG: Bad rss-counter state mm:ffff880241b3f500 idx:0 val:1 (Not tainted)
[18728.200706] BUG: Bad rss-counter state mm:ffff880241b3f500 idx:1 val:-1 (Not tainted)

This might be collateral damage from the swapops thing, I guess we won't know until that gets fixed, but I thought I'd mention that we might still have a problem here.

Quoted 2 lines

This might be collateral damage from the swapops thing, I guess we won't know until that gets fixed, but I thought I'd mention that we might still have a problem here.

Yes, those Bad rss-counters could well be collateral damage from the swapops BUG. To which I believe I now have the answer: again untested, but please give this a try...

(It's worth saying, by the way, that these bugs are not a consequence of recent changes at all, they've been there for ages; but trinity has just got better at taunting remap_file_pages and the rest of mm...)

Quoted 6 lines · Hugh Dickins

Quoted 2 lines

This might be collateral damage from the swapops thing, I guess we won't know until that gets fixed, but I thought I'd mention that we might still have a problem here.

Yes, those Bad rss-counters could well be collateral damage from the swapops BUG. To which I believe I now have the answer: again untested, but please give this a try...

This survived an overnight run. No swapops bug, and no bad RSS. Good job:)

Quoted 7 lines · Dave Jones

Quoted 1 line · Hugh Dickins

Quoted 1 line

This might be collateral damage from the swapops thing, I guess we

won't know until

Quoted 1 line

Quoted 1 line

that gets fixed, but I thought I'd mention that we might still have a

problem here.

Quoted 1 line

Quoted 5 lines

Quoted 3 lines

Yes, those Bad rss-counters could well be collateral damage from the swapops BUG. To which I believe I now have the answer: again untested, but please give this a try...

This survived an overnight run. No swapops bug, and no bad RSS. Good job:)

Same here, swapops bug is gone!

Quoted 31 lines · Hugh Dickins

Quoted 9 lines · Dave Jones

Quoted 3 lines · Cyrill Gorcunov

Quoted 2 lines · Dave Jones

Quoted 2 lines

Dave, iirc trinity can write log file pointing which exactly

syscall sequence

Quoted 3 lines

Quoted 3 lines

Quoted 1 line

was passed, right? Share it too please.

Hm, I may have been mistaken, and the damage was done by a previous

run.

Quoted 8 lines

Quoted 1 line

Quoted 1 line

I went from being able to reproduce it almost instantly to now not

being able

Quoted 3 lines

Quoted 1 line

to reproduce it at all. Will keep trying.

Sasha already gave a link to the syscalls sequence, so no rush.

It'd be nice to get a more concise reproducer, his list had a little of everything in there.

I've so far failed to find any explanation for your swapops.h BUG; but believe I have identified one cause for "Bad rss-counter"s.

My hunch is that the swapops.h BUG is "nearby", but I just cannot fit it together (the swapops.h BUG comes when rmap cannot find all all the migration entries it inserted earlier: it's a very useful BUG for validating rmap).

Untested patch below: I can't quite say Reported-by, because it may not even be one that you and Sasha have been seeing; but I'm hopeful, remap_file_pages is in the list.

Quoted 5 lines

Please give this a try, preferably on 3.14-rc or earlier: I've never seen "Bad rss-counter"s there myself (trinity uses remap_file_pages a lot more than most of us); but have seen them on mmotm/next, so some other trigger is coming up there, I'll worry about that once it reaches 3.15-rc.

The patch fixed the "Bad rss-counter" errors I've been seeing both in 3.14-rc7 and -next.

Quoted 38 lines · Sasha Levin

Quoted 36 lines · Hugh Dickins

Quoted 9 lines · Dave Jones

Quoted 3 lines · Cyrill Gorcunov

Quoted 2 lines · Dave Jones

Quoted 2 lines

Dave, iirc trinity can write log file pointing which exactly

syscall sequence

Quoted 3 lines

Quoted 3 lines

Quoted 1 line

was passed, right? Share it too please.

Hm, I may have been mistaken, and the damage was done by a previous

run.

Quoted 8 lines

Quoted 1 line

Quoted 1 line

I went from being able to reproduce it almost instantly to now not

being able

Quoted 3 lines

Quoted 1 line

to reproduce it at all. Will keep trying.

Sasha already gave a link to the syscalls sequence, so no rush.

It'd be nice to get a more concise reproducer, his list had a little of everything in there.

I've so far failed to find any explanation for your swapops.h BUG; but believe I have identified one cause for "Bad rss-counter"s.

My hunch is that the swapops.h BUG is "nearby", but I just cannot fit it together (the swapops.h BUG comes when rmap cannot find all all the migration entries it inserted earlier: it's a very useful BUG for validating rmap).

Untested patch below: I can't quite say Reported-by, because it may not even be one that you and Sasha have been seeing; but I'm hopeful, remap_file_pages is in the list.

Please give this a try, preferably on 3.14-rc or earlier: I've never seen "Bad rss-counter"s there myself (trinity uses remap_file_pages a lot more than most of us); but have seen them on mmotm/next, so some other trigger is coming up there, I'll worry about that once it reaches 3.15-rc.

Quoted 2 lines

The patch fixed the "Bad rss-counter" errors I've been seeing both in 3.14-rc7 and -next.

Great, thanks a lot, Sasha. I was afraid that you'd hit those swapops BUGs, which seemed perhaps to be paired with these; but glad to hear a positive. Let's see how Dave fares. (I've not forgotten shmem fallocate, by the way, but those probably aren't as high on my agenda as you'd like.)

(Cyrill, entirely unrelated, but in preparing this patch I noticed your soft_dirty work in install_file_pte(): which looked good at first, until I realized that it's propagating the soft_dirty of a pte it's about to zap completely, to the unrelated entry it's about to insert in its place. Which seems very odd to me.)