unify pagetable accessors patch causes double fault II

Subject was wrong of course -- it was a recursive oops, not a double fault. Sorry for the inaccuracy.

Hopefully it can be fixed soon because it inhibits further testing here.

Quoted 5 lines · Andi Kleen

Subject was wrong of course -- it was a recursive oops, not a double fault. Sorry for the inaccuracy.

Hopefully it can be fixed soon because it inhibits further testing here.

no context. Your first mail did not seem to make it to lkml. (yet)

Quoted 9 lines · Ingo Molnar

Quoted 4 lines · Andi Kleen

Subject was wrong of course -- it was a recursive oops, not a double fault. Sorry for the inaccuracy.

Hopefully it can be fixed soon because it inhibits further testing here.

no context. Your first mail did not seem to make it to lkml. (yet)

Sorry. Not sure what happened. Here is it again:

----

This is as of 2f42671697ea9abc7d10ea7f663d6ef6e8ec6358 git-x86 HEAD:

One of my test machines here when booted with git-x86 gives a double fault on entering user space. I bisected it down to the following commit.

commit c64ba9309275f2e89bd18adbe4d932b6ecc7eb07

Author: Jeremy Fitzhardinge <[email protected]>
Date:   Fri Jan 11 18:11:41 2008 +0100

x86/pgtable: unify pagetable accessors

Unify functions to test and set bits in pagetable entries.

This is with PAE and seems to only happen with enough RAM (6GB); a 2GB system boots. 64bit also works.

-Andi

VFS: Mounted root (nfs filesystem).
Freeing unused kernel memory: 260k freed
boot[1151]: segfault at 00000000 ip 00000000 sp bfe5c1fc error 14 Bad page state in process 'boot'
page:c27f8000 flags:0x80080010 mapping:00000000 mapcount:0 count:0 Trying to fix it up, but a reboot is needed

Backtrace:
Pid: 1151, comm: boot Not tainted 2.6.24-rc7-gc64ba930 #10
 [<c01474eb>] bad_page+0x48/0x6f
 [<c01478de>] free_hot_cold_page+0x5b/0x148
 [<c01479e3>] __pagevec_free+0x18/0x22
 [<c014a03d>] release_pages+0x13f/0x147
 [<c0151191>] free_pgtables+0x86/0x93
 [<c01568bb>] free_pages_and_swap_cache+0x6a/0x7e
 [<c01521e1>] exit_mmap+0xa2/0xcd
 [<c011fece>] mmput+0x25/0x79
 [<c01245fa>] do_exit+0x1a9/0x5eb
 [<c0124aa7>] sys_exit_group+0x0/0xd
 [<c012bda2>] get_signal_to_deliver+0x3e3/0x405
 [<c043577c>] do_page_fault+0x0/0x6a4
 [<c01043de>] do_notify_resume+0x7d/0x64e
 [<c0122346>] printk+0x14/0x18
 [<c0435b3a>] do_page_fault+0x3be/0x6a4
 [<c0435e17>] do_page_fault+0x69b/0x6a4
 [<c043577c>] do_page_fault+0x0/0x6a4
 [<c0104d26>] work_notifysig+0x13/0x19
 =======================
Bad page state in process 'boot'
page:c27f8120 flags:0x80000000 mapping:00000000 mapcount:1 count:1
Trying to fix it up, but a reboot is needed
Backtrace:
Pid: 1153, comm: boot Tainted: G    B   2.6.24-rc7-gc64ba930 #10
 [<c01474eb>] bad_page+0x48/0x6f
 [<c0147f27>] get_page_from_freelist+0x242/0x30f
 [<c014807f>] __alloc_pages+0x67/0x2c5
 [<c014e9b5>] do_wp_page+0x20e/0x494
 [<c0150ac2>] handle_mm_fault+0x6c1/0x75a
 [<c0435a2b>] do_page_fault+0x2af/0x6a4
 [<c043577c>] do_page_fault+0x0/0x6a4
 [<c043455a>] error_code+0x72/0x78
 [<c02122aa>] __put_user_4+0x12/0x18
 [<c011e09c>] schedule_tail+0x52/0x55
 [<c0104b6e>] ret_from_fork+0x6/0x1c
 =======================

boot[1153]: segfault at 0574c985 ip b7d9488a sp bfe5bfd8 error 6 Eeek! page_mapcount(page) went negative! (-1)
page pfn = bfc09
page->flags = 80000060
page->count = 1
page->mapping = f702d091
vma->vm_ops = 0x0

------------[ cut here ]------------
kernel BUG at /home/lsrc/git-arch-x86/linux-2.6-x86/mm/rmap.c:631!

invalid opcode: 0000 [#1] SMP Modules linked in:

Pid: 1153, comm: boot Tainted: G B (2.6.24-rc7-gc64ba930 #10)
EIP: 0060:[<c0154bd6>] EFLAGS: 00010246 CPU: 2
EIP is at page_remove_rmap+0xcc/0xe7
EAX: 00000000 EBX: c27f8120 ECX: 00000046 EDX: 00000046
ESI: f7074948 EDI: c27f8120 EBP: f7078198 ESP: f709ddfc
DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
Process boot (pid: 1153, ti=f709c000 task=f7460000 task.ti=f709c000)

Stack: bfc09045 00000000 c014f18f 00000000 f7074948 f709de80 bfc09045 00000000 
       00000000 00000001 b7e37000 f7070010 f70468c0 c4825240 ffffffff 00000000 
       c16e0f0c 00000000 f70aadf8 003f9ed9 b7e37000 b7e37000 00000000 b7e33000 
Call Trace:
 [<c014f18f>] unmap_vmas+0x334/0x5c9
 [<c015219e>] exit_mmap+0x5f/0xcd
 [<c011fece>] mmput+0x25/0x79
 [<c01245fa>] do_exit+0x1a9/0x5eb
 [<c0124aa7>] sys_exit_group+0x0/0xd
 [<c012bda2>] get_signal_to_deliver+0x3e3/0x405
 [<c043577c>] do_page_fault+0x0/0x6a4
 [<c01043de>] do_notify_resume+0x7d/0x64e
 [<c0122346>] printk+0x14/0x18
 [<c0435b3a>] do_page_fault+0x3be/0x6a4
 [<c0435e17>] do_page_fault+0x69b/0x6a4
 [<c043577c>] do_page_fault+0x0/0x6a4
 [<c0104d26>] work_notifysig+0x13/0x19
 =======================
Code: 8b 46 44 8b 50 08 b8 e3 aa 50 c0 e8 c0 ab fe ff 8b 46 4c 85 c0 74 14 8b 40 10 85 c0 74 0d 8b 50 2c b8 01 ab 50 c0 e8 a5 ab fe ff <0f> 0b eb fe 8b 53 10 89 d8 5b 5e 83 e2 01 f7 da 83 c2 04 e9 e1 EIP: [<c0154bd6>] page_remove_rmap+0xcc/0xe7 SS:ESP 0068:f709ddfc
---[ end trace 8cd8c46e6dae67bc ]---

Fixing recursive fault but reboot is needed!
eth0: no IPv6 routers present
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Quoted 2 lines

This is with PAE and seems to only happen with enough RAM (6GB); a 2GB system boots. 64bit also works.

Thanks, I'll have a close look at it. Presumably a pfn is getting truncated to 20/32-bits somewhere.

would be nice to have some debugging apparatus for bugs like this. Perhaps artificially add a large pfn, then convert/unconvert, then subtract it and expect the whole transformation to be an identity mapping? This way we could simulate most of the effects of >4GB RAM, right?

BTW there seems to be some other instability that hits even on 64bit. Since I updated my workstation to a recent git-x86 based kernel (admittedly with my patches too) I had at least one hard hang already.

Previously it ran an older -rc5 based kernel without git-x86 also with most of my patches and was stable for >23 days until a KVM oops required a reboot.

It unfortunately was not set up to receive console at this point and it hung in X, but I'm watching it now.

hm. Did the hanging kernel have the i387 regset fix from Roland already? That bug indeed manifested itself as rare desktop hangs.

other than that i know of no pending instability but the large-RAM bootup crash you reported.

Quoted 8 lines · Andi Kleen

BTW there seems to be some other instability that hits even on 64bit. Since I updated my workstation to a recent git-x86 based kernel (admittedly with my patches too) I had at least one hard hang already.

Previously it ran an older -rc5 based kernel without git-x86 also with most of my patches and was stable for >23 days until a KVM oops required a reboot.

Quoted 2 lines

It unfortunately was not set up to receive console at this point and it hung in X, but I'm watching it now.

hm. Did the hanging kernel have the i387 regset fix from Roland already? That bug indeed manifested itself as rare desktop hangs.

would be nice to have some debugging apparatus for bugs like this. Perhaps artificially add a large pfn, then convert/unconvert, then subtract it and expect the whole transformation to be an identity mapping? This way we could simulate most of the effects of >4GB RAM, right?

Quoted 9 lines · Ingo Molnar

Quoted 4 lines · Andi Kleen

Subject was wrong of course -- it was a recursive oops, not a double fault. Sorry for the inaccuracy.

Hopefully it can be fixed soon because it inhibits further testing here.

no context. Your first mail did not seem to make it to lkml. (yet)

Sorry. Not sure what happened. Here is it again:

----

This is as of 2f42671697ea9abc7d10ea7f663d6ef6e8ec6358 git-x86 HEAD:

One of my test machines here when booted with git-x86 gives a double fault on entering user space. I bisected it down to the following commit.

commit c64ba9309275f2e89bd18adbe4d932b6ecc7eb07

Author: Jeremy Fitzhardinge <[email protected]>
Date:   Fri Jan 11 18:11:41 2008 +0100

x86/pgtable: unify pagetable accessors

Unify functions to test and set bits in pagetable entries.

This is with PAE and seems to only happen with enough RAM (6GB); a 2GB system boots. 64bit also works.

-Andi

VFS: Mounted root (nfs filesystem).
Freeing unused kernel memory: 260k freed
boot[1151]: segfault at 00000000 ip 00000000 sp bfe5c1fc error 14 Bad page state in process 'boot'
page:c27f8000 flags:0x80080010 mapping:00000000 mapcount:0 count:0 Trying to fix it up, but a reboot is needed

Backtrace:
Pid: 1151, comm: boot Not tainted 2.6.24-rc7-gc64ba930 #10
 [<c01474eb>] bad_page+0x48/0x6f
 [<c01478de>] free_hot_cold_page+0x5b/0x148
 [<c01479e3>] __pagevec_free+0x18/0x22
 [<c014a03d>] release_pages+0x13f/0x147
 [<c0151191>] free_pgtables+0x86/0x93
 [<c01568bb>] free_pages_and_swap_cache+0x6a/0x7e
 [<c01521e1>] exit_mmap+0xa2/0xcd
 [<c011fece>] mmput+0x25/0x79
 [<c01245fa>] do_exit+0x1a9/0x5eb
 [<c0124aa7>] sys_exit_group+0x0/0xd
 [<c012bda2>] get_signal_to_deliver+0x3e3/0x405
 [<c043577c>] do_page_fault+0x0/0x6a4
 [<c01043de>] do_notify_resume+0x7d/0x64e
 [<c0122346>] printk+0x14/0x18
 [<c0435b3a>] do_page_fault+0x3be/0x6a4
 [<c0435e17>] do_page_fault+0x69b/0x6a4
 [<c043577c>] do_page_fault+0x0/0x6a4
 [<c0104d26>] work_notifysig+0x13/0x19
 =======================
Bad page state in process 'boot'
page:c27f8120 flags:0x80000000 mapping:00000000 mapcount:1 count:1
Trying to fix it up, but a reboot is needed
Backtrace:
Pid: 1153, comm: boot Tainted: G    B   2.6.24-rc7-gc64ba930 #10
 [<c01474eb>] bad_page+0x48/0x6f
 [<c0147f27>] get_page_from_freelist+0x242/0x30f
 [<c014807f>] __alloc_pages+0x67/0x2c5
 [<c014e9b5>] do_wp_page+0x20e/0x494
 [<c0150ac2>] handle_mm_fault+0x6c1/0x75a
 [<c0435a2b>] do_page_fault+0x2af/0x6a4
 [<c043577c>] do_page_fault+0x0/0x6a4
 [<c043455a>] error_code+0x72/0x78
 [<c02122aa>] __put_user_4+0x12/0x18
 [<c011e09c>] schedule_tail+0x52/0x55
 [<c0104b6e>] ret_from_fork+0x6/0x1c
 =======================

boot[1153]: segfault at 0574c985 ip b7d9488a sp bfe5bfd8 error 6 Eeek! page_mapcount(page) went negative! (-1)
page pfn = bfc09
page->flags = 80000060
page->count = 1
page->mapping = f702d091
vma->vm_ops = 0x0

------------[ cut here ]------------
kernel BUG at /home/lsrc/git-arch-x86/linux-2.6-x86/mm/rmap.c:631!

invalid opcode: 0000 [#1] SMP Modules linked in:

Pid: 1153, comm: boot Tainted: G B (2.6.24-rc7-gc64ba930 #10)
EIP: 0060:[<c0154bd6>] EFLAGS: 00010246 CPU: 2
EIP is at page_remove_rmap+0xcc/0xe7
EAX: 00000000 EBX: c27f8120 ECX: 00000046 EDX: 00000046
ESI: f7074948 EDI: c27f8120 EBP: f7078198 ESP: f709ddfc
DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
Process boot (pid: 1153, ti=f709c000 task=f7460000 task.ti=f709c000)

Stack: bfc09045 00000000 c014f18f 00000000 f7074948 f709de80 bfc09045 00000000 
       00000000 00000001 b7e37000 f7070010 f70468c0 c4825240 ffffffff 00000000 
       c16e0f0c 00000000 f70aadf8 003f9ed9 b7e37000 b7e37000 00000000 b7e33000 
Call Trace:
 [<c014f18f>] unmap_vmas+0x334/0x5c9
 [<c015219e>] exit_mmap+0x5f/0xcd
 [<c011fece>] mmput+0x25/0x79
 [<c01245fa>] do_exit+0x1a9/0x5eb
 [<c0124aa7>] sys_exit_group+0x0/0xd
 [<c012bda2>] get_signal_to_deliver+0x3e3/0x405
 [<c043577c>] do_page_fault+0x0/0x6a4
 [<c01043de>] do_notify_resume+0x7d/0x64e
 [<c0122346>] printk+0x14/0x18
 [<c0435b3a>] do_page_fault+0x3be/0x6a4
 [<c0435e17>] do_page_fault+0x69b/0x6a4
 [<c043577c>] do_page_fault+0x0/0x6a4
 [<c0104d26>] work_notifysig+0x13/0x19
 =======================
Code: 8b 46 44 8b 50 08 b8 e3 aa 50 c0 e8 c0 ab fe ff 8b 46 4c 85 c0 74 14 8b 40 10 85 c0 74 0d 8b 50 2c b8 01 ab 50 c0 e8 a5 ab fe ff <0f> 0b eb fe 8b 53 10 89 d8 5b 5e 83 e2 01 f7 da 83 c2 04 e9 e1 EIP: [<c0154bd6>] page_remove_rmap+0xcc/0xe7 SS:ESP 0068:f709ddfc
---[ end trace 8cd8c46e6dae67bc ]---

Fixing recursive fault but reboot is needed!
eth0: no IPv6 routers present

OK, I see the problem. The problem is that the _PAGE_X defines are defined with _AC(UL, 1 << _PAGE_BIT_X), which has unsigned long type. This means that ~_PAGE_X also has unsigned long type, and so when cast to 64-bit in pte_mkX, it ends up &ing the pte with 0x00000000ffffffxxx, with predictable results.

The original code just used signed constants for the _PAGE_X definitions, which will sign-extend when cast to 64-bit, and so have the upper bits set when masking. (Well, actually, the old code just operated on pte_low, so the problem didn't arise; however, pgtable_64.h also uses integers for its _PAGE_X, which has the same sign-extended 32->64 casting property).

I'll put together a fixup patch now.

Quoted 2 lines

OK, I see the problem. The problem is that the _PAGE_X defines are defined with _AC(UL, 1 << _PAGE_BIT_X), which has unsigned long type. This means that ~_PAGE_X also has unsigned long type, and so when cast to 64-bit in pte_mkX, it ends up &ing the pte with 0x00000000ffffffxxx, with predictable results.

Actually I fixed some of that -- see the pgtable-nx patch on firstfloor -- but it still doesn't work. Or maybe my patch was not complete.

Quoted 4 lines

The original code just used signed constants for the _PAGE_X definitions, which will sign-extend when cast to 64-bit, and so have the upper bits set when masking. (Well, actually, the old code just operated on pte_low, so the problem didn't arise; however, pgtable_64.h also uses integers for its _PAGE_X, which has the same sign-extended 32->64 casting property).

I'll put together a fixup patch now.

I'm leaving now but can test later.

Quoted 7 lines · Andi Kleen

Quoted 6 lines

OK, I see the problem. The problem is that the _PAGE_X defines are defined with _AC(UL, 1 << _PAGE_BIT_X), which has unsigned long type. This means that ~_PAGE_X also has unsigned long type, and so when cast to 64-bit in pte_mkX, it ends up &ing the pte with 0x00000000ffffffxxx, with predictable results.

Quoted 4 lines

Actually I fixed some of that -- see the pgtable-nx patch on firstfloor -- but
it still doesn't work. Or maybe my patch was not complete.

Yeah, that looks like the right sort of thing, but I wonder if there's other places doing an open-coded "pte_val(pte) & ~_PAGE_FOO". My patch changes the definition of _PAGE_FOO so it should be OK everywhere.

Quoted 8 lines

Quoted 8 lines

The original code just used signed constants for the _PAGE_X definitions, which will sign-extend when cast to 64-bit, and so have the upper bits set when masking. (Well, actually, the old code just operated on pte_low, so the problem didn't arise; however, pgtable_64.h also uses integers for its _PAGE_X, which has the same sign-extended 32->64 casting property).

Quoted 5 lines

Quoted 2 lines

I'll put together a fixup patch now.

I'm leaving now but can test later.

Can you try this out? It applies after "x86: move all asm/pgtable constants into one place".

Quoted 11 lines · Jeremy Fitzhardinge

Quoted 7 lines · Andi Kleen

Quoted 3 lines

OK, I see the problem. The problem is that the _PAGE_X defines are defined with _AC(UL, 1 << _PAGE_BIT_X), which has unsigned long type. This means that ~_PAGE_X also has unsigned long type, and so when cast to 64-bit in pte_mkX, it ends up &ing the pte with 0x00000000ffffffxxx, with predictable results.

Actually I fixed some of that -- see the pgtable-nx patch on firstfloor -- but it still doesn't work. Or maybe my patch was not complete.

Yeah, that looks like the right sort of thing, but I wonder if there's other places doing an open-coded "pte_val(pte) & ~_PAGE_FOO". My patch changes the definition of _PAGE_FOO so it should be OK everywhere.

Yes that's probably better. I just hit it because NX bits went missing.

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

The original code just used signed constants for the _PAGE_X definitions, which will sign-extend when cast to 64-bit, and so have the upper bits set when masking. (Well, actually, the old code just operated on pte_low, so the problem didn't arise; however, pgtable_64.h also uses integers for its _PAGE_X, which has the same sign-extended 32->64 casting property).

I'll put together a fixup patch now.

Quoted 5 lines

Quoted 3 lines

I'm leaving now but can test later.

Can you try this out? It applies after "x86: move all asm/pgtable constants into one place".

I just applied over the whole git-x86 patchkit (tip 2f42671697ea9abc7d10ea7f663d6ef6e8ec6358) + the two build fixes Unfortunately it didn't work, although the faulting loop is shorter now:

Freeing unused kernel memory: 260k freed
mount[1159]: segfault at b7fcfe98 ip b7ec6b7a sp bfcfcdec error 7 Eeek! page_mapcount(page) went negative! (-1)
page pfn = bfc33
page->flags = 80000014
page->count = 0
page->mapping = 00000000
vma->vm_ops = nfs_file_vm_ops+0x0/0x18 vma->vm_ops->nopage = 0x0
vma->vm_ops->fault = filemap_fault+0x0/0x34e vma->vm_file->f_op->mmap = nfs_file_mmap+0x0/0x61

------------[ cut here ]------------
kernel BUG at /home/lsrc/quilt/linux/mm/rmap.c:631!
invalid opcode: 0000 [#1] SMP
Modules linked in:

Pid: 1159, comm: mount Not tainted (2.6.24-rc7 #55)
EIP: 0060:[<c0152a32>] EFLAGS: 00010282 CPU: 3
EIP is at page_remove_rmap+0xcc/0xe7
EAX: 00000037 EBX: c27f8660 ECX: 00000046 EDX: 00000046
ESI: f7061c60 EDI: f7d87380 EBP: f7060e78 ESP: f7509df8
DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
Process mount (pid: 1159, ti=f7508000 task=f7460540 task.ti=f7508000)

Stack: c27f8660 bfc33065 c014d166 00000000 f7061c60 f7509e80 00000000 00000000
       00000000 00000001 b7fd0000 f7555010 f7d87380 c482e240 00000000 ffffffff
       c16e0c0c 00000000 00000000 f7084df8 003d5ee2 b7fd0000 b7fd0000 00000000
Call Trace:
 [<c014d166>] unmap_vmas+0x349/0x5d1
 [<c0150006>] exit_mmap+0x5f/0xcd
 [<c011df0a>] mmput+0x25/0x79
 [<c0122636>] do_exit+0x1a9/0x5eb
 [<c0122ae3>] sys_exit_group+0x0/0xd
 [<c0129dde>] get_signal_to_deliver+0x3e3/0x405
 [<c04334b6>] do_page_fault+0x0/0x69f
 [<c0102606>] do_notify_resume+0x7d/0x64e
 [<c0120383>] printk+0x14/0x18
 [<c0433874>] do_page_fault+0x3be/0x69f
 [<c0433b4c>] do_page_fault+0x696/0x69f
 [<c01508c1>] do_munmap+0x181/0x19b
 [<c04334b6>] do_page_fault+0x0/0x69f
 [<c0102f4e>] work_notifysig+0x13/0x19
 =======================
Code: 8b 46 44 8b 50 08 b8 d2 95 50 c0 e8 9c ad fe ff 8b 46 4c 85 c0 74 14 8b 40 10 85 c0 74 0d 8b 50 2c b8 f0 95 50 c0 e8 81 ad fe ff <0f> 0b eb fe 8b 53 10 89 d8 5b 5e 83 e2 01 f7 da 83 c2 04 e9 c5
EIP: [<c0152a32>] page_remove_rmap+0xcc/0xe7 SS:ESP 0068:f7509df8
---[ end trace eb3c259ee0695fdd ]---

Fixing recursive fault but reboot is needed!
eth0: no IPv6 routers present

Quoted 5 lines

I just applied over the whole git-x86 patchkit (tip 2f42671697ea9abc7d10ea7f663d6ef6e8ec6358) + the two build fixes Unfortunately it didn't work, although the faulting loop is shorter now:

Just to be clear, these two patches cause this oops to appear? Current

x86.git seems to be missing a pile of patches at the moment, so I presume you're using a version which does have my patch set?

I just tested x86.git and then x86.git + patch you send (+ build fixes in both cases), none of your patchkits. If you want me to test some other patchkit please point me to it.

The problem happens with plain git-x86 (+ build fixes) and also with git-x86 + your patch.

Quoted 4 lines · Andi Kleen

I just tested x86.git and then x86.git + patch you send (+ build fixes in both cases), none of your patchkits. If you want me to test some other patchkit please point me to it.

Quoted 2 lines

The problem happens with plain git-x86 (+ build fixes) and also with git-x86 + your patch.

i was held up from debugging your problem by a bug introduced by you into x86.git ;-) See the fix below.

I just applied over the whole git-x86 patchkit (tip 2f42671697ea9abc7d10ea7f663d6ef6e8ec6358) + the two build fixes Unfortunately it didn't work, although the faulting loop is shorter now:

Freeing unused kernel memory: 260k freed
mount[1159]: segfault at b7fcfe98 ip b7ec6b7a sp bfcfcdec error 7 Eeek! page_mapcount(page) went negative! (-1)

Quoted 5 lines · Andi Kleen

I just applied over the whole git-x86 patchkit (tip 2f42671697ea9abc7d10ea7f663d6ef6e8ec6358) + the two build fixes Unfortunately it didn't work, although the faulting loop is shorter now:

Quoted 3 lines

Freeing unused kernel memory: 260k freed
mount[1159]: segfault at b7fcfe98 ip b7ec6b7a sp bfcfcdec error 7 Eeek! page_mapcount(page) went negative! (-1)

could you please send me the .config you are using on that kernel?

Can you try this out? It applies after "x86: move all asm/pgtable constants into one place".

Quoted 6 lines

OK, I see the problem. The problem is that the _PAGE_X defines are defined with _AC(UL, 1 << _PAGE_BIT_X), which has unsigned long type. This means that ~_PAGE_X also has unsigned long type, and so when cast to 64-bit in pte_mkX, it ends up &ing the pte with 0x00000000ffffffxxx, with predictable results.

Actually I fixed some of that -- see the pgtable-nx patch on firstfloor -- but it still doesn't work. Or maybe my patch was not complete.

Quoted 9 lines

The original code just used signed constants for the _PAGE_X definitions, which will sign-extend when cast to 64-bit, and so have the upper bits set when masking. (Well, actually, the old code just operated on pte_low, so the problem didn't arise; however, pgtable_64.h also uses integers for its _PAGE_X, which has the same sign-extended 32->64 casting property).

I'll put together a fixup patch now.

I'm leaving now but can test later.

BTW, I just noticed that _PAGE_PWT has been folded into _PAGE_PCD. This seems like a really bad idea to me, since it breaks the rule that _PAGE_X == 1 << _PAGE_BIT_X. I can't think of a specific place where this would cause problems, but this kind of non-uniformity always ends up biting someone in the arse.

I think having a specific _PAGE_NOCACHE which combines these bits is a better approach.

Quoted 2 lines

BTW, I just noticed that _PAGE_PWT has been folded into _PAGE_PCD. This seems like a really bad idea to me, since it breaks the rule that _PAGE_X == 1 << _PAGE_BIT_X. I can't think of a specific place where this would cause problems, but this kind of non-uniformity always ends up biting someone in the arse.

Agreed that it's a bad idea.

Quoted 2 lines

I think having a specific _PAGE_NOCACHE which combines these bits is a better approach.

CPA series adds that already

Good, but isn't the name _PAGE_CACHE misleading? Or does it mean

something else in the context of PAT?

Review

Quoted 6 lines · Ingo Molnar

(quoted diff) +0 -5

-#define _PAGE_PRESENT	(_AC(1, UL)<<_PAGE_BIT_PRESENT)
-#define _PAGE_RW	(_AC(1, UL)<<_PAGE_BIT_RW)
-#define _PAGE_USER	(_AC(1, UL)<<_PAGE_BIT_USER)
-#define _PAGE_PWT	(_AC(1, UL)<<_PAGE_BIT_PWT)
-#define _PAGE_PCD	((_AC(1, UL)<<_PAGE_BIT_PCD) | _PAGE_PWT)

BTW, I just noticed that _PAGE_PWT has been folded into _PAGE_PCD. This seems like a really bad idea to me, since it breaks the rule that _PAGE_X == 1 << _PAGE_BIT_X. I can't think of a specific place where this would cause problems, but this kind of non-uniformity always ends up biting someone in the arse.

I think having a specific _PAGE_NOCACHE which combines these bits is a better approach.

J

Quoted 11 lines · Jeremy Fitzhardinge

Review

Quoted 7 lines · Ingo Molnar

(quoted diff) +0 -5

-#define _PAGE_PRESENT	(_AC(1, UL)<<_PAGE_BIT_PRESENT)
-#define _PAGE_RW	(_AC(1, UL)<<_PAGE_BIT_RW)
-#define _PAGE_USER	(_AC(1, UL)<<_PAGE_BIT_USER)
-#define _PAGE_PWT	(_AC(1, UL)<<_PAGE_BIT_PWT)
-#define _PAGE_PCD	((_AC(1, UL)<<_PAGE_BIT_PCD) | _PAGE_PWT)

BTW, I just noticed that _PAGE_PWT has been folded into _PAGE_PCD. This seems like a really bad idea to me, since it breaks the rule that _PAGE_X == 1 << _PAGE_BIT_X. I can't think of a specific place where this would cause problems, but this kind of non-uniformity always ends up biting someone in the arse.

Quoted 4 lines

I think having a specific _PAGE_NOCACHE which combines these bits is a better approach.

J

How about the patch below. It defines new _PAGE_UC. One concern is drivers continuing to use _PAGE_PCD and getting wrong attributes. May be we need to rename _PAGE_PCD to catch those errors as well?

Quoted 2 lines · Jeremy Fitzhardinge

Can you try this out? It applies after "x86: move all asm/pgtable constants into one place".

and here's the patch

Subject: Re: unify pagetable accessors patch causes double fault II
From: Jeremy Fitzhardinge <[email protected]>

The patch does not fix the problem here. When I boot plain git-x86 upto commit 67db3027d691ff8049d19be7bc440eca0e5826d7
on my 6GB machine I still get an recursive oops on entering user space.

The only difference to earlier kernels seems to be that the problem happens a little latter (in stty, not boot), but that might be just coincidence.