x86/mm/cpa: set PAGE_KERNEL in __set_pages_p()
Subsystems (3)
PATCH 1/1 x86/mm/cpa: set PAGE_KERNEL in __set_pages_p()
__set_pages_np() not only clears _PAGE_PRESENT and _PAGE_RW, but also clears _PAGE_GLOBAL to avoid confusing _PAGE_GLOBAL as _PAGE_PROTNONE when the PTE is not present.
Common usage for __set_pages_p() is to call it after __set_pages_np(). Therefore calling __set_pages_p() after __set_pages_np() clears _PAGE_GLOBAL, making it unable to globally shared in TLB.
As they are called by set_direct_map_{invalid,default}_noflush(), pages in direct map cannot be globally shared in TLB after being used by vmalloc, secretmem, and hibernation.
So set PAGE_KERNEL isntead of __pgprot(_PAGE_PRESENT | _PAGE_RW) in __set_pages_p().
--- arch/x86/mm/pat/set_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/mm/pat/set_memory.c +1 -1
@@ -2177,7 +2177,7 @@ static int __set_pages_p(struct page *page, int numpages)struct cpa_data cpa = { .vaddr = &tempaddr, .pgd =NULL , .numpages = numpages, - .mask_set =__pgprot (_PAGE_PRESENT | _PAGE_RW), + .mask_set =PAGE_KERNEL , .mask_clr =__pgprot (0 ), .flags =0 };
Signature · 1 line
2.32.0
Quoted 25 lines · Hyeonggon Yoo
__set_pages_np()not only clears_PAGE_PRESENTand_PAGE_RW, but also clears_PAGE_GLOBALto avoid confusing_PAGE_GLOBALas_PAGE_PROTNONEwhen the PTE is not present.Common usage for
__set_pages_p()is to call it after__set_pages_np(). Therefore calling__set_pages_p()after__set_pages_np()clears_PAGE_GLOBAL, making it unable to globally shared in TLB.As they are called by set_direct_map_{invalid,default}_noflush(), pages in direct map cannot be globally shared in TLB after being used by vmalloc, secretmem, and hibernation.
So set
PAGE_KERNELisntead of__pgprot(_PAGE_PRESENT | _PAGE_RW)in__set_pages_p().Signed-off-by Hyeonggon Yoo <[email protected]>--- arch/x86/mm/pat/set_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/mm/pat/set_memory.c +0
-0
arch/x86/mm/pat/set_memory.c static int __set_pages_p(struct page *page, int numpages) · line 2180
Quoted 5 lines
arch/x86/mm/pat/set_memory.c +1
-1@@ -2177,7 +2177,7 @@ static int __set_pages_p(struct page *page, int numpages)struct cpa_data cpa = { .vaddr = &tempaddr, .pgd =NULL , .numpages = numpages, - .mask_set =__pgprot (_PAGE_PRESENT | _PAGE_RW), + .mask_set =PAGE_KERNEL ,
With SME/SEV, this will also (unintentionally) set the encryption bit, so I don't think this is correct.
Thanks,
Tom
Quoted 3 lines
arch/x86/mm/pat/set_memory.c +0
-0.mask_clr = __pgprot(0), .flags = 0};
Quoted 21 lines · Tom Lendacky
Quoted 20 lines · Hyeonggon Yoo
__set_pages_np()not only clears_PAGE_PRESENTand_PAGE_RW, but also clears_PAGE_GLOBALto avoid confusing_PAGE_GLOBALas_PAGE_PROTNONEwhen the PTE is not present.Common usage for
__set_pages_p()is to call it after__set_pages_np(). Therefore calling__set_pages_p()after__set_pages_np()clears_PAGE_GLOBAL, making it unable to globally shared in TLB.As they are called by set_direct_map_{invalid,default}_noflush(), pages in direct map cannot be globally shared in TLB after being used by vmalloc, secretmem, and hibernation.
So set
PAGE_KERNELisntead of__pgprot(_PAGE_PRESENT | _PAGE_RW)in__set_pages_p().Signed-off-by Hyeonggon Yoo <[email protected]>--- arch/x86/mm/pat/set_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 14 lines
arch/x86/mm/pat/set_memory.c static int __set_pages_p(struct page *page, int numpages) · line 2180
Quoted 10 lines
arch/x86/mm/pat/set_memory.c +1
-1@@ -2177,7 +2177,7 @@ static int __set_pages_p(struct page *page, int numpages)struct cpa_data cpa = { .vaddr = &tempaddr, .pgd =NULL , .numpages = numpages, - .mask_set =__pgprot (_PAGE_PRESENT | _PAGE_RW), + .mask_set =PAGE_KERNEL ,With SME/SEV, this will also (unintentionally) set the encryption bit, so I don't think this is correct.
Thank you for catching this.
It seems PAGE_KERNEL was too much for __set_pages_p().
I think __pgprot_mask(_PAGE_KERNEL | _PAGE_RW | _PAGE_GLOBAL) would be correct. Thoughts?
Quoted 5 lines
Thanks,
TomQuoted 2 lines
arch/x86/mm/pat/set_memory.c +0
-0.mask_clr = __pgprot(0), .flags = 0};
Signature · 2 lines
Thanks, Hyeonggon
Quoted 37 lines · Hyeonggon Yoo
Quoted 21 lines · Tom Lendacky
Quoted 20 lines · Hyeonggon Yoo
__set_pages_np()not only clears_PAGE_PRESENTand_PAGE_RW, but also clears_PAGE_GLOBALto avoid confusing_PAGE_GLOBALas_PAGE_PROTNONEwhen the PTE is not present.Common usage for
__set_pages_p()is to call it after__set_pages_np(). Therefore calling__set_pages_p()after__set_pages_np()clears_PAGE_GLOBAL, making it unable to globally shared in TLB.As they are called by set_direct_map_{invalid,default}_noflush(), pages in direct map cannot be globally shared in TLB after being used by vmalloc, secretmem, and hibernation.
So set
PAGE_KERNELisntead of__pgprot(_PAGE_PRESENT | _PAGE_RW)in__set_pages_p().Signed-off-by Hyeonggon Yoo <[email protected]>--- arch/x86/mm/pat/set_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Quoted 14 lines
arch/x86/mm/pat/set_memory.c static int __set_pages_p(struct page *page, int numpages) · line 2180
Quoted 10 lines
arch/x86/mm/pat/set_memory.c +1
-1@@ -2177,7 +2177,7 @@ static int __set_pages_p(struct page *page, int numpages)struct cpa_data cpa = { .vaddr = &tempaddr, .pgd =NULL , .numpages = numpages, - .mask_set =__pgprot (_PAGE_PRESENT | _PAGE_RW), + .mask_set =PAGE_KERNEL ,With SME/SEV, this will also (unintentionally) set the encryption bit, so I don't think this is correct.
Quoted 5 lines
Thank you for catching this.
It seemsPAGE_KERNELwas too much for__set_pages_p().I think
__pgprot_mask(_PAGE_KERNEL | _PAGE_RW | _PAGE_GLOBAL)would be correct. Thoughts?
That should work, but you are counting on this function being called in a specific situation. Should this function ever be called from some other place in the kernel, in the future, that might inadvertently apply the global setting when it shouldn't.
I'm wondering if adding a specific helper that takes a boolean to indicate whether to set the global flag would be best. I'll let some of the MM maintainers comment about that.
Thanks,
Tom
Quoted 7 lines
Quoted 6 lines
Thanks,
TomQuoted 2 lines
arch/x86/mm/pat/set_memory.c +0
-0.mask_clr = __pgprot(0), .flags = 0};
Quoted 3 lines · Tom Lendacky
I'm wondering if adding a specific helper that takes a boolean to indicate whether to set the global flag would be best. I'll let some of the MM maintainers comment about that.
First of all, I'm not positive that _PAGE_BIT_PROTNONE is ever used for kernel mappings. This would all get a lot easier if we decided that _PAGE_BIT_PROTNONE is only for userspace mappings and we don't have to worry about it when _PAGE_USER is clear.
Second, the number of places that do these __set_pages_p()/__set_pages_np() pairs is pretty limited. Some of them are *quite* unambiguous over whether they are dealing with the direct map:
Quoted 9 lines
int
set_direct_map_invalid_noflush(struct page *page){ return __set_pages_np(page, 1); }int
set_direct_map_default_noflush(struct page *page){ return __set_pages_p(page, 1); }
which would make it patently obvious whether __set_pages_p() should restore the global bit. That would have been a problem in the "old" PTI days where _some_ of the direct map was exposed to Meltdown. I don't think we have any of those mappings left, though. They're all aliases like text and cpu_entry_area.
It would be nice if someone could look into unraveling _PAGE_BIT_PROTNONE. We could even probably move it to another bit for kernel mappings if we actually need it (I'm not convinced we do).
Is _PAGE_PROTNONE set only for user mappings?
Quoted 9 lines · Dave Hansen
Quoted 3 lines · Tom Lendacky
I'm wondering if adding a specific helper that takes a boolean to indicate whether to set the global flag would be best. I'll let some of the MM maintainers comment about that.
First of all, I'm not positive that
_PAGE_BIT_PROTNONEis ever used for kernel mappings. This would all get a lot easier if we decided that_PAGE_BIT_PROTNONEis only for userspace mappings and we don't have to worry about it when_PAGE_USERis clear.
After quickly skimming code it seems the place that actually sets _PAGE_PROTNONE is via mm/mmap.c's protection_map:
Quoted 4 lines
/* description of effects of mapping type and prot in current implementation.
* this is due to the limited x86 page protection hardware. The expected
* behavior is in parens:
*
Quoted 15 lines
* map_type prot
* PROT_NONE PROT_READ PROT_WRITE PROT_EXEC
* MAP_SHARED r: (no) no r: (yes) yes r: (no) yes r: (no) yes
* w: (no) no w: (no) no w: (yes) yes w: (no) no
* x: (no) no x: (no) yes x: (no) yes x: (yes) yes
*
* MAP_PRIVATE r: (no) no r: (yes) yes r: (no) yes r: (no) yes
* w: (no) no w: (no) no w: (copy) copy w: (no) no
* x: (no) no x: (no) yes x: (no) yes x: (yes) yes
*
*/
pgprot_t protection_map[16] = {
__P000, __P001, __P010, __P011, __P100, __P101, __P110, __P111,
__S000, __S001, __S010, __S011, __S100, __S101, __S110, __S111
};Where __P000, __S000 is PAGE_NONE (_PAGE_ACCESSED | _PAGE_PROTNONE).
And protection_map is accessed via:
Quoted 9 lines
pgprot_tvm_get_page_prot(unsigned long vm_flags){ pgprot_t ret = __pgprot(pgprot_val(protection_map[vm_flags & (VM_READ|VM_WRITE|VM_EXEC|VM_SHARED)]) | pgprot_val(arch_vm_get_page_prot(vm_flags))); return arch_filter_pgprot(ret); }
EXPORT_SYMBOL(vm_get_page_prot);
I guess it's only set for processes' VMA if no caller is abusing vm_get_page_prot() for kernel mappings.
But yeah, just quick guessing does not make us convinced. Let's Cc people working on mm.
If kernel never uses _PAGE_PROTNONE for kernel mappings, it's just okay not to clear _PAGE_GLOBAL at first in __change_page_attr() if it's not user address, because no user will confuse _PAGE_GLOBAL as _PAGE_PROTNONE if it's kernel address. right?
Quoted 24 lines
Second, the number of places that do these __set_pages_p()/__set_pages_np() pairs is pretty limited. Some of them are *quite* unambiguous over whether they are dealing with the direct map:
Quoted 9 lines
int
set_direct_map_invalid_noflush(struct page *page){ return __set_pages_np(page, 1); }int
set_direct_map_default_noflush(struct page *page){ return __set_pages_p(page, 1); }which would make it patently obvious whether
__set_pages_p()should restore the global bit. That would have been a problem in the "old" PTI days where _some_ of the direct map was exposed to Meltdown. I don't think we have any of those mappings left, though. They're all aliases like text andcpu_entry_area.It would be nice if someone could look into unraveling
_PAGE_BIT_PROTNONE. We could even probably move it to another bit for kernel mappings if we actually need it (I'm not convinced we do).
Signature · 2 lines
Thanks, Hyeonggon
Re: Is _PAGE_PROTNONE set only for user mappings?
Quoted 13 lines · Hyeonggon Yoo
Quoted 9 lines
pgprot_tvm_get_page_prot(unsigned long vm_flags){ pgprot_t ret = __pgprot(pgprot_val(protection_map[vm_flags & (VM_READ|VM_WRITE|VM_EXEC|VM_SHARED)]) | pgprot_val(arch_vm_get_page_prot(vm_flags))); return arch_filter_pgprot(ret); }
EXPORT_SYMBOL(vm_get_page_prot);I guess it's only set for processes' VMA if no caller is abusing
vm_get_page_prot()for kernel mappings.
Quoted 8 lines
But yeah, just quick guessing does not make us convinced. Let's Cc people working on mm.
If kernel never uses
_PAGE_PROTNONEfor kernel mappings, it's just okay not to clear_PAGE_GLOBALat first in__change_page_attr()if it's not user address, because no user will confuse_PAGE_GLOBALas_PAGE_PROTNONEif it's kernel address. right?
I'm not aware of a case where _PAGE_BIT_PROTNONE is used for a kernel address expecting PROT_NONE semantics instead of the global bit. NUMA Balancing is not going to accidentally treat a kernel address as if it's a NUMA hinting fault. By the time a fault is determining if a PTE access is a numa hinting fault or accesssing a PROT_NONE region, it has been established that it is a userspace address backed by a valid VMA.
Signature · 2 lines
Mel Gorman SUSE Labs
Re: Is _PAGE_PROTNONE set only for user mappings?
Quoted 23 lines · Mel Gorman
Quoted 13 lines · Hyeonggon Yoo
Quoted 9 lines
pgprot_tvm_get_page_prot(unsigned long vm_flags){ pgprot_t ret = __pgprot(pgprot_val(protection_map[vm_flags & (VM_READ|VM_WRITE|VM_EXEC|VM_SHARED)]) | pgprot_val(arch_vm_get_page_prot(vm_flags))); return arch_filter_pgprot(ret); }
EXPORT_SYMBOL(vm_get_page_prot);I guess it's only set for processes' VMA if no caller is abusing
vm_get_page_prot()for kernel mappings.Quoted 8 lines
But yeah, just quick guessing does not make us convinced. Let's Cc people working on mm.
If kernel never uses
_PAGE_PROTNONEfor kernel mappings, it's just okay not to clear_PAGE_GLOBALat first in__change_page_attr()if it's not user address, because no user will confuse_PAGE_GLOBALas_PAGE_PROTNONEif it's kernel address. right?
Quoted 7 lines
I'm not aware of a case where
_PAGE_BIT_PROTNONEis used for a kernel address expectingPROT_NONEsemantics instead of the global bit. NUMA Balancing is not going to accidentally treat a kernel address as if it's a NUMA hinting fault. By the time a fault is determining if a PTE access is a numa hinting fault or accesssing aPROT_NONEregion, it has been established that it is a userspace address backed by a valid VMA.
Thanks Mel, and IIUC nor does do_kern_addr_fault() in arch/x86/mm/fault.c expect _PAGE_PROTNONE instead of _PAGE_GLOBAL. I want to make it clear in the code that _PAGE_PROTNONE is only used for user mappings.
How does below look?
arch/x86/include/asm/pgtable_types.h +6 -2
@@ -35,7 +35,10 @@#define _PAGE_BIT_DEVMAP _PAGE_BIT_SOFTW4/* If _PAGE_BIT_PRESENT is clear, we use these: */ -/* - if the user mapped it with PROT_NONE; pte_present gives true */ +/* +* if the user mapped it with PROT_NONE; pte_present gives true +* this is only used for user mappings (with _PAGE_BIT_USER) +*/ #define _PAGE_BIT_PROTNONE _PAGE_BIT_GLOBAL#define _PAGE_PRESENT (_AT(pteval_t, 1) << _PAGE_BIT_PRESENT)
@@ -115,7 +118,8 @@#define _PAGE_DEVMAP (_AT(pteval_t, 0))#endif -#define _PAGE_PROTNONE (_AT(pteval_t, 1) << _PAGE_BIT_PROTNONE) +#define _PAGE_PROTNONE ((_AT(pteval_t, 1) << _PAGE_BIT_USER) | \ + (_AT(pteval_t, 1) << _PAGE_BIT_PROTNONE))/* * Set of bits not changed in pte_modify. The pte's
Signature · 2 lines
Thanks, Hyeonggon
Re: Is _PAGE_PROTNONE set only for user mappings?
Quoted 32 lines · Hyeonggon Yoo
Quoted 23 lines · Mel Gorman
Quoted 13 lines · Hyeonggon Yoo
Quoted 9 lines
pgprot_tvm_get_page_prot(unsigned long vm_flags){ pgprot_t ret = __pgprot(pgprot_val(protection_map[vm_flags & (VM_READ|VM_WRITE|VM_EXEC|VM_SHARED)]) | pgprot_val(arch_vm_get_page_prot(vm_flags))); return arch_filter_pgprot(ret); }
EXPORT_SYMBOL(vm_get_page_prot);I guess it's only set for processes' VMA if no caller is abusing
vm_get_page_prot()for kernel mappings.Quoted 8 lines
But yeah, just quick guessing does not make us convinced. Let's Cc people working on mm.
If kernel never uses
_PAGE_PROTNONEfor kernel mappings, it's just okay not to clear_PAGE_GLOBALat first in__change_page_attr()if it's not user address, because no user will confuse_PAGE_GLOBALas_PAGE_PROTNONEif it's kernel address. right?Quoted 7 lines
I'm not aware of a case where
_PAGE_BIT_PROTNONEis used for a kernel address expectingPROT_NONEsemantics instead of the global bit. NUMA Balancing is not going to accidentally treat a kernel address as if it's a NUMA hinting fault. By the time a fault is determining if a PTE access is a numa hinting fault or accesssing aPROT_NONEregion, it has been established that it is a userspace address backed by a valid VMA.
Quoted 6 lines
Thanks Mel, and IIUC nor does
do_kern_addr_fault()inarch/x86/mm/fault.cexpect_PAGE_PROTNONEinstead of_PAGE_GLOBAL. I want to make it clear in the code that_PAGE_PROTNONEis only used for user mappings.How does below look?
I've no strong objections. I worry that this somehow could be used to set PAGE_USER on a kernel mapping page and maybe a comment would be more appropriate. However, I'm failing to imagine how NUMA balancing could be fooled into doing that.
Signature · 2 lines
Mel Gorman SUSE Labs
Re: Is _PAGE_PROTNONE set only for user mappings?
In reply to Hyeonggon YooQuoted 23 lines · Hyeonggon Yoo
Thanks Mel, and IIUC nor does
do_kern_addr_fault()inarch/x86/mm/fault.cexpect_PAGE_PROTNONEinstead of_PAGE_GLOBAL. I want to make it clear in the code that_PAGE_PROTNONEis only used for user mappings.How does below look?
arch/x86/include/asm/pgtable_types.h +4
-1@@ -35,7 +35,10 @@#define _PAGE_BIT_DEVMAP _PAGE_BIT_SOFTW4/* If _PAGE_BIT_PRESENT is clear, we use these: */ -/* - if the user mapped it with PROT_NONE; pte_present gives true */ +/* +* if the user mapped it with PROT_NONE; pte_present gives true +* this is only used for user mappings (with _PAGE_BIT_USER) +*/ #define _PAGE_BIT_PROTNONE _PAGE_BIT_GLOBAL#define _PAGE_PRESENT (_AT(pteval_t, 1) << _PAGE_BIT_PRESENT)
arch/x86/include/asm/pgtable_types.h line 125
Quoted 9 lines
arch/x86/include/asm/pgtable_types.h +2
-1@@ -115,7 +118,8 @@#define _PAGE_DEVMAP (_AT(pteval_t, 0))#endif -#define _PAGE_PROTNONE (_AT(pteval_t, 1) << _PAGE_BIT_PROTNONE) +#define _PAGE_PROTNONE ((_AT(pteval_t, 1) << _PAGE_BIT_USER) | \ + (_AT(pteval_t, 1) << _PAGE_BIT_PROTNONE))/* * Set of bits not changed in pte_modify. The pte's
I don't like the idea of _PAGE_BIT_USER being so implicit. It is something kernel users should know explicitly that they are messing with.
I was thinking of something more along the lines of taking the set_memory.c code and ensuring that it never sets (or even observes) _PAGE_BIT_GLOBAL on a _PAGE_USER mapping. There was also a question of if set_memory.c is ever used on userspace mappings. It would be good to validate whether it's possible in-tree today and if not, enforce that _PAGE_USER PTEs should never even be observed with set_memory.c.
The arch/x86/mm/dump_pagetables.c code is also a reasonable place to put assumptions about the page tables since it walks *everything* when asked.
Re: Is _PAGE_PROTNONE set only for user mappings?
Quoted 31 lines · Dave Hansen
Quoted 30 lines · Hyeonggon Yoo
Thanks Mel, and IIUC nor does
do_kern_addr_fault()inarch/x86/mm/fault.cexpect_PAGE_PROTNONEinstead of_PAGE_GLOBAL. I want to make it clear in the code that_PAGE_PROTNONEis only used for user mappings.How does below look?
arch/x86/include/asm/pgtable_types.h +6
-2@@ -35,7 +35,10 @@#define _PAGE_BIT_DEVMAP _PAGE_BIT_SOFTW4/* If _PAGE_BIT_PRESENT is clear, we use these: */ -/* - if the user mapped it with PROT_NONE; pte_present gives true */ +/* +* if the user mapped it with PROT_NONE; pte_present gives true +* this is only used for user mappings (with _PAGE_BIT_USER) +*/ #define _PAGE_BIT_PROTNONE _PAGE_BIT_GLOBAL#define _PAGE_PRESENT (_AT(pteval_t, 1) << _PAGE_BIT_PRESENT)@@ -115,7 +118,8 @@#define _PAGE_DEVMAP (_AT(pteval_t, 0))#endif -#define _PAGE_PROTNONE (_AT(pteval_t, 1) << _PAGE_BIT_PROTNONE) +#define _PAGE_PROTNONE ((_AT(pteval_t, 1) << _PAGE_BIT_USER) | \ + (_AT(pteval_t, 1) << _PAGE_BIT_PROTNONE))
Quoted 6 lines
arch/x86/include/asm/pgtable_types.h line 125
Quoted 2 lines
arch/x86/include/asm/pgtable_types.h +0
-0/* * Set of bits not changed in pte_modify. The pte's I don't like the idea of
_PAGE_BIT_USERbeing so implicit. It is something kernel users should know explicitly that they are messing with.
Sounds reasonable. Better explicit than implicit.
Quoted 3 lines
I was thinking of something more along the lines of taking the
set_memory.ccode and ensuring that it never sets (or even observes)_PAGE_BIT_GLOBALon a_PAGE_USERmapping.
Yeah that would be a bit more explicit solution.
Quoted 4 lines
There was also a question of
ifset_memory.cis ever used on userspace mappings. It would be good to validate whether it's possible in-tree today and if not, enforce that_PAGE_USERPTEs should never even be observed withset_memory.c.
Simply adding dump_stack() tells me my kernel on my machine does not use set_memory.c for userspace mappings but Hmm I'll take a look.
Quoted 2 lines
The
arch/x86/mm/dump_pagetables.ccode is also a reasonable place to put assumptions about the page tables since it walks *everything* when asked.
Thanks for that information!
Thanks,
Hyeonggon
Re: Is _PAGE_PROTNONE set only for user mappings?
Quoted 7 lines · Hyeonggon Yoo
Quoted 3 lines · Dave Hansen
I was thinking of something more along the lines of taking the
set_memory.ccode and ensuring that it never sets (or even observes)_PAGE_BIT_GLOBALon a_PAGE_USERmapping.Yeah that would be a bit more explicit solution.
Quoted 7 lines
Quoted 4 lines
There was also a question of
ifset_memory.cis ever used on userspace mappings. It would be good to validate whether it's possible in-tree today and if not, enforce that_PAGE_USERPTEs should never even be observed withset_memory.c.Simply adding
dump_stack()tells me my kernel on my machine does not useset_memory.cfor userspace mappings but Hmm I'll take a look.
vc_slow_virt_to_phys() uses lookup_address_in_pgd() with user mappings, but that code is all but guaranteed to be buggy, e.g. doesn't guard against concurrent modifications to user mappings.
show_fault_oops() can also call into lookup_address_in_pgd() with a user mapping, though at that point the kernel has bigger problems since it's executing from user memory.
And up until commits 44187235cbcc ("KVM: x86/mmu: fix potential races when walking host page table") and 643d95aac59a ("Revert "x86/mm: Introduce lookup_address_in_mm()""), KVM had a similar bug.
Generally speaking, set_memory.c is not equipped to play nice with user mappings. It mostly "works", but there are races galore. IMO, hardening set_memory.c to scream if it's handed a user address or encounters _PAGE_USER PTEs would be a very good thing.
Re: Is _PAGE_PROTNONE set only for user mappings?
Quoted 20 lines · Sean Christopherson
Quoted 7 lines · Hyeonggon Yoo
Quoted 3 lines · Dave Hansen
I was thinking of something more along the lines of taking the
set_memory.ccode and ensuring that it never sets (or even observes)_PAGE_BIT_GLOBALon a_PAGE_USERmapping.Yeah that would be a bit more explicit solution.
Quoted 7 lines
Quoted 4 lines
There was also a question of
ifset_memory.cis ever used on userspace mappings. It would be good to validate whether it's possible in-tree today and if not, enforce that_PAGE_USERPTEs should never even be observed withset_memory.c.Simply adding
dump_stack()tells me my kernel on my machine does not useset_memory.cfor userspace mappings but Hmm I'll take a look.
vc_slow_virt_to_phys()useslookup_address_in_pgd()with user mappings, but that code is all but guaranteed to be buggy, e.g. doesn't guard against concurrent modifications to user mappings.
Quoted 7 lines
show_fault_oops()can also call intolookup_address_in_pgd()with a user mapping, though at that point the kernel has bigger problems since it's executing from user memory.And up until commits
44187235cbcc("KVM: x86/mmu: fix potential races when walking host page table") and643d95aac59a("Revert "x86/mm: Introducelookup_address_in_mm()""), KVM had a similar bug.
Thanks for your helpful insight.
I was curious if set_memory*() helpers are used for user mappings.
with some quick look ptrace() and uprobes (where updating application's text is needed) use kmap + memcpy or replace_page() instead of set_memory*() API.
_lookup_address_cpa() uses init_mm.pgd when cpa.pgd is not specified and the only place that passes pgd is efi subsystem (efi_mm.pgd), which is not a userspace.
So it is *obvious* that set_memory*() functions should not be used for user mappings. because that will only result in updating only init_mm's page table.
Therefore answering to the first question ('do we really need to unset _PAGE_GLOBAL when we're clearing _PAGE_PRESENT in set_memory.c to avoid confusing it as _PAGE_PROTNONE?'):
we don't need to consider PROT_NONE semantics in set_memory.c
because we don't (shouldn't) change user mappings in it and _PAGE_PROTNONE is not used for kernel mappings.
Quoted 3 lines
Generally speaking,
set_memory.cis not equipped to play nice with user mappings. It mostly "works", but there are races galore. IMO, hardeningset_memory.cto scream if it's handed a user address or encounters_PAGE_USERPTEs would be a very good thing.
I agree that would be a good thing too.
Thanks,
Hyeonggon
Re: Is _PAGE_PROTNONE set only for user mappings?
In reply to Dave HansenQuoted 37 lines · Dave Hansen
Quoted 23 lines · Hyeonggon Yoo
Thanks Mel, and IIUC nor does
do_kern_addr_fault()inarch/x86/mm/fault.cexpect_PAGE_PROTNONEinstead of_PAGE_GLOBAL. I want to make it clear in the code that_PAGE_PROTNONEis only used for user mappings.How does below look?
arch/x86/include/asm/pgtable_types.h +4
-1@@ -35,7 +35,10 @@#define _PAGE_BIT_DEVMAP _PAGE_BIT_SOFTW4/* If _PAGE_BIT_PRESENT is clear, we use these: */ -/* - if the user mapped it with PROT_NONE; pte_present gives true */ +/* +* if the user mapped it with PROT_NONE; pte_present gives true +* this is only used for user mappings (with _PAGE_BIT_USER) +*/ #define _PAGE_BIT_PROTNONE _PAGE_BIT_GLOBAL#define _PAGE_PRESENT (_AT(pteval_t, 1) << _PAGE_BIT_PRESENT)arch/x86/include/asm/pgtable_types.h line 125
Quoted 9 lines
arch/x86/include/asm/pgtable_types.h +2
-1@@ -115,7 +118,8 @@#define _PAGE_DEVMAP (_AT(pteval_t, 0))#endif -#define _PAGE_PROTNONE (_AT(pteval_t, 1) << _PAGE_BIT_PROTNONE) +#define _PAGE_PROTNONE ((_AT(pteval_t, 1) << _PAGE_BIT_USER) | \ + (_AT(pteval_t, 1) << _PAGE_BIT_PROTNONE))/* * Set of bits not changed in pte_modify. The pte's I don't like the idea of
_PAGE_BIT_USERbeing so implicit. It is something kernel users should know explicitly that they are messing with.
Quoted 6 lines
I was thinking of something more along the lines of taking the
set_memory.ccode and ensuring that it never sets (or even observes)_PAGE_BIT_GLOBALon a_PAGE_USERmapping. There was also a question of ifset_memory.cis ever used on userspace mappings. It would be good to validate whether it's possible in-tree today and if not, enforce that_PAGE_USERPTEs should never even be observed withset_memory.c.
Writing code I'm a bit confused:
commit d1440b23c922d8 ("x86/mm: Factor out pageattr _PAGE_GLOBAL setting") says:
"This unconditional setting of _PAGE_GLOBAL is a problem when we have PTI and non-PTI and we want some areas to have _PAGE_GLOBAL and some not."
Is this this sentence not valid anymore in PTI,
and just unconditionally setting _PAGE_GLOBAL would be okay in kernel side regardless of PTI?
I'm wondering it because previously I thought "Let's not clear _PAGE_GLOBAL in set_memory.c for kernel mappings and make pmd/pte_present() not confuse when _PAGE_USER is not set"
But you don't like it as it's a bit implicit.
Then I wonder - how do we know when to set _PAGE_GLOBAL again?
Quoted 2 lines
The
arch/x86/mm/dump_pagetables.ccode is also a reasonable place to put assumptions about the page tables since it walks *everything* when asked.
Re: Is _PAGE_PROTNONE set only for user mappings?
Quoted 5 lines · Hyeonggon Yoo
On Mon, May 16, 2022 at 07:04:32AM -0700, Dave Hansen wrote:
Writing code I'm a bit confused:
commitd1440b23c922d8("x86/mm: Factor out pageattr_PAGE_GLOBALsetting") says:
Quoted 7 lines
"This unconditional setting of
_PAGE_GLOBALis a problem when we have PTI and non-PTI and we want some areas to have_PAGE_GLOBALand some not."Is this this sentence not valid anymore in PTI,
and just unconditionally setting_PAGE_GLOBALwould be okay in kernel side regardless of PTI?
I believe it's still valid.
IIRC, there are three cases:
1. No KPTI. All kernel mappings are _PAGE_GLOBAL. Basically, for present mappings, if _PAGE_USER is clear, _PAGE_GLOBAL is set.
2. KPTI with PCID hardware support (or in a few other cases): The kernel image is mostly non-global. Anything mapped into userspace *is* marked global, like entry text.
3. KPTI without PCIDs: Basically case #2, but with more of the kernel image left global.
So, not only are there different KPTI modes, there a different pars of the kernel that require different _PAGE_GLOBAL behavior.
pti_kernel_image_global_ok() in arch/x86/mm/pti.c explains it pretty well.
Quoted 9 lines · Hyeonggon Yoo
__set_pages_np()not only clears_PAGE_PRESENTand_PAGE_RW, but also clears_PAGE_GLOBALto avoid confusing_PAGE_GLOBALas_PAGE_PROTNONEwhen the PTE is not present.Common usage for
__set_pages_p()is to call it after__set_pages_np().
Therefore calling__set_pages_p()after__set_pages_np()clears_PAGE_GLOBAL, making it unable to globally shared in TLB.
Quoted 7 lines
As they are called by set_direct_map_{invalid,default}_noflush(), pages in direct map cannot be globally shared in TLB after being used by
vmalloc, secretmem, and hibernation.So set
PAGE_KERNELisntead of__pgprot(_PAGE_PRESENT | _PAGE_RW)in__set_pages_p().
Nice find. I think we can't always set PAGE_KERNEL also because of the PTI code. It sometimes wants the direct map to be non global.
Maybe something like this?
(_PAGE_PRESENT | _PAGE_RW | _PAGE_GLOBAL) & __default_kernel_pte_mask
That would add back in a little of the "default global" behavior that was removed in d1440b2, but I think it should be ok because it is limited to the direct map. Otherwise, I wonder if the existing global clearing logic is really needed.
Quoted 15 lines · Rick P
Quoted 14 lines · Hyeonggon Yoo
__set_pages_np()not only clears_PAGE_PRESENTand_PAGE_RW, but also clears_PAGE_GLOBALto avoid confusing_PAGE_GLOBALas_PAGE_PROTNONEwhen the PTE is not present.Common usage for
__set_pages_p()is to call it after__set_pages_np().
Therefore calling__set_pages_p()after__set_pages_np()clears_PAGE_GLOBAL, making it unable to globally shared in TLB.As they are called by set_direct_map_{invalid,default}_noflush(), pages in direct map cannot be globally shared in TLB after being used by
vmalloc, secretmem, and hibernation.
Quoted 6 lines
Quoted 2 lines
So set
PAGE_KERNELisntead of__pgprot(_PAGE_PRESENT | _PAGE_RW)in__set_pages_p().Nice find. I think we can't always set
PAGE_KERNELalso because of the PTI code. It sometimes wants the direct map to be non global.
Thanks for review!
IIUC __pgprot_mask() already clears _PAGE_GLOBAL from PAGE_KERNEL when PTI is used.
#define __pgprot_mask(x) __pgprot((x) & __default_kernel_pte_mask)
#define PAGE_KERNEL __pgprot_mask(__PAGE_KERNEL | _ENC)
But yeah, it seems PAGE_KERNEL is too much for this.
Quoted 3 lines
Maybe something like this?
(_PAGE_PRESENT|_PAGE_RW|_PAGE_GLOBAL) &__default_kernel_pte_mask
What about __pgprot_mask(_PAGE_PRESENT | _PAGE_RW | _PAGE_GLOBAL)?
Quoted 6 lines
That would add back in a little of the "default global" behavior that was removed in d1440b2, but I think it should be ok because it is limited to the direct map.
Otherwise, I wonder if the existing global
clearing logic is really needed.
I think it's still needed. pte_present() returning true due to _PAGE_PROTNONE after __set_pages_np() simply doesn't make sense. No?
Signature · 2 lines
Thanks, Hyeonggon
Quoted 10 lines · Hyeonggon Yoo
Thanks for review!
IIUC
__pgprot_mask()already clears_PAGE_GLOBALfromPAGE_KERNELwhen PTI is used.#define
__pgprot_mask(x)__pgprot((x) &__default_kernel_pte_mask)
#definePAGE_KERNEL__pgprot_mask(__PAGE_KERNEL | _ENC)But yeah, it seems
PAGE_KERNELis too much for this.
Oh, yep, you're right.
Quoted 15 lines
Quoted 5 lines
Maybe something like this?
(_PAGE_PRESENT|_PAGE_RW|_PAGE_GLOBAL) &__default_kernel_pte_maskWhat about
__pgprot_mask(_PAGE_PRESENT | _PAGE_RW | _PAGE_GLOBAL)?Quoted 6 lines
That would add back in a little of the "default global" behavior that
was removed in d1440b2, but I think it should be ok because it is limited to the direct map.
Otherwise, I wonder if the existing global
clearing logic is really needed.
Quoted 3 lines
I think it's still needed.
pte_present()returning true due to_PAGE_PROTNONE
after__set_pages_np()simply doesn't make sense. No?
Yea, I just meant we don't need PROTNONE in the kernel PTEs (I think), so the kernel is forced to do weird stuff in CPA so that it can share the pte helpers with userspace.
I have not thought this all the way through, but say we declared that PROT_NONE doesn't exist in kernel pte's, then pte_present() could be like:
static inline int pte_present(pte_t a)
{
pteval_t val = pte_flags(a)
bool kernel_p = val & (_PAGE_USER | _PAGE_PRESENT) ==
_PAGE_PRESENT;
bool user_p = (val & _PAGE_USER) &&
(val & (_PAGE_PRESENT | _PAGE_PROTNONE));
/*
* PROT_NONE does not exist for kernel PTEs, but global and
* not present can appear in other cases. Check each
* differently depending on the user bit.
*/
return kernel_p || user_p;
}Not sure where else it ties in. I think that is what Dave was getting at in the other thread - to figure out all the places that need to change to remove the meaning from kernel PTEs.
Re-setting global seems valid too.