x86/mm: Fix set_memory_rox()
Peter Zijlstra
maintainercover x86/mm: Fix set_memory_rox()
Hi,
The robot reported fail on commit:
b38994948567 ("x86/mm: Implement native set_memory_rox()")
which led to some non-obvious code in __change_page_attr_set_clr(). These few patches attempt to clean that up.
Peter Zijlstra
maintainerPATCH 1/4 x86/mm: Add a few comments
It's a shame to hide useful comments in Changelogs, add some to the code.
Shamelessly stolen from commit:
c40a56a7818c ("x86/mm/init: Remove freed kernel image areas from alias mapping")
--- arch/x86/mm/pat/set_memory.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
arch/x86/mm/pat/set_memory.c +14 -0
@@ -219,6 +219,17 @@ within_inclusive(unsigned long addr, uns#ifdef CONFIG_X86_64 +/* +* The kernel image is mapped into two places in the virtual address space +* (addresses without KASLR, of course): +* +* 1. The kernel direct map (0xffff880000000000) +* 2. The "high kernel map" (0xffffffff81000000) +* +* We actually execute out of #2. If we get the address of a kernel symbol, it +* points to #2, but almost all physical-to-virtual translations point to #1. +*/ +static inline unsigned long highmap_start_pfn (void ) {return __pa_symbol (_text) >>PAGE_SHIFT ;
@@ -1626,6 +1637,9 @@ static int __change_page_attr(struct cpastatic int __change_page_attr_set_clr (struct cpa_data *cpa,int checkalias); +/* +* Check the directmap and "high kernel map" 'aliases'. +*/ static int cpa_process_alias (struct cpa_data *cpa) {struct cpa_data alias_cpa;
Re: [PATCH 1/4] x86/mm: Add a few comments
Quoted 5 lines · Peter Zijlstra
It's a shame to hide useful comments in Changelogs, add some to the code.
Shamelessly stolen from commit:
When the comment says how the image is mapped into two places, wouldn't it be good to also have the reason *why* rather than just what..
That said, my real commentary for this patch set is not that particular instance, but the bigger picture - that this code is still a bit opaque as to why these things are done.
Do we need any of those alias passes at all for pure protection bit changes? I thought we only did these because things like cacheability bits have to be in sync due to machine checks etc?
Or am I missing some case where writability matters too?
Linus
Peter Zijlstra
maintainerRe: [PATCH 1/4] x86/mm: Add a few comments
Quoted 10 lines · Linus Torvalds
Quoted 5 lines · Peter Zijlstra
It's a shame to hide useful comments in Changelogs, add some to the code.
Shamelessly stolen from commit:
When the comment says how the image is mapped into two places, wouldn't it be good to also have the reason *why* rather than just what..
Agreed, and I think I can actually answer this. How about I change it into something like this:
/*
* The kernel image is mapped into two places in the virtual address space
* (addresses without KASLR, of course):
*
* 1. The kernel direct map (0xffff880000000000)
* 2. The "high kernel map" (0xffffffff81000000)
*
* We actually execute out of #2. If we get the address of a kernel symbol, it
* points to #2, but almost all physical-to-virtual translations point to #1. *
* This is so that we can have both a directmap of all physical memory
* *and* take full advantage of the limited (s32) immediate addressing
* range (2G) of x86_64.
*
* See Documentation/x86/x86_64/mm.rst for more details.
*/
Quoted 3 lines
That said, my real commentary for this patch set is not that particular instance, but the bigger picture - that this code is still a bit opaque as to why these things are done.
Yes, small steps I suppose. It took me way longer than it should have to figure out what this code was actually doing. And why we shouldn't propagate those NX bits in the various cases.
Quoted 5 lines
Do we need any of those alias passes at all for pure protection bit changes? I thought we only did these because things like cacheability bits have to be in sync due to machine checks etc?
Or am I missing some case where writability matters too?
I _think_, but I'm not actually sure, that it matters in exactly that case dhansen mentions, where we do a physical to virtual address translation and expect access to match whatever alias we originally came from.
Peter Zijlstra
maintainerRe: [PATCH 1/4] x86/mm: Add a few comments
Quoted 10 lines · Peter Zijlstra
Quoted 5 lines
Do we need any of those alias passes at all for pure protection bit changes? I thought we only did these because things like cacheability bits have to be in sync due to machine checks etc?
Or am I missing some case where writability matters too?
I _think_, but I'm not actually sure, that it matters in exactly that case dhansen mentions, where we do a physical to virtual address translation and expect access to match whatever alias we originally came from.
That of course only covers the directmap; for giggles I did the below patch on top of these and the testcase at hand boots and finishes just fine...
So yeah, no sodding clue why we do that :/
arch/x86/mm/pat/set_memory.c +1 -1
@@ -1656,7 +1676,7 @@ static int cpa_process_alias(struct cpa_data *cpa)return ret; } -#ifdef CONFIG_X86_64 +#if 0 // def CONFIG_X86_64 /* * If the primary call didn't touch the high mapping already * and the physical address is inside the kernel map, we need
Peter Zijlstra
maintainerPATCH 2/4 x86/mm: Untangle __change_page_attr_set_clr(.checkalias)
In reply to Peter ZijlstraThe .checkalias argument to __change_page_attr_set_clr() is overloaded and serves two different purposes:
- it inhibits the call to cpa_process_alias() -- as suggested by the name; however,
- it also serves as 'primary' indicator for __change_page_attr() ( which in turn also serves as a recursion terminator for
cpa_process_alias() ).
Untangle these by extending the use of CPA_NO_CHECK_ALIAS to all callsites that currently use .checkalias=0 for this purpose.
--- arch/x86/mm/pat/set_memory.c | 30 +++++++++++------------------- 1 file changed, 11 insertions(+), 19 deletions(-)
arch/x86/mm/pat/set_memory.c +11 -19
@@ -1721,7 +1721,7 @@ static int __change_page_attr_set_clr(stif (ret) goto out; -if (checkalias) { +if (checkalias && !(cpa->flags &CPA_NO_CHECK_ALIAS )) { ret =cpa_process_alias (cpa);if (ret) goto out;
@@ -1795,18 +1795,12 @@ static int change_page_attr_set_clr(unsi cpa.numpages = numpages; cpa.mask_set = mask_set; cpa.mask_clr = mask_clr; - cpa.flags =0 ; + cpa.flags = in_flag; cpa.curpage =0 ; cpa.force_split = force_split; -if (in_flag & (CPA_ARRAY |CPA_PAGES_ARRAY )) - cpa.flags |= in_flag; -/* No alias checking for _NX bit modifications */ checkalias = (pgprot_val (mask_set) |pgprot_val (mask_clr)) != _PAGE_NX; -/* Has caller explicitly disabled alias checking? */ -if (in_flag &CPA_NO_CHECK_ALIAS ) - checkalias =0 ; ret =__change_page_attr_set_clr (&cpa, checkalias);
@@ -2061,11 +2055,9 @@ int set_memory_np(unsigned long addr, inint set_memory_np_noalias (unsigned long addr,int numpages) { -int cpa_flags =CPA_NO_CHECK_ALIAS ; -return change_page_attr_set_clr (&addr, numpages,__pgprot (0 ),__pgprot (_PAGE_PRESENT),0 , - cpa_flags,NULL ); +CPA_NO_CHECK_ALIAS ,NULL ); }int set_memory_4k (unsigned long addr,int numpages)
@@ -2282,7 +2274,7 @@ static int __set_pages_p(struct page *pa .numpages = numpages, .mask_set = __pgprot(_PAGE_PRESENT | _PAGE_RW), .mask_clr = __pgprot(0), - .flags = 0}; + .flags = CPA_NO_CHECK_ALIAS };/* * No alias checking needed for setting present flag. otherwise,
@@ -2290,7 +2282,7 @@ static int __set_pages_p(struct page *pa* mappings (this adds to complexity if we want to do this from * atomic context especially). Let's keep it simple! */ -return __change_page_attr_set_clr (&cpa,0 ); +return __change_page_attr_set_clr (&cpa,1 ); }static int __set_pages_np (struct page *page,int numpages)
@@ -2301,7 +2293,7 @@ static int __set_pages_np(struct page *p .numpages = numpages, .mask_set = __pgprot(0), .mask_clr = __pgprot(_PAGE_PRESENT | _PAGE_RW), - .flags = 0}; + .flags = CPA_NO_CHECK_ALIAS };/* * No alias checking needed for setting not present flag. otherwise,
@@ -2309,7 +2301,7 @@ static int __set_pages_np(struct page *p* mappings (this adds to complexity if we want to do this from * atomic context especially). Let's keep it simple! */ -return __change_page_attr_set_clr (&cpa,0 ); +return __change_page_attr_set_clr (&cpa,1 ); }int set_direct_map_invalid_noflush (struct page *page)
@@ -2380,7 +2372,7 @@ int __init kernel_map_pages_in_pgd(pgd_t .numpages = numpages, .mask_set = __pgprot(0), .mask_clr = __pgprot(~page_flags & (_PAGE_NX|_PAGE_RW)), - .flags = 0, + .flags = CPA_NO_CHECK_ALIAS, };WARN_ONCE (num_online_cpus () >1 ,"Don't call after initializing SMP" );
@@ -2393,7 +2385,7 @@ int __init kernel_map_pages_in_pgd(pgd_t cpa.mask_set =__pgprot (_PAGE_PRESENT | page_flags); - retval =__change_page_attr_set_clr (&cpa,0 ); + retval =__change_page_attr_set_clr (&cpa,1 );__flush_tlb_all (); out:
@@ -2423,12 +2415,12 @@ int __init kernel_unmap_pages_in_pgd(pgd .numpages = numpages, .mask_set = __pgprot(0), .mask_clr = __pgprot(_PAGE_PRESENT | _PAGE_RW), - .flags = 0, + .flags = CPA_NO_CHECK_ALIAS, };WARN_ONCE (num_online_cpus () >1 ,"Don't call after initializing SMP" ); - retval =__change_page_attr_set_clr (&cpa,0 ); + retval =__change_page_attr_set_clr (&cpa,1 );__flush_tlb_all ();return retval;
Peter Zijlstra
maintainerPATCH 3/4 x86/mm: Inhibit _PAGE_NX changes from cpa_process_alias()
In reply to Peter ZijlstraThere is a cludge in change_page_attr_set_clr() that inhibits propagating NX changes to the aliases (directmap and highmap) -- this is a cludge twofold:
- it also inhibits the primary checks in __change_page_attr();
- it hard depends on single bit changes.
The introduction of set_memory_rox() triggered this last issue for clearing both _PAGE_RW and _PAGE_NX.
Explicitly ignore _PAGE_NX in cpa_process_alias() instead.
--- arch/x86/mm/pat/set_memory.c | 28 +++++++++++++++++++++++----- 1 file changed, 23 insertions(+), 5 deletions(-)
arch/x86/mm/pat/set_memory.c +23 -5
@@ -1663,6 +1663,12 @@ static int cpa_process_alias(struct cpa_ alias_cpa.flags &= ~(CPA_PAGES_ARRAY |CPA_ARRAY ); alias_cpa.curpage =0 ; +/* Directmap always has NX set, do not modify. */ +if (__supported_pte_mask & _PAGE_NX) { + alias_cpa.mask_clr.pgprot &= ~_PAGE_NX; + alias_cpa.mask_set.pgprot &= ~_PAGE_NX; + } + cpa->force_flush_all =1 ; ret =__change_page_attr_set_clr (&alias_cpa,0 );
@@ -1685,6 +1691,15 @@ static int cpa_process_alias(struct cpa_ alias_cpa.flags &= ~(CPA_PAGES_ARRAY |CPA_ARRAY ); alias_cpa.curpage =0 ; +/* +* [_text, _brk_end) also covers data, do not modify NX except +* in cases where the highmap is the primary target. +*/ +if (__supported_pte_mask & _PAGE_NX) { + alias_cpa.mask_clr.pgprot &= ~_PAGE_NX; + alias_cpa.mask_set.pgprot &= ~_PAGE_NX; + } + cpa->force_flush_all =1 ;/* * The high mapping range is imprecise, so ignore the
@@ -1703,6 +1718,12 @@ static int __change_page_attr_set_clr(stunsigned long rempages = numpages;int ret =0 ; +/* +* No changes, easy! +*/ +if (!(pgprot_val (cpa->mask_set) |pgprot_val (cpa->mask_clr))) +return ret; +while (rempages) {/* * Store the remaining nr of pages for the large page
@@ -1749,7 +1770,7 @@ static int change_page_attr_set_clr(unsi struct page **pages) {struct cpa_data cpa; -int ret, cache, checkalias; +int ret, cache;memset (&cpa,0 ,sizeof (cpa));
@@ -1799,10 +1820,7 @@ static int change_page_attr_set_clr(unsi cpa.curpage =0 ; cpa.force_split = force_split; -/* No alias checking for _NX bit modifications */ - checkalias = (pgprot_val (mask_set) |pgprot_val (mask_clr)) != _PAGE_NX; - - ret =__change_page_attr_set_clr (&cpa, checkalias); + ret =__change_page_attr_set_clr (&cpa,1 );/* * Check whether we really changed something:
Re: [PATCH 3/4] x86/mm: Inhibit _PAGE_NX changes from cpa_process_alias()
Quoted 10 lines · Peter Zijlstra
There is a cludge in
change_page_attr_set_clr()that inhibits propagating NX changes to the aliases (directmap and highmap) -- this is a cludge twofold:- it also inhibits the primary checks in
__change_page_attr();
- it hard depends on single bit changes.The introduction of
set_memory_rox()triggered this last issue for clearing both_PAGE_RWand_PAGE_NX.
Quoted 6 lines
Explicitly ignore
_PAGE_NXincpa_process_alias()instead.Fixes b38994948567 ("x86/mm: Implement native set_memory_rox()")Reported-by kernel test robot <[email protected]>Debugged-by Dave Hansen <[email protected]>Signed-off-by Peter Zijlstra (Intel) <[email protected]>
Looks sane. It's also nice how the checkalias stuff continues to fall out of there.
Peter Zijlstra
maintainerPATCH 4/4 x86/mm: Rename __change_page_attr_set_clr(.checkalias)
In reply to Peter ZijlstraNow that the checkalias functionality is taken by CPA_NO_CHECK_ALIAS rename the argument to better match is remaining purpose: primary, matching __change_page_attr().
--- arch/x86/mm/pat/set_memory.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
arch/x86/mm/pat/set_memory.c +4 -4
@@ -1635,7 +1635,7 @@ static int __change_page_attr(struct cpareturn err; } -static int __change_page_attr_set_clr (struct cpa_data *cpa,int checkalias); +static int __change_page_attr_set_clr (struct cpa_data *cpa,int primary);/* * Check the directmap and "high kernel map" 'aliases'.
@@ -1712,7 +1712,7 @@ static int cpa_process_alias(struct cpa_return 0 ; } -static int __change_page_attr_set_clr (struct cpa_data *cpa,int checkalias) +static int __change_page_attr_set_clr (struct cpa_data *cpa,int primary) {unsigned long numpages = cpa->numpages;unsigned long rempages = numpages;
@@ -1736,13 +1736,13 @@ static int __change_page_attr_set_clr(stif (!debug_pagealloc_enabled ())spin_lock (&cpa_lock); - ret =__change_page_attr (cpa, checkalias); + ret =__change_page_attr (cpa, primary);if (!debug_pagealloc_enabled ())spin_unlock (&cpa_lock);if (ret) goto out; -if (checkalias && !(cpa->flags &CPA_NO_CHECK_ALIAS )) { +if (primary && !(cpa->flags &CPA_NO_CHECK_ALIAS )) { ret =cpa_process_alias (cpa);if (ret) goto out;