x86/module: rework ROX cache to avoid writable copy
Subsystems (13)
Mike Rapoport
maintainerv3 cover x86/module: rework ROX cache to avoid writable copy
From: "Mike Rapoport (Microsoft)" <[email protected]>
Hi,
Following Peter's comments [1] these patches rework handling of ROX caches for module text allocations.
Instead of using a writable copy that really complicates alternatives patching, temporarily remap parts of a large ROX page as RW for the time of module formation and then restore it's ROX protections when the module is ready.
To keep the ROX memory mapped with large pages, make set_memory_rox() capable of restoring large pages (more details are in patch 3).
Since this is really about x86, I believe this should go in via tip tree.
The patches also available in git
https://git.kernel.org/rppt/h/execmem/x86-rox/v10
v3 changes:
* instead of adding a new module state handle ROX restoration locally in load_module() as Petr suggested
v2: https://lore.kernel.org/all/[email protected]
* only collapse large mappings in set_memory_rox()
* simplify RW <-> ROX remapping
* don't remove ROX cache pages from the direct map (patch 4)
v1: https://lore.kernel.org/all/[email protected]
[1] https://lore.kernel.org/all/[email protected]
Kirill A. Shutemov (1):
x86/mm/pat: restore large ROX pages after fragmentation
Mike Rapoport (Microsoft) (8):
x86/mm/pat: cpa-test: fix length for CPA_ARRAY test x86/mm/pat: drop duplicate variable in cpa_flush()
execmem: don't remove ROX cache from the direct map
execmem: add API for temporal remapping as RW and restoring ROX afterwards
module: switch to execmem API for remapping as RW and restoring ROX Revert "x86/module: prepare module loading for ROX allocations of text"
module: drop unused module_writable_address()
x86: re-enable EXECMEM_ROX support
arch/um/kernel/um_arch.c | 11 +- arch/x86/Kconfig | 1 + arch/x86/entry/vdso/vma.c | 3 +- arch/x86/include/asm/alternative.h | 14 +- arch/x86/include/asm/pgtable_types.h | 2 + arch/x86/kernel/alternative.c | 181 +++++++++------------- arch/x86/kernel/ftrace.c | 30 ++-- arch/x86/kernel/module.c | 45 ++---- arch/x86/mm/pat/cpa-test.c | 2 +- arch/x86/mm/pat/set_memory.c | 220 ++++++++++++++++++++++++++- include/linux/execmem.h | 31 ++++ include/linux/module.h | 16 -- include/linux/moduleloader.h | 4 - include/linux/vm_event_item.h | 2 + kernel/module/main.c | 78 +++------- kernel/module/strict_rwx.c | 9 +- mm/execmem.c | 39 +++-- mm/vmstat.c | 2 + 18 files changed, 422 insertions(+), 268 deletions(-)
base-commit: ffd294d346d185b70e28b1a28abe367bbfe53c04
Signature · 1 line
2.45.2
Mike Rapoport
maintainerPATCH v3 1/9 x86/mm/pat: cpa-test: fix length for CPA_ARRAY test
From: "Mike Rapoport (Microsoft)" <[email protected]>
The CPA_ARRAY test always uses len[1] as numpages argument to change_page_attr_set() although the addresses array is different each iteration of the test loop.
Replace len[1] with len[i] to have numpages matching the addresses array.
--- arch/x86/mm/pat/cpa-test.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/mm/pat/cpa-test.c +1 -1
@@ -183,7 +183,7 @@ static int pageattr_test(void)break ;case 1 : - err =change_page_attr_set (addrs, len[1 ],PAGE_CPA_TEST ,1 ); + err =change_page_attr_set (addrs, len[i],PAGE_CPA_TEST ,1 );break ;case 2 :
Signature · 1 line
2.45.2
[tip: x86/mm] x86/mm/pat: cpa-test: fix length for CPA_ARRAY test
The following commit has been merged into the x86/mm branch of tip:
The following commit has been merged into the x86/mm branch of tip:
Commit-ID: 33ea120582a638b2f2e380a50686c2b1d7cce795 Gitweb: https://git.kernel.org/tip/33ea120582a638b2f2e380a50686c2b1d7cce795 Author: Mike Rapoport (Microsoft) <[email protected]> AuthorDate: Sun, 26 Jan 2025 09:47:25 +02:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Mon, 03 Feb 2025 11:46:01 +01:00
x86/mm/pat: cpa-test: fix length for CPA_ARRAY test
The CPA_ARRAY test always uses len[1] as numpages argument to change_page_attr_set() although the addresses array is different each iteration of the test loop.
Replace len[1] with len[i] to have numpages matching the addresses array.
--- arch/x86/mm/pat/cpa-test.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/mm/pat/cpa-test.c +1 -1
@@ -183,7 +183,7 @@ static int pageattr_test(void)break ;case 1 : - err =change_page_attr_set (addrs, len[1 ],PAGE_CPA_TEST ,1 ); + err =change_page_attr_set (addrs, len[i],PAGE_CPA_TEST ,1 );break ;case 2 :
Mike Rapoport
maintainerPATCH v3 2/9 x86/mm/pat: drop duplicate variable in cpa_flush()
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
There is a 'struct cpa_data *data' parameter in cpa_flush() that is assigned to a local 'struct cpa_data *cpa' variable.
Rename the parameter from 'data' to 'cpa' and drop declaration of the local 'cpa' variable.
--- arch/x86/mm/pat/set_memory.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
arch/x86/mm/pat/set_memory.c +1 -2
@@ -396,9 +396,8 @@ static void __cpa_flush_tlb(void *data)flush_tlb_one_kernel (fix_addr (__cpa_addr (cpa, i))); } -static void cpa_flush (struct cpa_data *data,int cache) +static void cpa_flush (struct cpa_data *cpa,int cache) { -struct cpa_data *cpa = data;unsigned int i;BUG_ON (irqs_disabled () && !early_boot_irqs_disabled);
Signature · 1 line
2.45.2
[tip: x86/mm] x86/mm/pat: drop duplicate variable in cpa_flush()
The following commit has been merged into the x86/mm branch of tip:
The following commit has been merged into the x86/mm branch of tip:
Commit-ID: 4ee788eb0781ba082709c1ac1d5146ebcc40b967 Gitweb: https://git.kernel.org/tip/4ee788eb0781ba082709c1ac1d5146ebcc40b967 Author: Mike Rapoport (Microsoft) <[email protected]> AuthorDate: Sun, 26 Jan 2025 09:47:26 +02:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Mon, 03 Feb 2025 11:46:01 +01:00
x86/mm/pat: drop duplicate variable in cpa_flush()
There is a 'struct cpa_data *data' parameter in cpa_flush() that is assigned to a local 'struct cpa_data *cpa' variable.
Rename the parameter from 'data' to 'cpa' and drop declaration of the local 'cpa' variable.
--- arch/x86/mm/pat/set_memory.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
arch/x86/mm/pat/set_memory.c +1 -2
@@ -394,9 +394,8 @@ static void __cpa_flush_tlb(void *data)flush_tlb_one_kernel (fix_addr (__cpa_addr (cpa, i))); } -static void cpa_flush (struct cpa_data *data,int cache) +static void cpa_flush (struct cpa_data *cpa,int cache) { -struct cpa_data *cpa = data;unsigned int i;BUG_ON (irqs_disabled () && !early_boot_irqs_disabled);
Mike Rapoport
maintainerPATCH v3 3/9 x86/mm/pat: restore large ROX pages after fragmentation
In reply to Mike RapoportFrom: "Kirill A. Shutemov" <[email protected]>
Change of attributes of the pages may lead to fragmentation of direct mapping over time and performance degradation when these pages contain executable code.
With current code it's one way road: kernel tries to avoid splitting large pages, but it doesn't restore them back even if page attributes got compatible again.
Any change to the mapping may potentially allow to restore large page.
Add a hook to cpa_flush() path that will check if the pages in the range that were just touched can be mapped at PMD level. If the collapse at the PMD level succeeded, also attempt to collapse PUD level.
The collapse logic runs only when a set_memory_ method explicitly sets CPA_COLLAPSE flag, for now this is only enabled in set_memory_rox().
CPUs don't like[1] to have to have TLB entries of different size for the same memory, but looks like it's okay as long as these entries have matching attributes[2]. Therefore it's critical to flush TLB before any following changes to the mapping.
Note that we already allow for multiple TLB entries of different sizes for the same memory now in split_large_page() path. It's not a new situation.
set_memory_4k() provides a way to use 4k pages on purpose. Kernel must not remap such pages as large. Re-use one of software PTE bits to indicate such pages.
[1] See Erratum 383 of AMD Family 10h Processors
[2] https://lore.kernel.org/linux-mm/[email protected]/
[[email protected]:
* s/restore/collapse/
* update formatting per peterz
* use 'struct ptdesc' instead of 'struct page' for list of page tables to be freed
* try to collapse PMD first and if it succeeds move on to PUD as peterz suggested
* flush TLB twice: for changes done in the original CPA call and after collapsing of large pages
* update commit message
]
--- arch/x86/include/asm/pgtable_types.h | 2 + arch/x86/mm/pat/set_memory.c | 217 ++++++++++++++++++++++++++- include/linux/vm_event_item.h | 2 + mm/vmstat.c | 2 + 4 files changed, 219 insertions(+), 4 deletions(-)
arch/x86/include/asm/pgtable_types.h +2 -0
@@ -33,6 +33,7 @@#define _PAGE_BIT_CPA_TEST _PAGE_BIT_SOFTW1#define _PAGE_BIT_UFFD_WP _PAGE_BIT_SOFTW2/* userfaultfd wrprotected */ #define _PAGE_BIT_SOFT_DIRTY _PAGE_BIT_SOFTW3/* software dirty tracking */ +#define _PAGE_BIT_KERNEL_4K _PAGE_BIT_SOFTW3/* page must not be converted to large */ #define _PAGE_BIT_DEVMAP _PAGE_BIT_SOFTW4#ifdef CONFIG_X86_64
@@ -64,6 +65,7 @@#define _PAGE_PAT_LARGE (_AT(pteval_t, 1) << _PAGE_BIT_PAT_LARGE)#define _PAGE_SPECIAL (_AT(pteval_t, 1) << _PAGE_BIT_SPECIAL)#define _PAGE_CPA_TEST (_AT(pteval_t, 1) << _PAGE_BIT_CPA_TEST) +#define _PAGE_KERNEL_4K (_AT(pteval_t, 1) << _PAGE_BIT_KERNEL_4K)#ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS #define _PAGE_PKEY_BIT0 (_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT0)#define _PAGE_PKEY_BIT1 (_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT1)
arch/x86/mm/pat/set_memory.c +213 -4
277 lines not on this page · raw message
include/linux/vm_event_item.h +2 -0
@@ -151,6 +151,8 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,#ifdef CONFIG_X86 DIRECT_MAP_LEVEL2_SPLIT ,DIRECT_MAP_LEVEL3_SPLIT , +DIRECT_MAP_LEVEL2_COLLAPSE , +DIRECT_MAP_LEVEL3_COLLAPSE , #endif #ifdef CONFIG_PER_VMA_LOCK_STATS VMA_LOCK_SUCCESS,
mm/vmstat.c +2 -0
@@ -1435,6 +1435,8 @@ const char * const vmstat_text[] = {#ifdef CONFIG_X86 "direct_map_level2_splits" ,"direct_map_level3_splits" , +"direct_map_level2_collapses" , +"direct_map_level3_collapses" , #endif #ifdef CONFIG_PER_VMA_LOCK_STATS "vma_lock_success",
Signature · 1 line
2.45.2
[tip: x86/mm] x86/mm/pat: restore large ROX pages after fragmentation
The following commit has been merged into the x86/mm branch of tip:
The following commit has been merged into the x86/mm branch of tip:
Commit-ID: 41d88484c71cd4f659348da41b7b5b3dbd3be1f6 Gitweb: https://git.kernel.org/tip/41d88484c71cd4f659348da41b7b5b3dbd3be1f6 Author: Kirill A. Shutemov <[email protected]> AuthorDate: Sun, 26 Jan 2025 09:47:27 +02:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Mon, 03 Feb 2025 11:46:01 +01:00
x86/mm/pat: restore large ROX pages after fragmentation
Change of attributes of the pages may lead to fragmentation of direct mapping over time and performance degradation when these pages contain executable code.
With current code it's one way road: kernel tries to avoid splitting large pages, but it doesn't restore them back even if page attributes got compatible again.
Any change to the mapping may potentially allow to restore large page.
Add a hook to cpa_flush() path that will check if the pages in the range that were just touched can be mapped at PMD level. If the collapse at the PMD level succeeded, also attempt to collapse PUD level.
The collapse logic runs only when a set_memory_ method explicitly sets CPA_COLLAPSE flag, for now this is only enabled in set_memory_rox().
CPUs don't like[1] to have to have TLB entries of different size for the same memory, but looks like it's okay as long as these entries have matching attributes[2]. Therefore it's critical to flush TLB before any following changes to the mapping.
Note that we already allow for multiple TLB entries of different sizes for the same memory now in split_large_page() path. It's not a new situation.
set_memory_4k() provides a way to use 4k pages on purpose. Kernel must not remap such pages as large. Re-use one of software PTE bits to indicate such pages.
[1] See Erratum 383 of AMD Family 10h Processors
[2] https://lore.kernel.org/linux-mm/[email protected]/
[[email protected]:
* s/restore/collapse/
* update formatting per peterz
* use 'struct ptdesc' instead of 'struct page' for list of page tables to be freed
* try to collapse PMD first and if it succeeds move on to PUD as peterz suggested
* flush TLB twice: for changes done in the original CPA call and after collapsing of large pages
* update commit message
]
--- arch/x86/include/asm/pgtable_types.h | 2 +- arch/x86/mm/pat/set_memory.c | 217 +++++++++++++++++++++++++- include/linux/vm_event_item.h | 2 +- mm/vmstat.c | 2 +- 4 files changed, 219 insertions(+), 4 deletions(-)
arch/x86/include/asm/pgtable_types.h +2 -0
@@ -33,6 +33,7 @@#define _PAGE_BIT_CPA_TEST _PAGE_BIT_SOFTW1#define _PAGE_BIT_UFFD_WP _PAGE_BIT_SOFTW2/* userfaultfd wrprotected */ #define _PAGE_BIT_SOFT_DIRTY _PAGE_BIT_SOFTW3/* software dirty tracking */ +#define _PAGE_BIT_KERNEL_4K _PAGE_BIT_SOFTW3/* page must not be converted to large */ #define _PAGE_BIT_DEVMAP _PAGE_BIT_SOFTW4#ifdef CONFIG_X86_64
@@ -64,6 +65,7 @@#define _PAGE_PAT_LARGE (_AT(pteval_t, 1) << _PAGE_BIT_PAT_LARGE)#define _PAGE_SPECIAL (_AT(pteval_t, 1) << _PAGE_BIT_SPECIAL)#define _PAGE_CPA_TEST (_AT(pteval_t, 1) << _PAGE_BIT_CPA_TEST) +#define _PAGE_KERNEL_4K (_AT(pteval_t, 1) << _PAGE_BIT_KERNEL_4K)#ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS #define _PAGE_PKEY_BIT0 (_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT0)#define _PAGE_PKEY_BIT1 (_AT(pteval_t, 1) << _PAGE_BIT_PKEY_BIT1)
arch/x86/mm/pat/set_memory.c +213 -4
277 lines not on this page · raw message
include/linux/vm_event_item.h +2 -0
@@ -151,6 +151,8 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,#ifdef CONFIG_X86 DIRECT_MAP_LEVEL2_SPLIT ,DIRECT_MAP_LEVEL3_SPLIT , +DIRECT_MAP_LEVEL2_COLLAPSE , +DIRECT_MAP_LEVEL3_COLLAPSE , #endif #ifdef CONFIG_PER_VMA_LOCK_STATS VMA_LOCK_SUCCESS,
mm/vmstat.c +2 -0
@@ -1435,6 +1435,8 @@ const char * const vmstat_text[] = {#ifdef CONFIG_X86 "direct_map_level2_splits" ,"direct_map_level3_splits" , +"direct_map_level2_collapses" , +"direct_map_level3_collapses" , #endif #ifdef CONFIG_PER_VMA_LOCK_STATS "vma_lock_success",
Mike Rapoport
maintainerPATCH v3 4/9 execmem: don't remove ROX cache from the direct map
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
The memory allocated for the ROX cache was removed from the direct map to reduce amount of direct map updates, however this cannot be tolerated by /proc/kcore that accesses module memory using vread_iter() and the latter does vmalloc_to_page() and copy_page_to_iter_nofault().
Instead of removing ROX cache memory from the direct map and mapping it as ROX in vmalloc space, simply call set_memory_rox() that will take care of proper permissions on both vmalloc and in the direct map.
--- mm/execmem.c | 17 ++++------------- 1 file changed, 4 insertions(+), 13 deletions(-)
mm/execmem.c +4 -13
@@ -257,7 +257,6 @@ static void *__execmem_cache_alloc(struct execmem_range *range, size_t size)static int execmem_cache_populate (struct execmem_range *range,size_t size) {unsigned long vm_flags =VM_ALLOW_HUGE_VMAP ; -unsigned long start, end;struct vm_struct *vm;size_t alloc_size;int err = -ENOMEM ;
@@ -275,26 +274,18 @@ static int execmem_cache_populate(struct execmem_range *range, size_t size)/* fill memory with instructions that will trap */ execmem_fill_trapping_insns (p, alloc_size,/* writable = */ true); - start = (unsigned long )p; - end = start + alloc_size; - -vunmap_range (start, end); - - err =execmem_set_direct_map_valid (vm, false); -if (err) - goto err_free_mem; - - err =vmap_pages_range_noflush (start, end, range->pgprot, vm->pages, -PMD_SHIFT ); + err =set_memory_rox ((unsigned long )p, vm->nr_pages);if (err) goto err_free_mem; err =execmem_cache_add (p, alloc_size);if (err) - goto err_free_mem; + goto err_reset_direct_map;return 0 ; +err_reset_direct_map: +execmem_set_direct_map_valid (vm, true); err_free_mem:vfree (p);return err;
Signature · 1 line
2.45.2
[tip: x86/mm] execmem: don't remove ROX cache from the direct map
The following commit has been merged into the x86/mm branch of tip:
The following commit has been merged into the x86/mm branch of tip:
Commit-ID: 925f426451182a9f3e0f7f0e7e928f32f81a966a Gitweb: https://git.kernel.org/tip/925f426451182a9f3e0f7f0e7e928f32f81a966a Author: Mike Rapoport (Microsoft) <[email protected]> AuthorDate: Sun, 26 Jan 2025 09:47:28 +02:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Mon, 03 Feb 2025 11:46:01 +01:00
execmem: don't remove ROX cache from the direct map
The memory allocated for the ROX cache was removed from the direct map to reduce amount of direct map updates, however this cannot be tolerated by /proc/kcore that accesses module memory using vread_iter() and the latter does vmalloc_to_page() and copy_page_to_iter_nofault().
Instead of removing ROX cache memory from the direct map and mapping it as ROX in vmalloc space, simply call set_memory_rox() that will take care of proper permissions on both vmalloc and in the direct map.
--- mm/execmem.c | 17 ++++------------- 1 file changed, 4 insertions(+), 13 deletions(-)
mm/execmem.c +4 -13
@@ -257,7 +257,6 @@ out_unlock:static int execmem_cache_populate (struct execmem_range *range,size_t size) {unsigned long vm_flags =VM_ALLOW_HUGE_VMAP ; -unsigned long start, end;struct vm_struct *vm;size_t alloc_size;int err = -ENOMEM ;
@@ -275,26 +274,18 @@ static int execmem_cache_populate(struct execmem_range *range, size_t size)/* fill memory with instructions that will trap */ execmem_fill_trapping_insns (p, alloc_size,/* writable = */ true); - start = (unsigned long )p; - end = start + alloc_size; - -vunmap_range (start, end); - - err =execmem_set_direct_map_valid (vm, false); -if (err) - goto err_free_mem; - - err =vmap_pages_range_noflush (start, end, range->pgprot, vm->pages, -PMD_SHIFT ); + err =set_memory_rox ((unsigned long )p, vm->nr_pages);if (err) goto err_free_mem; err =execmem_cache_add (p, alloc_size);if (err) - goto err_free_mem; + goto err_reset_direct_map;return 0 ; +err_reset_direct_map: +execmem_set_direct_map_valid (vm, true); err_free_mem:vfree (p);return err;
Mike Rapoport
maintainerPATCH v3 5/9 execmem: add API for temporal remapping as RW and restoring ROX afterwards
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Using a writable copy for ROX memory is cumbersome and error prone.
Add API that allow temporarily remapping of ranges in the ROX cache as writable and then restoring their read-only-execute permissions.
This API will be later used in modules code and will allow removing nasty games with writable copy in alternatives patching on x86.
The restoring of the ROX permissions relies on the ability of architecture to reconstruct large pages in its set_memory_rox() method.
--- include/linux/execmem.h | 31 +++++++++++++++++++++++++++++++ mm/execmem.c | 22 ++++++++++++++++++++++ 2 files changed, 53 insertions(+)
include/linux/execmem.h +31 -0
@@ -65,6 +65,37 @@ enum execmem_range_flags {* Architectures that use EXECMEM_ROX_CACHE must implement this. */ void execmem_fill_trapping_insns (void *ptr,size_t size,bool writable); + +/** +* execmem_make_temp_rw - temporarily remap region with read-write +* permissions +* @ptr: address of the region to remap +* @size: size of the region to remap +* +* Remaps a part of the cached large page in the ROX cache in the range +* [@ptr, @ptr + @size) as writable and not executable. The caller must +* have exclusive ownership of this range and ensure nothing will try to +* execute code in this range. +* +* Return: 0 on success or negative error code on failure. +*/ +int execmem_make_temp_rw (void *ptr,size_t size); + +/** +* execmem_restore_rox - restore read-only-execute permissions +* @ptr: address of the region to remap +* @size: size of the region to remap +* +* Restores read-only-execute permissions on a range [@ptr, @ptr + @size) +* after it was temporarily remapped as writable. Relies on architecture +* implementation of set_memory_rox() to restore mapping using large pages. +* +* Return: 0 on success or negative error code on failure. +*/ +int execmem_restore_rox (void *ptr,size_t size); +#else +static inline int execmem_make_temp_rw (void *ptr,size_t size) {return 0 ; } +static inline int execmem_restore_rox (void *ptr,size_t size) {return 0 ; }#endif /**
mm/execmem.c +22 -0
@@ -335,6 +335,28 @@ static bool execmem_cache_free(void *ptr) return true; } + +int execmem_make_temp_rw (void *ptr,size_t size) +{ +unsigned int nr =PAGE_ALIGN (size) >>PAGE_SHIFT ; +unsigned long addr = (unsigned long )ptr; +int ret; + + ret =set_memory_nx (addr, nr); +if (ret) +return ret; + +return set_memory_rw (addr, nr); +} + +int execmem_restore_rox (void *ptr,size_t size) +{ +unsigned int nr =PAGE_ALIGN (size) >>PAGE_SHIFT ; +unsigned long addr = (unsigned long )ptr; + +return set_memory_rox (addr, nr); +} +#else /* CONFIG_ARCH_HAS_EXECMEM_ROX */ static void *execmem_cache_alloc (struct execmem_range *range,size_t size) {
Signature · 1 line
2.45.2
[tip: x86/mm] execmem: add API for temporal remapping as RW and restoring ROX afterwards
The following commit has been merged into the x86/mm branch of tip:
The following commit has been merged into the x86/mm branch of tip:
Commit-ID: 05e555b817262b5df6aa3a73df8b3dc9d388a3b4 Gitweb: https://git.kernel.org/tip/05e555b817262b5df6aa3a73df8b3dc9d388a3b4 Author: Mike Rapoport (Microsoft) <[email protected]> AuthorDate: Sun, 26 Jan 2025 09:47:29 +02:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Mon, 03 Feb 2025 11:46:02 +01:00
execmem: add API for temporal remapping as RW and restoring ROX afterwards
Using a writable copy for ROX memory is cumbersome and error prone.
Add API that allow temporarily remapping of ranges in the ROX cache as writable and then restoring their read-only-execute permissions.
This API will be later used in modules code and will allow removing nasty games with writable copy in alternatives patching on x86.
The restoring of the ROX permissions relies on the ability of architecture to reconstruct large pages in its set_memory_rox() method.
--- include/linux/execmem.h | 31 +++++++++++++++++++++++++++++++ mm/execmem.c | 22 ++++++++++++++++++++++ 2 files changed, 53 insertions(+)
include/linux/execmem.h +31 -0
@@ -65,6 +65,37 @@ enum execmem_range_flags {* Architectures that use EXECMEM_ROX_CACHE must implement this. */ void execmem_fill_trapping_insns (void *ptr,size_t size,bool writable); + +/** +* execmem_make_temp_rw - temporarily remap region with read-write +* permissions +* @ptr: address of the region to remap +* @size: size of the region to remap +* +* Remaps a part of the cached large page in the ROX cache in the range +* [@ptr, @ptr + @size) as writable and not executable. The caller must +* have exclusive ownership of this range and ensure nothing will try to +* execute code in this range. +* +* Return: 0 on success or negative error code on failure. +*/ +int execmem_make_temp_rw (void *ptr,size_t size); + +/** +* execmem_restore_rox - restore read-only-execute permissions +* @ptr: address of the region to remap +* @size: size of the region to remap +* +* Restores read-only-execute permissions on a range [@ptr, @ptr + @size) +* after it was temporarily remapped as writable. Relies on architecture +* implementation of set_memory_rox() to restore mapping using large pages. +* +* Return: 0 on success or negative error code on failure. +*/ +int execmem_restore_rox (void *ptr,size_t size); +#else +static inline int execmem_make_temp_rw (void *ptr,size_t size) {return 0 ; } +static inline int execmem_restore_rox (void *ptr,size_t size) {return 0 ; }#endif /**
mm/execmem.c +22 -0
@@ -335,6 +335,28 @@ static bool execmem_cache_free(void *ptr) return true; } + +int execmem_make_temp_rw (void *ptr,size_t size) +{ +unsigned int nr =PAGE_ALIGN (size) >>PAGE_SHIFT ; +unsigned long addr = (unsigned long )ptr; +int ret; + + ret =set_memory_nx (addr, nr); +if (ret) +return ret; + +return set_memory_rw (addr, nr); +} + +int execmem_restore_rox (void *ptr,size_t size) +{ +unsigned int nr =PAGE_ALIGN (size) >>PAGE_SHIFT ; +unsigned long addr = (unsigned long )ptr; + +return set_memory_rox (addr, nr); +} +#else /* CONFIG_ARCH_HAS_EXECMEM_ROX */ static void *execmem_cache_alloc (struct execmem_range *range,size_t size) {
Mike Rapoport
maintainerPATCH v3 6/9 module: switch to execmem API for remapping as RW and restoring ROX
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
Instead of using writable copy for module text sections, temporarily remap the memory allocated from execmem's ROX cache as writable and restore its ROX permissions after the module is formed.
This will allow removing nasty games with writable copy in alternatives patching on x86.
--- include/linux/module.h | 8 +--- include/linux/moduleloader.h | 4 -- kernel/module/main.c | 78 ++++++++++-------------------------- kernel/module/strict_rwx.c | 9 +++-- 4 files changed, 27 insertions(+), 72 deletions(-)
include/linux/module.h +1 -7
@@ -367,7 +367,6 @@ enum mod_mem_type {struct module_memory {void *base; -void *rw_copy;bool is_rox;unsigned int size;
@@ -769,14 +768,9 @@ static inline bool is_livepatch_module(struct module *mod)void set_module_sig_enforced (void ); -void *__module_writable_address (struct module *mod,void *loc); -static inline void *module_writable_address (struct module *mod,void *loc) { -if (!IS_ENABLED (CONFIG_ARCH_HAS_EXECMEM_ROX ) || !mod || - mod->state !=MODULE_STATE_UNFORMED ) -return loc; -return __module_writable_address (mod, loc); +return loc; }#else /* !CONFIG_MODULES... */
include/linux/moduleloader.h +0 -4
@@ -108,10 +108,6 @@ int module_finalize(const Elf_Ehdr *hdr,const Elf_Shdr *sechdrs, struct module *mod); -int module_post_finalize (const Elf_Ehdr *hdr, -const Elf_Shdr *sechdrs, -struct module *mod); -#ifdef CONFIG_MODULES void flush_module_init_free_work (void );#else
kernel/module/main.c +21 -57
164 lines not on this page · raw message
kernel/module/strict_rwx.c +5 -4
@@ -9,6 +9,7 @@#include <linux/mm.h> #include <linux/vmalloc.h> #include <linux/set_memory.h> +#include <linux/execmem.h> #include "internal.h" static int module_set_memory(const struct module *mod, enum mod_mem_type type,
@@ -32,12 +33,12 @@ static int module_set_memory(const struct module *mod, enum mod_mem_type type,int module_enable_text_rox (const struct module *mod) { for_class_mod_mem_type(type, text) { +const struct module_memory *mem = &mod->mem[type];int ret; -if (mod->mem[type].is_rox) -continue ; - -if (IS_ENABLED (CONFIG_STRICT_MODULE_RWX )) +if (mem->is_rox) + ret =execmem_restore_rox (mem->base, mem->size); +else if (IS_ENABLED (CONFIG_STRICT_MODULE_RWX )) ret =module_set_memory (mod, type, set_memory_rox);else ret =module_set_memory (mod, type, set_memory_x);
Signature · 1 line
2.45.2
Petr Pavlu
maintainerRe: [PATCH v3 6/9] module: switch to execmem API for remapping as RW and restoring ROX
Quoted 10 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Instead of using writable copy for module text sections, temporarily remap the memory allocated from execmem's ROX cache as writable and restore its ROX permissions after the module is formed.
This will allow removing nasty games with writable copy in alternatives patching on x86.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
[...]
kernel/module/main.c static int module_memory_alloc(struct module *mod, enum mod_mem_type type) · line 1280
Quoted 10 lines
kernel/module/main.c +10
-0+static void module_memory_restore_rox (struct module *mod) +{ + for_class_mod_mem_type(type, text) { +struct module_memory *mem = &mod->mem[type]; + +if (mem->is_rox) +execmem_restore_rox (mem->base, mem->size); + } +} +
Can the execmem_restore_rox() call here fail? I realize that there isn't much that the module loader can do if that happens, but should it be perhaps logged as a warning?
Signature · 2 lines
Thanks, Petr
Mike Rapoport
maintainerRe: [PATCH v3 6/9] module: switch to execmem API for remapping as RW and restoring ROX
Quoted 14 lines · Petr Pavlu
Quoted 10 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Instead of using writable copy for module text sections, temporarily remap the memory allocated from execmem's ROX cache as writable and restore its ROX permissions after the module is formed.
This will allow removing nasty games with writable copy in alternatives patching on x86.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>[...]
Quoted 14 lines
kernel/module/main.c static int module_memory_alloc(struct module *mod, enum mod_mem_type type) · line 1280
Quoted 10 lines
kernel/module/main.c +10
-0+static void module_memory_restore_rox (struct module *mod) +{ + for_class_mod_mem_type(type, text) { +struct module_memory *mem = &mod->mem[type]; + +if (mem->is_rox) +execmem_restore_rox (mem->base, mem->size); + } +} +Can the
execmem_restore_rox()call here fail? I realize that there isn't much that the module loader can do if that happens, but should it be perhaps logged as a warning?
It won't fail at this point. set_memory APIs may fail if they need to split a large page and could not allocate a new page table, but here all the splits were already done at module_memory_alloc() time.
Quoted 3 lines
Signature · 2 lines
Thanks, Petr
Signature · 2 lines
Sincerely yours, Mike.
Re: [PATCH v3 6/9] module: switch to execmem API for remapping as RW and restoring ROX
Quoted 30 lines · Mike Rapoport
Quoted 14 lines · Petr Pavlu
Quoted 10 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Instead of using writable copy for module text sections, temporarily remap the memory allocated from execmem's ROX cache as writable and restore its ROX permissions after the module is formed.
This will allow removing nasty games with writable copy in alternatives patching on x86.
Signed-off-by Mike Rapoport (Microsoft) <[email protected]>[...]
Quoted 14 lines
kernel/module/main.c static int module_memory_alloc(struct module *mod, enum mod_mem_type type) · line 1280
Quoted 10 lines
kernel/module/main.c +10
-0+static void module_memory_restore_rox (struct module *mod) +{ + for_class_mod_mem_type(type, text) { +struct module_memory *mem = &mod->mem[type]; + +if (mem->is_rox) +execmem_restore_rox (mem->base, mem->size); + } +} +Can the
execmem_restore_rox()call here fail? I realize that there isn't much that the module loader can do if that happens, but should it be perhaps logged as a warning?
Quoted 3 lines
It won't fail at this point.
set_memoryAPIs may fail if they need to split a large page and could not allocate a new page table, but here all the splits were already done atmodule_memory_alloc()time.
Ok, thanks for the explanation.
-- Petr
[tip: x86/mm] module: switch to execmem API for remapping as RW and restoring ROX
In reply to Mike RapoportThe following commit has been merged into the x86/mm branch of tip:
The following commit has been merged into the x86/mm branch of tip:
Commit-ID: c287c072332905b7d878a8aade86cfef6b396343 Gitweb: https://git.kernel.org/tip/c287c072332905b7d878a8aade86cfef6b396343 Author: Mike Rapoport (Microsoft) <[email protected]> AuthorDate: Sun, 26 Jan 2025 09:47:30 +02:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Mon, 03 Feb 2025 11:46:02 +01:00
module: switch to execmem API for remapping as RW and restoring ROX
Instead of using writable copy for module text sections, temporarily remap the memory allocated from execmem's ROX cache as writable and restore its ROX permissions after the module is formed.
This will allow removing nasty games with writable copy in alternatives patching on x86.
--- include/linux/module.h | 8 +---- include/linux/moduleloader.h | 4 +-- kernel/module/main.c | 78 +++++++++-------------------------- kernel/module/strict_rwx.c | 9 ++-- 4 files changed, 27 insertions(+), 72 deletions(-)
include/linux/module.h +1 -7
@@ -367,7 +367,6 @@ enum mod_mem_type {struct module_memory {void *base; -void *rw_copy;bool is_rox;unsigned int size;
@@ -769,14 +768,9 @@ static inline bool is_livepatch_module(struct module *mod)void set_module_sig_enforced (void ); -void *__module_writable_address (struct module *mod,void *loc); -static inline void *module_writable_address (struct module *mod,void *loc) { -if (!IS_ENABLED (CONFIG_ARCH_HAS_EXECMEM_ROX ) || !mod || - mod->state !=MODULE_STATE_UNFORMED ) -return loc; -return __module_writable_address (mod, loc); +return loc; }#else /* !CONFIG_MODULES... */
include/linux/moduleloader.h +0 -4
@@ -108,10 +108,6 @@ int module_finalize(const Elf_Ehdr *hdr,const Elf_Shdr *sechdrs, struct module *mod); -int module_post_finalize (const Elf_Ehdr *hdr, -const Elf_Shdr *sechdrs, -struct module *mod); -#ifdef CONFIG_MODULES void flush_module_init_free_work (void );#else
kernel/module/main.c +21 -57
164 lines not on this page · raw message
kernel/module/strict_rwx.c +5 -4
@@ -9,6 +9,7 @@#include <linux/mm.h> #include <linux/vmalloc.h> #include <linux/set_memory.h> +#include <linux/execmem.h> #include "internal.h" static int module_set_memory(const struct module *mod, enum mod_mem_type type,
@@ -32,12 +33,12 @@ static int module_set_memory(const struct module *mod, enum mod_mem_type type,int module_enable_text_rox (const struct module *mod) { for_class_mod_mem_type(type, text) { +const struct module_memory *mem = &mod->mem[type];int ret; -if (mod->mem[type].is_rox) -continue ; - -if (IS_ENABLED (CONFIG_STRICT_MODULE_RWX )) +if (mem->is_rox) + ret =execmem_restore_rox (mem->base, mem->size); +else if (IS_ENABLED (CONFIG_STRICT_MODULE_RWX )) ret =module_set_memory (mod, type, set_memory_rox);else ret =module_set_memory (mod, type, set_memory_x);
Mike Rapoport
maintainerPATCH v3 7/9 Revert "x86/module: prepare module loading for ROX allocations of text"
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
The module code does not create a writable copy of the executable memory anymore so there is no need to handle it in module relocation and alternatives patching.
This reverts commit 9bfc4824fd4836c16bb44f922bfaffba5da3e4f3.
--- arch/um/kernel/um_arch.c | 11 +- arch/x86/entry/vdso/vma.c | 3 +- arch/x86/include/asm/alternative.h | 14 +-- arch/x86/kernel/alternative.c | 181 ++++++++++++----------------- arch/x86/kernel/ftrace.c | 30 +++-- arch/x86/kernel/module.c | 45 +++---- 6 files changed, 117 insertions(+), 167 deletions(-)
arch/um/kernel/um_arch.c +5 -6
@@ -440,25 +440,24 @@ void __init arch_cpu_finalize_init(void)os_check_bugs (); } -void apply_seal_endbr (s32 *start,s32 *end,struct module *mod) +void apply_seal_endbr (s32 *start,s32 *end) { } -void apply_retpolines (s32 *start,s32 *end,struct module *mod) +void apply_retpolines (s32 *start,s32 *end) { } -void apply_returns (s32 *start,s32 *end,struct module *mod) +void apply_returns (s32 *start,s32 *end) { }void apply_fineibt (s32 *start_retpoline,s32 *end_retpoline, -s32 *start_cfi,s32 *end_cfi,struct module *mod) +s32 *start_cfi,s32 *end_cfi) { } -void apply_alternatives (struct alt_instr *start,struct alt_instr *end, -struct module *mod) +void apply_alternatives (struct alt_instr *start,struct alt_instr *end) { }
arch/x86/entry/vdso/vma.c +1 -2
@@ -48,8 +48,7 @@ int __init init_vdso_image(const struct vdso_image *image)apply_alternatives ((struct alt_instr *)(image->data + image->alt), (struct alt_instr *)(image->data + image->alt + - image->alt_len), -NULL ); + image->alt_len));return 0 ; }
arch/x86/include/asm/alternative.h +7 -7
@@ -96,16 +96,16 @@ extern struct alt_instr __alt_instructions[], __alt_instructions_end[];* instructions were patched in already: */ extern int alternatives_patched; -struct module ;extern void alternative_instructions (void ); -extern void apply_alternatives (struct alt_instr *start,struct alt_instr *end, -struct module *mod); -extern void apply_retpolines (s32 *start,s32 *end,struct module *mod); -extern void apply_returns (s32 *start,s32 *end,struct module *mod); -extern void apply_seal_endbr (s32 *start,s32 *end,struct module *mod); +extern void apply_alternatives (struct alt_instr *start,struct alt_instr *end); +extern void apply_retpolines (s32 *start,s32 *end); +extern void apply_returns (s32 *start,s32 *end); +extern void apply_seal_endbr (s32 *start,s32 *end);extern void apply_fineibt (s32 *start_retpoline,s32 *end_retpoine, -s32 *start_cfi,s32 *end_cfi,struct module *mod); +s32 *start_cfi,s32 *end_cfi); + +struct module ;struct callthunk_sites {s32 *call_start, *call_end;
arch/x86/kernel/alternative.c +76 -105
535 lines not on this page · raw message
arch/x86/kernel/ftrace.c +14 -16
@@ -118,13 +118,10 @@ ftrace_modify_code_direct(unsigned long ip, const char *old_code,return ret;/* replace the text with the new text */ -if (ftrace_poke_late) { +if (ftrace_poke_late)text_poke_queue ((void *)ip, new_code,MCOUNT_INSN_SIZE ,NULL ); - }else { -mutex_lock (&text_mutex); -text_poke ((void *)ip, new_code,MCOUNT_INSN_SIZE ); -mutex_unlock (&text_mutex); - } +else +text_poke_early ((void *)ip, new_code,MCOUNT_INSN_SIZE );return 0 ; }
@@ -321,7 +318,7 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)unsigned const char op_ref[] = {0x48 ,0x8b ,0x15 };unsigned const char retq[] = {RET_INSN_OPCODE ,INT3_INSN_OPCODE };union ftrace_op_code_union op_ptr; -void *ret; +int ret;if (ops->flags &FTRACE_OPS_FL_SAVE_REGS ) { start_offset = (unsigned long )ftrace_regs_caller;
@@ -352,15 +349,15 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size) npages =DIV_ROUND_UP (*tramp_size,PAGE_SIZE );/* Copy ftrace_caller onto the trampoline memory */ - ret =text_poke_copy (trampoline, (void *)start_offset, size); -if (WARN_ON (!ret)) + ret =copy_from_kernel_nofault (trampoline, (void *)start_offset, size); +if (WARN_ON (ret <0 )) goto fail; ip = trampoline + size;if (cpu_feature_enabled (X86_FEATURE_RETHUNK ))__text_gen_insn (ip,JMP32_INSN_OPCODE , ip, x86_return_thunk,JMP32_INSN_SIZE );else -text_poke_copy (ip, retq,sizeof (retq)); +memcpy (ip, retq,sizeof (retq));/* No need to test direct calls on created trampolines */ if (ops->flags &FTRACE_OPS_FL_SAVE_REGS ) {
@@ -368,7 +365,8 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size) ip = trampoline + (jmp_offset - start_offset);if (WARN_ON (*(char *)ip !=0x75 )) goto fail; -if (!text_poke_copy (ip, x86_nops[2 ],2 )) + ret =copy_from_kernel_nofault (ip, x86_nops[2 ],2 ); +if (ret <0 ) goto fail; }
@@ -381,7 +379,7 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)*/ ptr = (unsigned long *)(trampoline + size +RET_SIZE ); -text_poke_copy (ptr, &ops,sizeof (unsigned long )); + *ptr = (unsigned long )ops; op_offset -= start_offset;memcpy (&op_ptr, trampoline + op_offset,OP_REF_SIZE );
@@ -397,7 +395,7 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size) op_ptr.offset = offset;/* put in the new offset to the ftrace_ops */ -text_poke_copy (trampoline + op_offset, &op_ptr,OP_REF_SIZE ); +memcpy (trampoline + op_offset, &op_ptr,OP_REF_SIZE );/* put in the call to the function */ mutex_lock (&text_mutex);
@@ -407,9 +405,9 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)* the depth accounting before the call already. */ dest =ftrace_ops_get_func (ops); -text_poke_copy_locked (trampoline + call_offset, -text_gen_insn (CALL_INSN_OPCODE , trampoline + call_offset, dest), -CALL_INSN_SIZE , false); +memcpy (trampoline + call_offset, +text_gen_insn (CALL_INSN_OPCODE , trampoline + call_offset, dest), +CALL_INSN_SIZE );mutex_unlock (&text_mutex);/* ALLOC_TRAMP flags lets us know we created it */
arch/x86/kernel/module.c +14 -31
102 lines not on this page · raw message
Signature · 1 line
2.45.2
[tip: x86/mm] Revert "x86/module: prepare module loading for ROX allocations of text"
The following commit has been merged into the x86/mm branch of tip:
The following commit has been merged into the x86/mm branch of tip:
Commit-ID: 1d7e707af446134dd272ea8a89018c63cc17bb6a Gitweb: https://git.kernel.org/tip/1d7e707af446134dd272ea8a89018c63cc17bb6a Author: Mike Rapoport (Microsoft) <[email protected]> AuthorDate: Sun, 26 Jan 2025 09:47:31 +02:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Mon, 03 Feb 2025 11:46:02 +01:00
Revert "x86/module: prepare module loading for ROX allocations of text"
The module code does not create a writable copy of the executable memory anymore so there is no need to handle it in module relocation and alternatives patching.
This reverts commit 9bfc4824fd4836c16bb44f922bfaffba5da3e4f3.
--- arch/um/kernel/um_arch.c | 11 +-- arch/x86/entry/vdso/vma.c | 3 +- arch/x86/include/asm/alternative.h | 14 +- arch/x86/kernel/alternative.c | 181 +++++++++++----------------- arch/x86/kernel/ftrace.c | 30 ++--- arch/x86/kernel/module.c | 45 ++----- 6 files changed, 117 insertions(+), 167 deletions(-)
arch/um/kernel/um_arch.c +5 -6
@@ -440,25 +440,24 @@ void __init arch_cpu_finalize_init(void)os_check_bugs (); } -void apply_seal_endbr (s32 *start,s32 *end,struct module *mod) +void apply_seal_endbr (s32 *start,s32 *end) { } -void apply_retpolines (s32 *start,s32 *end,struct module *mod) +void apply_retpolines (s32 *start,s32 *end) { } -void apply_returns (s32 *start,s32 *end,struct module *mod) +void apply_returns (s32 *start,s32 *end) { }void apply_fineibt (s32 *start_retpoline,s32 *end_retpoline, -s32 *start_cfi,s32 *end_cfi,struct module *mod) +s32 *start_cfi,s32 *end_cfi) { } -void apply_alternatives (struct alt_instr *start,struct alt_instr *end, -struct module *mod) +void apply_alternatives (struct alt_instr *start,struct alt_instr *end) { }
arch/x86/entry/vdso/vma.c +1 -2
@@ -48,8 +48,7 @@ int __init init_vdso_image(const struct vdso_image *image)apply_alternatives ((struct alt_instr *)(image->data + image->alt), (struct alt_instr *)(image->data + image->alt + - image->alt_len), -NULL ); + image->alt_len));return 0 ; }
arch/x86/include/asm/alternative.h +7 -7
@@ -87,16 +87,16 @@ extern struct alt_instr __alt_instructions[], __alt_instructions_end[];* instructions were patched in already: */ extern int alternatives_patched; -struct module ;extern void alternative_instructions (void ); -extern void apply_alternatives (struct alt_instr *start,struct alt_instr *end, -struct module *mod); -extern void apply_retpolines (s32 *start,s32 *end,struct module *mod); -extern void apply_returns (s32 *start,s32 *end,struct module *mod); -extern void apply_seal_endbr (s32 *start,s32 *end,struct module *mod); +extern void apply_alternatives (struct alt_instr *start,struct alt_instr *end); +extern void apply_retpolines (s32 *start,s32 *end); +extern void apply_returns (s32 *start,s32 *end); +extern void apply_seal_endbr (s32 *start,s32 *end);extern void apply_fineibt (s32 *start_retpoline,s32 *end_retpoine, -s32 *start_cfi,s32 *end_cfi,struct module *mod); +s32 *start_cfi,s32 *end_cfi); + +struct module ;struct callthunk_sites {s32 *call_start, *call_end;
arch/x86/kernel/alternative.c +76 -105
535 lines not on this page · raw message
arch/x86/kernel/ftrace.c +14 -16
@@ -118,13 +118,10 @@ ftrace_modify_code_direct(unsigned long ip, const char *old_code,return ret;/* replace the text with the new text */ -if (ftrace_poke_late) { +if (ftrace_poke_late)text_poke_queue ((void *)ip, new_code,MCOUNT_INSN_SIZE ,NULL ); - }else { -mutex_lock (&text_mutex); -text_poke ((void *)ip, new_code,MCOUNT_INSN_SIZE ); -mutex_unlock (&text_mutex); - } +else +text_poke_early ((void *)ip, new_code,MCOUNT_INSN_SIZE );return 0 ; }
@@ -321,7 +318,7 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)unsigned const char op_ref[] = {0x48 ,0x8b ,0x15 };unsigned const char retq[] = {RET_INSN_OPCODE ,INT3_INSN_OPCODE };union ftrace_op_code_union op_ptr; -void *ret; +int ret;if (ops->flags &FTRACE_OPS_FL_SAVE_REGS ) { start_offset = (unsigned long )ftrace_regs_caller;
@@ -352,15 +349,15 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size) npages =DIV_ROUND_UP (*tramp_size,PAGE_SIZE );/* Copy ftrace_caller onto the trampoline memory */ - ret =text_poke_copy (trampoline, (void *)start_offset, size); -if (WARN_ON (!ret)) + ret =copy_from_kernel_nofault (trampoline, (void *)start_offset, size); +if (WARN_ON (ret <0 )) goto fail; ip = trampoline + size;if (cpu_feature_enabled (X86_FEATURE_RETHUNK ))__text_gen_insn (ip,JMP32_INSN_OPCODE , ip, x86_return_thunk,JMP32_INSN_SIZE );else -text_poke_copy (ip, retq,sizeof (retq)); +memcpy (ip, retq,sizeof (retq));/* No need to test direct calls on created trampolines */ if (ops->flags &FTRACE_OPS_FL_SAVE_REGS ) {
@@ -368,7 +365,8 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size) ip = trampoline + (jmp_offset - start_offset);if (WARN_ON (*(char *)ip !=0x75 )) goto fail; -if (!text_poke_copy (ip, x86_nops[2 ],2 )) + ret =copy_from_kernel_nofault (ip, x86_nops[2 ],2 ); +if (ret <0 ) goto fail; }
@@ -381,7 +379,7 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)*/ ptr = (unsigned long *)(trampoline + size +RET_SIZE ); -text_poke_copy (ptr, &ops,sizeof (unsigned long )); + *ptr = (unsigned long )ops; op_offset -= start_offset;memcpy (&op_ptr, trampoline + op_offset,OP_REF_SIZE );
@@ -397,7 +395,7 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size) op_ptr.offset = offset;/* put in the new offset to the ftrace_ops */ -text_poke_copy (trampoline + op_offset, &op_ptr,OP_REF_SIZE ); +memcpy (trampoline + op_offset, &op_ptr,OP_REF_SIZE );/* put in the call to the function */ mutex_lock (&text_mutex);
@@ -407,9 +405,9 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size)* the depth accounting before the call already. */ dest =ftrace_ops_get_func (ops); -text_poke_copy_locked (trampoline + call_offset, -text_gen_insn (CALL_INSN_OPCODE , trampoline + call_offset, dest), -CALL_INSN_SIZE , false); +memcpy (trampoline + call_offset, +text_gen_insn (CALL_INSN_OPCODE , trampoline + call_offset, dest), +CALL_INSN_SIZE );mutex_unlock (&text_mutex);/* ALLOC_TRAMP flags lets us know we created it */
arch/x86/kernel/module.c +14 -31
102 lines not on this page · raw message
Mike Rapoport
maintainerPATCH v3 8/9 module: drop unused module_writable_address()
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
module_writable_address() is unused and can be removed.
--- include/linux/module.h | 10 ---------- 1 file changed, 10 deletions(-)
include/linux/module.h +0 -10
@@ -768,11 +768,6 @@ static inline bool is_livepatch_module(struct module *mod)void set_module_sig_enforced (void ); -static inline void *module_writable_address (struct module *mod,void *loc) -{ -return loc; -} -#else /* !CONFIG_MODULES... */ static inline struct module *__module_address (unsigned long addr)
@@ -880,11 +875,6 @@ static inline bool module_is_coming(struct module *mod) {return false; } - -static inline void *module_writable_address (struct module *mod,void *loc) -{ -return loc; -}#endif /* CONFIG_MODULES */ #ifdef CONFIG_SYSFS
Signature · 1 line
2.45.2
[tip: x86/mm] module: drop unused module_writable_address()
The following commit has been merged into the x86/mm branch of tip:
The following commit has been merged into the x86/mm branch of tip:
Commit-ID: 602df3712979de594d268e8cbca46a93126c977d Gitweb: https://git.kernel.org/tip/602df3712979de594d268e8cbca46a93126c977d Author: Mike Rapoport (Microsoft) <[email protected]> AuthorDate: Sun, 26 Jan 2025 09:47:32 +02:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Mon, 03 Feb 2025 11:46:03 +01:00
module: drop unused module_writable_address()
module_writable_address() is unused and can be removed.
--- include/linux/module.h | 10 ---------- 1 file changed, 10 deletions(-)
include/linux/module.h +0 -10
@@ -768,11 +768,6 @@ static inline bool is_livepatch_module(struct module *mod)void set_module_sig_enforced (void ); -static inline void *module_writable_address (struct module *mod,void *loc) -{ -return loc; -} -#else /* !CONFIG_MODULES... */ static inline struct module *__module_address (unsigned long addr)
@@ -880,11 +875,6 @@ static inline bool module_is_coming(struct module *mod) {return false; } - -static inline void *module_writable_address (struct module *mod,void *loc) -{ -return loc; -}#endif /* CONFIG_MODULES */ #ifdef CONFIG_SYSFS
Mike Rapoport
maintainerPATCH v3 9/9 x86: re-enable EXECMEM_ROX support
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
after rework of execmem ROX caches
--- arch/x86/Kconfig | 1 + 1 file changed, 1 insertion(+)
arch/x86/Kconfig +1 -0
@@ -83,6 +83,7 @@ config X86 select ARCH_HAS_DMA_OPS if GART_IOMMU || XEN select ARCH_HAS_EARLY_DEBUG if KGDB select ARCH_HAS_ELF_RANDOMIZE + select ARCH_HAS_EXECMEM_ROX if X86_64 select ARCH_HAS_FAST_MULTIPLIER select ARCH_HAS_FORTIFY_SOURCE select ARCH_HAS_GCOV_PROFILE_ALL
Signature · 1 line
2.45.2
[tip: x86/mm] x86: re-enable EXECMEM_ROX support
The following commit has been merged into the x86/mm branch of tip:
The following commit has been merged into the x86/mm branch of tip:
Commit-ID: 64f6a4e10c05ed527f0f24b7954964255e0d3535 Gitweb: https://git.kernel.org/tip/64f6a4e10c05ed527f0f24b7954964255e0d3535 Author: Mike Rapoport (Microsoft) <[email protected]> AuthorDate: Sun, 26 Jan 2025 09:47:33 +02:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Mon, 03 Feb 2025 11:46:03 +01:00
x86: re-enable EXECMEM_ROX support
after rework of execmem ROX caches
--- arch/x86/Kconfig | 1 + 1 file changed, 1 insertion(+)
arch/x86/Kconfig +1 -0
@@ -85,6 +85,7 @@ config X86 select ARCH_HAS_DMA_OPS if GART_IOMMU || XEN select ARCH_HAS_EARLY_DEBUG if KGDB select ARCH_HAS_ELF_RANDOMIZE + select ARCH_HAS_EXECMEM_ROX if X86_64 select ARCH_HAS_FAST_MULTIPLIER select ARCH_HAS_FORTIFY_SOURCE select ARCH_HAS_GCOV_PROFILE_ALL
Re: [PATCH v3 9/9] x86: re-enable EXECMEM_ROX support
In reply to Mike RapoportHi,
A kernel BUG in __text_poke() is observed inside a Ubuntu 26.04 guest (v7.0.0.28 and also with upstream v7.2-4c5). Host is running on v7.2-rc5.
SETUP
-----
512 QEMU Q35 guests (2 vCPUs, 1G RAM each) boot concurrently across 200 iterations. The crash is seen during module loading in early boot. 2-7 guests out of ~100,000 boot attempts crash with the kernel BUG.
Reproduced on:
- AMD EPYC, QEMU Q35 guests
- Intel Icelake, QEMU Q35 guests
Apparently it was seen once in the non-guest Ubuntu 26.04 environment as well
https://discourse.ubuntu.com/t/failed-boot-one-time-on-ubuntu-unity-26-04-educational-mission/85300
SYMPTOMS
--------
Either of the below two crash signatures is observed:
Signature 1 — BUG_ON(!pages[0]) in __text_poke():
kernel BUG at arch/x86/kernel/alternative.c:2564!
Oops: invalid opcode: 0000 [#1] SMP NOPTI CPU: 0 UID: 0 PID: 741 Comm: (udev-worker) Not tainted 7.0.0-28-generic #28-Ubuntu PREEMPT(lazy) Hardware name: QEMU Standard PC (Q35 + ICH9, 2009) RIP: 0010:__text_poke+0x2a4/0x400 Call Trace: <TASK> smp_text_poke_batch_finish+0x25f/0x5b0 smp_text_poke_single+0x13/0x30 __static_call_transform+0x10e/0x210 arch_static_call_transform+0x5c/0xc0 __static_call_init+0x154/0x300 static_call_module_notify+0x144/0x170 notifier_call_chain+0x62/0xf0 blocking_notifier_call_chain_robust+0x62/0xc0 load_module+0x5ef/0x960 Modules linked in: ppdev(+) kvm(+) irqbypass ...
Signature 2 — BUG_ON(memcmp()) in __text_poke():
kernel BUG at arch/x86/kernel/alternative.c:2632!
Oops: invalid opcode: 0000 [#1] SMP NOPTI CPU: 0 UID: 0 PID: 742 Comm: (udev-worker) Not tainted 7.0.0-28-generic #28-Ubuntu PREEMPT(lazy) Hardware name: QEMU Standard PC (Q35 + ICH9, 2009) RIP: 0010:__text_poke+0x380/0x400 Call Trace: <TASK> smp_text_poke_batch_finish+0x323/0x5b0 smp_text_poke_single+0x13/0x30 __static_call_transform+0x10e/0x210 arch_static_call_transform+0x5c/0xc0 __static_call_init+0x154/0x300 static_call_module_notify+0x144/0x170 notifier_call_chain+0x62/0xf0 blocking_notifier_call_chain_robust+0x62/0xc0 load_module+0x5ef/0x960 Modules linked in: kvm(+) irqbypass ...
INSTRUMENTATION
---------------
The following instrumentation was applied to capture the page table state at the moment of failure:
In mm/vmalloc.c::vmalloc_to_page()
ptep = pte_offset_kernel(pmd, addr);
pte = ptep_get(ptep);
if (pte_present(pte))
page = pte_page(pte);
else
pr_err("vmalloc_to_page: pte_not_present for addr=%lx "
"PGD=%lx P4D=%lx PUD=%lx PMD=%lx PTE=%lx\n",
addr, pgd_val(*pgd), p4d_val(*p4d), pud_val(*pud),
pmd_val(*pmd), pte_val(pte));In arch/x86/kernel/alternative.c::__text_poke()
/* BUG 1: pages[0] NULL */
if (!pages[0] || ...) {
pr_err("%s: vmalloc_to_page failed for addr=%pK pages[0]=%pK pages[1]=%pK cross=%d\n", ...);
BUG();
}
...
/* BUG 2: memcmp mismatch */
if (memcmp(addr, src, len)) {
struct page *page_now = vmalloc_to_page(addr);
pr_err("%s: memcmp failed at addr=%pK len=%zu\n", __func__, addr, len);
pr_err("%s: expected (src): %*ph\n", __func__, (int)len, src);
pr_err("%s: found (dst): %*ph\n", __func__, (int)len, (u8 *)addr);
pr_err("%s: pfn at poke time (vmalloc_to_page): %lx\n", __func__, page_to_pfn(pages[0]));
pr_err("%s: pfn at readback time (vmalloc_to_page): %lx\n", __func__, page_to_pfn(page_now));
text_poke_dump_pagetable((unsigned long)addr); /* reads CR3 */
BUG();
}Signature 1 — vmalloc_to_page() returned NULL:
vmalloc_to_page: pte_not_present for addr=ffffffffc0697979 PGD=28242067 P4D=28243067 PUD=28245067 PMD=3be001a1 PTE=0 SMP alternatives: __text_poke: vmalloc_to_page failed for addr=ffffffffc0697979 pages[0]=0000000000000000 pages[1]=0000000000000000 cross=0
PMD was not a leaf, and the code reached a point to check pte_present() and finds the PTE to be not present. But in the dump in else case, pmd_val(*pmd) shows that it is a PMD leaf (PSE bit set).
Signature 2 — vmalloc_to_page() returned a wrong page:
SMP alternatives: __text_poke: memcmp failed at addr=ffffffffc06fc4bd len=1 SMP alternatives: __text_poke: expected (src): cc
SMP alternatives: __text_poke: found (dst): e8
SMP alternatives: __text_poke: pfn at poke time (vmalloc_to_page): 3884a SMP alternatives: __text_poke: pfn at readback time (vmalloc_to_page): 52fc SMP alternatives: __text_poke: page table dump for addr:
PGD 1ba42067 P4D 1ba43067 PUD 1ba45067 PMD 52001a1
vmalloc_to_page() returned pfn=0x3884a when pages[0] was queried. The write of 0xcc landed there. When vmalloc_to_page() was called again during the memcmp readback in the instrumentation code, the vmap mapping resolved through PMD=52001a1 to a different pfn=0x52fc, which contains 0xe8 (existing instruction bytes). This is silent memory corruption — 0xcc was written to the wrong physical page (pfn=0x3884a).
BISECTION
---------
The following experiments all ran 200 iterations of 512 concurrent guest boots with zero crashes.
1. Remove "select ARCH_HAS_EXECMEM_ROX" from arch/x86/Kconfig
2. Boot with "nohugevmalloc"
3. Remove VM_ALLOW_HUGE_VMAP from vm_flags in execmem_cache_populate()
- unsigned long vm_flags = VM_ALLOW_HUGE_VMAP;
+ unsigned long vm_flags = 0;
4. Remove CPA_COLLAPSE from set_memory_rox()
return change_page_attr_set_clr(&addr, numpages, __pgprot(0), clr, 0,
- CPA_COLLAPSE, NULL);
+ 0, NULL);
Without CPA_COLLAPSE, set_memory_rox() changes 4K PTE permissions in-place and the PMD stays permanently as a PTE-page pointer after the first split — no re-collapse, no future race window. This confirms that CPA_COLLAPSE is a necessary part of the race sequence.
5. Hold pgd_lock around vmalloc_to_page() in __text_poke()
if (!core_kernel_text((unsigned long)addr)) {
+ spin_lock(&pgd_lock);
pages[0] = vmalloc_to_page(addr);
if (cross_page_boundary)
pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+ spin_unlock(&pgd_lock);
} else {
__split_large_page() holds pgd_lock while installing the new PMD and populating all 512 4K PTEs. Holding pgd_lock around vmalloc_to_page() prevents CPA from modifying the PMD concurrently with the page table walk. This confirms the race is precisely between vmalloc_to_page() reading the PMD/PTE and __split_large_page() modifying it under pgd_lock.
Note: this is a diagnostic experiment, not a production fix.
The experiments point to a race between vmalloc_to_page() in __text_poke() and __split_large_page() modifying the shared PMD under pgd_lock. __split_large_page() holds pgd_lock while modifying the PMD; vmalloc_to_page() holds no lock and can observe the PMD in an intermediate state.
What would be the correct fix for this issue?
Happy to test any proposed fix or run further experiments.
Regards,
Nikunj
Re: [PATCH v3 9/9] x86: re-enable EXECMEM_ROX support
Quoted 181 lines · Nikunj A Dadhania
Hi,
A kernel BUG in
__text_poke()is observed inside a Ubuntu 26.04 guest (v7.0.0.28 and also with upstream v7.2-4c5). Host is running on v7.2-rc5.SETUP
-----512 QEMU Q35 guests (2 vCPUs, 1G RAM each) boot concurrently across 200 iterations. The crash is seen during module loading in early boot. 2-7 guests out of ~100,000 boot attempts crash with the kernel BUG.
Reproduced on:
- AMD EPYC, QEMU Q35 guests
- Intel Icelake, QEMU Q35 guestsApparently it was seen once in the non-guest Ubuntu 26.04 environment as well
https://discourse.ubuntu.com/t/failed-boot-one-time-on-ubuntu-unity-26-04-educational-mission/85300SYMPTOMS
--------Either of the below two crash signatures is observed:
Signature 1 —
BUG_ON(!pages[0])in__text_poke():kernel BUG at
arch/x86/kernel/alternative.c:2564!Oops: invalid opcode: 0000 [#1] SMP NOPTI CPU: 0 UID: 0 PID: 741 Comm: (udev-worker) Not tainted 7.0.0-28-generic #28-Ubuntu PREEMPT(lazy) Hardware name: QEMU Standard PC (Q35 + ICH9, 2009) RIP: 0010:__text_poke+0x2a4/0x400 Call Trace: <TASK> smp_text_poke_batch_finish+0x25f/0x5b0 smp_text_poke_single+0x13/0x30 __static_call_transform+0x10e/0x210 arch_static_call_transform+0x5c/0xc0 __static_call_init+0x154/0x300 static_call_module_notify+0x144/0x170 notifier_call_chain+0x62/0xf0 blocking_notifier_call_chain_robust+0x62/0xc0 load_module+0x5ef/0x960 Modules linked in: ppdev(+) kvm(+) irqbypass ...Signature 2 — BUG_ON(memcmp()) in
__text_poke():kernel BUG at
arch/x86/kernel/alternative.c:2632!Oops: invalid opcode: 0000 [#1] SMP NOPTI CPU: 0 UID: 0 PID: 742 Comm: (udev-worker) Not tainted 7.0.0-28-generic #28-Ubuntu PREEMPT(lazy) Hardware name: QEMU Standard PC (Q35 + ICH9, 2009) RIP: 0010:__text_poke+0x380/0x400 Call Trace: <TASK> smp_text_poke_batch_finish+0x323/0x5b0 smp_text_poke_single+0x13/0x30 __static_call_transform+0x10e/0x210 arch_static_call_transform+0x5c/0xc0 __static_call_init+0x154/0x300 static_call_module_notify+0x144/0x170 notifier_call_chain+0x62/0xf0 blocking_notifier_call_chain_robust+0x62/0xc0 load_module+0x5ef/0x960 Modules linked in: kvm(+) irqbypass ...INSTRUMENTATION
---------------The following instrumentation was applied to capture the page table state at the moment of failure:
In mm/vmalloc.c::vmalloc_to_page()
ptep = pte_offset_kernel(pmd, addr); pte = ptep_get(ptep); if (pte_present(pte)) page = pte_page(pte); else pr_err("vmalloc_to_page: pte_not_present for addr=%lx " "PGD=%lx P4D=%lx PUD=%lx PMD=%lx PTE=%lx\n", addr, pgd_val(*pgd), p4d_val(*p4d), pud_val(*pud), pmd_val(*pmd), pte_val(pte));In arch/x86/kernel/alternative.c::__text_poke()
/* BUG 1: pages[0] NULL */ if (!pages[0] || ...) { pr_err("%s: vmalloc_to_page failed for addr=%pK pages[0]=%pK pages[1]=%pK cross=%d\n", ...); BUG(); } ... /* BUG 2: memcmp mismatch */ if (memcmp(addr, src, len)) { struct page *page_now = vmalloc_to_page(addr); pr_err("%s: memcmp failed at addr=%pK len=%zu\n", __func__, addr, len); pr_err("%s: expected (src): %*ph\n", __func__, (int)len, src); pr_err("%s: found (dst): %*ph\n", __func__, (int)len, (u8 *)addr); pr_err("%s: pfn at poke time (vmalloc_to_page): %lx\n", __func__, page_to_pfn(pages[0])); pr_err("%s: pfn at readback time (vmalloc_to_page): %lx\n", __func__, page_to_pfn(page_now)); text_poke_dump_pagetable((unsigned long)addr); /* reads CR3 */ BUG(); }Signature 1 —
vmalloc_to_page()returned NULL:
vmalloc_to_page:pte_not_presentfor addr=ffffffffc0697979 PGD=28242067 P4D=28243067 PUD=28245067 PMD=3be001a1 PTE=0 SMP alternatives:__text_poke:vmalloc_to_pagefailed for addr=ffffffffc0697979 pages[0]=0000000000000000 pages[1]=0000000000000000 cross=0PMD was not a leaf, and the code reached a point to check
pte_present()and finds the PTE to be not present. But in the dump in else case,pmd_val(*pmd)shows that it is a PMD leaf (PSE bit set).Signature 2 —
vmalloc_to_page()returned a wrong page:SMP alternatives:
__text_poke: memcmp failed at addr=ffffffffc06fc4bd len=1 SMP alternatives:__text_poke: expected (src): cc
SMP alternatives:__text_poke: found (dst): e8
SMP alternatives:__text_poke: pfn at poke time (vmalloc_to_page): 3884a SMP alternatives:__text_poke: pfn at readback time (vmalloc_to_page): 52fc SMP alternatives:__text_poke: page table dump for addr:PGD 1ba42067 P4D 1ba43067 PUD 1ba45067 PMD 52001a1
vmalloc_to_page()returned pfn=0x3884a when pages[0] was queried. The write of0xcclanded there. Whenvmalloc_to_page()was called again during the memcmp readback in the instrumentation code, the vmap mapping resolved through PMD=52001a1 to a different pfn=0x52fc, which contains0xe8(existing instruction bytes). This is silent memory corruption —0xccwas written to the wrong physical page (pfn=0x3884a).BISECTION
---------The following experiments all ran 200 iterations of 512 concurrent guest boots with zero crashes.
1. Remove "select
ARCH_HAS_EXECMEM_ROX" fromarch/x86/Kconfig2. Boot with "nohugevmalloc"
3. Remove
VM_ALLOW_HUGE_VMAPfromvm_flagsinexecmem_cache_populate()- unsigned long
vm_flags=VM_ALLOW_HUGE_VMAP;
+ unsigned longvm_flags= 0;4. Remove
CPA_COLLAPSEfromset_memory_rox()return change_page_attr_set_clr(&addr, numpages,
__pgprot(0), clr, 0,
-CPA_COLLAPSE, NULL);
+ 0, NULL);Without
CPA_COLLAPSE,set_memory_rox()changes 4K PTE permissions in-place and the PMD stays permanently as a PTE-page pointer after the first split — no re-collapse, no future race window. This confirms thatCPA_COLLAPSEis a necessary part of the race sequence.5. Hold
pgd_lockaroundvmalloc_to_page()in__text_poke()if (!core_kernel_text((unsigned long)addr)) {
+spin_lock(&pgd_lock);
pages[0] =vmalloc_to_page(addr);
if (cross_page_boundary)
pages[1] =vmalloc_to_page(addr + PAGE_SIZE);
+spin_unlock(&pgd_lock);
} else {
__split_large_page()holdspgd_lockwhile installing the new PMD and populating all 512 4K PTEs. Holdingpgd_lockaroundvmalloc_to_page()prevents CPA from modifying the PMD concurrently with the page table walk. This confirms the race is precisely betweenvmalloc_to_page()reading the PMD/PTE and__split_large_page()modifying it underpgd_lock.Note: this is a diagnostic experiment, not a production fix.
The experiments point to a race between
vmalloc_to_page()in__text_poke()and__split_large_page()modifying the shared PMD underpgd_lock.__split_large_page()holdspgd_lockwhile modifying the PMD;vmalloc_to_page()holds no lock and can observe the PMD in an intermediate state.
Quoted 3 lines
What would be the correct fix for this issue?
Happy to test any proposed fix or run further experiments.
https://lore.kernel.org/linux-mm/anCK3eWFMwZqq5ka@pedro-suse/
Signature · 1 line
Pedro
Re: [PATCH v3 9/9] x86: re-enable EXECMEM_ROX support
Quoted 2 lines · Pedro Falcato
https://lore.kernel.org/linux-mm/anCK3eWFMwZqq5ka@pedro-suse/
Yes, this is the same issue. Thanks for the pointer.
Will get back with test results on the above thread.
Regards,
Nikunj
Peter Zijlstra
maintainerQuoted 3 lines · Mike Rapoport
v3 changes:
* instead of adding a new module state handle ROX restoration locally inload_module()as Petr suggested
OK, updated the patches I have pending for post -rc1.
Hi Mike,
I bisected issues with the kernel booting correctly to an earlier revision of this series [2]. I have now tested this series in the same environment and found that it functions correctly with no issues, so feel free to add:
Cheers, Lorenzo
[2]:https://lore.kernel.org/linux-mm/[email protected]/
Quoted 71 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
Hi,
Following Peter's comments [1] these patches rework handling of ROX caches for module text allocations.
Instead of using a writable copy that really complicates alternatives patching, temporarily remap parts of a large ROX page as RW for the time of module formation and then restore it's ROX protections when the module is ready.
To keep the ROX memory mapped with large pages, make
set_memory_rox()capable of restoring large pages (more details are in patch 3).Since this is really about x86, I believe this should go in via tip tree.
The patches also available in git
https://git.kernel.org/rppt/h/execmem/x86-rox/v10v3 changes:
* instead of adding a new module state handle ROX restoration locally inload_module()as Petr suggestedv2: https://lore.kernel.org/all/[email protected]
* only collapse large mappings inset_memory_rox()
* simplify RW <-> ROX remapping
* don't remove ROX cache pages from the direct map (patch 4)v1: https://lore.kernel.org/all/[email protected]
[1] https://lore.kernel.org/all/[email protected]
Kirill A. Shutemov (1):
x86/mm/pat: restore large ROX pages after fragmentationMike Rapoport (Microsoft) (8):
x86/mm/pat: cpa-test: fix length forCPA_ARRAYtest x86/mm/pat: drop duplicate variable incpa_flush()
execmem: don't remove ROX cache from the direct map
execmem: add API for temporal remapping as RW and restoring ROX afterwards
module: switch to execmem API for remapping as RW and restoring ROX Revert "x86/module: prepare module loading for ROX allocations of text"
module: drop unusedmodule_writable_address()
x86: re-enableEXECMEM_ROXsupportarch/um/kernel/um_arch.c | 11 +- arch/x86/Kconfig | 1 + arch/x86/entry/vdso/vma.c | 3 +- arch/x86/include/asm/alternative.h | 14 +- arch/x86/include/asm/pgtable_types.h | 2 + arch/x86/kernel/alternative.c | 181 +++++++++------------- arch/x86/kernel/ftrace.c | 30 ++-- arch/x86/kernel/module.c | 45 ++---- arch/x86/mm/pat/cpa-test.c | 2 +- arch/x86/mm/pat/set_memory.c | 220 ++++++++++++++++++++++++++- include/linux/execmem.h | 31 ++++ include/linux/module.h | 16 -- include/linux/moduleloader.h | 4 - include/linux/vm_event_item.h | 2 + kernel/module/main.c | 78 +++------- kernel/module/strict_rwx.c | 9 +- mm/execmem.c | 39 +++-- mm/vmstat.c | 2 + 18 files changed, 422 insertions(+), 268 deletions(-)base-commit:
ffd294d346d185b70e28b1a28abe367bbfe53c04--
2.45.2