mm/vmalloc: track which page-table levels were modified
Subsystems (8)
Quoted 16 lines · Joerg Roedel
Track at which levels in the page-table entries were modified by vmap/vunmap.
After the page-table has been modified, use that information do decide whether the new
arch_sync_kernel_mappings()needs to be called.[[email protected]: map_kernel_range_noflush() needs the arch_sync_kernel_mappings() call]Signed-off-by Joerg Roedel <[email protected]>Signed-off-by Andrew Morton <[email protected]>Acked-by Andy Lutomirski <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Christoph Hellwig <[email protected]>Cc Dave Hansen <[email protected]>Cc "H . Peter Anvin" <[email protected]>Cc Ingo Molnar <[email protected]>
Quoted 8 lines
Cc Matthew Wilcox (Oracle) <[email protected]>Cc Michal Hocko <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc Steven Rostedt (VMware) <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Vlastimil Babka <[email protected]>Signed-off-by Linus Torvalds <[email protected]>
This patch causes crashes with all my ppc64 boot tests, in various locations depending on the platform. Reverting it together with its companion "mm/ioremap: track which page-table levels were modified" fixes the problem.
Various logs are at
https://kerneltests.org/builders/qemu-ppc64-master/builds/1442/steps/qemubuildcommand/logs/stdio
Just for completeness, bisect log attached.
Guenter
---
# bad: [f6aee505c71bbb035dde146caf5a6abbf3ccbe47] Merge tag 'x86-timers-2020-06-03' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
# good: [b23c4771ff62de8ca9b5e4a2d64491b2fb6f8f69] Merge tag 'docs-5.8' of git://git.lwn.net/linux
git bisect start 'HEAD' 'b23c4771ff62'
# good: [c41219fda6e04255c44d37fd2c0d898c1c46abf1] Merge tag 'drm-intel-next-fixes-2020-05-20' of git://anongit.freedesktop.org/drm/drm-intel into drm-next git bisect good c41219fda6e04255c44d37fd2c0d898c1c46abf1
# bad: [1966391fa576e1fb2701be8bcca197d8f72737b7] mm/migrate.c: attach_page_private already does the get_page
git bisect bad 1966391fa576e1fb2701be8bcca197d8f72737b7
# bad: [c5d6c13843880ad0112f0513f3eb041b258be66e] Merge tag 'mmc-v5.8' of git://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc
git bisect bad c5d6c13843880ad0112f0513f3eb041b258be66e
# bad: [94709049fb8442fb2f7b91fbec3c2897a75e18df] Merge branch 'akpm' (patches from Andrew)
git bisect bad 94709049fb8442fb2f7b91fbec3c2897a75e18df
# good: [a29adb6209cead1f6c34a8d72481fb183bfc2d68] mm: rename vmap_page_range to map_kernel_range
git bisect good a29adb6209cead1f6c34a8d72481fb183bfc2d68
# good: [56446efab9ce4961fe0fe6bbc5bc66374b08b9f3] Merge branch 'uaccess.__copy_from_user' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
git bisect good 56446efab9ce4961fe0fe6bbc5bc66374b08b9f3
# good: [17839856fd588f4ab6b789f482ed3ffd7c403e1f] gup: document and work around "COW can break either way" issue
git bisect good 17839856fd588f4ab6b789f482ed3ffd7c403e1f
# good: [ebcdd3074a729f9ba351278e1b42d7ae7fcdf236] arm64: use __vmalloc_node in arch_alloc_vmap_stack
git bisect good ebcdd3074a729f9ba351278e1b42d7ae7fcdf236
# bad: [86cf69f1d893d48fdb0382a940f2523409406423] x86/mm/32: implement arch_sync_kernel_mappings()
git bisect bad 86cf69f1d893d48fdb0382a940f2523409406423
# good: [d8626138009ba58ae2c22356966c2edaa1f1c3b5] mm: add functions to track page directory modifications
git bisect good d8626138009ba58ae2c22356966c2edaa1f1c3b5
# bad: [6c0c7d2b365b21a413f6d75772a8a4a2c7d36916] mm/ioremap: track which page-table levels were modified
git bisect bad 6c0c7d2b365b21a413f6d75772a8a4a2c7d36916
# bad: [2ba3e6947aed9bb9575eb1603c0ac6e39185d32a] mm/vmalloc: track which page-table levels were modified
git bisect bad 2ba3e6947aed9bb9575eb1603c0ac6e39185d32a
# first bad commit: [2ba3e6947aed9bb9575eb1603c0ac6e39185d32a] mm/vmalloc: track which page-table levels were modified
Hi Guenter,
Quoted 7 lines · Guenter Roeck
This patch causes crashes with all my ppc64 boot tests, in various locations depending on the platform. Reverting it together with its companion "mm/ioremap: track which page-table levels were modified" fixes the problem.
Various logs are at
https://kerneltests.org/builders/qemu-ppc64-master/builds/1442/steps/qemubuildcommand/logs/stdio
I posted the fix for this already:
https://lore.kernel.org/lkml/[email protected]/
Regards,
Joerg
Quoted 4 lines · Joerg Roedel
I posted the fix for this already:
https://lore.kernel.org/lkml/[email protected]/
Ugh.
I was going to apply this directly, but as I looked at the patch I just found it fairly illegible.
Is there some reason why the 5level-fixup.h versions use that very-hard-to-follow macro, rather than the inline functions that the main mm.h file uses?
I'm _assuming_ it's because it gets included in some place where not everything is defined yet, so making it a macro means that it works (later on) when everything has come together..
But the solution to that would seem to make all the p.._alloc_track() macros just be in a different header file, and make them be all together. We already have that
#if !__ARCH_HAS_5LEVEL_HACK
in linux/mm.h, so it's not like we really have isolated that issue into just 5level-fixup.h anyway, and creating a new <linux/pagetable-alloc.h> header that has all the variations in one place, and that is only included by the two (!) users of these things would seem to be a good idea regardless.
Because <linux/mm.h> is included by pretty much everything. Why do we have those alloc_track functions defined in such a common header when they are _so_ special?
Please? I'd obviously like this to be fixed on ppc asap, but I'd also like the fix to improve on the current somewhat confusing situation..
For extra point, the p??_alloc_track() functions could even be generated from a macro pattern, because the pattern is pretty much set in stone.
I think the only thing that really differs is the types and the PGTBL_xyz_MODIFIED mask, and which entry is tested for "none" (which is also the only thing that makes the 5level fixup case different - no?
Linus
Andrew Morton
maintainerQuoted 38 lines · On Thu, 4 Jun 2020 10:16:07 -0700 Linus Torvalds…
Quoted 4 lines · Joerg Roedel
I posted the fix for this already:
https://lore.kernel.org/lkml/[email protected]/Ugh.
I was going to apply this directly, but as I looked at the patch I just found it fairly illegible.
Is there some reason why the
5level-fixup.hversions use that very-hard-to-follow macro, rather than the inline functions that the mainmm.hfile uses?I'm _assuming_ it's because it gets included in some place where not everything is defined yet, so making it a macro means that it works (later on) when everything has come together..
But the solution to that would seem to make all the p.._alloc_track() macros just be in a different header file, and make them be all together. We already have that
#if !__ARCH_HAS_5LEVEL_HACKin
linux/mm.h, so it's not like we really have isolated that issue into just5level-fixup.hanyway, and creating a new <linux/pagetable-alloc.h> header that has all the variations in one place, and that is only included by the two (!) users of these things would seem to be a good idea regardless.Because <
linux/mm.h> is included by pretty much everything. Why do we have thosealloc_trackfunctions defined in such a common header when they are _so_ special?Please? I'd obviously like this to be fixed on ppc asap, but I'd also like the fix to improve on the current somewhat confusing situation..
Quoted 8 lines
For extra point, the p??_alloc_track() functions could even be generated from a macro pattern, because the pattern is pretty much set in stone.
I think the only thing that really differs is the types and the
PGTBL_xyz_MODIFIEDmask, and which entry is tested for "none" (which is also the only thing that makes the 5level fixup case different - no?
As discussed over in
https://lore.kernel.org/linux-mm/[email protected]/,
Mike's "mm: remove __ARCH_HAS_5LEVEL_HACK" patchset
(http://lkml.kernel.org/r/[email protected]) is expected to fix this. 5level-fixup.h gets removed.
I hope to have that patchset sent over later today.
Quoted 6 lines · Andrew Morton
As discussed over in
https://lore.kernel.org/linux-mm/[email protected]/,
Mike's "mm: remove__ARCH_HAS_5LEVEL_HACK" patchset
(http://lkml.kernel.org/r/[email protected]) is expected to fix this.5level-fixup.hgets removed.
Ok, even better.
That said, the commentary about "why is p.._alloc_track() in such a core header file, when it's only used by two special cases" is probably still true regardless of the 5-level fixup header.. I assume Mike didn't do those kinds of changes?
Yeah, I'm probably flailing at windmills, but I do dislike how we often end up just growing the very core headers that get included by everybody without ever trying to fix that uncontrolled growth..
Linus
Quoted 11 lines · Linus Torvalds
Quoted 4 lines · Joerg Roedel
I posted the fix for this already:
https://lore.kernel.org/lkml/[email protected]/Ugh.
I was going to apply this directly, but as I looked at the patch I just found it fairly illegible.
Quoted 7 lines
Is there some reason why the
5level-fixup.hversions use that very-hard-to-follow macro, rather than the inline functions that the mainmm.hfile uses?I'm _assuming_ it's because it gets included in some place where not everything is defined yet, so making it a macro means that it works (later on) when everything has come together..
Exactly, I had all of the p?d_alloc_track() functions as inlines first, but that broke compilation on some obscure architectures, so I followed the rule to make them macros when the p?d_alloc() was also a macro and defined them close together.
Joerg
Quoted 4 lines · Linus Torvalds
That said, the commentary about "why is p.._alloc_track() in such a core header file, when it's only used by two special cases" is probably still true regardless of the 5-level fixup header.. I assume Mike didn't do those kinds of changes?
I'll try to move them to a separate header next week. The compile testing I set up for this patch-set will certainly be helpful for that :)
Joerg
Quoted 9 lines · Joerg Roedel
Quoted 4 lines · Linus Torvalds
That said, the commentary about "why is p.._alloc_track() in such a core header file, when it's only used by two special cases" is probably still true regardless of the 5-level fixup header.. I assume Mike didn't do those kinds of changes?
I'll try to move them to a separate header next week. The compile testing I set up for this patch-set will certainly be helpful for that :)
We already have include/asm-generic/pgalloc.h, so maybe something like that patch below would fork. This is not even compile tested.
include/asm-generic/pgalloc.h +60 -0
@@ -102,6 +102,66 @@ static inline void pte_free(struct mm_struct *mm, struct page *pte_page)__free_page (pte_page); } +static inline p4d_t *p4d_alloc (struct mm_struct *mm,pgd_t *pgd, +unsigned long address) +{ +return (unlikely (pgd_none (*pgd)) &&__p4d_alloc (mm, pgd, address)) ? +NULL :p4d_offset (pgd, address); +} + +static inline pud_t *pud_alloc (struct mm_struct *mm,p4d_t *p4d, +unsigned long address) +{ +return (unlikely (p4d_none (*p4d)) &&__pud_alloc (mm, p4d, address)) ? +NULL :pud_offset (p4d, address); +} + +static inline p4d_t *p4d_alloc_track (struct mm_struct *mm,pgd_t *pgd, +unsigned long address, +pgtbl_mod_mask *mod_mask) + +{ +if (unlikely (pgd_none (*pgd))) { +if (__p4d_alloc (mm, pgd, address)) +return NULL ; + *mod_mask |=PGTBL_PGD_MODIFIED ; + } + +return p4d_offset (pgd, address); +} + +static inline pud_t *pud_alloc_track (struct mm_struct *mm,p4d_t *p4d, +unsigned long address, +pgtbl_mod_mask *mod_mask) +{ +if (unlikely (p4d_none (*p4d))) { +if (__pud_alloc (mm, p4d, address)) +return NULL ; + *mod_mask |=PGTBL_P4D_MODIFIED ; + } + +return pud_offset (p4d, address); +} + +static inline pmd_t *pmd_alloc (struct mm_struct *mm,pud_t *pud,unsigned long address) +{ +return (unlikely (pud_none (*pud)) &&__pmd_alloc (mm, pud, address))? +NULL :pmd_offset (pud, address); +} + +static inline pmd_t *pmd_alloc_track (struct mm_struct *mm,pud_t *pud, +unsigned long address, +pgtbl_mod_mask *mod_mask) +{ +if (unlikely (pud_none (*pud))) { +if (__pmd_alloc (mm, pud, address)) +return NULL ; + *mod_mask |=PGTBL_PUD_MODIFIED ; + } + +return pmd_offset (pud, address); +} +#endif /* CONFIG_MMU */ #endif /* __ASM_GENERIC_PGALLOC_H */
include/asm-generic/tlb.h +0 -1
@@ -14,7 +14,6 @@#include <linux/mmu_notifier.h> #include <linux/swap.h> #include <linux/hugetlb_inline.h> -#include <asm/pgalloc.h> #include <asm/tlbflush.h> #include <asm/cacheflush.h>
include/linux/mm.h +0 -63
69 lines not on this page · raw message
lib/ioremap.c +1 -0
@@ -13,6 +13,7 @@#include <linux/export.h> #include <asm/cacheflush.h> #include <asm/pgtable.h> +#include <asm/pgalloc.h> #ifdef CONFIG_HAVE_ARCH_HUGE_VMAP static int __read_mostly ioremap_p4d_capable;
mm/mremap.c +1 -0
@@ -27,6 +27,7 @@#include <asm/cacheflush.h> #include <asm/tlbflush.h> +#include <asm/pgalloc.h> #include "internal.h"
mm/vmalloc.c +1 -0
@@ -39,6 +39,7 @@#include <linux/uaccess.h> #include <asm/tlbflush.h> #include <asm/shmparam.h> +#include <asm/pgalloc.h> #include "internal.h"
Quoted 3 lines
Joerg
Signature · 2 lines
Sincerely yours, Mike.
Hi Mike,
include/asm-generic/pgalloc.h
Quoted 4 lines · Mike Rapoport
We already have
include/asm-generic/pgalloc.h, so maybe something like that patch below would fork. This is not even compile tested.include/asm-generic/pgalloc.h +0
-0
I experimented a bit with your diff, but it turned out that moving the page-table allocation functions/macros to asm-generic/pgalloc.h does not work on all architectures.
The reason is that some archs don't use that header at all (e.g. ARC) and have their own version of the functions defined there. That could all be made working, but I decided to no open this can of worms for now.
So I sent out a patch which moves the p?d_alloc_track() functions/macros to a separate header and include it only in mm/vmalloc.c and lib/ioremap.c. That compiles on all architectures where current Linus' master also builds (it does not for Alpha, CSky, Mips and Mips64), and as usual Hexagon and Unicore32 are not tested because I have no working compiler for those.
Regards,
Joerg
Quoted 16 lines · Joerg Roedel
Hi Mike,
include/asm-generic/pgalloc.h
Quoted 4 lines · Mike Rapoport
We already have
include/asm-generic/pgalloc.h, so maybe something like that patch below would fork. This is not even compile tested.include/asm-generic/pgalloc.h +0
-0I experimented a bit with your diff, but it turned out that moving the page-table allocation functions/macros to
asm-generic/pgalloc.hdoes not work on all architectures.The reason is that some archs don't use that header at all (e.g. ARC) and have their own version of the functions defined there. That could all be made working, but I decided to no open this can of worms for now.
Quoted 7 lines
So I sent out a patch which moves the p?d_alloc_track() functions/macros to a separate header and include it only in
mm/vmalloc.candlib/ioremap.c. That compiles on all architectures where current Linus' master also builds (it does not for Alpha, CSky, Mips and Mips64), and as usual Hexagon and Unicore32 are not tested because I have no working compiler for those.
To build csky images, you have to disable CONFIG_FRAME_POINTER or use a non-upstream compiler. To build any images reliably, you have to disable CONFIG_GCC_PLUGIN_RANDSTRUCT or use a version of gcc old enough to not support it (gcc 6.x is fine). For mips, you have to specify ARCH and CROSS_COMPILE as environment variables.
alpha is a lost case. The offending commit is 0f1c9688a19 ("tty/sysrq:
alpha: export and use __sysrq_get_key_op()"); it looks like that wasn't build tested. It can not be reverted easily because of subsequent changes.
I have stopped building unicore32 images since v4.19 since there is no available compiler that is still supported by the kernel. I am surprised that support for it has not been removed from the kernel.
I am told that hexagon only supports llvm/clang, and the only version of gcc that supports it is v4.6.1 (from Sourcery CodeBench Lite 2012.03-66). The minimum version of gcc will be raised to v4.8; that has already happened in linux-next and will presumably be applied to mainline shortly. With that, I'll stop build testing hexagon images as well until I find the time to build a llvm/clang hexagon toolchain.
So you are in good company, and thanks for testing everything else.
Guenter
Hi Joerg,
Quoted 12 lines · Joerg Roedel
Hi Mike,
include/asm-generic/pgalloc.h
Quoted 4 lines · Mike Rapoport
We already have
include/asm-generic/pgalloc.h, so maybe something like that patch below would fork. This is not even compile tested.include/asm-generic/pgalloc.h +0
-0I experimented a bit with your diff, but it turned out that moving the page-table allocation functions/macros to
asm-generic/pgalloc.hdoes not work on all architectures.
Quoted 7 lines
The reason is that some archs don't use that header at all (e.g. ARC) and have their own version of the functions defined there. That could all be made working, but I decided to no open this can of worms for now.
So I sent out a patch which moves the p?d_alloc_track() functions/macros to a separate header and include it only in
mm/vmalloc.candlib/ioremap.c.
I'm planning to open this can of worms eventually, but having p?d_alloc_track() in a separate header wouldn't be the biggest of my problems :)
Quoted 8 lines
That compiles on all architectures where current Linus' master also builds (it does not for Alpha, CSky, Mips and Mips64), and as usual Hexagon and Unicore32 are not tested because I have no working compiler for those.
Regards,
Joerg
Signature · 2 lines
Sincerely yours, Mike.
Hi Guenter,
Quoted 5 lines · Guenter Roeck
To build csky images, you have to disable
CONFIG_FRAME_POINTERor use a non-upstream compiler. To build any images reliably, you have to disableCONFIG_GCC_PLUGIN_RANDSTRUCTor use a version of gcc old enough to not support it (gcc 6.x is fine). For mips, you have to specify ARCH andCROSS_COMPILEas environment variables.
My test setup builds defconfigs for all architectures using the gcc-9.3 based cross-compilers from kernel.org (except the ones I have no compiler for). That used to work for CSky and MIPS(64) when working on my vmalloc changes.
On MIPS the build failure looks like some Makefile breakage, but I didn't dive deeper into that.
For CSky the compiler complains about not supporting '-mbacktrace'.
Quoted 3 lines
alpha is a lost case. The offending commit is
0f1c9688a19("tty/sysrq:
alpha: export and use__sysrq_get_key_op()"); it looks like that wasn't build tested. It can not be reverted easily because of subsequent changes.
The below diff fixed the alpha build for me, but I am sure another fix is already queued somewhere.
arch/alpha/kernel/setup.c +6 -1
@@ -431,8 +431,13 @@ register_cpus(void)arch_initcall (register_cpus);#ifdef CONFIG_MAGIC_SYSRQ +static void sysrq_reboot_handler (int unused) +{ +machine_halt (); +} +static const struct sysrq_key_op srm_sysrq_reboot_op = { - .handler = machine_halt, + .handler = sysrq_reboot_handler, .help_msg ="reboot(b)" , .action_msg ="Resetting" , .enable_mask = SYSRQ_ENABLE_BOOT,
Quoted 9 lines · Joerg Roedel
Hi Guenter,
Quoted 5 lines · Guenter Roeck
To build csky images, you have to disable
CONFIG_FRAME_POINTERor use a non-upstream compiler. To build any images reliably, you have to disableCONFIG_GCC_PLUGIN_RANDSTRUCTor use a version of gcc old enough to not support it (gcc 6.x is fine). For mips, you have to specify ARCH andCROSS_COMPILEas environment variables.
Quoted 8 lines
My test setup builds defconfigs for all architectures using the gcc-9.3 based cross-compilers from kernel.org (except the ones I have no compiler for). That used to work for CSky and MIPS(64) when working on my vmalloc changes.
On MIPS the build failure looks like some
Makefilebreakage, but I didn't dive deeper into that.
Yes, that was discussed elsewhere. A fix has been submitted and should hopefully find its way into the kernel sometime soon.
Quoted 2 lines
For CSky the compiler complains about not supporting '-mbacktrace'.
Yes, that is why you have to disable CONFIG_FRAME_POINTER. '-mbacktrace' is not supported in upstream versions of gcc for csky; it is only supported in the out-of-tree csky toolchain.
Quoted 7 lines
Quoted 3 lines
alpha is a lost case. The offending commit is
0f1c9688a19("tty/sysrq:
alpha: export and use__sysrq_get_key_op()"); it looks like that wasn't build tested. It can not be reverted easily because of subsequent changes.The below diff fixed the alpha build for me, but I am sure another fix is already queued somewhere.
Let's hope it is.
Thanks,
Guenter
Quoted 20 lines
arch/alpha/kernel/setup.c +6
-1@@ -431,8 +431,13 @@ register_cpus(void)arch_initcall (register_cpus);#ifdef CONFIG_MAGIC_SYSRQ +static void sysrq_reboot_handler (int unused) +{ +machine_halt (); +} +static const struct sysrq_key_op srm_sysrq_reboot_op = { - .handler = machine_halt, + .handler = sysrq_reboot_handler, .help_msg ="reboot(b)" , .action_msg ="Resetting" , .enable_mask = SYSRQ_ENABLE_BOOT,
Quoted 5 lines · Guenter Roeck
Quoted 1 line · Joerg Roedel
I have stopped building unicore32 images since v4.19 since there is no available compiler that is still supported by the kernel. I am surprised that support for it has not been removed from the kernel.
Well, I've done the patches [1] and I'll send them after -rc1 is out. It'll also give kbuild some time to verify I didn't remove too much :)
[1] https://git.kernel.org/pub/scm/linux/kernel/git/rppt/linux.git/log/?h=rm-unicore
Quoted 1 line
Guenter
Signature · 2 lines
Sincerely yours, Mike.