Skip to contentSkip to conversation

mm/vmalloc: track which page-table levels were modified

PATCH Guenter Roeck 5 participants 14 messages Jun 3, 2020 – Jun 10, 2020

Subsystems (8)
lore
Share & tools
lore
#0 lore raw
Quoted 16 lines · Joerg Roedel

Track at which levels in the page-table entries were modified by vmap/vunmap.

After the page-table has been modified, use that information do decide whether the new arch_sync_kernel_mappings() needs to be called.

[[email protected]: map_kernel_range_noflush() needs the arch_sync_kernel_mappings() call]
Cc Arnd Bergmann <[email protected]>
Cc Christoph Hellwig <[email protected]>
Cc Dave Hansen <[email protected]>
Cc "H . Peter Anvin" <[email protected]>
Cc Ingo Molnar <[email protected]>
Quoted 8 lines
Cc Matthew Wilcox (Oracle) <[email protected]>
Cc Michal Hocko <[email protected]>
Cc "Rafael J. Wysocki" <[email protected]>
Cc Steven Rostedt (VMware) <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Vlastimil Babka <[email protected]>

This patch causes crashes with all my ppc64 boot tests, in various locations depending on the platform. Reverting it together with its companion "mm/ioremap: track which page-table levels were modified" fixes the problem.

Various logs are at
https://kerneltests.org/builders/qemu-ppc64-master/builds/1442/steps/qemubuildcommand/logs/stdio

Just for completeness, bisect log attached.

Guenter

---
# bad: [f6aee505c71bbb035dde146caf5a6abbf3ccbe47] Merge tag 'x86-timers-2020-06-03' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
# good: [b23c4771ff62de8ca9b5e4a2d64491b2fb6f8f69] Merge tag 'docs-5.8' of git://git.lwn.net/linux
git bisect start 'HEAD' 'b23c4771ff62'
# good: [c41219fda6e04255c44d37fd2c0d898c1c46abf1] Merge tag 'drm-intel-next-fixes-2020-05-20' of git://anongit.freedesktop.org/drm/drm-intel into drm-next git bisect good c41219fda6e04255c44d37fd2c0d898c1c46abf1
# bad: [1966391fa576e1fb2701be8bcca197d8f72737b7] mm/migrate.c: attach_page_private already does the get_page
git bisect bad 1966391fa576e1fb2701be8bcca197d8f72737b7
# bad: [c5d6c13843880ad0112f0513f3eb041b258be66e] Merge tag 'mmc-v5.8' of git://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc
git bisect bad c5d6c13843880ad0112f0513f3eb041b258be66e
# bad: [94709049fb8442fb2f7b91fbec3c2897a75e18df] Merge branch 'akpm' (patches from Andrew)
git bisect bad 94709049fb8442fb2f7b91fbec3c2897a75e18df
# good: [a29adb6209cead1f6c34a8d72481fb183bfc2d68] mm: rename vmap_page_range to map_kernel_range
git bisect good a29adb6209cead1f6c34a8d72481fb183bfc2d68
# good: [56446efab9ce4961fe0fe6bbc5bc66374b08b9f3] Merge branch 'uaccess.__copy_from_user' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
git bisect good 56446efab9ce4961fe0fe6bbc5bc66374b08b9f3
# good: [17839856fd588f4ab6b789f482ed3ffd7c403e1f] gup: document and work around "COW can break either way" issue
git bisect good 17839856fd588f4ab6b789f482ed3ffd7c403e1f
# good: [ebcdd3074a729f9ba351278e1b42d7ae7fcdf236] arm64: use __vmalloc_node in arch_alloc_vmap_stack
git bisect good ebcdd3074a729f9ba351278e1b42d7ae7fcdf236
# bad: [86cf69f1d893d48fdb0382a940f2523409406423] x86/mm/32: implement arch_sync_kernel_mappings()
git bisect bad 86cf69f1d893d48fdb0382a940f2523409406423
# good: [d8626138009ba58ae2c22356966c2edaa1f1c3b5] mm: add functions to track page directory modifications
git bisect good d8626138009ba58ae2c22356966c2edaa1f1c3b5
# bad: [6c0c7d2b365b21a413f6d75772a8a4a2c7d36916] mm/ioremap: track which page-table levels were modified
git bisect bad 6c0c7d2b365b21a413f6d75772a8a4a2c7d36916
# bad: [2ba3e6947aed9bb9575eb1603c0ac6e39185d32a] mm/vmalloc: track which page-table levels were modified
git bisect bad 2ba3e6947aed9bb9575eb1603c0ac6e39185d32a

# first bad commit: [2ba3e6947aed9bb9575eb1603c0ac6e39185d32a] mm/vmalloc: track which page-table levels were modified
#1 lore raw

Hi Guenter,

Quoted 7 lines · Guenter Roeck

This patch causes crashes with all my ppc64 boot tests, in various locations depending on the platform. Reverting it together with its companion "mm/ioremap: track which page-table levels were modified" fixes the problem.

Various logs are at
https://kerneltests.org/builders/qemu-ppc64-master/builds/1442/steps/qemubuildcommand/logs/stdio

I posted the fix for this already:

	https://lore.kernel.org/lkml/[email protected]/

Regards,

Joerg

#2 lore raw
Quoted 4 lines · Joerg Roedel

I posted the fix for this already:

        https://lore.kernel.org/lkml/[email protected]/

Ugh.

I was going to apply this directly, but as I looked at the patch I just found it fairly illegible.

Is there some reason why the 5level-fixup.h versions use that very-hard-to-follow macro, rather than the inline functions that the main mm.h file uses?

I'm _assuming_ it's because it gets included in some place where not everything is defined yet, so making it a macro means that it works (later on) when everything has come together..

But the solution to that would seem to make all the p.._alloc_track() macros just be in a different header file, and make them be all together. We already have that

   #if !__ARCH_HAS_5LEVEL_HACK

in linux/mm.h, so it's not like we really have isolated that issue into just 5level-fixup.h anyway, and creating a new <linux/pagetable-alloc.h> header that has all the variations in one place, and that is only included by the two (!) users of these things would seem to be a good idea regardless.

Because <linux/mm.h> is included by pretty much everything. Why do we have those alloc_track functions defined in such a common header when they are _so_ special?

Please? I'd obviously like this to be fixed on ppc asap, but I'd also like the fix to improve on the current somewhat confusing situation..

For extra point, the p??_alloc_track() functions could even be generated from a macro pattern, because the pattern is pretty much set in stone.

I think the only thing that really differs is the types and the PGTBL_xyz_MODIFIED mask, and which entry is tested for "none" (which is also the only thing that makes the 5level fixup case different - no?

Linus

maintainer
#3 lore raw
Quoted 38 lines · On Thu, 4 Jun 2020 10:16:07 -0700 Linus Torvalds…
Quoted 4 lines · Joerg Roedel

I posted the fix for this already:

        https://lore.kernel.org/lkml/[email protected]/

Ugh.

I was going to apply this directly, but as I looked at the patch I just found it fairly illegible.

Is there some reason why the 5level-fixup.h versions use that very-hard-to-follow macro, rather than the inline functions that the main mm.h file uses?

I'm _assuming_ it's because it gets included in some place where not everything is defined yet, so making it a macro means that it works (later on) when everything has come together..

But the solution to that would seem to make all the p.._alloc_track() macros just be in a different header file, and make them be all together. We already have that

   #if !__ARCH_HAS_5LEVEL_HACK

in linux/mm.h, so it's not like we really have isolated that issue into just 5level-fixup.h anyway, and creating a new <linux/pagetable-alloc.h> header that has all the variations in one place, and that is only included by the two (!) users of these things would seem to be a good idea regardless.

Because <linux/mm.h> is included by pretty much everything. Why do we have those alloc_track functions defined in such a common header when they are _so_ special?

Please? I'd obviously like this to be fixed on ppc asap, but I'd also like the fix to improve on the current somewhat confusing situation..

Quoted 8 lines

For extra point, the p??_alloc_track() functions could even be generated from a macro pattern, because the pattern is pretty much set in stone.

I think the only thing that really differs is the types and the PGTBL_xyz_MODIFIED mask, and which entry is tested for "none" (which is also the only thing that makes the 5level fixup case different - no?

As discussed over in
https://lore.kernel.org/linux-mm/[email protected]/,
Mike's "mm: remove __ARCH_HAS_5LEVEL_HACK" patchset
(http://lkml.kernel.org/r/[email protected]) is expected to fix this. 5level-fixup.h gets removed.

I hope to have that patchset sent over later today.

#4 lore raw
Quoted 6 lines · Andrew Morton

As discussed over in
https://lore.kernel.org/linux-mm/[email protected]/,
Mike's "mm: remove __ARCH_HAS_5LEVEL_HACK" patchset
(http://lkml.kernel.org/r/[email protected]) is expected to fix this. 5level-fixup.h gets removed.

Ok, even better.

That said, the commentary about "why is p.._alloc_track() in such a core header file, when it's only used by two special cases" is probably still true regardless of the 5-level fixup header.. I assume Mike didn't do those kinds of changes?

Yeah, I'm probably flailing at windmills, but I do dislike how we often end up just growing the very core headers that get included by everybody without ever trying to fix that uncontrolled growth..

Linus

#5 lore raw
Quoted 4 lines · Linus Torvalds

That said, the commentary about "why is p.._alloc_track() in such a core header file, when it's only used by two special cases" is probably still true regardless of the 5-level fixup header.. I assume Mike didn't do those kinds of changes?

I'll try to move them to a separate header next week. The compile testing I set up for this patch-set will certainly be helpful for that :)

Joerg

#6 lore raw
Quoted 9 lines · Joerg Roedel
Quoted 4 lines · Linus Torvalds

That said, the commentary about "why is p.._alloc_track() in such a core header file, when it's only used by two special cases" is probably still true regardless of the 5-level fixup header.. I assume Mike didn't do those kinds of changes?

I'll try to move them to a separate header next week. The compile testing I set up for this patch-set will certainly be helpful for that :)

We already have include/asm-generic/pgalloc.h, so maybe something like that patch below would fork. This is not even compile tested.

include/asm-generic/pgalloc.h +60 -0
index 73f7421413cb..7aa9c98aff47 100644
@@ -102,6 +102,66 @@ static inline void pte_free(struct mm_struct *mm, struct page *pte_page)
 	__free_page(pte_page);
 }
 
+static inline p4d_t *p4d_alloc(struct mm_struct *mm, pgd_t *pgd,
+		unsigned long address)
+{
+	return (unlikely(pgd_none(*pgd)) && __p4d_alloc(mm, pgd, address)) ?
+		NULL : p4d_offset(pgd, address);
+}
+
+static inline pud_t *pud_alloc(struct mm_struct *mm, p4d_t *p4d,
+		unsigned long address)
+{
+	return (unlikely(p4d_none(*p4d)) && __pud_alloc(mm, p4d, address)) ?
+		NULL : pud_offset(p4d, address);
+}
+
+static inline p4d_t *p4d_alloc_track(struct mm_struct *mm, pgd_t *pgd,
+				     unsigned long address,
+				     pgtbl_mod_mask *mod_mask)
+
+{
+	if (unlikely(pgd_none(*pgd))) {
+		if (__p4d_alloc(mm, pgd, address))
+			return NULL;
+		*mod_mask |= PGTBL_PGD_MODIFIED;
+	}
+
+	return p4d_offset(pgd, address);
+}
+
+static inline pud_t *pud_alloc_track(struct mm_struct *mm, p4d_t *p4d,
+				     unsigned long address,
+				     pgtbl_mod_mask *mod_mask)
+{
+	if (unlikely(p4d_none(*p4d))) {
+		if (__pud_alloc(mm, p4d, address))
+			return NULL;
+		*mod_mask |= PGTBL_P4D_MODIFIED;
+	}
+
+	return pud_offset(p4d, address);
+}
+
+static inline pmd_t *pmd_alloc(struct mm_struct *mm, pud_t *pud, unsigned long address)
+{
+	return (unlikely(pud_none(*pud)) && __pmd_alloc(mm, pud, address))?
+		NULL: pmd_offset(pud, address);
+}
+
+static inline pmd_t *pmd_alloc_track(struct mm_struct *mm, pud_t *pud,
+				     unsigned long address,
+				     pgtbl_mod_mask *mod_mask)
+{
+	if (unlikely(pud_none(*pud))) {
+		if (__pmd_alloc(mm, pud, address))
+			return NULL;
+		*mod_mask |= PGTBL_PUD_MODIFIED;
+	}
+
+	return pmd_offset(pud, address);
+}
+
 #endif /* CONFIG_MMU */
 
 #endif /* __ASM_GENERIC_PGALLOC_H */
include/asm-generic/tlb.h +0 -1
index 3f1649a8cf55..36ac65f4744f 100644
@@ -14,7 +14,6 @@
 #include <linux/mmu_notifier.h>
 #include <linux/swap.h>
 #include <linux/hugetlb_inline.h>
-#include <asm/pgalloc.h>
 #include <asm/tlbflush.h>
 #include <asm/cacheflush.h>
 
include/linux/mm.h +0 -63

69 lines not on this page · raw message

lib/ioremap.c +1 -0
index ad485f08173b..357e5909a364 100644
@@ -13,6 +13,7 @@
 #include <linux/export.h>
 #include <asm/cacheflush.h>
 #include <asm/pgtable.h>
+#include <asm/pgalloc.h>
 
 #ifdef CONFIG_HAVE_ARCH_HUGE_VMAP
 static int __read_mostly ioremap_p4d_capable;
mm/mremap.c +1 -0
index 7d69e3fe51aa..4a6c4db8747b 100644
@@ -27,6 +27,7 @@
 
 #include <asm/cacheflush.h>
 #include <asm/tlbflush.h>
+#include <asm/pgalloc.h>
 
 #include "internal.h"
 
mm/vmalloc.c +1 -0
index 3091c2ca60df..84a6d08416a2 100644
@@ -39,6 +39,7 @@
 #include <linux/uaccess.h>
 #include <asm/tlbflush.h>
 #include <asm/shmparam.h>
+#include <asm/pgalloc.h>
 
 #include "internal.h"
 
Quoted 3 lines

Joerg

Signature · 2 lines
Sincerely yours,
Mike.
#7 lore raw

Hi Mike,

include/asm-generic/pgalloc.h

Quoted 4 lines · Mike Rapoport

We already have include/asm-generic/pgalloc.h, so maybe something like that patch below would fork. This is not even compile tested.

include/asm-generic/pgalloc.h +0 -0

I experimented a bit with your diff, but it turned out that moving the page-table allocation functions/macros to asm-generic/pgalloc.h does not work on all architectures.

The reason is that some archs don't use that header at all (e.g. ARC) and have their own version of the functions defined there. That could all be made working, but I decided to no open this can of worms for now.

So I sent out a patch which moves the p?d_alloc_track() functions/macros to a separate header and include it only in mm/vmalloc.c and lib/ioremap.c. That compiles on all architectures where current Linus' master also builds (it does not for Alpha, CSky, Mips and Mips64), and as usual Hexagon and Unicore32 are not tested because I have no working compiler for those.

Regards,

Joerg

#8 lore raw
Quoted 16 lines · Joerg Roedel

Hi Mike,

include/asm-generic/pgalloc.h

Quoted 4 lines · Mike Rapoport

We already have include/asm-generic/pgalloc.h, so maybe something like that patch below would fork. This is not even compile tested.

include/asm-generic/pgalloc.h +0 -0

I experimented a bit with your diff, but it turned out that moving the page-table allocation functions/macros to asm-generic/pgalloc.h does not work on all architectures.

The reason is that some archs don't use that header at all (e.g. ARC) and have their own version of the functions defined there. That could all be made working, but I decided to no open this can of worms for now.

Quoted 7 lines

So I sent out a patch which moves the p?d_alloc_track() functions/macros to a separate header and include it only in mm/vmalloc.c and lib/ioremap.c. That compiles on all architectures where current Linus' master also builds (it does not for Alpha, CSky, Mips and Mips64), and as usual Hexagon and Unicore32 are not tested because I have no working compiler for those.

To build csky images, you have to disable CONFIG_FRAME_POINTER or use a non-upstream compiler. To build any images reliably, you have to disable CONFIG_GCC_PLUGIN_RANDSTRUCT or use a version of gcc old enough to not support it (gcc 6.x is fine). For mips, you have to specify ARCH and CROSS_COMPILE as environment variables.

alpha is a lost case. The offending commit is 0f1c9688a19 ("tty/sysrq:
alpha: export and use __sysrq_get_key_op()"); it looks like that wasn't build tested. It can not be reverted easily because of subsequent changes.

I have stopped building unicore32 images since v4.19 since there is no available compiler that is still supported by the kernel. I am surprised that support for it has not been removed from the kernel.

I am told that hexagon only supports llvm/clang, and the only version of gcc that supports it is v4.6.1 (from Sourcery CodeBench Lite 2012.03-66). The minimum version of gcc will be raised to v4.8; that has already happened in linux-next and will presumably be applied to mainline shortly. With that, I'll stop build testing hexagon images as well until I find the time to build a llvm/clang hexagon toolchain.

So you are in good company, and thanks for testing everything else.

Guenter

#9 lore raw

Hi Guenter,

Quoted 5 lines · Guenter Roeck

To build csky images, you have to disable CONFIG_FRAME_POINTER or use a non-upstream compiler. To build any images reliably, you have to disable CONFIG_GCC_PLUGIN_RANDSTRUCT or use a version of gcc old enough to not support it (gcc 6.x is fine). For mips, you have to specify ARCH and CROSS_COMPILE as environment variables.

My test setup builds defconfigs for all architectures using the gcc-9.3 based cross-compilers from kernel.org (except the ones I have no compiler for). That used to work for CSky and MIPS(64) when working on my vmalloc changes.

On MIPS the build failure looks like some Makefile breakage, but I didn't dive deeper into that.

For CSky the compiler complains about not supporting '-mbacktrace'.

Quoted 3 lines

alpha is a lost case. The offending commit is 0f1c9688a19 ("tty/sysrq:
alpha: export and use __sysrq_get_key_op()"); it looks like that wasn't build tested. It can not be reverted easily because of subsequent changes.

The below diff fixed the alpha build for me, but I am sure another fix is already queued somewhere.

arch/alpha/kernel/setup.c +6 -1
index 6fa802c495b4..8f4ad63a3a9a 100644
@@ -431,8 +431,13 @@ register_cpus(void)
 arch_initcall(register_cpus);
 
 #ifdef CONFIG_MAGIC_SYSRQ
+static void sysrq_reboot_handler(int unused)
+{
+	machine_halt();
+}
+
 static const struct sysrq_key_op srm_sysrq_reboot_op = {
-	.handler	= machine_halt,
+	.handler	= sysrq_reboot_handler,
 	.help_msg       = "reboot(b)",
 	.action_msg     = "Resetting",
 	.enable_mask    = SYSRQ_ENABLE_BOOT,
#10 lore raw
Quoted 9 lines · Joerg Roedel

Hi Guenter,

Quoted 5 lines · Guenter Roeck

To build csky images, you have to disable CONFIG_FRAME_POINTER or use a non-upstream compiler. To build any images reliably, you have to disable CONFIG_GCC_PLUGIN_RANDSTRUCT or use a version of gcc old enough to not support it (gcc 6.x is fine). For mips, you have to specify ARCH and CROSS_COMPILE as environment variables.

Quoted 8 lines

My test setup builds defconfigs for all architectures using the gcc-9.3 based cross-compilers from kernel.org (except the ones I have no compiler for). That used to work for CSky and MIPS(64) when working on my vmalloc changes.

On MIPS the build failure looks like some Makefile breakage, but I didn't dive deeper into that.

Yes, that was discussed elsewhere. A fix has been submitted and should hopefully find its way into the kernel sometime soon.

Quoted 2 lines

For CSky the compiler complains about not supporting '-mbacktrace'.

Yes, that is why you have to disable CONFIG_FRAME_POINTER. '-mbacktrace' is not supported in upstream versions of gcc for csky; it is only supported in the out-of-tree csky toolchain.

Quoted 7 lines
Quoted 3 lines

alpha is a lost case. The offending commit is 0f1c9688a19 ("tty/sysrq:
alpha: export and use __sysrq_get_key_op()"); it looks like that wasn't build tested. It can not be reverted easily because of subsequent changes.

The below diff fixed the alpha build for me, but I am sure another fix is already queued somewhere.

Let's hope it is.

Thanks,
Guenter

Quoted 20 lines
arch/alpha/kernel/setup.c +6 -1
index 6fa802c495b4..8f4ad63a3a9a 100644
@@ -431,8 +431,13 @@ register_cpus(void)
 arch_initcall(register_cpus);
 
 #ifdef CONFIG_MAGIC_SYSRQ
+static void sysrq_reboot_handler(int unused)
+{
+	machine_halt();
+}
+
 static const struct sysrq_key_op srm_sysrq_reboot_op = {
-	.handler	= machine_halt,
+	.handler	= sysrq_reboot_handler,
 	.help_msg       = "reboot(b)",
 	.action_msg     = "Resetting",
 	.enable_mask    = SYSRQ_ENABLE_BOOT,
#11 lore raw
In reply to Guenter Roeck
Quoted 5 lines · Guenter Roeck
Quoted 1 line · Joerg Roedel

I have stopped building unicore32 images since v4.19 since there is no available compiler that is still supported by the kernel. I am surprised that support for it has not been removed from the kernel.

Well, I've done the patches [1] and I'll send them after -rc1 is out. It'll also give kbuild some time to verify I didn't remove too much :)

[1] https://git.kernel.org/pub/scm/linux/kernel/git/rppt/linux.git/log/?h=rm-unicore

Quoted 1 line

Guenter

Signature · 2 lines
Sincerely yours,
Mike.
#12 lore raw
In reply to Joerg Roedel

Hi Joerg,

Quoted 12 lines · Joerg Roedel

Hi Mike,

include/asm-generic/pgalloc.h

Quoted 4 lines · Mike Rapoport

We already have include/asm-generic/pgalloc.h, so maybe something like that patch below would fork. This is not even compile tested.

include/asm-generic/pgalloc.h +0 -0

I experimented a bit with your diff, but it turned out that moving the page-table allocation functions/macros to asm-generic/pgalloc.h does not work on all architectures.

Quoted 7 lines

The reason is that some archs don't use that header at all (e.g. ARC) and have their own version of the functions defined there. That could all be made working, but I decided to no open this can of worms for now.

So I sent out a patch which moves the p?d_alloc_track() functions/macros to a separate header and include it only in mm/vmalloc.c and lib/ioremap.c.

I'm planning to open this can of worms eventually, but having p?d_alloc_track() in a separate header wouldn't be the biggest of my problems :)

Quoted 8 lines

That compiles on all architectures where current Linus' master also builds (it does not for Alpha, CSky, Mips and Mips64), and as usual Hexagon and Unicore32 are not tested because I have no working compiler for those.

Regards,

Joerg

Signature · 2 lines
Sincerely yours,
Mike.
#13 lore raw
In reply to Linus Torvalds
Quoted 11 lines · Linus Torvalds
Quoted 4 lines · Joerg Roedel

I posted the fix for this already:

        https://lore.kernel.org/lkml/[email protected]/

Ugh.

I was going to apply this directly, but as I looked at the patch I just found it fairly illegible.

Quoted 7 lines

Is there some reason why the 5level-fixup.h versions use that very-hard-to-follow macro, rather than the inline functions that the main mm.h file uses?

I'm _assuming_ it's because it gets included in some place where not everything is defined yet, so making it a macro means that it works (later on) when everything has come together..

Exactly, I had all of the p?d_alloc_track() functions as inlines first, but that broke compilation on some obscure architectures, so I followed the rule to make them macros when the p?d_alloc() was also a macro and defined them close together.

Joerg