x86/setup: call early_reserve_memory() earlier

PATCH Applied Juergen Gross 10 participants 36 messages Sep 20, 2021 – Dec 20, 2021 1 message not in the archive

Subsystems (2)

Stable branch x86

lore

Share & tools

lore

Juergen Gross

author

2021-09-20 12:04 UTC #0 lore raw

PATCH v2 1/1 x86/setup: call early_reserve_memory() earlier

Commit a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations") introduced early_reserve_memory() to do all needed initial memblock_reserve() calls in one function. Unfortunately the call of early_reserve_memory() is done too late for Xen dom0, as in some cases a Xen hook called by e820__memory_setup() will need those memory reservations to have happened already.

Move the call of early_reserve_memory() before the call of e820__memory_setup() in order to avoid such problems.

Cc [email protected]

Fixes a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations")

Reported-by Marek Marczykowski-Górecki <[email protected]>

Signed-off-by Juergen Gross <[email protected]>

---
V2:
- update comment (Jan Beulich, Boris Petkov)
- move call down in setup_arch() (Mike Galbraith)

---
 arch/x86/kernel/setup.c | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)

arch/x86/kernel/setup.c +14 ~~-12~~

index 79f164141116..40ed44ead063 100644
@@ -830,6 +830,20 @@ void __init setup_arch(char **cmdline_p)
 
 	x86_init.oem.arch_setup();
 
+	/*
+	 * Do some memory reservations *before* memory is added to memblock, so
+	 * memblock allocations won't overwrite it.
+	 *
+	 * After this point, everything still needed from the boot loader or
+	 * firmware or kernel text should be early reserved or marked not RAM in
+	 * e820. All other memory is free game.
+	 *
+	 * This call needs to happen before e820__memory_setup() which calls the
+	 * xen_memory_setup() on Xen dom0 which relies on the fact that those
+	 * early reservations have happened already.
+	 */
+	early_reserve_memory();
+
 	iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
 	e820__memory_setup();
 	parse_setup_data();

@@ -876,18 +890,6 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
-	/*
-	 * Do some memory reservations *before* memory is added to
-	 * memblock, so memblock allocations won't overwrite it.
-	 * Do it after early param, so we could get (unlikely) panic from
-	 * serial.
-	 *
-	 * After this point everything still needed from the boot loader or
-	 * firmware or kernel text should be early reserved or marked not
-	 * RAM in e820. All other memory is free game.
-	 */
-	early_reserve_memory();
-
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux

Signature · 1 line

2.26.2

Marek Marczykowski-Górecki

Tested-by on v2 1/1

2021-09-21 00:01 UTC #1 lore raw

Quoted 7 lines · Juergen Gross

Commit a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations") introduced early_reserve_memory() to do all needed initial memblock_reserve() calls in one function. Unfortunately the call of early_reserve_memory() is done too late for Xen dom0, as in some cases a Xen hook called by e820__memory_setup() will need those memory reservations to have happened already.

Quoted 7 lines

Move the call of early_reserve_memory() before the call of e820__memory_setup() in order to avoid such problems.
Cc [email protected]
Fixes a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations")
Reported-by Marek Marczykowski-Górecki <[email protected]>
Signed-off-by Juergen Gross <[email protected]>

I confirm this fixes my boot issue too.

Tested-by Marek Marczykowski-Górecki <[email protected]>

Quoted 55 lines

---
V2:
- update comment (Jan Beulich, Boris Petkov)
- move call down in setup_arch() (Mike Galbraith)

---
 arch/x86/kernel/setup.c | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)

arch/x86/kernel/setup.c +14 ~~-12~~

index 79f164141116..40ed44ead063 100644
@@ -830,6 +830,20 @@ void __init setup_arch(char **cmdline_p)
 
 	x86_init.oem.arch_setup();
 
+	/*
+	 * Do some memory reservations *before* memory is added to memblock, so
+	 * memblock allocations won't overwrite it.
+	 *
+	 * After this point, everything still needed from the boot loader or
+	 * firmware or kernel text should be early reserved or marked not RAM in
+	 * e820. All other memory is free game.
+	 *
+	 * This call needs to happen before e820__memory_setup() which calls the
+	 * xen_memory_setup() on Xen dom0 which relies on the fact that those
+	 * early reservations have happened already.
+	 */
+	early_reserve_memory();
+
 	iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
 	e820__memory_setup();
 	parse_setup_data();

@@ -876,18 +890,6 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
-	/*
-	 * Do some memory reservations *before* memory is added to
-	 * memblock, so memblock allocations won't overwrite it.
-	 * Do it after early param, so we could get (unlikely) panic from
-	 * serial.
-	 *
-	 * After this point everything still needed from the boot loader or
-	 * firmware or kernel text should be early reserved or marked not
-	 * RAM in e820. All other memory is free game.
-	 */
-	early_reserve_memory();
-
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux

Signature · 1 line

2.26.2

Signature · 3 lines

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

Nathan Chancellor

Tested-by on v2 1/1

2021-09-21 03:59 UTC #2 lore raw

In reply to Juergen Gross

Quoted 7 lines · Juergen Gross

Commit a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations") introduced early_reserve_memory() to do all needed initial memblock_reserve() calls in one function. Unfortunately the call of early_reserve_memory() is done too late for Xen dom0, as in some cases a Xen hook called by e820__memory_setup() will need those memory reservations to have happened already.

Quoted 7 lines

Move the call of early_reserve_memory() before the call of e820__memory_setup() in order to avoid such problems.
Cc [email protected]
Fixes a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations")
Reported-by Marek Marczykowski-Górecki <[email protected]>
Signed-off-by Juergen Gross <[email protected]>

I had issues on an AMD Ryzen 3 4300G based system with v1. v2 does not trigger any boot issues on that same machine or an Intel i5-4210U based system that I also test with.

Tested-by Nathan Chancellor <[email protected]>

Quoted 56 lines

---
V2:
- update comment (Jan Beulich, Boris Petkov)
- move call down in setup_arch() (Mike Galbraith)

---
 arch/x86/kernel/setup.c | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)

arch/x86/kernel/setup.c +14 ~~-12~~

index 79f164141116..40ed44ead063 100644
@@ -830,6 +830,20 @@ void __init setup_arch(char **cmdline_p)
 
 	x86_init.oem.arch_setup();
 
+	/*
+	 * Do some memory reservations *before* memory is added to memblock, so
+	 * memblock allocations won't overwrite it.
+	 *
+	 * After this point, everything still needed from the boot loader or
+	 * firmware or kernel text should be early reserved or marked not RAM in
+	 * e820. All other memory is free game.
+	 *
+	 * This call needs to happen before e820__memory_setup() which calls the
+	 * xen_memory_setup() on Xen dom0 which relies on the fact that those
+	 * early reservations have happened already.
+	 */
+	early_reserve_memory();
+
 	iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
 	e820__memory_setup();
 	parse_setup_data();

@@ -876,18 +890,6 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
-	/*
-	 * Do some memory reservations *before* memory is added to
-	 * memblock, so memblock allocations won't overwrite it.
-	 * Do it after early param, so we could get (unlikely) panic from
-	 * serial.
-	 *
-	 * After this point everything still needed from the boot loader or
-	 * firmware or kernel text should be early reserved or marked not
-	 * RAM in e820. All other memory is free game.
-	 */
-	early_reserve_memory();
-
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux

Signature · 1 line

2.26.2

tip-bot2 for Juergen Gross

merge bot Applied

2021-09-22 17:19 UTC #3 lore raw

[tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

In reply to Juergen Gross

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     8aa83e6395ce047a506f0b16edca45f36c1ae7f8
Gitweb:        https://git.kernel.org/tip/8aa83e6395ce047a506f0b16edca45f36c1ae7f8
Author:        Juergen Gross <[email protected]>
AuthorDate:    Mon, 20 Sep 2021 14:04:21 +02:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Tue, 21 Sep 2021 09:52:08 +02:00

x86/setup: Call early_reserve_memory() earlier

Commit in Fixes introduced early_reserve_memory() to do all needed initial memblock_reserve() calls in one function. Unfortunately, the call of early_reserve_memory() is done too late for Xen dom0, as in some cases a Xen hook called by e820__memory_setup() will need those memory reservations to have happened already.

Move the call of early_reserve_memory() before the call of e820__memory_setup() in order to avoid such problems.

Fixes a799c2bd29d1 ("x86/setup: Consolidate early memory reservations")

Reported-by Marek Marczykowski-Górecki <[email protected]>

Signed-off-by Juergen Gross <[email protected]>

Signed-off-by Borislav Petkov <[email protected]>

Tested-by Marek Marczykowski-Górecki <[email protected]>

Tested-by Nathan Chancellor <[email protected]>

Cc [email protected]

Link https://lkml.kernel.org/r/[email protected] open here

---
 arch/x86/kernel/setup.c | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)

arch/x86/kernel/setup.c +14 ~~-12~~

index 79f1641..40ed44e 100644
@@ -830,6 +830,20 @@ void __init setup_arch(char **cmdline_p)
 
 	x86_init.oem.arch_setup();
 
+	/*
+	 * Do some memory reservations *before* memory is added to memblock, so
+	 * memblock allocations won't overwrite it.
+	 *
+	 * After this point, everything still needed from the boot loader or
+	 * firmware or kernel text should be early reserved or marked not RAM in
+	 * e820. All other memory is free game.
+	 *
+	 * This call needs to happen before e820__memory_setup() which calls the
+	 * xen_memory_setup() on Xen dom0 which relies on the fact that those
+	 * early reservations have happened already.
+	 */
+	early_reserve_memory();
+
 	iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
 	e820__memory_setup();
 	parse_setup_data();

@@ -876,18 +890,6 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
-	/*
-	 * Do some memory reservations *before* memory is added to
-	 * memblock, so memblock allocations won't overwrite it.
-	 * Do it after early param, so we could get (unlikely) panic from
-	 * serial.
-	 *
-	 * After this point everything still needed from the boot loader or
-	 * firmware or kernel text should be early reserved or marked not
-	 * RAM in e820. All other memory is free game.
-	 */
-	early_reserve_memory();
-
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux

Williams, Dan J

2021-11-04 05:38 UTC #4 lore raw

Re: [tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

Quoted 11 lines · On Wed, 2021-09-22 at 17:18 +0000, tip-bot2 for Juergen Gross wrote:

The following commit has been merged into the x86/urgent branch of tip:
Commit-ID: 8aa83e6395ce047a506f0b16edca45f36c1ae7f8 Gitweb:
https://git.kernel.org/tip/8aa83e6395ce047a506f0b16edca45f36c1ae7f8
Author: Juergen Gross <[email protected]> AuthorDate: Mon, 20 Sep 2021 14:04:21 +02:00 Committer: Borislav Petkov <[email protected]>
CommitterDate: Tue, 21 Sep 2021 09:52:08 +02:00

Quoted 1 line

x86/setup: Call early_reserve_memory() earlier

Hi,

I got a report from Anjaneya that starting with the v5.15 kernel he can no longer use the "efi=nosoftreserve" kernel command line parameter to suppress "soft reservation" behavior. Recall that "soft reserved" is the Linux designation for memory that is marked with the EFI "Special Purpose" attribute.

By inspection, this commit looks like the source of the problem because early_reserve_memory() now runs before parse_early_param(). I suspect that this also affects memmap= since it is also an early_param(). I'll verify that tomorrow when I'm more awake, but wanted to give a heads up in the meantime.

Quoted 66 lines

Commit in Fixes introduced early_reserve_memory() to do all needed initial memblock_reserve() calls in one function. Unfortunately, the call of early_reserve_memory() is done too late for Xen dom0, as in some cases a Xen hook called by e820__memory_setup() will need those memory reservations to have happened already.
Move the call of early_reserve_memory() before the call of e820__memory_setup() in order to avoid such problems.
Fixes a799c2bd29d1 ("x86/setup: Consolidate early memory reservations")
Reported-by Marek Marczykowski-Górecki <[email protected]>
Signed-off-by Juergen Gross <[email protected]>
Signed-off-by Borislav Petkov <[email protected]>
Tested-by Marek Marczykowski-Górecki <[email protected]>
Tested-by Nathan Chancellor <[email protected]>
Cc [email protected]
Link https://lkml.kernel.org/r/[email protected] open here
---
arch/x86/kernel/setup.c | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)
arch/x86/kernel/setup.c +0 -0
index 79f1641..40ed44e 100644
@@ -830,6 +830,20 @@ void __init setup_arch(char **cmdline_p)
        x86_init.oem.arch_setup();

+       /*
+        * Do some memory reservations *before* memory is added to memblock, so +        * memblock allocations won't overwrite it.
+        *
+        * After this point, everything still needed from the boot loader or +        * firmware or kernel text should be early reserved or marked not RAM in +        * e820. All other memory is free game.
+        *
+        * This call needs to happen before e820__memory_setup() which calls the +        * xen_memory_setup() on Xen dom0 which relies on the fact that those +        * early reservations have happened already.
+        */
+       early_reserve_memory();
+
        iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1; e820__memory_setup();
        parse_setup_data();
arch/x86/kernel/setup.c +0 -0
@@ -876,18 +890,6 @@ void __init setup_arch(char **cmdline_p)
        parse_early_param();

-       /*
-        * Do some memory reservations *before* memory is added to -        * memblock, so memblock allocations won't overwrite it. -        * Do it after early param, so we could get (unlikely) panic from -        * serial.
-        *
-        * After this point everything still needed from the boot loader or -        * firmware or kernel text should be early reserved or marked not -        * RAM in e820. All other memory is free game.
-        */
-       early_reserve_memory();
-
#ifdef CONFIG_MEMORY_HOTPLUG
        /*
         * Memory used by the kernel cannot be hot-removed because Linux

Borislav Petkov

maintainer

2021-11-04 11:40 UTC #5 lore raw

Re: [tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

+ Mike.

Quoted 2 lines · Dan J

By inspection, this commit looks like the source of the problem because early_reserve_memory() now runs before parse_early_param().

Yah, I see it too:

early_reserve_memory
|-> efi_memblock_x86_reserve_range

    |-> efi_fake_memmap_early

which does

        if (!efi_soft_reserve_enabled())
                return;

and that would have set EFI_MEM_NO_SOFT_RESERVE after having parsed "nosoftreserve".

And parse_early_param() happens later.

Uuuf, that early memory reservation dance is not going to be over, ever...

Well, I guess we can do something like this below. The intent being to carve out the command line preparation into a separate function which does the early param parsing too. So that it all goes together.

And then call that function before early_reserve_memory() so that the params have been parsed by then.

Looking at the changed flow, I think we should be ok but I've said that a bunch of times already regarding this memory reservation early and something in our house of cards called early boot order always broke.

Damn.

---

arch/x86/kernel/setup.c +39 ~~-27~~

index 40ed44ead063..05f69e7d84dc 100644
@@ -742,6 +742,28 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
 	return 0;
 }
 
+static char *prepare_command_line(void)
+{
+#ifdef CONFIG_CMDLINE_BOOL
+#ifdef CONFIG_CMDLINE_OVERRIDE
+	strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
+#else
+	if (builtin_cmdline[0]) {
+		/* append boot loader cmdline to builtin */
+		strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
+		strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
+		strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
+	}
+#endif
+#endif
+
+	strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
+
+	parse_early_param();
+
+	return command_line;
+}
+
 /*
  * Determine if we were loaded by an EFI loader.  If so, then we have also been
  * passed the efi memmap, systab, etc., so we should use these data structures

@@ -830,6 +852,23 @@ void __init setup_arch(char **cmdline_p)
 
 	x86_init.oem.arch_setup();
 
+	/*
+	 * x86_configure_nx() is called before parse_early_param() (called by
+	 * prepare_command_line()) to detect whether hardware doesn't support
+	 * NX (so that the early EHCI debug console setup can safely call
+	 * set_fixmap()). It may then be called again from within noexec_setup()
+	 * during parsing early parameters to honor the respective command line
+	 * option.
+	 */
+	x86_configure_nx();
+
+	/*
+	 * This parses early params and it needs to run before
+	 * early_reserve_memory() because latter relies on such settings
+	 * supplies as early params.
+	 */
+	*cmdline_p = prepare_command_line();
+
 	/*
 	 * Do some memory reservations *before* memory is added to memblock, so
 	 * memblock allocations won't overwrite it.

@@ -863,33 +902,6 @@ void __init setup_arch(char **cmdline_p)
 	bss_resource.start = __pa_symbol(__bss_start);
 	bss_resource.end = __pa_symbol(__bss_stop)-1;
 
-#ifdef CONFIG_CMDLINE_BOOL
-#ifdef CONFIG_CMDLINE_OVERRIDE
-	strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
-#else
-	if (builtin_cmdline[0]) {
-		/* append boot loader cmdline to builtin */
-		strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
-		strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
-		strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
-	}
-#endif
-#endif
-
-	strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
-	*cmdline_p = command_line;
-
-	/*
-	 * x86_configure_nx() is called before parse_early_param() to detect
-	 * whether hardware doesn't support NX (so that the early EHCI debug
-	 * console setup can safely call set_fixmap()). It may then be called
-	 * again from within noexec_setup() during parsing early parameters
-	 * to honor the respective command line option.
-	 */
-	x86_configure_nx();
-
-	parse_early_param();
-
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux

Signature · 4 lines

Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Dan Williams

Tested-by Anjaneya Chagam Reviewed-by

2021-11-04 17:36 UTC #6 lore raw

Re: [tip: x86/urgent] x86/setup: Call early_reserve_memory() earlier

Quoted 30 lines · Borislav Petkov

+ Mike.
Quoted 2 lines · Dan J
By inspection, this commit looks like the source of the problem because early_reserve_memory() now runs before parse_early_param().
Yah, I see it too:
early_reserve_memory
|-> efi_memblock_x86_reserve_range
    |-> efi_fake_memmap_early
which does
        if (!efi_soft_reserve_enabled())
                return;
and that would have set EFI_MEM_NO_SOFT_RESERVE after having parsed "nosoftreserve".
And parse_early_param() happens later.
Uuuf, that early memory reservation dance is not going to be over, ever...
Well, I guess we can do something like this below. The intent being to carve out the command line preparation into a separate function which does the early param parsing too. So that it all goes together.

Quoted 8 lines

And then call that function before early_reserve_memory() so that the params have been parsed by then.
Looking at the changed flow, I think we should be ok but I've said that a bunch of times already regarding this memory reservation early and something in our house of cards called early boot order always broke.
Damn.

Thanks Boris!

You can add:

Tested-by Anjaneya Chagam <[email protected]>

Reviewed-by Dan Williams <[email protected]>

Quoted 100 lines

---

arch/x86/kernel/setup.c +39 ~~-28~~

index 40ed44ead063..05f69e7d84dc 100644
@@ -742,6 +742,28 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
        return 0;
 }
 
+static char *prepare_command_line(void)
+{
+#ifdef CONFIG_CMDLINE_BOOL
+#ifdef CONFIG_CMDLINE_OVERRIDE
+       strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
+#else
+       if (builtin_cmdline[0]) {
+               /* append boot loader cmdline to builtin */
+               strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
+               strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
+               strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
+       }
+#endif
+#endif
+
+       strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
+
+       parse_early_param();
+
+       return command_line;
+}
+
 /*
  * Determine if we were loaded by an EFI loader.  If so, then we have also been
  * passed the efi memmap, systab, etc., so we should use these data structures

@@ -830,6 +852,23 @@ void __init setup_arch(char **cmdline_p)
 
        x86_init.oem.arch_setup();
 
+       /*
+        * x86_configure_nx() is called before parse_early_param() (called by
+        * prepare_command_line()) to detect whether hardware doesn't support
+        * NX (so that the early EHCI debug console setup can safely call
+        * set_fixmap()). It may then be called again from within noexec_setup()
+        * during parsing early parameters to honor the respective command line
+        * option.
+        */
+       x86_configure_nx();
+
+       /*
+        * This parses early params and it needs to run before
+        * early_reserve_memory() because latter relies on such settings
+        * supplies as early params.
+        */
+       *cmdline_p = prepare_command_line();
+
        /*
         * Do some memory reservations *before* memory is added to memblock, so
         * memblock allocations won't overwrite it.

@@ -863,33 +902,6 @@ void __init setup_arch(char **cmdline_p)
        bss_resource.start = __pa_symbol(__bss_start);
        bss_resource.end = __pa_symbol(__bss_stop)-1;
 
-#ifdef CONFIG_CMDLINE_BOOL
-#ifdef CONFIG_CMDLINE_OVERRIDE
-       strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
-#else
-       if (builtin_cmdline[0]) {
-               /* append boot loader cmdline to builtin */
-               strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
-               strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
-               strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
-       }
-#endif
-#endif
-
-       strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
-       *cmdline_p = command_line;
-
-       /*
-        * x86_configure_nx() is called before parse_early_param() to detect
-        * whether hardware doesn't support NX (so that the early EHCI debug
-        * console setup can safely call set_fixmap()). It may then be called
-        * again from within noexec_setup() during parsing early parameters
-        * to honor the respective command line option.
-        */
-       x86_configure_nx();
-
-       parse_early_param();
-
 #ifdef CONFIG_MEMORY_HOTPLUG
        /*
         * Memory used by the kernel cannot be hot-removed because Linux
 
 
--

Regards/Gruss,
Boris.

https://people.kernel.org/tglx/notes-about-netiquette

tip-bot2 for Borislav Petkov

merge bot Applied

2021-11-15 11:36 UTC #7 lore raw

[tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

In reply to Williams, Dan J

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     8d48bf8206f77aa8687f0e241e901e5197e52423
Gitweb:        https://git.kernel.org/tip/8d48bf8206f77aa8687f0e241e901e5197e52423
Author:        Borislav Petkov <[email protected]>
AuthorDate:    Fri, 05 Nov 2021 10:41:51 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Mon, 15 Nov 2021 12:27:40 +01:00

x86/boot: Pull up cmdline preparation and early param parsing

Dan reports that Anjaneya Chagam can no longer use the efi=nosoftreserve kernel command line parameter to suppress "soft reservation" behavior.

This is due to the fact that the following call-chain happens at boot:

early_reserve_memory
|-> efi_memblock_x86_reserve_range

    |-> efi_fake_memmap_early

which does

        if (!efi_soft_reserve_enabled())
                return;

and that would have set EFI_MEM_NO_SOFT_RESERVE after having parsed "nosoftreserve".

However, parse_early_param() gets called *after* it, leading to the boot cmdline not being taken into account.

Therefore, carve out the command line preparation into a separate function which does the early param parsing too. So that it all goes together.

And then call that function before early_reserve_memory() so that the params would have been parsed by then.

Fixes 8aa83e6395ce ("x86/setup: Call early_reserve_memory() earlier")

Reported-by Dan Williams <[email protected]>

Reviewed-by Dan Williams <[email protected]>

Signed-off-by Borislav Petkov <[email protected]>

Tested-by Anjaneya Chagam <[email protected]>

Cc <[email protected]>

Link https://lore.kernel.org/r/[email protected] open here

---
 arch/x86/kernel/setup.c | 66 +++++++++++++++++++++++-----------------
 1 file changed, 39 insertions(+), 27 deletions(-)

arch/x86/kernel/setup.c +39 ~~-27~~

index 49b596d..c410be7 100644
@@ -742,6 +742,28 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
 	return 0;
 }
 
+static char *prepare_command_line(void)
+{
+#ifdef CONFIG_CMDLINE_BOOL
+#ifdef CONFIG_CMDLINE_OVERRIDE
+	strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
+#else
+	if (builtin_cmdline[0]) {
+		/* append boot loader cmdline to builtin */
+		strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
+		strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
+		strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
+	}
+#endif
+#endif
+
+	strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
+
+	parse_early_param();
+
+	return command_line;
+}
+
 /*
  * Determine if we were loaded by an EFI loader.  If so, then we have also been
  * passed the efi memmap, systab, etc., so we should use these data structures

@@ -831,6 +853,23 @@ void __init setup_arch(char **cmdline_p)
 	x86_init.oem.arch_setup();
 
 	/*
+	 * x86_configure_nx() is called before parse_early_param() (called by
+	 * prepare_command_line()) to detect whether hardware doesn't support
+	 * NX (so that the early EHCI debug console setup can safely call
+	 * set_fixmap()). It may then be called again from within noexec_setup()
+	 * during parsing early parameters to honor the respective command line
+	 * option.
+	 */
+	x86_configure_nx();
+
+	/*
+	 * This parses early params and it needs to run before
+	 * early_reserve_memory() because latter relies on such settings
+	 * supplied as early params.
+	 */
+	*cmdline_p = prepare_command_line();
+
+	/*
 	 * Do some memory reservations *before* memory is added to memblock, so
 	 * memblock allocations won't overwrite it.
 	 *

@@ -863,33 +902,6 @@ void __init setup_arch(char **cmdline_p)
 	bss_resource.start = __pa_symbol(__bss_start);
 	bss_resource.end = __pa_symbol(__bss_stop)-1;
 
-#ifdef CONFIG_CMDLINE_BOOL
-#ifdef CONFIG_CMDLINE_OVERRIDE
-	strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
-#else
-	if (builtin_cmdline[0]) {
-		/* append boot loader cmdline to builtin */
-		strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
-		strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
-		strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
-	}
-#endif
-#endif
-
-	strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
-	*cmdline_p = command_line;
-
-	/*
-	 * x86_configure_nx() is called before parse_early_param() to detect
-	 * whether hardware doesn't support NX (so that the early EHCI debug
-	 * console setup can safely call set_fixmap()). It may then be called
-	 * again from within noexec_setup() during parsing early parameters
-	 * to honor the respective command line option.
-	 */
-	x86_configure_nx();
-
-	parse_early_param();
-
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux

John Dorminy

2021-12-09 14:38 UTC #8 lore raw

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

Greetings;

It seems that this patch causes a mem= parameter to the kernel to have no effect, unfortunately...

As far as I understand, the x86 mem parameter handler parse_memopt() (called by parse_early_param()) relies on being called after e820__memory_setup(): it simply removes any memory above the specified limit at that moment, allowing memory to later be hotplugged without regard for the initial limit. However, the initial non-hotplugged memory must already have been set up, in e820__memory_setup(), so that it can be removed in parse_memopt(); if parse_early_param() is called before e820__memory_setup(), as this change does, the parameter ends up having no effect.

I apologize that I don't know how to fix this, but I'm happy to test patches.

Typical dmesg output showing the lack of effect, built from the prior change and this change:

With a git tree synced to 8d48bf8206f77aa8687f0e241e901e5197e52423^ (working):

[    0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc1 root=UUID=a4f7bd84-4f29-40bc-8c98-f4a72d0856c4 ro net.ifnames=0 crashkernel=128M mem=4G
...
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable
[    0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable
[    0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] user-defined physical RAM map:
[    0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable
[    0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved
[    0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable
[    0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
[    0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
[    0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
[    0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
[    0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved
[    0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
[    0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
...
[    0.025617] Memory: 1762876K/2061136K available (16394K kernel code, 3568K rwdata, 10324K rodata, 2676K init, 4924K bss, 298000K reserved, 0K cma-reserved)

Synced 8d48bf8206f77aa8687f0e241e901e5197e52423 (not working):

[    0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc4+ root=UUID=0e750e61-b92e-4708-a974-c50a3fb7e969 ro net.ifnames=0 crashkernel=128M mem=4G
[    0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable
[    0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] user-defined physical RAM map:
[    0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable
[    0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved
[    0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable
[    0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
[    0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
[    0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
[    0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
[    0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved
[    0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
[    0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] user: [mem 0x0000000100000000-0x000000207fffffff] usable
...
[    0.695267] Memory: 131657608K/134181712K available (16394K kernel code, 3568K rwdata, 10328K rodata, 2676K init, 4924K bss, 2523844K reserved, 0K cma-reserved)

Borislav Petkov

maintainer

2021-12-09 15:19 UTC #9 lore raw

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

+ Hugh and Patrick.

Quoted 7 lines · John Dorminy

Greetings;
It seems that this patch causes a mem= parameter to the kernel to have no effect, unfortunately...
As far as I understand, the x86 mem parameter handler parse_memopt() (called by parse_early_param()) relies on being called after e820__memory_setup(): it simply removes any memory above the specified limit at that moment, allowing memory to later be hotplugged without regard for the initial limit. However, the initial non-hotplugged memory must already have been set up, in e820__memory_setup(), so that it can be removed in parse_memopt(); if parse_early_param() is called before e820__memory_setup(), as this change does, the parameter ends up having no effect.
I apologize that I don't know how to fix this, but I'm happy to test patches.

Yeah, people have been reporting boot failures with mem= on the cmdline.

I think I see why, can you try this one:

---

arch/x86/kernel/setup.c +2 -1

index 6a190c7f4d71..6db971e61e4b 100644
@@ -862,6 +862,8 @@ void __init setup_arch(char **cmdline_p)
 	 */
 	x86_configure_nx();
 
+	e820__memory_setup();
+
 	/*
 	 * This parses early params and it needs to run before
 	 * early_reserve_memory() because latter relies on such settings

@@ -884,7 +886,6 @@ void __init setup_arch(char **cmdline_p)
 	early_reserve_memory();
 
 	iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
-	e820__memory_setup();
 	parse_setup_data();
 
 	copy_edd();

---

Leaving in the rest for the newly added folks.

Quoted 69 lines

Typical dmesg output showing the lack of effect, built from the prior change and this change:

With a git tree synced to 8d48bf8206f77aa8687f0e241e901e5197e52423^ (working):

[    0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc1 root=UUID=a4f7bd84-4f29-40bc-8c98-f4a72d0856c4 ro net.ifnames=0 crashkernel=128M mem=4G
...
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable
[    0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable
[    0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] user-defined physical RAM map:
[    0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable
[    0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved
[    0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable
[    0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
[    0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
[    0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
[    0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
[    0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved
[    0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
[    0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
...
[    0.025617] Memory: 1762876K/2061136K available (16394K kernel code, 3568K rwdata, 10324K rodata, 2676K init, 4924K bss, 298000K reserved, 0K cma-reserved)

Synced 8d48bf8206f77aa8687f0e241e901e5197e52423 (not working):

[    0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc4+ root=UUID=0e750e61-b92e-4708-a974-c50a3fb7e969 ro net.ifnames=0 crashkernel=128M mem=4G
[    0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable
[    0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] user-defined physical RAM map:
[    0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable
[    0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved
[    0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable
[    0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
[    0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
[    0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
[    0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
[    0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved
[    0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
[    0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] user: [mem 0x0000000100000000-0x000000207fffffff] usable
...
[    0.695267] Memory: 131657608K/134181712K available (16394K kernel code, 3568K rwdata, 10328K rodata, 2676K init, 4924K bss, 2523844K reserved, 0K cma-reserved)

Signature · 4 lines

Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Juergen Gross

author

2021-12-09 15:27 UTC #10 lore raw

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

Quoted 7 lines · Borislav Petkov

+ Hugh and Patrick.
Quoted 4 lines · John Dorminy
Greetings;
It seems that this patch causes a mem= parameter to the kernel to have no effect, unfortunately...

Quoted 7 lines

Quoted 3 lines
As far as I understand, the x86 mem parameter handler parse_memopt() (called by parse_early_param()) relies on being called after e820__memory_setup(): it simply removes any memory above the specified limit at that moment, allowing memory to later be hotplugged without regard for the initial limit. However, the initial non-hotplugged memory must already have been set up, in e820__memory_setup(), so that it can be removed in parse_memopt(); if parse_early_param() is called before e820__memory_setup(), as this change does, the parameter ends up having no effect.
I apologize that I don't know how to fix this, but I'm happy to test patches.
Yeah, people have been reporting boot failures with mem= on the cmdline.
I think I see why, can you try this one:

Sigh. This will break Xen PV. Again. The comment above the call of early_reserve_memory() tells you why.

Juergen

Quoted 97 lines

---

arch/x86/kernel/setup.c +2 -2

index 6a190c7f4d71..6db971e61e4b 100644
@@ -862,6 +862,8 @@ void __init setup_arch(char **cmdline_p)
  	 */
  	x86_configure_nx();
 
+	e820__memory_setup();
+
  	/*
  	 * This parses early params and it needs to run before
  	 * early_reserve_memory() because latter relies on such settings

@@ -884,7 +886,6 @@ void __init setup_arch(char **cmdline_p)
  	early_reserve_memory();
 
  	iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
-	e820__memory_setup();
  	parse_setup_data();
 
  	copy_edd();
---

Leaving in the rest for the newly added folks.

Quoted 69 lines

Typical dmesg output showing the lack of effect, built from the prior change and this change:

With a git tree synced to 8d48bf8206f77aa8687f0e241e901e5197e52423^ (working):

[    0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc1 root=UUID=a4f7bd84-4f29-40bc-8c98-f4a72d0856c4 ro net.ifnames=0 crashkernel=128M mem=4G
...
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable
[    0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable
[    0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] user-defined physical RAM map:
[    0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable
[    0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved
[    0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable
[    0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
[    0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
[    0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
[    0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
[    0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved
[    0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
[    0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
...
[    0.025617] Memory: 1762876K/2061136K available (16394K kernel code, 3568K rwdata, 10324K rodata, 2676K init, 4924K bss, 298000K reserved, 0K cma-reserved)

Synced 8d48bf8206f77aa8687f0e241e901e5197e52423 (not working):

[    0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc4+ root=UUID=0e750e61-b92e-4708-a974-c50a3fb7e969 ro net.ifnames=0 crashkernel=128M mem=4G
[    0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable
[    0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] user-defined physical RAM map:
[    0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable
[    0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved
[    0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable
[    0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
[    0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
[    0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
[    0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
[    0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved
[    0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
[    0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] user: [mem 0x0000000100000000-0x000000207fffffff] usable
...
[    0.695267] Memory: 131657608K/134181712K available (16394K kernel code, 3568K rwdata, 10328K rodata, 2676K init, 4924K bss, 2523844K reserved, 0K cma-reserved)

Borislav Petkov

maintainer

2021-12-09 15:28 UTC #11 lore raw

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

Quoted 2 lines · Juergen Gross

Sigh. This will break Xen PV. Again. The comment above the call of early_reserve_memory() tells you why.

I know. I was just looking at how to fix that particular thing and was going to find you on IRC to talk to you about it...

Signature · 4 lines

Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Mike Rapoport

2021-12-09 16:29 UTC #12 lore raw

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

Quoted 6 lines · Borislav Petkov

Quoted 2 lines · Juergen Gross
Sigh. This will break Xen PV. Again. The comment above the call of early_reserve_memory() tells you why.
I know. I was just looking at how to fix that particular thing and was going to find you on IRC to talk to you about it...

The memory reservation in arch/x86/platform/efi/efi.c depends on at least two command line parameters, I think it's better put it back later in the boot process and move efi_memblock_x86_reserve_range() out of early_memory_reserve().

I.e. revert c0f2077baa41 ("x86/boot: Mark prepare_command_line() __init") and 8d48bf8206f7 ("x86/boot: Pull up cmdline preparation and early param parsing") and add the patch below on top.

arch/x86/kernel/setup.c +3 -3

index 49b596db5631..da36b8f8430b 100644
@@ -713,9 +713,6 @@ static void __init early_reserve_memory(void)
 
 	early_reserve_initrd();
 
-	if (efi_enabled(EFI_BOOT))
-		efi_memblock_x86_reserve_range();
-
 	memblock_x86_reserve_range_setup_data();
 
 	reserve_ibft_region();

@@ -890,6 +887,9 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
+	if (efi_enabled(EFI_BOOT)) {
+		efi_memblock_x86_reserve_range();
+
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux

Signature · 2 lines

Sincerely yours,
Mike.

Borislav Petkov

maintainer

2021-12-09 16:37 UTC #13 lore raw

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

Quoted 43 lines · Mike Rapoport

Quoted 6 lines · Borislav Petkov
Quoted 2 lines · Juergen Gross
Sigh. This will break Xen PV. Again. The comment above the call of early_reserve_memory() tells you why.
I know. I was just looking at how to fix that particular thing and was going to find you on IRC to talk to you about it...
The memory reservation in arch/x86/platform/efi/efi.c depends on at least two command line parameters, I think it's better put it back later in the boot process and move efi_memblock_x86_reserve_range() out of early_memory_reserve().
I.e. revert c0f2077baa41 ("x86/boot: Mark prepare_command_line() __init") and 8d48bf8206f7 ("x86/boot: Pull up cmdline preparation and early param parsing") and add the patch below on top.
arch/x86/kernel/setup.c +3 -3
index 49b596db5631..da36b8f8430b 100644
@@ -713,9 +713,6 @@ static void __init early_reserve_memory(void)
 
 	early_reserve_initrd();
 
-	if (efi_enabled(EFI_BOOT))
-		efi_memblock_x86_reserve_range();
-
 	memblock_x86_reserve_range_setup_data();
 
 	reserve_ibft_region();
@@ -890,6 +887,9 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
+	if (efi_enabled(EFI_BOOT)) {
+		efi_memblock_x86_reserve_range();
+
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux
Signature · 1 line
-- 

Jürgen and I were thinking about a different fix but that's probably ok too. But I've said that already about this mess and there's always something we haven't thought about.

Whatever we do, it needs to be tested by all folks on Cc who already reported regressions, i.e., Anjaneya, Hugh, John and Patrick.

Thx.

Signature · 4 lines

Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Borislav Petkov

maintainer

2021-12-10 11:28 UTC #14 lore raw

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

Quoted 2 lines · Borislav Petkov

Whatever we do, it needs to be tested by all folks on Cc who already reported regressions, i.e., Anjaneya, Hugh, John and Patrick.

Ok, Mike is busy so here are some patches for testing:

https://git.kernel.org/pub/scm/linux/kernel/git/bp/bp.git/log/?h=rc4-boot

I'd appreciate it if folks who reported an issue, verify those.

The first two are reverts which should address the issues with mem= folks have reported. And the last one should address Anjaneya's issue.

I guess doing it the way as Mike suggested is cleaner/better.

Thx!

Signature · 4 lines

Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Hugh Dickins

2021-12-10 20:11 UTC #15 lore raw

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

Quoted 6 lines · Borislav Petkov

Quoted 2 lines · Borislav Petkov
Whatever we do, it needs to be tested by all folks on Cc who already reported regressions, i.e., Anjaneya, Hugh, John and Patrick.
Ok, Mike is busy so here are some patches for testing:

Quoted 8 lines

https://git.kernel.org/pub/scm/linux/kernel/git/bp/bp.git/log/?h=rc4-boot
I'd appreciate it if folks who reported an issue, verify those.
The first two are reverts which should address the issues with mem= folks have reported. And the last one should address Anjaneya's issue.
I guess doing it the way as Mike suggested is cleaner/better.

Yes, mem= works fine for me, on both machines, 64-bit and 32-bit, thanks; but I'm not exercising the troublesome EFI case at all.

Hugh

Borislav Petkov

maintainer

2021-12-11 10:14 UTC #16 lore raw

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

Quoted 2 lines · Hugh Dickins

Yes, mem= works fine for me, on both machines, 64-bit and 32-bit, thanks;

Thanks!

Quoted 1 line

but I'm not exercising the troublesome EFI case at all.

Yeah, I added some debug printks in a VM yesterday to confirm the ordering. But will give Anjaneya some more time to verify, before I queue them next week.

Signature · 4 lines

Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Patrick J. Volkerding

2021-12-10 20:32 UTC #17 lore raw

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

In reply to Borislav Petkov

Quoted 5 lines · Borislav Petkov

Quoted 2 lines · Borislav Petkov
Whatever we do, it needs to be tested by all folks on Cc who already reported regressions, i.e., Anjaneya, Hugh, John and Patrick.

Quoted 8 lines

Ok, Mike is busy so here are some patches for testing:
https://git.kernel.org/pub/scm/linux/kernel/git/bp/bp.git/log/?h=rc4-boot
I'd appreciate it if folks who reported an issue, verify those.
The first two are reverts which should address the issues with mem= folks have reported. And the last one should address Anjaneya's issue.

I applied the two revert patches to 5.15.7 (the last one won't apply so I skipped it) and the resulting x86 32-bit kernel boots fine here on the Thinkpad X1E that was having issues previously.

Then I tested an unpatched 5.16-rc4, which (as expected) got the boot hang on the affected machine. Applied the three patches, and the resulting kernel boots fine.

Take care,

Pat

Borislav Petkov

maintainer

2021-12-11 10:29 UTC #18 lore raw

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

Quoted 2 lines · Patrick J. Volkerding

I applied the two revert patches to 5.15.7 (the last one won't apply so I skipped it)

Looks like it is only a contextual conflict.

Quoted 2 lines

and the resulting x86 32-bit kernel boots fine here on the Thinkpad X1E that was having issues previously.

Good.

Quoted 3 lines

Then I tested an unpatched 5.16-rc4, which (as expected) got the boot hang on the affected machine. Applied the three patches, and the resulting kernel boots fine.

Thanks for testing!

Signature · 4 lines

Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

John Dorminy

2021-12-11 05:24 UTC #19 lore raw

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

In reply to Borislav Petkov

Apologies for delay; my dev machine was broken much of today. But I have tested this patch under the same conditions as previously, and agree with Hugh that the patches make mem= work correctly.

Thanks!

John Dorminy

Borislav Petkov

maintainer

2021-12-11 10:30 UTC #20 lore raw

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

Quoted 5 lines · John Dorminy

Apologies for delay; my dev machine was broken much of today. But I have tested this patch under the same conditions as previously, and agree with Hugh that the patches make mem= work correctly.
Thanks!

Thanks for testing!

Signature · 4 lines

Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Mike Rapoport

2021-12-13 08:20 UTC #21 lore raw

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

In reply to Borislav Petkov

Quoted 7 lines · Borislav Petkov

Quoted 2 lines · Borislav Petkov
Whatever we do, it needs to be tested by all folks on Cc who already reported regressions, i.e., Anjaneya, Hugh, John and Patrick.
Ok, Mike is busy so here are some patches for testing:
https://git.kernel.org/pub/scm/linux/kernel/git/bp/bp.git/log/?h=rc4-boot

Thanks for taking care of this!

Signature · 2 lines

Sincerely yours,
Mike.

Borislav Petkov

maintainer

2021-12-13 09:34 UTC #22 lore raw

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

Quoted 1 line · Mike Rapoport

Thanks for taking care of this!

Sure, no probs.

Lemme send them out officially so they're on the list. Will queue them this week.

Thx.

Signature · 4 lines

Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Borislav Petkov

maintainer

2021-12-13 11:28 UTC #23 lore raw

cover x86: Fix boot ordering issues yet again

From: Borislav Petkov <[email protected]>

Hi,

here are the three I'm going to queue this week. Thanks again to everyone for the quick testing.

Borislav Petkov (2):
Revert "x86/boot: Mark prepare_command_line() __init" Revert "x86/boot: Pull up cmdline preparation and early param parsing"

Mike Rapoport (1):
x86/boot: Move EFI range reservation after cmdline parsing

 arch/x86/kernel/setup.c | 72 +++++++++++++++++------------------------
 1 file changed, 30 insertions(+), 42 deletions(-)

Signature · 1 line

2.29.2

Borislav Petkov

maintainer

2021-12-13 11:28 UTC #24 lore raw

PATCH 1/3 Revert "x86/boot: Mark prepare_command_line() __init"

From: Borislav Petkov <[email protected]>

This reverts commit c0f2077baa4113f38f008b8e912b9fb3ff8d43df.

Signed-off-by Borislav Petkov <[email protected]>

---
 arch/x86/kernel/setup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

arch/x86/kernel/setup.c +1 -1

index 6a190c7f4d71..c410be738ae7 100644
@@ -742,7 +742,7 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
 	return 0;
 }
 
-static char * __init prepare_command_line(void)
+static char *prepare_command_line(void)
 {
 #ifdef CONFIG_CMDLINE_BOOL
 #ifdef CONFIG_CMDLINE_OVERRIDE

Signature · 1 line

2.29.2

tip-bot2 for Borislav Petkov

merge bot Applied

2021-12-15 13:05 UTC #25 lore raw

[tip: x86/urgent] Revert "x86/boot: Mark prepare_command_line() __init"

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     58e138d62476fc5f889252dcf73848beeaa54789
Gitweb:        https://git.kernel.org/tip/58e138d62476fc5f889252dcf73848beeaa54789
Author:        Borislav Petkov <[email protected]>
AuthorDate:    Mon, 13 Dec 2021 12:27:55 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Wed, 15 Dec 2021 11:14:28 +01:00

Revert "x86/boot: Mark prepare_command_line() __init"

This reverts commit c0f2077baa4113f38f008b8e912b9fb3ff8d43df.

Signed-off-by Borislav Petkov <[email protected]>

Link https://lore.kernel.org/r/[email protected] open here

---
 arch/x86/kernel/setup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

arch/x86/kernel/setup.c +1 -1

index 6a190c7..c410be7 100644
@@ -742,7 +742,7 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
 	return 0;
 }
 
-static char * __init prepare_command_line(void)
+static char *prepare_command_line(void)
 {
 #ifdef CONFIG_CMDLINE_BOOL
 #ifdef CONFIG_CMDLINE_OVERRIDE

Borislav Petkov

maintainer

2021-12-13 11:28 UTC #26 lore raw

PATCH 2/3 Revert "x86/boot: Pull up cmdline preparation and early param parsing"

In reply to Borislav Petkov

From: Borislav Petkov <[email protected]>

This reverts commit 8d48bf8206f77aa8687f0e241e901e5197e52423.

It turned out to be a bad idea as it broke supplying mem= cmdline parameters due to parse_memopt() requiring preparatory work like setting up the e820 table in e820__memory_setup() in order to be able to exclude the range specified by mem=.

Pulling that up would've broken Xen PV again, see threads at

https://lkml.kernel.org/r/[email protected]

due to xen_memory_setup() needing the first reservations in early_reserve_memory() - kernel and initrd - to have happened already.

This could be fixed again by having Xen do those reservations itself...

Long story short, revert this and do a simpler fix in a later patch.

Signed-off-by Borislav Petkov <[email protected]>

---
 arch/x86/kernel/setup.c | 66 +++++++++++++++++------------------------
 1 file changed, 27 insertions(+), 39 deletions(-)

arch/x86/kernel/setup.c +27 ~~-39~~

index c410be738ae7..49b596db5631 100644
@@ -742,28 +742,6 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
 	return 0;
 }
 
-static char *prepare_command_line(void)
-{
-#ifdef CONFIG_CMDLINE_BOOL
-#ifdef CONFIG_CMDLINE_OVERRIDE
-	strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
-#else
-	if (builtin_cmdline[0]) {
-		/* append boot loader cmdline to builtin */
-		strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
-		strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
-		strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
-	}
-#endif
-#endif
-
-	strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
-
-	parse_early_param();
-
-	return command_line;
-}
-
 /*
  * Determine if we were loaded by an EFI loader.  If so, then we have also been
  * passed the efi memmap, systab, etc., so we should use these data structures

@@ -852,23 +830,6 @@ void __init setup_arch(char **cmdline_p)
 
 	x86_init.oem.arch_setup();
 
-	/*
-	 * x86_configure_nx() is called before parse_early_param() (called by
-	 * prepare_command_line()) to detect whether hardware doesn't support
-	 * NX (so that the early EHCI debug console setup can safely call
-	 * set_fixmap()). It may then be called again from within noexec_setup()
-	 * during parsing early parameters to honor the respective command line
-	 * option.
-	 */
-	x86_configure_nx();
-
-	/*
-	 * This parses early params and it needs to run before
-	 * early_reserve_memory() because latter relies on such settings
-	 * supplied as early params.
-	 */
-	*cmdline_p = prepare_command_line();
-
 	/*
 	 * Do some memory reservations *before* memory is added to memblock, so
 	 * memblock allocations won't overwrite it.

@@ -902,6 +863,33 @@ void __init setup_arch(char **cmdline_p)
 	bss_resource.start = __pa_symbol(__bss_start);
 	bss_resource.end = __pa_symbol(__bss_stop)-1;
 
+#ifdef CONFIG_CMDLINE_BOOL
+#ifdef CONFIG_CMDLINE_OVERRIDE
+	strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
+#else
+	if (builtin_cmdline[0]) {
+		/* append boot loader cmdline to builtin */
+		strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
+		strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
+		strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
+	}
+#endif
+#endif
+
+	strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
+	*cmdline_p = command_line;
+
+	/*
+	 * x86_configure_nx() is called before parse_early_param() to detect
+	 * whether hardware doesn't support NX (so that the early EHCI debug
+	 * console setup can safely call set_fixmap()). It may then be called
+	 * again from within noexec_setup() during parsing early parameters
+	 * to honor the respective command line option.
+	 */
+	x86_configure_nx();
+
+	parse_early_param();
+
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux

Signature · 1 line

2.29.2

tip-bot2 for Borislav Petkov

merge bot Applied

2021-12-15 13:05 UTC #27 lore raw

[tip: x86/urgent] Revert "x86/boot: Pull up cmdline preparation and early param parsing"

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     fbe6183998546f8896ee0b620ece86deff5a2fd1
Gitweb:        https://git.kernel.org/tip/fbe6183998546f8896ee0b620ece86deff5a2fd1
Author:        Borislav Petkov <[email protected]>
AuthorDate:    Mon, 13 Dec 2021 12:27:56 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Wed, 15 Dec 2021 11:38:57 +01:00

Revert "x86/boot: Pull up cmdline preparation and early param parsing"

This reverts commit 8d48bf8206f77aa8687f0e241e901e5197e52423.

Pulling that up would've broken Xen PV again, see threads at

https://lkml.kernel.org/r/[email protected]

due to xen_memory_setup() needing the first reservations in early_reserve_memory() - kernel and initrd - to have happened already.

This could be fixed again by having Xen do those reservations itself...

Long story short, revert this and do a simpler fix in a later patch.

Signed-off-by Borislav Petkov <[email protected]>

Link https://lore.kernel.org/r/[email protected] open here

---
 arch/x86/kernel/setup.c | 66 ++++++++++++++++------------------------
 1 file changed, 27 insertions(+), 39 deletions(-)

arch/x86/kernel/setup.c +27 ~~-39~~

index c410be7..49b596d 100644
@@ -742,28 +742,6 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
 	return 0;
 }
 
-static char *prepare_command_line(void)
-{
-#ifdef CONFIG_CMDLINE_BOOL
-#ifdef CONFIG_CMDLINE_OVERRIDE
-	strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
-#else
-	if (builtin_cmdline[0]) {
-		/* append boot loader cmdline to builtin */
-		strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
-		strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
-		strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
-	}
-#endif
-#endif
-
-	strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
-
-	parse_early_param();
-
-	return command_line;
-}
-
 /*
  * Determine if we were loaded by an EFI loader.  If so, then we have also been
  * passed the efi memmap, systab, etc., so we should use these data structures

@@ -853,23 +831,6 @@ void __init setup_arch(char **cmdline_p)
 	x86_init.oem.arch_setup();
 
 	/*
-	 * x86_configure_nx() is called before parse_early_param() (called by
-	 * prepare_command_line()) to detect whether hardware doesn't support
-	 * NX (so that the early EHCI debug console setup can safely call
-	 * set_fixmap()). It may then be called again from within noexec_setup()
-	 * during parsing early parameters to honor the respective command line
-	 * option.
-	 */
-	x86_configure_nx();
-
-	/*
-	 * This parses early params and it needs to run before
-	 * early_reserve_memory() because latter relies on such settings
-	 * supplied as early params.
-	 */
-	*cmdline_p = prepare_command_line();
-
-	/*
 	 * Do some memory reservations *before* memory is added to memblock, so
 	 * memblock allocations won't overwrite it.
 	 *

@@ -902,6 +863,33 @@ void __init setup_arch(char **cmdline_p)
 	bss_resource.start = __pa_symbol(__bss_start);
 	bss_resource.end = __pa_symbol(__bss_stop)-1;
 
+#ifdef CONFIG_CMDLINE_BOOL
+#ifdef CONFIG_CMDLINE_OVERRIDE
+	strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
+#else
+	if (builtin_cmdline[0]) {
+		/* append boot loader cmdline to builtin */
+		strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);
+		strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);
+		strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);
+	}
+#endif
+#endif
+
+	strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
+	*cmdline_p = command_line;
+
+	/*
+	 * x86_configure_nx() is called before parse_early_param() to detect
+	 * whether hardware doesn't support NX (so that the early EHCI debug
+	 * console setup can safely call set_fixmap()). It may then be called
+	 * again from within noexec_setup() during parsing early parameters
+	 * to honor the respective command line option.
+	 */
+	x86_configure_nx();
+
+	parse_early_param();
+
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux

Borislav Petkov

maintainer

2021-12-13 11:28 UTC #28 lore raw

PATCH 3/3 x86/boot: Move EFI range reservation after cmdline parsing

In reply to Borislav Petkov

From: Mike Rapoport <[email protected]>

The memory reservation in arch/x86/platform/efi/efi.c depends on at least two command line parameters. Put it back later in the boot process and move efi_memblock_x86_reserve_range() out of early_memory_reserve().

An attempt to fix this was done in

8d48bf8206f7 ("x86/boot: Pull up cmdline preparation and early param parsing")

but that caused other troubles so it got reverted.

The bug this is addressing is:

Dan reports that Anjaneya Chagam can no longer use the efi=nosoftreserve kernel command line parameter to suppress "soft reservation" behavior.

This is due to the fact that the following call-chain happens at boot:

early_reserve_memory
|-> efi_memblock_x86_reserve_range

    |-> efi_fake_memmap_early

which does

        if (!efi_soft_reserve_enabled())
                return;

and that would have set EFI_MEM_NO_SOFT_RESERVE after having parsed "nosoftreserve".

However, parse_early_param() gets called *after* it, leading to the boot cmdline not being taken into account.

[ bp: Productize into a proper patch. ]

Signed-off-by Mike Rapoport <[email protected]>

Signed-off-by Borislav Petkov <[email protected]>

Link https://lore.kernel.org/r/[email protected] open here

---
 arch/x86/kernel/setup.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

arch/x86/kernel/setup.c +3 -3

index 49b596db5631..e04f5e6eb33f 100644
@@ -713,9 +713,6 @@ static void __init early_reserve_memory(void)
 
 	early_reserve_initrd();
 
-	if (efi_enabled(EFI_BOOT))
-		efi_memblock_x86_reserve_range();
-
 	memblock_x86_reserve_range_setup_data();
 
 	reserve_ibft_region();

@@ -890,6 +887,9 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
+	if (efi_enabled(EFI_BOOT))
+		efi_memblock_x86_reserve_range();
+
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux

Signature · 1 line

2.29.2

tip-bot2 for Mike Rapoport

merge bot Applied

2021-12-15 13:12 UTC #29 lore raw

[tip: x86/urgent] x86/boot: Move EFI range reservation after cmdline parsing

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     2f5b3514c33fecad4003ce0f22ca9691492d310b
Gitweb:        https://git.kernel.org/tip/2f5b3514c33fecad4003ce0f22ca9691492d310b
Author:        Mike Rapoport <[email protected]>
AuthorDate:    Mon, 13 Dec 2021 12:27:57 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Wed, 15 Dec 2021 14:07:54 +01:00

x86/boot: Move EFI range reservation after cmdline parsing

An attempt to fix this was done in

8d48bf8206f7 ("x86/boot: Pull up cmdline preparation and early param parsing")

but that caused other troubles so it got reverted.

The bug this is addressing is:

Dan reports that Anjaneya Chagam can no longer use the efi=nosoftreserve kernel command line parameter to suppress "soft reservation" behavior.

This is due to the fact that the following call-chain happens at boot:

early_reserve_memory
|-> efi_memblock_x86_reserve_range

      |-> efi_fake_memmap_early

which does

        if (!efi_soft_reserve_enabled())
                return;

and that would have set EFI_MEM_NO_SOFT_RESERVE after having parsed "nosoftreserve".

However, parse_early_param() gets called *after* it, leading to the boot cmdline not being taken into account.

[ bp: Turn into a proper patch. ]

Signed-off-by Mike Rapoport <[email protected]>

Signed-off-by Borislav Petkov <[email protected]>

Link https://lore.kernel.org/r/[email protected] open here

---
 arch/x86/kernel/setup.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

arch/x86/kernel/setup.c +3 -3

index 49b596d..e04f5e6 100644
@@ -713,9 +713,6 @@ static void __init early_reserve_memory(void)
 
 	early_reserve_initrd();
 
-	if (efi_enabled(EFI_BOOT))
-		efi_memblock_x86_reserve_range();
-
 	memblock_x86_reserve_range_setup_data();
 
 	reserve_ibft_region();

@@ -890,6 +887,9 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
+	if (efi_enabled(EFI_BOOT))
+		efi_memblock_x86_reserve_range();
+
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux

A message that is not in the archive (sent off-list or not yet received): CANGBn69pGb-nscv8tXN1UKDEQGEMWRKuPVPLgg+q2m7V_sBvHw@mail.gmail.com

Patrick J. Volkerding

2021-12-20 18:49 UTC #31 lore raw

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

Trying again since gmail didn't use plain text and the message got rejected.

We're waiting for these patches to appear in a 5.15 kernel so that we can ship with an unpatched kernel. Will they be queued for the stable kernels sometime soon?

Thanks,

Pat

Quoted 24 lines · Patrick J. Volkerding

Will these patches be queued for the stable kernels soon?
Thanks,
Pat
Quoted 16 lines · On Mon, Dec 13, 2021, 03:33 Borislav Petkov <[email protected]> wrote:
Quoted 1 line · Mike Rapoport
Thanks for taking care of this!
Sure, no probs.
Lemme send them out officially so they're on the list. Will queue them this week.
Thx.
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette

Borislav Petkov

maintainer

2021-12-20 18:59 UTC #32 lore raw

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

Quoted 5 lines · Patrick J. Volkerding

Trying again since gmail didn't use plain text and the message got rejected.
We're waiting for these patches to appear in a 5.15 kernel so that we can ship with an unpatched kernel. Will they be queued for the stable kernels sometime soon?

Yes, they're currently queued here:

https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/log/?h=x86/urgent

and will go to Linus on the weekend.

Which reminds me - they need stable tags. Lemme add those.

Thx.

Signature · 4 lines

Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Borislav Petkov

maintainer

2021-12-20 19:08 UTC #33 lore raw

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

Quoted 1 line · Borislav Petkov

Which reminds me - they need stable tags. Lemme add those.

Actually, it'll be a lot easier if I send backporting instructions to the stable@ folks next week. Yap, I'll do that.

/me adds a TODO.

Signature · 4 lines

Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

John Dorminy

2021-12-09 15:49 UTC #34 lore raw

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

In reply to Borislav Petkov

Quoted 32 lines · Borislav Petkov

+ Hugh and Patrick.
Quoted 7 lines · John Dorminy
Greetings;
It seems that this patch causes a mem= parameter to the kernel to have no effect, unfortunately...
As far as I understand, the x86 mem parameter handler parse_memopt() (called by parse_early_param()) relies on being called after e820__memory_setup(): it simply removes any memory above the specified limit at that moment, allowing memory to later be hotplugged without regard for the initial limit. However, the initial non-hotplugged memory must already have been set up, in e820__memory_setup(), so that it can be removed in parse_memopt(); if parse_early_param() is called before e820__memory_setup(), as this change does, the parameter ends up having no effect.
I apologize that I don't know how to fix this, but I'm happy to test patches.
Yeah, people have been reporting boot failures with mem= on the cmdline.
I think I see why, can you try this one:
---
arch/x86/kernel/setup.c +2 -0
index 6a190c7f4d71..6db971e61e4b 100644
@@ -862,6 +862,8 @@ void __init setup_arch(char **cmdline_p)
         */
        x86_configure_nx();
 
+       e820__memory_setup();
+
        /*
         * This parses early params and it needs to run before
         * early_reserve_memory() because latter relies on such settings

arch/x86/kernel/setup.c void __init setup_arch(char **cmdline_p)

Quoted 8 lines

arch/x86/kernel/setup.c +0 -2

@@ -884,7 +886,6 @@ void __init setup_arch(char **cmdline_p)
        early_reserve_memory();
 
        iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
-       e820__memory_setup();
        parse_setup_data();
 
        copy_edd();
---

Confirmed that that patch makes mem= work again:

[    0.000000] Command line:
BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc4+ root=UUID=0e
750e61-b92e-4708-a974-c50a3fb7e969 ro net.ifnames=0 crashkernel=128M mem=4G
...
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable
[    0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable
[    0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] user-defined physical RAM map:
[    0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable
[    0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved
[    0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable
[    0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved
[    0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS
[    0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved
[    0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS
[    0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved
[    0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved
[    0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
...
[    0.025520] Memory: 1762976K/2061136K available (16394K kernel

code, 3568K rwdata, 10328K rodata, 2676K init, 4924K bss, 297900K reserved, 0K cma-reserved)

Borislav Petkov

maintainer

2021-12-09 16:07 UTC #35 lore raw

Re: [tip: x86/urgent] x86/boot: Pull up cmdline preparation and early param parsing

Quoted 1 line · John Dorminy

Confirmed that that patch makes mem= work again:

Thanks!

Signature · 4 lines

Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

tip-bot2 for Mike Rapoport

merge bot Applied

2021-12-15 13:05 UTC #36 lore raw

[tip: x86/urgent] x86/boot: Move EFI range reservation after cmdline parsing

In reply to Williams, Dan J

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     83757bbb9fe029b704fb28e80c3f2b92f23a1994
Gitweb:        https://git.kernel.org/tip/83757bbb9fe029b704fb28e80c3f2b92f23a1994
Author:        Mike Rapoport <[email protected]>
AuthorDate:    Mon, 13 Dec 2021 12:27:57 +01:00
Committer:     Borislav Petkov <[email protected]>
CommitterDate: Wed, 15 Dec 2021 12:36:47 +01:00

x86/boot: Move EFI range reservation after cmdline parsing

An attempt to fix this was done in

8d48bf8206f7 ("x86/boot: Pull up cmdline preparation and early param parsing")

but that caused other troubles so it got reverted.

The bug this is addressing is:

Dan reports that Anjaneya Chagam can no longer use the efi=nosoftreserve kernel command line parameter to suppress "soft reservation" behavior.

This is due to the fact that the following call-chain happens at boot:

early_reserve_memory
|-> efi_memblock_x86_reserve_range

    |-> efi_fake_memmap_early

which does

        if (!efi_soft_reserve_enabled())
                return;

and that would have set EFI_MEM_NO_SOFT_RESERVE after having parsed "nosoftreserve".

However, parse_early_param() gets called *after* it, leading to the boot cmdline not being taken into account.

[ bp: Produce into a proper patch. ]

Signed-off-by Borislav Petkov <[email protected]>

Signed-off-by Mike Rapoport <[email protected]>

Signed-off-by Borislav Petkov <[email protected]>

Link https://lore.kernel.org/r/[email protected] open here

---
 arch/x86/kernel/setup.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

arch/x86/kernel/setup.c +3 -3

index 49b596d..e04f5e6 100644
@@ -713,9 +713,6 @@ static void __init early_reserve_memory(void)
 
 	early_reserve_initrd();
 
-	if (efi_enabled(EFI_BOOT))
-		efi_memblock_x86_reserve_range();
-
 	memblock_x86_reserve_range_setup_data();
 
 	reserve_ibft_region();

@@ -890,6 +887,9 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
+	if (efi_enabled(EFI_BOOT))
+		efi_memblock_x86_reserve_range();
+
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux