Skip to contentSkip to conversation

x86: Configure NX support earlier in setup_arch

PATCH In review Stuart Hayes 5 participants 12 messages Jul 3, 2014 – Nov 13, 2014 x86

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86: Configure NX support earlier in setup_arch

A page fault can crash the kernel very early if an NX bit is set in a page table entry, if the CPU doesn't support NX (or if NX support is disabled in the CPU). Move the call to x86_configure_nx() earlier than parse_setup_data(), since that calls early_memremap().

---

arch/x86/kernel/setup.c renamed from arch/x86/kernel/setup.c.orig +11 -9
@@ -940,6 +940,17 @@ void __init setup_arch(char **cmdline_p)
 
 	iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
 	setup_memory_map();
+
+	/*
+	 * x86_configure_nx() is called before parse_setup_data() to detect
+	 * whether hardware doesn't support NX (so that parse_setup_data()
+	 * can safely call early_memremap() without setting the XD bit in
+	 * the PTE, which can cause an unrecoverable page fault). It may
+	 * then be called again from within noexec_setup() during parsing
+	 * early parameters to honor the respective command line option.
+	 */
+	x86_configure_nx();
+
 	parse_setup_data();
 
 	copy_edd();
@@ -974,15 +985,6 @@ void __init setup_arch(char **cmdline_p)
 	strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
 	*cmdline_p = command_line;
 
-	/*
-	 * x86_configure_nx() is called before parse_early_param() to detect
-	 * whether hardware doesn't support NX (so that the early EHCI debug
-	 * console setup can safely call set_fixmap()). It may then be called
-	 * again from within noexec_setup() during parsing early parameters
-	 * to honor the respective command line option.
-	 */
-	x86_configure_nx();
-
 	parse_early_param();
 
 	x86_report_nx();
#1 lore raw
Quoted 34 lines · Stuart Hayes

A page fault can crash the kernel very early if an NX bit is set in a page table entry, if the CPU doesn't support NX (or if NX support is disabled in the CPU). Move the call to x86_configure_nx() earlier than parse_setup_data(), since that calls early_memremap().

---

arch/x86/kernel/setup.c renamed from arch/x86/kernel/setup.c.orig +11 -2
@@ -940,6 +940,17 @@ void __init setup_arch(char **cmdline_p)
 
 	iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
 	setup_memory_map();
+
+	/*
+	 * x86_configure_nx() is called before parse_setup_data() to detect
+	 * whether hardware doesn't support NX (so that parse_setup_data()
+	 * can safely call early_memremap() without setting the XD bit in
+	 * the PTE, which can cause an unrecoverable page fault). It may
+	 * then be called again from within noexec_setup() during parsing
+	 * early parameters to honor the respective command line option.
+	 */
+	x86_configure_nx();
+
 	parse_setup_data();
 
 	copy_edd();
@@ -974,15 +985,6 @@ void __init setup_arch(char **cmdline_p)
 	strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);
 	*cmdline_p = command_line;
 
-	/*
-	 * x86_configure_nx() is called before parse_early_param() to detect

arch/x86/kernel/setup.c void __init setup_arch(char **cmdline_p) · line 990

Quoted 10 lines
arch/x86/kernel/setup.c renamed from arch/x86/kernel/setup.c.orig +0 -7
-	 * whether hardware doesn't support NX (so that the early EHCI debug
-	 * console setup can safely call set_fixmap()). It may then be called
-	 * again from within noexec_setup() during parsing early parameters
-	 * to honor the respective command line option.
-	 */
-	x86_configure_nx();
-
 	parse_early_param();
 
 	x86_report_nx();

I haven't received any responses... is there a problem with the patch? Also CCing a couple people.

#2 lore raw
Quoted 3 lines · Stuart Hayes

I haven't received any responses... is there a problem with the patch? Also CCing a couple people.

I was on vacation last week and am still catching up.

It would also help if you describe the real-world scenario that made you trip over this.

-hpa

#3 lore raw
Quoted 5 lines · H. Peter Anvin
Quoted 3 lines · Stuart Hayes

I haven't received any responses... is there a problem with the patch? Also CCing a couple people.

Quoted 7 lines

I was on vacation last week and am still catching up.

It would also help if you describe the real-world scenario that made you trip over this.

-hpa

Well... I got this issue because a co-worker tripped over it. He had NX disabled in BIOS for some reason, and found that linux wouldn't boot--it hung right after grub2. I guess it took a while to figure out that it was the fact that NX was disabled that caused linux not to come up--and that could happen to other people. I don't know of any real-world scenarios in which someone would actually prefer to run a recent linux kernel with NX disabled, though.

It looks like some of the other boot paths into the kernel automatically clear the XD_DISABLE bit in the MISC_ENABLE MSR in the CPU (in verify_cpu), but that doesn't happen when grub2 jumps to startup_64 in arch/x86/boot/compressed/head_64.S. I guess instead of this patch, I could try to make a patch that turns NX back on (somewhere in startup_64), but since the kernel already supports NX being disabled, so I thought maybe just fixing that would be better. I didn't like seeing the kernel just die without giving any indication of what the problem is.

Stuart

#4 lore raw
Quoted 8 lines · Stuart Hayes

Well... I got this issue because a co-worker tripped over it. He had NX disabled in BIOS for some reason, and found that linux wouldn't boot--it hung right after grub2. I guess it took a while to figure out that it was the fact that NX was disabled that caused linux not to come up--and that could happen to other people. I don't know of any real-world scenarios in which someone would actually prefer to run a recent linux kernel with NX disabled, though.

Quoted 9 lines

It looks like some of the other boot paths into the kernel automatically clear the XD_DISABLE bit in the MISC_ENABLE MSR in the CPU (in verify_cpu), but that doesn't happen when grub2 jumps to startup_64 in arch/x86/boot/compressed/head_64.S. I guess instead of this patch, I could try to make a patch that turns NX back on (somewhere in startup_64), but since the kernel already supports NX being disabled, so I thought maybe just fixing that would be better. I didn't like seeing the kernel just die without giving any indication of what the problem is.

Well, hpa and I were talking about this briefly and this NX disabling in the BIOS is probably for some broken legacy applications/OSes. Linux enables NX unconditionally very early because disabling it is a very bad idea anyway, security-wise.

So, if this is just a random trip over of a co-worker and doesn't have any sensible use case, I'd rather leave it as is an don't fix it at all.

Signature · 5 lines
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--
#5 lore raw

Oh, it is a case of Grub2 utter braindamage. That figures. I guess we need to invoke verify_cpu in yet another place.

Quoted 30 lines · Borislav Petkov
Quoted 3 lines · Stuart Hayes

Well... I got this issue because a co-worker tripped over it. He had NX disabled in BIOS for some reason, and found that linux wouldn't boot--it hung right after grub2. I guess it took a while to figure

out

Quoted 5 lines

that it was the fact that NX was disabled that caused linux not to come up--and that could happen to other people. I don't know of any real-world scenarios in which someone would actually prefer to run a recent linux kernel with NX disabled, though.

Quoted 7 lines

It looks like some of the other boot paths into the kernel automatically clear the XD_DISABLE bit in the MISC_ENABLE MSR in the CPU (in verify_cpu), but that doesn't happen when grub2 jumps to startup_64 in arch/x86/boot/compressed/head_64.S. I guess instead of this patch, I could try to make a patch that turns NX back on (somewhere in startup_64), but since the kernel already supports NX being disabled, so I thought maybe just fixing that would be better.

I

Quoted 2 lines

didn't like seeing the kernel just die without giving any indication of what the problem is.

Well, hpa and I were talking about this briefly and this NX disabling in the BIOS is probably for some broken legacy applications/OSes. Linux enables NX unconditionally very early because disabling it is a very bad
idea anyway, security-wise.

So, if this is just a random trip over of a co-worker and doesn't have any sensible use case, I'd rather leave it as is an don't fix it at all.

Signature · 1 line
Sent from my mobile phone.  Please pardon brevity and lack of formatting.
#6 lore raw
Quoted 2 lines · H. Peter Anvin

Oh, it is a case of Grub2 utter braindamage. That figures. I guess we need to invoke verify_cpu in yet another place.

How so? The bug report speaks of disabling NX in the BIOS, I'm assuming they're setting bit 34 in IA32_MISC_ENABLE.

How would that affect grub2?

Signature · 5 lines
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--
#7 lore raw
In reply to Borislav Petkov

Hi Boris,

I feel this can be a valid scenario where user wants to disable the memory protection (NX flag disable) for his requirement, in that case he will hit this issue. So request you to please revisit this patch.

Regards
Balaji Singh

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Borislav Petkov
Sent: Monday, July 14, 2014 10:52 PM
To: Stuart Hayes
Cc: H. Peter Anvin; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH] x86: Configure NX support earlier in setup_arch

Quoted 8 lines · Stuart Hayes

Well... I got this issue because a co-worker tripped over it. He had NX disabled in BIOS for some reason, and found that linux wouldn't boot--it hung right after grub2. I guess it took a while to figure out that it was the fact that NX was disabled that caused linux not to come up--and that could happen to other people. I don't know of any real-world scenarios in which someone would actually prefer to run a recent linux kernel with NX disabled, though.

Quoted 9 lines

It looks like some of the other boot paths into the kernel automatically clear the XD_DISABLE bit in the MISC_ENABLE MSR in the CPU (in verify_cpu), but that doesn't happen when grub2 jumps to startup_64 in arch/x86/boot/compressed/head_64.S. I guess instead of this patch, I could try to make a patch that turns NX back on (somewhere in startup_64), but since the kernel already supports NX being disabled, so I thought maybe just fixing that would be better. I didn't like seeing the kernel just die without giving any indication of what the problem is.

Well, hpa and I were talking about this briefly and this NX disabling in the BIOS is probably for some broken legacy applications/OSes. Linux enables NX unconditionally very early because disabling it is a very bad idea anyway, security-wise.

So, if this is just a random trip over of a co-worker and doesn't have any sensible use case, I'd rather leave it as is an don't fix it at all.

--
Regards/Gruss,
Boris.

Sent from a fat crate under my desk. Formatting is fine.
--
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m���� ������?�I�

#8 lore raw
Quoted 3 lines · On Thu, Nov 13, 2014 at 12:09:02PM +0530, [email protected] wrote:

I feel this can be a valid scenario where user wants to disable the memory protection (NX flag disable) for his requirement, in that case he will hit this issue. So request you to please revisit this patch.

First of all, please do not top-post. Here's why:

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

Revisit this patch how? I'm not sure I understand...

Signature · 5 lines
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--
#9 lore raw
Quoted 15 lines

-----Original Message-----
From: Borislav Petkov [mailto:[email protected]]
Sent: Thursday, November 13, 2014 6:01 PM
To: Singh, B B
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Jiri Kosina
Subject: Re: [PATCH] x86: Configure NX support earlier in setup_arch

Quoted 3 lines · On Thu, Nov 13, 2014 at 12:09:02PM +0530, [email protected] wrote:

I feel this can be a valid scenario where user wants to disable the memory protection (NX flag disable) for his requirement, in that case he will hit this issue. So request you to please revisit this patch.

Quoted 7 lines

First of all, please do not top-post. Here's why:

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

Sorry Boris, will make note on this & follow the same.

Quoted 1 line

Revisit this patch how? I'm not sure I understand...

I got an answer from Thomas, will check with BIOS team & come back.

Thanks
Balaji Singh ���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m���� ������?�I�

#10 lore raw
In reply to Borislav Petkov
Quoted 1 line · Borislav Petkov

Revisit this patch how? I'm not sure I understand...

X86_64 starts with:

pteval_t __supported_pte_mask __read_mostly = ~0;

while i386 starts with:

pteval_t __supported_pte_mask __read_mostly = ~(_PAGE_NX | _PAGE_GLOBAL);

Now if the stupid BIOS disabled NX via setting bit 34 in IA32_MISC_ENABLE, then X86_FEATURE_NX is off, but at the point where we call x86_configure_nx() we already used the supported_pte_mask with the NX bit set in the early remap code and accessed the mappings.

On 32bit we are safe because the early maps exclude NX at startup and only enable it in x86_configure_nx().

That's one part of the issue. The other is that grub2 does not call in via the trampoline, so we don't call verify_cpu. verify_cpu clears bit 34 in IA32_MISC_ENABLE depending on the cpu family/model, which is true for 64bit machines.

So I think moving x86_configure_nx() to a point before we actually create mappings is a sane thing to do in any case.

But of course we should also clear the stupid disable bit unconditionally independent of the entry path for all cpus which support it.

Thanks,

tglx

#11 lore raw
In reply to [email protected]
Quoted 3 lines · On Thu, 13 Nov 2014, [email protected] wrote:

I feel this can be a valid scenario where user wants to disable the memory protection (NX flag disable) for his requirement, in that case he will hit this issue.

It's not about feelings. If the user wants to disable NX he can do so already via the kernel command line.

What that patch is useful for is to prevent an hard to debug boot crash if the kernel happens to run on some wreckaged BIOS. But it's not there to support and encourage braindamaged BIOS nonsense.

Thanks,

tglx