x86-32: Early microcode loading stumbles over disabled DYNAMIC_FTRACE
Hi all,
32-bit kernels currently crash/reboot in early microcode loading when non-dynamic function tracing is enabled. Several functions in that path get instrumented with mcount, but its non-dynamic implementation does not work before paging is enabled (it accesses global variables at wrong addresses).
Below some hunks to get it working again - at least in the absence of any microcode in the initrd. Marking all involved functions as __init is another option (as __init implies notrace). But I bet there is more hidden. I see e.g. a pr_warn() in find_cpio_init that should trigger the issue as well if we hit the error it reports (btw. printing at this point of the boot should not work anyway, should it?).
Better ideas?
Jan
arch/x86/kernel/Makefile +3 -0
@@ -14,6 +14,9 @@ CFLAGS_REMOVE_pvclock.o = -pg CFLAGS_REMOVE_kvmclock.o = -pg CFLAGS_REMOVE_ftrace.o = -pg CFLAGS_REMOVE_early_printk.o = -pg +CFLAGS_REMOVE_microcode_core_early.o = -pg +CFLAGS_REMOVE_microcode_intel_early.o = -pg +CFLAGS_REMOVE_microcode_amd_early.o = -pg endif CFLAGS_irq.o := -I$(src)/../include/asm/trace
arch/x86/lib/Makefile +4 -0
@@ -2,6 +2,10 @@ # Makefile for x86 specific library files. # +ifdef CONFIG_FUNCTION_TRACER +CFLAGS_REMOVE_string_32.o = -pg +endif + inat_tables_script = $(srctree)/arch/x86/tools/gen-insn-attr-x86.awk inat_tables_maps = $(srctree)/arch/x86/lib/x86-opcode-map.txt quiet_cmd_inat_tables = GEN $@
Signature · 2 lines
Siemens AG, Corporate Technology, CT RTC ITP SES-DE Corporate Competence Center Embedded Linux
Borislav Petkov
maintainerQuoted 6 lines · Jan Kiszka
Below some hunks to get it working again - at least in the absence of any microcode in the initrd. Marking all involved functions as
__initis another option (as__initimplies notrace). But I bet there is more hidden. I see e.g. apr_warn()infind_cpio_initthat should trigger the issue as well if we hit the error it reports (btw. printing at this point of the boot should not work anyway, should it?).
I guess we can do early_printk there instead as x86_64_start_kernel() does it but I'm not sure for the 32-bit case where we call load_ucode_bsp/ap before we've even enabled paging.
Signature · 5 lines
Regards/Gruss,
Boris.
Sent from a fat crate under my desk. Formatting is fine.
--Quoted 11 lines · Borislav Petkov
Quoted 6 lines · Jan Kiszka
Below some hunks to get it working again - at least in the absence of any microcode in the initrd. Marking all involved functions as
__initis another option (as__initimplies notrace). But I bet there is more hidden. I see e.g. apr_warn()infind_cpio_initthat should trigger the issue as well if we hit the error it reports (btw. printing at this point of the boot should not work anyway, should it?).I guess we can do
early_printkthere instead asx86_64_start_kernel()does it but I'm not sure for the 32-bit case where we call load_ucode_bsp/ap before we've even enabled paging.
early_printk doesn't seem to work as well on 32-bit until paging was enabled.
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT RTC ITP SES-DE Corporate Competence Center Embedded Linux
H. Peter Anvin
reviewerQuoted 15 lines · Jan Kiszka
Hi all,
32-bit kernels currently crash/reboot in early microcode loading when non-dynamic function tracing is enabled. Several functions in that path get instrumented with mcount, but its non-dynamic implementation does not work before paging is enabled (it accesses global variables at wrong addresses).
Below some hunks to get it working again - at least in the absence of any microcode in the initrd. Marking all involved functions as
__initis another option (as__initimplies notrace). But I bet there is more hidden. I see e.g. apr_warn()infind_cpio_initthat should trigger the issue as well if we hit the error it reports (btw. printing at this point of the boot should not work anyway, should it?).
Quoted 4 lines
Better ideas?
Jan
A better idea would be for the mcount/__fentry__ function to simply return until the function tracing stuff is actually ready.
-hpa
Quoted 16 lines · On Fri, 30 Aug 2013 08:36:58 -0700 "H. Peter Anvin" <[email protected]> wrote:
Quoted 15 lines · Jan Kiszka
Hi all,
32-bit kernels currently crash/reboot in early microcode loading when non-dynamic function tracing is enabled. Several functions in that path get instrumented with mcount, but its non-dynamic implementation does not work before paging is enabled (it accesses global variables at wrong addresses).
Below some hunks to get it working again - at least in the absence of any microcode in the initrd. Marking all involved functions as
__initis another option (as__initimplies notrace). But I bet there is more hidden. I see e.g. apr_warn()infind_cpio_initthat should trigger the issue as well if we hit the error it reports (btw. printing at this point of the boot should not work anyway, should it?).
Quoted 8 lines
Quoted 4 lines
Better ideas?
Jan
A better idea would be for the mcount/__fentry__ function to simply return until the function tracing stuff is actually ready.
It does when dynamic tracing is enabled. But this issue is with static tracing, there's no code modification, thus all functions call mcount/__fentry__ and we need to look at a variable to determine if we should trace or not.
-- Steve
H. Peter Anvin
reviewerQuoted 6 lines · Steven Rostedt
It does when dynamic tracing is enabled. But this issue is with static tracing, there's no code modification, thus all functions call mcount/__fentry__ and we need to look at a variable to determine if we should trace or not.
And of course global variables are a no-no without special handling in pre-paging mode. Sigh.
You *could* bail out of the calling address is < PAGE_OFFSET.
-hpa
Quoted 8 lines · On Fri, 30 Aug 2013 09:35:18 -0700 "H. Peter Anvin" <[email protected]> wrote:
Quoted 6 lines · Steven Rostedt
It does when dynamic tracing is enabled. But this issue is with static tracing, there's no code modification, thus all functions call mcount/__fentry__ and we need to look at a variable to determine if we should trace or not.
Quoted 5 lines
And of course global variables are a no-no without special handling in pre-paging mode. Sigh.
You *could* bail out of the calling address is <
PAGE_OFFSET.
We could do that too, as static tracing already gives us high overhead, I'm not sure how much more overhead another check like this even matters.
Who uses static tracing? I only use it to test that it still works ;-)
-- Steve
H. Peter Anvin
reviewerQuoted 14 lines · Steven Rostedt
Quoted 11 lines · On Fri, 30 Aug 2013 09:35:18 -0700 "H. Peter Anvin" <[email protected]> wrote:
Quoted 6 lines · Steven Rostedt
It does when dynamic tracing is enabled. But this issue is with static tracing, there's no code modification, thus all functions call mcount/__fentry__ and we need to look at a variable to determine if we should trace or not.
And of course global variables are a no-no without special handling in pre-paging mode. Sigh.
Quoted 6 lines
Quoted 2 lines
You *could* bail out of the calling address is <
PAGE_OFFSET.We could do that too, as static tracing already gives us high overhead, I'm not sure how much more overhead another check like this even matters.
It's not too expensive: a compare and a jump.
Quoted 1 line
Who uses static tracing? I only use it to test that it still works ;-)
Can we get rid of it?
Quoted 4 lines · On Fri, 30 Aug 2013 10:16:00 -0700 "H. Peter Anvin" <[email protected]> wrote:
Quoted 1 line
Who uses static tracing? I only use it to test that it still works ;-)
Can we get rid of it?
I've though about it. But as some archs only have static tracing available (I need to revisit to make sure that's still the case), I kept it because I only test static tracing on x86.
I'm not sure if people still use it for x86 or not.
Jan, you use static tracing on x86?
-- Steve
Quoted 8 lines · Steven Rostedt
Quoted 4 lines · On Fri, 30 Aug 2013 10:16:00 -0700 "H. Peter Anvin" <[email protected]> wrote:
Quoted 1 line
Who uses static tracing? I only use it to test that it still works ;-)
Can we get rid of it?
Quoted 7 lines
I've though about it. But as some archs only have static tracing available (I need to revisit to make sure that's still the case), I kept it because I only test static tracing on x86.
I'm not sure if people still use it for x86 or not.
Jan, you use static tracing on x86?
Yes, due to out-of-tree code that is not compatible with dynamic tracing. It might be feasible to change this, I didn't try again recently. If static tracing goes away upstream, I'll have to sooner or later.
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT RTC ITP SES-DE Corporate Competence Center Embedded Linux
H. Peter Anvin
reviewerQuoted 14 lines · Steven Rostedt
Quoted 11 lines · On Fri, 30 Aug 2013 09:35:18 -0700 "H. Peter Anvin" <[email protected]> wrote:
Quoted 6 lines · Steven Rostedt
It does when dynamic tracing is enabled. But this issue is with static tracing, there's no code modification, thus all functions call mcount/__fentry__ and we need to look at a variable to determine if we should trace or not.
And of course global variables are a no-no without special handling in pre-paging mode. Sigh.
Quoted 9 lines
Quoted 2 lines
You *could* bail out of the calling address is <
PAGE_OFFSET.We could do that too, as static tracing already gives us high overhead, I'm not sure how much more overhead another check like this even matters.
Who uses static tracing? I only use it to test that it still works ;-)
Anyway... can we add this to the static tracing code? It is two instructions and is only needed on x86-32. For performance, use dynamic.
-hpa
Quoted 4 lines · On Fri, 30 Aug 2013 13:48:55 -0700 "H. Peter Anvin" <[email protected]> wrote:
Quoted 1 line
Quoted 1 line
You *could* bail out of the calling address is <
PAGE_OFFSET.Anyway... can we add this to the static tracing code? It is two instructions and is only needed on x86-32. For performance, use dynamic.
Sure, do you want to add it ?
-- Steve
Borislav Petkov
maintainerQuoted 9 lines · Steven Rostedt
Quoted 4 lines · On Fri, 30 Aug 2013 13:48:55 -0700 "H. Peter Anvin" <[email protected]> wrote:
Quoted 1 line
Quoted 1 line
You *could* bail out of the calling address is <
PAGE_OFFSET.Anyway... can we add this to the static tracing code? It is two instructions and is only needed on x86-32. For performance, use dynamic.
Sure, do you want to add it ?
Another, simpler option could be to make static tracing depend on !MICROCODE until the first has been removed.
Signature · 2 lines
Regards/Gruss, Boris.
Quoted 6 lines · On Sat, 31 Aug 2013 07:07:37 +0200 Borislav Petkov <[email protected]> wrote:
Quoted 5 lines · Steven Rostedt
Quoted 2 lines · On Fri, 30 Aug 2013 13:48:55 -0700 "H. Peter Anvin" <[email protected]> wrote:
Quoted 1 line
Quoted 1 line
You *could* bail out of the calling address is <
PAGE_OFFSET.
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Anyway... can we add this to the static tracing code? It is two instructions and is only needed on x86-32. For performance, use dynamic.
Sure, do you want to add it ?
Another, simpler option could be to make static tracing depend on !MICROCODE until the first has been removed.
Might be trickier than you think. Static tracing is the default. Dynamic tracing is the extension. By making static tracing depend on !MICROCODE, you basically made function tracing depend on !MICROCODE.
I guess you could make MICROCODE select DYNAMIC_FTRACE, but that could get messy.
-- Steve