x86: fixes for running KUT as EFI on non-QEMU KVM hosts
Subsystems (2)
v2 cover x86: fixes for running KUT as EFI on non-QEMU KVM hosts
This series fixes several issues encountered when running KUT x86 tests as EFI applications on a KVM-based hypervisor that lacks any ad-hoc para-virtualized interface (e.g. fw_cfg), on arbitrary VM shapes (variable CPU count and memory size).
Patches 1-4 fix crashes and faults:
- Stack size mismatch between EFI and non-EFI paths
- Memory allocator selecting regions above the 4 GiB identity map
- #GP on systems without a PMU
- ISR thunks using rel32 jumps that overflow beyond 2 GiB
Patches 5-6 fix SMP bringup:
- Replace per-AP serial prints that cause watchdog timeouts on large systems
- Add timeout-based AP detection when fw_cfg is unavailable
Patches 7-8 fix command line handling:
- Correct the load_options_size byte-to-character conversion
- Add KUT_ENV= parsing for passing environment variables via the command line
Regression-tested by running the full KUT suite under QEMU+KVM+OVMF, in both ./configure and ./configure --enable-efi. Results are identical with and without this series.
v1 -> v2:
- 8/8: reworked per Drew's review. Moved the implementation from lib/efi.c to lib/argv.c so the feature is available to non-EFI builds as well, use a strncmp() prefix match instead of strstr(), shift argv down instead of swapping with the last element so argument order is preserved, and drop the redundant isspace() check and COMMAND_LINE_SIZE cap. Retitled accordingly ("efi: parse KUT_ENV= from load options into environ" -> "lib: parse KUT_ENV= from command line into environ"). This patch was already posted standalone as v2 and reviewed [2]; it is included here unchanged so the series is complete.
- 1/8 - 7/8: unchanged.
- Picked up Andrew Jones' Reviewed-by on 7/8 and 8/8.
- Rebased onto current master.
Patches 1-6 received no review in v1, so they are unchanged and still need a look. Resending the full series to make it reviewable as a whole.
[1] v1: https://lore.kernel.org/kvm/[email protected]/
[2] v2 8/8 standalone: https://lore.kernel.org/kvm/[email protected]/
Giacomo Mazzola (8):
x86: efi: use PER_CPU_SIZE for per-CPU stack allocation
x86: fix EFI memory allocator to clamp regions to 4 GiB
x86: skip PMU init when no PMU is advertised
x86: fix ISR thunk to use absolute indirect jump
x86: replace per-AP bringup prints with a single summary line
x86: add timeout-based SMP bringup when fw_cfg is unavailable
efi: fix load_options_size conversion to character count
lib: parse KUT_ENV= from command line into environ
lib/argv.c | 35 +++++++++++++++++++++++++++++++++++ lib/efi.c | 2 +- lib/x86/apic.c | 1 - lib/x86/isr.c | 8 +++++--- lib/x86/setup.c | 24 ++++++++++++++++++++---- lib/x86/smp.c | 31 +++++++++++++++++++++++++++---- x86/efi/efistart64.S | 2 +- 7 files changed, 89 insertions(+), 14 deletions(-)
base-commit: 5a221342a025948fa7d84f0c404748f6ba57f3ad
Signature · 1 line
2.47.3
PATCH v2 1/8 x86: efi: use PER_CPU_SIZE for per-CPU stack allocation
The EFI startup code reserves PAGE_SIZE (4 KiB) per-CPU stacks, but the non-EFI path (cstart64.S) uses PER_CPU_SIZE (12 KiB). The smaller EFI stack causes stack overflows. Make the two paths consistent.
--- x86/efi/efistart64.S | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
x86/efi/efistart64.S +1 -1
@@ -8,7 +8,7 @@ /* Reserve stack in .data */ .data .align PAGE_SIZE - . = . + PAGE_SIZE * MAX_TEST_CPUS + . = . + PER_CPU_SIZE * MAX_TEST_CPUS .globl stacktop stacktop:
Signature · 1 line
2.47.3
PATCH v2 2/8 x86: fix EFI memory allocator to clamp regions to 4 GiB
In reply to Giacomo Mazzolasetup_page_table() creates an identity map covering only the first 4 GiB (4 PDPT entries × 512 PDE entries × 2 MiB pages). However, setup_memory_allocator() picks the largest EFI_CONVENTIONAL_MEMORY region without regard to this limit.
On machines with more than 4 GiB of RAM, the largest conventional memory region often starts below 4 GiB but extends well above it, or resides entirely above 4 GiB. When the allocator selects such a region, any access to memory beyond the mapped range triggers a page fault, crashing the test before it can run.
Fix this by clamping every candidate region to the 4 GiB boundary:
skip regions that start at or above 4 GiB entirely, and truncate regions that straddle the boundary so only the mapped portion is considered. The allocator then picks the largest usable region that is fully covered by the identity map.
--- lib/x86/setup.c | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-)
lib/x86/setup.c +17 -3
@@ -215,6 +215,7 @@ static efi_status_t setup_memory_allocator(efi_bootinfo_t *efi_bootinfo)struct efi_boot_memmap *map = &(efi_bootinfo->mem_map);efi_memory_desc_t *buffer = *map->map;efi_memory_desc_t *d =NULL ; +unsigned long max_addr =1UL <<32 ;/* * The 'buffer' contains multiple descriptors that describe memory
@@ -222,13 +223,26 @@ static efi_status_t setup_memory_allocator(efi_bootinfo_t *efi_bootinfo)* EFI_CONVENTIONAL_MEMORY region which will be used to set up the * memory allocator, so that the memory allocator can work in the * largest free continuous memory region. +* +* Regions are clamped to the first 4 GiB because the EFI page +* tables set up by setup_page_table() only map that range. */ for (i =0 ; i < *(map->map_size); i += *(map->desc_size)) { d = (efi_memory_desc_t *)(&((u8 *)buffer)[i]);if (d->type ==EFI_CONVENTIONAL_MEMORY ) { -if (free_mem_pages < d->num_pages) { - free_mem_pages = d->num_pages; - free_mem_start = d->phys_addr; +unsigned long start = d->phys_addr; +unsigned long end = start + (d->num_pages <<EFI_PAGE_SHIFT ); +unsigned long pages; + +if (start >= max_addr) +continue ; +if (end > max_addr) + end = max_addr; + pages = (end - start) >>EFI_PAGE_SHIFT ; + +if (free_mem_pages < pages) { + free_mem_pages = pages; + free_mem_start = start; } } }
Signature · 1 line
2.47.3
PATCH v2 3/8 x86: skip PMU init when no PMU is advertised
In reply to Giacomo MazzolaGuard pmu_init() behind CPUID checks: Intel's architectural PMU version (CPUID 0xA EAX[7:0]) and AMD's PERFCTR_CORE feature bit. Without a PMU, accessing performance counter MSRs triggers #GP.
--- lib/x86/setup.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
lib/x86/setup.c +3 -1
@@ -416,5 +416,7 @@ void bsp_rest_init(void)bringup_aps ();enable_x2apic ();smp_init (); -pmu_init (); +if (this_cpu_property (X86_PROPERTY_PMU_VERSION ) || +this_cpu_has (X86_FEATURE_PERFCTR_CORE )) +pmu_init (); }
Signature · 1 line
2.47.3
PATCH v2 4/8 x86: fix ISR thunk to use absolute indirect jump
In reply to Giacomo MazzolaReplace the relative jmp (e9 rel32) with an absolute indirect jmp *0(%rip) (ff 25 00000000 + addr64). The rel32 displacement is sign-extended to 64 bits, so when the displacement between the thunk and isr_entry_point exceeds 2 GiB the sign extension corrupts the target address.
--- lib/x86/isr.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
lib/x86/isr.c +5 -3
@@ -79,9 +79,11 @@ void handle_irq(unsigned vec, void (*func)(isr_regs_t *regs))/* mov $func_high, %(rsp+4) */ *thunk++ =0xc7 ; *thunk++ =0x44 ; *thunk++ =0x24 ; *thunk++ =0x04 ; *(u32 *)thunk = (ulong )func >>32 ; thunk +=4 ; -/* jmp isr_entry_point */ - *thunk ++ =0xe9 ; - *(u32 *)thunk = (ulong )isr_entry_point - (ulong)(thunk +4 ); +/* jmp *0(%rip): absolute indirect jump to isr_entry_point */ + *thunk++ =0xff ; *thunk++ =0x25 ; +/* disp32 = 0: address follows */ + *(u32 *)thunk =0 ; thunk +=4 ; + *(u64 *)thunk = (ulong )isr_entry_point;#else /* push $func */ *thunk++ =0x68 ;
Signature · 1 line
2.47.3
PATCH v2 5/8 x86: replace per-AP bringup prints with a single summary line
In reply to Giacomo MazzolaRemove the per-AP "CPU %d online" printf from ap_online() and the per-CPU "enabling apic" printf from enable_apic(). Replace them with a single "smp: %d CPUs online" message printed once after all APs have come up.
On systems with many CPUs, the serial lock contention from per-AP prints delays bringup long enough to trigger platform watchdogs. A single summary line provides the same information without O(ncpus) serial transactions during bringup.
--- lib/x86/apic.c | 1 - lib/x86/smp.c | 4 ++-- 2 files changed, 2 insertions(+), 3 deletions(-)
lib/x86/apic.c +0 -1
@@ -237,7 +237,6 @@ void set_irq_line(unsigned line, int val)void enable_apic (void ) { -printf ("enabling apic\n" );xapic_write (APIC_SPIV ,0x1ff ); }
lib/x86/smp.c +2 -2
@@ -97,7 +97,6 @@ void ap_online(void) {sti (); -printf ("setup: CPU %" PRId32" online\n" ,apic_id ());atomic_inc (&cpu_online_count);/* Only the BSP runs the test's main(), APs are given work via IPIs. */
@@ -294,7 +293,8 @@ void bringup_aps(void) _cpu_count =fwcfg_get_nb_cpus (); -printf ("smp: waiting for %d APs\n" , _cpu_count -1 );while (_cpu_count !=atomic_read (&cpu_online_count))cpu_relax (); + +printf ("smp: %d CPUs online\n" , _cpu_count); }
Signature · 1 line
2.47.3
PATCH v2 6/8 x86: add timeout-based SMP bringup when fw_cfg is unavailable
In reply to Giacomo MazzolaWhen fw_cfg is unavailable, fwcfg_get_nb_cpus()
returns 0 and the existing bringup loop spins forever waiting for cpu_online_count to match. Add a timeout-based fallback that watches cpu_online_count and declares bringup complete once no new AP has checked in for AP_QUIESCE_TICKS (~4 billion TSC ticks).
Assuming a TSC frequency of at least 1 GHz, this gives between ~1 s (at 4 GHz) and ~4 s (at 1 GHz) of quiescence after the last AP before declaring bringup complete.
--- lib/x86/smp.c | 27 +++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-)
lib/x86/smp.c +25 -2
@@ -14,6 +14,13 @@#define IPI_VECTOR 0x20 +/* +* Quiescence timeout for AP bringup when the CPU count is not known +* (bare-metal EFI without fw_cfg). Assumes a TSC frequency of at +* least 1 GHz; at 4 GHz this gives ~1 s, at 1 GHz ~4 s. +*/ +#define AP_QUIESCE_TICKS (4ull * 1000 * 1000 * 1000) +typedef void (*ipi_function_type )(void *data);static struct spinlock ipi_lock;
@@ -293,8 +300,24 @@ void bringup_aps(void) _cpu_count =fwcfg_get_nb_cpus (); -while (_cpu_count !=atomic_read (&cpu_online_count)) -cpu_relax (); +if (_cpu_count) { +while (_cpu_count !=atomic_read (&cpu_online_count)) +cpu_relax (); + }else { +int prev =atomic_read (&cpu_online_count); +u64 last_change =rdtsc (); + +while (rdtsc () - last_change <AP_QUIESCE_TICKS ) { +int cur =atomic_read (&cpu_online_count); + +if (cur != prev) { + prev = cur; + last_change =rdtsc (); + } +cpu_relax (); + } + _cpu_count = prev; + }printf ("smp: %d CPUs online\n" , _cpu_count); }
Signature · 1 line
2.47.3
PATCH v2 7/8 efi: fix load_options_size conversion to character count
In reply to Giacomo Mazzolaload_options_size is in bytes, but options_chars is used as the iteration count over the u16 load options array. Divide by sizeof(u16) to convert from bytes to UTF-16 code units; without this, the loop reads past the end of the load options buffer.
The original Linux kernel code (drivers/firmware/efi/libstub/ efi-stub-helper.c) from which this function was adapted performs the same conversion:
options_chars = options_size / sizeof(efi_char16_t);
This was lost when the function was ported to KUT.
--- lib/efi.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/efi.c +1 -1
@@ -134,7 +134,7 @@ static char *efi_convert_cmdline(struct efi_loaded_image_64 *image, int *cmd_lin {const u16 *s2;unsigned long cmdline_addr =0 ; -int options_chars = image->load_options_size; +int options_chars = image->load_options_size /sizeof (u16);const u16 *options = image->load_options;int options_bytes =0 , safe_options_bytes =0 ;/* UTF-8 bytes */ bool in_quote = false;
Signature · 1 line
2.47.3
PATCH v2 8/8 lib: parse KUT_ENV= from command line into environ
In reply to Giacomo MazzolaAdd support for passing environment variables via the command line. If any argv element starts with 'KUT_ENV=', the value after the '=' is parsed as a comma-separated list of KEY=VAL pairs and fed to setup_env(). The KUT_ENV= argument is then removed from argv so tests do not see it as a spurious positional argument.
Example: KUT_ENV=PARAM_1=VALUE_1,PARAM_2=VALUE_2
--- lib/argv.c | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+)
lib/argv.c +35 -0
@@ -27,6 +27,36 @@ static const char *skip_blanks(const char *p)return p; } +static void setup_args_env (void ) +{ +static const char prefix[] ="KUT_ENV=" ; +char *env, *p; +int i, j; + +for (i =0 ; i < __argc; i++) { +if (strncmp (__argv[i], prefix,sizeof (prefix) -1 ) ==0 ) +break ; + } + +if (i == __argc) +return ; + + env = __argv[i] +sizeof (prefix) -1 ; + +/* Shift remaining args down to preserve order. */ +for (j = i; j < __argc -1 ; j++) + __argv[j] = __argv[j +1 ]; + __argv[--__argc] =NULL ; + +/* Convert comma separators to newlines for setup_env(). */ +for (p = env; *p; p++) { +if (*p ==',' ) + *p ='\n' ; + } + +setup_env (env, p - env); +} +void __setup_args (void ) {const char *args = __args;
@@ -39,6 +69,8 @@ void __setup_args(void) *copy_ptr++ ='\0' ; } __argc = argv - __argv; + +setup_args_env (); }void setup_args (const char *args)
@@ -130,6 +162,9 @@ void setup_env(char *env, int size)bool newline = false;int i =0 ; +while (__environ[i]) + i++; +while (*p) ++p;if (p == eof)
Signature · 1 line
2.47.3