x86: Add support for running as secondary Jailhouse guest
Subsystems (3)
Jan Kiszka
maintainercover x86: Add support for running as secondary Jailhouse guest
This series paves the way to run Linux in so-called non-root cells (guest partitions) of the Jailhouse hypervisor.
Jailhouse [1] was started 4 years ago as an open-source (GPL) leight- weight hypervisor that statically partitions SMP systems. It's unique in that it uses one Linux instance, the root cell, as boot loader and management console. Jailhouse targets use cases for hard real-time and safety-critical systems that KVM cannot cater due to its inherent complexity.
Jaihouse can run bare-metal, free and closed-source RTOSes as secondary guests and, with this series, also x86 Linux instances. While ARM and ARM64 non-root Linux guests are feasible without extra patches, thanks to the high configurability via device trees, x86 requires special platform support, mostly to step away from non-existing resources in a non-root Jailhouse cell.
This series ensures that Linux can boot in a non-root cell, including SMP cells, has working timekeeping and can use the platform UARTs and PCI devices as assigned to it. In follow-up series, we will propose optimizations and enhancements for the PCI support, a simplistic debug console, and some improvement for Linux guests on ARM.
What is not yet in upstream-ready state is a driver for inter-cell communication. The current implementation of virtual peer-to-peer network [2] uses an enhanced version of the QEMU ivshmem shared memory device. However we still need to finish the evaluation of virtio / vhost-pci options prior to settling over the final interface.
This patch series is also available at
git://git.kiszka.org/linux.git d0036688b2da
Jan
[1] http://jailhouse-project.org
[2] http://git.kiszka.org/?p=linux.git;a=shortlog;h=refs/heads/queues/jailhouse
Jan Kiszka (10):
x86/apic: Install an empty physflat_init_apic_ldr
x86: jailhouse: Add infrastructure for running in non-root cell
x86: jailhouse: Enable APIC and SMP support
x86: jailhouse: Enable PMTIMER
x86: jailhouse: Set up timekeeping
x86: jailhouse: Avoid access of unsupported platform resources
x86: jailhouse: Silence ACPI warning
x86: jailhouse: Halt instead of failing to restart
x86: jailhouse: Wire up IOAPIC for legacy UART ports
x86: jailhouse: Initialize PCI support
arch/x86/Kconfig | 13 ++ arch/x86/include/asm/hypervisor.h | 1 + arch/x86/include/asm/jailhouse_para.h | 27 ++++ arch/x86/include/asm/tsc.h | 3 + arch/x86/kernel/Makefile | 2 + arch/x86/kernel/apic/apic_flat_64.c | 12 +- arch/x86/kernel/cpu/hypervisor.c | 4 + arch/x86/kernel/jailhouse.c | 252 ++++++++++++++++++++++++++++++++++ arch/x86/kernel/smpboot.c | 7 +- arch/x86/kernel/tsc.c | 14 +- drivers/acpi/Kconfig | 32 ++--- 11 files changed, 339 insertions(+), 28 deletions(-) create mode 100644 arch/x86/include/asm/jailhouse_para.h create mode 100644 arch/x86/kernel/jailhouse.c
Signature · 1 line
2.12.3
Jan Kiszka
maintainerPATCH 1/10 x86/apic: Install an empty physflat_init_apic_ldr
From: Jan Kiszka <[email protected]>
As the comment already stated, there is no need for setting up LDR in physflat mode as it remains unused. flat_init_apic_ldr only served as a placeholder for a nop operation so far, causing no harm.
That will change when running over the Jailhouse hypervisor. Here we must not touch LDR in a way that destroys the mapping originally set up by the Linux root cell. Jailhouse enforces this setting in order to efficiently validate any IPI requests sent by a cell.
Avoid a needless clash caused by flat_init_apic_ldr by installing a true nop handler.
--- arch/x86/kernel/apic/apic_flat_64.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
arch/x86/kernel/apic/apic_flat_64.c +6 -2
@@ -218,6 +218,11 @@ static int physflat_acpi_madt_oem_check(char *oem_id, char *oem_table_id)return 0 ; } +void physflat_init_apic_ldr (void ) +{ +/* LDR is not used in physflat mode. */ +} +static void physflat_send_IPI_allbutself (int vector) {default_send_IPI_mask_allbutself_phys (cpu_online_mask, vector);
@@ -251,8 +256,7 @@ static struct apic apic_physflat __ro_after_init = { .dest_logical = 0, .check_apicid_used = NULL, -/* not needed, but shouldn't hurt: */ - .init_apic_ldr = flat_init_apic_ldr, + .init_apic_ldr = physflat_init_apic_ldr, .ioapic_phys_id_map = NULL, .setup_apic_routing = NULL,
Signature · 1 line
2.12.3
Re: [PATCH 01/10] x86/apic: Install an empty physflat_init_apic_ldr
Quoted 3 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
As the comment already stated, there is no need for setting up LDR in
I prefer to have a proper reference to documentation why this is not needed than to a comment which is technically dubious.
Quoted 21 lines
physflat mode as it remains unused.
flat_init_apic_ldronly served as a placeholder for a nop operation so far, causing no harm.That will change when running over the Jailhouse hypervisor. Here we must not touch LDR in a way that destroys the mapping originally set up by the Linux root cell. Jailhouse enforces this setting in order to efficiently validate any IPI requests sent by a cell.
Avoid a needless clash caused by
flat_init_apic_ldrby installing a true nop handler.Signed-off-by Jan Kiszka <[email protected]>--- arch/x86/kernel/apic/apic_flat_64.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)arch/x86/kernel/apic/apic_flat_64.c +0
-0
arch/x86/kernel/apic/apic_flat_64.c static int physflat_acpi_madt_oem_check(char *oem_id, char *oem_table_id) · line 221
Quoted 4 lines
arch/x86/kernel/apic/apic_flat_64.c +1
-0@@ -218,6 +218,11 @@ static int physflat_acpi_madt_oem_check(char *oem_id, char *oem_table_id)return 0 ; } +void physflat_init_apic_ldr (void )
static ?
arch/x86/kernel/apic/apic_flat_64.c static int physflat_acpi_madt_oem_check(char *oem_id, char *oem_table_id) · line 223
Quoted 2 lines
arch/x86/kernel/apic/apic_flat_64.c +2
-0+{ +/* LDR is not used in physflat mode. */
flat_init_apic_ldr() also fiddles with DFR ...
Quoted 19 lines
arch/x86/kernel/apic/apic_flat_64.c +3
-2+} +static void physflat_send_IPI_allbutself (int vector) {default_send_IPI_mask_allbutself_phys (cpu_online_mask, vector);@@ -251,8 +256,7 @@ static struct apic apic_physflat __ro_after_init = { .dest_logical = 0, .check_apicid_used = NULL, -/* not needed, but shouldn't hurt: */ - .init_apic_ldr = flat_init_apic_ldr, + .init_apic_ldr = physflat_init_apic_ldr, .ioapic_phys_id_map = NULL, .setup_apic_routing = NULL,Signature · 1 line
2.12.3
Jan Kiszka
maintainerPATCH 2/10 x86: jailhouse: Add infrastructure for running in non-root cell
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
The Jailhouse hypervisor is able to statically partition a multicore system into multiple so-called cells. Linux is used as boot loader and continues to run in the root cell after Jailhouse is enabled. Linux can also run in non-root cells.
Jailhouse does not emulate usual x86 devices. It also provides no complex ACPI but basic platform information that the boot loader forwards via setup data. This adds the infrastructure to detect when running in a non-root cell so that the platform can be configured as required in succeeding steps.
Support is limited to x86-64 so far, primarily because no boot loader stub exists for i386 and, thus, we wouldn't be able to test the 32-bit path.
--- arch/x86/Kconfig | 12 ++++++ arch/x86/include/asm/hypervisor.h | 1 + arch/x86/include/asm/jailhouse_para.h | 27 +++++++++++++ arch/x86/kernel/Makefile | 2 + arch/x86/kernel/cpu/hypervisor.c | 4 ++ arch/x86/kernel/jailhouse.c | 76 +++++++++++++++++++++++++++++++++++ 6 files changed, 122 insertions(+) create mode 100644 arch/x86/include/asm/jailhouse_para.h create mode 100644 arch/x86/kernel/jailhouse.c
arch/x86/Kconfig +12 -0
@@ -782,6 +782,18 @@ config KVM_DEBUG_FS Statistics are displayed in debugfs filesystem. Enabling this option may incur significant overhead. +config JAILHOUSE_GUEST + bool "Jailhouse non-root cell support" + depends on PARAVIRT && X86_64 + ---help--- + This option allows to run Linux as guest in a Jailhouse non-root + cell. You can leave this option disabled if you only want to start + Jailhouse and run Linux afterwards in the root cell. + + You likely also want to disable CONFIG_SUSPEND and CONFIG_SERIO to + avoid access to I/O resources that are usually not assigned to the + non-root cell. + config PARAVIRT_TIME_ACCOUNTING bool "Paravirtual steal time accounting" depends on PARAVIRT
arch/x86/include/asm/hypervisor.h +1 -0
@@ -37,6 +37,7 @@ enum x86_hypervisor_type {X86_HYPER_XEN_PV ,X86_HYPER_XEN_HVM ,X86_HYPER_KVM , + X86_HYPER_JAILHOUSE, }; struct hypervisor_x86 {
arch/x86/include/asm/jailhouse_para.h new file +27 -0
@@ -0,0 +1,27 @@ +/* +* Jailhouse paravirt_ops implementation +* +* Copyright (c) Siemens AG, 2015-2017 +* +* Authors: +* Jan Kiszka <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2. See +* the COPYING file in the top-level directory. +*/ + +#ifndef _ASM_X86_JAILHOUSE_PARA_H +#define _ASM_X86_JAILHOUSE_PARA_H + +#include <linux/types.h> + +#ifdef CONFIG_JAILHOUSE_GUEST +bool jailhouse_paravirt (void ); +#else +static inline bool jailhouse_paravirt (void ) +{ +return false; +} +#endif + +#endif /* _ASM_X86_JAILHOUSE_PARA_H */
arch/x86/kernel/Makefile +2 -0
@@ -112,6 +112,8 @@ obj-$(CONFIG_PARAVIRT_SPINLOCKS)+= paravirt-spinlocks.o obj-$(CONFIG_PARAVIRT_CLOCK) += pvclock.o obj-$(CONFIG_X86_PMEM_LEGACY_DEVICE) += pmem.o +obj-$(CONFIG_JAILHOUSE_GUEST) += jailhouse.o + obj-$(CONFIG_EISA) += eisa.o obj-$(CONFIG_PCSPKR_PLATFORM) += pcspeaker.o
arch/x86/kernel/cpu/hypervisor.c +4 -0
@@ -31,6 +31,7 @@ extern const struct hypervisor_x86 x86_hyper_ms_hyperv;extern const struct hypervisor_x86 x86_hyper_xen_pv;extern const struct hypervisor_x86 x86_hyper_xen_hvm;extern const struct hypervisor_x86 x86_hyper_kvm; +extern const struct hypervisor_x86 x86_hyper_jailhouse;static const __initconst struct hypervisor_x86 *const hypervisors[] = {
@@ -45,6 +46,9 @@ static const __initconst struct hypervisor_x86 * const hypervisors[] =#ifdef CONFIG_KVM_GUEST &x86_hyper_kvm,#endif +#ifdef CONFIG_JAILHOUSE_GUEST + &x86_hyper_jailhouse, +#endif };enum x86_hypervisor_type x86_hyper_type;
arch/x86/kernel/jailhouse.c new file +76 -0
76 lines not on this page · raw message
Signature · 1 line
2.12.3
Re: [PATCH 02/10] x86: jailhouse: Add infrastructure for running in non-root cell
arch/x86/Kconfig config KVM_DEBUG_FS · line 795
Quoted 11 lines · Jan Kiszka
arch/x86/Kconfig +11
-0+config JAILHOUSE_GUEST + bool "Jailhouse non-root cell support" + depends on PARAVIRT && X86_64 + ---help--- + This option allows to run Linux as guest in a Jailhouse non-root + cell. You can leave this option disabled if you only want to start + Jailhouse and run Linux afterwards in the root cell. + + You likely also want to disable CONFIG_SUSPEND and CONFIG_SERIO to + avoid access to I/O resources that are usually not assigned to the + non-root cell.
That should be prevented programatically.
Quoted 12 lines
arch/x86/kernel/jailhouse.c new file +12
-0+#include <linux/kernel.h> +#include <asm/cpu.h> +#include <asm/hypervisor.h> +#include <asm/setup.h> + +#define SETUP_JAILHOUSE 0x53484c4a/* "JLHS" */ + +#define SETUP_REQUIRED_VERSION 1 + +/* +* The boot loader is passing platform information via this Jailhouse-specific +* setup data structure.
arch/x86/kernel/jailhouse.c line 34
Quoted 10 lines
arch/x86/kernel/jailhouse.c new file +10
-0+*/ +struct jailhouse_setup_data { +struct setup_data header; +u16 version; +u16 compatible_version; +u16 pm_timer_address; +u16 num_cpus; +u64 pci_mmconfig_base; +u8 standard_ioapic; +u8 cpu_ids[255 ];
Shouldn't this structure and SETUP_JAILHOUSE be defined in a header file which can be exported to boot loaders?
Aside of that please write the struct in tabular fashion:
struct jailhouse_setup_data {
struct setup_data header;
u16 version;
....arch/x86/kernel/jailhouse.c line 74
Quoted 12 lines
arch/x86/kernel/jailhouse.c new file +12
-0+}; + +bool jailhouse_paravirt (void ) +{ +return jailhouse_cpuid_base () !=0 ; +} + +const struct hypervisor_x86 x86_hyper_jailhouse__refconst = { + .name ="Jailhouse" , + .detect = jailhouse_detect, + .init = { + .init_platform = jailhouse_init_platform,
See above.
Thanks,
tglx
Re: [PATCH 02/10] x86: jailhouse: Add infrastructure for running in non-root cell
Quoted 15 lines · Thomas Gleixner
arch/x86/Kconfig config KVM_DEBUG_FS · line 795
Quoted 11 lines · Jan Kiszka
arch/x86/Kconfig +11
-0+config JAILHOUSE_GUEST + bool "Jailhouse non-root cell support" + depends on PARAVIRT && X86_64 + ---help--- + This option allows to run Linux as guest in a Jailhouse non-root + cell. You can leave this option disabled if you only want to start + Jailhouse and run Linux afterwards in the root cell. + + You likely also want to disable CONFIG_SUSPEND and CONFIG_SERIO to + avoid access to I/O resources that are usually not assigned to the + non-root cell.That should be prevented programatically.
Theoretically, serio access could also be assigned to a non-root cell. But excluding SUSPEND may make sense unconditionally, will check again.
Quoted 18 lines
Quoted 18 lines
arch/x86/kernel/jailhouse.c new file +17
-0+#include <linux/kernel.h> +#include <asm/cpu.h> +#include <asm/hypervisor.h> +#include <asm/setup.h> + +#define SETUP_JAILHOUSE 0x53484c4a/* "JLHS" */ + +#define SETUP_REQUIRED_VERSION 1 + +/* +* The boot loader is passing platform information via this Jailhouse-specific +* setup data structure. +*/ +struct jailhouse_setup_data { +struct setup_data header; +u16 version; +u16 compatible_version;
Quoted 8 lines
arch/x86/kernel/jailhouse.c line 34
Quoted 5 lines
arch/x86/kernel/jailhouse.c new file +5
-0+u16 pm_timer_address; +u16 num_cpus; +u64 pci_mmconfig_base; +u8 standard_ioapic; +u8 cpu_ids[255 ];Shouldn't this structure and
SETUP_JAILHOUSEbe defined in a header file which can be exported to boot loaders?
Something like arch/x86/include/uapi/asm/jailhouse_setup.h?
Jan
Re: [PATCH 02/10] x86: jailhouse: Add infrastructure for running in non-root cell
Quoted 5 lines · Jan Kiszka
Quoted 2 lines · Thomas Gleixner
Shouldn't this structure and
SETUP_JAILHOUSEbe defined in a header file which can be exported to boot loaders?Something like
arch/x86/include/uapi/asm/jailhouse_setup.h?
Something like that.
Thanks,
tglx
Jan Kiszka
maintainerPATCH 3/10 x86: jailhouse: Enable APIC and SMP support
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
Register the APIC which Jailhouse always exposes at 0xfee00000 if in xAPIC mode or via MSRs as x2APIC. The latter is only available if it was already activated because there is no virtualization to switch its mode during runtime.
Jailhouse requires the APIC to be operated in phys-flat mode. Ensure that this mode is selected by Linux.
The available CPUs are taken from the setup data structure that the loader filled and registered with the kernel.
--- arch/x86/kernel/apic/apic_flat_64.c | 4 +++- arch/x86/kernel/jailhouse.c | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+), 1 deletion(-)
arch/x86/kernel/apic/apic_flat_64.c +3 -1
@@ -19,6 +19,7 @@#include <asm/smp.h> #include <asm/apic.h> #include <asm/ipi.h> +#include <asm/jailhouse_para.h> #include <linux/acpi.h>
@@ -235,7 +236,8 @@ static void physflat_send_IPI_all(int vector)static int physflat_probe (void ) { -if (apic == &apic_physflat ||num_possible_cpus () >8 ) +if (apic == &apic_physflat ||num_possible_cpus () >8 || +jailhouse_paravirt ())return 1 ;return 0 ;
arch/x86/kernel/jailhouse.c +34 -0
@@ -11,6 +11,7 @@*/ #include <linux/kernel.h> +#include <asm/apic.h> #include <asm/cpu.h> #include <asm/hypervisor.h> #include <asm/setup.h>
@@ -48,10 +49,16 @@ static uint32_t __init jailhouse_detect(void)return jailhouse_cpuid_base (); } +static unsigned int x2apic_get_apic_id (unsigned long id) +{ +return id; +} +static void __init jailhouse_init_platform (void ) {u64 pa_data = boot_params.hdr.setup_data;struct jailhouse_setup_data *data; +unsigned int cpu; data =early_memremap (pa_data,sizeof (*data));
@@ -59,6 +66,23 @@ static void __init jailhouse_init_platform(void) data->compatible_version > SETUP_REQUIRED_VERSION)panic ("Jailhouse: Unsupported setup data structure" ); +#ifdef CONFIG_X86_X2APIC +/* +* Register x2APIC handlers early. We need them when running +* register_lapic_address. +*/ +if (x2apic_enabled ()) { + apic->read = native_apic_msr_read; + apic->write = native_apic_msr_write; + apic->get_apic_id = x2apic_get_apic_id; + } +#endif +register_lapic_address (0xfee00000 ); +for (cpu =0 ; cpu < data->num_cpus; cpu++) +generic_processor_info (data->cpu_ids[cpu], + boot_cpu_apic_version); + smp_found_config =1 ; +early_memunmap (data,sizeof (*data)); }
@@ -67,10 +91,20 @@ bool jailhouse_paravirt(void)return jailhouse_cpuid_base () !=0 ; } +static bool jailhouse_x2apic_available (void ) +{ +/* +* The x2APIC is only available if the root cell enabled it. Jailhouse +* does not support switching between xAPIC and x2APIC. +*/ +return x2apic_enabled (); +} +const struct hypervisor_x86 x86_hyper_jailhouse__refconst = { .name ="Jailhouse" , .detect = jailhouse_detect, .init = { .init_platform = jailhouse_init_platform, + .x2apic_available = jailhouse_x2apic_available, }, };
Signature · 1 line
2.12.3
Re: [PATCH 03/10] x86: jailhouse: Enable APIC and SMP support
On Thu, 16 Nov 2017, Jan Kiszka wrote:
Just noticed, that all prefixes should be 'x86/jailhouse:' please
Quoted 16 lines
arch/x86/kernel/jailhouse.c +6
-0+static unsigned int x2apic_get_apic_id (unsigned long id) +{ +return id; +} +static void __init jailhouse_init_platform (void ) {u64 pa_data = boot_params.hdr.setup_data;struct jailhouse_setup_data *data; +unsigned int cpu; data =early_memremap (pa_data,sizeof (*data));@@ -59,6 +66,23 @@ static void __init jailhouse_init_platform(void) data->compatible_version > SETUP_REQUIRED_VERSION)panic ("Jailhouse: Unsupported setup data structure" );
arch/x86/kernel/jailhouse.c static void __init jailhouse_init_platform(void) · line 77
Quoted 10 lines
arch/x86/kernel/jailhouse.c +9
-0+#ifdef CONFIG_X86_X2APIC +/* +* Register x2APIC handlers early. We need them when running +* register_lapic_address. +*/ +if (x2apic_enabled ()) { + apic->read = native_apic_msr_read; + apic->write = native_apic_msr_write; + apic->get_apic_id = x2apic_get_apic_id;
Uurgh. That looks hacky.
arch/x86/kernel/jailhouse.c static void __init jailhouse_init_platform(void) · line 80
Quoted 3 lines
arch/x86/kernel/jailhouse.c +3
-0+ } +#endif +register_lapic_address (0xfee00000 );
No need to do all of this here. The right thing to do is to override
x86_init.mpparse.get_smp_config
which is invoked after generic_apic_probe() which should select x2apic_phys, but I might be wrong.
arch/x86/kernel/jailhouse.c static void __init jailhouse_init_platform(void) · line 83
Quoted 3 lines
arch/x86/kernel/jailhouse.c +3
-0+for (cpu =0 ; cpu < data->num_cpus; cpu++) +generic_processor_info (data->cpu_ids[cpu], + boot_cpu_apic_version);
Please add brackets around multi line statements.
Thanks,
tglx
Re: [PATCH 03/10] x86: jailhouse: Enable APIC and SMP support
Quoted 21 lines · Thomas Gleixner
On Thu, 16 Nov 2017, Jan Kiszka wrote:
Just noticed, that all prefixes should be 'x86/jailhouse:' please
Quoted 17 lines
arch/x86/kernel/jailhouse.c +6
-0+static unsigned int x2apic_get_apic_id (unsigned long id) +{ +return id; +} +static void __init jailhouse_init_platform (void ) {u64 pa_data = boot_params.hdr.setup_data;struct jailhouse_setup_data *data; +unsigned int cpu; data =early_memremap (pa_data,sizeof (*data));@@ -59,6 +66,23 @@ static void __init jailhouse_init_platform(void) data->compatible_version > SETUP_REQUIRED_VERSION)panic ("Jailhouse: Unsupported setup data structure" );
Quoted 11 lines
arch/x86/kernel/jailhouse.c static void __init jailhouse_init_platform(void) · line 77
Quoted 9 lines
arch/x86/kernel/jailhouse.c +9
-0+#ifdef CONFIG_X86_X2APIC +/* +* Register x2APIC handlers early. We need them when running +* register_lapic_address. +*/ +if (x2apic_enabled ()) { + apic->read = native_apic_msr_read; + apic->write = native_apic_msr_write; + apic->get_apic_id = x2apic_get_apic_id;Uurgh. That looks hacky.
Could just create a new APIC driver (which would call most of these)?
Quoted 22 lines
arch/x86/kernel/jailhouse.c static void __init jailhouse_init_platform(void) · line 80
Quoted 4 lines
arch/x86/kernel/jailhouse.c +3
-0+ } +#endif +register_lapic_address (0xfee00000 );No need to do all of this here. The right thing to do is to override
x86_init.mpparse.get_smp_config
which is invoked after
generic_apic_probe()which should selectx2apic_phys, but I might be wrong.arch/x86/kernel/jailhouse.c static void __init jailhouse_init_platform(void) · line 83
Quoted 3 lines
arch/x86/kernel/jailhouse.c +3
-0+for (cpu =0 ; cpu < data->num_cpus; cpu++) +generic_processor_info (data->cpu_ids[cpu], + boot_cpu_apic_version);Please add brackets around multi line statements.
Thanks,
tglx
Re: [PATCH 03/10] x86: jailhouse: Enable APIC and SMP support
Quoted 34 lines · Konrad Rzeszutek Wilk
Quoted 21 lines · Thomas Gleixner
On Thu, 16 Nov 2017, Jan Kiszka wrote:
Just noticed, that all prefixes should be 'x86/jailhouse:' please
Quoted 17 lines
arch/x86/kernel/jailhouse.c +6
-0+static unsigned int x2apic_get_apic_id (unsigned long id) +{ +return id; +} +static void __init jailhouse_init_platform (void ) {u64 pa_data = boot_params.hdr.setup_data;struct jailhouse_setup_data *data; +unsigned int cpu; data =early_memremap (pa_data,sizeof (*data));@@ -59,6 +66,23 @@ static void __init jailhouse_init_platform(void) data->compatible_version > SETUP_REQUIRED_VERSION)panic ("Jailhouse: Unsupported setup data structure" );Quoted 11 lines
arch/x86/kernel/jailhouse.c static void __init jailhouse_init_platform(void) · line 77
Quoted 9 lines
arch/x86/kernel/jailhouse.c +9
-0+#ifdef CONFIG_X86_X2APIC +/* +* Register x2APIC handlers early. We need them when running +* register_lapic_address. +*/ +if (x2apic_enabled ()) { + apic->read = native_apic_msr_read; + apic->write = native_apic_msr_write; + apic->get_apic_id = x2apic_get_apic_id;Uurgh. That looks hacky.
Quoted 1 line
Could just create a new APIC driver (which would call most of these)?
Why so? See further down.
Quoted 23 lines
Quoted 22 lines
arch/x86/kernel/jailhouse.c static void __init jailhouse_init_platform(void) · line 80
Quoted 4 lines
arch/x86/kernel/jailhouse.c +3
-0+ } +#endif +register_lapic_address (0xfee00000 );No need to do all of this here. The right thing to do is to override
x86_init.mpparse.get_smp_config
which is invoked after
generic_apic_probe()which should selectx2apic_phys, but I might be wrong.arch/x86/kernel/jailhouse.c static void __init jailhouse_init_platform(void) · line 83
Quoted 3 lines
arch/x86/kernel/jailhouse.c +3
-0+for (cpu =0 ; cpu < data->num_cpus; cpu++) +generic_processor_info (data->cpu_ids[cpu], + boot_cpu_apic_version);Please add brackets around multi line statements.
Thanks,
tglx
Jan Kiszka
maintainerPATCH 4/10 x86: jailhouse: Enable PMTIMER
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
Jailhouse exposes the PMTIMER as only reference clock to all cells. Pick up its address from the setup data. Allow to enable the Linux support of it by relaxing its strict dependency on ACPI.
--- arch/x86/Kconfig | 1 + arch/x86/kernel/jailhouse.c | 4 ++++ drivers/acpi/Kconfig | 32 ++++++++++++++++---------------- 3 files changed, 21 insertions(+), 16 deletions(-)
arch/x86/Kconfig +1 -0
@@ -785,6 +785,7 @@ config KVM_DEBUG_FS config JAILHOUSE_GUEST bool "Jailhouse non-root cell support" depends on PARAVIRT && X86_64 + select X86_PM_TIMER ---help--- This option allows to run Linux as guest in a Jailhouse non-root cell. You can leave this option disabled if you only want to start
arch/x86/kernel/jailhouse.c +4 -0
@@ -10,6 +10,7 @@* the COPYING file in the top-level directory. */ +#include <linux/acpi_pmtmr.h> #include <linux/kernel.h> #include <asm/apic.h> #include <asm/cpu.h>
@@ -66,6 +67,9 @@ static void __init jailhouse_init_platform(void) data->compatible_version > SETUP_REQUIRED_VERSION)panic ("Jailhouse: Unsupported setup data structure" ); + pmtmr_ioport = data->pm_timer_address; +pr_debug ("Jailhouse: PM-Timer IO Port: %#x\n" , pmtmr_ioport); +#ifdef CONFIG_X86_X2APIC /* * Register x2APIC handlers early. We need them when running
drivers/acpi/Kconfig +16 -16
@@ -361,22 +361,6 @@ config ACPI_PCI_SLOT i.e., segment/bus/device/function tuples, with physical slots in the system. If you are unsure, say N. -config X86_PM_TIMER - bool "Power Management Timer Support" if EXPERT - depends on X86 - default y - help - The Power Management Timer is available on all ACPI-capable, - in most cases even if ACPI is unusable or blacklisted. - - This timing source is not affected by power management features - like aggressive processor idling, throttling, frequency and/or - voltage scaling, unlike the commonly used Time Stamp Counter - (TSC) timing source. - - You should nearly always say Y here because many modern - systems require this timer. - config ACPI_CONTAINER bool "Container and Module Devices" default (ACPI_HOTPLUG_MEMORY || ACPI_HOTPLUG_CPU)
@@ -558,3 +542,19 @@ config TPS68470_PMIC_OPREGION using this, are probed. endif # ACPI + +config X86_PM_TIMER + bool "Power Management Timer Support" if EXPERT + depends on X86 && (ACPI || JAILHOUSE_GUEST) + default y + help + The Power Management Timer is available on all ACPI-capable, + in most cases even if ACPI is unusable or blacklisted. + + This timing source is not affected by power management features + like aggressive processor idling, throttling, frequency and/or + voltage scaling, unlike the commonly used Time Stamp Counter + (TSC) timing source. + + You should nearly always say Y here because many modern + systems require this timer.
Signature · 1 line
2.12.3
Re: [PATCH 04/10] x86: jailhouse: Enable PMTIMER
Quoted 3 lines · Jan Kiszka
Jailhouse exposes the PMTIMER as only reference clock to all cells. Pick up its address from the setup data. Allow to enable the Linux support of it by relaxing its strict dependency on ACPI.
Yes, that should work w/o problems.
Thanks,
tglx
Jan Kiszka
maintainerPATCH 5/10 x86: jailhouse: Set up timekeeping
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
Calibrate the TSC and, where necessary, the APIC timer against the TMTIMER. We need our own implementation as neither the PIC nor the HPET are available, and the standard calibration routines try to make use of them.
--- arch/x86/include/asm/tsc.h | 3 ++ arch/x86/kernel/jailhouse.c | 82 +++++++++++++++++++++++++++++++++++++++++++++ arch/x86/kernel/tsc.c | 14 ++++---- 3 files changed, 92 insertions(+), 7 deletions(-)
arch/x86/include/asm/tsc.h +3 -0
@@ -71,4 +71,7 @@ extern void tsc_restore_sched_clock_state(void);unsigned long cpu_khz_from_msr (void ); +u64 tsc_read_refs (u64 *ref,int hpet); +unsigned long tsc_calc_pmtimer_ref (u64 deltatsc,u64 pm1,u64 pm2); +#endif /* _ASM_X86_TSC_H */
arch/x86/kernel/jailhouse.c +82 -0
94 lines not on this page · raw message
arch/x86/kernel/tsc.c +7 -7
@@ -286,7 +286,7 @@ __setup("tsc=", tsc_setup);/* * Read TSC and the reference counters. Take care of SMI disturbance */ -static u64 tsc_read_refs (u64 *p,int hpet) +u64 tsc_read_refs (u64 *p,int hpet) {u64 t1, t2;int i;
@@ -307,7 +307,7 @@ static u64 tsc_read_refs(u64 *p, int hpet)/* * Calculate the TSC frequency from HPET reference */ -static unsigned long calc_hpet_ref (u64 deltatsc,u64 hpet1,u64 hpet2) +static unsigned long tsc_calc_hpet_ref (u64 deltatsc,u64 hpet1,u64 hpet2) {u64 tmp;
@@ -324,7 +324,7 @@ static unsigned long calc_hpet_ref(u64 deltatsc, u64 hpet1, u64 hpet2)/* * Calculate the TSC frequency from PMTimer reference */ -static unsigned long calc_pmtimer_ref (u64 deltatsc,u64 pm1,u64 pm2) +unsigned long tsc_calc_pmtimer_ref (u64 deltatsc,u64 pm1,u64 pm2) {u64 tmp;
@@ -728,9 +728,9 @@ unsigned long native_calibrate_cpu(void) tsc2 = (tsc2 - tsc1) *1000000LL ;if (hpet) - tsc2 =calc_hpet_ref (tsc2, ref1, ref2); + tsc2 =tsc_calc_hpet_ref (tsc2, ref1, ref2);else - tsc2 =calc_pmtimer_ref (tsc2, ref1, ref2); + tsc2 =tsc_calc_pmtimer_ref (tsc2, ref1, ref2); tsc_ref_min =min (tsc_ref_min, (unsigned long ) tsc2);
@@ -1200,9 +1200,9 @@ static void tsc_refine_calibration_work(struct work_struct *work) delta = tsc_stop - tsc_start; delta *=1000000LL ;if (hpet) - freq =calc_hpet_ref (delta, ref_start, ref_stop); + freq =tsc_calc_hpet_ref (delta, ref_start, ref_stop);else - freq =calc_pmtimer_ref (delta, ref_start, ref_stop); + freq =tsc_calc_pmtimer_ref (delta, ref_start, ref_stop);/* Make sure we're within 1% */ if (abs (tsc_khz - freq) > tsc_khz/100 )
Signature · 1 line
2.12.3
Re: [PATCH 05/10] x86: jailhouse: Set up timekeeping
Quoted 4 lines · Jan Kiszka
Calibrate the TSC and, where necessary, the APIC timer against the TMTIMER. We need our own implementation as neither the PIC nor the HPET are available, and the standard calibration routines try to make use of them.
Why is this needed at all?
The host the frequency already. So this can be done w/o pmtimer and extra calibration routine.
Thanks,
tglx
Re: [PATCH 05/10] x86: jailhouse: Set up timekeeping
Quoted 10 lines · Thomas Gleixner
Quoted 4 lines · Jan Kiszka
Calibrate the TSC and, where necessary, the APIC timer against the TMTIMER. We need our own implementation as neither the PIC nor the HPET are available, and the standard calibration routines try to make use of them.
Why is this needed at all?
The host the frequency already. So this can be done w/o pmtimer and extra calibration routine.
The hypervisor does not have the frequencies. It will never use the APIC timer (it's owned by the guests), and it has no use case for the TSC so far. Only the root cell (the Linux that booted the system) has that data. Now we could
- trust the root cell to provide the right values and export them during startup to the hypervisor and from there to the non-root cells.
- calculate the frequencies once and store them in the hyperivsor config, just like other system-specific information, for re-export to the cells.
But I don't think option 1 will be ok for all use cases. Maybe a combination of both, falling back to the root cell data if nothing is defined in the config. Let me think about this.
Jan
Re: [PATCH 05/10] x86: jailhouse: Set up timekeeping
Quoted 20 lines · Jan Kiszka
Quoted 10 lines · Thomas Gleixner
Quoted 4 lines · Jan Kiszka
Calibrate the TSC and, where necessary, the APIC timer against the TMTIMER. We need our own implementation as neither the PIC nor the HPET are available, and the standard calibration routines try to make use of them.
Why is this needed at all?
The host the frequency already. So this can be done w/o pmtimer and extra calibration routine.
The hypervisor does not have the frequencies. It will never use the APIC timer (it's owned by the guests), and it has no use case for the TSC so far. Only the root cell (the Linux that booted the system) has that data. Now we could
- trust the root cell to provide the right values and export them during startup to the hypervisor and from there to the non-root cells.
Quoted 7 lines
- calculate the frequencies once and store them in the hyperivsor config, just like other system-specific information, for re-export to the cells.
But I don't think option 1 will be ok for all use cases. Maybe a combination of both, falling back to the root cell data if nothing is defined in the config. Let me think about this.
Another question is whether systems which can support jailhouse, have the frequencies available via cpuid/msr and can avoid that calibration thing completely.
Thanks,
tglx
Re: [PATCH 05/10] x86: jailhouse: Set up timekeeping
Quoted 25 lines · Thomas Gleixner
Quoted 24 lines · Jan Kiszka
Quoted 10 lines · Thomas Gleixner
Quoted 4 lines · Jan Kiszka
Calibrate the TSC and, where necessary, the APIC timer against the TMTIMER. We need our own implementation as neither the PIC nor the HPET are available, and the standard calibration routines try to make use of them.
Why is this needed at all?
The host the frequency already. So this can be done w/o pmtimer and extra calibration routine.
The hypervisor does not have the frequencies. It will never use the APIC timer (it's owned by the guests), and it has no use case for the TSC so far. Only the root cell (the Linux that booted the system) has that data. Now we could
- trust the root cell to provide the right values and export them during startup to the hypervisor and from there to the non-root cells.
- calculate the frequencies once and store them in the hyperivsor config, just like other system-specific information, for re-export to the cells.
Quoted 7 lines
Quoted 3 lines
But I don't think option 1 will be ok for all use cases. Maybe a combination of both, falling back to the root cell data if nothing is defined in the config. Let me think about this.
Another question is whether systems which can support jailhouse, have the frequencies available via cpuid/msr and can avoid that calibration thing completely.
OK, some may (not Xeons, though), and we would not exploit it with this approach.
Jan
Re: [PATCH 05/10] x86: jailhouse: Set up timekeeping
Quoted 30 lines · Jan Kiszka
Quoted 29 lines · Thomas Gleixner
Quoted 27 lines · Jan Kiszka
Quoted 10 lines · Thomas Gleixner
Quoted 4 lines · Jan Kiszka
Calibrate the TSC and, where necessary, the APIC timer against the TMTIMER. We need our own implementation as neither the PIC nor the HPET are available, and the standard calibration routines try to make use of them.
Why is this needed at all?
The host the frequency already. So this can be done w/o pmtimer and extra calibration routine.
The hypervisor does not have the frequencies. It will never use the APIC timer (it's owned by the guests), and it has no use case for the TSC so far. Only the root cell (the Linux that booted the system) has that data. Now we could
- trust the root cell to provide the right values and export them during startup to the hypervisor and from there to the non-root cells.
- calculate the frequencies once and store them in the hyperivsor config, just like other system-specific information, for re-export to the cells.
But I don't think option 1 will be ok for all use cases. Maybe a combination of both, falling back to the root cell data if nothing is defined in the config. Let me think about this.
Quoted 6 lines
Quoted 3 lines
Another question is whether systems which can support jailhouse, have the frequencies available via cpuid/msr and can avoid that calibration thing completely.
OK, some may (not Xeons, though), and we would not exploit it with this approach.
Ok. Let's expose pmtimer it's not a big issue.
Thanks,
tglx
Re: [PATCH 05/10] x86: jailhouse: Set up timekeeping
Quoted 31 lines · Thomas Gleixner
Quoted 30 lines · Jan Kiszka
Quoted 29 lines · Thomas Gleixner
Quoted 27 lines · Jan Kiszka
Quoted 10 lines · Thomas Gleixner
Quoted 4 lines · Jan Kiszka
Calibrate the TSC and, where necessary, the APIC timer against the TMTIMER. We need our own implementation as neither the PIC nor the HPET are available, and the standard calibration routines try to make use of them.
Why is this needed at all?
The host the frequency already. So this can be done w/o pmtimer and extra calibration routine.
The hypervisor does not have the frequencies. It will never use the APIC timer (it's owned by the guests), and it has no use case for the TSC so far. Only the root cell (the Linux that booted the system) has that data. Now we could
- trust the root cell to provide the right values and export them during startup to the hypervisor and from there to the non-root cells.
- calculate the frequencies once and store them in the hyperivsor config, just like other system-specific information, for re-export to the cells.
But I don't think option 1 will be ok for all use cases. Maybe a combination of both, falling back to the root cell data if nothing is defined in the config. Let me think about this.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Another question is whether systems which can support jailhouse, have the frequencies available via cpuid/msr and can avoid that calibration thing completely.
OK, some may (not Xeons, though), and we would not exploit it with this approach.
Ok. Let's expose pmtimer it's not a big issue.
I think PM timer should to be exposed independently of the question wheather calibrate or not. If - for whatever reason - TSC is/becomes unstable, we should have a fallback clock.
I've started to hack on forwarding calibration results from (primary) Linux to the other guests with the option to alternatively hard-code the values in the configuration. Seems to work find, just requires to export lapic_timer_frequency. I'll push updates the next days that exploit this and simplifies this patch.
Jan
Jan Kiszka
maintainerPATCH 6/10 x86: jailhouse: Avoid access of unsupported platform resources
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
We don't have CMOS access, thus we can't set the warm-reset vectors in do_boot_cpu. There is no RTC, thus also no wall clock. Furthermore, there are no ISA IRQs and no PIC. So fill the platform callbacks accordingly.
--- arch/x86/kernel/jailhouse.c | 10 ++++++++++ arch/x86/kernel/smpboot.c | 7 +++++-- 2 files changed, 15 insertions(+), 2 deletions(-)
arch/x86/kernel/jailhouse.c +10 -0
@@ -15,6 +15,7 @@#include <asm/apic.h> #include <asm/cpu.h> #include <asm/hypervisor.h> +#include <asm/i8259.h> #include <asm/setup.h> #define SETUP_JAILHOUSE 0x53484c4a/* "JLHS" */
@@ -50,6 +51,11 @@ static uint32_t __init jailhouse_detect(void)return jailhouse_cpuid_base (); } +static void jailhouse_get_wallclock (struct timespec *now) +{ +memset (now,0 ,sizeof (*now)); +} +#define MAX_RETRIES 5#define SMI_TRESHOLD 50000
@@ -139,7 +145,11 @@ static void __init jailhouse_init_platform(void)unsigned int cpu; x86_init.timers.timer_init = jailhouse_timer_init; + x86_init.irqs.pre_vector_init = x86_init_noop; + legacy_pic = &null_legacy_pic; + x86_platform.legacy.rtc =0 ; + x86_platform.get_wallclock = jailhouse_get_wallclock; x86_platform.calibrate_cpu = jailhouse_calibrate_cpu; x86_platform.calibrate_tsc = jailhouse_calibrate_tsc;
arch/x86/kernel/smpboot.c +5 -2
@@ -78,6 +78,7 @@#include <asm/realmode.h> #include <asm/misc.h> #include <asm/qspinlock.h> +#include <asm/jailhouse_para.h> /* Number of siblings per CPU package */ int smp_num_siblings =1 ;
@@ -1006,7 +1007,8 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle,* the targeted processor. */ -if (get_uv_system_type () !=UV_NON_UNIQUE_APIC ) { +if (get_uv_system_type () !=UV_NON_UNIQUE_APIC && + !jailhouse_paravirt ()) {pr_debug ("Setting warm reset code and vector.\n" );
@@ -1078,7 +1080,8 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle,/* mark "stuck" area as not stuck */ *trampoline_status =0 ; -if (get_uv_system_type () !=UV_NON_UNIQUE_APIC ) { +if (get_uv_system_type () !=UV_NON_UNIQUE_APIC && + !jailhouse_paravirt ()) {/* * Cleanup possible dangling ends... */
Signature · 1 line
2.12.3
Re: [PATCH 06/10] x86: jailhouse: Avoid access of unsupported platform resources
Quoted 3 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
We don't have CMOS access, thus we can't set the warm-reset vectors in
s/We/A jailhouse cell/ or s/We/A jailhouse guest/
Quoted 10 lines
do_boot_cpu. There is no RTC, thus also no wall clock. Furthermore, there are no ISA IRQs and no PIC. So fill the platform callbacks accordingly.arch/x86/kernel/jailhouse.c +4
-0+static void jailhouse_get_wallclock (struct timespec *now) +{ +memset (now,0 ,sizeof (*now)); +}
arch/x86/kernel/jailhouse.c static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle, · line 1013
Quoted 8 lines
arch/x86/kernel/jailhouse.c +2
-1@@ -1006,7 +1007,8 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle,* the targeted processor. */ -if (get_uv_system_type () !=UV_NON_UNIQUE_APIC ) { +if (get_uv_system_type () !=UV_NON_UNIQUE_APIC && + !jailhouse_paravirt ()) {pr_debug ("Setting warm reset code and vector.\n" );
Adding more here might justify a function pointer in x86_platform_ops. Not sure though, but we have an increasing amount of xen()/kvm()/whatever conditionals popping up all over the place. And now you add jailhouse() ones :)
Thanks,
tglx
Re: [PATCH 06/10] x86: jailhouse: Avoid access of unsupported platform resources
Quoted 25 lines · Thomas Gleixner
Quoted 3 lines · Jan Kiszka
From: Jan Kiszka <[email protected]>
We don't have CMOS access, thus we can't set the warm-reset vectors in
s/We/A jailhouse cell/ or s/We/A jailhouse guest/
Quoted 17 lines
do_boot_cpu. There is no RTC, thus also no wall clock. Furthermore, there are no ISA IRQs and no PIC. So fill the platform callbacks accordingly.arch/x86/kernel/jailhouse.c +6
-1+static void jailhouse_get_wallclock (struct timespec *now) +{ +memset (now,0 ,sizeof (*now)); +}@@ -1006,7 +1007,8 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle,* the targeted processor. */ -if (get_uv_system_type () !=UV_NON_UNIQUE_APIC ) { +if (get_uv_system_type () !=UV_NON_UNIQUE_APIC && + !jailhouse_paravirt ()) {
Quoted 6 lines
arch/x86/kernel/smpboot.c static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle, · line 1013
Quoted 1 line
arch/x86/kernel/smpboot.c +0
-0pr_debug ("Setting warm reset code and vector.\n" );Adding more here might justify a function pointer in
x86_platform_ops. Not sure though, but we have an increasing amount of xen()/kvm()/whatever conditionals popping up all over the place. And now you addjailhouse()ones :)
Actually we don't want a function pointer as we would need two. A simple flag would be sufficient.
Thanks,
tglx
Jan Kiszka
maintainerPATCH 7/10 x86: jailhouse: Silence ACPI warning
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
Jailhouse support does not depend on ACPI, and does not even use it. But if it should be enabled, avoid warning about its absence in the platform.
--- arch/x86/kernel/jailhouse.c | 6 ++++++ 1 file changed, 6 insertions(+)
arch/x86/kernel/jailhouse.c +6 -0
@@ -180,6 +180,12 @@ static void __init jailhouse_init_platform(void) smp_found_config =1 ;early_memunmap (data,sizeof (*data)); + +/* +* Avoid that the kernel complains about missing ACPI tables - there +* are none in a non-root cell. +*/ +disable_acpi (); }bool jailhouse_paravirt (void )
Signature · 1 line
2.12.3
Jan Kiszka
maintainerPATCH 8/10 x86: jailhouse: Halt instead of failing to restart
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
Jailhouse provides no guest-initiated restart. So, do not even try to.
--- arch/x86/kernel/jailhouse.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
arch/x86/kernel/jailhouse.c +10 -0
@@ -12,10 +12,12 @@#include <linux/acpi_pmtmr.h> #include <linux/kernel.h> +#include <linux/reboot.h> #include <asm/apic.h> #include <asm/cpu.h> #include <asm/hypervisor.h> #include <asm/i8259.h> +#include <asm/reboot.h> #include <asm/setup.h> #define SETUP_JAILHOUSE 0x53484c4a/* "JLHS" */
@@ -133,6 +135,12 @@ static unsigned long jailhouse_calibrate_tsc(void)return 0 ; } +static void jailhouse_no_restart (void ) +{ +pr_notice ("Jailhouse: Restart not supported, halting\n" ); +machine_halt (); +} +static unsigned int x2apic_get_apic_id (unsigned long id) {return id;
@@ -153,6 +161,8 @@ static void __init jailhouse_init_platform(void) x86_platform.calibrate_cpu = jailhouse_calibrate_cpu; x86_platform.calibrate_tsc = jailhouse_calibrate_tsc; + machine_ops.emergency_restart = jailhouse_no_restart; + data =early_memremap (pa_data,sizeof (*data)); if (data->header.type != SETUP_JAILHOUSE ||
Signature · 1 line
2.12.3
Jan Kiszka
maintainerPATCH 9/10 x86: jailhouse: Wire up IOAPIC for legacy UART ports
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
The typical I/O interrupts in non-root cells are MSI-based. However, the platform UARTs do not support MSI. In order to run a non-root cell that shall be use one of them, we need to register the standard IOAPIC and set 1:1 routing for IRQ 3 and 4.
If an IOAPIC is not available, the boot loader clears standard_ioapic in the setup data, and we skip the registration. If we should not be allowed to use one of those pins, Jailhouse will simply ignore our accesses.
--- arch/x86/kernel/jailhouse.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
arch/x86/kernel/jailhouse.c +20 -0
@@ -17,6 +17,7 @@#include <asm/cpu.h> #include <asm/hypervisor.h> #include <asm/i8259.h> +#include <asm/irqdomain.h> #include <asm/reboot.h> #include <asm/setup.h>
@@ -148,6 +149,14 @@ static unsigned int x2apic_get_apic_id(unsigned long id)static void __init jailhouse_init_platform (void ) { +struct ioapic_domain_cfg ioapic_cfg = { + .type =IOAPIC_DOMAIN_STRICT , + .ops = &mp_ioapic_irqdomain_ops, + }; +struct mpc_intsrc mp_irq = { + .type =MP_INTSRC , + .irqtype = mp_INT, + };u64 pa_data = boot_params.hdr.setup_data;struct jailhouse_setup_data *data;unsigned int cpu;
@@ -189,6 +198,17 @@ static void __init jailhouse_init_platform(void) boot_cpu_apic_version); smp_found_config =1 ; +if (data->standard_ioapic) { +mp_register_ioapic (0 ,0xfec00000 , gsi_top, &ioapic_cfg); + +/* Register 1:1 mapping for legacy UART IRQs 3 and 4 */ + mp_irq.srcbusirq = mp_irq.dstirq =3 ; +mp_save_irq (&mp_irq); + + mp_irq.srcbusirq = mp_irq.dstirq =4 ; +mp_save_irq (&mp_irq); + } +early_memunmap (data,sizeof (*data));/*
Signature · 1 line
2.12.3
Re: [PATCH 09/10] x86: jailhouse: Wire up IOAPIC for legacy UART ports
Quoted 17 lines · Jan Kiszka
static void
__initjailhouse_init_platform(void)
{arch/x86/kernel/jailhouse.c +8
-0+struct ioapic_domain_cfg ioapic_cfg = { + .type =IOAPIC_DOMAIN_STRICT , + .ops = &mp_ioapic_irqdomain_ops, + }; +struct mpc_intsrc mp_irq = { + .type =MP_INTSRC , + .irqtype = mp_INT, + };u64 pa_data = boot_params.hdr.setup_data;struct jailhouse_setup_data *data;unsigned int cpu;@@ -189,6 +198,17 @@ static void __init jailhouse_init_platform(void) boot_cpu_apic_version); smp_found_config =1 ;
arch/x86/kernel/jailhouse.c static void __init jailhouse_init_platform(void) · line 209
Quoted 10 lines
arch/x86/kernel/jailhouse.c +9
-0+if (data->standard_ioapic) { +mp_register_ioapic (0 ,0xfec00000 , gsi_top, &ioapic_cfg); + +/* Register 1:1 mapping for legacy UART IRQs 3 and 4 */ + mp_irq.srcbusirq = mp_irq.dstirq =3 ; +mp_save_irq (&mp_irq); + + mp_irq.srcbusirq = mp_irq.dstirq =4 ; +mp_save_irq (&mp_irq);
Again, that can run later. And please split out initialization for particular features into separate functions. Otherwise the parse function will become a kitchen sink of all sorts.
Thanks,
tglx
Jan Kiszka
maintainerPATCH 10/10 x86: jailhouse: Initialize PCI support
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
With this change, PCI devices can be detected and used inside a non-root cell.
--- arch/x86/kernel/jailhouse.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
arch/x86/kernel/jailhouse.c +10 -0
@@ -18,6 +18,7 @@#include <asm/hypervisor.h> #include <asm/i8259.h> #include <asm/irqdomain.h> +#include <asm/pci_x86.h> #include <asm/reboot.h> #include <asm/setup.h>
@@ -211,6 +212,15 @@ static void __init jailhouse_init_platform(void)early_memunmap (data,sizeof (*data)); + pci_probe =0 ; +pci_direct_init (1 ); + +/* +* There are no bridges on the virtual PCI root bus under Jailhouse, +* thus no other way to discover all devices than a full scan. +*/ + pcibios_last_bus =0xff ; +/* * Avoid that the kernel complains about missing ACPI tables - there * are none in a non-root cell.
Signature · 1 line
2.12.3
H. Peter Anvin
reviewerQuoted 10 lines · Jan Kiszka
This series paves the way to run Linux in so-called non-root cells (guest partitions) of the Jailhouse hypervisor.
Jailhouse [1] was started 4 years ago as an open-source (GPL) leight- weight hypervisor that statically partitions SMP systems. It's unique in that it uses one Linux instance, the root cell, as boot loader and management console. Jailhouse targets use cases for hard real-time and safety-critical systems that KVM cannot cater due to its inherent complexity.
Quoted 7 lines
Jaihouse can run bare-metal, free and closed-source RTOSes as secondary guests and, with this series, also x86 Linux instances. While ARM and ARM64 non-root Linux guests are feasible without extra patches, thanks to the high configurability via device trees, x86 requires special platform support, mostly to step away from non-existing resources in a non-root Jailhouse cell.
Could you please write a single summary about the virtualization holes in Jailhouse that you are papering over?
-hpa
Jan Kiszka
maintainerQuoted 19 lines · H. Peter Anvin
Quoted 10 lines · Jan Kiszka
This series paves the way to run Linux in so-called non-root cells (guest partitions) of the Jailhouse hypervisor.
Jailhouse [1] was started 4 years ago as an open-source (GPL) leight- weight hypervisor that statically partitions SMP systems. It's unique in that it uses one Linux instance, the root cell, as boot loader and management console. Jailhouse targets use cases for hard real-time and safety-critical systems that KVM cannot cater due to its inherent complexity.
Quoted 7 lines
Jaihouse can run bare-metal, free and closed-source RTOSes as secondary guests and, with this series, also x86 Linux instances. While ARM and ARM64 non-root Linux guests are feasible without extra patches, thanks to the high configurability via device trees, x86 requires special platform support, mostly to step away from non-existing resources in a non-root Jailhouse cell.
Quoted 2 lines
Could you please write a single summary about the virtualization holes in Jailhouse that you are papering over?
Actually, we are not virtualizing any full device in Jailhouse. That is an architectural decision that allows to keep the critical code base very small (9200 LOC on Intel right now). So, anything that is not there multiple times is not exposed or problematic state modifications are blocked.
That leaves the non-root cells with:
- local CPU resources (processor, LAPIC etc.)
- exclusive memory regions
- exclusive PCI devices (or functions)
- read access to the PM timer (as clocksource)
- exclusively assigned pins on the IOAPIC (if any - it's not recommended to partition it)
- virtual shared memory devices for inter-cell communication
Thus, the list of non-existing x86 resources and features:
- LPC with all its legacy devices like PIT, PIC, RTC, SMBus etc. (unless you decide to pass one through exclusively)
- HPET
- normal PCI bus topology
- BIOS / UEFI firmware services, including ACPI enumeration (that lacks means to describe absence of PC platform devices anyway)
- mass storage or network virtualization - sharing such devices is not in the scope of the hypervisor
- no restart interface
Deviations and restrictions:
- CPU start address is configurable, typically set to 0 - but that's handled by the boot loader (comes with Jailhouse)
- LAPIC can only be operated in flat mode, and the content of LDR is frozen
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT RDA ITP SES-DE Corporate Competence Center Embedded Linux