Skip to contentSkip to conversation

x86: Add support for running as secondary Jailhouse guest

PATCH In review Jan Kiszka 3 participants 27 messages Nov 27, 2017 – Jan 23, 2018

Subsystems (5)
lore
Share & tools
lore

Jan Kiszka

author maintainer
#0 lore raw

v2 cover x86: Add support for running as secondary Jailhouse guest

This series paves the way to run Linux in so-called non-root cells (guest partitions) of the Jailhouse hypervisor.

Jailhouse [1] was started 4 years ago as an open-source (GPL) leight- weight hypervisor that statically partitions SMP systems. It's unique in that it uses one Linux instance, the root cell, as boot loader and management console. Jailhouse targets use cases for hard real-time and safety-critical systems that KVM cannot cater due to its inherent complexity.

Jaihouse can run bare-metal, free and closed-source RTOSes as secondary guests and, with this series, also x86 Linux instances. While ARM and ARM64 non-root Linux guests are feasible without extra patches, thanks to the high configurability via device trees, x86 requires special platform support, mostly to step away from non-existing resources in a non-root Jailhouse cell. These differences are:

- no legacy devices such as PIC, PIT, RTC/CMOS, i8042
- no HPET
- no ACPI, no other BIOS services; platform information is provided from hypervisor config
- APIC and IOAPIC (if any) are available at fixed MMIO addresses and the APIC is limited to flat physical mode
- no complete PCI topology
- no restart supported
- use precalibrated TSC and APIC timer frequencies

This series ensures that Linux can boot in a non-root cell, including SMP cells, has working timekeeping and can use the platform UARTs and PCI devices as assigned to it. In follow-up series, we will propose optimizations and enhancements for the PCI support, a simplistic debug console, and some improvement for Linux guests on ARM.

What is not yet in upstream-ready state is a driver for inter-cell communication. The current implementation of virtual peer-to-peer network [2] uses an enhanced version of the QEMU ivshmem shared memory device. However we still need to finish the evaluation of virtio / vhost-pci options prior to settling over the final interface.

This patch series is also available at

git://git.kiszka.org/linux.git e8d19494b96b

Changes in v2:
- removed calibration in favor of static values now passed from boot loader
- refactored hypervisor platform setup according to feedback, specifically
- proper switch to x2APIC ops
- APIC and IOAPIC registration from x86_init.mpparse.get_smp_config
- PCI setup from x86_init.pci.arch_init
- disabled i8042 devices
- control of warm reset setup via platform feature flag
- proper walk of setup data linked list
- decoupled CONFIG_JAILHOUSE from CONFIG_PARAVIRT
- fixed IOAPIC routing setup for UARTs (trigger and polarity set)
- symbolic irqflag values in MP config tables

Jan

[1] http://jailhouse-project.org
[2] http://git.kiszka.org/?p=linux.git;a=shortlog;h=refs/heads/queues/jailhouse

Jan Kiszka (12):
x86/apic: Install an empty physflat_init_apic_ldr
x86: Control warm reset setup via legacy feature flag
x86: Introduce and use MP IRQ trigger and polarity defines x86/jailhouse: Add infrastructure for running in non-root cell x86/jailhouse: Enable APIC and SMP support x86/jailhouse: Enable PMTIMER
x86/jailhouse: Set up timekeeping
x86/jailhouse: Avoid access of unsupported platform resources x86/jailhouse: Silence ACPI warning
x86/jailhouse: Halt instead of failing to restart x86/jailhouse: Wire up IOAPIC for legacy UART ports x86/jailhouse: Initialize PCI support

 arch/x86/Kconfig                      |   9 ++
 arch/x86/include/asm/hypervisor.h     |   1 +
 arch/x86/include/asm/jailhouse_para.h |  27 +++++
 arch/x86/include/asm/mpspec_def.h     |  14 ++-
 arch/x86/include/asm/x86_init.h       |   1 +
 arch/x86/include/uapi/asm/bootparam.h |  22 ++++
 arch/x86/kernel/Makefile              |   2 +
 arch/x86/kernel/apic/apic_flat_64.c   |  16 ++-
 arch/x86/kernel/apic/io_apic.c        |  20 ++--
 arch/x86/kernel/apic/x2apic_uv_x.c    |   1 +
 arch/x86/kernel/cpu/hypervisor.c      |   4 +
 arch/x86/kernel/jailhouse.c           | 205 ++++++++++++++++++++++++++++++++++
 arch/x86/kernel/mpparse.c             |  23 ++--
 arch/x86/kernel/platform-quirks.c     |   1 +
 arch/x86/kernel/smpboot.c             |   4 +-
 arch/x86/platform/intel-mid/sfi.c     |   5 +-
 drivers/acpi/Kconfig                  |  32 +++---
 17 files changed, 341 insertions(+), 46 deletions(-)
 create mode 100644 arch/x86/include/asm/jailhouse_para.h
 create mode 100644 arch/x86/kernel/jailhouse.c
Signature · 1 line
2.12.3
author maintainer
#1 lore raw

PATCH v2 1/12 x86/apic: Install an empty physflat_init_apic_ldr

From: Jan Kiszka <[email protected]>

As the comment already stated, there is no need for setting up LDR (and DFR) in physflat mode as it remains unused (see SDM, 10.6.2.1). flat_init_apic_ldr only served as a placeholder for a nop operation so far, causing no harm.

That will change when running over the Jailhouse hypervisor. Here we must not touch LDR in a way that destroys the mapping originally set up by the Linux root cell. Jailhouse enforces this setting in order to efficiently validate any IPI requests sent by a cell.

Avoid a needless clash caused by flat_init_apic_ldr by installing a true nop handler.

---
 arch/x86/kernel/apic/apic_flat_64.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
arch/x86/kernel/apic/apic_flat_64.c +10 -2
index aa85690e9b64..226a6ab22b35 100644
@@ -218,6 +218,15 @@ static int physflat_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
 	return 0;
 }
 
+static void physflat_init_apic_ldr(void)
+{
+	/*
+	 * LDR and DFR are not involved in physflat mode, rather:
+	 * "In physical destination mode, the destination processor is
+	 * specified by its local APIC ID [...]." (Intel SDM, 10.6.2.1)
+	 */
+}
+
 static void physflat_send_IPI_allbutself(int vector)
 {
 	default_send_IPI_mask_allbutself_phys(cpu_online_mask, vector);
@@ -251,8 +260,7 @@ static struct apic apic_physflat __ro_after_init = {
 	.dest_logical			= 0,
 	.check_apicid_used		= NULL,
 
-	/* not needed, but shouldn't hurt: */
-	.init_apic_ldr			= flat_init_apic_ldr,
+	.init_apic_ldr			= physflat_init_apic_ldr,
 
 	.ioapic_phys_id_map		= NULL,
 	.setup_apic_routing		= NULL,
Signature · 1 line
2.12.3
#2 lore raw

[tip:x86/platform] x86/apic: Install an empty physflat_init_apic_ldr

x86/apic: Install an empty physflat_init_apic_ldr
Commit-ID:  32c9c801a853f181448ed4e8730168c556f9e05a
Gitweb:     https://git.kernel.org/tip/32c9c801a853f181448ed4e8730168c556f9e05a
Author:     Jan Kiszka <[email protected]>
AuthorDate: Mon, 27 Nov 2017 09:11:43 +0100
Committer:  Thomas Gleixner <[email protected]>
CommitDate: Sun, 14 Jan 2018 21:11:53 +0100

x86/apic: Install an empty physflat_init_apic_ldr

As the comment already stated, there is no need for setting up LDR (and DFR) in physflat mode as it remains unused (see SDM, 10.6.2.1). flat_init_apic_ldr only served as a placeholder for a nop operation so far, causing no harm.

That will change when running over the Jailhouse hypervisor. Here we must not touch LDR in a way that destroys the mapping originally set up by the Linux root cell. Jailhouse enforces this setting in order to efficiently validate any IPI requests sent by a cell.

Avoid a needless clash caused by flat_init_apic_ldr by installing a true nop handler.

---
 arch/x86/kernel/apic/apic_flat_64.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
arch/x86/kernel/apic/apic_flat_64.c +10 -2
index 25a8702..4b55477 100644
@@ -218,6 +218,15 @@ static int physflat_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
 	return 0;
 }
 
+static void physflat_init_apic_ldr(void)
+{
+	/*
+	 * LDR and DFR are not involved in physflat mode, rather:
+	 * "In physical destination mode, the destination processor is
+	 * specified by its local APIC ID [...]." (Intel SDM, 10.6.2.1)
+	 */
+}
+
 static void physflat_send_IPI_allbutself(int vector)
 {
 	default_send_IPI_mask_allbutself_phys(cpu_online_mask, vector);
@@ -251,8 +260,7 @@ static struct apic apic_physflat __ro_after_init = {
 	.dest_logical			= 0,
 	.check_apicid_used		= NULL,
 
-	/* not needed, but shouldn't hurt: */
-	.init_apic_ldr			= flat_init_apic_ldr,
+	.init_apic_ldr			= physflat_init_apic_ldr,
 
 	.ioapic_phys_id_map		= NULL,
 	.setup_apic_routing		= NULL,
author maintainer
#3 lore raw

PATCH v2 2/12 x86: Control warm reset setup via legacy feature flag

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

Allow to turn off the setup of BIOS-managed warm reset via a new flag in x86_legacy_features. Besides the UV1, we will soon add another platform that needs this switched off.

---
 arch/x86/include/asm/x86_init.h    | 1 +
 arch/x86/kernel/apic/x2apic_uv_x.c | 1 +
 arch/x86/kernel/platform-quirks.c  | 1 +
 arch/x86/kernel/smpboot.c          | 4 ++--
 4 files changed, 5 insertions(+), 2 deletions(-)
arch/x86/include/asm/x86_init.h +1 -0
index aa4747569e23..fc2f082ac635 100644
@@ -212,6 +212,7 @@ enum x86_legacy_i8042_state {
 struct x86_legacy_features {
 	enum x86_legacy_i8042_state i8042;
 	int rtc;
+	int warm_reset;
 	int no_vga;
 	int reserve_bios_regions;
 	struct x86_legacy_devices devices;
arch/x86/kernel/apic/x2apic_uv_x.c +1 -0
index e1b8e8bf6b3c..6de35fc8fb3a 100644
@@ -316,6 +316,7 @@ static int __init uv_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
 	} else if (!strcmp(oem_table_id, "UVH")) {
 		/* Only UV1 systems: */
 		uv_system_type = UV_NON_UNIQUE_APIC;
+		x86_platform.legacy.warm_reset = 0;
 		__this_cpu_write(x2apic_extra_bits, pnodeid << uvh_apicid.s.pnode_shift);
 		uv_set_apicid_hibit();
 		uv_apic = 1;
arch/x86/kernel/platform-quirks.c +1 -0
index 39a59299bfa0..235fe6008ac8 100644
@@ -9,6 +9,7 @@ void __init x86_early_init_platform_quirks(void)
 {
 	x86_platform.legacy.i8042 = X86_LEGACY_I8042_EXPECTED_PRESENT;
 	x86_platform.legacy.rtc = 1;
+	x86_platform.legacy.warm_reset = 1;
 	x86_platform.legacy.reserve_bios_regions = 0;
 	x86_platform.legacy.devices.pnpbios = 1;
 
arch/x86/kernel/smpboot.c +2 -2
index 3d01df7d7cf6..21995490fc5e 100644
@@ -947,7 +947,7 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle,
 	 * the targeted processor.
 	 */
 
-	if (get_uv_system_type() != UV_NON_UNIQUE_APIC) {
+	if (x86_platform.legacy.warm_reset) {
 
 		pr_debug("Setting warm reset code and vector.\n");
 
@@ -1019,7 +1019,7 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle,
 	/* mark "stuck" area as not stuck */
 	*trampoline_status = 0;
 
-	if (get_uv_system_type() != UV_NON_UNIQUE_APIC) {
+	if (x86_platform.legacy.warm_reset) {
 		/*
 		 * Cleanup possible dangling ends...
 		 */
Signature · 1 line
2.12.3
#4 lore raw

[tip:x86/platform] x86/platform: Control warm reset setup via legacy feature flag

x86/platform: Control warm reset setup via legacy feature flag
Commit-ID:  e348caef8b4a161cc27bec8f7500b7e100370ef1
Gitweb:     https://git.kernel.org/tip/e348caef8b4a161cc27bec8f7500b7e100370ef1
Author:     Jan Kiszka <[email protected]>
AuthorDate: Mon, 27 Nov 2017 09:11:44 +0100
Committer:  Thomas Gleixner <[email protected]>
CommitDate: Sun, 14 Jan 2018 21:11:53 +0100

x86/platform: Control warm reset setup via legacy feature flag

Allow to turn off the setup of BIOS-managed warm reset via a new flag in x86_legacy_features. Besides the UV1, the upcoming jailhose guest support needs this switched off.

---
 arch/x86/include/asm/x86_init.h    | 1 +
 arch/x86/kernel/apic/x2apic_uv_x.c | 1 +
 arch/x86/kernel/platform-quirks.c  | 1 +
 arch/x86/kernel/smpboot.c          | 4 ++--
 4 files changed, 5 insertions(+), 2 deletions(-)
arch/x86/include/asm/x86_init.h +1 -0
index aa47475..fc2f082 100644
@@ -212,6 +212,7 @@ enum x86_legacy_i8042_state {
 struct x86_legacy_features {
 	enum x86_legacy_i8042_state i8042;
 	int rtc;
+	int warm_reset;
 	int no_vga;
 	int reserve_bios_regions;
 	struct x86_legacy_devices devices;
arch/x86/kernel/apic/x2apic_uv_x.c +1 -0
index e1b8e8b..6de35fc 100644
@@ -316,6 +316,7 @@ static int __init uv_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
 	} else if (!strcmp(oem_table_id, "UVH")) {
 		/* Only UV1 systems: */
 		uv_system_type = UV_NON_UNIQUE_APIC;
+		x86_platform.legacy.warm_reset = 0;
 		__this_cpu_write(x2apic_extra_bits, pnodeid << uvh_apicid.s.pnode_shift);
 		uv_set_apicid_hibit();
 		uv_apic = 1;
arch/x86/kernel/platform-quirks.c +1 -0
index 39a5929..235fe60 100644
@@ -9,6 +9,7 @@ void __init x86_early_init_platform_quirks(void)
 {
 	x86_platform.legacy.i8042 = X86_LEGACY_I8042_EXPECTED_PRESENT;
 	x86_platform.legacy.rtc = 1;
+	x86_platform.legacy.warm_reset = 1;
 	x86_platform.legacy.reserve_bios_regions = 0;
 	x86_platform.legacy.devices.pnpbios = 1;
 
arch/x86/kernel/smpboot.c +2 -2
index ed556d5..9adcae1 100644
@@ -934,7 +934,7 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle,
 	 * the targeted processor.
 	 */
 
-	if (get_uv_system_type() != UV_NON_UNIQUE_APIC) {
+	if (x86_platform.legacy.warm_reset) {
 
 		pr_debug("Setting warm reset code and vector.\n");
 
@@ -1006,7 +1006,7 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle,
 	/* mark "stuck" area as not stuck */
 	*trampoline_status = 0;
 
-	if (get_uv_system_type() != UV_NON_UNIQUE_APIC) {
+	if (x86_platform.legacy.warm_reset) {
 		/*
 		 * Cleanup possible dangling ends...
 		 */
author maintainer
#5 lore raw

PATCH v2 3/12 x86: Introduce and use MP IRQ trigger and polarity defines

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

MP_IRQDIR_* constants pointed in the right direction but remained unused so far: It's cleaer to use symbolic values for the IRQ flags in the MP config table. That also saves some comments.

---
 arch/x86/include/asm/mpspec_def.h | 14 +++++++++++---
 arch/x86/kernel/apic/io_apic.c    | 20 ++++++++++----------
 arch/x86/kernel/mpparse.c         | 23 ++++++++++++++---------
 arch/x86/platform/intel-mid/sfi.c |  5 ++---
 4 files changed, 37 insertions(+), 25 deletions(-)
arch/x86/include/asm/mpspec_def.h +11 -3
index a6bec8028480..6fb923a34309 100644
@@ -128,9 +128,17 @@ enum mp_irq_source_types {
 	mp_ExtINT = 3
 };
 
-#define MP_IRQDIR_DEFAULT	0
-#define MP_IRQDIR_HIGH		1
-#define MP_IRQDIR_LOW		3
+#define MP_IRQPOL_DEFAULT	0x0
+#define MP_IRQPOL_ACTIVE_HIGH	0x1
+#define MP_IRQPOL_RESERVED	0x2
+#define MP_IRQPOL_ACTIVE_LOW	0x3
+#define MP_IRQPOL_MASK		0x3
+
+#define MP_IRQTRIG_DEFAULT	0x0
+#define MP_IRQTRIG_EDGE		0x4
+#define MP_IRQTRIG_RESERVED	0x8
+#define MP_IRQTRIG_LEVEL	0xc
+#define MP_IRQTRIG_MASK		0xc
 
 #define MP_APIC_ALL	0xFF
 
arch/x86/kernel/apic/io_apic.c +10 -10
index 201579dc5242..6a3405a8a13c 100644
@@ -800,18 +800,18 @@ static int irq_polarity(int idx)
 	/*
 	 * Determine IRQ line polarity (high active or low active):
 	 */
-	switch (mp_irqs[idx].irqflag & 0x03) {
-	case 0:
+	switch (mp_irqs[idx].irqflag & MP_IRQPOL_MASK) {
+	case MP_IRQPOL_DEFAULT:
 		/* conforms to spec, ie. bus-type dependent polarity */
 		if (test_bit(bus, mp_bus_not_pci))
 			return default_ISA_polarity(idx);
 		else
 			return default_PCI_polarity(idx);
-	case 1:
+	case MP_IRQPOL_ACTIVE_HIGH:
 		return IOAPIC_POL_HIGH;
-	case 2:
+	case MP_IRQPOL_RESERVED:
 		pr_warn("IOAPIC: Invalid polarity: 2, defaulting to low\n");
-	case 3:
+	case MP_IRQPOL_ACTIVE_LOW:
 	default: /* Pointless default required due to do gcc stupidity */
 		return IOAPIC_POL_LOW;
 	}
@@ -845,8 +845,8 @@ static int irq_trigger(int idx)
 	/*
 	 * Determine IRQ trigger mode (edge or level sensitive):
 	 */
-	switch ((mp_irqs[idx].irqflag >> 2) & 0x03) {
-	case 0:
+	switch (mp_irqs[idx].irqflag & MP_IRQTRIG_MASK) {
+	case MP_IRQTRIG_DEFAULT:
 		/* conforms to spec, ie. bus-type dependent trigger mode */
 		if (test_bit(bus, mp_bus_not_pci))
 			trigger = default_ISA_trigger(idx);
@@ -854,11 +854,11 @@ static int irq_trigger(int idx)
 			trigger = default_PCI_trigger(idx);
 		/* Take EISA into account */
 		return eisa_irq_trigger(idx, bus, trigger);
-	case 1:
+	case MP_IRQTRIG_EDGE:
 		return IOAPIC_EDGE;
-	case 2:
+	case MP_IRQTRIG_RESERVED:
 		pr_warn("IOAPIC: Invalid trigger mode 2 defaulting to level\n");
-	case 3:
+	case MP_IRQTRIG_LEVEL:
 	default: /* Pointless default required due to do gcc stupidity */
 		return IOAPIC_LEVEL;
 	}
arch/x86/kernel/mpparse.c +14 -9
index 3a4b12809ab5..3a43119a60d4 100644
@@ -281,7 +281,7 @@ static void __init construct_default_ioirq_mptable(int mpc_default_type)
 	int ELCR_fallback = 0;
 
 	intsrc.type = MP_INTSRC;
-	intsrc.irqflag = 0;	/* conforming */
+	intsrc.irqflag = MP_IRQTRIG_DEFAULT | MP_IRQPOL_DEFAULT;
 	intsrc.srcbus = 0;
 	intsrc.dstapic = mpc_ioapic_id(0);
 
@@ -324,10 +324,13 @@ static void __init construct_default_ioirq_mptable(int mpc_default_type)
 			 *  copy that information over to the MP table in the
 			 *  irqflag field (level sensitive, active high polarity).
 			 */
-			if (ELCR_trigger(i))
-				intsrc.irqflag = 13;
-			else
-				intsrc.irqflag = 0;
+			if (ELCR_trigger(i)) {
+				intsrc.irqflag = MP_IRQTRIG_LEVEL |
+					MP_IRQPOL_ACTIVE_HIGH;
+			} else {
+				intsrc.irqflag = MP_IRQTRIG_DEFAULT |
+					MP_IRQPOL_DEFAULT;
+			}
 		}
 
 		intsrc.srcbusirq = i;
@@ -419,7 +422,7 @@ static inline void __init construct_default_ISA_mptable(int mpc_default_type)
 	construct_ioapic_table(mpc_default_type);
 
 	lintsrc.type = MP_LINTSRC;
-	lintsrc.irqflag = 0;		/* conforming */
+	lintsrc.irqflag = MP_IRQTRIG_DEFAULT | MP_IRQPOL_DEFAULT;
 	lintsrc.srcbusid = 0;
 	lintsrc.srcbusirq = 0;
 	lintsrc.destapic = MP_APIC_ALL;
@@ -664,7 +667,7 @@ static int  __init get_MP_intsrc_index(struct mpc_intsrc *m)
 	if (m->irqtype != mp_INT)
 		return 0;
 
-	if (m->irqflag != 0x0f)
+	if (m->irqflag != (MP_IRQTRIG_LEVEL | MP_IRQPOL_ACTIVE_LOW))
 		return 0;
 
 	/* not legacy */
@@ -673,7 +676,8 @@ static int  __init get_MP_intsrc_index(struct mpc_intsrc *m)
 		if (mp_irqs[i].irqtype != mp_INT)
 			continue;
 
-		if (mp_irqs[i].irqflag != 0x0f)
+		if (mp_irqs[i].irqflag != (MP_IRQTRIG_LEVEL |
+					   MP_IRQPOL_ACTIVE_LOW))
 			continue;
 
 		if (mp_irqs[i].srcbus != m->srcbus)
@@ -784,7 +788,8 @@ static int  __init replace_intsrc_all(struct mpc_table *mpc,
 		if (mp_irqs[i].irqtype != mp_INT)
 			continue;
 
-		if (mp_irqs[i].irqflag != 0x0f)
+		if (mp_irqs[i].irqflag != (MP_IRQTRIG_LEVEL |
+					   MP_IRQPOL_ACTIVE_LOW))
 			continue;
 
 		if (nr_m_spare > 0) {
arch/x86/platform/intel-mid/sfi.c +2 -3
index 19b43e3a9f0f..7be1e1fe9ae3 100644
@@ -96,8 +96,7 @@ int __init sfi_parse_mtmr(struct sfi_table_header *table)
 			pentry->freq_hz, pentry->irq);
 		mp_irq.type = MP_INTSRC;
 		mp_irq.irqtype = mp_INT;
-		/* triggering mode edge bit 2-3, active high polarity bit 0-1 */
-		mp_irq.irqflag = 5;
+		mp_irq.irqflag = MP_IRQTRIG_EDGE | MP_IRQPOL_ACTIVE_HIGH;
 		mp_irq.srcbus = MP_BUS_ISA;
 		mp_irq.srcbusirq = pentry->irq;	/* IRQ */
 		mp_irq.dstapic = MP_APIC_ALL;
@@ -168,7 +167,7 @@ int __init sfi_parse_mrtc(struct sfi_table_header *table)
 			totallen, (u32)pentry->phys_addr, pentry->irq);
 		mp_irq.type = MP_INTSRC;
 		mp_irq.irqtype = mp_INT;
-		mp_irq.irqflag = 0xf;	/* level trigger and active low */
+		mp_irq.irqflag = MP_IRQTRIG_LEVEL | MP_IRQPOL_ACTIVE_LOW;
 		mp_irq.srcbus = MP_BUS_ISA;
 		mp_irq.srcbusirq = pentry->irq;	/* IRQ */
 		mp_irq.dstapic = MP_APIC_ALL;
Signature · 1 line
2.12.3
#6 lore raw

[tip:x86/platform] x86: Introduce and use MP IRQ trigger and polarity defines

x86: Introduce and use MP IRQ trigger and polarity defines
Commit-ID:  a09c5ec00a120dae52eceef3eebff93ed729bb43
Gitweb:     https://git.kernel.org/tip/a09c5ec00a120dae52eceef3eebff93ed729bb43
Author:     Jan Kiszka <[email protected]>
AuthorDate: Mon, 27 Nov 2017 09:11:45 +0100
Committer:  Thomas Gleixner <[email protected]>
CommitDate: Sun, 14 Jan 2018 21:11:54 +0100

x86: Introduce and use MP IRQ trigger and polarity defines

MP_IRQDIR_* constants pointed in the right direction but remained unused so
far: It's cleaner to use symbolic values for the IRQ flags in the MP config table. That also saves some comments.

---
 arch/x86/include/asm/mpspec_def.h | 14 +++++++++++---
 arch/x86/kernel/apic/io_apic.c    | 20 ++++++++++----------
 arch/x86/kernel/mpparse.c         | 23 ++++++++++++++---------
 arch/x86/platform/intel-mid/sfi.c |  5 ++---
 4 files changed, 37 insertions(+), 25 deletions(-)
arch/x86/include/asm/mpspec_def.h +11 -3
index a6bec80..6fb923a 100644
@@ -128,9 +128,17 @@ enum mp_irq_source_types {
 	mp_ExtINT = 3
 };
 
-#define MP_IRQDIR_DEFAULT	0
-#define MP_IRQDIR_HIGH		1
-#define MP_IRQDIR_LOW		3
+#define MP_IRQPOL_DEFAULT	0x0
+#define MP_IRQPOL_ACTIVE_HIGH	0x1
+#define MP_IRQPOL_RESERVED	0x2
+#define MP_IRQPOL_ACTIVE_LOW	0x3
+#define MP_IRQPOL_MASK		0x3
+
+#define MP_IRQTRIG_DEFAULT	0x0
+#define MP_IRQTRIG_EDGE		0x4
+#define MP_IRQTRIG_RESERVED	0x8
+#define MP_IRQTRIG_LEVEL	0xc
+#define MP_IRQTRIG_MASK		0xc
 
 #define MP_APIC_ALL	0xFF
 
arch/x86/kernel/apic/io_apic.c +10 -10
index 8a79634..8ad2e41 100644
@@ -800,18 +800,18 @@ static int irq_polarity(int idx)
 	/*
 	 * Determine IRQ line polarity (high active or low active):
 	 */
-	switch (mp_irqs[idx].irqflag & 0x03) {
-	case 0:
+	switch (mp_irqs[idx].irqflag & MP_IRQPOL_MASK) {
+	case MP_IRQPOL_DEFAULT:
 		/* conforms to spec, ie. bus-type dependent polarity */
 		if (test_bit(bus, mp_bus_not_pci))
 			return default_ISA_polarity(idx);
 		else
 			return default_PCI_polarity(idx);
-	case 1:
+	case MP_IRQPOL_ACTIVE_HIGH:
 		return IOAPIC_POL_HIGH;
-	case 2:
+	case MP_IRQPOL_RESERVED:
 		pr_warn("IOAPIC: Invalid polarity: 2, defaulting to low\n");
-	case 3:
+	case MP_IRQPOL_ACTIVE_LOW:
 	default: /* Pointless default required due to do gcc stupidity */
 		return IOAPIC_POL_LOW;
 	}
@@ -845,8 +845,8 @@ static int irq_trigger(int idx)
 	/*
 	 * Determine IRQ trigger mode (edge or level sensitive):
 	 */
-	switch ((mp_irqs[idx].irqflag >> 2) & 0x03) {
-	case 0:
+	switch (mp_irqs[idx].irqflag & MP_IRQTRIG_MASK) {
+	case MP_IRQTRIG_DEFAULT:
 		/* conforms to spec, ie. bus-type dependent trigger mode */
 		if (test_bit(bus, mp_bus_not_pci))
 			trigger = default_ISA_trigger(idx);
@@ -854,11 +854,11 @@ static int irq_trigger(int idx)
 			trigger = default_PCI_trigger(idx);
 		/* Take EISA into account */
 		return eisa_irq_trigger(idx, bus, trigger);
-	case 1:
+	case MP_IRQTRIG_EDGE:
 		return IOAPIC_EDGE;
-	case 2:
+	case MP_IRQTRIG_RESERVED:
 		pr_warn("IOAPIC: Invalid trigger mode 2 defaulting to level\n");
-	case 3:
+	case MP_IRQTRIG_LEVEL:
 	default: /* Pointless default required due to do gcc stupidity */
 		return IOAPIC_LEVEL;
 	}
arch/x86/kernel/mpparse.c +14 -9
index 3a4b128..27d0a17 100644
@@ -281,7 +281,7 @@ static void __init construct_default_ioirq_mptable(int mpc_default_type)
 	int ELCR_fallback = 0;
 
 	intsrc.type = MP_INTSRC;
-	intsrc.irqflag = 0;	/* conforming */
+	intsrc.irqflag = MP_IRQTRIG_DEFAULT | MP_IRQPOL_DEFAULT;
 	intsrc.srcbus = 0;
 	intsrc.dstapic = mpc_ioapic_id(0);
 
@@ -324,10 +324,13 @@ static void __init construct_default_ioirq_mptable(int mpc_default_type)
 			 *  copy that information over to the MP table in the
 			 *  irqflag field (level sensitive, active high polarity).
 			 */
-			if (ELCR_trigger(i))
-				intsrc.irqflag = 13;
-			else
-				intsrc.irqflag = 0;
+			if (ELCR_trigger(i)) {
+				intsrc.irqflag = MP_IRQTRIG_LEVEL |
+						 MP_IRQPOL_ACTIVE_HIGH;
+			} else {
+				intsrc.irqflag = MP_IRQTRIG_DEFAULT |
+						 MP_IRQPOL_DEFAULT;
+			}
 		}
 
 		intsrc.srcbusirq = i;
@@ -419,7 +422,7 @@ static inline void __init construct_default_ISA_mptable(int mpc_default_type)
 	construct_ioapic_table(mpc_default_type);
 
 	lintsrc.type = MP_LINTSRC;
-	lintsrc.irqflag = 0;		/* conforming */
+	lintsrc.irqflag = MP_IRQTRIG_DEFAULT | MP_IRQPOL_DEFAULT;
 	lintsrc.srcbusid = 0;
 	lintsrc.srcbusirq = 0;
 	lintsrc.destapic = MP_APIC_ALL;
@@ -664,7 +667,7 @@ static int  __init get_MP_intsrc_index(struct mpc_intsrc *m)
 	if (m->irqtype != mp_INT)
 		return 0;
 
-	if (m->irqflag != 0x0f)
+	if (m->irqflag != (MP_IRQTRIG_LEVEL | MP_IRQPOL_ACTIVE_LOW))
 		return 0;
 
 	/* not legacy */
@@ -673,7 +676,8 @@ static int  __init get_MP_intsrc_index(struct mpc_intsrc *m)
 		if (mp_irqs[i].irqtype != mp_INT)
 			continue;
 
-		if (mp_irqs[i].irqflag != 0x0f)
+		if (mp_irqs[i].irqflag != (MP_IRQTRIG_LEVEL |
+					   MP_IRQPOL_ACTIVE_LOW))
 			continue;
 
 		if (mp_irqs[i].srcbus != m->srcbus)
@@ -784,7 +788,8 @@ static int  __init replace_intsrc_all(struct mpc_table *mpc,
 		if (mp_irqs[i].irqtype != mp_INT)
 			continue;
 
-		if (mp_irqs[i].irqflag != 0x0f)
+		if (mp_irqs[i].irqflag != (MP_IRQTRIG_LEVEL |
+					   MP_IRQPOL_ACTIVE_LOW))
 			continue;
 
 		if (nr_m_spare > 0) {
arch/x86/platform/intel-mid/sfi.c +2 -3
index 19b43e3..7be1e1f 100644
@@ -96,8 +96,7 @@ int __init sfi_parse_mtmr(struct sfi_table_header *table)
 			pentry->freq_hz, pentry->irq);
 		mp_irq.type = MP_INTSRC;
 		mp_irq.irqtype = mp_INT;
-		/* triggering mode edge bit 2-3, active high polarity bit 0-1 */
-		mp_irq.irqflag = 5;
+		mp_irq.irqflag = MP_IRQTRIG_EDGE | MP_IRQPOL_ACTIVE_HIGH;
 		mp_irq.srcbus = MP_BUS_ISA;
 		mp_irq.srcbusirq = pentry->irq;	/* IRQ */
 		mp_irq.dstapic = MP_APIC_ALL;
@@ -168,7 +167,7 @@ int __init sfi_parse_mrtc(struct sfi_table_header *table)
 			totallen, (u32)pentry->phys_addr, pentry->irq);
 		mp_irq.type = MP_INTSRC;
 		mp_irq.irqtype = mp_INT;
-		mp_irq.irqflag = 0xf;	/* level trigger and active low */
+		mp_irq.irqflag = MP_IRQTRIG_LEVEL | MP_IRQPOL_ACTIVE_LOW;
 		mp_irq.srcbus = MP_BUS_ISA;
 		mp_irq.srcbusirq = pentry->irq;	/* IRQ */
 		mp_irq.dstapic = MP_APIC_ALL;
author maintainer
#7 lore raw

PATCH v2 4/12 x86/jailhouse: Add infrastructure for running in non-root cell

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

The Jailhouse hypervisor is able to statically partition a multicore system into multiple so-called cells. Linux is used as boot loader and continues to run in the root cell after Jailhouse is enabled. Linux can also run in non-root cells.

Jailhouse does not emulate usual x86 devices. It also provides no complex ACPI but basic platform information that the boot loader forwards via setup data. This adds the infrastructure to detect when running in a non-root cell so that the platform can be configured as required in succeeding steps.

Support is limited to x86-64 so far, primarily because no boot loader stub exists for i386 and, thus, we wouldn't be able to test the 32-bit path.

---
 arch/x86/Kconfig                      |  8 ++++
 arch/x86/include/asm/hypervisor.h     |  1 +
 arch/x86/include/asm/jailhouse_para.h | 27 +++++++++++++
 arch/x86/include/uapi/asm/bootparam.h | 22 ++++++++++
 arch/x86/kernel/Makefile              |  2 +
 arch/x86/kernel/cpu/hypervisor.c      |  4 ++
 arch/x86/kernel/jailhouse.c           | 75 +++++++++++++++++++++++++++++++++++
 7 files changed, 139 insertions(+)
 create mode 100644 arch/x86/include/asm/jailhouse_para.h
 create mode 100644 arch/x86/kernel/jailhouse.c
arch/x86/Kconfig +8 -0
index 8eed3f94bfc7..93c67ae5cfaf 100644
@@ -797,6 +797,14 @@ config PARAVIRT_TIME_ACCOUNTING
 config PARAVIRT_CLOCK
 	bool
 
+config JAILHOUSE_GUEST
+	bool "Jailhouse non-root cell support"
+	depends on X86_64
+	---help---
+	  This option allows to run Linux as guest in a Jailhouse non-root
+	  cell. You can leave this option disabled if you only want to start
+	  Jailhouse and run Linux afterwards in the root cell.
+
 endif #HYPERVISOR_GUEST
 
 config NO_BOOTMEM
arch/x86/include/asm/hypervisor.h +1 -0
index 1b0a5abcd8ae..376085cb6244 100644
@@ -37,6 +37,7 @@ enum x86_hypervisor_type {
 	X86_HYPER_XEN_PV,
 	X86_HYPER_XEN_HVM,
 	X86_HYPER_KVM,
+	X86_HYPER_JAILHOUSE,
 };
 
 struct hypervisor_x86 {
arch/x86/include/asm/jailhouse_para.h new file +27 -0
new file mode 100644
index 000000000000..06a5f41d5451
@@ -0,0 +1,27 @@
+/*
+ * Jailhouse paravirt_ops implementation
+ *
+ * Copyright (c) Siemens AG, 2015-2017
+ *
+ * Authors:
+ *  Jan Kiszka <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ */
+
+#ifndef _ASM_X86_JAILHOUSE_PARA_H
+#define _ASM_X86_JAILHOUSE_PARA_H
+
+#include <linux/types.h>
+
+#ifdef CONFIG_JAILHOUSE_GUEST
+bool jailhouse_paravirt(void);
+#else
+static inline bool jailhouse_paravirt(void)
+{
+	return false;
+}
+#endif
+
+#endif /* _ASM_X86_JAILHOUSE_PARA_H */
arch/x86/include/uapi/asm/bootparam.h +22 -0
index afdd5ae0fcc4..aebf60357758 100644
@@ -9,6 +9,7 @@
 #define SETUP_PCI			3
 #define SETUP_EFI			4
 #define SETUP_APPLE_PROPERTIES		5
+#define SETUP_JAILHOUSE			6
 
 /* ram_size flags */
 #define RAMDISK_IMAGE_START_MASK	0x07FF
@@ -126,6 +127,27 @@ struct boot_e820_entry {
 	__u32 type;
 } __attribute__((packed));
 
+/*
+ * Smallest compatible version of jailhouse_setup_data required by this kernel.
+ */
+#define JAILHOUSE_SETUP_REQUIRED_VERSION	1
+
+/*
+ * The boot loader is passing platform information via this Jailhouse-specific
+ * setup data structure.
+ */
+struct jailhouse_setup_data {
+	u16	version;
+	u16	compatible_version;
+	u16	pm_timer_address;
+	u16	num_cpus;
+	u64	pci_mmconfig_base;
+	u32	tsc_khz;
+	u32	apic_khz;
+	u8	standard_ioapic;
+	u8	cpu_ids[255];
+} __attribute__((packed));
+
 /* The so-called "zeropage" */
 struct boot_params {
 	struct screen_info screen_info;			/* 0x000 */
arch/x86/kernel/Makefile +2 -0
index 81bb565f4497..aed9296dccd3 100644
@@ -112,6 +112,8 @@ obj-$(CONFIG_PARAVIRT_SPINLOCKS)+= paravirt-spinlocks.o
 obj-$(CONFIG_PARAVIRT_CLOCK)	+= pvclock.o
 obj-$(CONFIG_X86_PMEM_LEGACY_DEVICE) += pmem.o
 
+obj-$(CONFIG_JAILHOUSE_GUEST)	+= jailhouse.o
+
 obj-$(CONFIG_EISA)		+= eisa.o
 obj-$(CONFIG_PCSPKR_PLATFORM)	+= pcspeaker.o
 
arch/x86/kernel/cpu/hypervisor.c +4 -0
index bea8d3e24f50..479ca4728de0 100644
@@ -31,6 +31,7 @@ extern const struct hypervisor_x86 x86_hyper_ms_hyperv;
 extern const struct hypervisor_x86 x86_hyper_xen_pv;
 extern const struct hypervisor_x86 x86_hyper_xen_hvm;
 extern const struct hypervisor_x86 x86_hyper_kvm;
+extern const struct hypervisor_x86 x86_hyper_jailhouse;
 
 static const __initconst struct hypervisor_x86 * const hypervisors[] =
 {
@@ -45,6 +46,9 @@ static const __initconst struct hypervisor_x86 * const hypervisors[] =
 #ifdef CONFIG_KVM_GUEST
 	&x86_hyper_kvm,
 #endif
+#ifdef CONFIG_JAILHOUSE_GUEST
+	&x86_hyper_jailhouse,
+#endif
 };
 
 enum x86_hypervisor_type x86_hyper_type;
arch/x86/kernel/jailhouse.c new file +75 -0

75 lines not on this page · raw message

Signature · 1 line
2.12.3
#8 lore raw

[tip:x86/platform] x86/jailhouse: Add infrastructure for running in non-root cell

x86/jailhouse: Add infrastructure for running in non-root cell
Commit-ID:  4a362601baa6fff92b576d85199f1948cec2fb3b
Gitweb:     https://git.kernel.org/tip/4a362601baa6fff92b576d85199f1948cec2fb3b
Author:     Jan Kiszka <[email protected]>
AuthorDate: Mon, 27 Nov 2017 09:11:46 +0100
Committer:  Thomas Gleixner <[email protected]>
CommitDate: Sun, 14 Jan 2018 21:11:54 +0100

x86/jailhouse: Add infrastructure for running in non-root cell

The Jailhouse hypervisor is able to statically partition a multicore system into multiple so-called cells. Linux is used as boot loader and continues to run in the root cell after Jailhouse is enabled. Linux can also run in non-root cells.

Jailhouse does not emulate usual x86 devices. It also provides no complex ACPI but basic platform information that the boot loader forwards via setup data. This adds the infrastructure to detect when running in a non-root cell so that the platform can be configured as required in succeeding steps.

Support is limited to x86-64 so far, primarily because no boot loader stub exists for i386 and, thus, we wouldn't be able to test the 32-bit path.

---
 arch/x86/Kconfig                      |  8 ++++
 arch/x86/include/asm/hypervisor.h     |  1 +
 arch/x86/include/asm/jailhouse_para.h | 26 +++++++++++++
 arch/x86/include/uapi/asm/bootparam.h | 22 +++++++++++
 arch/x86/kernel/Makefile              |  2 +
 arch/x86/kernel/cpu/hypervisor.c      |  4 ++
 arch/x86/kernel/jailhouse.c           | 73 +++++++++++++++++++++++++++++++++++
 7 files changed, 136 insertions(+)
arch/x86/Kconfig +8 -0
index ff4e9cd..fbea8d1 100644
@@ -796,6 +796,14 @@ config PARAVIRT_TIME_ACCOUNTING
 config PARAVIRT_CLOCK
 	bool
 
+config JAILHOUSE_GUEST
+	bool "Jailhouse non-root cell support"
+	depends on X86_64
+	---help---
+	  This option allows to run Linux as guest in a Jailhouse non-root
+	  cell. You can leave this option disabled if you only want to start
+	  Jailhouse and run Linux afterwards in the root cell.
+
 endif #HYPERVISOR_GUEST
 
 config NO_BOOTMEM
arch/x86/include/asm/hypervisor.h +1 -0
index 96aa6b9..8c5aaba 100644
@@ -28,6 +28,7 @@ enum x86_hypervisor_type {
 	X86_HYPER_XEN_PV,
 	X86_HYPER_XEN_HVM,
 	X86_HYPER_KVM,
+	X86_HYPER_JAILHOUSE,
 };
 
 #ifdef CONFIG_HYPERVISOR_GUEST
arch/x86/include/asm/jailhouse_para.h new file +26 -0
new file mode 100644
index 0000000..875b543
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL2.0 */
+
+/*
+ * Jailhouse paravirt_ops implementation
+ *
+ * Copyright (c) Siemens AG, 2015-2017
+ *
+ * Authors:
+ *  Jan Kiszka <[email protected]>
+ */
+
+#ifndef _ASM_X86_JAILHOUSE_PARA_H
+#define _ASM_X86_JAILHOUSE_PARA_H
+
+#include <linux/types.h>
+
+#ifdef CONFIG_JAILHOUSE_GUEST
+bool jailhouse_paravirt(void);
+#else
+static inline bool jailhouse_paravirt(void)
+{
+	return false;
+}
+#endif
+
+#endif /* _ASM_X86_JAILHOUSE_PARA_H */
arch/x86/include/uapi/asm/bootparam.h +22 -0
index afdd5ae..aebf603 100644
@@ -9,6 +9,7 @@
 #define SETUP_PCI			3
 #define SETUP_EFI			4
 #define SETUP_APPLE_PROPERTIES		5
+#define SETUP_JAILHOUSE			6
 
 /* ram_size flags */
 #define RAMDISK_IMAGE_START_MASK	0x07FF
@@ -126,6 +127,27 @@ struct boot_e820_entry {
 	__u32 type;
 } __attribute__((packed));
 
+/*
+ * Smallest compatible version of jailhouse_setup_data required by this kernel.
+ */
+#define JAILHOUSE_SETUP_REQUIRED_VERSION	1
+
+/*
+ * The boot loader is passing platform information via this Jailhouse-specific
+ * setup data structure.
+ */
+struct jailhouse_setup_data {
+	u16	version;
+	u16	compatible_version;
+	u16	pm_timer_address;
+	u16	num_cpus;
+	u64	pci_mmconfig_base;
+	u32	tsc_khz;
+	u32	apic_khz;
+	u8	standard_ioapic;
+	u8	cpu_ids[255];
+} __attribute__((packed));
+
 /* The so-called "zeropage" */
 struct boot_params {
 	struct screen_info screen_info;			/* 0x000 */
arch/x86/kernel/Makefile +2 -0
index 81bb565..aed9296 100644
@@ -112,6 +112,8 @@ obj-$(CONFIG_PARAVIRT_SPINLOCKS)+= paravirt-spinlocks.o
 obj-$(CONFIG_PARAVIRT_CLOCK)	+= pvclock.o
 obj-$(CONFIG_X86_PMEM_LEGACY_DEVICE) += pmem.o
 
+obj-$(CONFIG_JAILHOUSE_GUEST)	+= jailhouse.o
+
 obj-$(CONFIG_EISA)		+= eisa.o
 obj-$(CONFIG_PCSPKR_PLATFORM)	+= pcspeaker.o
 
arch/x86/kernel/cpu/hypervisor.c +4 -0
index bea8d3e..479ca47 100644
@@ -31,6 +31,7 @@ extern const struct hypervisor_x86 x86_hyper_ms_hyperv;
 extern const struct hypervisor_x86 x86_hyper_xen_pv;
 extern const struct hypervisor_x86 x86_hyper_xen_hvm;
 extern const struct hypervisor_x86 x86_hyper_kvm;
+extern const struct hypervisor_x86 x86_hyper_jailhouse;
 
 static const __initconst struct hypervisor_x86 * const hypervisors[] =
 {
@@ -45,6 +46,9 @@ static const __initconst struct hypervisor_x86 * const hypervisors[] =
 #ifdef CONFIG_KVM_GUEST
 	&x86_hyper_kvm,
 #endif
+#ifdef CONFIG_JAILHOUSE_GUEST
+	&x86_hyper_jailhouse,
+#endif
 };
 
 enum x86_hypervisor_type x86_hyper_type;
arch/x86/kernel/jailhouse.c new file +73 -0

73 lines not on this page · raw message

author maintainer
#9 lore raw

PATCH v2 5/12 x86/jailhouse: Enable APIC and SMP support

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

Register the APIC which Jailhouse always exposes at 0xfee00000 if in xAPIC mode or via MSRs as x2APIC. The latter is only available if it was already activated because there is no support for switching its mode during runtime.

Jailhouse requires the APIC to be operated in phys-flat mode. Ensure that this mode is selected by Linux.

The available CPUs are taken from the setup data structure that the loader filled and registered with the kernel.

---
 arch/x86/kernel/apic/apic_flat_64.c |  4 +++-
 arch/x86/kernel/jailhouse.c         | 42 +++++++++++++++++++++++++++++++++++++
 2 files changed, 45 insertions(+), 1 deletion(-)
arch/x86/kernel/apic/apic_flat_64.c +3 -1
index 226a6ab22b35..408bf5abad4c 100644
@@ -19,6 +19,7 @@
 #include <asm/smp.h>
 #include <asm/apic.h>
 #include <asm/ipi.h>
+#include <asm/jailhouse_para.h>
 
 #include <linux/acpi.h>
 
@@ -239,7 +240,8 @@ static void physflat_send_IPI_all(int vector)
 
 static int physflat_probe(void)
 {
-	if (apic == &apic_physflat || num_possible_cpus() > 8)
+	if (apic == &apic_physflat || num_possible_cpus() > 8 ||
+	    jailhouse_paravirt())
 		return 1;
 
 	return 0;
arch/x86/kernel/jailhouse.c +42 -0

68 lines not on this page · raw message

Signature · 1 line
2.12.3
#10 lore raw

[tip:x86/platform] x86/jailhouse: Enable APIC and SMP support

x86/jailhouse: Enable APIC and SMP support
Commit-ID:  11c8dc419bbc7b5acef812043feefc53c45ef558
Gitweb:     https://git.kernel.org/tip/11c8dc419bbc7b5acef812043feefc53c45ef558
Author:     Jan Kiszka <[email protected]>
AuthorDate: Mon, 27 Nov 2017 09:11:47 +0100
Committer:  Thomas Gleixner <[email protected]>
CommitDate: Sun, 14 Jan 2018 21:11:55 +0100

x86/jailhouse: Enable APIC and SMP support

Register the APIC which Jailhouse always exposes at 0xfee00000 if in xAPIC mode or via MSRs as x2APIC. The latter is only available if it was already activated because there is no support for switching its mode during runtime.

Jailhouse requires the APIC to be operated in phys-flat mode. Ensure that this mode is selected by Linux.

The available CPUs are taken from the setup data structure that the loader filled and registered with the kernel.

---
 arch/x86/kernel/apic/apic_flat_64.c |  4 +++-
 arch/x86/kernel/jailhouse.c         | 42 +++++++++++++++++++++++++++++++++++++
 2 files changed, 45 insertions(+), 1 deletion(-)
arch/x86/kernel/apic/apic_flat_64.c +3 -1
index 4b55477..fcce5a7 100644
@@ -19,6 +19,7 @@
 #include <asm/smp.h>
 #include <asm/apic.h>
 #include <asm/ipi.h>
+#include <asm/jailhouse_para.h>
 
 #include <linux/acpi.h>
 
@@ -239,7 +240,8 @@ static void physflat_send_IPI_all(int vector)
 
 static int physflat_probe(void)
 {
-	if (apic == &apic_physflat || num_possible_cpus() > 8)
+	if (apic == &apic_physflat || num_possible_cpus() > 8 ||
+	    jailhouse_paravirt())
 		return 1;
 
 	return 0;
arch/x86/kernel/jailhouse.c +42 -0

68 lines not on this page · raw message

author maintainer
#11 lore raw

PATCH v2 6/12 x86/jailhouse: Enable PMTIMER

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

Jailhouse exposes the PMTIMER as only reference clock to all cells. Pick up its address from the setup data. Allow to enable the Linux support of it by relaxing its strict dependency on ACPI.

---
 arch/x86/Kconfig            |  1 +
 arch/x86/kernel/jailhouse.c |  4 ++++
 drivers/acpi/Kconfig        | 32 ++++++++++++++++----------------
 3 files changed, 21 insertions(+), 16 deletions(-)
arch/x86/Kconfig +1 -0
index 93c67ae5cfaf..bbcbe21ca664 100644
@@ -800,6 +800,7 @@ config PARAVIRT_CLOCK
 config JAILHOUSE_GUEST
 	bool "Jailhouse non-root cell support"
 	depends on X86_64
+	select X86_PM_TIMER
 	---help---
 	  This option allows to run Linux as guest in a Jailhouse non-root
 	  cell. You can leave this option disabled if you only want to start
arch/x86/kernel/jailhouse.c +4 -0
index 9682e7089402..267547162a2c 100644
@@ -10,6 +10,7 @@
  * the COPYING file in the top-level directory.
  */
 
+#include <linux/acpi_pmtmr.h>
 #include <linux/kernel.h>
 #include <asm/apic.h>
 #include <asm/cpu.h>
@@ -93,6 +94,9 @@ static void __init jailhouse_init_platform(void)
 
 	if (setup_data.compatible_version > JAILHOUSE_SETUP_REQUIRED_VERSION)
 		panic("Jailhouse: Unsupported setup data structure");
+
+	pmtmr_ioport = setup_data.pm_timer_address;
+	pr_debug("Jailhouse: PM-Timer IO Port: %#x\n", pmtmr_ioport);
 }
 
 bool jailhouse_paravirt(void)
drivers/acpi/Kconfig +16 -16
index 46505396869e..d650c5b6ec90 100644
@@ -361,22 +361,6 @@ config ACPI_PCI_SLOT
 	  i.e., segment/bus/device/function tuples, with physical slots in
 	  the system.  If you are unsure, say N.
 
-config X86_PM_TIMER
-	bool "Power Management Timer Support" if EXPERT
-	depends on X86
-	default y
-	help
-	  The Power Management Timer is available on all ACPI-capable,
-	  in most cases even if ACPI is unusable or blacklisted.
-
-	  This timing source is not affected by power management features
-	  like aggressive processor idling, throttling, frequency and/or
-	  voltage scaling, unlike the commonly used Time Stamp Counter
-	  (TSC) timing source.
-
-	  You should nearly always say Y here because many modern
-	  systems require this timer. 
-
 config ACPI_CONTAINER
 	bool "Container and Module Devices"
 	default (ACPI_HOTPLUG_MEMORY || ACPI_HOTPLUG_CPU)
@@ -564,3 +548,19 @@ config TPS68470_PMIC_OPREGION
 	  using this, are probed.
 
 endif	# ACPI
+
+config X86_PM_TIMER
+	bool "Power Management Timer Support" if EXPERT
+	depends on X86 && (ACPI || JAILHOUSE_GUEST)
+	default y
+	help
+	  The Power Management Timer is available on all ACPI-capable,
+	  in most cases even if ACPI is unusable or blacklisted.
+
+	  This timing source is not affected by power management features
+	  like aggressive processor idling, throttling, frequency and/or
+	  voltage scaling, unlike the commonly used Time Stamp Counter
+	  (TSC) timing source.
+
+	  You should nearly always say Y here because many modern
+	  systems require this timer.
Signature · 1 line
2.12.3
#12 lore raw

[tip:x86/platform] x86/jailhouse: Enable PMTIMER

x86/jailhouse: Enable PMTIMER
Commit-ID:  87e65d05bb0a18e00655a58159790bc8d38e219e
Gitweb:     https://git.kernel.org/tip/87e65d05bb0a18e00655a58159790bc8d38e219e
Author:     Jan Kiszka <[email protected]>
AuthorDate: Mon, 27 Nov 2017 09:11:48 +0100
Committer:  Thomas Gleixner <[email protected]>
CommitDate: Sun, 14 Jan 2018 21:11:55 +0100

x86/jailhouse: Enable PMTIMER

Jailhouse exposes the PMTIMER as only reference clock to all cells. Pick up its address from the setup data. Allow to enable the Linux support of it by relaxing its strict dependency on ACPI.

---
 arch/x86/Kconfig            |  1 +
 arch/x86/kernel/jailhouse.c |  4 ++++
 drivers/acpi/Kconfig        | 32 ++++++++++++++++----------------
 3 files changed, 21 insertions(+), 16 deletions(-)
arch/x86/Kconfig +1 -0
index fbea8d1..a936e29 100644
@@ -799,6 +799,7 @@ config PARAVIRT_CLOCK
 config JAILHOUSE_GUEST
 	bool "Jailhouse non-root cell support"
 	depends on X86_64
+	select X86_PM_TIMER
 	---help---
 	  This option allows to run Linux as guest in a Jailhouse non-root
 	  cell. You can leave this option disabled if you only want to start
arch/x86/kernel/jailhouse.c +4 -0
index 57f4996..21c1077 100644
@@ -8,6 +8,7 @@
  *  Jan Kiszka <[email protected]>
  */
 
+#include <linux/acpi_pmtmr.h>
 #include <linux/kernel.h>
 #include <asm/apic.h>
 #include <asm/cpu.h>
@@ -91,6 +92,9 @@ static void __init jailhouse_init_platform(void)
 
 	if (setup_data.compatible_version > JAILHOUSE_SETUP_REQUIRED_VERSION)
 		panic("Jailhouse: Unsupported setup data structure");
+
+	pmtmr_ioport = setup_data.pm_timer_address;
+	pr_debug("Jailhouse: PM-Timer IO Port: %#x\n", pmtmr_ioport);
 }
 
 bool jailhouse_paravirt(void)
drivers/acpi/Kconfig +16 -16
index 4650539..d650c5b 100644
@@ -361,22 +361,6 @@ config ACPI_PCI_SLOT
 	  i.e., segment/bus/device/function tuples, with physical slots in
 	  the system.  If you are unsure, say N.
 
-config X86_PM_TIMER
-	bool "Power Management Timer Support" if EXPERT
-	depends on X86
-	default y
-	help
-	  The Power Management Timer is available on all ACPI-capable,
-	  in most cases even if ACPI is unusable or blacklisted.
-
-	  This timing source is not affected by power management features
-	  like aggressive processor idling, throttling, frequency and/or
-	  voltage scaling, unlike the commonly used Time Stamp Counter
-	  (TSC) timing source.
-
-	  You should nearly always say Y here because many modern
-	  systems require this timer. 
-
 config ACPI_CONTAINER
 	bool "Container and Module Devices"
 	default (ACPI_HOTPLUG_MEMORY || ACPI_HOTPLUG_CPU)
@@ -564,3 +548,19 @@ config TPS68470_PMIC_OPREGION
 	  using this, are probed.
 
 endif	# ACPI
+
+config X86_PM_TIMER
+	bool "Power Management Timer Support" if EXPERT
+	depends on X86 && (ACPI || JAILHOUSE_GUEST)
+	default y
+	help
+	  The Power Management Timer is available on all ACPI-capable,
+	  in most cases even if ACPI is unusable or blacklisted.
+
+	  This timing source is not affected by power management features
+	  like aggressive processor idling, throttling, frequency and/or
+	  voltage scaling, unlike the commonly used Time Stamp Counter
+	  (TSC) timing source.
+
+	  You should nearly always say Y here because many modern
+	  systems require this timer.
author maintainer
#13 lore raw

PATCH v2 7/12 x86/jailhouse: Set up timekeeping

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

Get the precalibrated frequencies for the TSC and the APIC timer from the Jailhouse platform info and set the kernel values accordingly.

---
 arch/x86/kernel/jailhouse.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
arch/x86/kernel/jailhouse.c +17 -0
index 267547162a2c..effec7d2b480 100644
@@ -18,6 +18,7 @@
 #include <asm/setup.h>
 
 static __initdata struct jailhouse_setup_data setup_data;
+static unsigned int precalibrated_tsc_khz;
 
 static uint32_t jailhouse_cpuid_base(void)
 {
@@ -33,6 +34,16 @@ static uint32_t __init jailhouse_detect(void)
 	return jailhouse_cpuid_base();
 }
 
+static void __init jailhouse_timer_init(void)
+{
+	lapic_timer_frequency = setup_data.apic_khz * (1000 / HZ);
+}
+
+static unsigned long jailhouse_get_tsc(void)
+{
+	return precalibrated_tsc_khz;
+}
+
 static void __init jailhouse_get_smp_config(unsigned int early)
 {
 	unsigned int cpu;
@@ -68,8 +79,12 @@ static void __init jailhouse_init_platform(void)
 	struct setup_data header;
 	void *mapping;
 
+	x86_init.timers.timer_init	= jailhouse_timer_init;
 	x86_init.mpparse.get_smp_config	= jailhouse_get_smp_config;
 
+	x86_platform.calibrate_cpu	= jailhouse_get_tsc;
+	x86_platform.calibrate_tsc	= jailhouse_get_tsc;
+
 	while (pa_data) {
 		mapping = early_memremap(pa_data, sizeof(header));
 		memcpy(&header, mapping, sizeof(header));
@@ -97,6 +112,8 @@ static void __init jailhouse_init_platform(void)
 
 	pmtmr_ioport = setup_data.pm_timer_address;
 	pr_debug("Jailhouse: PM-Timer IO Port: %#x\n", pmtmr_ioport);
+
+	precalibrated_tsc_khz = setup_data.tsc_khz;
 }
 
 bool jailhouse_paravirt(void)
Signature · 1 line
2.12.3
#14 lore raw

[tip:x86/platform] x86/jailhouse: Set up timekeeping

x86/jailhouse: Set up timekeeping
Commit-ID:  e85eb632f651e70252bb18b292efaf6961164e32
Gitweb:     https://git.kernel.org/tip/e85eb632f651e70252bb18b292efaf6961164e32
Author:     Jan Kiszka <[email protected]>
AuthorDate: Mon, 27 Nov 2017 09:11:49 +0100
Committer:  Thomas Gleixner <[email protected]>
CommitDate: Sun, 14 Jan 2018 21:11:56 +0100

x86/jailhouse: Set up timekeeping

Get the precalibrated frequencies for the TSC and the APIC timer from the Jailhouse platform info and set the kernel values accordingly.

---
 arch/x86/kernel/jailhouse.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
arch/x86/kernel/jailhouse.c +17 -0
index 21c1077..34cf9d3 100644
@@ -16,6 +16,7 @@
 #include <asm/setup.h>
 
 static __initdata struct jailhouse_setup_data setup_data;
+static unsigned int precalibrated_tsc_khz;
 
 static uint32_t jailhouse_cpuid_base(void)
 {
@@ -31,6 +32,16 @@ static uint32_t __init jailhouse_detect(void)
 	return jailhouse_cpuid_base();
 }
 
+static void __init jailhouse_timer_init(void)
+{
+	lapic_timer_frequency = setup_data.apic_khz * (1000 / HZ);
+}
+
+static unsigned long jailhouse_get_tsc(void)
+{
+	return precalibrated_tsc_khz;
+}
+
 static void __init jailhouse_get_smp_config(unsigned int early)
 {
 	unsigned int cpu;
@@ -66,8 +77,12 @@ static void __init jailhouse_init_platform(void)
 	struct setup_data header;
 	void *mapping;
 
+	x86_init.timers.timer_init	= jailhouse_timer_init;
 	x86_init.mpparse.get_smp_config	= jailhouse_get_smp_config;
 
+	x86_platform.calibrate_cpu	= jailhouse_get_tsc;
+	x86_platform.calibrate_tsc	= jailhouse_get_tsc;
+
 	while (pa_data) {
 		mapping = early_memremap(pa_data, sizeof(header));
 		memcpy(&header, mapping, sizeof(header));
@@ -95,6 +110,8 @@ static void __init jailhouse_init_platform(void)
 
 	pmtmr_ioport = setup_data.pm_timer_address;
 	pr_debug("Jailhouse: PM-Timer IO Port: %#x\n", pmtmr_ioport);
+
+	precalibrated_tsc_khz = setup_data.tsc_khz;
 }
 
 bool jailhouse_paravirt(void)
author maintainer
#15 lore raw

PATCH v2 8/12 x86/jailhouse: Avoid access of unsupported platform resources

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

Non-root cells do not have CMOS access, thus the warm reset cannot be enabled. There is no RTC, thus also no wall clock. Furthermore, there are no ISA IRQs and no PIC.

Also disable probing of i8042 devices that are typically blocked for non-root cells. In theory, access could also be granted to a non-root cell, provided the root cell is not using the devices. But there is no concrete scenario in sight, and disabling probing over Jailhouse allows to build generic kernels that keep CONFIG_SERIO enabled for use in normal systems.

---
 arch/x86/kernel/jailhouse.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
arch/x86/kernel/jailhouse.c +13 -0
index effec7d2b480..f5f459bb7aac 100644
@@ -15,6 +15,7 @@
 #include <asm/apic.h>
 #include <asm/cpu.h>
 #include <asm/hypervisor.h>
+#include <asm/i8259.h>
 #include <asm/setup.h>
 
 static __initdata struct jailhouse_setup_data setup_data;
@@ -34,6 +35,11 @@ static uint32_t __init jailhouse_detect(void)
 	return jailhouse_cpuid_base();
 }
 
+static void jailhouse_get_wallclock(struct timespec *now)
+{
+	memset(now, 0, sizeof(*now));
+}
+
 static void __init jailhouse_timer_init(void)
 {
 	lapic_timer_frequency = setup_data.apic_khz * (1000 / HZ);
@@ -79,11 +85,18 @@ static void __init jailhouse_init_platform(void)
 	struct setup_data header;
 	void *mapping;
 
+	x86_init.irqs.pre_vector_init	= x86_init_noop;
 	x86_init.timers.timer_init	= jailhouse_timer_init;
 	x86_init.mpparse.get_smp_config	= jailhouse_get_smp_config;
 
 	x86_platform.calibrate_cpu	= jailhouse_get_tsc;
 	x86_platform.calibrate_tsc	= jailhouse_get_tsc;
+	x86_platform.get_wallclock	= jailhouse_get_wallclock;
+	x86_platform.legacy.rtc		= 0;
+	x86_platform.legacy.warm_reset	= 0;
+	x86_platform.legacy.i8042	= X86_LEGACY_I8042_PLATFORM_ABSENT;
+
+	legacy_pic			= &null_legacy_pic;
 
 	while (pa_data) {
 		mapping = early_memremap(pa_data, sizeof(header));
Signature · 1 line
2.12.3
#16 lore raw

[tip:x86/platform] x86/jailhouse: Avoid access of unsupported platform resources

x86/jailhouse: Avoid access of unsupported platform resources
Commit-ID:  0d7c1e22183b9ddaa0b3bf30ece6577741bc13b3
Gitweb:     https://git.kernel.org/tip/0d7c1e22183b9ddaa0b3bf30ece6577741bc13b3
Author:     Jan Kiszka <[email protected]>
AuthorDate: Mon, 27 Nov 2017 09:11:50 +0100
Committer:  Thomas Gleixner <[email protected]>
CommitDate: Sun, 14 Jan 2018 21:11:56 +0100

x86/jailhouse: Avoid access of unsupported platform resources

Non-root cells do not have CMOS access, thus the warm reset cannot be enabled. There is no RTC, thus also no wall clock. Furthermore, there are no ISA IRQs and no PIC.

Also disable probing of i8042 devices that are typically blocked for non-root cells. In theory, access could also be granted to a non-root cell, provided the root cell is not using the devices. But there is no concrete scenario in sight, and disabling probing over Jailhouse allows to build generic kernels that keep CONFIG_SERIO enabled for use in normal systems.

---
 arch/x86/kernel/jailhouse.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
arch/x86/kernel/jailhouse.c +13 -0
index 34cf9d3..b9f116d 100644
@@ -13,6 +13,7 @@
 #include <asm/apic.h>
 #include <asm/cpu.h>
 #include <asm/hypervisor.h>
+#include <asm/i8259.h>
 #include <asm/setup.h>
 
 static __initdata struct jailhouse_setup_data setup_data;
@@ -32,6 +33,11 @@ static uint32_t __init jailhouse_detect(void)
 	return jailhouse_cpuid_base();
 }
 
+static void jailhouse_get_wallclock(struct timespec *now)
+{
+	memset(now, 0, sizeof(*now));
+}
+
 static void __init jailhouse_timer_init(void)
 {
 	lapic_timer_frequency = setup_data.apic_khz * (1000 / HZ);
@@ -77,11 +83,18 @@ static void __init jailhouse_init_platform(void)
 	struct setup_data header;
 	void *mapping;
 
+	x86_init.irqs.pre_vector_init	= x86_init_noop;
 	x86_init.timers.timer_init	= jailhouse_timer_init;
 	x86_init.mpparse.get_smp_config	= jailhouse_get_smp_config;
 
 	x86_platform.calibrate_cpu	= jailhouse_get_tsc;
 	x86_platform.calibrate_tsc	= jailhouse_get_tsc;
+	x86_platform.get_wallclock	= jailhouse_get_wallclock;
+	x86_platform.legacy.rtc		= 0;
+	x86_platform.legacy.warm_reset	= 0;
+	x86_platform.legacy.i8042	= X86_LEGACY_I8042_PLATFORM_ABSENT;
+
+	legacy_pic			= &null_legacy_pic;
 
 	while (pa_data) {
 		mapping = early_memremap(pa_data, sizeof(header));
author maintainer
#17 lore raw

PATCH v2 9/12 x86/jailhouse: Silence ACPI warning

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

Jailhouse support does not depend on ACPI, and does not even use it. But if it should be enabled, avoid warning about its absence in the platform.

---
 arch/x86/kernel/jailhouse.c | 6 ++++++
 1 file changed, 6 insertions(+)
arch/x86/kernel/jailhouse.c +6 -0
index f5f459bb7aac..36feb0e4b4fe 100644
@@ -127,6 +127,12 @@ static void __init jailhouse_init_platform(void)
 	pr_debug("Jailhouse: PM-Timer IO Port: %#x\n", pmtmr_ioport);
 
 	precalibrated_tsc_khz = setup_data.tsc_khz;
+
+	/*
+	 * Avoid that the kernel complains about missing ACPI tables - there
+	 * are none in a non-root cell.
+	 */
+	disable_acpi();
 }
 
 bool jailhouse_paravirt(void)
Signature · 1 line
2.12.3
#18 lore raw

[tip:x86/platform] x86/jailhouse: Silence ACPI warning

x86/jailhouse: Silence ACPI warning
Commit-ID:  5ae4443010b83cce3d55ce5259870e542a7c9551
Gitweb:     https://git.kernel.org/tip/5ae4443010b83cce3d55ce5259870e542a7c9551
Author:     Jan Kiszka <[email protected]>
AuthorDate: Mon, 27 Nov 2017 09:11:51 +0100
Committer:  Thomas Gleixner <[email protected]>
CommitDate: Sun, 14 Jan 2018 21:11:56 +0100

x86/jailhouse: Silence ACPI warning

Jailhouse support does not depend on ACPI, and does not even use it. But if it should be enabled, avoid warning about its absence in the platform.

---
 arch/x86/kernel/jailhouse.c | 6 ++++++
 1 file changed, 6 insertions(+)
arch/x86/kernel/jailhouse.c +6 -0
index b9f116d..54469ef 100644
@@ -125,6 +125,12 @@ static void __init jailhouse_init_platform(void)
 	pr_debug("Jailhouse: PM-Timer IO Port: %#x\n", pmtmr_ioport);
 
 	precalibrated_tsc_khz = setup_data.tsc_khz;
+
+	/*
+	 * Avoid that the kernel complains about missing ACPI tables - there
+	 * are none in a non-root cell.
+	 */
+	disable_acpi();
 }
 
 bool jailhouse_paravirt(void)
author maintainer
#19 lore raw

PATCH v2 10/12 x86/jailhouse: Halt instead of failing to restart

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

Jailhouse provides no guest-initiated restart. So, do not even try to.

---
 arch/x86/kernel/jailhouse.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
arch/x86/kernel/jailhouse.c +10 -0
index 36feb0e4b4fe..c8ee638b6bce 100644
@@ -12,10 +12,12 @@
 
 #include <linux/acpi_pmtmr.h>
 #include <linux/kernel.h>
+#include <linux/reboot.h>
 #include <asm/apic.h>
 #include <asm/cpu.h>
 #include <asm/hypervisor.h>
 #include <asm/i8259.h>
+#include <asm/reboot.h>
 #include <asm/setup.h>
 
 static __initdata struct jailhouse_setup_data setup_data;
@@ -79,6 +81,12 @@ static void __init jailhouse_get_smp_config(unsigned int early)
 	smp_found_config = 1;
 }
 
+static void jailhouse_no_restart(void)
+{
+	pr_notice("Jailhouse: Restart not supported, halting\n");
+	machine_halt();
+}
+
 static void __init jailhouse_init_platform(void)
 {
 	u64 pa_data = boot_params.hdr.setup_data;
@@ -98,6 +106,8 @@ static void __init jailhouse_init_platform(void)
 
 	legacy_pic			= &null_legacy_pic;
 
+	machine_ops.emergency_restart	= jailhouse_no_restart;
+
 	while (pa_data) {
 		mapping = early_memremap(pa_data, sizeof(header));
 		memcpy(&header, mapping, sizeof(header));
Signature · 1 line
2.12.3
#20 lore raw

[tip:x86/platform] x86/jailhouse: Halt instead of failing to restart

x86/jailhouse: Halt instead of failing to restart
Commit-ID:  fd498076821739db38babe72602f7c227587cbb5
Gitweb:     https://git.kernel.org/tip/fd498076821739db38babe72602f7c227587cbb5
Author:     Jan Kiszka <[email protected]>
AuthorDate: Mon, 27 Nov 2017 09:11:52 +0100
Committer:  Thomas Gleixner <[email protected]>
CommitDate: Sun, 14 Jan 2018 21:11:57 +0100

x86/jailhouse: Halt instead of failing to restart

Jailhouse provides no guest-initiated restart. So, do not even try to.

---
 arch/x86/kernel/jailhouse.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
arch/x86/kernel/jailhouse.c +10 -0
index 54469ef..2b55672 100644
@@ -10,10 +10,12 @@
 
 #include <linux/acpi_pmtmr.h>
 #include <linux/kernel.h>
+#include <linux/reboot.h>
 #include <asm/apic.h>
 #include <asm/cpu.h>
 #include <asm/hypervisor.h>
 #include <asm/i8259.h>
+#include <asm/reboot.h>
 #include <asm/setup.h>
 
 static __initdata struct jailhouse_setup_data setup_data;
@@ -77,6 +79,12 @@ static void __init jailhouse_get_smp_config(unsigned int early)
 	smp_found_config = 1;
 }
 
+static void jailhouse_no_restart(void)
+{
+	pr_notice("Jailhouse: Restart not supported, halting\n");
+	machine_halt();
+}
+
 static void __init jailhouse_init_platform(void)
 {
 	u64 pa_data = boot_params.hdr.setup_data;
@@ -96,6 +104,8 @@ static void __init jailhouse_init_platform(void)
 
 	legacy_pic			= &null_legacy_pic;
 
+	machine_ops.emergency_restart	= jailhouse_no_restart;
+
 	while (pa_data) {
 		mapping = early_memremap(pa_data, sizeof(header));
 		memcpy(&header, mapping, sizeof(header));
author maintainer
#21 lore raw

PATCH v2 11/12 x86/jailhouse: Wire up IOAPIC for legacy UART ports

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

The typical I/O interrupts in non-root cells are MSI-based. However, the platform UARTs do not support MSI. In order to run a non-root cell that shall be use one of them, we need to register the standard IOAPIC and set 1:1 routing for IRQ 3 and 4.

If an IOAPIC is not available, the boot loader clears standard_ioapic in the setup data, and we skip the registration. If we should not be allowed to use one of those pins, Jailhouse will simply ignore our accesses.

---
 arch/x86/kernel/jailhouse.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
arch/x86/kernel/jailhouse.c +21 -0
index c8ee638b6bce..8ff21e1534de 100644
@@ -17,6 +17,7 @@
 #include <asm/cpu.h>
 #include <asm/hypervisor.h>
 #include <asm/i8259.h>
+#include <asm/irqdomain.h>
 #include <asm/reboot.h>
 #include <asm/setup.h>
 
@@ -54,6 +55,15 @@ static unsigned long jailhouse_get_tsc(void)
 
 static void __init jailhouse_get_smp_config(unsigned int early)
 {
+	struct ioapic_domain_cfg ioapic_cfg = {
+		.type = IOAPIC_DOMAIN_STRICT,
+		.ops = &mp_ioapic_irqdomain_ops,
+	};
+	struct mpc_intsrc mp_irq = {
+		.type = MP_INTSRC,
+		.irqtype = mp_INT,
+		.irqflag = MP_IRQPOL_ACTIVE_HIGH | MP_IRQTRIG_EDGE,
+	};
 	unsigned int cpu;
 
 	if (x2apic_enabled()) {
@@ -79,6 +89,17 @@ static void __init jailhouse_get_smp_config(unsigned int early)
 	}
 
 	smp_found_config = 1;
+
+	if (setup_data.standard_ioapic) {
+		mp_register_ioapic(0, 0xfec00000, gsi_top, &ioapic_cfg);
+
+		/* Register 1:1 mapping for legacy UART IRQs 3 and 4 */
+		mp_irq.srcbusirq = mp_irq.dstirq = 3;
+		mp_save_irq(&mp_irq);
+
+		mp_irq.srcbusirq = mp_irq.dstirq = 4;
+		mp_save_irq(&mp_irq);
+	}
 }
 
 static void jailhouse_no_restart(void)
Signature · 1 line
2.12.3
#22 lore raw

[tip:x86/platform] x86/jailhouse: Wire up IOAPIC for legacy UART ports

x86/jailhouse: Wire up IOAPIC for legacy UART ports
Commit-ID:  cf878e169d37b596de41322291523951540984c1
Gitweb:     https://git.kernel.org/tip/cf878e169d37b596de41322291523951540984c1
Author:     Jan Kiszka <[email protected]>
AuthorDate: Mon, 27 Nov 2017 09:11:53 +0100
Committer:  Thomas Gleixner <[email protected]>
CommitDate: Sun, 14 Jan 2018 21:11:57 +0100

x86/jailhouse: Wire up IOAPIC for legacy UART ports

The typical I/O interrupts in non-root cells are MSI-based. However, the platform UARTs do not support MSI. In order to run a non-root cell that shall use one of them, the standard IOAPIC must be registered and 1:1 routing for IRQ 3 and 4 set up.

If an IOAPIC is not available, the boot loader clears standard_ioapic in the setup data, so registration is skipped. If the guest is not allowed to to use one of those pins, Jailhouse will simply ignore the access.

---
 arch/x86/kernel/jailhouse.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
arch/x86/kernel/jailhouse.c +21 -0
index 2b55672..01d5b06 100644
@@ -15,6 +15,7 @@
 #include <asm/cpu.h>
 #include <asm/hypervisor.h>
 #include <asm/i8259.h>
+#include <asm/irqdomain.h>
 #include <asm/reboot.h>
 #include <asm/setup.h>
 
@@ -52,6 +53,15 @@ static unsigned long jailhouse_get_tsc(void)
 
 static void __init jailhouse_get_smp_config(unsigned int early)
 {
+	struct ioapic_domain_cfg ioapic_cfg = {
+		.type = IOAPIC_DOMAIN_STRICT,
+		.ops = &mp_ioapic_irqdomain_ops,
+	};
+	struct mpc_intsrc mp_irq = {
+		.type = MP_INTSRC,
+		.irqtype = mp_INT,
+		.irqflag = MP_IRQPOL_ACTIVE_HIGH | MP_IRQTRIG_EDGE,
+	};
 	unsigned int cpu;
 
 	if (x2apic_enabled()) {
@@ -77,6 +87,17 @@ static void __init jailhouse_get_smp_config(unsigned int early)
 	}
 
 	smp_found_config = 1;
+
+	if (setup_data.standard_ioapic) {
+		mp_register_ioapic(0, 0xfec00000, gsi_top, &ioapic_cfg);
+
+		/* Register 1:1 mapping for legacy UART IRQs 3 and 4 */
+		mp_irq.srcbusirq = mp_irq.dstirq = 3;
+		mp_save_irq(&mp_irq);
+
+		mp_irq.srcbusirq = mp_irq.dstirq = 4;
+		mp_save_irq(&mp_irq);
+	}
 }
 
 static void jailhouse_no_restart(void)
author maintainer
#23 lore raw

PATCH v2 12/12 x86/jailhouse: Initialize PCI support

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

With this change, PCI devices can be detected and used inside a non-root cell.

---
 arch/x86/kernel/jailhouse.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
arch/x86/kernel/jailhouse.c +17 -0
index 8ff21e1534de..70b857d4b1f5 100644
@@ -18,6 +18,7 @@
 #include <asm/hypervisor.h>
 #include <asm/i8259.h>
 #include <asm/irqdomain.h>
+#include <asm/pci_x86.h>
 #include <asm/reboot.h>
 #include <asm/setup.h>
 
@@ -108,6 +109,19 @@ static void jailhouse_no_restart(void)
 	machine_halt();
 }
 
+static int __init jailhouse_pci_arch_init(void)
+{
+	pci_direct_init(1);
+
+	/*
+	 * There are no bridges on the virtual PCI root bus under Jailhouse,
+	 * thus no other way to discover all devices than a full scan.
+	 */
+	pcibios_last_bus = 0xff;
+
+	return 0;
+}
+
 static void __init jailhouse_init_platform(void)
 {
 	u64 pa_data = boot_params.hdr.setup_data;
@@ -117,6 +131,7 @@ static void __init jailhouse_init_platform(void)
 	x86_init.irqs.pre_vector_init	= x86_init_noop;
 	x86_init.timers.timer_init	= jailhouse_timer_init;
 	x86_init.mpparse.get_smp_config	= jailhouse_get_smp_config;
+	x86_init.pci.arch_init		= jailhouse_pci_arch_init;
 
 	x86_platform.calibrate_cpu	= jailhouse_get_tsc;
 	x86_platform.calibrate_tsc	= jailhouse_get_tsc;
@@ -159,6 +174,8 @@ static void __init jailhouse_init_platform(void)
 
 	precalibrated_tsc_khz = setup_data.tsc_khz;
 
+	pci_probe = 0;
+
 	/*
 	 * Avoid that the kernel complains about missing ACPI tables - there
 	 * are none in a non-root cell.
Signature · 1 line
2.12.3
#24 lore raw

[tip:x86/platform] x86/jailhouse: Initialize PCI support

x86/jailhouse: Initialize PCI support
Commit-ID:  a0c01e4bb92d085462c293091a521cb9e7000371
Gitweb:     https://git.kernel.org/tip/a0c01e4bb92d085462c293091a521cb9e7000371
Author:     Jan Kiszka <[email protected]>
AuthorDate: Mon, 27 Nov 2017 09:11:54 +0100
Committer:  Thomas Gleixner <[email protected]>
CommitDate: Sun, 14 Jan 2018 21:11:57 +0100

x86/jailhouse: Initialize PCI support

With this change, PCI devices can be detected and used inside a non-root cell.

---
 arch/x86/kernel/jailhouse.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
arch/x86/kernel/jailhouse.c +17 -0
index 01d5b06..d6d5976 100644
@@ -16,6 +16,7 @@
 #include <asm/hypervisor.h>
 #include <asm/i8259.h>
 #include <asm/irqdomain.h>
+#include <asm/pci_x86.h>
 #include <asm/reboot.h>
 #include <asm/setup.h>
 
@@ -106,6 +107,19 @@ static void jailhouse_no_restart(void)
 	machine_halt();
 }
 
+static int __init jailhouse_pci_arch_init(void)
+{
+	pci_direct_init(1);
+
+	/*
+	 * There are no bridges on the virtual PCI root bus under Jailhouse,
+	 * thus no other way to discover all devices than a full scan.
+	 */
+	pcibios_last_bus = 0xff;
+
+	return 0;
+}
+
 static void __init jailhouse_init_platform(void)
 {
 	u64 pa_data = boot_params.hdr.setup_data;
@@ -115,6 +129,7 @@ static void __init jailhouse_init_platform(void)
 	x86_init.irqs.pre_vector_init	= x86_init_noop;
 	x86_init.timers.timer_init	= jailhouse_timer_init;
 	x86_init.mpparse.get_smp_config	= jailhouse_get_smp_config;
+	x86_init.pci.arch_init		= jailhouse_pci_arch_init;
 
 	x86_platform.calibrate_cpu	= jailhouse_get_tsc;
 	x86_platform.calibrate_tsc	= jailhouse_get_tsc;
@@ -157,6 +172,8 @@ static void __init jailhouse_init_platform(void)
 
 	precalibrated_tsc_khz = setup_data.tsc_khz;
 
+	pci_probe = 0;
+
 	/*
 	 * Avoid that the kernel complains about missing ACPI tables - there
 	 * are none in a non-root cell.
#25 lore raw

Re: [PATCH v2 12/12] x86/jailhouse: Initialize PCI support

In reply to Jan Kiszka
Quoted 25 lines · Jan Kiszka

From: Jan Kiszka <[email protected]>

With this change, PCI devices can be detected and used inside a non-root cell.

---
 arch/x86/kernel/jailhouse.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
arch/x86/kernel/jailhouse.c +1 -0
index 8ff21e1534de..70b857d4b1f5 100644
@@ -18,6 +18,7 @@
 #include <asm/hypervisor.h>
 #include <asm/i8259.h>
 #include <asm/irqdomain.h>
+#include <asm/pci_x86.h>
 #include <asm/reboot.h>
 #include <asm/setup.h>
 
@@ -108,6 +109,19 @@ static void jailhouse_no_restart(void)
 	machine_halt();
 }

arch/x86/kernel/jailhouse.c static void jailhouse_no_restart(void) · line 120

Quoted 10 lines
arch/x86/kernel/jailhouse.c +9 -0
 
+static int __init jailhouse_pci_arch_init(void)
+{
+	pci_direct_init(1);
+
+	/*
+	 * There are no bridges on the virtual PCI root bus under Jailhouse,
+	 * thus no other way to discover all devices than a full scan.
+	 */
+	pcibios_last_bus = 0xff;

Can you help me understand the comment here? If the virtual root bus is bus 00, are you saying the guest might see devices on bus 00 and bus 01, with no bus 00 bridge that leads to bus 01?

I suspect you mean something different because you say elsewhere that ARM "just works" because DT provides more configurability. But even on ARM with DT, we probe the root bus and only probe other buses when we find bridges leading to them.

So I suspect the purpose of this may be to discover devices that are below host bridges not exposed by ACPI. For example, my BIOS may expose one host bridge:

ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-7e])

but the chipset may implement devices on bus 7f even though the BIOS did not advertise the host bridge leading to that bus. This is a case of a missing host bridge, not a missing bridge on the root bus.

Can you show an example "lspci -v" output to make this concrete?

Quoted 26 lines
arch/x86/kernel/jailhouse.c +6 -0
+	return 0;
+}
+
 static void __init jailhouse_init_platform(void)
 {
 	u64 pa_data = boot_params.hdr.setup_data;
@@ -117,6 +131,7 @@ static void __init jailhouse_init_platform(void)
 	x86_init.irqs.pre_vector_init	= x86_init_noop;
 	x86_init.timers.timer_init	= jailhouse_timer_init;
 	x86_init.mpparse.get_smp_config	= jailhouse_get_smp_config;
+	x86_init.pci.arch_init		= jailhouse_pci_arch_init;
 
 	x86_platform.calibrate_cpu	= jailhouse_get_tsc;
 	x86_platform.calibrate_tsc	= jailhouse_get_tsc;
@@ -159,6 +174,8 @@ static void __init jailhouse_init_platform(void)
 
 	precalibrated_tsc_khz = setup_data.tsc_khz;
 
+	pci_probe = 0;
+
 	/*
 	 * Avoid that the kernel complains about missing ACPI tables - there
 	 * are none in a non-root cell.
Signature · 1 line
2.12.3
author maintainer
#26 lore raw

Re: [PATCH v2 12/12] x86/jailhouse: Initialize PCI support

Quoted 27 lines · Bjorn Helgaas
Quoted 26 lines · Jan Kiszka

From: Jan Kiszka <[email protected]>

With this change, PCI devices can be detected and used inside a non-root cell.

---
 arch/x86/kernel/jailhouse.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
arch/x86/kernel/jailhouse.c +1 -0
index 8ff21e1534de..70b857d4b1f5 100644
@@ -18,6 +18,7 @@
 #include <asm/hypervisor.h>
 #include <asm/i8259.h>
 #include <asm/irqdomain.h>
+#include <asm/pci_x86.h>
 #include <asm/reboot.h>
 #include <asm/setup.h>
 
@@ -108,6 +109,19 @@ static void jailhouse_no_restart(void)
 	machine_halt();
 }
 
Quoted 13 lines

arch/x86/kernel/jailhouse.c static void jailhouse_no_restart(void) · line 120

Quoted 9 lines
arch/x86/kernel/jailhouse.c +9 -0
+static int __init jailhouse_pci_arch_init(void)
+{
+	pci_direct_init(1);
+
+	/*
+	 * There are no bridges on the virtual PCI root bus under Jailhouse,
+	 * thus no other way to discover all devices than a full scan.
+	 */
+	pcibios_last_bus = 0xff;

Can you help me understand the comment here? If the virtual root bus is bus 00, are you saying the guest might see devices on bus 00 and bus 01, with no bus 00 bridge that leads to bus 01?

Exactly, and there is even no root bridge for bus 0 as well. We just hand out the devices. We try hard to keep it that simple in order to avoid emulation bloat in the hypervisor (which < 9K LoC on Intel right now).

Quoted 5 lines

I suspect you mean something different because you say elsewhere that ARM "just works" because DT provides more configurability. But even on ARM with DT, we probe the root bus and only probe other buses when we find bridges leading to them.

This statement about ARM probably does not apply to incomplete PCI hierarchy. I guess we cannot fill that gap with device tree descriptions, but I also didn't run into that scenario yet as most ARM systems we tested do not even allow to partition PCI buses (many IOMMU implementations on SoCs, where available at all, consider the root complex as single device).

Quoted 5 lines

So I suspect the purpose of this may be to discover devices that are below host bridges not exposed by ACPI. For example, my BIOS may expose one host bridge:

Quoted 7 lines

ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-7e])

but the chipset may implement devices on bus 7f even though the BIOS did not advertise the host bridge leading to that bus. This is a case of a missing host bridge, not a missing bridge on the root bus.

Can you show an example "lspci -v" output to make this concrete?

We we go, using a QEMU/KVM setup
(... -device pci-bridge,chassis_nr=1,id=b1
-device e1000e,addr=1.0,netdev=net,bus=b1):

# lspci -v
00:0e.0 Unassigned class [ff01]: Red Hat, Inc Inter-VM shared memory
        Subsystem: Red Hat, Inc Inter-VM shared memory
        Flags: bus master, fast devsel, latency 0
        Memory at 100000000 (64-bit, non-prefetchable) [size=256]
        Memory at 100000100 (64-bit, non-prefetchable) [size=32]
        Capabilities: [50] MSI-X: Enable+ Count=1 Masked-
        Kernel driver in use: ivshmem-net

01:01.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
Subsystem: Intel Corporation Device 0000
Flags: bus master, fast devsel, latency 0, IRQ 11 Memory at fe840000 (32-bit, non-prefetchable) [size=128K] Memory at fe860000 (32-bit, non-prefetchable) [size=128K] I/O ports at c000 [disabled] [size=32]
Memory at fe880000 (32-bit, non-prefetchable) [size=16K]
Capabilities: [c8] Power Management version 2
Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [e0] Express Endpoint, MSI 00
Capabilities: [a0] MSI-X: Enable+ Count=5 Masked-
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Device Serial Number 52-54-00-ff-ff-12-34-56 Kernel driver in use: e1000e

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux