Skip to contentSkip to conversation

x86_64 UEFI and AMD SEV/SEV-ES support

PATCH In review Zixuan Wang 6 participants 40 messages Oct 4, 2021 – Nov 29, 2021

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v3 cover x86_64 UEFI and AMD SEV/SEV-ES support

Hello,

This patch series updates the x86_64 KVM-Unit-Tests to run under UEFI and culminates in enabling AMD SEV/SEV-ES. The patches are organized as four parts.

The first part (patch 1) refactors the current Multiboot start-up code by converting assembly data structures into C. This enables the follow-up UEFI patches to reuse these data structures without redefining or duplicating them in assembly.

The second part (patches 2-3) copies code from Varad's patch set [1] that builds EFI stubs without depending on GNU-EFI. Part 3 and 4 are built on top of this part.

The third part (patches 4-11) enables the x86_64 test cases to run under UEFI. In particular, these patches allow the x86_64 test cases to be built as EFI executables and take full control of the guest VM. The efi_main() function sets up the KVM-Unit-Tests framework to run under UEFI and then launches the test cases' main() functions. To date, we have 38/43 test cases running with UEFI using this approach.

The fourth part of the series (patches 12-17) focuses on SEV. In particular, these patches introduce SEV/SEV-ES set up code into the EFI set up process, including checking if SEV is supported, setting c-bits for page table entries, and (notably) reusing the UEFI #VC handler so that the set up process does not need to re-implement it (a test case can always implement a new #VC handler and load it after set up is finished). Using this approach, we are able to launch the x86_64 test cases under SEV-ES and exercise KVM's VMGEXIT handler.

Note, a previous feedback [3] indicated that long-term we'd like to instrument KVM-Unit-Tests with it's own #VC handler. However, we still believe that the current approach is good as an intermediate solution, because it unlocks a lot of testing and we do not expect that testing to be inherently tied to the UEFI's #VC handler. Rather, test cases should be tied to the underlying GHCB spec implemented by an arbitrary #VC handler.

See the Part 1 to Part 4 summaries, below, for a high-level breakdown of how the patches are organized.

Part 1 Summary:
Commit 1 refactors boot-related data structures from assembly to C.

Part 2 Summary:
Commits 2-3 copy code from Varad's patch set [1] that implements EFI-related helper functions to replace the GNU-EFI library.

Part 3 Summary:
Commits 4-5 introduce support to build test cases with EFI support.

Commits 6-10 set up KVM-Unit-Tests to run under UEFI. In doing so, these patches incrementally enable most existing x86_64 test cases to run under UEFI.

Commit 11 fixes several test cases that fail to compile with EFI due to UEFI's position independent code (PIC) requirement.

Part 4 Summary:
Commits 12-13 introduce support for SEV by adding code to set the SEV c-bit in page table entries.

Commits 14-16 introduce support for SEV-ES by reusing the UEFI #VC handler in KVM-Unit-Tests. They also fix GDT and IDT issues that occur when reusing UEFI functions in KVM-Unit-Tests.

Commit 17 adds additional test cases for SEV-ES.

Changes V2 -> V3:

V3 Patch #  Changes
----------  -------
     01/17  (New patch) refactors assembly data structures in C
     02/17  Adds a missing alignment attribute

Renames the file uefi.h to efi.h
03/17 Adds an SPDX header, fixes a comment style issue 06/17 Removes assembly data structure definitions 07/17 Removes assembly data structure definitions 12/17 Simplifies an if condition code
14/17 Simplifies an if condition code
15/17 Removes GDT copying for SEV-ES #VC handler

Notes on page table set up code:
Paolo suggested unifying the page table definitions in cstart64.S and UEFI start-up code [5]. We tried but found it hard to implement due to the real/long mode issue: a page table set up function written in C is by default compiled to run in long mode. However, cstart64.S requires page table setup before entering long mode. Calling a long mode function from real/protected mode crashes the guest VM. Thus we chose not to implement this feature in this patch set. More details can be found in our off-list GitHub review [6].

Changes V1 -> V2:
1. Merge Varad's patches [1] as the foundation of our V2 patch set [4].
2. Remove AMD SEV/SEV-ES config flags and macros (patches 11-17)
3. Drop one commit 'x86 UEFI: Move setjmp.h out of desc.h' because we do not link GNU-EFI library.

Notes on authorships and attributions:
The first two commits are from Varad's patch set [1], so they are tagged as 'From:' and 'Signed-off-by:' Varad. Commits 3-7 are from our V1 patch set [2], and since Varad implemented similar code [1], these commits are tagged as 'Co-developed-by:' and 'Signed-off-by:' Varad.

Notes on patch sets merging strategy:
We understand that the current merging strategy (reorganizing and squeezing Varad's patches into two) reduces Varad's authorships, and we hope the additional attribution tags make up for it. We see another approach which is to build our patch set on top of Varad's original patch set, but this creates some noise in the final patch set, e.g., x86/cstart64.S is modified in Varad's part and later reverted in our part as we implement start up code in C. For the sake of the clarity of the code history, we believe the current approach is the best effort so far, and we are open to all kinds of opinions.

[1] https://lore.kernel.org/kvm/[email protected]/
[2] https://lore.kernel.org/kvm/[email protected]/
[3] https://lore.kernel.org/kvm/YSA%[email protected]/
[4] https://lore.kernel.org/kvm/[email protected]/
[5] https://lore.kernel.org/kvm/[email protected]/
[6] https://github.com/marc-orr/KVM-Unit-Tests-dev-fork/pull/1

Regards,
Zixuan Wang

Varad Gautam (2):
x86 UEFI: Copy code from Linux
x86 UEFI: Implement UEFI function calls

Zixuan Wang (15):
x86: Move IDT, GDT and TSS to desc.c
x86 UEFI: Copy code from GNU-EFI
x86 UEFI: Boot from UEFI
x86 UEFI: Load IDT after UEFI boot up
x86 UEFI: Load GDT and TSS after UEFI boot up
x86 UEFI: Set up memory allocator
x86 UEFI: Set up RSDP after UEFI boot up
x86 UEFI: Set up page tables
x86 UEFI: Convert x86 test cases to PIC
x86 AMD SEV: Initial support
x86 AMD SEV: Page table with c-bit
x86 AMD SEV-ES: Check SEV-ES status
x86 AMD SEV-ES: Copy UEFI #VC IDT entry
x86 AMD SEV-ES: Set up GHCB page
x86 AMD SEV-ES: Add test cases

 .gitignore                 |   3 +
 Makefile                   |  29 +-
 README.md                  |   6 +
 configure                  |   6 +
 lib/efi.c                  | 118 ++++++++
 lib/efi.h                  |  22 ++
 lib/linux/efi.h            | 539 +++++++++++++++++++++++++++++++++++++
 lib/x86/acpi.c             |  38 ++-
 lib/x86/acpi.h             |  11 +
 lib/x86/amd_sev.c          | 174 ++++++++++++
 lib/x86/amd_sev.h          |  63 +++++
 lib/x86/asm/page.h         |  28 +-
 lib/x86/asm/setup.h        |  35 +++
 lib/x86/desc.c             |  46 +++-
 lib/x86/desc.h             |   6 +-
 lib/x86/setup.c            | 246 +++++++++++++++++
 lib/x86/usermode.c         |   3 +-
 lib/x86/vm.c               |  18 +-
 x86/Makefile.common        |  68 +++--
 x86/Makefile.i386          |   5 +-
 x86/Makefile.x86_64        |  58 ++--
 x86/access.c               |   9 +-
 x86/amd_sev.c              |  94 +++++++
 x86/cet.c                  |   8 +-
 x86/cstart64.S             |  77 +-----
 x86/efi/README.md          |  63 +++++
 x86/efi/crt0-efi-x86_64.S  |  79 ++++++
 x86/efi/efistart64.S       |  77 ++++++
 x86/efi/elf_x86_64_efi.lds |  81 ++++++
 x86/efi/reloc_x86_64.c     |  96 +++++++
 x86/efi/run                |  63 +++++
 x86/emulator.c             |   5 +-
 x86/eventinj.c             |   6 +-
 x86/run                    |  16 +-
 x86/smap.c                 |   8 +-
 x86/umip.c                 |  10 +-
 x86/vmx.c                  |   8 +-
 37 files changed, 2067 insertions(+), 155 deletions(-)
 create mode 100644 lib/efi.c
 create mode 100644 lib/efi.h
 create mode 100644 lib/linux/efi.h
 create mode 100644 lib/x86/amd_sev.c
 create mode 100644 lib/x86/amd_sev.h
 create mode 100644 lib/x86/asm/setup.h
 create mode 100644 x86/amd_sev.c
 create mode 100644 x86/efi/README.md
 create mode 100644 x86/efi/crt0-efi-x86_64.S
 create mode 100644 x86/efi/efistart64.S
 create mode 100644 x86/efi/elf_x86_64_efi.lds
 create mode 100644 x86/efi/reloc_x86_64.c
 create mode 100755 x86/efi/run
Signature · 1 line
2.33.0
#1 lore raw

PATCH v3 1/17 x86: Move IDT, GDT and TSS to desc.c

From: Zixuan Wang <[email protected]>

Move the IDT, GDT and TSS data structures from x86/cstart64.S to lib/x86/desc.c, so that the follow-up UEFI support commits can reuse these definitions, without re-defining them in UEFI's boot up assembly code.

In this commit, tss_descr is defined as a pointer, instead of an assembly label. This type change leads to several updates in the x86/vmx.c. Fortunately x86/vmx.c is only used in x86_64, so it is not necessary to be compatible with i386's tss_descr type which is an assembly label.

---
 lib/x86/asm/setup.h |  8 +++++
 lib/x86/desc.c      | 46 ++++++++++++++++++++++++++-
 lib/x86/desc.h      |  6 +++-
 lib/x86/setup.c     | 43 +++++++++++++++++++++++++
 x86/cstart64.S      | 77 ++-------------------------------------------
 x86/vmx.c           |  8 ++---
 6 files changed, 107 insertions(+), 81 deletions(-)
 create mode 100644 lib/x86/asm/setup.h
lib/x86/asm/setup.h new file +8 -0
new file mode 100644
index 0000000..19ded12
@@ -0,0 +1,8 @@
+#ifndef _X86_ASM_SETUP_H_
+#define _X86_ASM_SETUP_H_
+
+#ifdef __x86_64__
+unsigned long setup_tss(void);
+#endif /* __x86_64__ */
+
+#endif /* _X86_ASM_SETUP_H_ */
lib/x86/desc.c +45 -1

58 lines not on this page · raw message

lib/x86/desc.h +5 -1
index a6ffb38..c7ee881 100644
@@ -208,7 +208,11 @@ void set_idt_task_gate(int vec, u16 sel);
 void set_intr_task_gate(int vec, void *fn);
 void setup_tss32(void);
 #else
-extern tss64_t tss;
+extern tss64_t tss[];
+/* In gdt64, there are 16 entries before TSS entries */
+#define GDT64_PRE_TSS_ENTRIES (16)
+#define GDT64_TSS_OFFSET (GDT64_PRE_TSS_ENTRIES)
+extern gdt_entry_t gdt64[];
 #endif
 
 unsigned exception_vector(void);
lib/x86/setup.c +43 -0

61 lines not on this page · raw message

x86/cstart64.S +2 -75

100 lines not on this page · raw message

x86/vmx.c +4 -4
index f0b853a..37aff12 100644
@@ -75,7 +75,7 @@ union vmx_ept_vpid  ept_vpid;
 
 extern struct descriptor_table_ptr gdt64_desc;
 extern struct descriptor_table_ptr idt_descr;
-extern struct descriptor_table_ptr tss_descr;
+extern struct descriptor_table_ptr *tss_descr;
 extern void *vmx_return;
 extern void *entry_sysenter;
 extern void *guest_entry;
@@ -1276,7 +1276,7 @@ static void init_vmcs_host(void)
 	vmcs_write(HOST_SEL_FS, KERNEL_DS);
 	vmcs_write(HOST_SEL_GS, KERNEL_DS);
 	vmcs_write(HOST_SEL_TR, TSS_MAIN);
-	vmcs_write(HOST_BASE_TR, tss_descr.base);
+	vmcs_write(HOST_BASE_TR, tss_descr->base);
 	vmcs_write(HOST_BASE_GDTR, gdt64_desc.base);
 	vmcs_write(HOST_BASE_IDTR, idt_descr.base);
 	vmcs_write(HOST_BASE_FS, 0);
@@ -1332,7 +1332,7 @@ static void init_vmcs_guest(void)
 	vmcs_write(GUEST_BASE_DS, 0);
 	vmcs_write(GUEST_BASE_FS, 0);
 	vmcs_write(GUEST_BASE_GS, 0);
-	vmcs_write(GUEST_BASE_TR, tss_descr.base);
+	vmcs_write(GUEST_BASE_TR, tss_descr->base);
 	vmcs_write(GUEST_BASE_LDTR, 0);
 
 	vmcs_write(GUEST_LIMIT_CS, 0xFFFFFFFF);
@@ -1342,7 +1342,7 @@ static void init_vmcs_guest(void)
 	vmcs_write(GUEST_LIMIT_FS, 0xFFFFFFFF);
 	vmcs_write(GUEST_LIMIT_GS, 0xFFFFFFFF);
 	vmcs_write(GUEST_LIMIT_LDTR, 0xffff);
-	vmcs_write(GUEST_LIMIT_TR, tss_descr.limit);
+	vmcs_write(GUEST_LIMIT_TR, tss_descr->limit);
 
 	vmcs_write(GUEST_AR_CS, 0xa09b);
 	vmcs_write(GUEST_AR_DS, 0xc093);
Signature · 1 line
2.33.0
maintainer
#2 lore raw

Re: [kvm-unit-tests PATCH v3 01/17] x86: Move IDT, GDT and TSS to desc.c

lib/x86/desc.h void set_idt_task_gate(int vec, u16 sel); · line 214

Quoted 3 lines · Zixuan Wang
lib/x86/desc.h +3 -0
+/* In gdt64, there are 16 entries before TSS entries */
+#define GDT64_PRE_TSS_ENTRIES (16)
+#define GDT64_TSS_OFFSET (GDT64_PRE_TSS_ENTRIES)

No need to have both; in fact the definition can also be changed to TSS_MAIN/8.

tss_descr is completely broken in the current code, I'll send a patch to fix it so you don't have to deal with it.

Paolo

Quoted 1 line

+extern gdt_entry_t gdt64[];

#3 lore raw

Re: [kvm-unit-tests PATCH v3 01/17] x86: Move IDT, GDT and TSS to desc.c

Quoted 8 lines · Paolo Bonzini

lib/x86/desc.h void set_idt_task_gate(int vec, u16 sel); · line 214

Quoted 3 lines · Zixuan Wang
lib/x86/desc.h +3 -0
+/* In gdt64, there are 16 entries before TSS entries */
+#define GDT64_PRE_TSS_ENTRIES (16)
+#define GDT64_TSS_OFFSET (GDT64_PRE_TSS_ENTRIES)

No need to have both; in fact the definition can also be changed to TSS_MAIN/8.

I did not realize there is a TSS_MAIN, I will update this part.

Quoted 4 lines

tss_descr is completely broken in the current code, I'll send a patch to fix it so you don't have to deal with it.

Paolo

I just noticed the new patchset to fix the tss_descr, I will build the V4 patchset based on that fix.

Best regards,
Zixuan

maintainer
#4 lore raw

Re: [kvm-unit-tests PATCH v3 01/17] x86: Move IDT, GDT and TSS to desc.c

Quoted 6 lines · Zixuan Wang
Quoted 4 lines

tss_descr is completely broken in the current code, I'll send a patch to fix it so you don't have to deal with it.

Paolo

I just noticed the new patchset to fix the tss_descr, I will build the V4 patchset based on that fix.

Don't worry, there's a lot more work to cleanup GDT/TSS/IDT and I don't want to "force" you to do that. Let me go on with the review of this series so that you can concentrate on the UEFI bits.

Paolo

#5 lore raw

PATCH v3 2/17 x86 UEFI: Copy code from Linux

In reply to Zixuan Wang

From: Varad Gautam <[email protected]>

Copy UEFI-related definitions from Linux, so the follow-up commits can develop UEFI function calls based on these definitions, without relying on GNU-EFI library.

---
 lib/linux/efi.h | 518 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 518 insertions(+)
 create mode 100644 lib/linux/efi.h
lib/linux/efi.h new file +518 -0
new file mode 100644
index 0000000..bb4601b
@@ -0,0 +1,518 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Relevant definitions from linux/efi.h. */
+
+#ifndef __LINUX_UEFI_H
+#define __LINUX_UEFI_H
+
+#define BITS_PER_LONG 64
+
+#define EFI_SUCCESS		0
+#define EFI_LOAD_ERROR		( 1 | (1UL << (BITS_PER_LONG-1)))
+#define EFI_INVALID_PARAMETER	( 2 | (1UL << (BITS_PER_LONG-1)))
+#define EFI_UNSUPPORTED		( 3 | (1UL << (BITS_PER_LONG-1)))
+#define EFI_BAD_BUFFER_SIZE	( 4 | (1UL << (BITS_PER_LONG-1)))
+#define EFI_BUFFER_TOO_SMALL	( 5 | (1UL << (BITS_PER_LONG-1)))
+#define EFI_NOT_READY		( 6 | (1UL << (BITS_PER_LONG-1)))
+#define EFI_DEVICE_ERROR	( 7 | (1UL << (BITS_PER_LONG-1)))
+#define EFI_WRITE_PROTECTED	( 8 | (1UL << (BITS_PER_LONG-1)))
+#define EFI_OUT_OF_RESOURCES	( 9 | (1UL << (BITS_PER_LONG-1)))
+#define EFI_NOT_FOUND		(14 | (1UL << (BITS_PER_LONG-1)))
+#define EFI_TIMEOUT		(18 | (1UL << (BITS_PER_LONG-1)))
+#define EFI_ABORTED		(21 | (1UL << (BITS_PER_LONG-1)))
+#define EFI_SECURITY_VIOLATION	(26 | (1UL << (BITS_PER_LONG-1)))
+
+typedef unsigned long efi_status_t;
+typedef u8 efi_bool_t;
+typedef u16 efi_char16_t;		/* UNICODE character */
+typedef u64 efi_physical_addr_t;
+typedef void *efi_handle_t;
+
+#define __efiapi __attribute__((ms_abi))
+
+/*
+ * The UEFI spec and EDK2 reference implementation both define EFI_GUID as
+ * struct { u32 a; u16; b; u16 c; u8 d[8]; }; and so the implied alignment
+ * is 32 bits not 8 bits like our guid_t. In some cases (i.e., on 32-bit ARM),
+ * this means that firmware services invoked by the kernel may assume that
+ * efi_guid_t* arguments are 32-bit aligned, and use memory accessors that
+ * do not tolerate misalignment. So let's set the minimum alignment to 32 bits.
+ *
+ * Note that the UEFI spec as well as some comments in the EDK2 code base
+ * suggest that EFI_GUID should be 64-bit aligned, but this appears to be
+ * a mistake, given that no code seems to exist that actually enforces that
+ * or relies on it.
+ */
+typedef struct {
+	u8 b[16];
+} guid_t __attribute__((aligned(__alignof__(u32))));
+typedef guid_t efi_guid_t;
+
+#define EFI_GUID(a, b, c, d...) (efi_guid_t){ {					\
+	(a) & 0xff, ((a) >> 8) & 0xff, ((a) >> 16) & 0xff, ((a) >> 24) & 0xff,	\
+	(b) & 0xff, ((b) >> 8) & 0xff,						\
+	(c) & 0xff, ((c) >> 8) & 0xff, d } }
+
+/*
+ * Generic EFI table header
+ */
+typedef	struct {
+	u64 signature;
+	u32 revision;
+	u32 headersize;
+	u32 crc32;
+	u32 reserved;
+} efi_table_hdr_t;
+
+/*
+ * Memory map descriptor:
+ */
+
+/* Memory types: */
+#define EFI_RESERVED_TYPE		 0
+#define EFI_LOADER_CODE			 1
+#define EFI_LOADER_DATA			 2
+#define EFI_BOOT_SERVICES_CODE		 3
+#define EFI_BOOT_SERVICES_DATA		 4
+#define EFI_RUNTIME_SERVICES_CODE	 5
+#define EFI_RUNTIME_SERVICES_DATA	 6
+#define EFI_CONVENTIONAL_MEMORY		 7
+#define EFI_UNUSABLE_MEMORY		 8
+#define EFI_ACPI_RECLAIM_MEMORY		 9
+#define EFI_ACPI_MEMORY_NVS		10
+#define EFI_MEMORY_MAPPED_IO		11
+#define EFI_MEMORY_MAPPED_IO_PORT_SPACE	12
+#define EFI_PAL_CODE			13
+#define EFI_PERSISTENT_MEMORY		14
+#define EFI_MAX_MEMORY_TYPE		15
+
+/* Attribute values: */
+#define EFI_MEMORY_UC		((u64)0x0000000000000001ULL)	/* uncached */
+#define EFI_MEMORY_WC		((u64)0x0000000000000002ULL)	/* write-coalescing */
+#define EFI_MEMORY_WT		((u64)0x0000000000000004ULL)	/* write-through */
+#define EFI_MEMORY_WB		((u64)0x0000000000000008ULL)	/* write-back */
+#define EFI_MEMORY_UCE		((u64)0x0000000000000010ULL)	/* uncached, exported */
+#define EFI_MEMORY_WP		((u64)0x0000000000001000ULL)	/* write-protect */
+#define EFI_MEMORY_RP		((u64)0x0000000000002000ULL)	/* read-protect */
+#define EFI_MEMORY_XP		((u64)0x0000000000004000ULL)	/* execute-protect */
+#define EFI_MEMORY_NV		((u64)0x0000000000008000ULL)	/* non-volatile */
+#define EFI_MEMORY_MORE_RELIABLE \
+				((u64)0x0000000000010000ULL)	/* higher reliability */
+#define EFI_MEMORY_RO		((u64)0x0000000000020000ULL)	/* read-only */
+#define EFI_MEMORY_SP		((u64)0x0000000000040000ULL)	/* soft reserved */
+#define EFI_MEMORY_CPU_CRYPTO	((u64)0x0000000000080000ULL)	/* supports encryption */
+#define EFI_MEMORY_RUNTIME	((u64)0x8000000000000000ULL)	/* range requires runtime mapping */
+#define EFI_MEMORY_DESCRIPTOR_VERSION	1
+
+#define EFI_PAGE_SHIFT		12
+#define EFI_PAGE_SIZE		(1UL << EFI_PAGE_SHIFT)
+#define EFI_PAGES_MAX		(U64_MAX >> EFI_PAGE_SHIFT)
+
+typedef struct {
+	u32 type;
+	u32 pad;
+	u64 phys_addr;
+	u64 virt_addr;
+	u64 num_pages;
+	u64 attribute;
+} efi_memory_desc_t;
+
+typedef struct {
+	efi_guid_t guid;
+	u32 headersize;
+	u32 flags;
+	u32 imagesize;
+} efi_capsule_header_t;
+
+/*
+ * EFI capsule flags
+ */
+#define EFI_CAPSULE_PERSIST_ACROSS_RESET	0x00010000
+#define EFI_CAPSULE_POPULATE_SYSTEM_TABLE	0x00020000
+#define EFI_CAPSULE_INITIATE_RESET		0x00040000
+
+struct capsule_info {
+	efi_capsule_header_t	header;
+	efi_capsule_header_t	*capsule;
+	int			reset_type;
+	long			index;
+	size_t			count;
+	size_t			total_size;
+	struct page		**pages;
+	phys_addr_t		*phys;
+	size_t			page_bytes_remain;
+};
+
+int __efi_capsule_setup_info(struct capsule_info *cap_info);
+
+/*
+ * Types and defines for Time Services
+ */
+#define EFI_TIME_ADJUST_DAYLIGHT 0x1
+#define EFI_TIME_IN_DAYLIGHT     0x2
+#define EFI_UNSPECIFIED_TIMEZONE 0x07ff
+
+typedef struct {
+	u16 year;
+	u8 month;
+	u8 day;
+	u8 hour;
+	u8 minute;
+	u8 second;
+	u8 pad1;
+	u32 nanosecond;
+	s16 timezone;
+	u8 daylight;
+	u8 pad2;
+} efi_time_t;
+
+typedef struct {
+	u32 resolution;
+	u32 accuracy;
+	u8 sets_to_zero;
+} efi_time_cap_t;
+
+typedef void *efi_event_t;
+/* Note that notifications won't work in mixed mode */
+typedef void (__efiapi *efi_event_notify_t)(efi_event_t, void *);
+
+typedef enum {
+	EfiTimerCancel,
+	EfiTimerPeriodic,
+	EfiTimerRelative
+} EFI_TIMER_DELAY;
+
+/*
+ * EFI Device Path information
+ */
+#define EFI_DEV_HW			0x01
+#define  EFI_DEV_PCI				 1
+#define  EFI_DEV_PCCARD				 2
+#define  EFI_DEV_MEM_MAPPED			 3
+#define  EFI_DEV_VENDOR				 4
+#define  EFI_DEV_CONTROLLER			 5
+#define EFI_DEV_ACPI			0x02
+#define   EFI_DEV_BASIC_ACPI			 1
+#define   EFI_DEV_EXPANDED_ACPI			 2
+#define EFI_DEV_MSG			0x03
+#define   EFI_DEV_MSG_ATAPI			 1
+#define   EFI_DEV_MSG_SCSI			 2
+#define   EFI_DEV_MSG_FC			 3
+#define   EFI_DEV_MSG_1394			 4
+#define   EFI_DEV_MSG_USB			 5
+#define   EFI_DEV_MSG_USB_CLASS			15
+#define   EFI_DEV_MSG_I20			 6
+#define   EFI_DEV_MSG_MAC			11
+#define   EFI_DEV_MSG_IPV4			12
+#define   EFI_DEV_MSG_IPV6			13
+#define   EFI_DEV_MSG_INFINIBAND		 9
+#define   EFI_DEV_MSG_UART			14
+#define   EFI_DEV_MSG_VENDOR			10
+#define EFI_DEV_MEDIA			0x04
+#define   EFI_DEV_MEDIA_HARD_DRIVE		 1
+#define   EFI_DEV_MEDIA_CDROM			 2
+#define   EFI_DEV_MEDIA_VENDOR			 3
+#define   EFI_DEV_MEDIA_FILE			 4
+#define   EFI_DEV_MEDIA_PROTOCOL		 5
+#define EFI_DEV_BIOS_BOOT		0x05
+#define EFI_DEV_END_PATH		0x7F
+#define EFI_DEV_END_PATH2		0xFF
+#define   EFI_DEV_END_INSTANCE			0x01
+#define   EFI_DEV_END_ENTIRE			0xFF
+
+struct efi_generic_dev_path {
+	u8				type;
+	u8				sub_type;
+	u16				length;
+} __packed;
+
+typedef struct efi_generic_dev_path efi_device_path_protocol_t;
+
+/*
+ * EFI Boot Services table
+ */
+union efi_boot_services {
+	struct {
+		efi_table_hdr_t hdr;
+		void *raise_tpl;
+		void *restore_tpl;
+		efi_status_t (__efiapi *allocate_pages)(int, int, unsigned long,
+							efi_physical_addr_t *);
+		efi_status_t (__efiapi *free_pages)(efi_physical_addr_t,
+						    unsigned long);
+		efi_status_t (__efiapi *get_memory_map)(unsigned long *, void *,
+							unsigned long *,
+							unsigned long *, u32 *);
+		efi_status_t (__efiapi *allocate_pool)(int, unsigned long,
+						       void **);
+		efi_status_t (__efiapi *free_pool)(void *);
+		efi_status_t (__efiapi *create_event)(u32, unsigned long,
+						      efi_event_notify_t, void *,
+						      efi_event_t *);
+		efi_status_t (__efiapi *set_timer)(efi_event_t,
+						  EFI_TIMER_DELAY, u64);
+		efi_status_t (__efiapi *wait_for_event)(unsigned long,
+							efi_event_t *,
+							unsigned long *);
+		void *signal_event;
+		efi_status_t (__efiapi *close_event)(efi_event_t);
+		void *check_event;
+		void *install_protocol_interface;
+		void *reinstall_protocol_interface;
+		void *uninstall_protocol_interface;
+		efi_status_t (__efiapi *handle_protocol)(efi_handle_t,
+							 efi_guid_t *, void **);
+		void *__reserved;
+		void *register_protocol_notify;
+		efi_status_t (__efiapi *locate_handle)(int, efi_guid_t *,
+						       void *, unsigned long *,
+						       efi_handle_t *);
+		efi_status_t (__efiapi *locate_device_path)(efi_guid_t *,
+							    efi_device_path_protocol_t **,
+							    efi_handle_t *);
+		efi_status_t (__efiapi *install_configuration_table)(efi_guid_t *,
+								     void *);
+		void *load_image;
+		void *start_image;
+		efi_status_t (__efiapi *exit)(efi_handle_t,
+							 efi_status_t,
+							 unsigned long,
+							 efi_char16_t *);
+		void *unload_image;
+		efi_status_t (__efiapi *exit_boot_services)(efi_handle_t,
+							    unsigned long);
+		void *get_next_monotonic_count;
+		efi_status_t (__efiapi *stall)(unsigned long);
+		void *set_watchdog_timer;
+		void *connect_controller;
+		efi_status_t (__efiapi *disconnect_controller)(efi_handle_t,
+							       efi_handle_t,
+							       efi_handle_t);
+		void *open_protocol;
+		void *close_protocol;
+		void *open_protocol_information;
+		void *protocols_per_handle;
+		void *locate_handle_buffer;
+		efi_status_t (__efiapi *locate_protocol)(efi_guid_t *, void *,
+							 void **);
+		void *install_multiple_protocol_interfaces;
+		void *uninstall_multiple_protocol_interfaces;
+		void *calculate_crc32;
+		void *copy_mem;
+		void *set_mem;
+		void *create_event_ex;
+	};
+	struct {
+		efi_table_hdr_t hdr;
+		u32 raise_tpl;
+		u32 restore_tpl;
+		u32 allocate_pages;
+		u32 free_pages;
+		u32 get_memory_map;
+		u32 allocate_pool;
+		u32 free_pool;
+		u32 create_event;
+		u32 set_timer;
+		u32 wait_for_event;
+		u32 signal_event;
+		u32 close_event;
+		u32 check_event;
+		u32 install_protocol_interface;
+		u32 reinstall_protocol_interface;
+		u32 uninstall_protocol_interface;
+		u32 handle_protocol;
+		u32 __reserved;
+		u32 register_protocol_notify;
+		u32 locate_handle;
+		u32 locate_device_path;
+		u32 install_configuration_table;
+		u32 load_image;
+		u32 start_image;
+		u32 exit;
+		u32 unload_image;
+		u32 exit_boot_services;
+		u32 get_next_monotonic_count;
+		u32 stall;
+		u32 set_watchdog_timer;
+		u32 connect_controller;
+		u32 disconnect_controller;
+		u32 open_protocol;
+		u32 close_protocol;
+		u32 open_protocol_information;
+		u32 protocols_per_handle;
+		u32 locate_handle_buffer;
+		u32 locate_protocol;
+		u32 install_multiple_protocol_interfaces;
+		u32 uninstall_multiple_protocol_interfaces;
+		u32 calculate_crc32;
+		u32 copy_mem;
+		u32 set_mem;
+		u32 create_event_ex;
+	} mixed_mode;
+};
+
+typedef union efi_boot_services efi_boot_services_t;
+
+/*
+ * Types and defines for EFI ResetSystem
+ */
+#define EFI_RESET_COLD 0
+#define EFI_RESET_WARM 1
+#define EFI_RESET_SHUTDOWN 2
+
+/*
+ * EFI Runtime Services table
+ */
+#define EFI_RUNTIME_SERVICES_SIGNATURE ((u64)0x5652453544e5552ULL)
+#define EFI_RUNTIME_SERVICES_REVISION  0x00010000
+
+typedef struct {
+	efi_table_hdr_t hdr;
+	u32 get_time;
+	u32 set_time;
+	u32 get_wakeup_time;
+	u32 set_wakeup_time;
+	u32 set_virtual_address_map;
+	u32 convert_pointer;
+	u32 get_variable;
+	u32 get_next_variable;
+	u32 set_variable;
+	u32 get_next_high_mono_count;
+	u32 reset_system;
+	u32 update_capsule;
+	u32 query_capsule_caps;
+	u32 query_variable_info;
+} efi_runtime_services_32_t;
+
+typedef efi_status_t efi_get_time_t (efi_time_t *tm, efi_time_cap_t *tc);
+typedef efi_status_t efi_set_time_t (efi_time_t *tm);
+typedef efi_status_t efi_get_wakeup_time_t (efi_bool_t *enabled, efi_bool_t *pending,
+					    efi_time_t *tm);
+typedef efi_status_t efi_set_wakeup_time_t (efi_bool_t enabled, efi_time_t *tm);
+typedef efi_status_t efi_get_variable_t (efi_char16_t *name, efi_guid_t *vendor, u32 *attr,
+					 unsigned long *data_size, void *data);
+typedef efi_status_t efi_get_next_variable_t (unsigned long *name_size, efi_char16_t *name,
+					      efi_guid_t *vendor);
+typedef efi_status_t efi_set_variable_t (efi_char16_t *name, efi_guid_t *vendor,
+					 u32 attr, unsigned long data_size,
+					 void *data);
+typedef efi_status_t efi_get_next_high_mono_count_t (u32 *count);
+typedef void efi_reset_system_t (int reset_type, efi_status_t status,
+				 unsigned long data_size, efi_char16_t *data);
+typedef efi_status_t efi_set_virtual_address_map_t (unsigned long memory_map_size,
+						unsigned long descriptor_size,
+						u32 descriptor_version,
+						efi_memory_desc_t *virtual_map);
+typedef efi_status_t efi_query_variable_info_t(u32 attr,
+					       u64 *storage_space,
+					       u64 *remaining_space,
+					       u64 *max_variable_size);
+typedef efi_status_t efi_update_capsule_t(efi_capsule_header_t **capsules,
+					  unsigned long count,
+					  unsigned long sg_list);
+typedef efi_status_t efi_query_capsule_caps_t(efi_capsule_header_t **capsules,
+					      unsigned long count,
+					      u64 *max_size,
+					      int *reset_type);
+typedef efi_status_t efi_query_variable_store_t(u32 attributes,
+						unsigned long size,
+						bool nonblocking);
+
+typedef union {
+	struct {
+		efi_table_hdr_t				hdr;
+		efi_get_time_t __efiapi			*get_time;
+		efi_set_time_t __efiapi			*set_time;
+		efi_get_wakeup_time_t __efiapi		*get_wakeup_time;
+		efi_set_wakeup_time_t __efiapi		*set_wakeup_time;
+		efi_set_virtual_address_map_t __efiapi	*set_virtual_address_map;
+		void					*convert_pointer;
+		efi_get_variable_t __efiapi		*get_variable;
+		efi_get_next_variable_t __efiapi	*get_next_variable;
+		efi_set_variable_t __efiapi		*set_variable;
+		efi_get_next_high_mono_count_t __efiapi	*get_next_high_mono_count;
+		efi_reset_system_t __efiapi		*reset_system;
+		efi_update_capsule_t __efiapi		*update_capsule;
+		efi_query_capsule_caps_t __efiapi	*query_capsule_caps;
+		efi_query_variable_info_t __efiapi	*query_variable_info;
+	};
+	efi_runtime_services_32_t mixed_mode;
+} efi_runtime_services_t;
+
+#define EFI_SYSTEM_TABLE_SIGNATURE ((u64)0x5453595320494249ULL)
+
+#define EFI_2_30_SYSTEM_TABLE_REVISION  ((2 << 16) | (30))
+#define EFI_2_20_SYSTEM_TABLE_REVISION  ((2 << 16) | (20))
+#define EFI_2_10_SYSTEM_TABLE_REVISION  ((2 << 16) | (10))
+#define EFI_2_00_SYSTEM_TABLE_REVISION  ((2 << 16) | (00))
+#define EFI_1_10_SYSTEM_TABLE_REVISION  ((1 << 16) | (10))
+#define EFI_1_02_SYSTEM_TABLE_REVISION  ((1 << 16) | (02))
+
+typedef struct {
+	efi_table_hdr_t hdr;
+	u64 fw_vendor;	/* physical addr of CHAR16 vendor string */
+	u32 fw_revision;
+	u32 __pad1;
+	u64 con_in_handle;
+	u64 con_in;
+	u64 con_out_handle;
+	u64 con_out;
+	u64 stderr_handle;
+	u64 stderr;
+	u64 runtime;
+	u64 boottime;
+	u32 nr_tables;
+	u32 __pad2;
+	u64 tables;
+} efi_system_table_64_t;
+
+typedef struct {
+	efi_table_hdr_t hdr;
+	u32 fw_vendor;	/* physical addr of CHAR16 vendor string */
+	u32 fw_revision;
+	u32 con_in_handle;
+	u32 con_in;
+	u32 con_out_handle;
+	u32 con_out;
+	u32 stderr_handle;
+	u32 stderr;
+	u32 runtime;
+	u32 boottime;
+	u32 nr_tables;
+	u32 tables;
+} efi_system_table_32_t;
+
+typedef union efi_simple_text_input_protocol efi_simple_text_input_protocol_t;
+typedef union efi_simple_text_output_protocol efi_simple_text_output_protocol_t;
+
+typedef union {
+	struct {
+		efi_table_hdr_t hdr;
+		unsigned long fw_vendor;	/* physical addr of CHAR16 vendor string */
+		u32 fw_revision;
+		unsigned long con_in_handle;
+		efi_simple_text_input_protocol_t *con_in;
+		unsigned long con_out_handle;
+		efi_simple_text_output_protocol_t *con_out;
+		unsigned long stderr_handle;
+		unsigned long stderr;
+		efi_runtime_services_t *runtime;
+		efi_boot_services_t *boottime;
+		unsigned long nr_tables;
+		unsigned long tables;
+	};
+	efi_system_table_32_t mixed_mode;
+} efi_system_table_t;
+
+struct efi_boot_memmap {
+	efi_memory_desc_t       **map;
+	unsigned long           *map_size;
+	unsigned long           *desc_size;
+	u32                     *desc_ver;
+	unsigned long           *key_ptr;
+	unsigned long           *buff_size;
+};
+
+#define efi_bs_call(func, ...)						\
+	efi_system_table->boottime->func(__VA_ARGS__)
+
+#endif /* __LINUX_UEFI_H */
Signature · 1 line
2.33.0
#6 lore raw

PATCH v3 3/17 x86 UEFI: Implement UEFI function calls

In reply to Zixuan Wang

From: Varad Gautam <[email protected]>

This commit implements helper functions that call UEFI services and assist the boot up process.

---
 lib/efi.c | 67 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)
 create mode 100644 lib/efi.c
lib/efi.c new file +67 -0
new file mode 100644
index 0000000..7f08f0e
@@ -0,0 +1,67 @@
+/*
+ * EFI-related functions to set up and run test cases in EFI
+ *
+ * Copyright (c) 2021, SUSE, Varad Gautam <[email protected]>
+ *
+ * SPDX-License-Identifier: LGPL-2.0-or-later
+ */
+#include <linux/efi.h>
+
+unsigned long __efiapi efi_main(efi_handle_t handle, efi_system_table_t *sys_tab);
+efi_system_table_t *efi_system_table = NULL;
+
+static void efi_free_pool(void *ptr)
+{
+	efi_bs_call(free_pool, ptr);
+}
+
+static efi_status_t efi_get_memory_map(struct efi_boot_memmap *map)
+{
+	efi_memory_desc_t *m = NULL;
+	efi_status_t status;
+	unsigned long key = 0, map_size = 0, desc_size = 0;
+
+	status = efi_bs_call(get_memory_map, &map_size,
+			     NULL, &key, &desc_size, NULL);
+	if (status != EFI_BUFFER_TOO_SMALL || map_size == 0)
+		goto out;
+
+	/*
+	 * Pad map_size with additional descriptors so we don't need to
+	 * retry.
+	 */
+	map_size += 4 * desc_size;
+	*map->buff_size = map_size;
+	status = efi_bs_call(allocate_pool, EFI_LOADER_DATA,
+			     map_size, (void **)&m);
+	if (status != EFI_SUCCESS)
+		goto out;
+
+	/* Get the map. */
+	status = efi_bs_call(get_memory_map, &map_size,
+			     m, &key, &desc_size, NULL);
+	if (status != EFI_SUCCESS) {
+		efi_free_pool(m);
+		goto out;
+	}
+
+	*map->desc_size = desc_size;
+	*map->map_size = map_size;
+	*map->key_ptr = key;
+out:
+	*map->map = m;
+	return status;
+}
+
+static efi_status_t efi_exit_boot_services(void *handle,
+					   struct efi_boot_memmap *map)
+{
+	return efi_bs_call(exit_boot_services, handle, *map->key_ptr);
+}
+
+unsigned long __efiapi efi_main(efi_handle_t handle, efi_system_table_t *sys_tab)
+{
+	efi_system_table = sys_tab;
+
+	return 0;
+}
Signature · 1 line
2.33.0
#7 lore raw

PATCH v3 4/17 x86 UEFI: Copy code from GNU-EFI

In reply to Zixuan Wang

From: Zixuan Wang <[email protected]>

To build x86 test cases with UEFI, we need to borrow some source code from GNU-EFI, which includes the initialization code and linker scripts. This commit only copies the source code, without any modification. These source code files are not used by KVM-Unit-Tests in this commit.

The following source code is copied from GNU-EFI:
1. x86/efi/elf_x86_64_efi.lds
2. x86/efi/reloc_x86_64.c
3. x86/efi/crt0-efi-x86_64.S

We put these EFI-related files under a new dir x86/efi because:
1. EFI-related code is easy to find
2. EFI-related code is separated from the original code in x86/
3. EFI-related code can still reuse the Makefile and test case code in its parent dir x86/

GNU-EFI repo and version:
GIT URL: https://git.code.sf.net/p/gnu-efi/code
Commit ID: 4fe83e102674
Website: https://sourceforge.net/p/gnu-efi/code/ci/4fe83e/tree/

---
 x86/efi/README.md          |  25 ++++++++++
 x86/efi/crt0-efi-x86_64.S  |  79 +++++++++++++++++++++++++++++
 x86/efi/elf_x86_64_efi.lds |  77 ++++++++++++++++++++++++++++
 x86/efi/reloc_x86_64.c     | 100 +++++++++++++++++++++++++++++++++++++
 4 files changed, 281 insertions(+)
 create mode 100644 x86/efi/README.md
 create mode 100644 x86/efi/crt0-efi-x86_64.S
 create mode 100644 x86/efi/elf_x86_64_efi.lds
 create mode 100644 x86/efi/reloc_x86_64.c
x86/efi/README.md new file +25 -0
new file mode 100644
index 0000000..bc1f733
@@ -0,0 +1,25 @@
+# EFI Startup Code and Linker Script
+
+This dir contains source code and linker script copied from
+[GNU-EFI](https://sourceforge.net/projects/gnu-efi/):
+   - crt0-efi-x86_64.S: startup code of an EFI application
+   - elf_x86_64_efi.lds: linker script to build an EFI application
+   - reloc_x86_64.c: position independent x86_64 ELF shared object relocator
+
+EFI application binaries should be relocatable as UEFI loads binaries to dynamic
+runtime addresses. To build such relocatable binaries, GNU-EFI utilizes the
+above-mentioned files in its build process:
+
+   1. build an ELF shared object and link it using linker script
+      `elf_x86_64_efi.lds` to organize the sections in a way UEFI recognizes
+   2. link the shared object with self-relocator `reloc_x86_64.c` that applies
+      dynamic relocations that may be present in the shared object
+   3. link the entry point code `crt0-efi-x86_64.S` that invokes self-relocator
+      and then jumps to EFI application's `efi_main()` function
+   4. convert the shared object to an EFI binary
+
+More details can be found in `GNU-EFI/README.gnuefi`, section "Building
+Relocatable Binaries".
+
+kvm-unit-tests follows a similar build process, but does not link with GNU-EFI
+library.
x86/efi/crt0-efi-x86_64.S new file +79 -0

79 lines not on this page · raw message

x86/efi/elf_x86_64_efi.lds new file +77 -0

77 lines not on this page · raw message

x86/efi/reloc_x86_64.c new file +100 -0

100 lines not on this page · raw message

Signature · 1 line
2.33.0
#8 lore raw

PATCH v3 5/17 x86 UEFI: Boot from UEFI

In reply to Zixuan Wang

From: Zixuan Wang <[email protected]>

This commit provides initial support for x86 test cases to boot from UEFI:

1. UEFI compiler flags are added to Makefile
2. A new TARGET_EFI macro is added to turn on/off UEFI startup code
3. Previous Multiboot setup code is refactored and updated for supporting UEFI, including the following changes:
1. x86/efi/crt0-efi-x86_64.S: provides entry point and jumps to setup code in lib/efi.c.
2. lib/efi.c: performs UEFI setup, calls arch-related setup functions, then jumps to test case main() function
3. lib/x86/setup.c: provides arch-related setup under UEFI

To build test cases for UEFI, please first install the GNU-EFI library. Check x86/efi/README.md for more details.

This commit is tested by a simple test calling report() and report_summayr(). This commit does not include such a test to avoid unnecessary files added into git history. To build and run this test in UEFI (assuming file name is x86/dummy.c):

   ./configure --target-efi
   make x86/dummy.efi
   ./x86/efi/run ./x86/dummy.efi

To use the default Multiboot instead of UEFI:

./configure
make x86/dummy.flat
./x86/run ./x86/dummy.flat

Some x86 test cases require additional fixes to work in UEFI, e.g., converting to position independent code (PIC), setting up page tables, etc. This commit does not provide these fixes, so compiling and running UEFI test cases other than x86/dummy.c may trigger compiler errors or QEMU crashes. These test cases will be fixed by the follow-up commits in this series.

The following code is ported from github.com/rhdrjones/kvm-unit-tests
- ./configure: 'target-efi'-related code

See original code:
- Repo: https://github.com/rhdrjones/kvm-unit-tests
- Branch: target-efi

---
 .gitignore             |  3 ++
 Makefile               | 29 +++++++++++++++++-
 README.md              |  6 ++++
 configure              |  6 ++++
 lib/efi.c              | 32 +++++++++++++++-----
 lib/efi.h              | 21 +++++++++++++
 lib/linux/efi.h        | 10 +++++--
 lib/x86/asm/setup.h    |  7 +++++
 lib/x86/setup.c        | 13 ++++++++
 x86/Makefile.common    | 67 +++++++++++++++++++++++++++++++-----------
 x86/Makefile.i386      |  5 ++--
 x86/Makefile.x86_64    | 54 ++++++++++++++++++++++------------
 x86/efi/README.md      | 40 ++++++++++++++++++++++++-
 x86/efi/reloc_x86_64.c |  8 ++---
 x86/efi/run            | 63 +++++++++++++++++++++++++++++++++++++++
 x86/run                | 16 ++++++++--
 16 files changed, 323 insertions(+), 57 deletions(-)
 create mode 100644 lib/efi.h
 create mode 100755 x86/efi/run
.gitignore +3 -0
index b3cf2cb..dca6d29 100644
@@ -3,7 +3,9 @@ tags
 *.a
 *.d
 *.o
+*.so
 *.flat
+*.efi
 *.elf
 .pc
 patches
@@ -24,3 +26,4 @@ cscope.*
 /api/dirty-log-perf
 /s390x/*.bin
 /s390x/snippets/*/*.gbin
+/efi-tests/*
Makefile +28 -1
index 6792b93..6465929 100644
@@ -38,6 +38,29 @@ LIBFDT_archive = $(LIBFDT_objdir)/libfdt.a
 
 OBJDIRS += $(LIBFDT_objdir)
 
+# EFI App
+ifeq ($(TARGET_EFI),y)
+ifeq ($(ARCH_NAME),x86_64)
+EFI_ARCH = x86_64
+else
+$(error Cannot build $(ARCH_NAME) tests as EFI apps)
+endif
+EFI_CFLAGS := -DTARGET_EFI
+# The following CFLAGS and LDFLAGS come from:
+#   - GNU-EFI/Makefile.defaults
+#   - GNU-EFI/apps/Makefile
+# Function calls must include the number of arguments passed to the functions
+# More details: https://wiki.osdev.org/GNU-EFI
+EFI_CFLAGS += -maccumulate-outgoing-args
+# GCC defines wchar to be 32 bits, but EFI expects 16 bits
+EFI_CFLAGS += -fshort-wchar
+# EFI applications use PIC as they are loaded to dynamic addresses, not a fixed
+# starting address
+EFI_CFLAGS += -fPIC
+# Create shared library
+EFI_LDFLAGS := -Bsymbolic -shared -nostdlib
+endif
+
 #include architecture specific make rules
 include $(SRCDIR)/$(TEST_DIR)/Makefile
 
@@ -62,7 +85,11 @@ COMMON_CFLAGS += $(fno_stack_protector)
 COMMON_CFLAGS += $(fno_stack_protector_all)
 COMMON_CFLAGS += $(wno_frame_address)
 COMMON_CFLAGS += $(if $(U32_LONG_FMT),-D__U32_LONG_FMT__,)
+ifeq ($(TARGET_EFI),y)
+COMMON_CFLAGS += $(EFI_CFLAGS)
+else
 COMMON_CFLAGS += $(fno_pic) $(no_pie)
+endif
 COMMON_CFLAGS += $(wclobbered)
 COMMON_CFLAGS += $(wunused_but_set_parameter)
 
@@ -113,7 +140,7 @@ clean: arch_clean libfdt_clean
 
 distclean: clean
 	$(RM) lib/asm lib/config.h config.mak $(TEST_DIR)-run msr.out cscope.* build-head
-	$(RM) -r tests logs logs.old
+	$(RM) -r tests logs logs.old efi-tests
 
 cscope: cscope_dirs = lib lib/libfdt lib/linux $(TEST_DIR) $(ARCH_LIBDIRS) lib/asm-generic
 cscope:
README.md +6 -0
index b498aaf..6edacfe 100644
@@ -17,6 +17,8 @@ in this directory.  Test images are created in ./ARCH/\*.flat
 
 NOTE: GCC cross-compiler is required for [build on macOS](README.macOS.md).
 
+To build with UEFI, check [build and run with UEFI](./x86/efi/README.md).
+
 ## Standalone tests
 
 The tests can be built as standalone.  To create and use standalone tests do:
@@ -54,6 +56,10 @@ ACCEL=name environment variable:
 
     ACCEL=kvm ./x86-run ./x86/msr.flat
 
+## Running the tests with UEFI
+
+Check [build and run with UEFI](./x86/efi/README.md).
+
 # Tests configuration file
 
 The test case may need specific runtime configurations, for
configure +6 -0
index 1d4d855..b6c09b3 100755
@@ -28,6 +28,7 @@ erratatxt="$srcdir/errata.txt"
 host_key_document=
 page_size=
 earlycon=
+target_efi=
 
 usage() {
     cat <<-EOF
@@ -69,6 +70,7 @@ usage() {
 	               pl011,mmio32,ADDR
 	                           Specify a PL011 compatible UART at address ADDR. Supported
 	                           register stride is 32 bit only.
+	    --target-efi           Boot and run from UEFI
 EOF
     exit 1
 }
@@ -133,6 +135,9 @@ while [[ "$1" = -* ]]; do
 	--earlycon)
 	    earlycon="$arg"
 	    ;;
+	--target-efi)
+	    target_efi=y
+	    ;;
 	--help)
 	    usage
 	    ;;
@@ -341,6 +346,7 @@ U32_LONG_FMT=$u32_long
 WA_DIVIDE=$wa_divide
 GENPROTIMG=${GENPROTIMG-genprotimg}
 HOST_KEY_DOCUMENT=$host_key_document
+TARGET_EFI=$target_efi
 EOF
 if [ "$arch" = "arm" ] || [ "$arch" = "arm64" ]; then
     echo "TARGET=$target" >> config.mak
lib/efi.c +25 -7
index 7f08f0e..f3214b8 100644
@@ -2,12 +2,22 @@
  * EFI-related functions to set up and run test cases in EFI
  *
  * Copyright (c) 2021, SUSE, Varad Gautam <[email protected]>
+ * Copyright (c) 2021, Google Inc, Zixuan Wang <[email protected]>
  *
  * SPDX-License-Identifier: LGPL-2.0-or-later
  */
-#include <linux/efi.h>
 
-unsigned long __efiapi efi_main(efi_handle_t handle, efi_system_table_t *sys_tab);
+#include "efi.h"
+#include <libcflat.h>
+#include <asm/setup.h>
+
+/* From lib/argv.c */
+extern int __argc, __envc;
+extern char *__argv[100];
+extern char *__environ[200];
+
+extern int main(int argc, char **argv, char **envp);
+
 efi_system_table_t *efi_system_table = NULL;
 
 static void efi_free_pool(void *ptr)
@@ -15,7 +25,7 @@ static void efi_free_pool(void *ptr)
 	efi_bs_call(free_pool, ptr);
 }
 
-static efi_status_t efi_get_memory_map(struct efi_boot_memmap *map)
+efi_status_t efi_get_memory_map(struct efi_boot_memmap *map)
 {
 	efi_memory_desc_t *m = NULL;
 	efi_status_t status;
@@ -53,15 +63,23 @@ out:
 	return status;
 }
 
-static efi_status_t efi_exit_boot_services(void *handle,
-					   struct efi_boot_memmap *map)
+efi_status_t efi_exit_boot_services(void *handle, struct efi_boot_memmap *map)
 {
 	return efi_bs_call(exit_boot_services, handle, *map->key_ptr);
 }
 
-unsigned long __efiapi efi_main(efi_handle_t handle, efi_system_table_t *sys_tab)
+efi_status_t efi_main(efi_handle_t handle, efi_system_table_t *sys_tab)
 {
+	int ret;
+
 	efi_system_table = sys_tab;
 
-	return 0;
+	setup_efi();
+	ret = main(__argc, __argv, __environ);
+
+	/* Shutdown the guest VM */
+	efi_rs_call(reset_system, EFI_RESET_SHUTDOWN, ret, 0, NULL);
+
+	/* Unreachable */
+	return EFI_UNSUPPORTED;
 }
lib/efi.h new file +21 -0
new file mode 100644
index 0000000..889de18
@@ -0,0 +1,21 @@
+#ifndef _EFI_H_
+#define _EFI_H_
+
+/*
+ * EFI-related functions in . This file's name "efi.h" is in
+ * conflict with GNU-EFI library's "efi.h", but  does not include
+ * GNU-EFI headers or links against GNU-EFI.
+ *
+ * Copyright (c) 2021, Google Inc, Zixuan Wang <[email protected]>
+ *
+ * SPDX-License-Identifier: LGPL-2.0-or-later
+ */
+#include "linux/efi.h"
+#include <elf.h>
+
+efi_status_t _relocate(long ldbase, Elf64_Dyn *dyn, efi_handle_t handle, efi_system_table_t *sys_tab);
+efi_status_t efi_get_memory_map(struct efi_boot_memmap *map);
+efi_status_t efi_exit_boot_services(void *handle, struct efi_boot_memmap *map);
+efi_status_t efi_main(efi_handle_t handle, efi_system_table_t *sys_tab);
+
+#endif /* _EFI_H_ */
lib/linux/efi.h +8 -2
index bb4601b..3d68c28 100644
@@ -4,6 +4,12 @@
 #ifndef __LINUX_UEFI_H
 #define __LINUX_UEFI_H
 
+#include "libcflat.h"
+
+#ifndef __packed
+# define __packed		__attribute__((__packed__))
+#endif
+
 #define BITS_PER_LONG 64
 
 #define EFI_SUCCESS		0
@@ -512,7 +518,7 @@ struct efi_boot_memmap {
 	unsigned long           *buff_size;
 };
 
-#define efi_bs_call(func, ...)						\
-	efi_system_table->boottime->func(__VA_ARGS__)
+#define efi_bs_call(func, ...) efi_system_table->boottime->func(__VA_ARGS__)
+#define efi_rs_call(func, ...) efi_system_table->runtime->func(__VA_ARGS__)
 
 #endif /* __LINUX_UEFI_H */
lib/x86/asm/setup.h +7 -0
index 19ded12..a1f869c 100644
@@ -5,4 +5,11 @@
 unsigned long setup_tss(void);
 #endif /* __x86_64__ */
 
+#ifdef TARGET_EFI
+#include "x86/apic.h"
+#include "x86/smp.h"
+
+void setup_efi(void);
+#endif /* TARGET_EFI */
+
 #endif /* _X86_ASM_SETUP_H_ */
lib/x86/setup.c +13 -0
index 8c73156..2fbf04f 100644
@@ -161,6 +161,19 @@ void setup_multiboot(struct mbi_bootinfo *bi)
 	initrd_size = mods->end - mods->start;
 }
 
+#ifdef TARGET_EFI
+
+void setup_efi(void)
+{
+	reset_apic();
+	mask_pic_interrupts();
+	enable_apic();
+	enable_x2apic();
+	smp_init();
+}
+
+#endif /* TARGET_EFI */
+
 void setup_libcflat(void)
 {
 	if (initrd) {
x86/Makefile.common +50 -17

98 lines not on this page · raw message

x86/Makefile.i386 +3 -2
index 960e274..340c561 100644
@@ -1,11 +1,12 @@
 cstart.o = $(TEST_DIR)/cstart.o
 bits = 32
 ldarch = elf32-i386
+exe = flat
 COMMON_CFLAGS += -mno-sse -mno-sse2
 
 cflatobjs += lib/x86/setjmp32.o lib/ldiv32.o
 
-tests = $(TEST_DIR)/taskswitch.flat $(TEST_DIR)/taskswitch2.flat \
-	$(TEST_DIR)/cmpxchg8b.flat $(TEST_DIR)/la57.flat
+tests = $(TEST_DIR)/taskswitch.$(exe) $(TEST_DIR)/taskswitch2.$(exe) \
+	$(TEST_DIR)/cmpxchg8b.$(exe) $(TEST_DIR)/la57.$(exe)
 
 include $(SRCDIR)/$(TEST_DIR)/Makefile.common
x86/Makefile.x86_64 +35 -19

70 lines not on this page · raw message

x86/efi/README.md +39 -1
index bc1f733..d62758c 100644
@@ -1,4 +1,38 @@
-# EFI Startup Code and Linker Script
+# Build kvm-unit-tests and run under UEFI
+
+## Introduction
+
+This dir provides code to build kvm-unit-tests test cases and run them under
+QEMU and UEFI.
+
+### Install dependencies
+
+The following dependencies should be installed:
+
+- [UEFI firmware](https://github.com/tianocore/edk2): to run test cases in QEMU
+
+### Build
+
+To build:
+
+    ./configure --target-efi
+    make
+
+### Run test cases with UEFI
+
+To run a test case with UEFI:
+
+    ./x86/efi/run ./x86/dummy.efi
+
+By default the runner script loads the UEFI firmware `/usr/share/ovmf/OVMF.fd`;
+please install UEFI firmware to this path, or specify the correct path through
+the env variable `EFI_UEFI`:
+
+    EFI_UEFI=/path/to/OVMF.fd ./x86/efi/run ./x86/dummy.efi
+
+## Code structure
+
+### Code from GNU-EFI
 
 This dir contains source code and linker script copied from
 [GNU-EFI](https://sourceforge.net/projects/gnu-efi/):
@@ -23,3 +57,7 @@ Relocatable Binaries".
 
 kvm-unit-tests follows a similar build process, but does not link with GNU-EFI
 library.
+### Startup code for kvm-unit-tests in UEFI
+
+This dir also contains kvm-unit-tests startup code in UEFI:
+   - efistart64.S: startup code for kvm-unit-tests in UEFI
x86/efi/reloc_x86_64.c +2 -6
index d13b53e..7b6068e 100644
@@ -37,14 +37,10 @@
     SUCH DAMAGE.
 */
 
-#include <efi.h>
-#include <efilib.h>
-
+#include "efi.h"
 #include <elf.h>
 
-EFI_STATUS _relocate (long ldbase, Elf64_Dyn *dyn,
-		      EFI_HANDLE image EFI_UNUSED,
-		      EFI_SYSTEM_TABLE *systab EFI_UNUSED)
+efi_status_t _relocate(long ldbase, Elf64_Dyn *dyn, efi_handle_t handle, efi_system_table_t *sys_tab)
 {
 	long relsz = 0, relent = 0;
 	Elf64_Rel *rel = 0;
x86/efi/run new file +63 -0

63 lines not on this page · raw message

x86/run +14 -2
index 8b2425f..4eba2b9 100755
@@ -38,7 +38,19 @@ else
 fi
 
 command="${qemu} --no-reboot -nodefaults $pc_testdev -vnc none -serial stdio $pci_testdev"
-command+=" -machine accel=$ACCEL -kernel"
+command+=" -machine accel=$ACCEL"
+if ! [ "$EFI_RUN" ]; then
+	command+=" -kernel"
+fi
 command="$(timeout_cmd) $command"
 
-run_qemu ${command} "$@"
+if [ "$EFI_RUN" ]; then
+	# Set ENVIRON_DEFAULT=n to remove '-initrd' flag for QEMU (see
+	# 'scripts/arch-run.bash' for more details). This is because when using
+	# UEFI, the test case binaries are passed to QEMU through the disk
+	# image, not through the '-kernel' flag. And QEMU reports an error if it
+	# gets '-initrd' without a '-kernel'
+	ENVIRON_DEFAULT=n run_qemu ${command} "$@"
+else
+	run_qemu ${command} "$@"
+fi
Signature · 1 line
2.33.0
maintainer
#9 lore raw

Re: [kvm-unit-tests PATCH v3 05/17] x86 UEFI: Boot from UEFI

Quoted 43 lines · Zixuan Wang

From: Zixuan Wang <[email protected]>

This commit provides initial support for x86 test cases to boot from UEFI:

1. UEFI compiler flags are added to Makefile
2. A new TARGET_EFI macro is added to turn on/off UEFI startup code
3. Previous Multiboot setup code is refactored and updated for supporting UEFI, including the following changes:
1. x86/efi/crt0-efi-x86_64.S: provides entry point and jumps to setup code in lib/efi.c.
2. lib/efi.c: performs UEFI setup, calls arch-related setup functions, then jumps to test case main() function
3. lib/x86/setup.c: provides arch-related setup under UEFI

To build test cases for UEFI, please first install the GNU-EFI library. Check x86/efi/README.md for more details.

This commit is tested by a simple test calling report() and report_summayr(). This commit does not include such a test to avoid unnecessary files added into git history. To build and run this test in UEFI (assuming file name is x86/dummy.c):

    ./configure --target-efi
    make x86/dummy.efi
    ./x86/efi/run ./x86/dummy.efi

To use the default Multiboot instead of UEFI:

    ./configure
    make x86/dummy.flat
    ./x86/run ./x86/dummy.flat

Some x86 test cases require additional fixes to work in UEFI, e.g., converting to position independent code (PIC), setting up page tables, etc. This commit does not provide these fixes, so compiling and running UEFI test cases other than x86/dummy.c may trigger compiler errors or QEMU crashes. These test cases will be fixed by the follow-up commits in this series.

The following code is ported from github.com/rhdrjones/kvm-unit-tests
- ./configure: 'target-efi'-related code

Quoted 7 lines

See original code:
- Repo: https://github.com/rhdrjones/kvm-unit-tests
- Branch: target-efi

This is missing

x86/Makefile.i386 +1 -0
index 340c561..7d19d55 100644
@@ -2,6 +2,7 @@ cstart.o = $(TEST_DIR)/cstart.o
 bits = 32
 ldarch = elf32-i386
 exe = flat
+bin = elf
 COMMON_CFLAGS += -mno-sse -mno-sse2
 
 cflatobjs += lib/x86/setjmp32.o lib/ldiv32.o

for 32-bit tests to build; and also:

configure +10 -1
index b6c09b3..6f4a8f0 100755
@@ -265,6 +265,11 @@ if [ -f "$srcdir/$testdir/run" ]; then
     ln -fs "$srcdir/$testdir/run" $testdir-run
 fi
 
+testsubdir=$testdir
+if [ -n "$target_efi" ]; then
+    testsubdir=$testdir/efi
+fi
+
 # check if uint32_t needs a long format modifier
 cat << EOF > lib-test.c
 __UINT32_TYPE__
@@ -291,8 +301,11 @@ if test ! -e Makefile; then
     ln -s "$srcdir/Makefile" .
 
     echo "linking tests..."
-    mkdir -p $testdir
+    mkdir -p $testsubdir
     ln -sf "$srcdir/$testdir/run" $testdir/
+    if test "$testdir" != "$testsubdir"; then
+        ln -sf "$srcdir/$testsubdir/run" $testsubdir/
+    fi
     ln -sf "$srcdir/$testdir/unittests.cfg" $testdir/
     ln -sf "$srcdir/run_tests.sh"
 
@@ -332,6 +345,7 @@ OBJDUMP=$cross_prefix$objdump
 AR=$cross_prefix$ar
 ADDR2LINE=$cross_prefix$addr2line
 TEST_DIR=$testdir
+TEST_SUBDIR=$testsubdir
 FIRMWARE=$firmware
 ENDIAN=$endian
 PRETTY_PRINT_STACKS=$pretty_print_stacks
run_tests.sh +1 -1
index 9f233c5..f61e005 100755
@@ -31,7 +31,7 @@ specify the appropriate qemu binary for ARCH-run.
 EOF
 }
 
-RUNTIME_arch_run="./$TEST_DIR/run"
+RUNTIME_arch_run="./$TEST_SUBDIR/run"
 source scripts/runtime.bash
 
 # require enhanced getopt

As of this patch tests don't seem to work correctly, but at least the build machinery works (they build and ./x86/efi/run starts them).

./run_tests.sh does not work, either.

Paolo

maintainer
#10 lore raw

Re: [kvm-unit-tests PATCH v3 05/17] x86 UEFI: Boot from UEFI

In reply to Zixuan Wang

x86/efi/run line 58

Quoted 3 lines · Zixuan Wang

+"$TEST_DIR/run" \

x86/efi/run new file +2 -0
+	-drive file="$EFI_UEFI",format=raw,if=pflash \
+	-drive file.dir="$EFI_TEST/$EFI_CASE/",file.driver=vvfat,file.rw=on,format=raw \

For what it's worth, SEV doesn't work for me unless I add ",if=virtio" to the second -drive option. Since it's a good idea anyway, I squashed it into the patch.

Paolo

#11 lore raw

PATCH v3 6/17 x86 UEFI: Load IDT after UEFI boot up

In reply to Zixuan Wang

From: Zixuan Wang <[email protected]>

Interrupt descriptor table (IDT) is used by x86 arch to describe the interrupt handlers. UEFI already setup an IDT before running the test binaries, but this IDT is not compatible with the existing KVM-Unit-Tests test cases, e.g., x86/msr.c triggers a #GP fault when using UEFI IDT. This is because test cases setup new interrupt handlers and register them into KVM-Unit-Tests' IDT, but it takes no effect if we do not load KVM-Unit-Tests' IDT.

This commit fixes this issue by setting up and loding the IDT.

In this commit, the x86/msr.c can run in UEFI and generates same output as the default Seabios version.

---
 lib/x86/setup.c            |  5 +++++
 x86/Makefile.x86_64        |  2 +-
 x86/efi/README.md          |  4 ++--
 x86/efi/efistart64.S       | 12 ++++++++++++
 x86/efi/elf_x86_64_efi.lds |  6 +++++-
 5 files changed, 25 insertions(+), 4 deletions(-)
 create mode 100644 x86/efi/efistart64.S
lib/x86/setup.c +5 -0
index 2fbf04f..8606fe8 100644
@@ -163,9 +163,14 @@ void setup_multiboot(struct mbi_bootinfo *bi)
 
 #ifdef TARGET_EFI
 
+/* From x86/efi/efistart64.S */
+extern void load_idt(void);
+
 void setup_efi(void)
 {
 	reset_apic();
+	setup_idt();
+	load_idt();
 	mask_pic_interrupts();
 	enable_apic();
 	enable_x2apic();
x86/Makefile.x86_64 +1 -1
index a5f8923..aa23b22 100644
@@ -5,7 +5,7 @@ ifeq ($(TARGET_EFI),y)
 exe = efi
 bin = so
 FORMAT = efi-app-x86_64
-cstart.o = x86/efi/crt0-efi-x86_64.o
+cstart.o = $(TEST_DIR)/efi/efistart64.o
 else
 exe = flat
 bin = elf
x86/efi/README.md +2 -2
index d62758c..a39f509 100644
@@ -22,13 +22,13 @@ To build:
 
 To run a test case with UEFI:
 
-    ./x86/efi/run ./x86/dummy.efi
+    ./x86/efi/run ./x86/msr.efi
 
 By default the runner script loads the UEFI firmware `/usr/share/ovmf/OVMF.fd`;
 please install UEFI firmware to this path, or specify the correct path through
 the env variable `EFI_UEFI`:
 
-    EFI_UEFI=/path/to/OVMF.fd ./x86/efi/run ./x86/dummy.efi
+    EFI_UEFI=/path/to/OVMF.fd ./x86/efi/run ./x86/msr.efi
 
 ## Code structure
 
x86/efi/efistart64.S new file +12 -0
new file mode 100644
index 0000000..41e9185
@@ -0,0 +1,12 @@
+/* Startup code and pre-defined data structures */
+
+#include "crt0-efi-x86_64.S"
+
+.section .init
+.code64
+.text
+
+.globl load_idt
+load_idt:
+	lidtq idt_descr(%rip)
+	retq
x86/efi/elf_x86_64_efi.lds +5 -1
index 5eae376..3d92c86 100644
@@ -1,4 +1,4 @@
-/* Copied from GNU-EFI/gnuefi/elf_x86_64_efi.lds, licensed under GNU GPL */
+/* Developed based on GNU-EFI/gnuefi/elf_x86_64_efi.lds, licensed under GNU GPL */
 /* Same as elf_x86_64_fbsd_efi.lds, except for OUTPUT_FORMAT below - KEEP IN SYNC */
 OUTPUT_FORMAT("elf64-x86-64", "elf64-x86-64", "elf64-x86-64")
 OUTPUT_ARCH(i386:x86-64)
@@ -38,6 +38,10 @@ SECTIONS
    *(.rodata*)
    *(.got.plt)
    *(.got)
+   /* Expected by lib/x86/desc.c to store exception_table */
+   exception_table_start = .;
+   *(.data.ex)
+   exception_table_end = .;
    *(.data*)
    *(.sdata)
    /* the EFI loader doesn't seem to like a .bss section, so we stick
Signature · 1 line
2.33.0
#12 lore raw

PATCH v3 7/17 x86 UEFI: Load GDT and TSS after UEFI boot up

In reply to Zixuan Wang

From: Zixuan Wang <[email protected]>

Global Descriptor Table (GDT) defines x86 memory areas, e.g. memory area for data or code. UEFI has a different GDT compared to KVM-Unit-Tests, e.g., in UEFI, 3rd GDT entry defines a code segment, while in KVM-Unit-Tests, 3rd GDT entry is data segment.

Without loading KVM-Unit-Tests GDT, the UEFI GDT is used and is incompatible with KVM-Unit-Tests. This causes QEMU to silently crash when a test case changes segments.

This commit fixes this issue by loading KVM-Unit-Tests GDT to replace UEFI GDT. And since Task State Segment (TSS) is tightly coupled with GDT, this commit also loads TSS on boot-up.

In this commit, x86/debug.c can run in UEFI and pass all sub-tests.

---
 lib/x86/setup.c      | 10 ++++++++++
 x86/efi/efistart64.S | 42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 52 insertions(+)
lib/x86/setup.c +10 -0
index 8606fe8..f4f9e1b 100644
@@ -165,10 +165,20 @@ void setup_multiboot(struct mbi_bootinfo *bi)
 
 /* From x86/efi/efistart64.S */
 extern void load_idt(void);
+extern void load_gdt_tss(size_t tss_offset);
+
+static void setup_gdt_tss(void)
+{
+	size_t tss_offset;
+
+	tss_offset = setup_tss();
+	load_gdt_tss(tss_offset);
+}
 
 void setup_efi(void)
 {
 	reset_apic();
+	setup_gdt_tss();
 	setup_idt();
 	load_idt();
 	mask_pic_interrupts();
x86/efi/efistart64.S +42 -0

52 lines not on this page · raw message

Signature · 1 line
2.33.0
#13 lore raw

PATCH v3 8/17 x86 UEFI: Set up memory allocator

In reply to Zixuan Wang

From: Zixuan Wang <[email protected]>

KVM-Unit-Tests library implements a memory allocator which requires two arguments to set up (See lib/alloc_phys.c:phys_alloc_init() for more details):
1. A base (start) physical address
2. Size of available memory for allocation

To get this memory info, we scan all the memory regions returned by LibMemoryMap(), find out the largest free memory region and use it for memory allocation.

After retrieving this memory info, we call ExitBootServices so that KVM-Unit-Tests has full control of the machine, and UEFI will not touch the memory after this point.

Starting from this commit, x86/hypercall.c test case can run in UEFI and generates the same output as in Seabios.

---
 lib/efi.c           | 28 +++++++++++++---
 lib/efi.h           |  2 +-
 lib/x86/asm/setup.h | 16 +++++++++-
 lib/x86/setup.c     | 78 ++++++++++++++++++++++++++++++++++++++++++++-
 4 files changed, 116 insertions(+), 8 deletions(-)
lib/efi.c +23 -5
index f3214b8..c1c3806 100644
@@ -30,9 +30,10 @@ efi_status_t efi_get_memory_map(struct efi_boot_memmap *map)
 	efi_memory_desc_t *m = NULL;
 	efi_status_t status;
 	unsigned long key = 0, map_size = 0, desc_size = 0;
+	u32 desc_ver;
 
 	status = efi_bs_call(get_memory_map, &map_size,
-			     NULL, &key, &desc_size, NULL);
+			     NULL, &key, &desc_size, &desc_ver);
 	if (status != EFI_BUFFER_TOO_SMALL || map_size == 0)
 		goto out;
 
@@ -49,12 +50,13 @@ efi_status_t efi_get_memory_map(struct efi_boot_memmap *map)
 
 	/* Get the map. */
 	status = efi_bs_call(get_memory_map, &map_size,
-			     m, &key, &desc_size, NULL);
+			     m, &key, &desc_size, &desc_ver);
 	if (status != EFI_SUCCESS) {
 		efi_free_pool(m);
 		goto out;
 	}
 
+	*map->desc_ver = desc_ver;
 	*map->desc_size = desc_size;
 	*map->map_size = map_size;
 	*map->key_ptr = key;
@@ -63,18 +65,34 @@ out:
 	return status;
 }
 
-efi_status_t efi_exit_boot_services(void *handle, struct efi_boot_memmap *map)
+efi_status_t efi_exit_boot_services(void *handle, unsigned long mapkey)
 {
-	return efi_bs_call(exit_boot_services, handle, *map->key_ptr);
+	return efi_bs_call(exit_boot_services, handle, mapkey);
 }
 
 efi_status_t efi_main(efi_handle_t handle, efi_system_table_t *sys_tab)
 {
 	int ret;
+	unsigned long mapkey = 0;
+	efi_status_t status;
+	efi_bootinfo_t efi_bootinfo;
 
 	efi_system_table = sys_tab;
 
-	setup_efi();
+	setup_efi_bootinfo(&efi_bootinfo);
+	status = setup_efi_pre_boot(&mapkey, &efi_bootinfo);
+	if (status != EFI_SUCCESS) {
+		printf("Failed to set up before ExitBootServices, exiting.\n");
+		return status;
+	}
+
+	status = efi_exit_boot_services(handle, mapkey);
+	if (status != EFI_SUCCESS) {
+		printf("Failed to exit boot services\n");
+		return status;
+	}
+
+	setup_efi(&efi_bootinfo);
 	ret = main(__argc, __argv, __environ);
 
 	/* Shutdown the guest VM */
lib/efi.h +1 -1
index 889de18..0f1dafd 100644
@@ -15,7 +15,7 @@
 
 efi_status_t _relocate(long ldbase, Elf64_Dyn *dyn, efi_handle_t handle, efi_system_table_t *sys_tab);
 efi_status_t efi_get_memory_map(struct efi_boot_memmap *map);
-efi_status_t efi_exit_boot_services(void *handle, struct efi_boot_memmap *map);
+efi_status_t efi_exit_boot_services(void *handle, unsigned long mapkey);
 efi_status_t efi_main(efi_handle_t handle, efi_system_table_t *sys_tab);
 
 #endif /* _EFI_H_ */
lib/x86/asm/setup.h +15 -1
index a1f869c..3f0a870 100644
@@ -8,8 +8,22 @@ unsigned long setup_tss(void);
 #ifdef TARGET_EFI
 #include "x86/apic.h"
 #include "x86/smp.h"
+#include "efi.h"
 
-void setup_efi(void);
+/*
+ * efi_bootinfo_t: stores EFI-related machine info retrieved by
+ * setup_efi_pre_boot(), and is then used by setup_efi(). setup_efi() cannot
+ * retrieve this info as it is called after ExitBootServices and thus some EFI
+ * resources are not available.
+ */
+typedef struct {
+	phys_addr_t free_mem_start;
+	phys_addr_t free_mem_size;
+} efi_bootinfo_t;
+
+void setup_efi_bootinfo(efi_bootinfo_t *efi_bootinfo);
+void setup_efi(efi_bootinfo_t *efi_bootinfo);
+efi_status_t setup_efi_pre_boot(unsigned long *mapkey, efi_bootinfo_t *efi_bootinfo);
 #endif /* TARGET_EFI */
 
 #endif /* _X86_ASM_SETUP_H_ */
lib/x86/setup.c +77 -1

96 lines not on this page · raw message

Signature · 1 line
2.33.0
#14 lore raw

PATCH v3 9/17 x86 UEFI: Set up RSDP after UEFI boot up

In reply to Zixuan Wang

From: Zixuan Wang <[email protected]>

Root system description pointer (RSDP) is a data structure used in the ACPI programming interface. In BIOS, RSDP is located within a predefined memory area, so a program can scan the memory area and find RSDP. But in UEFI, RSDP may not appear in that memory area, instead, a program should find it in the EFI system table.

This commit provides RSDP set up code in UEFI:
1. Read RSDP from EFI system table
2. Pass RSDP pointer to find_acpi_table_attr() function

From this commit, the x86/s3.c test can run in UEFI and generates similar output as in Seabios, note that:
1. In its output, memory addresses are different than Seabios's, this is because EFI application starts from a dynamic runtime address, not a fixed predefined memory address
2. There is a short delay (~5 secs) after the test case prints "PM1a event registers" line. This test case sleeps for a few seconds and then wakes up, so give it a few seconds to run.

---
 lib/efi.c           | 15 +++++++++++++++
 lib/efi.h           |  1 +
 lib/linux/efi.h     | 15 +++++++++++++++
 lib/x86/acpi.c      | 38 +++++++++++++++++++++++++++++++-------
 lib/x86/acpi.h      | 11 +++++++++++
 lib/x86/asm/setup.h |  2 ++
 lib/x86/setup.c     | 13 +++++++++++++
 7 files changed, 88 insertions(+), 7 deletions(-)
lib/efi.c +15 -0
index c1c3806..9506830 100644
@@ -70,6 +70,21 @@ efi_status_t efi_exit_boot_services(void *handle, unsigned long mapkey)
 	return efi_bs_call(exit_boot_services, handle, mapkey);
 }
 
+efi_status_t efi_get_system_config_table(efi_guid_t table_guid, void **table)
+{
+	size_t i;
+	efi_config_table_t *tables;
+
+	tables = (efi_config_table_t *)efi_system_table->tables;
+	for (i = 0; i < efi_system_table->nr_tables; i++) {
+		if (!memcmp(&table_guid, &tables[i].guid, sizeof(efi_guid_t))) {
+			*table = tables[i].table;
+			return EFI_SUCCESS;
+		}
+	}
+	return EFI_NOT_FOUND;
+}
+
 efi_status_t efi_main(efi_handle_t handle, efi_system_table_t *sys_tab)
 {
 	int ret;
lib/efi.h +1 -0
index 0f1dafd..1b3abd0 100644
@@ -16,6 +16,7 @@
 efi_status_t _relocate(long ldbase, Elf64_Dyn *dyn, efi_handle_t handle, efi_system_table_t *sys_tab);
 efi_status_t efi_get_memory_map(struct efi_boot_memmap *map);
 efi_status_t efi_exit_boot_services(void *handle, unsigned long mapkey);
+efi_status_t efi_get_system_config_table(efi_guid_t table_guid, void **table);
 efi_status_t efi_main(efi_handle_t handle, efi_system_table_t *sys_tab);
 
 #endif /* _EFI_H_ */
lib/linux/efi.h +15 -0
index 3d68c28..7ac1082 100644
@@ -58,6 +58,21 @@ typedef guid_t efi_guid_t;
 	(b) & 0xff, ((b) >> 8) & 0xff,						\
 	(c) & 0xff, ((c) >> 8) & 0xff, d } }
 
+#define ACPI_TABLE_GUID EFI_GUID(0xeb9d2d30, 0x2d88, 0x11d3, 0x9a, 0x16, 0x00, 0x90, 0x27, 0x3f, 0xc1, 0x4d)
+
+typedef struct {
+	efi_guid_t guid;
+	u32 table;
+} efi_config_table_32_t;
+
+typedef union {
+	struct {
+		efi_guid_t guid;
+		void *table;
+	};
+	efi_config_table_32_t mixed_mode;
+} efi_config_table_t;
+
 /*
  * Generic EFI table header
  */
lib/x86/acpi.c +31 -7
index 4373106..0f75d79 100644
@@ -1,9 +1,37 @@
 #include "libcflat.h"
 #include "acpi.h"
 
+#ifdef TARGET_EFI
+struct rsdp_descriptor *efi_rsdp = NULL;
+
+void setup_efi_rsdp(struct rsdp_descriptor *rsdp) {
+	efi_rsdp = rsdp;
+}
+
+static struct rsdp_descriptor *get_rsdp(void) {
+	if (efi_rsdp == NULL) {
+		printf("Can't find RSDP from UEFI, maybe setup_efi_rsdp() was not called\n");
+	}
+	return efi_rsdp;
+}
+#else
+static struct rsdp_descriptor *get_rsdp(void) {
+    struct rsdp_descriptor *rsdp;
+    unsigned long addr;
+    for(addr = 0xf0000; addr < 0x100000; addr += 16) {
+	rsdp = (void*)addr;
+	if (rsdp->signature == RSDP_SIGNATURE_8BYTE)
+          break;
+    }
+    if (addr == 0x100000) {
+        return NULL;
+    }
+    return rsdp;
+}
+#endif /* TARGET_EFI */
+
 void* find_acpi_table_addr(u32 sig)
 {
-    unsigned long addr;
     struct rsdp_descriptor *rsdp;
     struct rsdt_descriptor_rev1 *rsdt;
     void *end;
@@ -19,12 +47,8 @@ void* find_acpi_table_addr(u32 sig)
         return (void*)(ulong)fadt->firmware_ctrl;
     }
 
-    for(addr = 0xf0000; addr < 0x100000; addr += 16) {
-	rsdp = (void*)addr;
-	if (rsdp->signature == 0x2052545020445352LL)
-          break;
-    }
-    if (addr == 0x100000) {
+    rsdp = get_rsdp();
+    if (rsdp == NULL) {
         printf("Can't find RSDP\n");
         return 0;
     }
lib/x86/acpi.h +11 -0
index 1b80374..db8ee56 100644
@@ -11,6 +11,13 @@
 #define FACP_SIGNATURE ACPI_SIGNATURE('F','A','C','P')
 #define FACS_SIGNATURE ACPI_SIGNATURE('F','A','C','S')
 
+
+#define ACPI_SIGNATURE_8BYTE(c1, c2, c3, c4, c5, c6, c7, c8) \
+	((uint64_t)(ACPI_SIGNATURE(c1, c2, c3, c4))) |       \
+	((uint64_t)(ACPI_SIGNATURE(c5, c6, c7, c8)) << 32)
+
+#define RSDP_SIGNATURE_8BYTE (ACPI_SIGNATURE_8BYTE('R', 'S', 'D', ' ', 'P', 'T', 'R', ' '))
+
 struct rsdp_descriptor {        /* Root System Descriptor Pointer */
     u64 signature;              /* ACPI signature, contains "RSD PTR " */
     u8  checksum;               /* To make sum of struct == 0 */
@@ -101,4 +108,8 @@ struct facs_descriptor_rev1
 
 void* find_acpi_table_addr(u32 sig);
 
+#ifdef TARGET_EFI
+void setup_efi_rsdp(struct rsdp_descriptor *rsdp);
+#endif /* TARGET_EFI */
+
 #endif
lib/x86/asm/setup.h +2 -0
index 3f0a870..ecfcd5c 100644
@@ -6,6 +6,7 @@ unsigned long setup_tss(void);
 #endif /* __x86_64__ */
 
 #ifdef TARGET_EFI
+#include "x86/acpi.h"
 #include "x86/apic.h"
 #include "x86/smp.h"
 #include "efi.h"
@@ -19,6 +20,7 @@ unsigned long setup_tss(void);
 typedef struct {
 	phys_addr_t free_mem_start;
 	phys_addr_t free_mem_size;
+	struct rsdp_descriptor *rsdp;
 } efi_bootinfo_t;
 
 void setup_efi_bootinfo(efi_bootinfo_t *efi_bootinfo);
lib/x86/setup.c +13 -0
index 90f95a3..6d81ab6 100644
@@ -171,6 +171,7 @@ void setup_efi_bootinfo(efi_bootinfo_t *efi_bootinfo)
 {
 	efi_bootinfo->free_mem_size = 0;
 	efi_bootinfo->free_mem_start = 0;
+	efi_bootinfo->rsdp = NULL;
 }
 
 static efi_status_t setup_pre_boot_memory(unsigned long *mapkey, efi_bootinfo_t *efi_bootinfo)
@@ -221,6 +222,11 @@ static efi_status_t setup_pre_boot_memory(unsigned long *mapkey, efi_bootinfo_t
 	return EFI_SUCCESS;
 }
 
+static efi_status_t setup_pre_boot_rsdp(efi_bootinfo_t *efi_bootinfo)
+{
+	return efi_get_system_config_table(ACPI_TABLE_GUID, (void **)&efi_bootinfo->rsdp);
+}
+
 efi_status_t setup_efi_pre_boot(unsigned long *mapkey, efi_bootinfo_t *efi_bootinfo)
 {
 	efi_status_t status;
@@ -239,6 +245,12 @@ efi_status_t setup_efi_pre_boot(unsigned long *mapkey, efi_bootinfo_t *efi_booti
 		return status;
 	}
 
+	status = setup_pre_boot_rsdp(efi_bootinfo);
+	if (status != EFI_SUCCESS) {
+		printf("Cannot find RSDP in EFI system table\n");
+		return status;
+	}
+
 	return EFI_SUCCESS;
 }
 
@@ -261,6 +273,7 @@ void setup_efi(efi_bootinfo_t *efi_bootinfo)
 	enable_x2apic();
 	smp_init();
 	phys_alloc_init(efi_bootinfo->free_mem_start, efi_bootinfo->free_mem_size);
+	setup_efi_rsdp(efi_bootinfo->rsdp);
 }
 
 #endif /* TARGET_EFI */
Signature · 1 line
2.33.0
#15 lore raw

PATCH v3 10/17 x86 UEFI: Set up page tables

In reply to Zixuan Wang

From: Zixuan Wang <[email protected]>

UEFI sets up page tables before executing EFI application binaries. These page tables do not allow user space code to access kernel space memory. But x86/syscall.c test case places a user space function syscall_tf_user32 inside kernel space memory. When using UEFI page tables, fetching this kernel memory from user space triggers a #PF fault, which is not expected by this test case.

KVM-Unit-Tests defines page tables that allow such behavior. So the solution to this problem is to load KVM-Unit-Tests' page tables:

1. Copy the page table definition from x86/cstart64.S
2. Update page table entries with runtime memory addresses
3. Update CR3 register with the new page table root address

Since this commit, x86/syscall.c can run in UEFI and generate same output as in Seabios, using the following command:

   ./x86/efi/run ./x86/syscall.efi --cpu Opteron_G1,vendor=AuthenticAMD
---
 lib/x86/asm/page.h   |  1 +
 lib/x86/asm/setup.h  |  3 +++
 lib/x86/setup.c      | 55 ++++++++++++++++++++++++++++++++++++++++++++
 x86/efi/efistart64.S | 23 ++++++++++++++++++
 4 files changed, 82 insertions(+)
lib/x86/asm/page.h +1 -0
index fc14160..f6f740b 100644
@@ -31,6 +31,7 @@ typedef unsigned long pgd_t;
 #define PT_ACCESSED_MASK	(1ull << 5)
 #define PT_DIRTY_MASK		(1ull << 6)
 #define PT_PAGE_SIZE_MASK	(1ull << 7)
+#define PT_GLOBAL_MASK		(1ull << 8)
 #define PT64_NX_MASK		(1ull << 63)
 #define PT_ADDR_MASK		GENMASK_ULL(51, 12)
 
lib/x86/asm/setup.h +3 -0
index ecfcd5c..9cc135a 100644
@@ -8,7 +8,9 @@ unsigned long setup_tss(void);
 #ifdef TARGET_EFI
 #include "x86/acpi.h"
 #include "x86/apic.h"
+#include "x86/processor.h"
 #include "x86/smp.h"
+#include "asm/page.h"
 #include "efi.h"
 
 /*
@@ -26,6 +28,7 @@ typedef struct {
 void setup_efi_bootinfo(efi_bootinfo_t *efi_bootinfo);
 void setup_efi(efi_bootinfo_t *efi_bootinfo);
 efi_status_t setup_efi_pre_boot(unsigned long *mapkey, efi_bootinfo_t *efi_bootinfo);
+void setup_5level_page_table(void);
 #endif /* TARGET_EFI */
 
 #endif /* _X86_ASM_SETUP_H_ */
lib/x86/setup.c +55 -0

67 lines not on this page · raw message

x86/efi/efistart64.S +23 -0
index 57299a5..adfff3a 100644
@@ -17,6 +17,29 @@ ring0stacksize = PAGE_SIZE
 	.align 16
 ring0stacktop:
 
+.data
+
+.align PAGE_SIZE
+.globl ptl2
+ptl2:
+	. = . + 4 * PAGE_SIZE
+.align PAGE_SIZE
+
+.globl ptl3
+ptl3:
+	. = . + PAGE_SIZE
+.align PAGE_SIZE
+
+.globl ptl4
+ptl4:
+	. = . + PAGE_SIZE
+.align PAGE_SIZE
+
+.globl ptl5
+ptl5:
+	. = . + PAGE_SIZE
+.align PAGE_SIZE
+
 .section .init
 .code64
 .text
Signature · 1 line
2.33.0
#16 lore raw

PATCH v3 11/17 x86 UEFI: Convert x86 test cases to PIC

In reply to Zixuan Wang

From: Zixuan Wang <[email protected]>

UEFI loads EFI applications to dynamic runtime addresses, so it requires all applications to be compiled as PIC (position independent code). PIC does not allow the usage of compile time absolute address.

This commit converts multiple x86 test cases to PIC so they can compile and run in UEFI:

- x86/cet.efi

- x86/emulator.c: x86/emulator.c depends on lib/x86/usermode.c. But usermode.c contains non-PIC inline assembly code. This commit converts lib/x86/usermode.c and x86/emulator.c to PIC, so x86/emulator.c can compile and run in UEFI.

- x86/vmware_backdoors.c: it depends on lib/x86/usermode.c and now works without modifications

- x86/eventinj.c

- x86/smap.c

- x86/access.c

- x86/umip.c

---
 lib/x86/usermode.c  |  3 ++-
 x86/Makefile.common | 10 +++++-----
 x86/Makefile.x86_64 |  7 ++++---
 x86/access.c        |  9 +++++----
 x86/cet.c           |  8 +++++---
 x86/emulator.c      |  5 +++--
 x86/eventinj.c      |  6 ++++--
 x86/smap.c          |  8 ++++----
 x86/umip.c          | 10 +++++++---
 9 files changed, 39 insertions(+), 27 deletions(-)
lib/x86/usermode.c +2 -1
index f032523..c550545 100644
@@ -58,7 +58,8 @@ uint64_t run_in_user(usermode_func func, unsigned int fault_vector,
 			"pushq %[user_stack_top]\n\t"
 			"pushfq\n\t"
 			"pushq %[user_cs]\n\t"
-			"pushq $user_mode\n\t"
+			"lea user_mode(%%rip), %%rdx\n\t"
+			"pushq %%rdx\n\t"
 			"iretq\n"
 
 			"user_mode:\n\t"
x86/Makefile.common +5 -5
index 4859bf3..959379c 100644
@@ -81,16 +81,16 @@ tests-common = $(TEST_DIR)/vmexit.$(exe) $(TEST_DIR)/tsc.$(exe) \
                $(TEST_DIR)/init.$(exe) \
                $(TEST_DIR)/hyperv_synic.$(exe) $(TEST_DIR)/hyperv_stimer.$(exe) \
                $(TEST_DIR)/hyperv_connections.$(exe) \
-               $(TEST_DIR)/tsx-ctrl.$(exe)
+               $(TEST_DIR)/tsx-ctrl.$(exe) \
+	       $(TEST_DIR)/eventinj.$(exe) \
+               $(TEST_DIR)/umip.$(exe)
 
 # The following test cases are disabled when building EFI tests because they
 # use absolute addresses in their inline assembly code, which cannot compile
 # with the '-fPIC' flag
 ifneq ($(TARGET_EFI),y)
-tests-common += $(TEST_DIR)/eventinj.$(exe) \
-                $(TEST_DIR)/smap.$(exe) \
-                $(TEST_DIR)/realmode.$(exe) \
-                $(TEST_DIR)/umip.$(exe)
+tests-common += $(TEST_DIR)/smap.$(exe) \
+                $(TEST_DIR)/realmode.$(exe)
 endif
 
 test_cases: $(tests-common) $(tests)
x86/Makefile.x86_64 +4 -3
index aa23b22..7e8a57a 100644
@@ -30,20 +30,21 @@ tests += $(TEST_DIR)/intel-iommu.$(exe)
 tests += $(TEST_DIR)/rdpru.$(exe)
 tests += $(TEST_DIR)/pks.$(exe)
 tests += $(TEST_DIR)/pmu_lbr.$(exe)
+tests += $(TEST_DIR)/emulator.$(exe)
+tests += $(TEST_DIR)/vmware_backdoors.$(exe)
 
 # The following test cases are disabled when building EFI tests because they
 # use absolute addresses in their inline assembly code, which cannot compile
 # with the '-fPIC' flag
 ifneq ($(TARGET_EFI),y)
 tests += $(TEST_DIR)/access.$(exe)
-tests += $(TEST_DIR)/emulator.$(exe)
 tests += $(TEST_DIR)/svm.$(exe)
 tests += $(TEST_DIR)/vmx.$(exe)
-tests += $(TEST_DIR)/vmware_backdoors.$(exe)
+endif
+
 ifneq ($(fcf_protection_full),)
 tests += $(TEST_DIR)/cet.$(exe)
 endif
-endif
 
 include $(SRCDIR)/$(TEST_DIR)/Makefile.common
 
x86/access.c +5 -4
index 4725bbd..8d620a7 100644
@@ -700,7 +700,7 @@ static int ac_test_do_access(ac_test_t *at)
 
     if (F(AC_ACCESS_TWICE)) {
 	asm volatile (
-	    "mov $fixed2, %%rsi \n\t"
+	    "lea fixed2(%%rip), %%rsi \n\t"
 	    "mov (%[addr]), %[reg] \n\t"
 	    "fixed2:"
 	    : [reg]"=r"(r), [fault]"=a"(fault), "=b"(e)
@@ -710,7 +710,7 @@ static int ac_test_do_access(ac_test_t *at)
 	fault = 0;
     }
 
-    asm volatile ("mov $fixed1, %%rsi \n\t"
+    asm volatile ("lea fixed1(%%rip), %%rsi \n\t"
 		  "mov %%rsp, %%rdx \n\t"
 		  "cmp $0, %[user] \n\t"
 		  "jz do_access \n\t"
@@ -719,7 +719,8 @@ static int ac_test_do_access(ac_test_t *at)
 		  "pushq %[user_stack_top] \n\t"
 		  "pushfq \n\t"
 		  "pushq %[user_cs] \n\t"
-		  "pushq $do_access \n\t"
+		  "lea do_access(%%rip), %%r8\n\t"
+		  "pushq %%r8\n\t"
 		  "iretq \n"
 		  "do_access: \n\t"
 		  "cmp $0, %[fetch] \n\t"
@@ -744,7 +745,7 @@ static int ac_test_do_access(ac_test_t *at)
 		    [user_cs]"i"(USER_CS),
 		    [user_stack_top]"r"(user_stack + sizeof user_stack),
 		    [kernel_entry_vector]"i"(0x20)
-		  : "rsi");
+		  : "rsi", "r8");
 
     asm volatile (".section .text.pf \n\t"
 		  "page_fault: \n\t"
x86/cet.c +5 -3
index a21577a..a4b79cb 100644
@@ -52,7 +52,7 @@ static u64 cet_ibt_func(void)
 	printf("No endbr64 instruction at jmp target, this triggers #CP...\n");
 	asm volatile ("movq $2, %rcx\n"
 		      "dec %rcx\n"
-		      "leaq 2f, %rax\n"
+		      "leaq 2f(%rip), %rax\n"
 		      "jmp *%rax \n"
 		      "2:\n"
 		      "dec %rcx\n");
@@ -67,7 +67,8 @@ void test_func(void) {
 			"pushq %[user_stack_top]\n\t"
 			"pushfq\n\t"
 			"pushq %[user_cs]\n\t"
-			"pushq $user_mode\n\t"
+			"lea user_mode(%%rip), %%rax\n\t"
+			"pushq %%rax\n\t"
 			"iretq\n"
 
 			"user_mode:\n\t"
@@ -77,7 +78,8 @@ void test_func(void) {
 			[user_ds]"i"(USER_DS),
 			[user_cs]"i"(USER_CS),
 			[user_stack_top]"r"(user_stack +
-					sizeof(user_stack)));
+					sizeof(user_stack))
+			: "rax");
 }
 
 #define SAVE_REGS() \
x86/emulator.c +3 -2
index 9fda1a0..4d2de24 100644
@@ -262,12 +262,13 @@ static void test_pop(void *mem)
 
 	asm volatile("mov %%rsp, %[tmp] \n\t"
 		     "mov %[stack_top], %%rsp \n\t"
-		     "push $1f \n\t"
+		     "lea 1f(%%rip), %%rax \n\t"
+		     "push %%rax \n\t"
 		     "ret \n\t"
 		     "2: jmp 2b \n\t"
 		     "1: mov %[tmp], %%rsp"
 		     : [tmp]"=&r"(tmp) : [stack_top]"r"(stack_top)
-		     : "memory");
+		     : "memory", "rax");
 	report(1, "ret");
 
 	stack_top[-1] = 0x778899;
x86/eventinj.c +4 -2
index 46593c9..0cd68e8 100644
@@ -155,9 +155,11 @@ asm("do_iret:"
 	"pushf"W" \n\t"
 	"mov %cs, %ecx \n\t"
 	"push"W" %"R "cx \n\t"
-	"push"W" $2f \n\t"
+	"lea"W" 2f(%"R "ip), %"R "bx \n\t"
+	"push"W" %"R "bx \n\t"
 
-	"cmpb $0, no_test_device\n\t"	// see if need to flush
+	"mov no_test_device(%"R "ip), %bl \n\t"
+	"cmpb $0, %bl\n\t"		// see if need to flush
 	"jnz 1f\n\t"
 	"outl %eax, $0xe4 \n\t"		// flush page
 	"1: \n\t"
x86/smap.c +4 -4
index ac2c8d5..b3ee16f 100644
@@ -161,10 +161,10 @@ int main(int ac, char **av)
 		test = -1;
 		asm("or $(" xstr(USER_BASE) "), %"R "sp \n"
 		    "push $44 \n "
-		    "decl test\n"
+		    "decl test(%"R "ip)\n"
 		    "and $~(" xstr(USER_BASE) "), %"R "sp \n"
 		    "pop %"R "ax\n"
-		    "movl %eax, test");
+		    "movl %eax, test(%"R "ip)");
 		report(pf_count == 0 && test == 44,
 		       "write to user stack with AC=1");
 
@@ -173,10 +173,10 @@ int main(int ac, char **av)
 		test = -1;
 		asm("or $(" xstr(USER_BASE) "), %"R "sp \n"
 		    "push $45 \n "
-		    "decl test\n"
+		    "decl test(%"R "ip)\n"
 		    "and $~(" xstr(USER_BASE) "), %"R "sp \n"
 		    "pop %"R "ax\n"
-		    "movl %eax, test");
+		    "movl %eax, test(%"R "ip)");
 		report(pf_count == 1 && test == 45 && save == -1,
 		       "write to user stack with AC=0");
 
x86/umip.c +7 -3
index c5700b3..8b4e798 100644
@@ -23,7 +23,10 @@ static void gp_handler(struct ex_regs *regs)
 
 #define GP_ASM(stmt, in, clobber)                  \
     asm volatile (                                 \
-          "mov" W " $1f, %[expected_rip]\n\t"      \
+          "push" W " %%" R "ax\n\t"                \
+	  "lea 1f(%%" R "ip), %%" R "ax\n\t"       \
+          "mov %%" R "ax, %[expected_rip]\n\t"     \
+          "pop" W " %%" R "ax\n\t"                 \
           "movl $2f-1f, %[skip_count]\n\t"         \
           "1: " stmt "\n\t"                        \
           "2: "                                    \
@@ -130,7 +133,8 @@ static int do_ring3(void (*fn)(const char *), const char *arg)
 		  "push" W " %%" R "dx \n\t"
 		  "pushf" W "\n\t"
 		  "push" W " %[user_cs] \n\t"
-		  "push" W " $1f \n\t"
+		  "lea 1f(%%" R "ip), %%" R "dx \n\t"
+		  "push" W " %%" R "dx \n\t"
 		  "iret" W "\n"
 		  "1: \n\t"
 		  "push %%" R "cx\n\t"   /* save kernel SP */
@@ -144,7 +148,7 @@ static int do_ring3(void (*fn)(const char *), const char *arg)
 #endif
 
 		  "pop %%" R "cx\n\t"
-		  "mov $1f, %%" R "dx\n\t"
+		  "lea 1f(%%" R "ip), %%" R "dx\n\t"
 		  "int %[kernel_entry_vector]\n\t"
 		  ".section .text.entry \n\t"
 		  "kernel_entry: \n\t"
Signature · 1 line
2.33.0
maintainer
#17 lore raw

Re: [kvm-unit-tests PATCH v3 11/17] x86 UEFI: Convert x86 test cases to PIC

Quoted 279 lines · Zixuan Wang

From: Zixuan Wang <[email protected]>

UEFI loads EFI applications to dynamic runtime addresses, so it requires all applications to be compiled as PIC (position independent code). PIC does not allow the usage of compile time absolute address.

This commit converts multiple x86 test cases to PIC so they can compile and run in UEFI:

- x86/cet.efi

- x86/emulator.c: x86/emulator.c depends on lib/x86/usermode.c. But usermode.c contains non-PIC inline assembly code. This commit converts lib/x86/usermode.c and x86/emulator.c to PIC, so x86/emulator.c can compile and run in UEFI.

- x86/vmware_backdoors.c: it depends on lib/x86/usermode.c and now works without modifications

- x86/eventinj.c

- x86/smap.c

- x86/access.c

- x86/umip.c

---
  lib/x86/usermode.c  |  3 ++-
  x86/Makefile.common | 10 +++++-----
  x86/Makefile.x86_64 |  7 ++++---
  x86/access.c        |  9 +++++----
  x86/cet.c           |  8 +++++---
  x86/emulator.c      |  5 +++--
  x86/eventinj.c      |  6 ++++--
  x86/smap.c          |  8 ++++----
  x86/umip.c          | 10 +++++++---
  9 files changed, 39 insertions(+), 27 deletions(-)
lib/x86/usermode.c +2 -1
index f032523..c550545 100644
@@ -58,7 +58,8 @@ uint64_t run_in_user(usermode_func func, unsigned int fault_vector,
  			"pushq %[user_stack_top]\n\t"
  			"pushfq\n\t"
  			"pushq %[user_cs]\n\t"
-			"pushq $user_mode\n\t"
+			"lea user_mode(%%rip), %%rdx\n\t"
+			"pushq %%rdx\n\t"
  			"iretq\n"
 
  			"user_mode:\n\t"
x86/Makefile.common +5 -5
index 4859bf3..959379c 100644
@@ -81,16 +81,16 @@ tests-common = $(TEST_DIR)/vmexit.$(exe) $(TEST_DIR)/tsc.$(exe) \
                 $(TEST_DIR)/init.$(exe) \
                 $(TEST_DIR)/hyperv_synic.$(exe) $(TEST_DIR)/hyperv_stimer.$(exe) \
                 $(TEST_DIR)/hyperv_connections.$(exe) \
-               $(TEST_DIR)/tsx-ctrl.$(exe)
+               $(TEST_DIR)/tsx-ctrl.$(exe) \
+	       $(TEST_DIR)/eventinj.$(exe) \
+               $(TEST_DIR)/umip.$(exe)
 
  # The following test cases are disabled when building EFI tests because they
  # use absolute addresses in their inline assembly code, which cannot compile
  # with the '-fPIC' flag
  ifneq ($(TARGET_EFI),y)
-tests-common += $(TEST_DIR)/eventinj.$(exe) \
-                $(TEST_DIR)/smap.$(exe) \
-                $(TEST_DIR)/realmode.$(exe) \
-                $(TEST_DIR)/umip.$(exe)
+tests-common += $(TEST_DIR)/smap.$(exe) \
+                $(TEST_DIR)/realmode.$(exe)
  endif
 
  test_cases: $(tests-common) $(tests)
x86/Makefile.x86_64 +4 -3
index aa23b22..7e8a57a 100644
@@ -30,20 +30,21 @@ tests += $(TEST_DIR)/intel-iommu.$(exe)
  tests += $(TEST_DIR)/rdpru.$(exe)
  tests += $(TEST_DIR)/pks.$(exe)
  tests += $(TEST_DIR)/pmu_lbr.$(exe)
+tests += $(TEST_DIR)/emulator.$(exe)
+tests += $(TEST_DIR)/vmware_backdoors.$(exe)
 
  # The following test cases are disabled when building EFI tests because they
  # use absolute addresses in their inline assembly code, which cannot compile
  # with the '-fPIC' flag
  ifneq ($(TARGET_EFI),y)
  tests += $(TEST_DIR)/access.$(exe)
-tests += $(TEST_DIR)/emulator.$(exe)
  tests += $(TEST_DIR)/svm.$(exe)
  tests += $(TEST_DIR)/vmx.$(exe)
-tests += $(TEST_DIR)/vmware_backdoors.$(exe)
+endif
+
  ifneq ($(fcf_protection_full),)
  tests += $(TEST_DIR)/cet.$(exe)
  endif
-endif
 
  include $(SRCDIR)/$(TEST_DIR)/Makefile.common
 
x86/access.c +5 -4
index 4725bbd..8d620a7 100644
@@ -700,7 +700,7 @@ static int ac_test_do_access(ac_test_t *at)
 
      if (F(AC_ACCESS_TWICE)) {
  	asm volatile (
-	    "mov $fixed2, %%rsi \n\t"
+	    "lea fixed2(%%rip), %%rsi \n\t"
  	    "mov (%[addr]), %[reg] \n\t"
  	    "fixed2:"
  	    : [reg]"=r"(r), [fault]"=a"(fault), "=b"(e)
@@ -710,7 +710,7 @@ static int ac_test_do_access(ac_test_t *at)
  	fault = 0;
      }
 
-    asm volatile ("mov $fixed1, %%rsi \n\t"
+    asm volatile ("lea fixed1(%%rip), %%rsi \n\t"
  		  "mov %%rsp, %%rdx \n\t"
  		  "cmp $0, %[user] \n\t"
  		  "jz do_access \n\t"
@@ -719,7 +719,8 @@ static int ac_test_do_access(ac_test_t *at)
  		  "pushq %[user_stack_top] \n\t"
  		  "pushfq \n\t"
  		  "pushq %[user_cs] \n\t"
-		  "pushq $do_access \n\t"
+		  "lea do_access(%%rip), %%r8\n\t"
+		  "pushq %%r8\n\t"
  		  "iretq \n"
  		  "do_access: \n\t"
  		  "cmp $0, %[fetch] \n\t"
@@ -744,7 +745,7 @@ static int ac_test_do_access(ac_test_t *at)
  		    [user_cs]"i"(USER_CS),
  		    [user_stack_top]"r"(user_stack + sizeof user_stack),
  		    [kernel_entry_vector]"i"(0x20)
-		  : "rsi");
+		  : "rsi", "r8");
 
      asm volatile (".section .text.pf \n\t"
  		  "page_fault: \n\t"
x86/cet.c +5 -3
index a21577a..a4b79cb 100644
@@ -52,7 +52,7 @@ static u64 cet_ibt_func(void)
  	printf("No endbr64 instruction at jmp target, this triggers #CP...\n");
  	asm volatile ("movq $2, %rcx\n"
  		      "dec %rcx\n"
-		      "leaq 2f, %rax\n"
+		      "leaq 2f(%rip), %rax\n"
  		      "jmp *%rax \n"
  		      "2:\n"
  		      "dec %rcx\n");
@@ -67,7 +67,8 @@ void test_func(void) {
  			"pushq %[user_stack_top]\n\t"
  			"pushfq\n\t"
  			"pushq %[user_cs]\n\t"
-			"pushq $user_mode\n\t"
+			"lea user_mode(%%rip), %%rax\n\t"
+			"pushq %%rax\n\t"
  			"iretq\n"
 
  			"user_mode:\n\t"
@@ -77,7 +78,8 @@ void test_func(void) {
  			[user_ds]"i"(USER_DS),
  			[user_cs]"i"(USER_CS),
  			[user_stack_top]"r"(user_stack +
-					sizeof(user_stack)));
+					sizeof(user_stack))
+			: "rax");
  }
 
  #define SAVE_REGS() \
x86/emulator.c +3 -2
index 9fda1a0..4d2de24 100644
@@ -262,12 +262,13 @@ static void test_pop(void *mem)
 
  	asm volatile("mov %%rsp, %[tmp] \n\t"
  		     "mov %[stack_top], %%rsp \n\t"
-		     "push $1f \n\t"
+		     "lea 1f(%%rip), %%rax \n\t"
+		     "push %%rax \n\t"
  		     "ret \n\t"
  		     "2: jmp 2b \n\t"
  		     "1: mov %[tmp], %%rsp"
  		     : [tmp]"=&r"(tmp) : [stack_top]"r"(stack_top)
-		     : "memory");
+		     : "memory", "rax");
  	report(1, "ret");
 
  	stack_top[-1] = 0x778899;
x86/eventinj.c +4 -2
index 46593c9..0cd68e8 100644
@@ -155,9 +155,11 @@ asm("do_iret:"
  	"pushf"W" \n\t"
  	"mov %cs, %ecx \n\t"
  	"push"W" %"R "cx \n\t"
-	"push"W" $2f \n\t"
+	"lea"W" 2f(%"R "ip), %"R "bx \n\t"
+	"push"W" %"R "bx \n\t"
 
-	"cmpb $0, no_test_device\n\t"	// see if need to flush
+	"mov no_test_device(%"R "ip), %bl \n\t"
+	"cmpb $0, %bl\n\t"		// see if need to flush
  	"jnz 1f\n\t"
  	"outl %eax, $0xe4 \n\t"		// flush page
  	"1: \n\t"
x86/smap.c +4 -4
index ac2c8d5..b3ee16f 100644
@@ -161,10 +161,10 @@ int main(int ac, char **av)
  		test = -1;
  		asm("or $(" xstr(USER_BASE) "), %"R "sp \n"
  		    "push $44 \n "
-		    "decl test\n"
+		    "decl test(%"R "ip)\n"
  		    "and $~(" xstr(USER_BASE) "), %"R "sp \n"
  		    "pop %"R "ax\n"
-		    "movl %eax, test");
+		    "movl %eax, test(%"R "ip)");
  		report(pf_count == 0 && test == 44,
  		       "write to user stack with AC=1");
 
@@ -173,10 +173,10 @@ int main(int ac, char **av)
  		test = -1;
  		asm("or $(" xstr(USER_BASE) "), %"R "sp \n"
  		    "push $45 \n "
-		    "decl test\n"
+		    "decl test(%"R "ip)\n"
  		    "and $~(" xstr(USER_BASE) "), %"R "sp \n"
  		    "pop %"R "ax\n"
-		    "movl %eax, test");
+		    "movl %eax, test(%"R "ip)");
  		report(pf_count == 1 && test == 45 && save == -1,
  		       "write to user stack with AC=0");
 
x86/umip.c +6 -2
index c5700b3..8b4e798 100644
@@ -23,7 +23,10 @@ static void gp_handler(struct ex_regs *regs)
 
  #define GP_ASM(stmt, in, clobber)                  \
      asm volatile (                                 \
-          "mov" W " $1f, %[expected_rip]\n\t"      \
+          "push" W " %%" R "ax\n\t"                \
+	  "lea 1f(%%" R "ip), %%" R "ax\n\t"       \
+          "mov %%" R "ax, %[expected_rip]\n\t"     \
+          "pop" W " %%" R "ax\n\t"                 \
            "movl $2f-1f, %[skip_count]\n\t"         \
            "1: " stmt "\n\t"                        \
            "2: "                                    \
@@ -130,7 +133,8 @@ static int do_ring3(void (*fn)(const char *), const char *arg)
  		  "push" W " %%" R "dx \n\t"
  		  "pushf" W "\n\t"
  		  "push" W " %[user_cs] \n\t"
-		  "push" W " $1f \n\t"
+		  "lea 1f(%%" R "ip), %%" R "dx \n\t"
+		  "push" W " %%" R "dx \n\t"
  		  "iret" W "\n"
  		  "1: \n\t"
  		  "push %%" R "cx\n\t"   /* save kernel SP */

x86/umip.c static int do_ring3(void (*fn)(const char *), const char *arg) · line 154

Quoted 8 lines
x86/umip.c +1 -1
@@ -144,7 +148,7 @@ static int do_ring3(void (*fn)(const char *), const char *arg)
  #endif
 
  		  "pop %%" R "cx\n\t"
-		  "mov $1f, %%" R "dx\n\t"
+		  "lea 1f(%%" R "ip), %%" R "dx\n\t"
  		  "int %[kernel_entry_vector]\n\t"
  		  ".section .text.entry \n\t"
  		  "kernel_entry: \n\t"

I have left this patch out for now, because it breaks 32-bit builds. It's not a huge deal and can be redone on top of the rest.

Paolo

#18 lore raw

Re: [kvm-unit-tests PATCH v3 11/17] x86 UEFI: Convert x86 test cases to PIC

Quoted 27 lines · Paolo Bonzini
Quoted 25 lines · Zixuan Wang

From: Zixuan Wang <[email protected]>

UEFI loads EFI applications to dynamic runtime addresses, so it requires all applications to be compiled as PIC (position independent code). PIC does not allow the usage of compile time absolute address.

This commit converts multiple x86 test cases to PIC so they can compile and run in UEFI:

- x86/cet.efi

- x86/emulator.c: x86/emulator.c depends on lib/x86/usermode.c. But usermode.c contains non-PIC inline assembly code. This commit converts lib/x86/usermode.c and x86/emulator.c to PIC, so x86/emulator.c can compile and run in UEFI.

- x86/vmware_backdoors.c: it depends on lib/x86/usermode.c and now works without modifications

- x86/eventinj.c

- x86/smap.c

- x86/access.c

Quoted 9 lines
Quoted 3 lines

- x86/umip.c

I have left this patch out for now, because it breaks 32-bit builds. It's not a huge deal and can be redone on top of the rest.

Paolo

Marc and I are working on a follow-up patch set that includes the fixes for this patch under 32-bit mode. I have also identified a potential bug in x86/umip.c and fixed it in the next patch set. The full debugging detail is described in our off-list GitHub discussion [1].

In summary, in the following line, %[sp0] can be compiled as a %r8-based offset address:

x86/umip.c:127
"mov %%" R "sp, %[sp0]\n\t" /* kernel sp for exception handlers */

%r8 is then modified in the function call without saving/restoring, thus making the following line using the wrong address.

x86/umip.c:148
"mov %[sp0], %%" R "sp\n\t"

This register is selected by the compiler, it's not guaranteed to be %r8 so we cannot just push/pop %r8 before/after the function call. A simple fix is to save the %rsp to %rbx (in addition to saving it to %[sp0]). %rbx is a callee-saved register, so its value is not modified.

We already have this fix in the patch set draft [1] and will post it once ready.

[1] https://github.com/marc-orr/KVM-Unit-Tests-dev-fork/pull/9

Best regards,
Zixuan

#19 lore raw

PATCH v3 12/17 x86 AMD SEV: Initial support

In reply to Zixuan Wang

From: Zixuan Wang <[email protected]>

AMD Secure Encrypted Virtualization (SEV) is a hardware accelerated memory encryption feature that protects guest VMs from host attacks.

This commit provides set up code and a test case for AMD SEV. The set up code checks if SEV is supported and enabled, and then sets SEV c-bit for each page table entry.

---
 lib/x86/amd_sev.c   | 75 +++++++++++++++++++++++++++++++++++++++++++++
 lib/x86/amd_sev.h   | 45 +++++++++++++++++++++++++++
 lib/x86/asm/setup.h |  1 +
 lib/x86/setup.c     | 15 +++++++++
 x86/Makefile.common |  1 +
 x86/Makefile.x86_64 |  3 ++
 x86/amd_sev.c       | 64 ++++++++++++++++++++++++++++++++++++++
 7 files changed, 204 insertions(+)
 create mode 100644 lib/x86/amd_sev.c
 create mode 100644 lib/x86/amd_sev.h
 create mode 100644 x86/amd_sev.c
lib/x86/amd_sev.c new file +75 -0
new file mode 100644
index 0000000..13e6455
@@ -0,0 +1,75 @@
+/*
+ * AMD SEV support in kvm-unit-tests
+ *
+ * Copyright (c) 2021, Google Inc
+ *
+ * Authors:
+ *   Zixuan Wang <[email protected]>
+ *
+ * SPDX-License-Identifier: LGPL-2.0-or-later
+ */
+
+#include "amd_sev.h"
+#include "x86/processor.h"
+
+static unsigned short amd_sev_c_bit_pos;
+
+bool amd_sev_enabled(void)
+{
+	struct cpuid cpuid_out;
+	static bool sev_enabled;
+	static bool initialized = false;
+
+	/* Check CPUID and MSR for SEV status and store it for future function calls. */
+	if (!initialized) {
+		sev_enabled = false;
+		initialized = true;
+
+		/* Test if we can query SEV features */
+		cpuid_out = cpuid(CPUID_FN_LARGEST_EXT_FUNC_NUM);
+		if (cpuid_out.a < CPUID_FN_ENCRYPT_MEM_CAPAB) {
+			return sev_enabled;
+		}
+
+		/* Test if SEV is supported */
+		cpuid_out = cpuid(CPUID_FN_ENCRYPT_MEM_CAPAB);
+		if (!(cpuid_out.a & SEV_SUPPORT_MASK)) {
+			return sev_enabled;
+		}
+
+		/* Test if SEV is enabled */
+		if (rdmsr(MSR_SEV_STATUS) & SEV_ENABLED_MASK) {
+			sev_enabled = true;
+		}
+	}
+
+	return sev_enabled;
+}
+
+efi_status_t setup_amd_sev(void)
+{
+	struct cpuid cpuid_out;
+
+	if (!amd_sev_enabled()) {
+		return EFI_UNSUPPORTED;
+	}
+
+	/*
+	 * Extract C-Bit position from ebx[5:0]
+	 * AMD64 Architecture Programmer's Manual Volume 3
+	 *   - Section " Function 8000_001Fh - Encrypted Memory Capabilities"
+	 */
+	cpuid_out = cpuid(CPUID_FN_ENCRYPT_MEM_CAPAB);
+	amd_sev_c_bit_pos = (unsigned short)(cpuid_out.b & 0x3f);
+
+	return EFI_SUCCESS;
+}
+
+unsigned long long get_amd_sev_c_bit_mask(void)
+{
+	if (amd_sev_enabled()) {
+		return 1ull << amd_sev_c_bit_pos;
+	} else {
+		return 0;
+	}
+}
lib/x86/amd_sev.h new file +45 -0

45 lines not on this page · raw message

lib/x86/asm/setup.h +1 -0
index 9cc135a..30ca341 100644
@@ -12,6 +12,7 @@ unsigned long setup_tss(void);
 #include "x86/smp.h"
 #include "asm/page.h"
 #include "efi.h"
+#include "x86/amd_sev.h"
 
 /*
  * efi_bootinfo_t: stores EFI-related machine info retrieved by
lib/x86/setup.c +15 -0
index c1aa82a..ad7f725 100644
@@ -251,6 +251,18 @@ efi_status_t setup_efi_pre_boot(unsigned long *mapkey, efi_bootinfo_t *efi_booti
 		return status;
 	}
 
+	status = setup_amd_sev();
+	if (status != EFI_SUCCESS) {
+		switch (status) {
+		case EFI_UNSUPPORTED:
+			/* Continue if AMD SEV is not supported */
+			break;
+		default:
+			printf("Set up AMD SEV failed\n");
+			return status;
+		}
+	}
+
 	return EFI_SUCCESS;
 }
 
@@ -269,6 +281,9 @@ static void setup_page_table(void)
 	/* Set default flags */
 	flags = PT_PRESENT_MASK | PT_WRITABLE_MASK | PT_USER_MASK;
 
+	/* Set AMD SEV C-Bit for page table entries */
+	flags |= get_amd_sev_c_bit_mask();
+
 	/* Level 5 */
 	curr_pt = (pgd_t *)&ptl5;
 	curr_pt[0] = ((phys_addr_t)&ptl4) | flags;
x86/Makefile.common +1 -0
index 959379c..0913083 100644
@@ -23,6 +23,7 @@ cflatobjs += lib/x86/stack.o
 cflatobjs += lib/x86/fault_test.o
 cflatobjs += lib/x86/delay.o
 ifeq ($(TARGET_EFI),y)
+cflatobjs += lib/x86/amd_sev.o
 cflatobjs += lib/x86/setup.o
 cflatobjs += lib/efi.o
 cflatobjs += x86/efi/reloc_x86_64.o
x86/Makefile.x86_64 +3 -0
index 7e8a57a..8304939 100644
@@ -32,6 +32,9 @@ tests += $(TEST_DIR)/pks.$(exe)
 tests += $(TEST_DIR)/pmu_lbr.$(exe)
 tests += $(TEST_DIR)/emulator.$(exe)
 tests += $(TEST_DIR)/vmware_backdoors.$(exe)
+ifeq ($(TARGET_EFI),y)
+tests += $(TEST_DIR)/amd_sev.$(exe)
+endif
 
 # The following test cases are disabled when building EFI tests because they
 # use absolute addresses in their inline assembly code, which cannot compile
x86/amd_sev.c new file +64 -0

64 lines not on this page · raw message

Signature · 1 line
2.33.0
maintainer
#20 lore raw

Re: [kvm-unit-tests PATCH v3 12/17] x86 AMD SEV: Initial support

Quoted 5 lines · Zixuan Wang

From: Zixuan Wang <[email protected]>

AMD Secure Encrypted Virtualization (SEV) is a hardware accelerated memory encryption feature that protects guest VMs from host attacks.

Quoted 7 lines

This commit provides set up code and a test case for AMD SEV. The set up code checks if SEV is supported and enabled, and then sets SEV c-bit for each page table entry.

Whee, it works!

qemu/qemu/build/qemu-system-x86_64 \
-object sev-guest,id=sev0,cbitpos=47,reduced-phys-bits=1,policy=0x7 \
-machine q35,memory-encryption=sev0 --no-reboot -nodefaults \
-device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 \
-vnc none -serial stdio -device pci-testdev -machine accel=kvm \
-drive file=/usr/share/edk2/ovmf/OVMF_CODE.cc.fd,format=raw,if=pflash \
-drive file.dir=efi-tests/amd_sev/,file.driver=vvfat,file.rw=on,format=raw,if=virtio \
-net none -nographic -smp 1 -m 256 --cpu EPYC-Rome

So the "magic" flags are

	-object sev-guest,id=sev0,cbitpos=47,reduced-phys-bits=1,policy=0x3
	-machine memory-encryption=sev0

Paolo

#21 lore raw

PATCH v3 13/17 x86 AMD SEV: Page table with c-bit

In reply to Zixuan Wang

From: Zixuan Wang <[email protected]>

AMD SEV introduces c-bit to page table entries. To work with AMD SEV:

1. c-bit should be set for new page table entries
2. address calculation should not use c-bit as part of address

---
 lib/x86/amd_sev.c  | 10 ++++++++++
 lib/x86/amd_sev.h  |  1 +
 lib/x86/asm/page.h | 27 ++++++++++++++++++++++++---
 lib/x86/vm.c       | 18 ++++++++++++++----
 4 files changed, 49 insertions(+), 7 deletions(-)
lib/x86/amd_sev.c +10 -0
index 13e6455..b555355 100644
@@ -73,3 +73,13 @@ unsigned long long get_amd_sev_c_bit_mask(void)
 		return 0;
 	}
 }
+
+unsigned long long get_amd_sev_addr_upperbound(void)
+{
+	if (amd_sev_enabled()) {
+		return amd_sev_c_bit_pos - 1;
+	} else {
+		/* Default memory upper bound */
+		return PT_ADDR_UPPER_BOUND_DEFAULT;
+	}
+}
lib/x86/amd_sev.h +1 -0
index 68483c9..7de12f0 100644
@@ -39,6 +39,7 @@ bool amd_sev_enabled(void);
 efi_status_t setup_amd_sev(void);
 
 unsigned long long get_amd_sev_c_bit_mask(void);
+unsigned long long get_amd_sev_addr_upperbound(void);
 
 #endif /* TARGET_EFI */
 
lib/x86/asm/page.h +24 -3
index f6f740b..c25bc66 100644
@@ -25,6 +25,12 @@ typedef unsigned long pgd_t;
 #define LARGE_PAGE_SIZE	(1024 * PAGE_SIZE)
 #endif
 
+#ifdef TARGET_EFI
+/* lib/x86/amd_sev.c */
+extern unsigned long long get_amd_sev_c_bit_mask(void);
+extern unsigned long long get_amd_sev_addr_upperbound(void);
+#endif /* TARGET_EFI */
+
 #define PT_PRESENT_MASK		(1ull << 0)
 #define PT_WRITABLE_MASK	(1ull << 1)
 #define PT_USER_MASK		(1ull << 2)
@@ -33,10 +39,25 @@ typedef unsigned long pgd_t;
 #define PT_PAGE_SIZE_MASK	(1ull << 7)
 #define PT_GLOBAL_MASK		(1ull << 8)
 #define PT64_NX_MASK		(1ull << 63)
-#define PT_ADDR_MASK		GENMASK_ULL(51, 12)
 
-#define PDPTE64_PAGE_SIZE_MASK	  (1ull << 7)
-#define PDPTE64_RSVD_MASK	  GENMASK_ULL(51, cpuid_maxphyaddr())
+/*
+ * Without AMD SEV, the default address upper bound is 51 (i.e., pte bit 51 and
+ * lower bits are addresses). But with AMD SEV enabled, the upper bound is one
+ * bit lower than the c-bit position.
+ */
+#define PT_ADDR_UPPER_BOUND_DEFAULT	(51)
+
+#ifdef TARGET_EFI
+#define PT_ADDR_UPPER_BOUND	(get_amd_sev_addr_upperbound())
+#else
+#define PT_ADDR_UPPER_BOUND	(PT_ADDR_UPPER_BOUND_DEFAULT)
+#endif /* TARGET_EFI */
+
+#define PT_ADDR_LOWER_BOUND	(PAGE_SHIFT)
+#define PT_ADDR_MASK		GENMASK_ULL(PT_ADDR_UPPER_BOUND, PT_ADDR_LOWER_BOUND)
+
+#define PDPTE64_PAGE_SIZE_MASK	(1ull << 7)
+#define PDPTE64_RSVD_MASK	GENMASK_ULL(PT_ADDR_UPPER_BOUND, cpuid_maxphyaddr())
 
 #define PT_AD_MASK              (PT_ACCESSED_MASK | PT_DIRTY_MASK)
 
lib/x86/vm.c +14 -4
index 5cd2ee4..0ebc860 100644
@@ -26,6 +26,9 @@ pteval_t *install_pte(pgd_t *cr3,
                 pt_page = 0;
 	    memset(new_pt, 0, PAGE_SIZE);
 	    pt[offset] = virt_to_phys(new_pt) | PT_PRESENT_MASK | PT_WRITABLE_MASK | pte_opt_mask;
+#ifdef TARGET_EFI
+	    pt[offset] |= get_amd_sev_c_bit_mask();
+#endif /* TARGET_EFI */
 	}
 	pt = phys_to_virt(pt[offset] & PT_ADDR_MASK);
     }
@@ -63,7 +66,7 @@ struct pte_search find_pte_level(pgd_t *cr3, void *virt,
 		if (r.level == lowest_level)
 			return r;
 
-		pt = phys_to_virt(pte & 0xffffffffff000ull);
+		pt = phys_to_virt(pte & PT_ADDR_MASK);
 	}
 }
 
@@ -94,13 +97,20 @@ pteval_t *get_pte_level(pgd_t *cr3, void *virt, int pte_level)
 
 pteval_t *install_large_page(pgd_t *cr3, phys_addr_t phys, void *virt)
 {
-    return install_pte(cr3, 2, virt,
-		       phys | PT_PRESENT_MASK | PT_WRITABLE_MASK | pte_opt_mask | PT_PAGE_SIZE_MASK, 0);
+    phys_addr_t flags = PT_PRESENT_MASK | PT_WRITABLE_MASK | pte_opt_mask | PT_PAGE_SIZE_MASK;
+#ifdef TARGET_EFI
+    flags |= get_amd_sev_c_bit_mask();
+#endif /* TARGET_EFI */
+    return install_pte(cr3, 2, virt, phys | flags, 0);
 }
 
 pteval_t *install_page(pgd_t *cr3, phys_addr_t phys, void *virt)
 {
-    return install_pte(cr3, 1, virt, phys | PT_PRESENT_MASK | PT_WRITABLE_MASK | pte_opt_mask, 0);
+    phys_addr_t flags = PT_PRESENT_MASK | PT_WRITABLE_MASK | pte_opt_mask;
+#ifdef TARGET_EFI
+    flags |= get_amd_sev_c_bit_mask();
+#endif /* TARGET_EFI */
+    return install_pte(cr3, 1, virt, phys | flags, 0);
 }
 
 void install_pages(pgd_t *cr3, phys_addr_t phys, size_t len, void *virt)
Signature · 1 line
2.33.0
#22 lore raw

PATCH v3 14/17 x86 AMD SEV-ES: Check SEV-ES status

In reply to Zixuan Wang

From: Zixuan Wang <[email protected]>

This commit provides initial start up code for KVM-Unit-Tests to run in an SEV-ES guest VM. This start up code checks if SEV-ES feature is supported and enabled for the guest.

In this commit, KVM-Unit-Tests can pass the SEV-ES check and enter setup_efi() function, but crashes in setup_gdt_tss(), which will be fixed by follow-up commits.

---
 lib/x86/amd_sev.c | 22 ++++++++++++++++++++++
 lib/x86/amd_sev.h |  7 +++++--
 2 files changed, 27 insertions(+), 2 deletions(-)
lib/x86/amd_sev.c +22 -0
index b555355..88cf283 100644
@@ -65,6 +65,28 @@ efi_status_t setup_amd_sev(void)
 	return EFI_SUCCESS;
 }
 
+bool amd_sev_es_enabled(void)
+{
+	static bool sev_es_enabled;
+	static bool initialized = false;
+
+	if (!initialized) {
+		sev_es_enabled = false;
+		initialized = true;
+
+		if (!amd_sev_enabled()) {
+			return sev_es_enabled;
+		}
+
+		/* Test if SEV-ES is enabled */
+		if (rdmsr(MSR_SEV_STATUS) & SEV_ES_ENABLED_MASK) {
+			sev_es_enabled = true;
+		}
+	}
+
+	return sev_es_enabled;
+}
+
 unsigned long long get_amd_sev_c_bit_mask(void)
 {
 	if (amd_sev_enabled()) {
lib/x86/amd_sev.h +5 -2
index 7de12f0..fb51fc2 100644
@@ -32,12 +32,15 @@
  * AMD Programmer's Manual Volume 2
  *   - Section "SEV_STATUS MSR"
  */
-#define MSR_SEV_STATUS   0xc0010131
-#define SEV_ENABLED_MASK 0b1
+#define MSR_SEV_STATUS      0xc0010131
+#define SEV_ENABLED_MASK    0b1
+#define SEV_ES_ENABLED_MASK 0b10
 
 bool amd_sev_enabled(void);
 efi_status_t setup_amd_sev(void);
 
+bool amd_sev_es_enabled(void);
+
 unsigned long long get_amd_sev_c_bit_mask(void);
 unsigned long long get_amd_sev_addr_upperbound(void);
 
Signature · 1 line
2.33.0
#23 lore raw

PATCH v3 15/17 x86 AMD SEV-ES: Copy UEFI #VC IDT entry

In reply to Zixuan Wang

From: Zixuan Wang <[email protected]>

AMD SEV-ES introduces a new #VC exception that handles the communication between guest and host. UEFI already implements a #VC handler so there is no need to re-implement it in KVM-Unit-Tests. To reuse this #VC handler, this commit reads UEFI's IDT, copy the #VC IDT entry into KVM-Unit-Tests' IDT.

Reusing UEFI #VC handler is a temporary workaround, and the long-term solution is to implement a #VC handler in KVM-Unit-Tests so it does not depend on specific UEFI's #VC handler. However, we still believe that the current approach is good as an intermediate solution, because it unlocks a lot of testing and we do not expect that testing to be inherently tied to the UEFI's #VC handler.

In this commit, load_idt() can work and now guest crashes in setup_page_table(), which will be fixed by follow-up commits.

---
 lib/x86/amd_sev.c | 30 ++++++++++++++++++++++++++++++
 lib/x86/amd_sev.h |  7 +++++++
 lib/x86/setup.c   | 12 ++++++++++++
 3 files changed, 49 insertions(+)
lib/x86/amd_sev.c +30 -0
index 88cf283..50352df 100644
@@ -87,6 +87,36 @@ bool amd_sev_es_enabled(void)
 	return sev_es_enabled;
 }
 
+efi_status_t setup_amd_sev_es(void)
+{
+	struct descriptor_table_ptr idtr;
+	idt_entry_t *idt;
+	idt_entry_t vc_handler_idt;
+
+	if (!amd_sev_es_enabled()) {
+		return EFI_UNSUPPORTED;
+	}
+
+	/*
+	 * Copy UEFI's #VC IDT entry, so KVM-Unit-Tests can reuse it and does
+	 * not have to re-implement a #VC handler. Also update the #VC IDT code
+	 * segment to use KVM-Unit-Tests segments, KERNEL_CS, so that we do not
+	 * have to copy the UEFI GDT entries into KVM-Unit-Tests GDT.
+	 *
+	 * TODO: Reusing UEFI #VC handler is a temporary workaround to simplify
+	 * the boot up process, the long-term solution is to implement a #VC
+	 * handler in kvm-unit-tests and load it, so that kvm-unit-tests does
+	 * not depend on specific UEFI #VC handler implementation.
+	 */
+	sidt(&idtr);
+	idt = (idt_entry_t *)idtr.base;
+	vc_handler_idt = idt[SEV_ES_VC_HANDLER_VECTOR];
+	vc_handler_idt.selector = KERNEL_CS;
+	boot_idt[SEV_ES_VC_HANDLER_VECTOR] = vc_handler_idt;
+
+	return EFI_SUCCESS;
+}
+
 unsigned long long get_amd_sev_c_bit_mask(void)
 {
 	if (amd_sev_enabled()) {
lib/x86/amd_sev.h +7 -0
index fb51fc2..0ea1fda 100644
@@ -39,7 +39,14 @@
 bool amd_sev_enabled(void);
 efi_status_t setup_amd_sev(void);
 
+/*
+ * AMD Programmer's Manual Volume 2
+ *   - Section "#VC Exception"
+ */
+#define SEV_ES_VC_HANDLER_VECTOR 29
+
 bool amd_sev_es_enabled(void);
+efi_status_t setup_amd_sev_es(void);
 
 unsigned long long get_amd_sev_c_bit_mask(void);
 unsigned long long get_amd_sev_addr_upperbound(void);
lib/x86/setup.c +12 -0
index ad7f725..529c3d0 100644
@@ -263,6 +263,18 @@ efi_status_t setup_efi_pre_boot(unsigned long *mapkey, efi_bootinfo_t *efi_booti
 		}
 	}
 
+	status = setup_amd_sev_es();
+	if (status != EFI_SUCCESS) {
+		switch (status) {
+		case EFI_UNSUPPORTED:
+			/* Continue if AMD SEV-ES is not supported */
+			break;
+		default:
+			printf("Set up AMD SEV-ES failed\n");
+			return status;
+		}
+	}
+
 	return EFI_SUCCESS;
 }
 
Signature · 1 line
2.33.0
#24 lore raw

PATCH v3 16/17 x86 AMD SEV-ES: Set up GHCB page

In reply to Zixuan Wang

From: Zixuan Wang <[email protected]>

AMD SEV-ES introduces a GHCB page for guest/host communication. This page should be unencrypted, i.e. its c-bit should be unset, otherwise the guest VM may crash when #VC exception happens.

By default, KVM-Unit-Tests only sets up 2MiB pages, i.e. only Level 2 page table entries are provided. Unsetting GHCB Level 2 pte's c-bit still crashes the guest VM. The solution is to unset only its Level 1 pte's c-bit.

This commit provides GHCB page set up code that:

1. finds GHCB Level 1 pte
2. if not found, installs corresponding Level 1 pages
3. unsets GHCB Level 1 pte's c-bit

In this commit, KVM-Unit-Tests can run in an SEV-ES VM and boot into test cases' main().

---
 lib/x86/amd_sev.c | 37 +++++++++++++++++++++++++++++++++++++
 lib/x86/amd_sev.h |  7 +++++++
 lib/x86/setup.c   |  4 ++++
 3 files changed, 48 insertions(+)
lib/x86/amd_sev.c +37 -0
index 50352df..6672214 100644
@@ -11,6 +11,7 @@
 
 #include "amd_sev.h"
 #include "x86/processor.h"
+#include "x86/vm.h"
 
 static unsigned short amd_sev_c_bit_pos;
 
@@ -117,6 +118,42 @@ efi_status_t setup_amd_sev_es(void)
 	return EFI_SUCCESS;
 }
 
+void setup_ghcb_pte(pgd_t *page_table)
+{
+	/*
+	 * SEV-ES guest uses GHCB page to communicate with the host. This page
+	 * must be unencrypted, i.e. its c-bit should be unset. To do so, this
+	 * function searches GHCB's L1 pte, creates corresponding L1 ptes if not
+	 * found, and unsets the c-bit of GHCB's L1 pte.
+	 */
+	phys_addr_t ghcb_addr, ghcb_base_addr;
+	pteval_t *pte;
+
+	/* Read the current GHCB page addr */
+	ghcb_addr = rdmsr(SEV_ES_GHCB_MSR_INDEX);
+
+	/* Search Level 1 page table entry for GHCB page */
+	pte = get_pte_level(page_table, (void *)ghcb_addr, 1);
+
+	/* Create Level 1 pte for GHCB page if not found */
+	if (pte == NULL) {
+		/* Find Level 2 page base address */
+		ghcb_base_addr = ghcb_addr & ~(LARGE_PAGE_SIZE - 1);
+		/* Install Level 1 ptes */
+		install_pages(page_table, ghcb_base_addr, LARGE_PAGE_SIZE, (void *)ghcb_base_addr);
+		/* Find Level 2 pte, set as 4KB pages */
+		pte = get_pte_level(page_table, (void *)ghcb_addr, 2);
+		assert(pte);
+		*pte &= ~(PT_PAGE_SIZE_MASK);
+		/* Find Level 1 GHCB pte */
+		pte = get_pte_level(page_table, (void *)ghcb_addr, 1);
+		assert(pte);
+	}
+
+	/* Unset c-bit in Level 1 GHCB pte */
+	*pte &= ~(get_amd_sev_c_bit_mask());
+}
+
 unsigned long long get_amd_sev_c_bit_mask(void)
 {
 	if (amd_sev_enabled()) {
lib/x86/amd_sev.h +7 -0
index 0ea1fda..6a10f84 100644
@@ -45,8 +45,15 @@ efi_status_t setup_amd_sev(void);
  */
 #define SEV_ES_VC_HANDLER_VECTOR 29
 
+/*
+ * AMD Programmer's Manual Volume 2
+ *   - Section "GHCB"
+ */
+#define SEV_ES_GHCB_MSR_INDEX 0xc0010130
+
 bool amd_sev_es_enabled(void);
 efi_status_t setup_amd_sev_es(void);
+void setup_ghcb_pte(pgd_t *page_table);
 
 unsigned long long get_amd_sev_c_bit_mask(void);
 unsigned long long get_amd_sev_addr_upperbound(void);
lib/x86/setup.c +4 -0
index 529c3d0..1f2cdde 100644
@@ -314,6 +314,10 @@ static void setup_page_table(void)
 		curr_pt[i] = ((phys_addr_t)(i << 21)) | flags;
 	}
 
+	if (amd_sev_es_enabled()) {
+		setup_ghcb_pte((pgd_t *)&ptl4);
+	}
+
 	/* Load 4-level page table */
 	write_cr3((ulong)&ptl4);
 }
Signature · 1 line
2.33.0
#25 lore raw

PATCH v3 17/17 x86 AMD SEV-ES: Add test cases

In reply to Zixuan Wang

From: Zixuan Wang <[email protected]>

SEV-ES introduces #VC handler for guest/host communications, e.g., accessing MSR, executing CPUID. This commit provides test cases to check if SEV-ES is enabled and if rdmsr/wrmsr are handled correctly in SEV-ES.

---
 x86/amd_sev.c | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
x86/amd_sev.c +30 -0
index a07a48f..21a491c 100644
@@ -13,6 +13,7 @@
 #include "libcflat.h"
 #include "x86/processor.h"
 #include "x86/amd_sev.h"
+#include "msr.h"
 
 #define EXIT_SUCCESS 0
 #define EXIT_FAILURE 1
@@ -55,10 +56,39 @@ static int test_sev_activation(void)
 	return EXIT_SUCCESS;
 }
 
+static int test_sev_es_activation(void)
+{
+	if (!(rdmsr(MSR_SEV_STATUS) & SEV_ES_ENABLED_MASK)) {
+		return EXIT_FAILURE;
+	}
+
+	return EXIT_SUCCESS;
+}
+
+static int test_sev_es_msr(void)
+{
+	/*
+	 * With SEV-ES, rdmsr/wrmsr trigger #VC exception. If #VC is handled
+	 * correctly, rdmsr/wrmsr should work like without SEV-ES and not crash
+	 * the guest VM.
+	 */
+	u64 val = 0x1234;
+	wrmsr(MSR_TSC_AUX, val);
+	if(val != rdmsr(MSR_TSC_AUX)) {
+		return EXIT_FAILURE;
+	}
+
+	return EXIT_SUCCESS;
+}
+
 int main(void)
 {
 	int rtn;
 	rtn = test_sev_activation();
 	report(rtn == EXIT_SUCCESS, "SEV activation test.");
+	rtn = test_sev_es_activation();
+	report(rtn == EXIT_SUCCESS, "SEV-ES activation test.");
+	rtn = test_sev_es_msr();
+	report(rtn == EXIT_SUCCESS, "SEV-ES MSR test.");
 	return report_summary();
 }
Signature · 1 line
2.33.0
#26 lore raw

Re: [kvm-unit-tests PATCH v3 17/17] x86 AMD SEV-ES: Add test cases

Hi Zixuan,

Quoted 37 lines · Zixuan Wang

From: Zixuan Wang <[email protected]>

SEV-ES introduces #VC handler for guest/host communications, e.g., accessing MSR, executing CPUID. This commit provides test cases to check if SEV-ES is enabled and if rdmsr/wrmsr are handled correctly in SEV-ES.

---
 x86/amd_sev.c | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
x86/amd_sev.c +11 -0
index a07a48f..21a491c 100644
@@ -13,6 +13,7 @@
 #include "libcflat.h"
 #include "x86/processor.h"
 #include "x86/amd_sev.h"
+#include "msr.h"
 
 #define EXIT_SUCCESS 0
 #define EXIT_FAILURE 1
@@ -55,10 +56,39 @@ static int test_sev_activation(void)
 	return EXIT_SUCCESS;
 }
 
+static int test_sev_es_activation(void)
+{
+	if (!(rdmsr(MSR_SEV_STATUS) & SEV_ES_ENABLED_MASK)) {
+		return EXIT_FAILURE;
+	}
+
+	return EXIT_SUCCESS;
+}
+
+static int test_sev_es_msr(void)

x86/amd_sev.c static int test_sev_activation(void) · line 78

Quoted 10 lines
x86/amd_sev.c +10 -0
+{
+	/*
+	 * With SEV-ES, rdmsr/wrmsr trigger #VC exception. If #VC is handled
+	 * correctly, rdmsr/wrmsr should work like without SEV-ES and not crash
+	 * the guest VM.
+	 */
+	u64 val = 0x1234;
+	wrmsr(MSR_TSC_AUX, val);
+	if(val != rdmsr(MSR_TSC_AUX)) {
+		return EXIT_FAILURE;

See note below.

x86/amd_sev.c static int test_sev_activation(void) · line 91

Quoted 13 lines
x86/amd_sev.c +8 -0
+	}
+
+	return EXIT_SUCCESS;
+}
+
 int main(void)
 {
 	int rtn;
 	rtn = test_sev_activation();
 	report(rtn == EXIT_SUCCESS, "SEV activation test.");
+	rtn = test_sev_es_activation();
+	report(rtn == EXIT_SUCCESS, "SEV-ES activation test.");
+	rtn = test_sev_es_msr();

There is nothing SEV-ES specific about this function, it only wraps rdmsr/wrmsr, which are supposed to generate #VC exceptions on SEV-ES. Since the same scenario can be covered by running the msr testcase as a SEV-ES guest and observing if it crashes, does testing rdmsr/wrmsr one more time here gain us any new information?

Also, the function gets called from main() even if test_sev_es_activation() failed or SEV-ES was inactive.

Note: More broadly, what are you looking to test for here?
1. wrmsr/rdmsr correctness (rdmsr reads what wrmsr wrote)? or,
2. A #VC exception not causing a guest crash on SEV-ES?

If you are looking to test 1., I suggest letting it be covered by the generic testcases for msr.

If you are looking to test 2., perhaps a better test is to trigger all scenarios that would cause a #VC exception (eg. test_sev_es_vc_exit) and check that a SEV-ES guest survives.

Regards,
Varad

Quoted 4 lines
x86/amd_sev.c +1 -0
+	report(rtn == EXIT_SUCCESS, "SEV-ES MSR test.");
 	return report_summary();
 }
#27 lore raw

Re: [kvm-unit-tests PATCH v3 17/17] x86 AMD SEV-ES: Add test cases

Quoted 49 lines · Varad Gautam

Hi Zixuan,

x86/amd_sev.c static int test_sev_activation(void) · line 78

Quoted 12 lines · Zixuan Wang

From: Zixuan Wang <[email protected]>

x86/amd_sev.c +11 -0
+static int test_sev_es_msr(void)
+{
+     /*
+      * With SEV-ES, rdmsr/wrmsr trigger #VC exception. If #VC is handled
+      * correctly, rdmsr/wrmsr should work like without SEV-ES and not crash
+      * the guest VM.
+      */
+     u64 val = 0x1234;
+     wrmsr(MSR_TSC_AUX, val);
+     if(val != rdmsr(MSR_TSC_AUX)) {
+             return EXIT_FAILURE;

See note below.

x86/amd_sev.c static int test_sev_activation(void) · line 91

Quoted 13 lines
x86/amd_sev.c +8 -0
+     }
+
+     return EXIT_SUCCESS;
+}
+
 int main(void)
 {
      int rtn;
      rtn = test_sev_activation();
      report(rtn == EXIT_SUCCESS, "SEV activation test.");
+     rtn = test_sev_es_activation();
+     report(rtn == EXIT_SUCCESS, "SEV-ES activation test.");
+     rtn = test_sev_es_msr();

There is nothing SEV-ES specific about this function, it only wraps rdmsr/wrmsr, which are supposed to generate #VC exceptions on SEV-ES. Since the same scenario can be covered by running the msr testcase as a SEV-ES guest and observing if it crashes, does testing rdmsr/wrmsr one more time here gain us any new information?

Also, the function gets called from main() even if test_sev_es_activation() failed or SEV-ES was inactive.

Note: More broadly, what are you looking to test for here?
1. wrmsr/rdmsr correctness (rdmsr reads what wrmsr wrote)? or,
2. A #VC exception not causing a guest crash on SEV-ES?

If you are looking to test 1., I suggest letting it be covered by the generic testcases for msr.

Quoted 7 lines

If you are looking to test 2., perhaps a better test is to trigger all scenarios that would cause a #VC exception (eg. test_sev_es_vc_exit) and check that a SEV-ES guest survives.

Regards,
Varad

Hi Varad,

This test case does not bring any SEV-related functionality testing. Instead, it is provided for development, i.e., one can check if SEV is properly set up by monitoring if this test case runs fine without crashes.

Since this test case is causing some confusion and does not bring any functionality testing, I can remove it from the next version. We can still verify the SEV setup process by checking if an existing test case (e.g., x86/msr.c) runs without crashes in a SEV guest.

It's hard for me to develop a meaningful SEV test case, because I just finished my Google internship and thus lost access to SEV-enabled machines.

Best regards,
Zixuan

#28 lore raw

Re: [kvm-unit-tests PATCH v3 17/17] x86 AMD SEV-ES: Add test cases

Quoted 66 lines · Zixuan Wang
Quoted 49 lines · Varad Gautam

Hi Zixuan,

x86/amd_sev.c static int test_sev_activation(void) · line 78

Quoted 12 lines · Zixuan Wang

From: Zixuan Wang <[email protected]>

x86/amd_sev.c +11 -0
+static int test_sev_es_msr(void)
+{
+     /*
+      * With SEV-ES, rdmsr/wrmsr trigger #VC exception. If #VC is handled
+      * correctly, rdmsr/wrmsr should work like without SEV-ES and not crash
+      * the guest VM.
+      */
+     u64 val = 0x1234;
+     wrmsr(MSR_TSC_AUX, val);
+     if(val != rdmsr(MSR_TSC_AUX)) {
+             return EXIT_FAILURE;

See note below.

x86/amd_sev.c static int test_sev_activation(void) · line 91

Quoted 13 lines
x86/amd_sev.c +8 -0
+     }
+
+     return EXIT_SUCCESS;
+}
+
 int main(void)
 {
      int rtn;
      rtn = test_sev_activation();
      report(rtn == EXIT_SUCCESS, "SEV activation test.");
+     rtn = test_sev_es_activation();
+     report(rtn == EXIT_SUCCESS, "SEV-ES activation test.");
+     rtn = test_sev_es_msr();

There is nothing SEV-ES specific about this function, it only wraps rdmsr/wrmsr, which are supposed to generate #VC exceptions on SEV-ES. Since the same scenario can be covered by running the msr testcase as a SEV-ES guest and observing if it crashes, does testing rdmsr/wrmsr one more time here gain us any new information?

Also, the function gets called from main() even if test_sev_es_activation() failed or SEV-ES was inactive.

Note: More broadly, what are you looking to test for here?
1. wrmsr/rdmsr correctness (rdmsr reads what wrmsr wrote)? or,
2. A #VC exception not causing a guest crash on SEV-ES?

If you are looking to test 1., I suggest letting it be covered by the generic testcases for msr.

Quoted 7 lines

If you are looking to test 2., perhaps a better test is to trigger all scenarios that would cause a #VC exception (eg. test_sev_es_vc_exit) and check that a SEV-ES guest survives.

Regards,
Varad

Hi Varad,

This test case does not bring any SEV-related functionality testing. Instead, it is provided for development, i.e., one can check if SEV is properly set up by monitoring if this test case runs fine without crashes.

Quoted 8 lines

Since this test case is causing some confusion and does not bring any functionality testing, I can remove it from the next version. We can still verify the SEV setup process by checking if an existing test case (e.g., x86/msr.c) runs without crashes in a SEV guest.

It's hard for me to develop a meaningful SEV test case, because I just finished my Google internship and thus lost access to SEV-enabled machines.

Removing this test case is fine. Though, it is convenient. But I agree, it's redundant. Maybe we can tag any tests that are good to run under SEV and/or SEV-ES via the groups field in the x86/unittests.cfg file. The name groups is plural. So I assume that a test can be a member of multiple groups. But I see no examples.

#29 lore raw

Re: [kvm-unit-tests PATCH v3 17/17] x86 AMD SEV-ES: Add test cases

Quoted 76 lines · Marc Orr
Quoted 66 lines · Zixuan Wang
Quoted 49 lines · Varad Gautam

Hi Zixuan,

x86/amd_sev.c static int test_sev_activation(void) · line 78

Quoted 12 lines · Zixuan Wang

From: Zixuan Wang <[email protected]>

x86/amd_sev.c +11 -0
+static int test_sev_es_msr(void)
+{
+     /*
+      * With SEV-ES, rdmsr/wrmsr trigger #VC exception. If #VC is handled
+      * correctly, rdmsr/wrmsr should work like without SEV-ES and not crash
+      * the guest VM.
+      */
+     u64 val = 0x1234;
+     wrmsr(MSR_TSC_AUX, val);
+     if(val != rdmsr(MSR_TSC_AUX)) {
+             return EXIT_FAILURE;

See note below.

x86/amd_sev.c static int test_sev_activation(void) · line 91

Quoted 13 lines
x86/amd_sev.c +8 -0
+     }
+
+     return EXIT_SUCCESS;
+}
+
 int main(void)
 {
      int rtn;
      rtn = test_sev_activation();
      report(rtn == EXIT_SUCCESS, "SEV activation test.");
+     rtn = test_sev_es_activation();
+     report(rtn == EXIT_SUCCESS, "SEV-ES activation test.");
+     rtn = test_sev_es_msr();

There is nothing SEV-ES specific about this function, it only wraps rdmsr/wrmsr, which are supposed to generate #VC exceptions on SEV-ES. Since the same scenario can be covered by running the msr testcase as a SEV-ES guest and observing if it crashes, does testing rdmsr/wrmsr one more time here gain us any new information?

Also, the function gets called from main() even if test_sev_es_activation() failed or SEV-ES was inactive.

Note: More broadly, what are you looking to test for here?
1. wrmsr/rdmsr correctness (rdmsr reads what wrmsr wrote)? or,
2. A #VC exception not causing a guest crash on SEV-ES?

If you are looking to test 1., I suggest letting it be covered by the generic testcases for msr.

Quoted 7 lines

If you are looking to test 2., perhaps a better test is to trigger all scenarios that would cause a #VC exception (eg. test_sev_es_vc_exit) and check that a SEV-ES guest survives.

Regards,
Varad

Hi Varad,

This test case does not bring any SEV-related functionality testing. Instead, it is provided for development, i.e., one can check if SEV is properly set up by monitoring if this test case runs fine without crashes.

Quoted 8 lines

Since this test case is causing some confusion and does not bring any functionality testing, I can remove it from the next version. We can still verify the SEV setup process by checking if an existing test case (e.g., x86/msr.c) runs without crashes in a SEV guest.

It's hard for me to develop a meaningful SEV test case, because I just finished my Google internship and thus lost access to SEV-enabled machines.

Quoted 6 lines

Removing this test case is fine. Though, it is convenient. But I agree, it's redundant. Maybe we can tag any tests that are good to run under SEV and/or SEV-ES via the groups field in the x86/unittests.cfg file. The name groups is plural. So I assume that a test can be a member of multiple groups. But I see no examples.

Yes, groups is specified to accept more than one group with space separation (see the comment block at the top of the unittests file). I see a couple instances where groups are comma separated, but that should be changed, especially since commit b373304853a0 ("scripts:
Fix the check whether testname is in the only_tests list") was merged. I'll send a patch for that.

Thanks,
drew

#30 lore raw

Re: [kvm-unit-tests PATCH v3 17/17] x86 AMD SEV-ES: Add test cases

Quoted 73 lines · Andrew Jones
Quoted 57 lines · Marc Orr
Quoted 47 lines · Zixuan Wang
Quoted 30 lines · Varad Gautam

Hi Zixuan,

x86/amd_sev.c static int test_sev_activation(void) · line 91

Quoted 9 lines · Zixuan Wang

From: Zixuan Wang <[email protected]>
int main(void)

 {
      int rtn;
      rtn = test_sev_activation();
      report(rtn == EXIT_SUCCESS, "SEV activation test.");
x86/amd_sev.c +3 -0
+     rtn = test_sev_es_activation();
+     report(rtn == EXIT_SUCCESS, "SEV-ES activation test.");
+     rtn = test_sev_es_msr();

There is nothing SEV-ES specific about this function, it only wraps rdmsr/wrmsr, which are supposed to generate #VC exceptions on SEV-ES. Since the same scenario can be covered by running the msr testcase as a SEV-ES guest and observing if it crashes, does testing rdmsr/wrmsr one more time here gain us any new information?

Also, the function gets called from main() even if test_sev_es_activation() failed or SEV-ES was inactive.

Note: More broadly, what are you looking to test for here?
1. wrmsr/rdmsr correctness (rdmsr reads what wrmsr wrote)? or,
2. A #VC exception not causing a guest crash on SEV-ES?

If you are looking to test 1., I suggest letting it be covered by the generic testcases for msr.

Quoted 7 lines

If you are looking to test 2., perhaps a better test is to trigger all scenarios that would cause a #VC exception (eg. test_sev_es_vc_exit) and check that a SEV-ES guest survives.

Regards,
Varad

Hi Varad,

This test case does not bring any SEV-related functionality testing. Instead, it is provided for development, i.e., one can check if SEV is properly set up by monitoring if this test case runs fine without crashes.

Quoted 8 lines

Since this test case is causing some confusion and does not bring any functionality testing, I can remove it from the next version. We can still verify the SEV setup process by checking if an existing test case (e.g., x86/msr.c) runs without crashes in a SEV guest.

It's hard for me to develop a meaningful SEV test case, because I just finished my Google internship and thus lost access to SEV-enabled machines.

Quoted 6 lines

Removing this test case is fine. Though, it is convenient. But I agree, it's redundant. Maybe we can tag any tests that are good to run under SEV and/or SEV-ES via the groups field in the x86/unittests.cfg file. The name groups is plural. So I assume that a test can be a member of multiple groups. But I see no examples.

Yes, groups is specified to accept more than one group with space separation (see the comment block at the top of the unittests file). I see a couple instances where groups are comma separated, but that should be changed, especially since commit b373304853a0 ("scripts:
Fix the check whether testname is in the only_tests list") was merged. I'll send a patch for that.

Quoted 3 lines

Thanks,
drew

Great! Marc and I are working on labeling test cases as different groups. We are now trying to label an 'efi' group for test cases that can run under UEFI, and if necessary, a 'sev' group for those that can run under SEV.

Best regards,
Zixuan

#31 lore raw

Re: [kvm-unit-tests PATCH v3 17/17] x86 AMD SEV-ES: Add test cases

In reply to Marc Orr
Quoted 76 lines · Marc Orr
Quoted 66 lines · Zixuan Wang
Quoted 49 lines · Varad Gautam

Hi Zixuan,

x86/amd_sev.c static int test_sev_activation(void) · line 78

Quoted 12 lines · Zixuan Wang

From: Zixuan Wang <[email protected]>

x86/amd_sev.c +11 -0
+static int test_sev_es_msr(void)
+{
+     /*
+      * With SEV-ES, rdmsr/wrmsr trigger #VC exception. If #VC is handled
+      * correctly, rdmsr/wrmsr should work like without SEV-ES and not crash
+      * the guest VM.
+      */
+     u64 val = 0x1234;
+     wrmsr(MSR_TSC_AUX, val);
+     if(val != rdmsr(MSR_TSC_AUX)) {
+             return EXIT_FAILURE;

See note below.

x86/amd_sev.c static int test_sev_activation(void) · line 91

Quoted 13 lines
x86/amd_sev.c +8 -0
+     }
+
+     return EXIT_SUCCESS;
+}
+
 int main(void)
 {
      int rtn;
      rtn = test_sev_activation();
      report(rtn == EXIT_SUCCESS, "SEV activation test.");
+     rtn = test_sev_es_activation();
+     report(rtn == EXIT_SUCCESS, "SEV-ES activation test.");
+     rtn = test_sev_es_msr();

There is nothing SEV-ES specific about this function, it only wraps rdmsr/wrmsr, which are supposed to generate #VC exceptions on SEV-ES. Since the same scenario can be covered by running the msr testcase as a SEV-ES guest and observing if it crashes, does testing rdmsr/wrmsr one more time here gain us any new information?

Also, the function gets called from main() even if test_sev_es_activation() failed or SEV-ES was inactive.

Note: More broadly, what are you looking to test for here?
1. wrmsr/rdmsr correctness (rdmsr reads what wrmsr wrote)? or,
2. A #VC exception not causing a guest crash on SEV-ES?

If you are looking to test 1., I suggest letting it be covered by the generic testcases for msr.

Quoted 7 lines

If you are looking to test 2., perhaps a better test is to trigger all scenarios that would cause a #VC exception (eg. test_sev_es_vc_exit) and check that a SEV-ES guest survives.

Regards,
Varad

Hi Varad,

This test case does not bring any SEV-related functionality testing. Instead, it is provided for development, i.e., one can check if SEV is properly set up by monitoring if this test case runs fine without crashes.

Quoted 8 lines

Since this test case is causing some confusion and does not bring any functionality testing, I can remove it from the next version. We can still verify the SEV setup process by checking if an existing test case (e.g., x86/msr.c) runs without crashes in a SEV guest.

It's hard for me to develop a meaningful SEV test case, because I just finished my Google internship and thus lost access to SEV-enabled machines.

Quoted 6 lines

Removing this test case is fine. Though, it is convenient. But I agree, it's redundant. Maybe we can tag any tests that are good to run under SEV and/or SEV-ES via the groups field in the x86/unittests.cfg file. The name groups is plural. So I assume that a test can be a member of multiple groups. But I see no examples.

Right, from a fleet owner perspective I can imagine the following scenarios being relevant to test for a SEV* offering (and I guess hence make sense to have a special test in kvm-unit-tests for):
- CPUID shows the right SEV level
- C-bit discovery
- GHCB validity (protocol version etc.)

Generic kvm behavior is better tested via the other dedicated tests, which, after the EFI-fication should be no problem to fit into a test plan. The SEV* implementation can then go through the whole battery of kvm-unit-tests plus the SEV* ones.

Regards,
Varad

#32 lore raw

Re: [kvm-unit-tests PATCH v3 17/17] x86 AMD SEV-ES: Add test cases

Quoted 73 lines · Varad Gautam
Quoted 57 lines · Marc Orr
Quoted 47 lines · Zixuan Wang
Quoted 30 lines · Varad Gautam

Hi Zixuan,

x86/amd_sev.c static int test_sev_activation(void) · line 91

Quoted 9 lines · Zixuan Wang

From: Zixuan Wang <[email protected]>
int main(void)

 {
      int rtn;
      rtn = test_sev_activation();
      report(rtn == EXIT_SUCCESS, "SEV activation test.");
x86/amd_sev.c +3 -0
+     rtn = test_sev_es_activation();
+     report(rtn == EXIT_SUCCESS, "SEV-ES activation test.");
+     rtn = test_sev_es_msr();

There is nothing SEV-ES specific about this function, it only wraps rdmsr/wrmsr, which are supposed to generate #VC exceptions on SEV-ES. Since the same scenario can be covered by running the msr testcase as a SEV-ES guest and observing if it crashes, does testing rdmsr/wrmsr one more time here gain us any new information?

Also, the function gets called from main() even if test_sev_es_activation() failed or SEV-ES was inactive.

Note: More broadly, what are you looking to test for here?
1. wrmsr/rdmsr correctness (rdmsr reads what wrmsr wrote)? or,
2. A #VC exception not causing a guest crash on SEV-ES?

If you are looking to test 1., I suggest letting it be covered by the generic testcases for msr.

Quoted 7 lines

If you are looking to test 2., perhaps a better test is to trigger all scenarios that would cause a #VC exception (eg. test_sev_es_vc_exit) and check that a SEV-ES guest survives.

Regards,
Varad

Hi Varad,

This test case does not bring any SEV-related functionality testing. Instead, it is provided for development, i.e., one can check if SEV is properly set up by monitoring if this test case runs fine without crashes.

Quoted 8 lines

Since this test case is causing some confusion and does not bring any functionality testing, I can remove it from the next version. We can still verify the SEV setup process by checking if an existing test case (e.g., x86/msr.c) runs without crashes in a SEV guest.

It's hard for me to develop a meaningful SEV test case, because I just finished my Google internship and thus lost access to SEV-enabled machines.

Quoted 6 lines

Removing this test case is fine. Though, it is convenient. But I agree, it's redundant. Maybe we can tag any tests that are good to run under SEV and/or SEV-ES via the groups field in the x86/unittests.cfg file. The name groups is plural. So I assume that a test can be a member of multiple groups. But I see no examples.

Right, from a fleet owner perspective I can imagine the following scenarios being relevant to test for a SEV* offering (and I guess hence make sense to have a special test in kvm-unit-tests for):
- CPUID shows the right SEV level
- C-bit discovery
- GHCB validity (protocol version etc.)

Quoted 8 lines

Generic kvm behavior is better tested via the other dedicated tests, which, after the EFI-fication should be no problem to fit into a test plan. The SEV* implementation can then go through the whole battery of kvm-unit-tests plus the SEV* ones.

Regards,
Varad

Thank you for the summary! I will put a brief description in the V4 patchset about the future test cases, and a link to this discussion.

Best regards,
ZIxuan

maintainer
#33 lore raw

Re: [kvm-unit-tests PATCH v3 17/17] x86 AMD SEV-ES: Add test cases

In reply to Varad Gautam
Quoted 73 lines · Varad Gautam

Hi Zixuan,

Quoted 37 lines · Zixuan Wang

From: Zixuan Wang <[email protected]>

SEV-ES introduces #VC handler for guest/host communications, e.g., accessing MSR, executing CPUID. This commit provides test cases to check if SEV-ES is enabled and if rdmsr/wrmsr are handled correctly in SEV-ES.

---
  x86/amd_sev.c | 30 ++++++++++++++++++++++++++++++
  1 file changed, 30 insertions(+)
x86/amd_sev.c +11 -0
index a07a48f..21a491c 100644
@@ -13,6 +13,7 @@
  #include "libcflat.h"
  #include "x86/processor.h"
  #include "x86/amd_sev.h"
+#include "msr.h"
 
  #define EXIT_SUCCESS 0
  #define EXIT_FAILURE 1
@@ -55,10 +56,39 @@ static int test_sev_activation(void)
  	return EXIT_SUCCESS;
  }
 
+static int test_sev_es_activation(void)
+{
+	if (!(rdmsr(MSR_SEV_STATUS) & SEV_ES_ENABLED_MASK)) {
+		return EXIT_FAILURE;
+	}
+
+	return EXIT_SUCCESS;
+}
+
+static int test_sev_es_msr(void)

x86/amd_sev.c static int test_sev_activation(void) · line 78

Quoted 10 lines
x86/amd_sev.c +10 -0
+{
+	/*
+	 * With SEV-ES, rdmsr/wrmsr trigger #VC exception. If #VC is handled
+	 * correctly, rdmsr/wrmsr should work like without SEV-ES and not crash
+	 * the guest VM.
+	 */
+	u64 val = 0x1234;
+	wrmsr(MSR_TSC_AUX, val);
+	if(val != rdmsr(MSR_TSC_AUX)) {
+		return EXIT_FAILURE;

See note below.

x86/amd_sev.c static int test_sev_activation(void) · line 91

Quoted 13 lines
x86/amd_sev.c +8 -0
+	}
+
+	return EXIT_SUCCESS;
+}
+
  int main(void)
  {
  	int rtn;
  	rtn = test_sev_activation();
  	report(rtn == EXIT_SUCCESS, "SEV activation test.");
+	rtn = test_sev_es_activation();
+	report(rtn == EXIT_SUCCESS, "SEV-ES activation test.");
+	rtn = test_sev_es_msr();

There is nothing SEV-ES specific about this function, it only wraps rdmsr/wrmsr, which are supposed to generate #VC exceptions on SEV-ES. Since the same scenario can be covered by running the msr testcase as a SEV-ES guest and observing if it crashes, does testing rdmsr/wrmsr one more time here gain us any new information?

Quoted 2 lines

Also, the function gets called from main() even if test_sev_es_activation() failed or SEV-ES was inactive.

Agreed, this patch is still a bit rough. However, a very simple change to report whether SEV-ES is enabled is a good addition to x86/amd_sev.c

Paolo

maintainer
#34 lore raw
In reply to Zixuan Wang
Quoted 1 line · Zixuan Wang

Hello,

WHOA IT WORKS! XD

There are still a few rough edges around the build system (and in general, the test harness is starting to really show its limits), but this is awesome work. Thanks Drew, Varad and Zixuan (in alphabetic and temporal order) for the combined contribution!

For now I've placed it at a 'uefi' branch on gitlab, while I'm waiting for some reviews of my GDT cleanup work. Any future improvements can be done on top.

Paolo

Quoted 196 lines

This patch series updates the x86_64 KVM-Unit-Tests to run under UEFI and culminates in enabling AMD SEV/SEV-ES. The patches are organized as four parts.

The first part (patch 1) refactors the current Multiboot start-up code by converting assembly data structures into C. This enables the follow-up UEFI patches to reuse these data structures without redefining or duplicating them in assembly.

The second part (patches 2-3) copies code from Varad's patch set [1] that builds EFI stubs without depending on GNU-EFI. Part 3 and 4 are built on top of this part.

The third part (patches 4-11) enables the x86_64 test cases to run under UEFI. In particular, these patches allow the x86_64 test cases to be built as EFI executables and take full control of the guest VM. The efi_main() function sets up the KVM-Unit-Tests framework to run under UEFI and then launches the test cases' main() functions. To date, we have 38/43 test cases running with UEFI using this approach.

The fourth part of the series (patches 12-17) focuses on SEV. In particular, these patches introduce SEV/SEV-ES set up code into the EFI set up process, including checking if SEV is supported, setting c-bits for page table entries, and (notably) reusing the UEFI #VC handler so that the set up process does not need to re-implement it (a test case can always implement a new #VC handler and load it after set up is finished). Using this approach, we are able to launch the x86_64 test cases under SEV-ES and exercise KVM's VMGEXIT handler.

Note, a previous feedback [3] indicated that long-term we'd like to instrument KVM-Unit-Tests with it's own #VC handler. However, we still believe that the current approach is good as an intermediate solution, because it unlocks a lot of testing and we do not expect that testing to be inherently tied to the UEFI's #VC handler. Rather, test cases should be tied to the underlying GHCB spec implemented by an arbitrary #VC handler.

See the Part 1 to Part 4 summaries, below, for a high-level breakdown of how the patches are organized.

Part 1 Summary:
Commit 1 refactors boot-related data structures from assembly to C.

Part 2 Summary:
Commits 2-3 copy code from Varad's patch set [1] that implements EFI-related helper functions to replace the GNU-EFI library.

Part 3 Summary:
Commits 4-5 introduce support to build test cases with EFI support.

Commits 6-10 set up KVM-Unit-Tests to run under UEFI. In doing so, these patches incrementally enable most existing x86_64 test cases to run under UEFI.

Commit 11 fixes several test cases that fail to compile with EFI due to UEFI's position independent code (PIC) requirement.

Part 4 Summary:
Commits 12-13 introduce support for SEV by adding code to set the SEV c-bit in page table entries.

Commits 14-16 introduce support for SEV-ES by reusing the UEFI #VC handler in KVM-Unit-Tests. They also fix GDT and IDT issues that occur when reusing UEFI functions in KVM-Unit-Tests.

Commit 17 adds additional test cases for SEV-ES.

Changes V2 -> V3:
V3 Patch # Changes
---------- -------
01/17 (New patch) refactors assembly data structures in C 02/17 Adds a missing alignment attribute
Renames the file uefi.h to efi.h
03/17 Adds an SPDX header, fixes a comment style issue 06/17 Removes assembly data structure definitions 07/17 Removes assembly data structure definitions 12/17 Simplifies an if condition code
14/17 Simplifies an if condition code
15/17 Removes GDT copying for SEV-ES #VC handler

Notes on page table set up code:
Paolo suggested unifying the page table definitions in cstart64.S and UEFI start-up code [5]. We tried but found it hard to implement due to the real/long mode issue: a page table set up function written in C is by default compiled to run in long mode. However, cstart64.S requires page table setup before entering long mode. Calling a long mode function from real/protected mode crashes the guest VM. Thus we chose not to implement this feature in this patch set. More details can be found in our off-list GitHub review [6].

Changes V1 -> V2:
1. Merge Varad's patches [1] as the foundation of our V2 patch set [4].
2. Remove AMD SEV/SEV-ES config flags and macros (patches 11-17)
3. Drop one commit 'x86 UEFI: Move setjmp.h out of desc.h' because we do not link GNU-EFI library.

Notes on authorships and attributions:
The first two commits are from Varad's patch set [1], so they are tagged as 'From:' and 'Signed-off-by:' Varad. Commits 3-7 are from our V1 patch set [2], and since Varad implemented similar code [1], these commits are tagged as 'Co-developed-by:' and 'Signed-off-by:' Varad.

Notes on patch sets merging strategy:
We understand that the current merging strategy (reorganizing and squeezing Varad's patches into two) reduces Varad's authorships, and we hope the additional attribution tags make up for it. We see another approach which is to build our patch set on top of Varad's original patch set, but this creates some noise in the final patch set, e.g., x86/cstart64.S is modified in Varad's part and later reverted in our part as we implement start up code in C. For the sake of the clarity of the code history, we believe the current approach is the best effort so far, and we are open to all kinds of opinions.

[1] https://lore.kernel.org/kvm/[email protected]/
[2] https://lore.kernel.org/kvm/[email protected]/
[3] https://lore.kernel.org/kvm/YSA%[email protected]/
[4] https://lore.kernel.org/kvm/[email protected]/
[5] https://lore.kernel.org/kvm/[email protected]/
[6] https://github.com/marc-orr/KVM-Unit-Tests-dev-fork/pull/1

Regards,
Zixuan Wang

Varad Gautam (2):
x86 UEFI: Copy code from Linux
x86 UEFI: Implement UEFI function calls

Zixuan Wang (15):
x86: Move IDT, GDT and TSS to desc.c
x86 UEFI: Copy code from GNU-EFI
x86 UEFI: Boot from UEFI
x86 UEFI: Load IDT after UEFI boot up
x86 UEFI: Load GDT and TSS after UEFI boot up
x86 UEFI: Set up memory allocator
x86 UEFI: Set up RSDP after UEFI boot up
x86 UEFI: Set up page tables
x86 UEFI: Convert x86 test cases to PIC
x86 AMD SEV: Initial support
x86 AMD SEV: Page table with c-bit
x86 AMD SEV-ES: Check SEV-ES status
x86 AMD SEV-ES: Copy UEFI #VC IDT entry
x86 AMD SEV-ES: Set up GHCB page
x86 AMD SEV-ES: Add test cases

  .gitignore                 |   3 +
  Makefile                   |  29 +-
  README.md                  |   6 +
  configure                  |   6 +
  lib/efi.c                  | 118 ++++++++
  lib/efi.h                  |  22 ++
  lib/linux/efi.h            | 539 +++++++++++++++++++++++++++++++++++++
  lib/x86/acpi.c             |  38 ++-
  lib/x86/acpi.h             |  11 +
  lib/x86/amd_sev.c          | 174 ++++++++++++
  lib/x86/amd_sev.h          |  63 +++++
  lib/x86/asm/page.h         |  28 +-
  lib/x86/asm/setup.h        |  35 +++
  lib/x86/desc.c             |  46 +++-
  lib/x86/desc.h             |   6 +-
  lib/x86/setup.c            | 246 +++++++++++++++++
  lib/x86/usermode.c         |   3 +-
  lib/x86/vm.c               |  18 +-
  x86/Makefile.common        |  68 +++--
  x86/Makefile.i386          |   5 +-
  x86/Makefile.x86_64        |  58 ++--
  x86/access.c               |   9 +-
  x86/amd_sev.c              |  94 +++++++
  x86/cet.c                  |   8 +-
  x86/cstart64.S             |  77 +-----
  x86/efi/README.md          |  63 +++++
  x86/efi/crt0-efi-x86_64.S  |  79 ++++++
  x86/efi/efistart64.S       |  77 ++++++
  x86/efi/elf_x86_64_efi.lds |  81 ++++++
  x86/efi/reloc_x86_64.c     |  96 +++++++
  x86/efi/run                |  63 +++++
  x86/emulator.c             |   5 +-
  x86/eventinj.c             |   6 +-
  x86/run                    |  16 +-
  x86/smap.c                 |   8 +-
  x86/umip.c                 |  10 +-
  x86/vmx.c                  |   8 +-
  37 files changed, 2067 insertions(+), 155 deletions(-)

create mode 100644 lib/efi.c
create mode 100644 lib/efi.h
create mode 100644 lib/linux/efi.h
create mode 100644 lib/x86/amd_sev.c
create mode 100644 lib/x86/amd_sev.h
create mode 100644 lib/x86/asm/setup.h
create mode 100644 x86/amd_sev.c
create mode 100644 x86/efi/README.md
create mode 100644 x86/efi/crt0-efi-x86_64.S
create mode 100644 x86/efi/efistart64.S
create mode 100644 x86/efi/elf_x86_64_efi.lds
create mode 100644 x86/efi/reloc_x86_64.c
create mode 100755 x86/efi/run

#35 lore raw
Quoted 6 lines · Paolo Bonzini
Quoted 1 line · Zixuan Wang

Hello,

WHOA IT WORKS! XD

Quoted 8 lines

There are still a few rough edges around the build system (and in general, the test harness is starting to really show its limits), but this is awesome work. Thanks Drew, Varad and Zixuan (in alphabetic and temporal order) for the combined contribution!

For now I've placed it at a 'uefi' branch on gitlab, while I'm waiting for some reviews of my GDT cleanup work. Any future improvements can be done on top.

Phenomenal! And +1 on thanking Drew, Varad, and Zixuan! (And thanks Paolo for reviewing, testing, and setting up a branch for this work :-).)

Zixuan has actually had a v4 ready to go since last week that incorporates Drew's last round of reviews, but we were holding off on posting it because we kept getting more comments and Zixuan wanted to incorporate all of the feedback :-).

Should we go ahead and post it to the list (and perhaps update the branch in the gitlab so everyone can work off of that)? Or would it be easier to wait for the GDT cleanup work to finalize, and then post it afterward?

Quoted 200 lines

Paolo

Quoted 196 lines

This patch series updates the x86_64 KVM-Unit-Tests to run under UEFI and culminates in enabling AMD SEV/SEV-ES. The patches are organized as four parts.

The first part (patch 1) refactors the current Multiboot start-up code by converting assembly data structures into C. This enables the follow-up UEFI patches to reuse these data structures without redefining or duplicating them in assembly.

The second part (patches 2-3) copies code from Varad's patch set [1] that builds EFI stubs without depending on GNU-EFI. Part 3 and 4 are built on top of this part.

The third part (patches 4-11) enables the x86_64 test cases to run under UEFI. In particular, these patches allow the x86_64 test cases to be built as EFI executables and take full control of the guest VM. The efi_main() function sets up the KVM-Unit-Tests framework to run under UEFI and then launches the test cases' main() functions. To date, we have 38/43 test cases running with UEFI using this approach.

The fourth part of the series (patches 12-17) focuses on SEV. In particular, these patches introduce SEV/SEV-ES set up code into the EFI set up process, including checking if SEV is supported, setting c-bits for page table entries, and (notably) reusing the UEFI #VC handler so that the set up process does not need to re-implement it (a test case can always implement a new #VC handler and load it after set up is finished). Using this approach, we are able to launch the x86_64 test cases under SEV-ES and exercise KVM's VMGEXIT handler.

Note, a previous feedback [3] indicated that long-term we'd like to instrument KVM-Unit-Tests with it's own #VC handler. However, we still believe that the current approach is good as an intermediate solution, because it unlocks a lot of testing and we do not expect that testing to be inherently tied to the UEFI's #VC handler. Rather, test cases should be tied to the underlying GHCB spec implemented by an arbitrary #VC handler.

See the Part 1 to Part 4 summaries, below, for a high-level breakdown of how the patches are organized.

Part 1 Summary:
Commit 1 refactors boot-related data structures from assembly to C.

Part 2 Summary:
Commits 2-3 copy code from Varad's patch set [1] that implements EFI-related helper functions to replace the GNU-EFI library.

Part 3 Summary:
Commits 4-5 introduce support to build test cases with EFI support.

Commits 6-10 set up KVM-Unit-Tests to run under UEFI. In doing so, these patches incrementally enable most existing x86_64 test cases to run under UEFI.

Commit 11 fixes several test cases that fail to compile with EFI due to UEFI's position independent code (PIC) requirement.

Part 4 Summary:
Commits 12-13 introduce support for SEV by adding code to set the SEV c-bit in page table entries.

Commits 14-16 introduce support for SEV-ES by reusing the UEFI #VC handler in KVM-Unit-Tests. They also fix GDT and IDT issues that occur when reusing UEFI functions in KVM-Unit-Tests.

Commit 17 adds additional test cases for SEV-ES.

Changes V2 -> V3:
V3 Patch # Changes
---------- -------
01/17 (New patch) refactors assembly data structures in C 02/17 Adds a missing alignment attribute
Renames the file uefi.h to efi.h
03/17 Adds an SPDX header, fixes a comment style issue 06/17 Removes assembly data structure definitions 07/17 Removes assembly data structure definitions 12/17 Simplifies an if condition code
14/17 Simplifies an if condition code
15/17 Removes GDT copying for SEV-ES #VC handler

Notes on page table set up code:
Paolo suggested unifying the page table definitions in cstart64.S and UEFI start-up code [5]. We tried but found it hard to implement due to the real/long mode issue: a page table set up function written in C is by default compiled to run in long mode. However, cstart64.S requires page table setup before entering long mode. Calling a long mode function from real/protected mode crashes the guest VM. Thus we chose not to implement this feature in this patch set. More details can be found in our off-list GitHub review [6].

Changes V1 -> V2:
1. Merge Varad's patches [1] as the foundation of our V2 patch set [4].
2. Remove AMD SEV/SEV-ES config flags and macros (patches 11-17)
3. Drop one commit 'x86 UEFI: Move setjmp.h out of desc.h' because we do not link GNU-EFI library.

Notes on authorships and attributions:
The first two commits are from Varad's patch set [1], so they are tagged as 'From:' and 'Signed-off-by:' Varad. Commits 3-7 are from our V1 patch set [2], and since Varad implemented similar code [1], these commits are tagged as 'Co-developed-by:' and 'Signed-off-by:' Varad.

Notes on patch sets merging strategy:
We understand that the current merging strategy (reorganizing and squeezing Varad's patches into two) reduces Varad's authorships, and we hope the additional attribution tags make up for it. We see another approach which is to build our patch set on top of Varad's original patch set, but this creates some noise in the final patch set, e.g., x86/cstart64.S is modified in Varad's part and later reverted in our part as we implement start up code in C. For the sake of the clarity of the code history, we believe the current approach is the best effort so far, and we are open to all kinds of opinions.

[1] https://lore.kernel.org/kvm/[email protected]/
[2] https://lore.kernel.org/kvm/[email protected]/
[3] https://lore.kernel.org/kvm/YSA%[email protected]/
[4] https://lore.kernel.org/kvm/[email protected]/
[5] https://lore.kernel.org/kvm/[email protected]/
[6] https://github.com/marc-orr/KVM-Unit-Tests-dev-fork/pull/1

Regards,
Zixuan Wang

Varad Gautam (2):
x86 UEFI: Copy code from Linux
x86 UEFI: Implement UEFI function calls

Zixuan Wang (15):
x86: Move IDT, GDT and TSS to desc.c
x86 UEFI: Copy code from GNU-EFI
x86 UEFI: Boot from UEFI
x86 UEFI: Load IDT after UEFI boot up
x86 UEFI: Load GDT and TSS after UEFI boot up
x86 UEFI: Set up memory allocator
x86 UEFI: Set up RSDP after UEFI boot up
x86 UEFI: Set up page tables
x86 UEFI: Convert x86 test cases to PIC
x86 AMD SEV: Initial support
x86 AMD SEV: Page table with c-bit
x86 AMD SEV-ES: Check SEV-ES status
x86 AMD SEV-ES: Copy UEFI #VC IDT entry
x86 AMD SEV-ES: Set up GHCB page
x86 AMD SEV-ES: Add test cases

  .gitignore                 |   3 +
  Makefile                   |  29 +-
  README.md                  |   6 +
  configure                  |   6 +
  lib/efi.c                  | 118 ++++++++
  lib/efi.h                  |  22 ++
  lib/linux/efi.h            | 539 +++++++++++++++++++++++++++++++++++++
  lib/x86/acpi.c             |  38 ++-
  lib/x86/acpi.h             |  11 +
  lib/x86/amd_sev.c          | 174 ++++++++++++
  lib/x86/amd_sev.h          |  63 +++++
  lib/x86/asm/page.h         |  28 +-
  lib/x86/asm/setup.h        |  35 +++
  lib/x86/desc.c             |  46 +++-
  lib/x86/desc.h             |   6 +-
  lib/x86/setup.c            | 246 +++++++++++++++++
  lib/x86/usermode.c         |   3 +-
  lib/x86/vm.c               |  18 +-
  x86/Makefile.common        |  68 +++--
  x86/Makefile.i386          |   5 +-
  x86/Makefile.x86_64        |  58 ++--
  x86/access.c               |   9 +-
  x86/amd_sev.c              |  94 +++++++
  x86/cet.c                  |   8 +-
  x86/cstart64.S             |  77 +-----
  x86/efi/README.md          |  63 +++++
  x86/efi/crt0-efi-x86_64.S  |  79 ++++++
  x86/efi/efistart64.S       |  77 ++++++
  x86/efi/elf_x86_64_efi.lds |  81 ++++++
  x86/efi/reloc_x86_64.c     |  96 +++++++
  x86/efi/run                |  63 +++++
  x86/emulator.c             |   5 +-
  x86/eventinj.c             |   6 +-
  x86/run                    |  16 +-
  x86/smap.c                 |   8 +-
  x86/umip.c                 |  10 +-
  x86/vmx.c                  |   8 +-
  37 files changed, 2067 insertions(+), 155 deletions(-)

create mode 100644 lib/efi.c
create mode 100644 lib/efi.h
create mode 100644 lib/linux/efi.h
create mode 100644 lib/x86/amd_sev.c
create mode 100644 lib/x86/amd_sev.h
create mode 100644 lib/x86/asm/setup.h
create mode 100644 x86/amd_sev.c
create mode 100644 x86/efi/README.md
create mode 100644 x86/efi/crt0-efi-x86_64.S
create mode 100644 x86/efi/efistart64.S
create mode 100644 x86/efi/elf_x86_64_efi.lds
create mode 100644 x86/efi/reloc_x86_64.c
create mode 100755 x86/efi/run

maintainer
#36 lore raw
Quoted 3 lines · Marc Orr

Should we go ahead and post it to the list (and perhaps update the branch in the gitlab so everyone can work off of that)? Or would it be easier to wait for the GDT cleanup work to finalize, and then post
it afterward?

I would say, just post it on top of the 'uefi' branch. I'm not sure how and when it will be merged in 'master', but for the next few weeks I suppose it's okay if I get incremental patches, and either squash them or commit them on top.

Paolo

#37 lore raw
In reply to Paolo Bonzini
Quoted 9 lines · Paolo Bonzini
Quoted 1 line · Zixuan Wang

Hello,

WHOA IT WORKS! XD

There are still a few rough edges around the build system (and in general, the test harness is starting to really show its limits), but this is awesome work.  Thanks Drew, Varad and Zixuan (in alphabetic and temporal order) for the combined contribution!

For now I've placed it at a 'uefi' branch on gitlab, while I'm waiting for some reviews of my GDT cleanup work.  Any future improvements can be done on top.

While doing the #VC handler support for test binaries [1], I realised I can't seem to run any of the tests from the uefi branch [2] that write to cr3 via setup_vm() on SEV-ES. These tests (eg., tscdeadline_latency) crash with SEV-ES, and work with uefi without SEV-ES (policy=0x0). I'm wondering if I am missing something, is setup_vm->setup_mmu->write_cr3() known to work on SEV-ES elsewhere?

[1] https://lore.kernel.org/all/[email protected]/
[2] https://gitlab.com/kvm-unit-tests/kvm-unit-tests/-/tree/uefi

Thanks,
Varad

Quoted 199 lines

Paolo

Quoted 196 lines

This patch series updates the x86_64 KVM-Unit-Tests to run under UEFI and culminates in enabling AMD SEV/SEV-ES. The patches are organized as four parts.

The first part (patch 1) refactors the current Multiboot start-up code by converting assembly data structures into C. This enables the follow-up UEFI patches to reuse these data structures without redefining or duplicating them in assembly.

The second part (patches 2-3) copies code from Varad's patch set [1] that builds EFI stubs without depending on GNU-EFI. Part 3 and 4 are built on top of this part.

The third part (patches 4-11) enables the x86_64 test cases to run under UEFI. In particular, these patches allow the x86_64 test cases to be built as EFI executables and take full control of the guest VM. The efi_main() function sets up the KVM-Unit-Tests framework to run under UEFI and then launches the test cases' main() functions. To date, we have 38/43 test cases running with UEFI using this approach.

The fourth part of the series (patches 12-17) focuses on SEV. In particular, these patches introduce SEV/SEV-ES set up code into the EFI set up process, including checking if SEV is supported, setting c-bits for page table entries, and (notably) reusing the UEFI #VC handler so that the set up process does not need to re-implement it (a test case can always implement a new #VC handler and load it after set up is finished). Using this approach, we are able to launch the x86_64 test cases under SEV-ES and exercise KVM's VMGEXIT handler.

Note, a previous feedback [3] indicated that long-term we'd like to instrument KVM-Unit-Tests with it's own #VC handler. However, we still believe that the current approach is good as an intermediate solution, because it unlocks a lot of testing and we do not expect that testing to be inherently tied to the UEFI's #VC handler. Rather, test cases should be tied to the underlying GHCB spec implemented by an arbitrary #VC handler.

See the Part 1 to Part 4 summaries, below, for a high-level breakdown of how the patches are organized.

Part 1 Summary:
Commit 1 refactors boot-related data structures from assembly to C.

Part 2 Summary:
Commits 2-3 copy code from Varad's patch set [1] that implements EFI-related helper functions to replace the GNU-EFI library.

Part 3 Summary:
Commits 4-5 introduce support to build test cases with EFI support.

Commits 6-10 set up KVM-Unit-Tests to run under UEFI. In doing so, these patches incrementally enable most existing x86_64 test cases to run under UEFI.

Commit 11 fixes several test cases that fail to compile with EFI due to UEFI's position independent code (PIC) requirement.

Part 4 Summary:
Commits 12-13 introduce support for SEV by adding code to set the SEV c-bit in page table entries.

Commits 14-16 introduce support for SEV-ES by reusing the UEFI #VC handler in KVM-Unit-Tests. They also fix GDT and IDT issues that occur when reusing UEFI functions in KVM-Unit-Tests.

Commit 17 adds additional test cases for SEV-ES.

Changes V2 -> V3:
V3 Patch #  Changes
----------  -------
      01/17  (New patch) refactors assembly data structures in C 02/17  Adds a missing alignment attribute Renames the file uefi.h to efi.h
      03/17  Adds an SPDX header, fixes a comment style issue 06/17  Removes assembly data structure definitions 07/17  Removes assembly data structure definitions 12/17  Simplifies an if condition code
      14/17  Simplifies an if condition code
      15/17  Removes GDT copying for SEV-ES #VC handler

Notes on page table set up code:
Paolo suggested unifying  the page table definitions in cstart64.S and UEFI start-up code [5]. We tried but found it hard to implement due to the real/long mode issue: a page table set up function written in C is by default compiled to run in long mode. However, cstart64.S requires page table setup before entering long mode. Calling a long mode function from real/protected mode crashes the guest VM. Thus we chose not to implement this feature in this patch set. More details can be found in our off-list GitHub review [6].

Changes V1 -> V2:
1. Merge Varad's patches [1] as the foundation of our V2 patch set [4].
2. Remove AMD SEV/SEV-ES config flags and macros (patches 11-17)
3. Drop one commit 'x86 UEFI: Move setjmp.h out of desc.h' because we do not link GNU-EFI library.

Notes on authorships and attributions:
The first two commits are from Varad's patch set [1], so they are tagged as 'From:' and 'Signed-off-by:' Varad. Commits 3-7 are from our V1 patch set [2], and since Varad implemented similar code [1], these commits are tagged as 'Co-developed-by:' and 'Signed-off-by:' Varad.

Notes on patch sets merging strategy:
We understand that the current merging strategy (reorganizing and squeezing Varad's patches into two) reduces Varad's authorships, and we hope the additional attribution tags make up for it. We see another approach which is to build our patch set on top of Varad's original patch set, but this creates some noise in the final patch set, e.g., x86/cstart64.S is modified in Varad's part and later reverted in our part as we implement start up code in C. For the sake of the clarity of the code history, we believe the current approach is the best effort so far, and we are open to all kinds of opinions.

[1] https://lore.kernel.org/kvm/[email protected]/
[2] https://lore.kernel.org/kvm/[email protected]/
[3] https://lore.kernel.org/kvm/YSA%[email protected]/
[4] https://lore.kernel.org/kvm/[email protected]/
[5] https://lore.kernel.org/kvm/[email protected]/
[6] https://github.com/marc-orr/KVM-Unit-Tests-dev-fork/pull/1

Regards,
Zixuan Wang

Varad Gautam (2):
   x86 UEFI: Copy code from Linux
   x86 UEFI: Implement UEFI function calls

Zixuan Wang (15):
   x86: Move IDT, GDT and TSS to desc.c
   x86 UEFI: Copy code from GNU-EFI
   x86 UEFI: Boot from UEFI
   x86 UEFI: Load IDT after UEFI boot up
   x86 UEFI: Load GDT and TSS after UEFI boot up
   x86 UEFI: Set up memory allocator
   x86 UEFI: Set up RSDP after UEFI boot up
   x86 UEFI: Set up page tables
   x86 UEFI: Convert x86 test cases to PIC
   x86 AMD SEV: Initial support
   x86 AMD SEV: Page table with c-bit
   x86 AMD SEV-ES: Check SEV-ES status
   x86 AMD SEV-ES: Copy UEFI #VC IDT entry
   x86 AMD SEV-ES: Set up GHCB page
   x86 AMD SEV-ES: Add test cases

  .gitignore                 |   3 +
  Makefile                   |  29 +-

  README.md                  |   6 +
  configure                  |   6 +
  lib/efi.c                  | 118 ++++++++
  lib/efi.h                  |  22 ++

  lib/linux/efi.h            | 539 +++++++++++++++++++++++++++++++++++++ lib/x86/acpi.c             |  38 ++-
  lib/x86/acpi.h             |  11 +
  lib/x86/amd_sev.c          | 174 ++++++++++++ lib/x86/amd_sev.h          |  63 +++++
  lib/x86/asm/page.h         |  28 +-
  lib/x86/asm/setup.h        |  35 +++
  lib/x86/desc.c             |  46 +++-
  lib/x86/desc.h             |   6 +-
  lib/x86/setup.c            | 246 +++++++++++++++++ lib/x86/usermode.c         |   3 +-
  lib/x86/vm.c               |  18 +-
  x86/Makefile.common        |  68 +++--
  x86/Makefile.i386          |   5 +-
  x86/Makefile.x86_64        |  58 ++--
  x86/access.c               |   9 +-
  x86/amd_sev.c              |  94 +++++++
  x86/cet.c                  |   8 +-
  x86/cstart64.S             |  77 +-----
  x86/efi/README.md          |  63 +++++ x86/efi/crt0-efi-x86_64.S  |  79 ++++++ x86/efi/efistart64.S       |  77 ++++++ x86/efi/elf_x86_64_efi.lds |  81 ++++++ x86/efi/reloc_x86_64.c     |  96 +++++++
  x86/efi/run                |  63 +++++
  x86/emulator.c             |   5 +-
  x86/eventinj.c             |   6 +-
  x86/run                    |  16 +-
  x86/smap.c                 |   8 +-
  x86/umip.c                 |  10 +-
  x86/vmx.c                  |   8 +-

  37 files changed, 2067 insertions(+), 155 deletions(-)

  create mode 100644 lib/efi.c
  create mode 100644 lib/efi.h
  create mode 100644 lib/linux/efi.h
  create mode 100644 lib/x86/amd_sev.c
  create mode 100644 lib/x86/amd_sev.h
  create mode 100644 lib/x86/asm/setup.h
  create mode 100644 x86/amd_sev.c
  create mode 100644 x86/efi/README.md
  create mode 100644 x86/efi/crt0-efi-x86_64.S
  create mode 100644 x86/efi/efistart64.S
  create mode 100644 x86/efi/elf_x86_64_efi.lds
  create mode 100644 x86/efi/reloc_x86_64.c
  create mode 100755 x86/efi/run

#38 lore raw
Quoted 12 lines · Varad Gautam
Quoted 9 lines · Paolo Bonzini
Quoted 1 line · Zixuan Wang

Hello,

WHOA IT WORKS! XD

There are still a few rough edges around the build system (and in general, the test harness is starting to really show its limits), but this is awesome work. Thanks Drew, Varad and Zixuan (in alphabetic and temporal order) for the combined contribution!

For now I've placed it at a 'uefi' branch on gitlab, while I'm waiting for some reviews of my GDT cleanup work. Any future improvements can be done on top.

Quoted 8 lines

While doing the #VC handler support for test binaries [1], I realised I can't seem to run any of the tests from the uefi branch [2] that write to cr3 via setup_vm() on SEV-ES. These tests (eg., tscdeadline_latency) crash with SEV-ES, and work with uefi without SEV-ES (policy=0x0). I'm wondering if I am missing something, is setup_vm->setup_mmu->write_cr3() known to work on SEV-ES elsewhere?

[1] https://lore.kernel.org/all/[email protected]/
[2] https://gitlab.com/kvm-unit-tests/kvm-unit-tests/-/tree/uefi

I've only been running amd_sev under SEV-ES up to now. I just tried tscdeadline_latency on my setup, and can confirm that it does indeed fail under SEV-ES.

#39 lore raw
Quoted 10 lines · Marc Orr
Quoted 9 lines · Varad Gautam
Quoted 7 lines · Paolo Bonzini
Quoted 1 line · Zixuan Wang

Hello,

WHOA IT WORKS! XD

There are still a few rough edges around the build system (and in general, the test harness is starting to really show its limits), but this is awesome work. Thanks Drew, Varad and Zixuan (in alphabetic and temporal order) for the combined contribution!

Quoted 8 lines
Quoted 8 lines
Quoted 2 lines

For now I've placed it at a 'uefi' branch on gitlab, while I'm waiting for some reviews of my GDT cleanup work. Any future improvements can be done on top.

While doing the #VC handler support for test binaries [1], I realised I can't seem to run any of the tests from the uefi branch [2] that write to cr3 via setup_vm() on SEV-ES. These tests (eg., tscdeadline_latency) crash with SEV-ES, and work with uefi without SEV-ES (policy=0x0). I'm wondering if I am missing something, is setup_vm->setup_mmu->write_cr3() known to work on SEV-ES elsewhere?

When writing a new CR3 value, do the new page tables have the GHCB(s) mapped shared?

Thanks,
Tom

Quoted 8 lines
Quoted 3 lines

[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fall%2F20211117134752.32662-1-varad.gautam%40suse.com%2F&amp;data=04%7C01%7CThomas.Lendacky%40amd.com%7C30e4810784c9456a7c4208d9b346bfe9%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637737938743453221%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=%2Fo0aGSzTWbVwLId4gEsnDpYfDsyMWNibjocX6whDK14%3D&amp;reserved=0
[2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.com%2Fkvm-unit-tests%2Fkvm-unit-tests%2F-%2Ftree%2Fuefi&amp;data=04%7C01%7CThomas.Lendacky%40amd.com%7C30e4810784c9456a7c4208d9b346bfe9%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637737938743463179%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=k2kQzSZwmSWNVWWV%2BHJI0cfT71zva3Ify3UHFbSEOyA%3D&amp;reserved=0

I've only been running amd_sev under SEV-ES up to now. I just tried tscdeadline_latency on my setup, and can confirm that it does indeed fail under SEV-ES.