Skip to contentSkip to conversation

Secure Virtual Machine Enablement

PATCH Superseded Thiago Jung Bauermann 3 participants 25 messages Aug 6, 2019 – Aug 16, 2019

Subsystems (2)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover Secure Virtual Machine Enablement

Hello,

There are three new patches from Claudio's "kvmppc: Paravirtualize KVM to support ultravisor" series. Two of them are borrowed (meaning that he will submit them with his series as well) and included here so that this series can stand on its own. They are the one introducing ucall_norets() (patch 1) and the one adding documentation on the Ultravisor (patch 15).

The third new patch is migrating from the kvmppc series to this one because it won't be needed there anymore but is needed here. This is the patch introducing the MSR_S bit (patch 6).

There are also changes to a couple of patches stemming from review comments by Alexey Kardashevskiy and Segher Boessenkool (thanks!). The changelog below has the details.

This series applies on top of v4 of the <asm/mem_encrypt.h> cleanup series:

https://lore.kernel.org/linuxppc-dev/[email protected]/

Everything is available in branch ultravisor-secure-vm at this repo:

https://github.com/bauermann/linux.git

Original cover letter below, and changelog at the bottom:

This series enables Secure Virtual Machines (SVMs) on powerpc. SVMs use the Protected Execution Facility (PEF) and request to be migrated to secure memory during prom_init() so by default all of their memory is inaccessible to the hypervisor. There is an Ultravisor call that the VM can use to request certain pages to be made accessible to (or shared with) the hypervisor.

The objective of these patches is to have the guest perform this request for buffers that need to be accessed by the hypervisor such as the LPPACAs, the SWIOTLB memory and the Debug Trace Log.

Patch 3 ("powerpc: Add support for adding an ESM blob to the zImage wrapper") is posted as RFC because we are still finalizing the details on how the ESM blob will be passed along with the kernel. All other patches are (hopefully) in upstreamable shape and don't depend on this patch.

Unfortunately this series still doesn't enable the use of virtio devices in the secure guest. This support depends on a discussion that is currently ongoing with the virtio community:

https://lore.kernel.org/linuxppc-dev/[email protected]/

I was able to test it using Claudio's patches in the host kernel, booting normally using an initramfs for the root filesystem.

This is the command used to start up the guest with QEMU 4.0:

qemu-system-ppc64				\
	-nodefaults				\
	-cpu host				\
	-machine pseries,accel=kvm,kvm-type=HV,cap-htm=off,cap-cfpc=broken,cap-sbbc=broken,cap-ibs=broken \
	-display none				\
	-serial mon:stdio			\
	-smp 1					\
	-m 4G					\
	-kernel /root/bauermann/vmlinux		\
	-initrd /root/bauermann/fs_small.cpio	\
	-append 'debug'

Changelog

Since v2:

- Patch "powerpc/kernel: Add ucall_norets() ultravisor call handler"
- Borrowed unchanged from Claudio's "kvmppc: Paravirtualize KVM to support ultravisor" series.

- Patch "powerpc/prom_init: Add the ESM call to prom_init"
- Briefly mention in the commit message why we pass the kernel base address and FDT to the Enter Secure Mode ultracall. Suggested by Alexey Kardashevskiy.
- Use enter_secure_mode() version provided by Segher Boessenkool.

- Patch "powerpc/pseries/svm: Add helpers for UV_SHARE_PAGE and UV_UNSHARE_PAGE"
- Use ucall_norets() which doesn't need to be passed a return buffer. Suggested by Alexey Kardashevskiy.

- Patch "powerpc: Introduce the MSR_S bit"
- Moved from Claudio's "kvmppc: Paravirtualize KVM to support ultravisor" series to this series.

- Patch "Documentation/powerpc: Ultravisor API"
- New patch from Sukadev Bhattiprolu. Will also appear on Claudio's kvmppc series.

Since v1:

- Patch "powerpc/pseries: Introduce option to build secure virtual machines"
- Dropped redundant "default n" from CONFIG_PPC_SVM. Suggested by Christoph Hellwig.

- Patch "powerpc: Add support for adding an ESM blob to the zImage wrapper"
- Renamed prom_rtas_os_term_hcall() to prom_rtas_hcall(). Suggested by Alexey Kardashevskiy.
- In prom_rtas_hcall(), changed prom_printf() calls to prom_debug(), and use H_RTAS constant instead of raw value.
- Changed enter_secure_mode() to new ABI passing ucall number in r3. Also changed it to accept kbase argument instead of ESM blob address.
- Changed setup_secure_guest() to only make the ESM ultracall if svm=1 was passed on the kernel command line.

- Patch "powerpc/pseries/svm: Unshare all pages before kexecing a new kernel"
- New patch from Ram Pai.

- Patch "powerpc/pseries/svm: Force SWIOTLB for secure guests"
- No need to define sme_me_mask, sme_active() and sev_active() anymore.
- Add definitions for mem_encrypt_active() and force_dma_unencrypted().
- Select ARCH_HAS_FORCE_DMA_UNENCRYPTED in CONFIG_PPC_SVM.

Anshuman Khandual (3):
powerpc/pseries/svm: Use shared memory for LPPACA structures powerpc/pseries/svm: Use shared memory for Debug Trace Log (DTL) powerpc/pseries/svm: Force SWIOTLB for secure guests

Benjamin Herrenschmidt (1):
powerpc: Add support for adding an ESM blob to the zImage wrapper

Claudio Carvalho (1):
powerpc/kernel: Add ucall_norets() ultravisor call handler

Ram Pai (3):
powerpc/prom_init: Add the ESM call to prom_init powerpc/pseries/svm: Add helpers for UV_SHARE_PAGE and UV_UNSHARE_PAGE powerpc/pseries/svm: Unshare all pages before kexecing a new kernel

Ryan Grimm (2):
powerpc/pseries/svm: Export guest SVM status to user space via sysfs powerpc/configs: Enable secure guest support in pseries and ppc64
defconfigs

Sukadev Bhattiprolu (3):
powerpc: Introduce the MSR_S bit powerpc/pseries/svm: Disable doorbells in SVM guests Documentation/powerpc: Ultravisor API

Thiago Jung Bauermann (3):
powerpc/pseries: Introduce option to build secure virtual machines powerpc/pseries: Add and use LPPACA_SIZE constant powerpc/pseries/iommu: Don't use dma_iommu_ops on secure guests

 .../admin-guide/kernel-parameters.txt         |    5 +
 Documentation/powerpc/ultravisor.rst          | 1048 +++++++++++++++++
 arch/powerpc/boot/main.c                      |   41 +
 arch/powerpc/boot/ops.h                       |    2 +
 arch/powerpc/boot/wrapper                     |   24 +-
 arch/powerpc/boot/zImage.lds.S                |    8 +
 arch/powerpc/configs/ppc64_defconfig          |    1 +
 arch/powerpc/configs/pseries_defconfig        |    1 +
 arch/powerpc/include/asm/asm-prototypes.h     |   11 +
 arch/powerpc/include/asm/mem_encrypt.h        |   26 +
 arch/powerpc/include/asm/reg.h                |    3 +
 arch/powerpc/include/asm/svm.h                |   31 +
 arch/powerpc/include/asm/ultravisor-api.h     |   29 +
 arch/powerpc/include/asm/ultravisor.h         |   29 +
 arch/powerpc/kernel/Makefile                  |    3 +
 arch/powerpc/kernel/machine_kexec_64.c        |    9 +
 arch/powerpc/kernel/paca.c                    |   52 +-
 arch/powerpc/kernel/prom_init.c               |   96 ++
 arch/powerpc/kernel/sysfs.c                   |   29 +
 arch/powerpc/kernel/ucall.S                   |   20 +
 arch/powerpc/platforms/pseries/Kconfig        |   14 +
 arch/powerpc/platforms/pseries/Makefile       |    1 +
 arch/powerpc/platforms/pseries/iommu.c        |    6 +-
 arch/powerpc/platforms/pseries/setup.c        |    5 +-
 arch/powerpc/platforms/pseries/smp.c          |    3 +-
 arch/powerpc/platforms/pseries/svm.c          |   85 ++
 26 files changed, 1571 insertions(+), 11 deletions(-)
 create mode 100644 Documentation/powerpc/ultravisor.rst
 create mode 100644 arch/powerpc/include/asm/mem_encrypt.h
 create mode 100644 arch/powerpc/include/asm/svm.h
 create mode 100644 arch/powerpc/include/asm/ultravisor-api.h
 create mode 100644 arch/powerpc/include/asm/ultravisor.h
 create mode 100644 arch/powerpc/kernel/ucall.S
 create mode 100644 arch/powerpc/platforms/pseries/svm.c
#1 lore raw

PATCH v3 1/16 powerpc/kernel: Add ucall_norets() ultravisor call handler

From: Claudio Carvalho <[email protected]>

The ultracalls (ucalls for short) allow the Secure Virtual Machines (SVM)s and hypervisor to request services from the ultravisor such as accessing a register or memory region that can only be accessed when running in ultravisor-privileged mode.

This patch adds ucall_norets() ultravisor call handler. Like plpar_hcall_norets(), it also saves and restores the Condition Register (CR).

The specific service needed from an ucall is specified in register R3 (the first parameter to the ucall). Other parameters to the ucall, if any, are specified in registers R4 through R12.

Return value of all ucalls is in register R3. Other output values from the ucall, if any, are returned in registers R4 through R12.

Each ucall returns specific error codes, applicable in the context of the ucall. However, like with the PowerPC Architecture Platform Reference (PAPR), if no specific error code is defined for a particular situation, then the ucall will fallback to an erroneous parameter-position based code. i.e U_PARAMETER, U_P2, U_P3 etc depending on the ucall parameter that may have caused the error.

Every host kernel (powernv) needs to be able to do ucalls in case it ends up being run in a machine with ultravisor enabled. Otherwise, the kernel may crash early in boot trying to access ultravisor resources, for instance, trying to set the partition table entry 0.

Since secure guests also need to be able to do ucalls and its kernel may not have CONFIG_PPC_POWERNV=y, the ucall.S file is placed under arch/powerpc/kernel.

If ultravisor is not enabled, the ucalls will be redirected to the hypervisor which must handle/fail the call.

Thanks to inputs from Ram Pai and Michael Anderson.

---
 arch/powerpc/include/asm/asm-prototypes.h | 11 +++++++++++
 arch/powerpc/include/asm/ultravisor-api.h | 23 +++++++++++++++++++++++
 arch/powerpc/kernel/Makefile              |  1 +
 arch/powerpc/kernel/ucall.S               | 20 ++++++++++++++++++++
 4 files changed, 55 insertions(+)
arch/powerpc/include/asm/asm-prototypes.h +11 -0
index ec1c97a8e8cb..e698f48cbc6d 100644
@@ -15,6 +15,7 @@
 #include <asm/epapr_hcalls.h>
 #include <asm/dcr.h>
 #include <asm/mmu_context.h>
+#include <asm/ultravisor-api.h>
 
 #include <uapi/asm/ucontext.h>
 
@@ -34,6 +35,16 @@ extern struct static_key hcall_tracepoint_key;
 void __trace_hcall_entry(unsigned long opcode, unsigned long *args);
 void __trace_hcall_exit(long opcode, long retval, unsigned long *retbuf);
 
+/* Ultravisor */
+#ifdef CONFIG_PPC_POWERNV
+long ucall_norets(unsigned long opcode, ...);
+#else
+static inline long ucall_norets(unsigned long opcode, ...)
+{
+	return U_NOT_AVAILABLE;
+}
+#endif
+
 /* OPAL */
 int64_t __opal_call(int64_t a0, int64_t a1, int64_t a2, int64_t a3,
 		    int64_t a4, int64_t a5, int64_t a6, int64_t a7,
arch/powerpc/include/asm/ultravisor-api.h new file +23 -0
new file mode 100644
index 000000000000..88ffa78f9d61
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Ultravisor API.
+ *
+ * Copyright 2019, IBM Corporation.
+ *
+ */
+#ifndef _ASM_POWERPC_ULTRAVISOR_API_H
+#define _ASM_POWERPC_ULTRAVISOR_API_H
+
+#include <asm/hvcall.h>
+
+/* Return codes */
+#define U_FUNCTION		H_FUNCTION
+#define U_NOT_AVAILABLE		H_NOT_AVAILABLE
+#define U_P2			H_P2
+#define U_P3			H_P3
+#define U_P4			H_P4
+#define U_P5			H_P5
+#define U_PARAMETER		H_PARAMETER
+#define U_SUCCESS		H_SUCCESS
+
+#endif /* _ASM_POWERPC_ULTRAVISOR_API_H */
arch/powerpc/kernel/Makefile +1 -0
index ea0c69236789..c6c4ea240b2a 100644
@@ -156,6 +156,7 @@ endif
 
 obj-$(CONFIG_EPAPR_PARAVIRT)	+= epapr_paravirt.o epapr_hcalls.o
 obj-$(CONFIG_KVM_GUEST)		+= kvm.o kvm_emul.o
+obj-$(CONFIG_PPC_POWERNV)	+= ucall.o
 
 # Disable GCOV, KCOV & sanitizers in odd or sensitive code
 GCOV_PROFILE_prom_init.o := n
arch/powerpc/kernel/ucall.S new file +20 -0
new file mode 100644
index 000000000000..de9133e45d21
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Generic code to perform an ultravisor call.
+ *
+ * Copyright 2019, IBM Corporation.
+ *
+ */
+#include <asm/ppc_asm.h>
+#include <asm/export.h>
+
+_GLOBAL(ucall_norets)
+EXPORT_SYMBOL_GPL(ucall_norets)
+	mfcr	r0
+	stw	r0,8(r1)
+
+	sc	2		/* Invoke the ultravisor */
+
+	lwz	r0,8(r1)
+	mtcrf	0xff,r0
+	blr			/* Return r3 = status */
#2 lore raw

PATCH v3 2/16 powerpc/pseries: Introduce option to build secure virtual machines

In reply to Thiago Jung Bauermann

Introduce CONFIG_PPC_SVM to control support for secure guests and include Ultravisor-related helpers when it is selected

---
 arch/powerpc/include/asm/asm-prototypes.h |  2 +-
 arch/powerpc/kernel/Makefile              |  4 +++-
 arch/powerpc/platforms/pseries/Kconfig    | 11 +++++++++++
 3 files changed, 15 insertions(+), 2 deletions(-)
arch/powerpc/include/asm/asm-prototypes.h +1 -1
index e698f48cbc6d..49196d35e3bb 100644
@@ -36,7 +36,7 @@ void __trace_hcall_entry(unsigned long opcode, unsigned long *args);
 void __trace_hcall_exit(long opcode, long retval, unsigned long *retbuf);
 
 /* Ultravisor */
-#ifdef CONFIG_PPC_POWERNV
+#if defined(CONFIG_PPC_POWERNV) || defined(CONFIG_PPC_SVM)
 long ucall_norets(unsigned long opcode, ...);
 #else
 static inline long ucall_norets(unsigned long opcode, ...)
arch/powerpc/kernel/Makefile +3 -1
index c6c4ea240b2a..ba379dfb8b83 100644
@@ -156,7 +156,9 @@ endif
 
 obj-$(CONFIG_EPAPR_PARAVIRT)	+= epapr_paravirt.o epapr_hcalls.o
 obj-$(CONFIG_KVM_GUEST)		+= kvm.o kvm_emul.o
-obj-$(CONFIG_PPC_POWERNV)	+= ucall.o
+ifneq ($(CONFIG_PPC_POWERNV)$(CONFIG_PPC_SVM),)
+obj-y				+= ucall.o
+endif
 
 # Disable GCOV, KCOV & sanitizers in odd or sensitive code
 GCOV_PROFILE_prom_init.o := n
arch/powerpc/platforms/pseries/Kconfig +11 -0
index f7b484f55553..d09deb05bb66 100644
@@ -145,3 +145,14 @@ config PAPR_SCM
 	tristate "Support for the PAPR Storage Class Memory interface"
 	help
 	  Enable access to hypervisor provided storage class memory.
+
+config PPC_SVM
+	bool "Secure virtual machine (SVM) support for POWER"
+	depends on PPC_PSERIES
+	help
+	 There are certain POWER platforms which support secure guests using
+	 the Protected Execution Facility, with the help of an Ultravisor
+	 executing below the hypervisor layer. This enables support for
+	 those guests.
+
+	 If unsure, say "N".
#3 lore raw

RFC PATCH v3 3/16 powerpc: Add support for adding an ESM blob to the zImage wrapper

In reply to Thiago Jung Bauermann

From: Benjamin Herrenschmidt <[email protected]>

For secure VMs, the signing tool will create a ticket called the "ESM blob" for the Enter Secure Mode ultravisor call with the signatures of the kernel and initrd among other things.

This adds support to the wrapper script for adding that blob via the "-e" option to the zImage.pseries.

It also adds code to the zImage wrapper itself to retrieve and if necessary relocate the blob, and pass its address to Linux via the device-tree, to be later consumed by prom_init.

[ bauerman: Minor adjustments to some comments. ]
---
 arch/powerpc/boot/main.c       | 41 ++++++++++++++++++++++++++++++++++
 arch/powerpc/boot/ops.h        |  2 ++
 arch/powerpc/boot/wrapper      | 24 +++++++++++++++++---
 arch/powerpc/boot/zImage.lds.S |  8 +++++++
 4 files changed, 72 insertions(+), 3 deletions(-)
arch/powerpc/boot/main.c +41 -0
index 102cc546444d..a9d209135975 100644
@@ -146,6 +146,46 @@ static struct addr_range prep_initrd(struct addr_range vmlinux, void *chosen,
 	return (struct addr_range){(void *)initrd_addr, initrd_size};
 }
 
+#ifdef __powerpc64__
+static void prep_esm_blob(struct addr_range vmlinux, void *chosen)
+{
+	unsigned long esm_blob_addr, esm_blob_size;
+
+	/* Do we have an ESM (Enter Secure Mode) blob? */
+	if (_esm_blob_end <= _esm_blob_start)
+		return;
+
+	printf("Attached ESM blob at 0x%p-0x%p\n\r",
+	       _esm_blob_start, _esm_blob_end);
+	esm_blob_addr = (unsigned long)_esm_blob_start;
+	esm_blob_size = _esm_blob_end - _esm_blob_start;
+
+	/*
+	 * If the ESM blob is too low it will be clobbered when the
+	 * kernel relocates to its final location.  In this case,
+	 * allocate a safer place and move it.
+	 */
+	if (esm_blob_addr < vmlinux.size) {
+		void *old_addr = (void *)esm_blob_addr;
+
+		printf("Allocating 0x%lx bytes for esm_blob ...\n\r",
+		       esm_blob_size);
+		esm_blob_addr = (unsigned long)malloc(esm_blob_size);
+		if (!esm_blob_addr)
+			fatal("Can't allocate memory for ESM blob !\n\r");
+		printf("Relocating ESM blob 0x%lx <- 0x%p (0x%lx bytes)\n\r",
+		       esm_blob_addr, old_addr, esm_blob_size);
+		memmove((void *)esm_blob_addr, old_addr, esm_blob_size);
+	}
+
+	/* Tell the kernel ESM blob address via device tree. */
+	setprop_val(chosen, "linux,esm-blob-start", (u32)(esm_blob_addr));
+	setprop_val(chosen, "linux,esm-blob-end", (u32)(esm_blob_addr + esm_blob_size));
+}
+#else
+static inline void prep_esm_blob(struct addr_range vmlinux, void *chosen) { }
+#endif
+
 /* A buffer that may be edited by tools operating on a zImage binary so as to
  * edit the command line passed to vmlinux (by setting /chosen/bootargs).
  * The buffer is put in it's own section so that tools may locate it easier.
@@ -214,6 +254,7 @@ void start(void)
 	vmlinux = prep_kernel();
 	initrd = prep_initrd(vmlinux, chosen,
 			     loader_info.initrd_addr, loader_info.initrd_size);
+	prep_esm_blob(vmlinux, chosen);
 	prep_cmdline(chosen);
 
 	printf("Finalizing device tree...");
arch/powerpc/boot/ops.h +2 -0
index cd043726ed88..e0606766480f 100644
@@ -251,6 +251,8 @@ extern char _initrd_start[];
 extern char _initrd_end[];
 extern char _dtb_start[];
 extern char _dtb_end[];
+extern char _esm_blob_start[];
+extern char _esm_blob_end[];
 
 static inline __attribute__((const))
 int __ilog2_u32(u32 n)
arch/powerpc/boot/wrapper +21 -3
index 5148ac271f28..ed6266367bc0 100755
@@ -13,6 +13,7 @@
 # -i initrd	specify initrd file
 # -d devtree	specify device-tree blob
 # -s tree.dts	specify device-tree source file (needs dtc installed)
+# -e esm_blob   specify ESM blob for secure images
 # -c		cache $kernel.strip.gz (use if present & newer, else make)
 # -C prefix	specify command prefix for cross-building tools
 #		(strip, objcopy, ld)
@@ -37,6 +38,7 @@ platform=of
 initrd=
 dtb=
 dts=
+esm_blob=
 cacheit=
 binary=
 compression=.gz
@@ -60,9 +62,9 @@ tmpdir=.
 
 usage() {
     echo 'Usage: wrapper [-o output] [-p platform] [-i initrd]' >&2
-    echo '       [-d devtree] [-s tree.dts] [-c] [-C cross-prefix]' >&2
-    echo '       [-D datadir] [-W workingdir] [-Z (gz|xz|none)]' >&2
-    echo '       [--no-compression] [vmlinux]' >&2
+    echo '       [-d devtree] [-s tree.dts] [-e esm_blob]' >&2
+    echo '       [-c] [-C cross-prefix] [-D datadir] [-W workingdir]' >&2
+    echo '       [-Z (gz|xz|none)] [--no-compression] [vmlinux]' >&2
     exit 1
 }
 
@@ -105,6 +107,11 @@ while [ "$#" -gt 0 ]; do
 	[ "$#" -gt 0 ] || usage
 	dtb="$1"
 	;;
+    -e)
+	shift
+	[ "$#" -gt 0 ] || usage
+	esm_blob="$1"
+	;;
     -s)
 	shift
 	[ "$#" -gt 0 ] || usage
@@ -218,9 +225,16 @@ objflags=-S
 tmp=$tmpdir/zImage.$$.o
 ksection=.kernel:vmlinux.strip
 isection=.kernel:initrd
+esection=.kernel:esm_blob
 link_address='0x400000'
 make_space=y
 
+
+if [ -n "$esm_blob" -a "$platform" != "pseries" ]; then
+    echo "ESM blob not support on non-pseries platforms" >&2
+    exit 1
+fi
+
 case "$platform" in
 of)
     platformo="$object/of.o $object/epapr.o"
@@ -477,6 +491,10 @@ if [ -n "$dtb" ]; then
     fi
 fi
 
+if [ -n "$esm_blob" ]; then
+    addsec $tmp "$esm_blob" $esection
+fi
+
 if [ "$platform" != "miboot" ]; then
     if [ -n "$link_address" ] ; then
         text_start="-Ttext $link_address"
arch/powerpc/boot/zImage.lds.S +8 -0
index 4ac1e36edfe7..a21f3a76e06f 100644
@@ -68,6 +68,14 @@ SECTIONS
     _initrd_end =  .;
   }
 
+  . = ALIGN(4096);
+  .kernel:esm_blob :
+  {
+    _esm_blob_start =  .;
+    *(.kernel:esm_blob)
+    _esm_blob_end =  .;
+  }
+
 #ifdef CONFIG_PPC64_BOOT_WRAPPER
   . = ALIGN(256);
   .got :
#4 lore raw

PATCH v3 4/16 powerpc/prom_init: Add the ESM call to prom_init

In reply to Thiago Jung Bauermann

From: Ram Pai <[email protected]>

Make the Enter-Secure-Mode (ESM) ultravisor call to switch the VM to secure mode. Pass kernel base address and FDT address so that the Ultravisor is able to verify the integrity of the VM using information from the ESM blob.

Add "svm=" command line option to turn on switching to secure mode.

[ andmike: Generate an RTAS os-term hcall when the ESM ucall fails. ]
[ bauerman: Cleaned up the code a bit. ]
---
 .../admin-guide/kernel-parameters.txt         |  5 +
 arch/powerpc/include/asm/ultravisor-api.h     |  3 +
 arch/powerpc/kernel/prom_init.c               | 96 +++++++++++++++++++
 3 files changed, 104 insertions(+)
Documentation/admin-guide/kernel-parameters.txt +5 -0
index 7ccd158b3894..231a008b7961 100644
@@ -4620,6 +4620,11 @@
 			/sys/power/pm_test). Only available when CONFIG_PM_DEBUG
 			is set. Default value is 5.
 
+	svm=		[PPC]
+			Format: { on | off | y | n | 1 | 0 }
+			This parameter controls use of the Protected
+			Execution Facility on pSeries.
+
 	swapaccount=[0|1]
 			[KNL] Enable accounting of swap in memory resource
 			controller if no parameter or 1 is given or disable
arch/powerpc/include/asm/ultravisor-api.h +3 -0
index 88ffa78f9d61..d3503d1f447e 100644
@@ -20,4 +20,7 @@
 #define U_PARAMETER		H_PARAMETER
 #define U_SUCCESS		H_SUCCESS
 
+/* opcodes */
+#define UV_ESM				0xF110
+
 #endif /* _ASM_POWERPC_ULTRAVISOR_API_H */
arch/powerpc/kernel/prom_init.c +96 -0

132 lines not on this page · raw message

#5 lore raw

PATCH v3 5/16 powerpc/pseries/svm: Add helpers for UV_SHARE_PAGE and UV_UNSHARE_PAGE

In reply to Thiago Jung Bauermann

From: Ram Pai <[email protected]>

These functions are used when the guest wants to grant the hypervisor access to certain pages.

---
 arch/powerpc/include/asm/ultravisor-api.h |  2 ++
 arch/powerpc/include/asm/ultravisor.h     | 24 +++++++++++++++++++++++
 2 files changed, 26 insertions(+)
arch/powerpc/include/asm/ultravisor-api.h +2 -0
index d3503d1f447e..142b0576b89f 100644
@@ -22,5 +22,7 @@
 
 /* opcodes */
 #define UV_ESM				0xF110
+#define UV_SHARE_PAGE			0xF130
+#define UV_UNSHARE_PAGE			0xF134
 
 #endif /* _ASM_POWERPC_ULTRAVISOR_API_H */
arch/powerpc/include/asm/ultravisor.h new file +24 -0
new file mode 100644
index 000000000000..a930aec8c1e3
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Ultravisor definitions
+ *
+ * Copyright 2019, IBM Corporation.
+ *
+ */
+#ifndef _ASM_POWERPC_ULTRAVISOR_H
+#define _ASM_POWERPC_ULTRAVISOR_H
+
+#include <asm/asm-prototypes.h>
+#include <asm/ultravisor-api.h>
+
+static inline int uv_share_page(u64 pfn, u64 npages)
+{
+	return ucall_norets(UV_SHARE_PAGE, pfn, npages);
+}
+
+static inline int uv_unshare_page(u64 pfn, u64 npages)
+{
+	return ucall_norets(UV_UNSHARE_PAGE, pfn, npages);
+}
+
+#endif	/* _ASM_POWERPC_ULTRAVISOR_H */
#6 lore raw

PATCH v3 6/16 powerpc: Introduce the MSR_S bit

In reply to Thiago Jung Bauermann

From: Sukadev Bhattiprolu <[email protected]>

The ultravisor processor mode is introduced in POWER platforms that supports the Protected Execution Facility (PEF). Ultravisor is higher privileged than hypervisor mode.

In PEF enabled platforms, the MSR_S bit is used to indicate if the thread is in secure state. With the MSR_S bit, the privilege state of the thread is now determined by MSR_S, MSR_HV and MSR_PR, as follows:

HV PR S=0 S=1
---------------------------------------------
0 0 privileged privileged (secure guest kernel)
0 1 problem problem (secure guest userspace)
1 0 hypervisor ultravisor
1 1 problem reserved

---
 arch/powerpc/include/asm/reg.h | 3 +++
 1 file changed, 3 insertions(+)
arch/powerpc/include/asm/reg.h +3 -0
index 10caa145f98b..ec3714cf0989 100644
@@ -38,6 +38,7 @@
 #define MSR_TM_LG	32		/* Trans Mem Available */
 #define MSR_VEC_LG	25	        /* Enable AltiVec */
 #define MSR_VSX_LG	23		/* Enable VSX */
+#define MSR_S_LG	22		/* Secure state */
 #define MSR_POW_LG	18		/* Enable Power Management */
 #define MSR_WE_LG	18		/* Wait State Enable */
 #define MSR_TGPR_LG	17		/* TLB Update registers in use */
@@ -71,11 +72,13 @@
 #define MSR_SF		__MASK(MSR_SF_LG)	/* Enable 64 bit mode */
 #define MSR_ISF		__MASK(MSR_ISF_LG)	/* Interrupt 64b mode valid on 630 */
 #define MSR_HV 		__MASK(MSR_HV_LG)	/* Hypervisor state */
+#define MSR_S		__MASK(MSR_S_LG)	/* Secure state */
 #else
 /* so tests for these bits fail on 32-bit */
 #define MSR_SF		0
 #define MSR_ISF		0
 #define MSR_HV		0
+#define MSR_S		0
 #endif
 
 /*
#7 lore raw

PATCH v3 7/16 powerpc/pseries: Add and use LPPACA_SIZE constant

In reply to Thiago Jung Bauermann

Helps document what the hard-coded number means.

Also take the opportunity to fix an #endif comment.

---
 arch/powerpc/kernel/paca.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
arch/powerpc/kernel/paca.c +6 -5
index e3ad8aa4730d..612fc87ef785 100644
@@ -52,6 +52,8 @@ static void *__init alloc_paca_data(unsigned long size, unsigned long align,
 
 #ifdef CONFIG_PPC_PSERIES
 
+#define LPPACA_SIZE 0x400
+
 /*
  * See asm/lppaca.h for more detail.
  *
@@ -65,7 +67,7 @@ static inline void init_lppaca(struct lppaca *lppaca)
 
 	*lppaca = (struct lppaca) {
 		.desc = cpu_to_be32(0xd397d781),	/* "LpPa" */
-		.size = cpu_to_be16(0x400),
+		.size = cpu_to_be16(LPPACA_SIZE),
 		.fpregs_in_use = 1,
 		.slb_count = cpu_to_be16(64),
 		.vmxregs_in_use = 0,
@@ -75,19 +77,18 @@ static inline void init_lppaca(struct lppaca *lppaca)
 static struct lppaca * __init new_lppaca(int cpu, unsigned long limit)
 {
 	struct lppaca *lp;
-	size_t size = 0x400;
 
-	BUILD_BUG_ON(size < sizeof(struct lppaca));
+	BUILD_BUG_ON(sizeof(struct lppaca) > LPPACA_SIZE);
 
 	if (early_cpu_has_feature(CPU_FTR_HVMODE))
 		return NULL;
 
-	lp = alloc_paca_data(size, 0x400, limit, cpu);
+	lp = alloc_paca_data(LPPACA_SIZE, 0x400, limit, cpu);
 	init_lppaca(lp);
 
 	return lp;
 }
-#endif /* CONFIG_PPC_BOOK3S */
+#endif /* CONFIG_PPC_PSERIES */
 
 #ifdef CONFIG_PPC_BOOK3S_64
 
#8 lore raw

PATCH v3 8/16 powerpc/pseries/svm: Use shared memory for LPPACA structures

In reply to Thiago Jung Bauermann

From: Anshuman Khandual <[email protected]>

LPPACA structures need to be shared with the host. Hence they need to be in shared memory. Instead of allocating individual chunks of memory for a given structure from memblock, a contiguous chunk of memory is allocated and then converted into shared memory. Subsequent allocation requests will come from the contiguous chunk which will be always shared memory for all structures.

While we are able to use a kmem_cache constructor for the Debug Trace Log, LPPACAs are allocated very early in the boot process (before SLUB is available) so we need to use a simpler scheme here.

Introduce helper is_svm_platform() which uses the S bit of the MSR to tell whether we're running as a secure guest.

---
 arch/powerpc/include/asm/svm.h | 26 ++++++++++++++++++++
 arch/powerpc/kernel/paca.c     | 43 +++++++++++++++++++++++++++++++++-
 2 files changed, 68 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/svm.h new file +26 -0
new file mode 100644
index 000000000000..fef3740f46a6
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * SVM helper functions
+ *
+ * Copyright 2019 Anshuman Khandual, IBM Corporation.
+ */
+
+#ifndef _ASM_POWERPC_SVM_H
+#define _ASM_POWERPC_SVM_H
+
+#ifdef CONFIG_PPC_SVM
+
+static inline bool is_secure_guest(void)
+{
+	return mfmsr() & MSR_S;
+}
+
+#else /* CONFIG_PPC_SVM */
+
+static inline bool is_secure_guest(void)
+{
+	return false;
+}
+
+#endif /* CONFIG_PPC_SVM */
+#endif /* _ASM_POWERPC_SVM_H */
arch/powerpc/kernel/paca.c +42 -1

61 lines not on this page · raw message

#9 lore raw

Re: [PATCH v3 08/16] powerpc/pseries/svm: Use shared memory for LPPACA structures

Quoted 29 lines · Thiago Jung Bauermann

From: Anshuman Khandual <[email protected]>

LPPACA structures need to be shared with the host. Hence they need to be in shared memory. Instead of allocating individual chunks of memory for a given structure from memblock, a contiguous chunk of memory is allocated and then converted into shared memory. Subsequent allocation requests will come from the contiguous chunk which will be always shared memory for all structures.

While we are able to use a kmem_cache constructor for the Debug Trace Log, LPPACAs are allocated very early in the boot process (before SLUB is available) so we need to use a simpler scheme here.

Introduce helper is_svm_platform() which uses the S bit of the MSR to tell whether we're running as a secure guest.

---
 arch/powerpc/include/asm/svm.h | 26 ++++++++++++++++++++
 arch/powerpc/kernel/paca.c     | 43 +++++++++++++++++++++++++++++++++-
 2 files changed, 68 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/svm.h new file +0 -0
new file mode 100644
index 000000000000..fef3740f46a6

arch/powerpc/include/asm/svm.h line 5

Quoted 5 lines
arch/powerpc/include/asm/svm.h new file +5 -0
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * SVM helper functions
+ *
+ * Copyright 2019 Anshuman Khandual, IBM Corporation.

Are we sure this copyright date is correct?

cheers

#10 lore raw

Re: [PATCH v3 08/16] powerpc/pseries/svm: Use shared memory for LPPACA structures

Quoted 24 lines · Michael Ellerman
Quoted 23 lines · Thiago Jung Bauermann

From: Anshuman Khandual <[email protected]>

LPPACA structures need to be shared with the host. Hence they need to be in shared memory. Instead of allocating individual chunks of memory for a given structure from memblock, a contiguous chunk of memory is allocated and then converted into shared memory. Subsequent allocation requests will come from the contiguous chunk which will be always shared memory for all structures.

While we are able to use a kmem_cache constructor for the Debug Trace Log, LPPACAs are allocated very early in the boot process (before SLUB is available) so we need to use a simpler scheme here.

Introduce helper is_svm_platform() which uses the S bit of the MSR to tell whether we're running as a secure guest.

---
 arch/powerpc/include/asm/svm.h | 26 ++++++++++++++++++++
 arch/powerpc/kernel/paca.c     | 43 +++++++++++++++++++++++++++++++++-
 2 files changed, 68 insertions(+), 1 deletion(-)
Quoted 13 lines

arch/powerpc/include/asm/svm.h line 5

Quoted 11 lines
arch/powerpc/include/asm/svm.h new file +5 -0
new file mode 100644
index 000000000000..fef3740f46a6
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * SVM helper functions
+ *
+ * Copyright 2019 Anshuman Khandual, IBM Corporation.

Are we sure this copyright date is correct?

I may be confused about which year the copyright refers to. I thought it was the year when the patch was committed. If it is the first time the patch was published then this one should be 2018.

--
Thiago Jung Bauermann
IBM Linux Technology Center

#11 lore raw

Re: [PATCH v3 08/16] powerpc/pseries/svm: Use shared memory for LPPACA structures

Quoted 39 lines · Thiago Jung Bauermann
Quoted 24 lines · Michael Ellerman
Quoted 23 lines · Thiago Jung Bauermann

From: Anshuman Khandual <[email protected]>

LPPACA structures need to be shared with the host. Hence they need to be in shared memory. Instead of allocating individual chunks of memory for a given structure from memblock, a contiguous chunk of memory is allocated and then converted into shared memory. Subsequent allocation requests will come from the contiguous chunk which will be always shared memory for all structures.

While we are able to use a kmem_cache constructor for the Debug Trace Log, LPPACAs are allocated very early in the boot process (before SLUB is available) so we need to use a simpler scheme here.

Introduce helper is_svm_platform() which uses the S bit of the MSR to tell whether we're running as a secure guest.

---
 arch/powerpc/include/asm/svm.h | 26 ++++++++++++++++++++
 arch/powerpc/kernel/paca.c     | 43 +++++++++++++++++++++++++++++++++-
 2 files changed, 68 insertions(+), 1 deletion(-)
Quoted 13 lines

arch/powerpc/include/asm/svm.h line 5

Quoted 11 lines
arch/powerpc/include/asm/svm.h new file +5 -0
new file mode 100644
index 000000000000..fef3740f46a6
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * SVM helper functions
+ *
+ * Copyright 2019 Anshuman Khandual, IBM Corporation.

Are we sure this copyright date is correct?

Quoted 3 lines

I may be confused about which year the copyright refers to. I thought it was the year when the patch was committed. If it is the first time the patch was published then this one should be 2018.

I'm not a lawyer etc. but AIUI the date above is about the authorship, ie. when it was originally written, not when it was published or committed.

In general I don't think it matters too much, but in this case I'm pretty sure Anshuman can't have possibly written it in 2019 on behalf of IBM :)

So we can either change the date to 2018, or drop his name and just say it's copyright 2019 by IBM.

cheers

#12 lore raw

Re: [PATCH v3 08/16] powerpc/pseries/svm: Use shared memory for LPPACA structures

Quoted 48 lines · Michael Ellerman
Quoted 39 lines · Thiago Jung Bauermann
Quoted 24 lines · Michael Ellerman
Quoted 23 lines · Thiago Jung Bauermann

From: Anshuman Khandual <[email protected]>

LPPACA structures need to be shared with the host. Hence they need to be in shared memory. Instead of allocating individual chunks of memory for a given structure from memblock, a contiguous chunk of memory is allocated and then converted into shared memory. Subsequent allocation requests will come from the contiguous chunk which will be always shared memory for all structures.

While we are able to use a kmem_cache constructor for the Debug Trace Log, LPPACAs are allocated very early in the boot process (before SLUB is available) so we need to use a simpler scheme here.

Introduce helper is_svm_platform() which uses the S bit of the MSR to tell whether we're running as a secure guest.

---
 arch/powerpc/include/asm/svm.h | 26 ++++++++++++++++++++
 arch/powerpc/kernel/paca.c     | 43 +++++++++++++++++++++++++++++++++-
 2 files changed, 68 insertions(+), 1 deletion(-)
Quoted 13 lines

arch/powerpc/include/asm/svm.h line 5

Quoted 11 lines
arch/powerpc/include/asm/svm.h new file +5 -0
new file mode 100644
index 000000000000..fef3740f46a6
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * SVM helper functions
+ *
+ * Copyright 2019 Anshuman Khandual, IBM Corporation.

Are we sure this copyright date is correct?

Quoted 3 lines

I may be confused about which year the copyright refers to. I thought it was the year when the patch was committed. If it is the first time the patch was published then this one should be 2018.

I'm not a lawyer etc. but AIUI the date above is about the authorship, ie. when it was originally written, not when it was published or committed.

Quoted 6 lines

In general I don't think it matters too much, but in this case I'm pretty sure Anshuman can't have possibly written it in 2019 on behalf of IBM :)

So we can either change the date to 2018, or drop his name and just say it's copyright 2019 by IBM.

I think it's better to change the date to 2018. The same should be done for svm.c, svm.h and mem_encrypt.h. I'll send a new patch series with the correction.

Signature · 2 lines
Thiago Jung Bauermann
IBM Linux Technology Center
#13 lore raw

PATCH v3 9/16 powerpc/pseries/svm: Use shared memory for Debug Trace Log (DTL)

In reply to Thiago Jung Bauermann

From: Anshuman Khandual <[email protected]>

Secure guests need to share the DTL buffers with the hypervisor. To that end, use a kmem_cache constructor which converts the underlying buddy allocated SLUB cache pages into shared memory.

---
 arch/powerpc/include/asm/svm.h          |  5 ++++
 arch/powerpc/platforms/pseries/Makefile |  1 +
 arch/powerpc/platforms/pseries/setup.c  |  5 +++-
 arch/powerpc/platforms/pseries/svm.c    | 40 +++++++++++++++++++++++++
 4 files changed, 50 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/svm.h +5 -0
index fef3740f46a6..f253116c31fc 100644
@@ -15,6 +15,9 @@ static inline bool is_secure_guest(void)
 	return mfmsr() & MSR_S;
 }
 
+void dtl_cache_ctor(void *addr);
+#define get_dtl_cache_ctor()	(is_secure_guest() ? dtl_cache_ctor : NULL)
+
 #else /* CONFIG_PPC_SVM */
 
 static inline bool is_secure_guest(void)
@@ -22,5 +25,7 @@ static inline bool is_secure_guest(void)
 	return false;
 }
 
+#define get_dtl_cache_ctor() NULL
+
 #endif /* CONFIG_PPC_SVM */
 #endif /* _ASM_POWERPC_SVM_H */
arch/powerpc/platforms/pseries/Makefile +1 -0
index ab3d59aeacca..a420ef4c9d8e 100644
@@ -26,6 +26,7 @@ obj-$(CONFIG_IBMVIO)		+= vio.o
 obj-$(CONFIG_IBMEBUS)		+= ibmebus.o
 obj-$(CONFIG_PAPR_SCM)		+= papr_scm.o
 obj-$(CONFIG_PPC_SPLPAR)	+= vphn.o
+obj-$(CONFIG_PPC_SVM)		+= svm.o
 
 ifdef CONFIG_PPC_PSERIES
 obj-$(CONFIG_SUSPEND)		+= suspend.o
arch/powerpc/platforms/pseries/setup.c +4 -1
index f5940cc71c37..d8930c3a8a11 100644
@@ -69,6 +69,7 @@
 #include <asm/security_features.h>
 #include <asm/asm-const.h>
 #include <asm/swiotlb.h>
+#include <asm/svm.h>
 
 #include "pseries.h"
 #include "../../../../drivers/pci/pci.h"
@@ -297,8 +298,10 @@ static inline int alloc_dispatch_logs(void)
 
 static int alloc_dispatch_log_kmem_cache(void)
 {
+	void (*ctor)(void *) = get_dtl_cache_ctor();
+
 	dtl_cache = kmem_cache_create("dtl", DISPATCH_LOG_BYTES,
-						DISPATCH_LOG_BYTES, 0, NULL);
+						DISPATCH_LOG_BYTES, 0, ctor);
 	if (!dtl_cache) {
 		pr_warn("Failed to create dispatch trace log buffer cache\n");
 		pr_warn("Stolen time statistics will be unreliable\n");
arch/powerpc/platforms/pseries/svm.c new file +40 -0
new file mode 100644
index 000000000000..c508196f7c83
@@ -0,0 +1,40 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Secure VM platform
+ *
+ * Copyright 2019 IBM Corporation
+ * Author: Anshuman Khandual <[email protected]>
+ */
+
+#include <linux/mm.h>
+#include <asm/ultravisor.h>
+
+/* There's one dispatch log per CPU. */
+#define NR_DTL_PAGE (DISPATCH_LOG_BYTES * CONFIG_NR_CPUS / PAGE_SIZE)
+
+static struct page *dtl_page_store[NR_DTL_PAGE];
+static long dtl_nr_pages;
+
+static bool is_dtl_page_shared(struct page *page)
+{
+	long i;
+
+	for (i = 0; i < dtl_nr_pages; i++)
+		if (dtl_page_store[i] == page)
+			return true;
+
+	return false;
+}
+
+void dtl_cache_ctor(void *addr)
+{
+	unsigned long pfn = PHYS_PFN(__pa(addr));
+	struct page *page = pfn_to_page(pfn);
+
+	if (!is_dtl_page_shared(page)) {
+		dtl_page_store[dtl_nr_pages] = page;
+		dtl_nr_pages++;
+		WARN_ON(dtl_nr_pages >= NR_DTL_PAGE);
+		uv_share_page(pfn, 1);
+	}
+}
#14 lore raw

PATCH v3 10/16 powerpc/pseries/svm: Unshare all pages before kexecing a new kernel

In reply to Thiago Jung Bauermann

From: Ram Pai <[email protected]>

A new kernel deserves a clean slate. Any pages shared with the hypervisor is unshared before invoking the new kernel. However there are exceptions. If the new kernel is invoked to dump the current kernel, or if there is a explicit request to preserve the state of the current kernel, unsharing of pages is skipped.

NOTE: While testing crashkernel, make sure at least 256M is reserved for crashkernel. Otherwise SWIOTLB allocation will fail and crash kernel will fail to boot.

---
 arch/powerpc/include/asm/ultravisor-api.h | 1 +
 arch/powerpc/include/asm/ultravisor.h     | 5 +++++
 arch/powerpc/kernel/machine_kexec_64.c    | 9 +++++++++
 3 files changed, 15 insertions(+)
arch/powerpc/include/asm/ultravisor-api.h +1 -0
index 142b0576b89f..7e69c364bde0 100644
@@ -24,5 +24,6 @@
 #define UV_ESM				0xF110
 #define UV_SHARE_PAGE			0xF130
 #define UV_UNSHARE_PAGE			0xF134
+#define UV_UNSHARE_ALL_PAGES		0xF140
 
 #endif /* _ASM_POWERPC_ULTRAVISOR_API_H */
arch/powerpc/include/asm/ultravisor.h +5 -0
index a930aec8c1e3..e6f8a2b96694 100644
@@ -21,4 +21,9 @@ static inline int uv_unshare_page(u64 pfn, u64 npages)
 	return ucall_norets(UV_UNSHARE_PAGE, pfn, npages);
 }
 
+static inline int uv_unshare_all_pages(void)
+{
+	return ucall_norets(UV_UNSHARE_ALL_PAGES);
+}
+
 #endif	/* _ASM_POWERPC_ULTRAVISOR_H */
arch/powerpc/kernel/machine_kexec_64.c +9 -0
index 18481b0e2788..04a7cba58eff 100644
@@ -29,6 +29,8 @@
 #include <asm/smp.h>
 #include <asm/hw_breakpoint.h>
 #include <asm/asm-prototypes.h>
+#include <asm/svm.h>
+#include <asm/ultravisor.h>
 
 int default_machine_kexec_prepare(struct kimage *image)
 {
@@ -327,6 +329,13 @@ void default_machine_kexec(struct kimage *image)
 #ifdef CONFIG_PPC_PSERIES
 	kexec_paca.lppaca_ptr = NULL;
 #endif
+
+	if (is_secure_guest() && !(image->preserve_context ||
+				   image->type == KEXEC_TYPE_CRASH)) {
+		uv_unshare_all_pages();
+		printk("kexec: Unshared all shared pages.\n");
+	}
+
 	paca_ptrs[kexec_paca.paca_index] = &kexec_paca;
 
 	setup_paca(&kexec_paca);
#15 lore raw

PATCH v3 11/16 powerpc/pseries/svm: Export guest SVM status to user space via sysfs

In reply to Thiago Jung Bauermann

From: Ryan Grimm <[email protected]>

User space might want to know it's running in a secure VM. It can't do a mfmsr because mfmsr is a privileged instruction.

The solution here is to create a cpu attribute:

/sys/devices/system/cpu/svm

which will read 0 or 1 based on the S bit of the guest's CPU 0.

---
 arch/powerpc/kernel/sysfs.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)
arch/powerpc/kernel/sysfs.c +29 -0
index e2147d7c9e72..f7100ab77d29 100644
@@ -19,6 +19,7 @@
 #include <asm/smp.h>
 #include <asm/pmc.h>
 #include <asm/firmware.h>
+#include <asm/svm.h>
 
 #include "cacheinfo.h"
 #include "setup.h"
@@ -715,6 +716,32 @@ static struct device_attribute pa6t_attrs[] = {
 #endif /* HAS_PPC_PMC_PA6T */
 #endif /* HAS_PPC_PMC_CLASSIC */
 
+#ifdef CONFIG_PPC_SVM
+static void get_svm(void *val)
+{
+	u32 *value = val;
+
+	*value = is_secure_guest();
+}
+
+static ssize_t show_svm(struct device *dev, struct device_attribute *attr, char *buf)
+{
+	u32 val;
+	smp_call_function_single(0, get_svm, &val, 1);
+	return sprintf(buf, "%u\n", val);
+}
+static DEVICE_ATTR(svm, 0444, show_svm, NULL);
+
+static void create_svm_file(void)
+{
+	device_create_file(cpu_subsys.dev_root, &dev_attr_svm);
+}
+#else
+static void create_svm_file(void)
+{
+}
+#endif /* CONFIG_PPC_SVM */
+
 static int register_cpu_online(unsigned int cpu)
 {
 	struct cpu *c = &per_cpu(cpu_devices, cpu);
@@ -1058,6 +1085,8 @@ static int __init topology_init(void)
 	sysfs_create_dscr_default();
 #endif /* CONFIG_PPC64 */
 
+	create_svm_file();
+
 	return 0;
 }
 subsys_initcall(topology_init);
#16 lore raw

Re: [PATCH v3 11/16] powerpc/pseries/svm: Export guest SVM status to user space via sysfs

Quoted 10 lines · Thiago Jung Bauermann

From: Ryan Grimm <[email protected]>

User space might want to know it's running in a secure VM. It can't do a mfmsr because mfmsr is a privileged instruction.

The solution here is to create a cpu attribute:

/sys/devices/system/cpu/svm

which will read 0 or 1 based on the S bit of the guest's CPU 0.

Why CPU 0?

If we have different CPUs running with different MSR_S then something has gone badly wrong, no?

So can't we just read the MSR on whatever CPU the sysfs code happens to run on.

cheers

Quoted 54 lines
arch/powerpc/kernel/sysfs.c +29 -0
index e2147d7c9e72..f7100ab77d29 100644
@@ -19,6 +19,7 @@
 #include <asm/smp.h>
 #include <asm/pmc.h>
 #include <asm/firmware.h>
+#include <asm/svm.h>
 
 #include "cacheinfo.h"
 #include "setup.h"
@@ -715,6 +716,32 @@ static struct device_attribute pa6t_attrs[] = {
 #endif /* HAS_PPC_PMC_PA6T */
 #endif /* HAS_PPC_PMC_CLASSIC */
 
+#ifdef CONFIG_PPC_SVM
+static void get_svm(void *val)
+{
+	u32 *value = val;
+
+	*value = is_secure_guest();
+}
+
+static ssize_t show_svm(struct device *dev, struct device_attribute *attr, char *buf)
+{
+	u32 val;
+	smp_call_function_single(0, get_svm, &val, 1);
+	return sprintf(buf, "%u\n", val);
+}
+static DEVICE_ATTR(svm, 0444, show_svm, NULL);
+
+static void create_svm_file(void)
+{
+	device_create_file(cpu_subsys.dev_root, &dev_attr_svm);
+}
+#else
+static void create_svm_file(void)
+{
+}
+#endif /* CONFIG_PPC_SVM */
+
 static int register_cpu_online(unsigned int cpu)
 {
 	struct cpu *c = &per_cpu(cpu_devices, cpu);
@@ -1058,6 +1085,8 @@ static int __init topology_init(void)
 	sysfs_create_dscr_default();
 #endif /* CONFIG_PPC64 */
 
+	create_svm_file();
+
 	return 0;
 }
 subsys_initcall(topology_init);
#17 lore raw

Re: [PATCH v3 11/16] powerpc/pseries/svm: Export guest SVM status to user space via sysfs

Quoted 8 lines · Michael Ellerman
Quoted 7 lines · Thiago Jung Bauermann

From: Ryan Grimm <[email protected]>

User space might want to know it's running in a secure VM. It can't do a mfmsr because mfmsr is a privileged instruction.

The solution here is to create a cpu attribute:

Quoted 8 lines
Quoted 3 lines

/sys/devices/system/cpu/svm

which will read 0 or 1 based on the S bit of the guest's CPU 0.

Why CPU 0?

If we have different CPUs running with different MSR_S then something has gone badly wrong, no?

Yes, that would be very bad.

Quoted 2 lines

So can't we just read the MSR on whatever CPU the sysfs code happens to run on.

Good point. I made the change in the patch below.

Signature · 25 lines
Thiago Jung Bauermann
IBM Linux Technology Center



From 2d951305e118bf286f8e83cbf396448085186357 Mon Sep 17 00:00:00 2001
From: Ryan Grimm <[email protected]>
Date: Tue, 15 Jan 2019 11:56:29 -0600
Subject: [PATCH] powerpc/pseries/svm: Export guest SVM status to user space
 via sysfs

User space might want to know it's running in a secure VM.  It can't do
a mfmsr because mfmsr is a privileged instruction.

The solution here is to create a cpu attribute:

/sys/devices/system/cpu/svm

which will read 0 or 1 based on the S bit of the current CPU.

Signed-off-by: Ryan Grimm <[email protected]>
Signed-off-by: Thiago Jung Bauermann <[email protected]>
---
 arch/powerpc/kernel/sysfs.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
arch/powerpc/kernel/sysfs.c +20 -0
index e2147d7c9e72..80a676da11cb 100644
@@ -19,6 +19,7 @@
 #include <asm/smp.h>
 #include <asm/pmc.h>
 #include <asm/firmware.h>
+#include <asm/svm.h>
 
 #include "cacheinfo.h"
 #include "setup.h"
@@ -715,6 +716,23 @@ static struct device_attribute pa6t_attrs[] = {
 #endif /* HAS_PPC_PMC_PA6T */
 #endif /* HAS_PPC_PMC_CLASSIC */
 
+#ifdef CONFIG_PPC_SVM
+static ssize_t show_svm(struct device *dev, struct device_attribute *attr, char *buf)
+{
+	return sprintf(buf, "%u\n", is_secure_guest());
+}
+static DEVICE_ATTR(svm, 0444, show_svm, NULL);
+
+static void create_svm_file(void)
+{
+	device_create_file(cpu_subsys.dev_root, &dev_attr_svm);
+}
+#else
+static void create_svm_file(void)
+{
+}
+#endif /* CONFIG_PPC_SVM */
+
 static int register_cpu_online(unsigned int cpu)
 {
 	struct cpu *c = &per_cpu(cpu_devices, cpu);
@@ -1058,6 +1076,8 @@ static int __init topology_init(void)
 	sysfs_create_dscr_default();
 #endif /* CONFIG_PPC64 */
 
+	create_svm_file();
+
 	return 0;
 }
 subsys_initcall(topology_init);
#18 lore raw

Re: [PATCH v3 11/16] powerpc/pseries/svm: Export guest SVM status to user space via sysfs

Quoted 17 lines · Thiago Jung Bauermann
Quoted 7 lines · Michael Ellerman
Quoted 6 lines · Thiago Jung Bauermann

From: Ryan Grimm <[email protected]>
User space might want to know it's running in a secure VM. It can't do a mfmsr because mfmsr is a privileged instruction.

The solution here is to create a cpu attribute:

Quoted 8 lines
Quoted 3 lines

/sys/devices/system/cpu/svm

which will read 0 or 1 based on the S bit of the guest's CPU 0.

Why CPU 0?

If we have different CPUs running with different MSR_S then something has gone badly wrong, no?

Quoted 6 lines

Yes, that would be very bad.

Quoted 2 lines

So can't we just read the MSR on whatever CPU the sysfs code happens to run on.

Good point. I made the change in the patch below.

The patch looks good. Although, it raises the question of whether it should be an attribute of the CPU at all.

I guess there's not obviously anywhere better for it.

Still you should document the attribute in Documentation/ABI/testing/sysfs-devices-system-cpu

cheers

Quoted 66 lines

From 2d951305e118bf286f8e83cbf396448085186357 Mon Sep 17 00:00:00 2001
From: Ryan Grimm <[email protected]>
Date: Tue, 15 Jan 2019 11:56:29 -0600
Subject: [PATCH] powerpc/pseries/svm: Export guest SVM status to user space
via sysfs

User space might want to know it's running in a secure VM. It can't do a mfmsr because mfmsr is a privileged instruction.

The solution here is to create a cpu attribute:

/sys/devices/system/cpu/svm

which will read 0 or 1 based on the S bit of the current CPU.

---
 arch/powerpc/kernel/sysfs.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
arch/powerpc/kernel/sysfs.c +20 -0
index e2147d7c9e72..80a676da11cb 100644
@@ -19,6 +19,7 @@
 #include <asm/smp.h>
 #include <asm/pmc.h>
 #include <asm/firmware.h>
+#include <asm/svm.h>
 
 #include "cacheinfo.h"
 #include "setup.h"
@@ -715,6 +716,23 @@ static struct device_attribute pa6t_attrs[] = {
 #endif /* HAS_PPC_PMC_PA6T */
 #endif /* HAS_PPC_PMC_CLASSIC */
 
+#ifdef CONFIG_PPC_SVM
+static ssize_t show_svm(struct device *dev, struct device_attribute *attr, char *buf)
+{
+	return sprintf(buf, "%u\n", is_secure_guest());
+}
+static DEVICE_ATTR(svm, 0444, show_svm, NULL);
+
+static void create_svm_file(void)
+{
+	device_create_file(cpu_subsys.dev_root, &dev_attr_svm);
+}
+#else
+static void create_svm_file(void)
+{
+}
+#endif /* CONFIG_PPC_SVM */
+
 static int register_cpu_online(unsigned int cpu)
 {
 	struct cpu *c = &per_cpu(cpu_devices, cpu);
@@ -1058,6 +1076,8 @@ static int __init topology_init(void)
 	sysfs_create_dscr_default();
 #endif /* CONFIG_PPC64 */
 
+	create_svm_file();
+
 	return 0;
 }
 subsys_initcall(topology_init);
#19 lore raw

Re: [PATCH v3 11/16] powerpc/pseries/svm: Export guest SVM status to user space via sysfs

Quoted 23 lines · Michael Ellerman
Quoted 22 lines · Thiago Jung Bauermann
Quoted 7 lines · Michael Ellerman
Quoted 6 lines · Thiago Jung Bauermann

From: Ryan Grimm <[email protected]>
User space might want to know it's running in a secure VM. It can't do a mfmsr because mfmsr is a privileged instruction.

The solution here is to create a cpu attribute:

Quoted 8 lines
Quoted 3 lines

/sys/devices/system/cpu/svm

which will read 0 or 1 based on the S bit of the guest's CPU 0.

Why CPU 0?

If we have different CPUs running with different MSR_S then something has gone badly wrong, no?

Yes, that would be very bad.

Quoted 2 lines

So can't we just read the MSR on whatever CPU the sysfs code happens to run on.

Quoted 6 lines
Quoted 1 line

Good point. I made the change in the patch below.

The patch looks good. Although, it raises the question of whether it should be an attribute of the CPU at all.

I guess there's not obviously anywhere better for it.

Ok. TBH this patch is not as urgent as the others. It was added so that tests have an easy way to tell if they're in an SVM. I can leave it out for now to figure out if there's a better place for this information.

Quoted 1 line

Still you should document the attribute in Documentation/ABI/testing/sysfs-devices-system-cpu

Indedd, will do that.

--
Thiago Jung Bauermann
IBM Linux Technology Center

#20 lore raw

PATCH v3 12/16 powerpc/pseries/svm: Disable doorbells in SVM guests

In reply to Thiago Jung Bauermann

From: Sukadev Bhattiprolu <[email protected]>

Normally, the HV emulates some instructions like MSGSNDP, MSGCLRP from a KVM guest. To emulate the instructions, it must first read the instruction from the guest's memory and decode its parameters.

However for a secure guest (aka SVM), the page containing the instruction is in secure memory and the HV cannot access directly. It would need the Ultravisor (UV) to facilitate accessing the instruction and parameters but the UV currently does not have the support for such accesses.

Until the UV has such support, disable doorbells in SVMs. This might incur a performance hit but that is yet to be quantified.

With this patch applied (needed only in SVMs not needed for HV) we are able to launch SVM guests with multi-core support. Eg:

	qemu -smp sockets=2,cores=2,threads=2.

Fix suggested by Benjamin Herrenschmidt. Thanks to input from Paul Mackerras, Ram Pai and Michael Anderson.

---
 arch/powerpc/platforms/pseries/smp.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/powerpc/platforms/pseries/smp.c +2 -1
index 4b3ef8d9c63f..ad61e90032da 100644
@@ -41,6 +41,7 @@
 #include <asm/dbell.h>
 #include <asm/plpar_wrappers.h>
 #include <asm/code-patching.h>
+#include <asm/svm.h>
 
 #include "pseries.h"
 #include "offline_states.h"
@@ -221,7 +222,7 @@ static __init void pSeries_smp_probe_xics(void)
 {
 	xics_smp_probe();
 
-	if (cpu_has_feature(CPU_FTR_DBELL))
+	if (cpu_has_feature(CPU_FTR_DBELL) && !is_secure_guest())
 		smp_ops->cause_ipi = smp_pseries_cause_ipi;
 	else
 		smp_ops->cause_ipi = icp_ops->cause_ipi;
#21 lore raw

PATCH v3 13/16 powerpc/pseries/iommu: Don't use dma_iommu_ops on secure guests

In reply to Thiago Jung Bauermann

Secure guest memory is inacessible to devices so regular DMA isn't possible.

In that case set devices' dma_map_ops to NULL so that the generic DMA code path will use SWIOTLB and DMA to bounce buffers.

---
 arch/powerpc/platforms/pseries/iommu.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/powerpc/platforms/pseries/iommu.c +5 -1
index 889dc2e44b89..6fd8c8581873 100644
@@ -36,6 +36,7 @@
 #include <asm/udbg.h>
 #include <asm/mmzone.h>
 #include <asm/plpar_wrappers.h>
+#include <asm/svm.h>
 
 #include "pseries.h"
 
@@ -1318,7 +1319,10 @@ void iommu_init_early_pSeries(void)
 	of_reconfig_notifier_register(&iommu_reconfig_nb);
 	register_memory_notifier(&iommu_mem_nb);
 
-	set_pci_dma_ops(&dma_iommu_ops);
+	if (is_secure_guest())
+		set_pci_dma_ops(NULL);
+	else
+		set_pci_dma_ops(&dma_iommu_ops);
 }
 
 static int __init disable_multitce(char *str)
#22 lore raw

Re: [PATCH v3 13/16] powerpc/pseries/iommu: Don't use dma_iommu_ops on secure guests

arch/powerpc/platforms/pseries/iommu.c void iommu_init_early_pSeries(void) · line 1325

Quoted 9 lines · Thiago Jung Bauermann
arch/powerpc/platforms/pseries/iommu.c +4 -1
@@ -1318,7 +1319,10 @@ void iommu_init_early_pSeries(void)
 	of_reconfig_notifier_register(&iommu_reconfig_nb);
 	register_memory_notifier(&iommu_mem_nb);
 
-	set_pci_dma_ops(&dma_iommu_ops);
+	if (is_secure_guest())
+		set_pci_dma_ops(NULL);
+	else
+		set_pci_dma_ops(&dma_iommu_ops);

Shoudn't:

	if (!is_secure_guest())
		set_pci_dma_ops(&dma_iommu_ops);

be enough here, given that NULL is the default?

Also either way I think this conditional needs a comment explaining why it is there.

#23 lore raw

Re: [PATCH v3 13/16] powerpc/pseries/iommu: Don't use dma_iommu_ops on secure guests

Hello Christoph,

Thanks for your review.

Quoted 13 lines · Christoph Hellwig

Review void iommu_init_early_pSeries(void) · line 1325

Quoted 9 lines · Thiago Jung Bauermann
(quoted diff) +4 -1
@@ -1318,7 +1319,10 @@ void iommu_init_early_pSeries(void)
 	of_reconfig_notifier_register(&iommu_reconfig_nb);
 	register_memory_notifier(&iommu_mem_nb);
 
-	set_pci_dma_ops(&dma_iommu_ops);
+	if (is_secure_guest())
+		set_pci_dma_ops(NULL);
+	else
+		set_pci_dma_ops(&dma_iommu_ops);

Shoudn't:

Quoted 4 lines
	if (!is_secure_guest())
		set_pci_dma_ops(&dma_iommu_ops);

be enough here, given that NULL is the default?

Indeed, it is enough.

Quoted 2 lines

Also either way I think this conditional needs a comment explaining why it is there.

Good point. I added the commit message as a comment in the code. New version of this patch below.

From 5dc3914efa4765eef2869d554d4ea1c676bb1e75 Mon Sep 17 00:00:00 2001
From: Thiago Jung Bauermann <[email protected]>
Date: Thu, 24 Jan 2019 22:40:16 -0200
Subject: [PATCH] powerpc/pseries/iommu: Don't use dma_iommu_ops on secure
guests

Secure guest memory is inacessible to devices so regular DMA isn't possible.

In that case set devices' dma_map_ops to NULL so that the generic DMA code path will use SWIOTLB to bounce buffers for DMA.

---
 arch/powerpc/platforms/pseries/iommu.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
arch/powerpc/platforms/pseries/iommu.c +10 -1
index 889dc2e44b89..8d9c2b17ad54 100644
@@ -36,6 +36,7 @@
 #include <asm/udbg.h>
 #include <asm/mmzone.h>
 #include <asm/plpar_wrappers.h>
+#include <asm/svm.h>
 
 #include "pseries.h"
 
@@ -1318,7 +1319,15 @@ void iommu_init_early_pSeries(void)
 	of_reconfig_notifier_register(&iommu_reconfig_nb);
 	register_memory_notifier(&iommu_mem_nb);
 
-	set_pci_dma_ops(&dma_iommu_ops);
+	/*
+	 * Secure guest memory is inacessible to devices so regular DMA isn't
+	 * possible.
+	 *
+	 * In that case keep devices' dma_map_ops as NULL so that the generic
+	 * DMA code path will use SWIOTLB to bounce buffers for DMA.
+	 */
+	if (!is_secure_guest())
+		set_pci_dma_ops(&dma_iommu_ops);
 }
 
 static int __init disable_multitce(char *str)
#24 lore raw

PATCH v3 14/16 powerpc/pseries/svm: Force SWIOTLB for secure guests

In reply to Thiago Jung Bauermann

From: Anshuman Khandual <[email protected]>

SWIOTLB checks range of incoming CPU addresses to be bounced and sees if the device can access it through its DMA window without requiring bouncing. In such cases it just chooses to skip bouncing. But for cases like secure guests on powerpc platform all addresses need to be bounced into the shared pool of memory because the host cannot access it otherwise. Hence the need to do the bouncing is not related to device's DMA window and use of bounce buffers is forced by setting swiotlb_force.

Also, connect the shared memory conversion functions into the ARCH_HAS_MEM_ENCRYPT hooks and call swiotlb_update_mem_attributes() to convert SWIOTLB's memory pool to shared memory.

[ bauerman: Use ARCH_HAS_MEM_ENCRYPT hooks to share swiotlb memory pool. ]
---
 arch/powerpc/include/asm/mem_encrypt.h | 26 +++++++++++++++
 arch/powerpc/platforms/pseries/Kconfig |  3 ++
 arch/powerpc/platforms/pseries/svm.c   | 45 ++++++++++++++++++++++++++
 3 files changed, 74 insertions(+)
arch/powerpc/include/asm/mem_encrypt.h new file +26 -0
new file mode 100644
index 000000000000..d8d5a7fcf298
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * SVM helper functions
+ *
+ * Copyright 2019 IBM Corporation
+ */
+
+#ifndef _ASM_POWERPC_MEM_ENCRYPT_H
+#define _ASM_POWERPC_MEM_ENCRYPT_H
+
+#include <asm/svm.h>
+
+static inline bool mem_encrypt_active(void)
+{
+	return is_secure_guest();
+}
+
+static inline bool force_dma_unencrypted(struct device *dev)
+{
+	return is_secure_guest();
+}
+
+int set_memory_encrypted(unsigned long addr, int numpages);
+int set_memory_decrypted(unsigned long addr, int numpages);
+
+#endif /* _ASM_POWERPC_MEM_ENCRYPT_H */
arch/powerpc/platforms/pseries/Kconfig +3 -0
index d09deb05bb66..9e35cddddf73 100644
@@ -149,6 +149,9 @@ config PAPR_SCM
 config PPC_SVM
 	bool "Secure virtual machine (SVM) support for POWER"
 	depends on PPC_PSERIES
+	select SWIOTLB
+	select ARCH_HAS_MEM_ENCRYPT
+	select ARCH_HAS_FORCE_DMA_UNENCRYPTED
 	help
 	 There are certain POWER platforms which support secure guests using
 	 the Protected Execution Facility, with the help of an Ultravisor
arch/powerpc/platforms/pseries/svm.c +45 -0

53 lines not on this page · raw message