Skip to contentSkip to conversation

Secure Virtual Machine Enablement

PATCH Superseded Thiago Jung Bauermann 5 participants 23 messages May 21, 2019 – Jul 1, 2019

Subsystems (2)

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

cover Secure Virtual Machine Enablement

This series enables Secure Virtual Machines (SVMs) on powerpc. SVMs use the Protected Execution Facility (PEF) and request to be migrated to secure memory during prom_init() so by default all of their memory is inaccessible to the hypervisor. There is an Ultravisor call that the VM can use to request certain pages to be made accessible to (or shared with) the hypervisor.

The objective of these patches is to have the guest perform this request for buffers that need to be accessed by the hypervisor such as the LPPACAs, the SWIOTLB memory and the Debug Trace Log.

The patch set applies on top of Claudio Carvalho's "kvmppc: Paravirtualize KVM to support ultravisor" series:

https://lore.kernel.org/linuxppc-dev/[email protected]/

I only need the following two patches from his series:

[RFC PATCH v2 02/10] KVM: PPC: Ultravisor: Introduce the MSR_S bit [RFC PATCH v2 04/10] KVM: PPC: Ultravisor: Add generic ultravisor call handler

Patches 2 and 3 are posted as RFC because we are still finalizing the details on how the ESM blob will be passed to the kernel. All other patches are (hopefully) in upstreamable shape.

Unfortunately this series still doesn't enable the use of virtio devices in the secure guest. This support depends on a discussion that is currently ongoing with the virtio community:

https://lore.kernel.org/linuxppc-dev/[email protected]/

This was the last time I posted this patch set:

https://lore.kernel.org/linuxppc-dev/[email protected]/

At that time, it wasn't possible to launch a real secure guest because the Ultravisor was still in very early development. Now there is a relatively mature Ultravisor and I was able to test it using Claudio's patches in the host kernel, booting normally using an initramfs for the root filesystem.

This is the command used to start up the guest with QEMU 4.0:

qemu-system-ppc64				\
	-nodefaults				\
	-cpu host				\
	-machine pseries,accel=kvm,kvm-type=HV,cap-htm=off,cap-cfpc=broken,cap-sbbc=broken,cap-ibs=broken \
	-display none				\
	-serial mon:stdio			\
	-smp 1					\
	-m 4G					\
	-kernel /home/bauermann/vmlinux		\
	-initrd /home/bauermann/fs_small.cpio	\
	-append 'debug'

Changelog since the RFC from August:

- Patch "powerpc/pseries: Introduce option to build secure virtual machines"
- New patch.

- Patch "powerpc: Add support for adding an ESM blob to the zImage wrapper"
- Patch from Benjamin Herrenschmidt, first posted here:

    https://lore.kernel.org/linuxppc-dev/[email protected]/

- Made minor adjustments to some comments. Code is unchanged.

- Patch "powerpc/prom_init: Add the ESM call to prom_init"
- New patch from Ram Pai and Michael Anderson.

- Patch "powerpc/pseries/svm: Add helpers for UV_SHARE_PAGE and UV_UNSHARE_PAGE"
- New patch from Ram Pai.

- Patch "powerpc/pseries: Add and use LPPACA_SIZE constant"
- Moved LPPACA_SIZE macro inside the CONFIG_PPC_PSERIES #ifdef.
- Put sizeof() operand left of comparison operator in BUILD_BUG_ON() macro to appease a checkpatch warning.

- Patch "powerpc/pseries/svm: Use shared memory for LPPACA structures"
- Moved definition of is_secure_guest() helper to this patch.
- Changed shared_lppaca and shared_lppaca_size from globals to static variables inside alloc_shared_lppaca().
- Changed shared_lppaca to hold virtual address instead of physical address.

- Patch "powerpc/pseries/svm: Use shared memory for Debug Trace Log (DTL)"
- Add get_dtl_cache_ctor() macro. Suggested by Ram Pai.

- Patch "powerpc/pseries/svm: Export guest SVM status to user space via sysfs"
- New patch from Ryan Grimm.

- Patch "powerpc/pseries/svm: Disable doorbells in SVM guests"
- New patch from Sukadev Bhattiprolu.

- Patch "powerpc/pseries/iommu: Don't use dma_iommu_ops on secure guests"
- New patch.

- Patch "powerpc/pseries/svm: Force SWIOTLB for secure guests"
- New patch with code that was previously in other patches.

- Patch "powerpc/configs: Enable secure guest support in pseries and ppc64 defconfigs"
- New patch from Ryan Grimm.

- Patch "powerpc/pseries/svm: Detect Secure Virtual Machine (SVM) platform"
- Dropped this patch by moving its code to other patches.

- Patch "powerpc/svm: Select CONFIG_DMA_DIRECT_OPS and CONFIG_SWIOTLB"
- No need to select CONFIG_DMA_DIRECT_OPS anymore. The CONFIG_SWIOTLB change was moved to another patch and this patch was dropped.

- Patch "powerpc/pseries/svm: Add memory conversion (shared/secure) helper functions"
- Dropped patch since the helper functions were unnecessary wrappers around uv_share_page() and uv_unshare_page().

- Patch "powerpc/svm: Convert SWIOTLB buffers to shared memory"
- Squashed into patch "powerpc/pseries/svm: Force SWIOTLB for secure guests"

- Patch "powerpc/svm: Don't release SWIOTLB buffers on secure guests"
- Squashed into patch "powerpc/pseries/svm: Force SWIOTLB for secure guests"

- Patch "powerpc/svm: Use SWIOTLB DMA API for all virtio devices"
- Dropped patch. Enablement of virtio will use a difference approach.

- Patch "powerpc/svm: Force the use of bounce buffers"
- Squashed into patch "powerpc/pseries/svm: Force SWIOTLB for secure guests"
- Added comment explaining why it's necessary.to force use of SWIOTLB. Suggested by Christoph Hellwig.

- Patch "powerpc/svm: Increase SWIOTLB buffer size"
- Dropped patch.

Anshuman Khandual (3):
powerpc/pseries/svm: Use shared memory for LPPACA structures powerpc/pseries/svm: Use shared memory for Debug Trace Log (DTL) powerpc/pseries/svm: Force SWIOTLB for secure guests

Benjamin Herrenschmidt (1):
powerpc: Add support for adding an ESM blob to the zImage wrapper

Ram Pai (2):
powerpc/prom_init: Add the ESM call to prom_init powerpc/pseries/svm: Add helpers for UV_SHARE_PAGE and UV_UNSHARE_PAGE

Ryan Grimm (2):
powerpc/pseries/svm: Export guest SVM status to user space via sysfs powerpc/configs: Enable secure guest support in pseries and ppc64
defconfigs

Sukadev Bhattiprolu (1):
powerpc/pseries/svm: Disable doorbells in SVM guests

Thiago Jung Bauermann (3):
powerpc/pseries: Introduce option to build secure virtual machines powerpc/pseries: Add and use LPPACA_SIZE constant powerpc/pseries/iommu: Don't use dma_iommu_ops on secure guests

 .../admin-guide/kernel-parameters.txt         |   5 +
 arch/powerpc/boot/main.c                      |  41 ++++++
 arch/powerpc/boot/ops.h                       |   2 +
 arch/powerpc/boot/wrapper                     |  24 +++-
 arch/powerpc/boot/zImage.lds.S                |   8 ++
 arch/powerpc/configs/ppc64_defconfig          |   1 +
 arch/powerpc/configs/pseries_defconfig        |   1 +
 arch/powerpc/include/asm/mem_encrypt.h        |  19 +++
 arch/powerpc/include/asm/svm.h                |  31 +++++
 arch/powerpc/include/asm/ultravisor-api.h     |   3 +
 arch/powerpc/include/asm/ultravisor.h         |  16 ++-
 arch/powerpc/kernel/Makefile                  |   4 +-
 arch/powerpc/kernel/paca.c                    |  52 +++++++-
 arch/powerpc/kernel/prom_init.c               | 124 ++++++++++++++++++
 arch/powerpc/kernel/sysfs.c                   |  29 ++++
 arch/powerpc/platforms/pseries/Kconfig        |  17 +++
 arch/powerpc/platforms/pseries/Makefile       |   1 +
 arch/powerpc/platforms/pseries/iommu.c        |   6 +-
 arch/powerpc/platforms/pseries/setup.c        |   5 +-
 arch/powerpc/platforms/pseries/smp.c          |   3 +-
 arch/powerpc/platforms/pseries/svm.c          |  85 ++++++++++++
 21 files changed, 464 insertions(+), 13 deletions(-)
 create mode 100644 arch/powerpc/include/asm/mem_encrypt.h
 create mode 100644 arch/powerpc/include/asm/svm.h
 create mode 100644 arch/powerpc/platforms/pseries/svm.c
#1 lore raw

PATCH 1/12 powerpc/pseries: Introduce option to build secure virtual machines

Introduce CONFIG_PPC_SVM to control support for secure guests and include Ultravisor-related helpers when it is selected

---
 arch/powerpc/include/asm/ultravisor.h  |  2 +-
 arch/powerpc/kernel/Makefile           |  4 +++-
 arch/powerpc/platforms/pseries/Kconfig | 12 ++++++++++++
 3 files changed, 16 insertions(+), 2 deletions(-)
arch/powerpc/include/asm/ultravisor.h +1 -1
index 4ffec7a36acd..09e0a615d96f 100644
@@ -28,7 +28,7 @@ extern int early_init_dt_scan_ultravisor(unsigned long node, const char *uname,
  * This call supports up to 6 arguments and 4 return arguments. Use
  * UCALL_BUFSIZE to size the return argument buffer.
  */
-#if defined(CONFIG_PPC_UV)
+#if defined(CONFIG_PPC_UV) || defined(CONFIG_PPC_SVM)
 long ucall(unsigned long opcode, unsigned long *retbuf, ...);
 #else
 static long ucall(unsigned long opcode, unsigned long *retbuf, ...)
arch/powerpc/kernel/Makefile +3 -1
index 43ff4546e469..1e9b721634c8 100644
@@ -154,7 +154,9 @@ endif
 
 obj-$(CONFIG_EPAPR_PARAVIRT)	+= epapr_paravirt.o epapr_hcalls.o
 obj-$(CONFIG_KVM_GUEST)		+= kvm.o kvm_emul.o
-obj-$(CONFIG_PPC_UV)		+= ultravisor.o ucall.o
+ifneq ($(CONFIG_PPC_UV)$(CONFIG_PPC_SVM),)
+obj-y				+= ultravisor.o ucall.o
+endif
 
 # Disable GCOV, KCOV & sanitizers in odd or sensitive code
 GCOV_PROFILE_prom_init.o := n
arch/powerpc/platforms/pseries/Kconfig +12 -0
index 9c6b3d860518..82c16aa4f1ce 100644
@@ -144,3 +144,15 @@ config PAPR_SCM
 	tristate "Support for the PAPR Storage Class Memory interface"
 	help
 	  Enable access to hypervisor provided storage class memory.
+
+config PPC_SVM
+	bool "Secure virtual machine (SVM) support for POWER"
+	depends on PPC_PSERIES
+	default n
+	help
+	 Support secure guests on POWER. There are certain POWER platforms which
+	 support secure guests using the Protected Execution Facility, with the
+	 help of an Ultravisor executing below the hypervisor layer. This
+	 enables the support for those guests.
+
+	 If unsure, say "N".
#2 lore raw

RFC PATCH 2/12 powerpc: Add support for adding an ESM blob to the zImage wrapper

In reply to Thiago Jung Bauermann

From: Benjamin Herrenschmidt <[email protected]>

For secure VMs, the signing tool will create a ticket called the "ESM blob" for the Enter Secure Mode ultravisor call with the signatures of the kernel and initrd among other things.

This adds support to the wrapper script for adding that blob via the "-e" option to the zImage.pseries.

It also adds code to the zImage wrapper itself to retrieve and if necessary relocate the blob, and pass its address to Linux via the device-tree, to be later consumed by prom_init.

[ Minor adjustments to some comments. ]
---
 arch/powerpc/boot/main.c       | 41 ++++++++++++++++++++++++++++++++++
 arch/powerpc/boot/ops.h        |  2 ++
 arch/powerpc/boot/wrapper      | 24 +++++++++++++++++---
 arch/powerpc/boot/zImage.lds.S |  8 +++++++
 4 files changed, 72 insertions(+), 3 deletions(-)
arch/powerpc/boot/main.c +41 -0
index 78aaf4ffd7ab..ca612efd3e81 100644
@@ -150,6 +150,46 @@ static struct addr_range prep_initrd(struct addr_range vmlinux, void *chosen,
 	return (struct addr_range){(void *)initrd_addr, initrd_size};
 }
 
+#ifdef __powerpc64__
+static void prep_esm_blob(struct addr_range vmlinux, void *chosen)
+{
+	unsigned long esm_blob_addr, esm_blob_size;
+
+	/* Do we have an ESM (Enter Secure Mode) blob? */
+	if (_esm_blob_end <= _esm_blob_start)
+		return;
+
+	printf("Attached ESM blob at 0x%p-0x%p\n\r",
+	       _esm_blob_start, _esm_blob_end);
+	esm_blob_addr = (unsigned long)_esm_blob_start;
+	esm_blob_size = _esm_blob_end - _esm_blob_start;
+
+	/*
+	 * If the ESM blob is too low it will be clobbered when the
+	 * kernel relocates to its final location.  In this case,
+	 * allocate a safer place and move it.
+	 */
+	if (esm_blob_addr < vmlinux.size) {
+		void *old_addr = (void *)esm_blob_addr;
+
+		printf("Allocating 0x%lx bytes for esm_blob ...\n\r",
+		       esm_blob_size);
+		esm_blob_addr = (unsigned long)malloc(esm_blob_size);
+		if (!esm_blob_addr)
+			fatal("Can't allocate memory for ESM blob !\n\r");
+		printf("Relocating ESM blob 0x%lx <- 0x%p (0x%lx bytes)\n\r",
+		       esm_blob_addr, old_addr, esm_blob_size);
+		memmove((void *)esm_blob_addr, old_addr, esm_blob_size);
+	}
+
+	/* Tell the kernel ESM blob address via device tree. */
+	setprop_val(chosen, "linux,esm-blob-start", (u32)(esm_blob_addr));
+	setprop_val(chosen, "linux,esm-blob-end", (u32)(esm_blob_addr + esm_blob_size));
+}
+#else
+static inline void prep_esm_blob(struct addr_range vmlinux, void *chosen) { }
+#endif
+
 /* A buffer that may be edited by tools operating on a zImage binary so as to
  * edit the command line passed to vmlinux (by setting /chosen/bootargs).
  * The buffer is put in it's own section so that tools may locate it easier.
@@ -218,6 +258,7 @@ void start(void)
 	vmlinux = prep_kernel();
 	initrd = prep_initrd(vmlinux, chosen,
 			     loader_info.initrd_addr, loader_info.initrd_size);
+	prep_esm_blob(vmlinux, chosen);
 	prep_cmdline(chosen);
 
 	printf("Finalizing device tree...");
arch/powerpc/boot/ops.h +2 -0
index cd043726ed88..e0606766480f 100644
@@ -251,6 +251,8 @@ extern char _initrd_start[];
 extern char _initrd_end[];
 extern char _dtb_start[];
 extern char _dtb_end[];
+extern char _esm_blob_start[];
+extern char _esm_blob_end[];
 
 static inline __attribute__((const))
 int __ilog2_u32(u32 n)
arch/powerpc/boot/wrapper +21 -3
index f9141eaec6ff..36b2ad6cd5b7 100755
@@ -14,6 +14,7 @@
 # -i initrd	specify initrd file
 # -d devtree	specify device-tree blob
 # -s tree.dts	specify device-tree source file (needs dtc installed)
+# -e esm_blob   specify ESM blob for secure images
 # -c		cache $kernel.strip.gz (use if present & newer, else make)
 # -C prefix	specify command prefix for cross-building tools
 #		(strip, objcopy, ld)
@@ -38,6 +39,7 @@ platform=of
 initrd=
 dtb=
 dts=
+esm_blob=
 cacheit=
 binary=
 compression=.gz
@@ -60,9 +62,9 @@ tmpdir=.
 
 usage() {
     echo 'Usage: wrapper [-o output] [-p platform] [-i initrd]' >&2
-    echo '       [-d devtree] [-s tree.dts] [-c] [-C cross-prefix]' >&2
-    echo '       [-D datadir] [-W workingdir] [-Z (gz|xz|none)]' >&2
-    echo '       [--no-compression] [vmlinux]' >&2
+    echo '       [-d devtree] [-s tree.dts] [-e esm_blob]' >&2
+    echo '       [-c] [-C cross-prefix] [-D datadir] [-W workingdir]' >&2
+    echo '       [-Z (gz|xz|none)] [--no-compression] [vmlinux]' >&2
     exit 1
 }
 
@@ -105,6 +107,11 @@ while [ "$#" -gt 0 ]; do
 	[ "$#" -gt 0 ] || usage
 	dtb="$1"
 	;;
+    -e)
+	shift
+	[ "$#" -gt 0 ] || usage
+	esm_blob="$1"
+	;;
     -s)
 	shift
 	[ "$#" -gt 0 ] || usage
@@ -211,9 +218,16 @@ objflags=-S
 tmp=$tmpdir/zImage.$$.o
 ksection=.kernel:vmlinux.strip
 isection=.kernel:initrd
+esection=.kernel:esm_blob
 link_address='0x400000'
 make_space=y
 
+
+if [ -n "$esm_blob" -a "$platform" != "pseries" ]; then
+    echo "ESM blob not support on non-pseries platforms" >&2
+    exit 1
+fi
+
 case "$platform" in
 of)
     platformo="$object/of.o $object/epapr.o"
@@ -463,6 +477,10 @@ if [ -n "$dtb" ]; then
     fi
 fi
 
+if [ -n "$esm_blob" ]; then
+    addsec $tmp "$esm_blob" $esection
+fi
+
 if [ "$platform" != "miboot" ]; then
     if [ -n "$link_address" ] ; then
         text_start="-Ttext $link_address"
arch/powerpc/boot/zImage.lds.S +8 -0
index 4ac1e36edfe7..a21f3a76e06f 100644
@@ -68,6 +68,14 @@ SECTIONS
     _initrd_end =  .;
   }
 
+  . = ALIGN(4096);
+  .kernel:esm_blob :
+  {
+    _esm_blob_start =  .;
+    *(.kernel:esm_blob)
+    _esm_blob_end =  .;
+  }
+
 #ifdef CONFIG_PPC64_BOOT_WRAPPER
   . = ALIGN(256);
   .got :
#3 lore raw

Re: [RFC PATCH 02/12] powerpc: Add support for adding an ESM blob to the zImage wrapper

Quoted 6 lines · Thiago Jung Bauermann

From: Benjamin Herrenschmidt <[email protected]>

For secure VMs, the signing tool will create a ticket called the "ESM blob" for the Enter Secure Mode ultravisor call with the signatures of the kernel and initrd among other things.

Quoted 6 lines

This adds support to the wrapper script for adding that blob via the "-e" option to the zImage.pseries.

It also adds code to the zImage wrapper itself to retrieve and if necessary relocate the blob, and pass its address to Linux via the device-tree, to be later consumed by prom_init.

Where does the "BLOB" come from? How is it licensed and how can we satisfy the GPL with it?

#4 lore raw

Re: Re: [RFC PATCH 02/12] powerpc: Add support for adding an ESM blob to the zImage wrapper

Quoted 10 lines · Christoph Hellwig
Quoted 9 lines · Thiago Jung Bauermann

From: Benjamin Herrenschmidt <[email protected]>

For secure VMs, the signing tool will create a ticket called the "ESM blob" for the Enter Secure Mode ultravisor call with the signatures of the kernel and initrd among other things.

This adds support to the wrapper script for adding that blob via the "-e" option to the zImage.pseries.

Quoted 6 lines
Quoted 3 lines

It also adds code to the zImage wrapper itself to retrieve and if necessary relocate the blob, and pass its address to Linux via the device-tree, to be later consumed by prom_init.

Where does the "BLOB" come from? How is it licensed and how can we satisfy the GPL with it?

The "BLOB" is not a piece of code. Its just a piece of data that gets generated by our build tools. This data contains the
signed hash of the kernel, initrd, and kernel command line parameters. Also it contains any information that the creator the the BLOB wants to be made available to anyone needing it, inside the secure-virtual-machine. All of this is integrity-protected and encrypted to safegaurd it when at rest and at runtime.

Bottomline -- Blob is data, and hence no licensing implication. And due to some reason, even data needs to have licensing statement, we can make it available to have no conflicts with GPL.

Signature · 1 line
Ram Pai
#5 lore raw

Re: [RFC PATCH 02/12] powerpc: Add support for adding an ESM blob to the zImage wrapper

In reply to Christoph Hellwig
Quoted 10 lines · Christoph Hellwig
Quoted 9 lines · Thiago Jung Bauermann

From: Benjamin Herrenschmidt <[email protected]>

For secure VMs, the signing tool will create a ticket called the "ESM blob" for the Enter Secure Mode ultravisor call with the signatures of the kernel and initrd among other things.

This adds support to the wrapper script for adding that blob via the "-e" option to the zImage.pseries.

Quoted 6 lines
Quoted 3 lines

It also adds code to the zImage wrapper itself to retrieve and if necessary relocate the blob, and pass its address to Linux via the device-tree, to be later consumed by prom_init.

Where does the "BLOB" come from? How is it licensed and how can we satisfy the GPL with it?

The blob is data, not code, and it will be created by a tool that will be open source. My understanding is that most of it will be encrypted with a session key that is encrypted with the secret key of the ultravisor. Ram Pai's KVM Forum talk last year explained how this works.

Paul.

#6 lore raw

RFC PATCH 3/12 powerpc/prom_init: Add the ESM call to prom_init

In reply to Thiago Jung Bauermann

From: Ram Pai <[email protected]>

Make the Enter-Secure-Mode (ESM) ultravisor call to switch the VM to secure mode. Add "svm=" command line option to turn off switching to secure mode. Introduce CONFIG_PPC_SVM to control support for secure guests.

[ Generate an RTAS os-term hcall when the ESM ucall fails. ]
[ Cleaned up the code a bit. ]
---
 .../admin-guide/kernel-parameters.txt         |   5 +
 arch/powerpc/include/asm/ultravisor-api.h     |   1 +
 arch/powerpc/kernel/prom_init.c               | 124 ++++++++++++++++++
 3 files changed, 130 insertions(+)
Documentation/admin-guide/kernel-parameters.txt +5 -0
index c45a19d654f3..7237d86b25c6 100644
@@ -4501,6 +4501,11 @@
 			/sys/power/pm_test). Only available when CONFIG_PM_DEBUG
 			is set. Default value is 5.
 
+	svm=		[PPC]
+			Format: { on | off | y | n | 1 | 0 }
+			This parameter controls use of the Protected
+			Execution Facility on pSeries.
+
 	swapaccount=[0|1]
 			[KNL] Enable accounting of swap in memory resource
 			controller if no parameter or 1 is given or disable
arch/powerpc/include/asm/ultravisor-api.h +1 -0
index 15e6ce77a131..0e8b72081718 100644
@@ -19,6 +19,7 @@
 
 /* opcodes */
 #define UV_WRITE_PATE			0xF104
+#define UV_ESM				0xF110
 #define UV_RETURN			0xF11C
 
 #endif /* _ASM_POWERPC_ULTRAVISOR_API_H */
arch/powerpc/kernel/prom_init.c +124 -0

160 lines not on this page · raw message

#7 lore raw

Re: [RFC PATCH 03/12] powerpc/prom_init: Add the ESM call to prom_init

Quoted 87 lines · Thiago Jung Bauermann

From: Ram Pai <[email protected]>

Make the Enter-Secure-Mode (ESM) ultravisor call to switch the VM to secure mode. Add "svm=" command line option to turn off switching to secure mode. Introduce CONFIG_PPC_SVM to control support for secure guests.

[ Generate an RTAS os-term hcall when the ESM ucall fails. ]
[ Cleaned up the code a bit. ]
---
 .../admin-guide/kernel-parameters.txt         |   5 +
 arch/powerpc/include/asm/ultravisor-api.h     |   1 +
 arch/powerpc/kernel/prom_init.c               | 124 ++++++++++++++++++
 3 files changed, 130 insertions(+)
Documentation/admin-guide/kernel-parameters.txt +5 -0
index c45a19d654f3..7237d86b25c6 100644
@@ -4501,6 +4501,11 @@
 			/sys/power/pm_test). Only available when CONFIG_PM_DEBUG
 			is set. Default value is 5.
 
+	svm=		[PPC]
+			Format: { on | off | y | n | 1 | 0 }
+			This parameter controls use of the Protected
+			Execution Facility on pSeries.
+
 	swapaccount=[0|1]
 			[KNL] Enable accounting of swap in memory resource
 			controller if no parameter or 1 is given or disable
arch/powerpc/include/asm/ultravisor-api.h +1 -0
index 15e6ce77a131..0e8b72081718 100644
@@ -19,6 +19,7 @@
 
 /* opcodes */
 #define UV_WRITE_PATE			0xF104
+#define UV_ESM				0xF110
 #define UV_RETURN			0xF11C
 
 #endif /* _ASM_POWERPC_ULTRAVISOR_API_H */
arch/powerpc/kernel/prom_init.c +16 -0
index 523bb99d7676..5d8a3efb54f2 100644
@@ -44,6 +44,7 @@
 #include <asm/sections.h>
 #include <asm/machdep.h>
 #include <asm/asm-prototypes.h>
+#include <asm/ultravisor-api.h>
 
 #include <linux/linux_logo.h>
 
@@ -174,6 +175,10 @@ static unsigned long __prombss prom_tce_alloc_end;
 static bool __prombss prom_radix_disable;
 #endif
 
+#ifdef CONFIG_PPC_SVM
+static bool __prombss prom_svm_disable;
+#endif
+
 struct platform_support {
 	bool hash_mmu;
 	bool radix_mmu;
@@ -809,6 +814,17 @@ static void __init early_cmdline_parse(void)
 	if (prom_radix_disable)
 		prom_debug("Radix disabled from cmdline\n");
 #endif /* CONFIG_PPC_PSERIES */
+
+#ifdef CONFIG_PPC_SVM
+	opt = prom_strstr(prom_cmd_line, "svm=");
+	if (opt) {
+		bool val;
+
+		opt += sizeof("svm=") - 1;
+		if (!prom_strtobool(opt, &val))
+			prom_svm_disable = !val;
+	}
+#endif /* CONFIG_PPC_SVM */
 }
 
 #ifdef CONFIG_PPC_PSERIES

arch/powerpc/kernel/prom_init.c static void __init prom_close_stdin(void) · line 1727

Quoted 5 lines
arch/powerpc/kernel/prom_init.c +2 -0
@@ -1707,6 +1723,43 @@ static void __init prom_close_stdin(void)
 	}
 }
 
+#ifdef CONFIG_PPC_SVM
+static int prom_rtas_os_term_hcall(uint64_t args)

This is just an rtas hcall, nothing special about "os-term".

Quoted 124 lines
arch/powerpc/kernel/prom_init.c +106 -0
+{
+	register uint64_t arg1 asm("r3") = 0xf000;
+	register uint64_t arg2 asm("r4") = args;
+
+	asm volatile("sc 1\n" : "=r" (arg1) :
+			"r" (arg1),
+			"r" (arg2) :);
+	return arg1;
+}
+
+static struct rtas_args __prombss os_term_args;
+
+static void __init prom_rtas_os_term(char *str)
+{
+	phandle rtas_node;
+	__be32 val;
+	u32 token;
+
+	prom_printf("%s: start...\n", __func__);
+	rtas_node = call_prom("finddevice", 1, 1, ADDR("/rtas"));
+	prom_printf("rtas_node: %x\n", rtas_node);
+	if (!PHANDLE_VALID(rtas_node))
+		return;
+
+	val = 0;
+	prom_getprop(rtas_node, "ibm,os-term", &val, sizeof(val));
+	token = be32_to_cpu(val);
+	prom_printf("ibm,os-term: %x\n", token);
+	if (token == 0)
+		prom_panic("Could not get token for ibm,os-term\n");
+	os_term_args.token = cpu_to_be32(token);
+	prom_rtas_os_term_hcall((uint64_t)&os_term_args);
+}
+#endif /* CONFIG_PPC_SVM */
+
 /*
  * Allocate room for and instantiate RTAS
  */
@@ -3162,6 +3215,74 @@ static void unreloc_toc(void)
 #endif
 #endif
 
+#ifdef CONFIG_PPC_SVM
+/*
+ * The ESM blob is a data structure with information needed by the Ultravisor to
+ * validate the integrity of the secure guest.
+ */
+static void *get_esm_blob(void)
+{
+	/*
+	 * FIXME: We are still finalizing the details on how prom_init will grab
+	 * the ESM blob. When that is done, this function will be updated.
+	 */
+	return (void *)0xdeadbeef;
+}
+
+/*
+ * Perform the Enter Secure Mode ultracall.
+ */
+static int enter_secure_mode(void *esm_blob, void *retaddr, void *fdt)
+{
+	register uint64_t func asm("r0") = UV_ESM;
+	register uint64_t arg1 asm("r3") = (uint64_t)esm_blob;
+	register uint64_t arg2 asm("r4") = (uint64_t)retaddr;
+	register uint64_t arg3 asm("r5") = (uint64_t)fdt;
+
+	asm volatile("sc 2\n"
+		     : "=r"(arg1)
+		     : "r"(func), "0"(arg1), "r"(arg2), "r"(arg3)
+		     :);
+
+	return (int)arg1;
+}
+
+/*
+ * Call the Ultravisor to transfer us to secure memory if we have an ESM blob.
+ */
+static void setup_secure_guest(void *fdt)
+{
+	void *esm_blob;
+	int ret;
+
+	if (prom_svm_disable) {
+		prom_printf("Secure mode is OFF\n");
+		return;
+	}
+
+	esm_blob = get_esm_blob();
+	if (esm_blob == NULL)
+		/*
+		 * Absence of an ESM blob isn't an error, it just means we
+		 * shouldn't switch to secure mode.
+		 */
+		return;
+
+	/* Switch to secure mode. */
+	prom_printf("Switching to secure mode.\n");
+
+	ret = enter_secure_mode(esm_blob, NULL, fdt);
+	if (ret != U_SUCCESS) {
+		prom_printf("Returned %d from switching to secure mode.\n", ret);
+		prom_rtas_os_term("Switch to secure mode failed.\n");
+	}
+}
+#else
+static void setup_secure_guest(void *fdt)
+{
+}
+#endif /* CONFIG_PPC_SVM */
+
 /*
  * We enter here early on, when the Open Firmware prom is still
  * handling exceptions and the MMU hash table for us.
@@ -3360,6 +3481,9 @@ unsigned long __init prom_init(unsigned long r3, unsigned long r4,
 	unreloc_toc();
 #endif
 
+	/* Move to secure memory if we're supposed to be secure guests. */
+	setup_secure_guest((void *)hdr);
+
 	__start(hdr, kbase, 0, 0, 0, 0, 0);
 
 	return 0;
Signature · 1 line
Alexey
#8 lore raw

Re: [RFC PATCH 03/12] powerpc/prom_init: Add the ESM call to prom_init

Hello Alexey,

Thanks for reviewing this patch!

Quoted 10 lines · Alexey Kardashevskiy

arch/powerpc/kernel/prom_init.c static void __init prom_close_stdin(void) · line 1727

Quoted 6 lines · Thiago Jung Bauermann
arch/powerpc/kernel/prom_init.c +2 -0
@@ -1707,6 +1723,43 @@ static void __init prom_close_stdin(void)
 	}
 }
 
+#ifdef CONFIG_PPC_SVM
+static int prom_rtas_os_term_hcall(uint64_t args)

This is just an rtas hcall, nothing special about "os-term".

Sorry, unfortunately I don't understand how we're treating os-term especially. Do you mean that we should inline this function directly into prom_rtas_os_term()?

Quoted 38 lines
Quoted 38 lines
arch/powerpc/kernel/prom_init.c +35 -0
+{
+	register uint64_t arg1 asm("r3") = 0xf000;
+	register uint64_t arg2 asm("r4") = args;
+
+	asm volatile("sc 1\n" : "=r" (arg1) :
+			"r" (arg1),
+			"r" (arg2) :);
+	return arg1;
+}
+
+static struct rtas_args __prombss os_term_args;
+
+static void __init prom_rtas_os_term(char *str)
+{
+	phandle rtas_node;
+	__be32 val;
+	u32 token;
+
+	prom_printf("%s: start...\n", __func__);
+	rtas_node = call_prom("finddevice", 1, 1, ADDR("/rtas"));
+	prom_printf("rtas_node: %x\n", rtas_node);
+	if (!PHANDLE_VALID(rtas_node))
+		return;
+
+	val = 0;
+	prom_getprop(rtas_node, "ibm,os-term", &val, sizeof(val));
+	token = be32_to_cpu(val);
+	prom_printf("ibm,os-term: %x\n", token);
+	if (token == 0)
+		prom_panic("Could not get token for ibm,os-term\n");
+	os_term_args.token = cpu_to_be32(token);
+	prom_rtas_os_term_hcall((uint64_t)&os_term_args);
+}
+#endif /* CONFIG_PPC_SVM */
+
 /*
  * Allocate room for and instantiate RTAS
  */
Signature · 2 lines
Thiago Jung Bauermann
IBM Linux Technology Center
#9 lore raw

Re: [RFC PATCH 03/12] powerpc/prom_init: Add the ESM call to prom_init

Quoted 18 lines · Thiago Jung Bauermann

Hello Alexey,

Thanks for reviewing this patch!

Quoted 10 lines · Alexey Kardashevskiy

arch/powerpc/kernel/prom_init.c static void __init prom_close_stdin(void) · line 1727

Quoted 6 lines · Thiago Jung Bauermann
arch/powerpc/kernel/prom_init.c +2 -0
@@ -1707,6 +1723,43 @@ static void __init prom_close_stdin(void)
 	}
 }
 
+#ifdef CONFIG_PPC_SVM
+static int prom_rtas_os_term_hcall(uint64_t args)

This is just an rtas hcall, nothing special about "os-term".

Quoted 3 lines

Sorry, unfortunately I don't understand how we're treating os-term especially. Do you mean that we should inline this function directly into prom_rtas_os_term()?

I meant the function name - prom_rtas_os_term_hcall - should rather be prom_rtas_hcall.

Quoted 40 lines
Quoted 39 lines
Quoted 39 lines
arch/powerpc/kernel/prom_init.c +35 -0
+{
+	register uint64_t arg1 asm("r3") = 0xf000;
+	register uint64_t arg2 asm("r4") = args;
+
+	asm volatile("sc 1\n" : "=r" (arg1) :
+			"r" (arg1),
+			"r" (arg2) :);
+	return arg1;
+}
+
+static struct rtas_args __prombss os_term_args;
+
+static void __init prom_rtas_os_term(char *str)
+{
+	phandle rtas_node;
+	__be32 val;
+	u32 token;
+
+	prom_printf("%s: start...\n", __func__);
+	rtas_node = call_prom("finddevice", 1, 1, ADDR("/rtas"));
+	prom_printf("rtas_node: %x\n", rtas_node);
+	if (!PHANDLE_VALID(rtas_node))
+		return;
+
+	val = 0;
+	prom_getprop(rtas_node, "ibm,os-term", &val, sizeof(val));
+	token = be32_to_cpu(val);
+	prom_printf("ibm,os-term: %x\n", token);
+	if (token == 0)
+		prom_panic("Could not get token for ibm,os-term\n");
+	os_term_args.token = cpu_to_be32(token);
+	prom_rtas_os_term_hcall((uint64_t)&os_term_args);
+}
+#endif /* CONFIG_PPC_SVM */
+
 /*
  * Allocate room for and instantiate RTAS
  */
Signature · 1 line
Alexey
#10 lore raw

PATCH 4/12 powerpc/pseries/svm: Add helpers for UV_SHARE_PAGE and UV_UNSHARE_PAGE

In reply to Thiago Jung Bauermann

From: Ram Pai <[email protected]>

These functions are used when the guest wants to grant the hypervisor access to certain pages.

---
 arch/powerpc/include/asm/ultravisor-api.h |  2 ++
 arch/powerpc/include/asm/ultravisor.h     | 14 ++++++++++++++
 2 files changed, 16 insertions(+)
arch/powerpc/include/asm/ultravisor-api.h +2 -0
index 0e8b72081718..ed68b02869fd 100644
@@ -20,6 +20,8 @@
 /* opcodes */
 #define UV_WRITE_PATE			0xF104
 #define UV_ESM				0xF110
+#define UV_SHARE_PAGE			0xF130
+#define UV_UNSHARE_PAGE			0xF134
 #define UV_RETURN			0xF11C
 
 #endif /* _ASM_POWERPC_ULTRAVISOR_API_H */
arch/powerpc/include/asm/ultravisor.h +14 -0
index 09e0a615d96f..537f7717d21a 100644
@@ -44,6 +44,20 @@ static inline int uv_register_pate(u64 lpid, u64 dw0, u64 dw1)
 	return ucall(UV_WRITE_PATE, retbuf, lpid, dw0, dw1);
 }
 
+static inline int uv_share_page(u64 pfn, u64 npages)
+{
+	unsigned long retbuf[UCALL_BUFSIZE];
+
+	return ucall(UV_SHARE_PAGE, retbuf, pfn, npages);
+}
+
+static inline int uv_unshare_page(u64 pfn, u64 npages)
+{
+	unsigned long retbuf[UCALL_BUFSIZE];
+
+	return ucall(UV_UNSHARE_PAGE, retbuf, pfn, npages);
+}
+
 #endif /* !__ASSEMBLY__ */
 
 #endif	/* _ASM_POWERPC_ULTRAVISOR_H */
#11 lore raw

PATCH 5/12 powerpc/pseries: Add and use LPPACA_SIZE constant

In reply to Thiago Jung Bauermann

Helps document what the hard-coded number means.

Also take the opportunity to fix an #endif comment.

---
 arch/powerpc/kernel/paca.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
arch/powerpc/kernel/paca.c +6 -5
index 9cc91d03ab62..854105db5cff 100644
@@ -56,6 +56,8 @@ static void *__init alloc_paca_data(unsigned long size, unsigned long align,
 
 #ifdef CONFIG_PPC_PSERIES
 
+#define LPPACA_SIZE 0x400
+
 /*
  * See asm/lppaca.h for more detail.
  *
@@ -69,7 +71,7 @@ static inline void init_lppaca(struct lppaca *lppaca)
 
 	*lppaca = (struct lppaca) {
 		.desc = cpu_to_be32(0xd397d781),	/* "LpPa" */
-		.size = cpu_to_be16(0x400),
+		.size = cpu_to_be16(LPPACA_SIZE),
 		.fpregs_in_use = 1,
 		.slb_count = cpu_to_be16(64),
 		.vmxregs_in_use = 0,
@@ -79,19 +81,18 @@ static inline void init_lppaca(struct lppaca *lppaca)
 static struct lppaca * __init new_lppaca(int cpu, unsigned long limit)
 {
 	struct lppaca *lp;
-	size_t size = 0x400;
 
-	BUILD_BUG_ON(size < sizeof(struct lppaca));
+	BUILD_BUG_ON(sizeof(struct lppaca) > LPPACA_SIZE);
 
 	if (early_cpu_has_feature(CPU_FTR_HVMODE))
 		return NULL;
 
-	lp = alloc_paca_data(size, 0x400, limit, cpu);
+	lp = alloc_paca_data(LPPACA_SIZE, 0x400, limit, cpu);
 	init_lppaca(lp);
 
 	return lp;
 }
-#endif /* CONFIG_PPC_BOOK3S */
+#endif /* CONFIG_PPC_PSERIES */
 
 #ifdef CONFIG_PPC_BOOK3S_64
 
#12 lore raw

PATCH 6/12 powerpc/pseries/svm: Use shared memory for LPPACA structures

In reply to Thiago Jung Bauermann

From: Anshuman Khandual <[email protected]>

LPPACA structures need to be shared with the host. Hence they need to be in shared memory. Instead of allocating individual chunks of memory for a given structure from memblock, a contiguous chunk of memory is allocated and then converted into shared memory. Subsequent allocation requests will come from the contiguous chunk which will be always shared memory for all structures.

While we are able to use a kmem_cache constructor for the Debug Trace Log, LPPACAs are allocated very early in the boot process (before SLUB is available) so we need to use a simpler scheme here.

Introduce helper is_svm_platform() which uses the S bit of the MSR to tell whether we're running as a secure guest.

---
 arch/powerpc/include/asm/svm.h | 26 ++++++++++++++++++++
 arch/powerpc/kernel/paca.c     | 43 +++++++++++++++++++++++++++++++++-
 2 files changed, 68 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/svm.h new file +26 -0
new file mode 100644
index 000000000000..fef3740f46a6
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * SVM helper functions
+ *
+ * Copyright 2019 Anshuman Khandual, IBM Corporation.
+ */
+
+#ifndef _ASM_POWERPC_SVM_H
+#define _ASM_POWERPC_SVM_H
+
+#ifdef CONFIG_PPC_SVM
+
+static inline bool is_secure_guest(void)
+{
+	return mfmsr() & MSR_S;
+}
+
+#else /* CONFIG_PPC_SVM */
+
+static inline bool is_secure_guest(void)
+{
+	return false;
+}
+
+#endif /* CONFIG_PPC_SVM */
+#endif /* _ASM_POWERPC_SVM_H */
arch/powerpc/kernel/paca.c +42 -1

61 lines not on this page · raw message

#13 lore raw

PATCH 7/12 powerpc/pseries/svm: Use shared memory for Debug Trace Log (DTL)

In reply to Thiago Jung Bauermann

From: Anshuman Khandual <[email protected]>

Secure guests need to share the DTL buffers with the hypervisor. To that end, use a kmem_cache constructor which converts the underlying buddy allocated SLUB cache pages into shared memory.

---
 arch/powerpc/include/asm/svm.h          |  5 ++++
 arch/powerpc/platforms/pseries/Makefile |  1 +
 arch/powerpc/platforms/pseries/setup.c  |  5 +++-
 arch/powerpc/platforms/pseries/svm.c    | 40 +++++++++++++++++++++++++
 4 files changed, 50 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/svm.h +5 -0
index fef3740f46a6..f253116c31fc 100644
@@ -15,6 +15,9 @@ static inline bool is_secure_guest(void)
 	return mfmsr() & MSR_S;
 }
 
+void dtl_cache_ctor(void *addr);
+#define get_dtl_cache_ctor()	(is_secure_guest() ? dtl_cache_ctor : NULL)
+
 #else /* CONFIG_PPC_SVM */
 
 static inline bool is_secure_guest(void)
@@ -22,5 +25,7 @@ static inline bool is_secure_guest(void)
 	return false;
 }
 
+#define get_dtl_cache_ctor() NULL
+
 #endif /* CONFIG_PPC_SVM */
 #endif /* _ASM_POWERPC_SVM_H */
arch/powerpc/platforms/pseries/Makefile +1 -0
index a43ec843c8e2..b7b6e6f52bd0 100644
@@ -25,6 +25,7 @@ obj-$(CONFIG_LPARCFG)		+= lparcfg.o
 obj-$(CONFIG_IBMVIO)		+= vio.o
 obj-$(CONFIG_IBMEBUS)		+= ibmebus.o
 obj-$(CONFIG_PAPR_SCM)		+= papr_scm.o
+obj-$(CONFIG_PPC_SVM)		+= svm.o
 
 ifdef CONFIG_PPC_PSERIES
 obj-$(CONFIG_SUSPEND)		+= suspend.o
arch/powerpc/platforms/pseries/setup.c +4 -1
index e4f0dfd4ae33..c928e6e8a279 100644
@@ -71,6 +71,7 @@
 #include <asm/isa-bridge.h>
 #include <asm/security_features.h>
 #include <asm/asm-const.h>
+#include <asm/svm.h>
 
 #include "pseries.h"
 #include "../../../../drivers/pci/pci.h"
@@ -329,8 +330,10 @@ static inline int alloc_dispatch_logs(void)
 
 static int alloc_dispatch_log_kmem_cache(void)
 {
+	void (*ctor)(void *) = get_dtl_cache_ctor();
+
 	dtl_cache = kmem_cache_create("dtl", DISPATCH_LOG_BYTES,
-						DISPATCH_LOG_BYTES, 0, NULL);
+						DISPATCH_LOG_BYTES, 0, ctor);
 	if (!dtl_cache) {
 		pr_warn("Failed to create dispatch trace log buffer cache\n");
 		pr_warn("Stolen time statistics will be unreliable\n");
arch/powerpc/platforms/pseries/svm.c new file +40 -0
new file mode 100644
index 000000000000..c508196f7c83
@@ -0,0 +1,40 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Secure VM platform
+ *
+ * Copyright 2019 IBM Corporation
+ * Author: Anshuman Khandual <[email protected]>
+ */
+
+#include <linux/mm.h>
+#include <asm/ultravisor.h>
+
+/* There's one dispatch log per CPU. */
+#define NR_DTL_PAGE (DISPATCH_LOG_BYTES * CONFIG_NR_CPUS / PAGE_SIZE)
+
+static struct page *dtl_page_store[NR_DTL_PAGE];
+static long dtl_nr_pages;
+
+static bool is_dtl_page_shared(struct page *page)
+{
+	long i;
+
+	for (i = 0; i < dtl_nr_pages; i++)
+		if (dtl_page_store[i] == page)
+			return true;
+
+	return false;
+}
+
+void dtl_cache_ctor(void *addr)
+{
+	unsigned long pfn = PHYS_PFN(__pa(addr));
+	struct page *page = pfn_to_page(pfn);
+
+	if (!is_dtl_page_shared(page)) {
+		dtl_page_store[dtl_nr_pages] = page;
+		dtl_nr_pages++;
+		WARN_ON(dtl_nr_pages >= NR_DTL_PAGE);
+		uv_share_page(pfn, 1);
+	}
+}
#14 lore raw

PATCH 8/12 powerpc/pseries/svm: Export guest SVM status to user space via sysfs

In reply to Thiago Jung Bauermann

From: Ryan Grimm <[email protected]>

User space might want to know it's running in a secure VM. It can't do a mfmsr because mfmsr is a privileged instruction.

The solution here is to create a cpu attribute:

/sys/devices/system/cpu/svm

which will read 0 or 1 based on the S bit of the guest's CPU 0.

---
 arch/powerpc/kernel/sysfs.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)
arch/powerpc/kernel/sysfs.c +29 -0
index e8e93c2c7d03..8fdab134e9ae 100644
@@ -18,6 +18,7 @@
 #include <asm/smp.h>
 #include <asm/pmc.h>
 #include <asm/firmware.h>
+#include <asm/svm.h>
 
 #include "cacheinfo.h"
 #include "setup.h"
@@ -714,6 +715,32 @@ static struct device_attribute pa6t_attrs[] = {
 #endif /* HAS_PPC_PMC_PA6T */
 #endif /* HAS_PPC_PMC_CLASSIC */
 
+#ifdef CONFIG_PPC_SVM
+static void get_svm(void *val)
+{
+	u32 *value = val;
+
+	*value = is_secure_guest();
+}
+
+static ssize_t show_svm(struct device *dev, struct device_attribute *attr, char *buf)
+{
+	u32 val;
+	smp_call_function_single(0, get_svm, &val, 1);
+	return sprintf(buf, "%u\n", val);
+}
+static DEVICE_ATTR(svm, 0444, show_svm, NULL);
+
+static void create_svm_file(void)
+{
+	device_create_file(cpu_subsys.dev_root, &dev_attr_svm);
+}
+#else
+static void create_svm_file(void)
+{
+}
+#endif /* CONFIG_PPC_SVM */
+
 static int register_cpu_online(unsigned int cpu)
 {
 	struct cpu *c = &per_cpu(cpu_devices, cpu);
@@ -1057,6 +1084,8 @@ static int __init topology_init(void)
 	sysfs_create_dscr_default();
 #endif /* CONFIG_PPC64 */
 
+	create_svm_file();
+
 	return 0;
 }
 subsys_initcall(topology_init);
#15 lore raw

PATCH 9/12 powerpc/pseries/svm: Disable doorbells in SVM guests

In reply to Thiago Jung Bauermann

From: Sukadev Bhattiprolu <[email protected]>

Normally, the HV emulates some instructions like MSGSNDP, MSGCLRP from a KVM guest. To emulate the instructions, it must first read the instruction from the guest's memory and decode its parameters.

However for a secure guest (aka SVM), the page containing the instruction is in secure memory and the HV cannot access directly. It would need the Ultravisor (UV) to facilitate accessing the instruction and parameters but the UV currently does not have the support for such accesses.

Until the UV has such support, disable doorbells in SVMs. This might incur a performance hit but that is yet to be quantified.

With this patch applied (needed only in SVMs not needed for HV) we are able to launch SVM guests with multi-core support. Eg:

	qemu -smp sockets=2,cores=2,threads=2.

Fix suggested by Benjamin Herrenschmidt. Thanks to input from Paul Mackerras, Ram Pai and Michael Anderson.

---
 arch/powerpc/platforms/pseries/smp.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/powerpc/platforms/pseries/smp.c +2 -1
index 3df46123cce3..95a5c24a1544 100644
@@ -45,6 +45,7 @@
 #include <asm/dbell.h>
 #include <asm/plpar_wrappers.h>
 #include <asm/code-patching.h>
+#include <asm/svm.h>
 
 #include "pseries.h"
 #include "offline_states.h"
@@ -225,7 +226,7 @@ static __init void pSeries_smp_probe_xics(void)
 {
 	xics_smp_probe();
 
-	if (cpu_has_feature(CPU_FTR_DBELL))
+	if (cpu_has_feature(CPU_FTR_DBELL) && !is_secure_guest())
 		smp_ops->cause_ipi = smp_pseries_cause_ipi;
 	else
 		smp_ops->cause_ipi = icp_ops->cause_ipi;
#16 lore raw

PATCH 10/12 powerpc/pseries/iommu: Don't use dma_iommu_ops on secure guests

In reply to Thiago Jung Bauermann

Secure guest memory is inacessible to devices so regular DMA isn't possible.

In that case set devices' dma_map_ops to NULL so that the generic DMA code path will use SWIOTLB and DMA to bounce buffers.

---
 arch/powerpc/platforms/pseries/iommu.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
arch/powerpc/platforms/pseries/iommu.c +5 -1
index 03bbb299320e..7d9550edb700 100644
@@ -50,6 +50,7 @@
 #include <asm/udbg.h>
 #include <asm/mmzone.h>
 #include <asm/plpar_wrappers.h>
+#include <asm/svm.h>
 
 #include "pseries.h"
 
@@ -1332,7 +1333,10 @@ void iommu_init_early_pSeries(void)
 	of_reconfig_notifier_register(&iommu_reconfig_nb);
 	register_memory_notifier(&iommu_mem_nb);
 
-	set_pci_dma_ops(&dma_iommu_ops);
+	if (is_secure_guest())
+		set_pci_dma_ops(NULL);
+	else
+		set_pci_dma_ops(&dma_iommu_ops);
 }
 
 static int __init disable_multitce(char *str)
#17 lore raw

PATCH 11/12 powerpc/pseries/svm: Force SWIOTLB for secure guests

In reply to Thiago Jung Bauermann

From: Anshuman Khandual <[email protected]>

SWIOTLB checks range of incoming CPU addresses to be bounced and sees if the device can access it through its DMA window without requiring bouncing. In such cases it just chooses to skip bouncing. But for cases like secure guests on powerpc platform all addresses need to be bounced into the shared pool of memory because the host cannot access it otherwise. Hence the need to do the bouncing is not related to device's DMA window and use of bounce buffers is forced by setting swiotlb_force.

Also, connect the shared memory conversion functions into the ARCH_HAS_MEM_ENCRYPT hooks and call swiotlb_update_mem_attributes() to convert SWIOTLB's memory pool to shared memory.

[ Use ARCH_HAS_MEM_ENCRYPT hooks to share swiotlb memory pool. ]
---
 arch/powerpc/include/asm/mem_encrypt.h | 19 +++++++++++
 arch/powerpc/platforms/pseries/Kconfig |  5 +++
 arch/powerpc/platforms/pseries/svm.c   | 45 ++++++++++++++++++++++++++
 3 files changed, 69 insertions(+)
arch/powerpc/include/asm/mem_encrypt.h new file +19 -0
new file mode 100644
index 000000000000..45d5e4d0e6e0
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * SVM helper functions
+ *
+ * Copyright 2019 IBM Corporation
+ */
+
+#ifndef _ASM_POWERPC_MEM_ENCRYPT_H
+#define _ASM_POWERPC_MEM_ENCRYPT_H
+
+#define sme_me_mask	0ULL
+
+static inline bool sme_active(void) { return false; }
+static inline bool sev_active(void) { return false; }
+
+int set_memory_encrypted(unsigned long addr, int numpages);
+int set_memory_decrypted(unsigned long addr, int numpages);
+
+#endif /* _ASM_POWERPC_MEM_ENCRYPT_H */
arch/powerpc/platforms/pseries/Kconfig +5 -0
index 82c16aa4f1ce..41b10f3bc729 100644
@@ -145,9 +145,14 @@ config PAPR_SCM
 	help
 	  Enable access to hypervisor provided storage class memory.
 
+config ARCH_HAS_MEM_ENCRYPT
+	def_bool n
+
 config PPC_SVM
 	bool "Secure virtual machine (SVM) support for POWER"
 	depends on PPC_PSERIES
+	select SWIOTLB
+	select ARCH_HAS_MEM_ENCRYPT
 	default n
 	help
 	 Support secure guests on POWER. There are certain POWER platforms which
arch/powerpc/platforms/pseries/svm.c +45 -0

53 lines not on this page · raw message

#18 lore raw

Re: [PATCH 11/12] powerpc/pseries/svm: Force SWIOTLB for secure guests

Quoted 15 lines
arch/powerpc/include/asm/mem_encrypt.h new file +9 -0
new file mode 100644
index 000000000000..45d5e4d0e6e0
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * SVM helper functions
+ *
+ * Copyright 2019 IBM Corporation
+ */
+
+#ifndef _ASM_POWERPC_MEM_ENCRYPT_H
+#define _ASM_POWERPC_MEM_ENCRYPT_H

arch/powerpc/include/asm/mem_encrypt.h line 19

Quoted 10 lines
arch/powerpc/include/asm/mem_encrypt.h new file +10 -0
+
+#define sme_me_mask	0ULL
+
+static inline bool sme_active(void) { return false; }
+static inline bool sev_active(void) { return false; }
+
+int set_memory_encrypted(unsigned long addr, int numpages);
+int set_memory_decrypted(unsigned long addr, int numpages);
+
+#endif /* _ASM_POWERPC_MEM_ENCRYPT_H */

S/390 seems to be adding a stub header just like this. Can you please clean up the Kconfig and generic headers bits for memory encryption so that we don't need all this boilerplate code?

arch/powerpc/include/asm/mem_encrypt.h

Quoted 6 lines

config PPC_SVM

arch/powerpc/include/asm/mem_encrypt.h new file +2 -0
 	bool "Secure virtual machine (SVM) support for POWER"
 	depends on PPC_PSERIES
+	select SWIOTLB
+	select ARCH_HAS_MEM_ENCRYPT
 	default n

n is the default default, no need to explictly specify it.

#19 lore raw

Re: [PATCH 11/12] powerpc/pseries/svm: Force SWIOTLB for secure guests

Hello Christoph,

Thanks for reviewing the patch!

Quoted 21 lines · Christoph Hellwig
Quoted 21 lines
arch/powerpc/include/asm/mem_encrypt.h new file +15 -0
new file mode 100644
index 000000000000..45d5e4d0e6e0
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * SVM helper functions
+ *
+ * Copyright 2019 IBM Corporation
+ */
+
+#ifndef _ASM_POWERPC_MEM_ENCRYPT_H
+#define _ASM_POWERPC_MEM_ENCRYPT_H
+
+#define sme_me_mask	0ULL
+
+static inline bool sme_active(void) { return false; }
+static inline bool sev_active(void) { return false; }
+
Quoted 8 lines

arch/powerpc/include/asm/mem_encrypt.h line 19

Quoted 4 lines
arch/powerpc/include/asm/mem_encrypt.h new file +4 -0
+int set_memory_encrypted(unsigned long addr, int numpages);
+int set_memory_decrypted(unsigned long addr, int numpages);
+
+#endif /* _ASM_POWERPC_MEM_ENCRYPT_H */

S/390 seems to be adding a stub header just like this. Can you please clean up the Kconfig and generic headers bits for memory encryption so that we don't need all this boilerplate code?

Yes, that's a good idea. Will do.

Quoted 8 lines

arch/powerpc/include/asm/mem_encrypt.h

Quoted 6 lines

config PPC_SVM

arch/powerpc/include/asm/mem_encrypt.h new file +2 -0
 	bool "Secure virtual machine (SVM) support for POWER"
 	depends on PPC_PSERIES
+	select SWIOTLB
+	select ARCH_HAS_MEM_ENCRYPT
 	default n

n is the default default, no need to explictly specify it.

Indeed. Changed for the next version.

Signature · 2 lines
Thiago Jung Bauermann
IBM Linux Technology Center
#20 lore raw

PATCH 12/12 powerpc/configs: Enable secure guest support in pseries and ppc64 defconfigs

In reply to Thiago Jung Bauermann

From: Ryan Grimm <[email protected]>

Enables running as a secure guest in platforms with an Ultravisor.

---
 arch/powerpc/configs/ppc64_defconfig   | 1 +
 arch/powerpc/configs/pseries_defconfig | 1 +
 2 files changed, 2 insertions(+)
arch/powerpc/configs/ppc64_defconfig +1 -0
index d7c381009636..725297438320 100644
@@ -31,6 +31,7 @@ CONFIG_DTL=y
 CONFIG_SCANLOG=m
 CONFIG_PPC_SMLPAR=y
 CONFIG_IBMEBUS=y
+CONFIG_PPC_SVM=y
 CONFIG_PPC_MAPLE=y
 CONFIG_PPC_PASEMI=y
 CONFIG_PPC_PASEMI_IOMMU=y
arch/powerpc/configs/pseries_defconfig +1 -0
index 62e12f61a3b2..724a574fe4b2 100644
@@ -42,6 +42,7 @@ CONFIG_DTL=y
 CONFIG_SCANLOG=m
 CONFIG_PPC_SMLPAR=y
 CONFIG_IBMEBUS=y
+CONFIG_PPC_SVM=y
 # CONFIG_PPC_PMAC is not set
 CONFIG_RTAS_FLASH=m
 CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND=y
#21 lore raw

RFC PATCH 1/1 powerpc/pseries/svm: Unshare all pages before kexecing a new kernel

powerpc/pseries/svm: Unshare all pages before kexecing a new kernel.

A new kernel deserves a clean slate. Any pages shared with the hypervisor is unshared before invoking the new kernel. However there are exceptions. If the new kernel is invoked to dump the current kernel, or if there is a explicit request to preserve the state of the current kernel, unsharing of pages is skipped.

NOTE: Reserve atleast 256M for crashkernel. Otherwise SWIOTLB allocation fails and crash kernel fails to boot.

arch/powerpc/include/asm/ultravisor-api.h +1 -0
index 8a6c5b4d..c8dd470 100644
@@ -31,5 +31,6 @@
 #define UV_UNSHARE_PAGE			0xF134
 #define UV_PAGE_INVAL			0xF138
 #define UV_SVM_TERMINATE		0xF13C
+#define UV_UNSHARE_ALL_PAGES		0xF140
 
 #endif /* _ASM_POWERPC_ULTRAVISOR_API_H */
arch/powerpc/include/asm/ultravisor.h +6 -0
index bf5ac05..73c44ff 100644
@@ -120,6 +120,12 @@ static inline int uv_unshare_page(u64 pfn, u64 npages)
 	return ucall(UV_UNSHARE_PAGE, retbuf, pfn, npages);
 }
 
+static inline int uv_unshare_all_pages(void)
+{
+	unsigned long retbuf[UCALL_BUFSIZE];
+
+	return ucall(UV_UNSHARE_ALL_PAGES, retbuf);
+}
 #endif /* !__ASSEMBLY__ */
 
 #endif	/* _ASM_POWERPC_ULTRAVISOR_H */
arch/powerpc/kernel/machine_kexec_64.c +7 -0
index 75692c3..a93e3ab 100644
@@ -329,6 +329,13 @@ void default_machine_kexec(struct kimage *image)
 #ifdef CONFIG_PPC_PSERIES
 	kexec_paca.lppaca_ptr = NULL;
 #endif
+
+	if (is_svm_platform() && !(image->preserve_context ||
+				   image->type == KEXEC_TYPE_CRASH)) {
+		uv_unshare_all_pages();
+		printk("kexec: Unshared all shared pages.\n");
+	}
+
 	paca_ptrs[kexec_paca.paca_index] = &kexec_paca;
 
 	setup_paca(&kexec_paca);
#22 lore raw
In reply to Thiago Jung Bauermann

Hello,

Quoted 10 lines · Thiago Jung Bauermann

This series enables Secure Virtual Machines (SVMs) on powerpc. SVMs use the Protected Execution Facility (PEF) and request to be migrated to secure memory during prom_init() so by default all of their memory is inaccessible to the hypervisor. There is an Ultravisor call that the VM can use to request certain pages to be made accessible to (or shared with) the hypervisor.

The objective of these patches is to have the guest perform this request for buffers that need to be accessed by the hypervisor such as the LPPACAs, the SWIOTLB memory and the Debug Trace Log.

Ping? Any more comments on these patches? Or even acks? :-)

Signature · 2 lines
Thiago Jung Bauermann
IBM Linux Technology Center