Skip to contentSkip to conversation

Intel SGX Driver

PATCH Superseded Jarkko Sakkinen 9 participants 78 messages Nov 13, 2017 – Dec 20, 2017

Subsystems (6)
Documentation x86 VFS KVM +2 more subsystems

v5 of 10 Superseded Latest is v10 →

lore
Share & tools
lore
#0 lore raw

v5 cover Intel SGX Driver

Intel(R) SGX is a set of CPU instructions that can be used by applications to set aside private regions of code and data. The code outside the enclave is disallowed to access the memory inside the enclave by the CPU access control. In a way you can think that SGX provides inverted sandbox. It protects the application from a malicious host.

There is a new hardware unit in the processor called Memory Encryption Engine (MEE) starting from the Skylake microacrhitecture. BIOS can define one or many MEE regions that can hold enclave data by configuring them with PRMRR registers.

The MEE automatically encrypts the data leaving the processor package to the MEE regions. The data is encrypted using a random key whose life-time is exactly one power cycle.

You can tell if your CPU supports SGX by looking into /proc/cpuinfo:

cat /proc/cpuinfo | grep sgx

The GIT repositoy for SGX driver resides in

https://github.com/jsakkine-intel/linux-sgx.git

'le' branch contains the upstream candidate patches.

'master' branch contains the same patches with the following differences:

* top-level patch modifies the ioctl API to be SDK compatible
* does not use flexible launch control but instead relies on SDK provided Intel launch enclave.

v5:
* Described IPC between the Launch Enclave and kernel in the commit messages.
* Fixed all relevant checkpatch.pl issues that I have forgot fix in earlier versions except those that exist in the imported TinyCrypt code.
* Fixed spelling mistakes in the documentation.
* Forgot to check the return value of sgx_drv_subsys_init().
* Encapsulated properly page cache init and teardown.
* Collect epc pages to a temp list in sgx_add_epc_bank
* Removed SGX_ENCLAVE_INIT_ARCH constant.

v4:
* Tied life-cycle of the sgx_le_proxy process to /dev/sgx.
* Removed __exit annotation from sgx_drv_subsys_exit().
* Fixed a leak of a backing page in sgx_process_add_page_req() in the case when vm_insert_pfn() fails.
* Removed unused symbol exports for sgx_page_cache.c.
* Updated sgx_alloc_page() to require encl parameter and documented the behavior (Sean Christopherson).
* Refactored a more lean API for sgx_encl_find() and documented the behavior.
* Moved #PF handler to sgx_fault.c.
* Replaced subsys_system_register() with plain bus_register().
* Retry EINIT 2nd time only if MSRs are not locked.

v3:
* Check that FEATURE_CONTROL_LOCKED and FEATURE_CONTROL_SGX_ENABLE are set.
* Return -ERESTARTSYS in __sgx_encl_add_page() when sgx_alloc_page() fails.
* Use unused bits in epc_page->pa to store the bank number.
* Removed #ifdef for WQ_NONREENTRANT.
* If mmu_notifier_register() fails with -EINTR, return -ERESTARTSYS.
* Added --remove-section=.got.plt to objcopy flags in order to prevent a dummy .got.plt, which will cause an inconsistent size for the LE.
* Documented sgx_encl_* functions.
* Added remark about AES implementation used inside the LE.
* Removed redundant sgx_sys_exit() from le/main.c.
* Fixed struct sgx_secinfo alignment from 128 to 64 bytes.
* Validate miscselect in sgx_encl_create().
* Fixed SSA frame size calculation to take the misc region into account.
* Implemented consistent exception handling to __encls() and __encls_ret().
* Implemented a proper device model in order to allow sysfs attributes and in-kernel API.
* Cleaned up various "find enclave" implementations to the unified sgx_encl_find().
* Validate that vm_pgoff is zero.
* Discard backing pages with shmem_truncate_range() after EADD.
* Added missing EEXTEND operations to LE signing and launch.
* Fixed SSA size for GPRS region from 168 to 184 bytes.
* Fixed the checks for TCS flags. Now DBGOPTIN is allowed.
* Check that TCS addresses are in ELRANGE and not just page aligned.
* Require kernel to be compiled with X64_64 and CPU_SUP_INTEL.
* Fixed an incorrect value for SGX_ATTR_DEBUG from 0x01 to 0x02.

v2:
* get_rand_uint32() changed the value of the pointer instead of value where it is pointing at.
* Launch enclave incorrectly used sigstruct attributes-field instead of enclave attributes-field.
* Removed unused struct sgx_add_page_req from sgx_ioctl.c
* Removed unused sgx_has_sgx2.
* Updated arch/x86/include/asm/sgx.h so that it provides stub implementations when sgx in not enabled.
* Removed cruft rdmsr-calls from sgx_set_pubkeyhash_msrs().
* return -ENOMEM in sgx_alloc_page() when VA pages consume too much space
* removed unused global sgx_nr_pids
* moved sgx_encl_release to sgx_encl.c
* return -ERESTARTSYS instead of -EINTR in sgx_encl_init()

Haim Cohen (1):
x86: add SGX MSRs to msr-index.h

Jarkko Sakkinen (8):
intel_sgx: updated MAINTAINERS
x86: define the feature control MSR's SGX launch control bit
intel_sgx: driver for Intel Software Guard Extensions
intel_sgx: ptrace() support
intel_sgx: in-kernel launch enclave
fs/pipe.c: export create_pipe_files() and replace_fd()
intel_sgx: glue code for in-kernel LE
intel_sgx: driver documentation

Kai Huang (1):
x86: add SGX definition to cpufeature

Sean Christopherson (1):
x86: define the feature control MSR's SGX enable bit

 Documentation/index.rst                            |    1 +
 Documentation/x86/intel_sgx.rst                    |  131 +++
 MAINTAINERS                                        |    5 +
 arch/x86/include/asm/cpufeatures.h                 |    2 +
 arch/x86/include/asm/msr-index.h                   |    8 +
 arch/x86/include/asm/sgx.h                         |  233 +++++
 arch/x86/include/asm/sgx_arch.h                    |  268 ++++++
 arch/x86/include/uapi/asm/sgx.h                    |  138 +++
 drivers/platform/x86/Kconfig                       |    2 +
 drivers/platform/x86/Makefile                      |    1 +
 drivers/platform/x86/intel_sgx/Kconfig             |   34 +
 drivers/platform/x86/intel_sgx/Makefile            |   32 +
 drivers/platform/x86/intel_sgx/le/Makefile         |   26 +
 drivers/platform/x86/intel_sgx/le/enclave/Makefile |   46 +
 .../x86/intel_sgx/le/enclave/aes_encrypt.c         |  191 ++++
 .../platform/x86/intel_sgx/le/enclave/cmac_mode.c  |  254 +++++
 .../x86/intel_sgx/le/enclave/encl_bootstrap.S      |  163 ++++
 .../intel_sgx/le/enclave/include/tinycrypt/aes.h   |  133 +++
 .../le/enclave/include/tinycrypt/cmac_mode.h       |  194 ++++
 .../le/enclave/include/tinycrypt/constants.h       |   59 ++
 .../intel_sgx/le/enclave/include/tinycrypt/utils.h |   95 ++
 drivers/platform/x86/intel_sgx/le/enclave/main.c   |  203 ++++
 .../platform/x86/intel_sgx/le/enclave/sgx_le.lds   |   28 +
 .../platform/x86/intel_sgx/le/enclave/sgxsign.c    |  538 +++++++++++
 drivers/platform/x86/intel_sgx/le/enclave/utils.c  |   78 ++
 drivers/platform/x86/intel_sgx/le/entry.S          |  117 +++
 .../platform/x86/intel_sgx/le/include/sgx_asm.h    |   64 ++
 .../platform/x86/intel_sgx/le/include/sgx_encl.h   |  110 +++
 drivers/platform/x86/intel_sgx/le/main.c           |  214 +++++
 drivers/platform/x86/intel_sgx/le/sgx_le_piggy.S   |   15 +
 drivers/platform/x86/intel_sgx/sgx.h               |  277 ++++++
 drivers/platform/x86/intel_sgx/sgx_encl.c          | 1007 ++++++++++++++++++++
 drivers/platform/x86/intel_sgx/sgx_ioctl.c         |  279 ++++++
 drivers/platform/x86/intel_sgx/sgx_le.c            |  313 ++++++
 .../platform/x86/intel_sgx/sgx_le_proxy_piggy.S    |   15 +
 drivers/platform/x86/intel_sgx/sgx_main.c          |  457 +++++++++
 drivers/platform/x86/intel_sgx/sgx_page_cache.c    |  614 ++++++++++++
 drivers/platform/x86/intel_sgx/sgx_util.c          |  397 ++++++++
 drivers/platform/x86/intel_sgx/sgx_vma.c           |  232 +++++
 fs/file.c                                          |    1 +
 fs/pipe.c                                          |    1 +
 41 files changed, 6976 insertions(+)
 create mode 100644 Documentation/x86/intel_sgx.rst
 create mode 100644 arch/x86/include/asm/sgx.h
 create mode 100644 arch/x86/include/asm/sgx_arch.h
 create mode 100644 arch/x86/include/uapi/asm/sgx.h
 create mode 100644 drivers/platform/x86/intel_sgx/Kconfig
 create mode 100644 drivers/platform/x86/intel_sgx/Makefile
 create mode 100644 drivers/platform/x86/intel_sgx/le/Makefile
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/Makefile
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/aes_encrypt.c
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/cmac_mode.c
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/encl_bootstrap.S
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/aes.h
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/cmac_mode.h
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/constants.h
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/utils.h
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/main.c
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/sgx_le.lds
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/sgxsign.c
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/utils.c
 create mode 100644 drivers/platform/x86/intel_sgx/le/entry.S
 create mode 100644 drivers/platform/x86/intel_sgx/le/include/sgx_asm.h
 create mode 100644 drivers/platform/x86/intel_sgx/le/include/sgx_encl.h
 create mode 100644 drivers/platform/x86/intel_sgx/le/main.c
 create mode 100644 drivers/platform/x86/intel_sgx/le/sgx_le_piggy.S
 create mode 100644 drivers/platform/x86/intel_sgx/sgx.h
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_encl.c
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_ioctl.c
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_le.c
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_le_proxy_piggy.S
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_main.c
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_page_cache.c
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_util.c
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_vma.c
Signature · 1 line
2.14.1
#1 lore raw

PATCH v5 1/11 intel_sgx: updated MAINTAINERS

---
 MAINTAINERS | 5 +++++
 1 file changed, 5 insertions(+)
MAINTAINERS +5 -0
index 2d3d750b19c0..30a4b7f97a93 100644
@@ -14932,6 +14932,11 @@ L:	[email protected]
 S:	Maintained
 F:	mm/zswap.c
 
+INTEL SGX
+M:	Jarkko Sakkinen <[email protected]>
+L:	[email protected]
+Q:	https://patchwork.kernel.org/project/intel-sgx/list/
+
 THE REST
 M:	Linus Torvalds <[email protected]>
 L:	[email protected]
Signature · 1 line
2.14.1
#2 lore raw

Re: [PATCH v5 01/11] intel_sgx: updated MAINTAINERS

Quoted 10 lines · Jarkko Sakkinen
---
 MAINTAINERS | 5 +++++
 1 file changed, 5 insertions(+)
MAINTAINERS +0 -0
index 2d3d750b19c0..30a4b7f97a93 100644

MAINTAINERS L: [email protected] · line 14936

Quoted 5 lines
MAINTAINERS +2 -0
@@ -14932,6 +14932,11 @@ L:	[email protected]
 S:	Maintained
 F:	mm/zswap.c
 
+INTEL SGX
+M:	Jarkko Sakkinen <[email protected]>

It is clear from the review thus far that this is going to need additional entries here. Likely from the x86 folks at the very least. If not maintainers then at least reviewers (R:).

MAINTAINERS L: [email protected] · line 14938

Quoted 2 lines
MAINTAINERS +2 -0
+L:	[email protected]
+Q:	https://patchwork.kernel.org/project/intel-sgx/list/

F: entries here

Signature · 2 lines
Darren Hart
VMware Open Source Technology Center
#3 lore raw

Re: [PATCH v5 01/11] intel_sgx: updated MAINTAINERS

Quoted 14 lines · Darren Hart
Quoted 13 lines · Jarkko Sakkinen
---
 MAINTAINERS | 5 +++++
 1 file changed, 5 insertions(+)
MAINTAINERS +0 -0
index 2d3d750b19c0..30a4b7f97a93 100644
@@ -14932,6 +14932,11 @@ L:	[email protected]
 S:	Maintained
 F:	mm/zswap.c
 
Quoted 6 lines

MAINTAINERS L: [email protected] · line 14936

Quoted 2 lines
MAINTAINERS +2 -0
+INTEL SGX
+M:	Jarkko Sakkinen <[email protected]>

It is clear from the review thus far that this is going to need additional entries here. Likely from the x86 folks at the very least. If not maintainers then at least reviewers (R:).

Point taken. I think I can cope with maintaining SGX tree once it is in the mainline but additional R-entries would make sense. Who do you suggest? I'm thinking of Dave Hansen at least.

/Jarkko

#4 lore raw

PATCH v5 2/11 x86: add SGX definition to cpufeature

In reply to Jarkko Sakkinen

From: Kai Huang <[email protected]>

---
 arch/x86/include/asm/cpufeatures.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/include/asm/cpufeatures.h +1 -0
index 2519c6c801c9..31a7d1c0f204 100644
@@ -219,6 +219,7 @@
 /* Intel-defined CPU features, CPUID level 0x00000007:0 (ebx), word 9 */
 #define X86_FEATURE_FSGSBASE	( 9*32+ 0) /* {RD/WR}{FS/GS}BASE instructions*/
 #define X86_FEATURE_TSC_ADJUST	( 9*32+ 1) /* TSC adjustment MSR 0x3b */
+#define X86_FEATURE_SGX		( 9*32+ 2) /* Software Guard Extensions */
 #define X86_FEATURE_BMI1	( 9*32+ 3) /* 1st group bit manipulation extensions */
 #define X86_FEATURE_HLE		( 9*32+ 4) /* Hardware Lock Elision */
 #define X86_FEATURE_AVX2	( 9*32+ 5) /* AVX2 instructions */
Signature · 1 line
2.14.1
#5 lore raw

PATCH v5 3/11 x86: define the feature control MSR's SGX enable bit

In reply to Jarkko Sakkinen

From: Sean Christopherson <[email protected]>

---
 arch/x86/include/asm/msr-index.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/include/asm/msr-index.h +1 -0
index 17f5c12e1afd..b35cb98b5d60 100644
@@ -435,6 +435,7 @@
 #define FEATURE_CONTROL_LOCKED				(1<<0)
 #define FEATURE_CONTROL_VMXON_ENABLED_INSIDE_SMX	(1<<1)
 #define FEATURE_CONTROL_VMXON_ENABLED_OUTSIDE_SMX	(1<<2)
+#define FEATURE_CONTROL_SGX_ENABLE                      (1<<18)
 #define FEATURE_CONTROL_LMCE				(1<<20)
 
 #define MSR_IA32_APICBASE		0x0000001b
Signature · 1 line
2.14.1
#6 lore raw

Re: [PATCH v5 03/11] x86: define the feature control MSR's SGX enable bit

Quoted 2 lines · Jarkko Sakkinen

From: Sean Christopherson <[email protected]>

If you are at a loss regarding what more you could say beyond the single subject line, please refer to the git history of the file or subsystem in question. Several good examples for msr-index.h in the history.

Quoted 2 lines
Signature · 2 lines
Darren Hart
VMware Open Source Technology Center
#7 lore raw

PATCH v5 4/11 x86: define the feature control MSR's SGX launch control bit

In reply to Jarkko Sakkinen
---
 arch/x86/include/asm/cpufeatures.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/include/asm/cpufeatures.h +1 -0
index 31a7d1c0f204..43130f3c18a1 100644
@@ -298,6 +298,7 @@
 #define X86_FEATURE_AVX512_VPOPCNTDQ (16*32+14) /* POPCNT for vectors of DW/QW */
 #define X86_FEATURE_LA57	(16*32+16) /* 5-level page tables */
 #define X86_FEATURE_RDPID	(16*32+22) /* RDPID instruction */
+#define X86_FEATURE_SGX_LC	(16*32+30) /* supports SGX launch configuration */
 
 /* AMD-defined CPU features, CPUID level 0x80000007 (ebx), word 17 */
 #define X86_FEATURE_OVERFLOW_RECOV (17*32+0) /* MCA overflow recovery support */
Signature · 1 line
2.14.1
#8 lore raw

PATCH v5 5/11 x86: add SGX MSRs to msr-index.h

In reply to Jarkko Sakkinen

From: Haim Cohen <[email protected]>

These MSRs hold the SHA256 checksum of the currently configured launch enclave public key.

---
 arch/x86/include/asm/msr-index.h | 7 +++++++
 1 file changed, 7 insertions(+)
arch/x86/include/asm/msr-index.h +7 -0
index b35cb98b5d60..22e27d46d046 100644
@@ -436,6 +436,7 @@
 #define FEATURE_CONTROL_VMXON_ENABLED_INSIDE_SMX	(1<<1)
 #define FEATURE_CONTROL_VMXON_ENABLED_OUTSIDE_SMX	(1<<2)
 #define FEATURE_CONTROL_SGX_ENABLE                      (1<<18)
+#define FEATURE_CONTROL_SGX_LAUNCH_CONTROL_ENABLE	(1<<17)
 #define FEATURE_CONTROL_LMCE				(1<<20)
 
 #define MSR_IA32_APICBASE		0x0000001b
@@ -502,6 +503,12 @@
 #define PACKAGE_THERM_INT_LOW_ENABLE		(1 << 1)
 #define PACKAGE_THERM_INT_PLN_ENABLE		(1 << 24)
 
+/* Intel SGX MSRs */
+#define MSR_IA32_SGXLEPUBKEYHASH0	0x0000008C
+#define MSR_IA32_SGXLEPUBKEYHASH1	0x0000008D
+#define MSR_IA32_SGXLEPUBKEYHASH2	0x0000008E
+#define MSR_IA32_SGXLEPUBKEYHASH3	0x0000008F
+
 /* Thermal Thresholds Support */
 #define THERM_INT_THRESHOLD0_ENABLE    (1 << 15)
 #define THERM_SHIFT_THRESHOLD0        8
Signature · 1 line
2.14.1
#9 lore raw

PATCH v5 6/11 intel_sgx: driver for Intel Software Guard Extensions

In reply to Jarkko Sakkinen

Intel SGX is a set of CPU instructions that can be used by applications to set aside private regions of code and data. The code outside the enclave is disallowed to access the memory inside the enclave by the CPU access control.

SGX driver provides a ioctl API for loading and initializing enclaves. Address range for enclaves is reserved with mmap() and they are destroyed with munmap(). Enclave construction, measurement and initialization is done with the provided the ioctl API.

The driver implements also a swapper thread ksgxswapd for EPC pages backed by a private shmem file. Currently it has a limitation of not swapping VA pages but there is nothing preventing to implement it later on. Now it was scoped out in order to keep the implementation simple.

The parameter struct for SGX_IOC_ENCLAVE_INIT does not contain a parameter to supply a launch token. Generating and using tokens is best to be kept in the control of the kernel because it has direct binding to the IA32_SGXPUBKEYHASHx MSRs (a core must have MSRs set to the same value as the signer of token).

By giving user space any role in the launch process is a risk for introducing bottlenecks as kernel must exhibit behavior that user space launch daemon depends on, properietary risks (closed launch daemons on closed platforms) and stability risks as there would be division of semantics between user space and kernel.

---
 arch/x86/include/asm/sgx.h                      | 233 ++++++
 arch/x86/include/asm/sgx_arch.h                 | 268 +++++++
 arch/x86/include/uapi/asm/sgx.h                 | 138 ++++
 drivers/platform/x86/Kconfig                    |   2 +
 drivers/platform/x86/Makefile                   |   1 +
 drivers/platform/x86/intel_sgx/Kconfig          |  19 +
 drivers/platform/x86/intel_sgx/Makefile         |  13 +
 drivers/platform/x86/intel_sgx/sgx.h            | 254 ++++++
 drivers/platform/x86/intel_sgx/sgx_encl.c       | 988 ++++++++++++++++++++++++
 drivers/platform/x86/intel_sgx/sgx_ioctl.c      | 277 +++++++
 drivers/platform/x86/intel_sgx/sgx_main.c       | 410 ++++++++++
 drivers/platform/x86/intel_sgx/sgx_page_cache.c | 614 +++++++++++++++
 drivers/platform/x86/intel_sgx/sgx_util.c       | 372 +++++++++
 drivers/platform/x86/intel_sgx/sgx_vma.c        | 117 +++
 14 files changed, 3706 insertions(+)
 create mode 100644 arch/x86/include/asm/sgx.h
 create mode 100644 arch/x86/include/asm/sgx_arch.h
 create mode 100644 arch/x86/include/uapi/asm/sgx.h
 create mode 100644 drivers/platform/x86/intel_sgx/Kconfig
 create mode 100644 drivers/platform/x86/intel_sgx/Makefile
 create mode 100644 drivers/platform/x86/intel_sgx/sgx.h
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_encl.c
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_ioctl.c
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_main.c
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_page_cache.c
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_util.c
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_vma.c
arch/x86/include/asm/sgx.h new file +233 -0
new file mode 100644
index 000000000000..9c567c05a8bb
@@ -0,0 +1,233 @@
+/*
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright(c) 2016-2017 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * Contact Information:
+ * Jarkko Sakkinen <[email protected]>
+ * Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo
+ *
+ * BSD LICENSE
+ *
+ * Copyright(c) 2016-2017 Intel Corporation.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ *   * Redistributions of source code must retain the above copyright
+ *     notice, this list of conditions and the following disclaimer.
+ *   * Redistributions in binary form must reproduce the above copyright
+ *     notice, this list of conditions and the following disclaimer in
+ *     the documentation and/or other materials provided with the
+ *     distribution.
+ *   * Neither the name of Intel Corporation nor the names of its
+ *     contributors may be used to endorse or promote products derived
+ *     from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * Authors:
+ *
+ * Jarkko Sakkinen <[email protected]>
+ * Suresh Siddha <[email protected]>
+ */
+
+#ifndef _ASM_X86_SGX_H
+#define _ASM_X86_SGX_H
+
+#include <asm/sgx_arch.h>
+#include <asm/asm.h>
+#include <linux/bitops.h>
+#include <linux/err.h>
+#include <linux/types.h>
+
+#define SGX_CPUID 0x12
+
+enum sgx_cpuid {
+	SGX_CPUID_CAPABILITIES	= 0,
+	SGX_CPUID_ATTRIBUTES	= 1,
+	SGX_CPUID_EPC_BANKS	= 2,
+};
+
+enum sgx_commands {
+	ECREATE	= 0x0,
+	EADD	= 0x1,
+	EINIT	= 0x2,
+	EREMOVE	= 0x3,
+	EDGBRD	= 0x4,
+	EDGBWR	= 0x5,
+	EEXTEND	= 0x6,
+	ELDU	= 0x8,
+	EBLOCK	= 0x9,
+	EPA	= 0xA,
+	EWB	= 0xB,
+	ETRACK	= 0xC,
+	EAUG	= 0xD,
+	EMODPR	= 0xE,
+	EMODT	= 0xF,
+};
+
+#ifdef CONFIG_X86_64
+#define XAX "%%rax"
+#else
+#define XAX "%%eax"
+#endif
+
+#define __encls_ret(rax, rbx, rcx, rdx)			\
+	({						\
+	int ret;					\
+	asm volatile(					\
+	"1: .byte 0x0f, 0x01, 0xcf;\n\t"		\
+	"2:\n"						\
+	".section .fixup,\"ax\"\n"			\
+	"3: mov $-14,"XAX"\n"				\
+	"   jmp 2b\n"					\
+	".previous\n"					\
+	_ASM_EXTABLE(1b, 3b)				\
+	: "=a"(ret)					\
+	: "a"(rax), "b"(rbx), "c"(rcx), "d"(rdx)	\
+	: "memory");					\
+	ret;						\
+	})
+
+#define __encls(rax, rbx, rcx, rdx...)			\
+	({						\
+	int ret;					\
+	asm volatile(					\
+	"1: .byte 0x0f, 0x01, 0xcf;\n\t"		\
+	"   xor "XAX","XAX"\n"				\
+	"2:\n"						\
+	".section .fixup,\"ax\"\n"			\
+	"3: mov $-14,"XAX"\n"				\
+	"   jmp 2b\n"					\
+	".previous\n"					\
+	_ASM_EXTABLE(1b, 3b)				\
+	: "=a"(ret), "=b"(rbx), "=c"(rcx)		\
+	: "a"(rax), "b"(rbx), "c"(rcx), rdx		\
+	: "memory");					\
+	ret;						\
+	})
+
+static inline unsigned long __ecreate(struct sgx_pageinfo *pginfo, void *secs)
+{
+	return __encls(ECREATE, pginfo, secs, "d"(0));
+}
+
+static inline int __eextend(void *secs, void *epc)
+{
+	return __encls(EEXTEND, secs, epc, "d"(0));
+}
+
+static inline int __eadd(struct sgx_pageinfo *pginfo, void *epc)
+{
+	return __encls(EADD, pginfo, epc, "d"(0));
+}
+
+static inline int __einit(void *sigstruct, struct sgx_einittoken *einittoken,
+			  void *secs)
+{
+	return __encls_ret(EINIT, sigstruct, secs, einittoken);
+}
+
+static inline int __eremove(void *epc)
+{
+	unsigned long rbx = 0;
+	unsigned long rdx = 0;
+
+	return __encls_ret(EREMOVE, rbx, epc, rdx);
+}
+
+static inline int __edbgwr(void *epc, unsigned long *data)
+{
+	return __encls(EDGBWR, *data, epc, "d"(0));
+}
+
+static inline int __edbgrd(void *epc, unsigned long *data)
+{
+	unsigned long rbx = 0;
+	int ret;
+
+	ret = __encls(EDGBRD, rbx, epc, "d"(0));
+	if (!ret)
+		*(unsigned long *) data = rbx;
+
+	return ret;
+}
+
+static inline int __etrack(void *epc)
+{
+	unsigned long rbx = 0;
+	unsigned long rdx = 0;
+
+	return __encls_ret(ETRACK, rbx, epc, rdx);
+}
+
+static inline int __eldu(unsigned long rbx, unsigned long rcx,
+			 unsigned long rdx)
+{
+	return __encls_ret(ELDU, rbx, rcx, rdx);
+}
+
+static inline int __eblock(unsigned long rcx)
+{
+	unsigned long rbx = 0;
+	unsigned long rdx = 0;
+
+	return __encls_ret(EBLOCK, rbx, rcx, rdx);
+}
+
+static inline int __epa(void *epc)
+{
+	unsigned long rbx = SGX_PAGE_TYPE_VA;
+
+	return __encls(EPA, rbx, epc, "d"(0));
+}
+
+static inline int __ewb(struct sgx_pageinfo *pginfo, void *epc, void *va)
+{
+	return __encls_ret(EWB, pginfo, epc, va);
+}
+
+static inline int __eaug(struct sgx_pageinfo *pginfo, void *epc)
+{
+	return __encls(EAUG, pginfo, epc, "d"(0));
+}
+
+static inline int __emodpr(struct sgx_secinfo *secinfo, void *epc)
+{
+	unsigned long rdx = 0;
+
+	return __encls_ret(EMODPR, secinfo, epc, rdx);
+}
+
+static inline int __emodt(struct sgx_secinfo *secinfo, void *epc)
+{
+	unsigned long rdx = 0;
+
+	return __encls_ret(EMODT, secinfo, epc, rdx);
+}
+
+#endif /* _ASM_X86_SGX_H */
arch/x86/include/asm/sgx_arch.h new file +268 -0

268 lines not on this page · raw message

arch/x86/include/uapi/asm/sgx.h new file +138 -0

138 lines not on this page · raw message

drivers/platform/x86/Kconfig +2 -0
index 80b87954f6dd..51d429751235 100644
@@ -1139,6 +1139,8 @@ config SILEAD_DMI
 	  with the OS-image for the device. This option supplies the missing
 	  information. Enable this for x86 tablets with Silead touchscreens.
 
+source "drivers/platform/x86/intel_sgx/Kconfig"
+
 endif # X86_PLATFORM_DEVICES
 
 config PMC_ATOM
drivers/platform/x86/Makefile +1 -0
index 91cec1751461..b3a75ee512d2 100644
@@ -83,3 +83,4 @@ obj-$(CONFIG_PMC_ATOM)		+= pmc_atom.o
 obj-$(CONFIG_MLX_PLATFORM)	+= mlx-platform.o
 obj-$(CONFIG_MLX_CPLD_PLATFORM)	+= mlxcpld-hotplug.o
 obj-$(CONFIG_INTEL_TURBO_MAX_3) += intel_turbo_max_3.o
+obj-$(CONFIG_INTEL_SGX) += intel_sgx/
drivers/platform/x86/intel_sgx/Kconfig new file +19 -0
new file mode 100644
index 000000000000..5c7e61ecb524
@@ -0,0 +1,19 @@
+#
+# Intel SGX
+#
+
+config INTEL_SGX
+	tristate "Intel(R) SGX Driver"
+	default n
+	depends on X86_64 && CPU_SUP_INTEL
+	select MMU_NOTIFIER
+	---help---
+	Intel(R) SGX is a set of CPU instructions that can be used by
+	applications to set aside private regions of code and data.  The code
+	outside the enclave is disallowed to access the memory inside the
+	enclave by the CPU access control.
+
+	The firmware uses PRMRR registers to reserve an area of physical memory
+	called Enclave Page Cache (EPC). There is a hardware unit in the
+	processor called Memory Encryption Engine. The MEE encrypts and decrypts
+	the EPC pages as they enter and leave the processor package.
drivers/platform/x86/intel_sgx/Makefile new file +13 -0
new file mode 100644
index 000000000000..92af94668508
@@ -0,0 +1,13 @@
+#
+# Intel SGX
+#
+
+obj-$(CONFIG_INTEL_SGX) += intel_sgx.o
+
+intel_sgx-$(CONFIG_INTEL_SGX) += \
+	sgx_ioctl.o \
+	sgx_encl.o \
+	sgx_main.o \
+	sgx_page_cache.o \
+	sgx_util.o \
+	sgx_vma.o \
drivers/platform/x86/intel_sgx/sgx.h new file +254 -0

254 lines not on this page · raw message

drivers/platform/x86/intel_sgx/sgx_encl.c new file +988 -0

988 lines not on this page · raw message

drivers/platform/x86/intel_sgx/sgx_ioctl.c new file +277 -0

277 lines not on this page · raw message

drivers/platform/x86/intel_sgx/sgx_main.c new file +410 -0

410 lines not on this page · raw message

drivers/platform/x86/intel_sgx/sgx_page_cache.c new file +614 -0

388 lines not on this page · raw message

drivers/platform/x86/intel_sgx/sgx_util.c new file +372 -0

372 lines not on this page · raw message

drivers/platform/x86/intel_sgx/sgx_vma.c new file +117 -0

117 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.14.1

Part of the diff is not shown here. View the raw message.

#10 lore raw

Re: [PATCH v5 06/11] intel_sgx: driver for Intel Software Guard Extensions

drivers/platform/x86/intel_sgx/sgx_encl.c line 582

Quoted 2 lines · Jarkko Sakkinen
drivers/platform/x86/intel_sgx/sgx_encl.c new file +2 -0
+
+	secs_epc = sgx_alloc_page(0);

Use SGX_ALLOC_ATOMIC instead of 0 ?

drivers/platform/x86/intel_sgx/sgx_encl.c line 656

Quoted 10 lines
drivers/platform/x86/intel_sgx/sgx_encl.c new file +9 -0
+	if (IS_ERR(secs_epc)) {
+		ret = PTR_ERR(secs_epc);
+		goto out;
+	}
 
+out:
+	if (encl)
+		kref_put(&encl->refcount, sgx_encl_release);
+	return ret;
+}

Don't you need an sgx_free_page() somewhere here?

Signature · 2 lines
James Morris
<[email protected]>
#11 lore raw

Re: [PATCH v5 06/11] intel_sgx: driver for Intel Software Guard Extensions

Quoted 6 lines · James Morris

drivers/platform/x86/intel_sgx/sgx_encl.c line 582

Quoted 2 lines · Jarkko Sakkinen
drivers/platform/x86/intel_sgx/sgx_encl.c new file +2 -0
+
+	secs_epc = sgx_alloc_page(0);

Use SGX_ALLOC_ATOMIC instead of 0 ?

It is used in the #PF handler where it is not safe to swap pages in the call context. I.e. you cannot lock mmap_sem without potentially causing a deadlock.

Quoted 5 lines
Quoted 4 lines
drivers/platform/x86/intel_sgx/sgx_encl.c new file +4 -0
+	if (IS_ERR(secs_epc)) {
+		ret = PTR_ERR(secs_epc);
+		goto out;
+	}
Quoted 7 lines

drivers/platform/x86/intel_sgx/sgx_encl.c line 656

Quoted 5 lines
drivers/platform/x86/intel_sgx/sgx_encl.c new file +5 -0
+out:
+	if (encl)
+		kref_put(&encl->refcount, sgx_encl_release);
+	return ret;
+}

Don't you need an sgx_free_page() somewhere here?

It will get freed by sgx_encl_release().

Quoted 3 lines
Signature · 2 lines
James Morris
<[email protected]>

/Jarkko

#12 lore raw

Re: [PATCH v5 06/11] intel_sgx: driver for Intel Software Guard Extensions

Quoted 14 lines · Jarkko Sakkinen
Quoted 5 lines · James Morris
Quoted 4 lines
drivers/platform/x86/intel_sgx/sgx_encl.c new file +4 -0
+	if (IS_ERR(secs_epc)) {
+		ret = PTR_ERR(secs_epc);
+		goto out;
+	}
Quoted 7 lines

drivers/platform/x86/intel_sgx/sgx_encl.c line 656

Quoted 5 lines
drivers/platform/x86/intel_sgx/sgx_encl.c new file +5 -0
+out:
+	if (encl)
+		kref_put(&encl->refcount, sgx_encl_release);
+	return ret;
+}

Don't you need an sgx_free_page() somewhere here?

Quoted 1 line

It will get freed by sgx_encl_release().

The call flow was somewhat messy so I just moved sgx_encl_alloc() call outside of sgx_encl_create().

/Jarkko

#13 lore raw

Re: [intel-sgx-kernel-dev] [PATCH v5 06/11] intel_sgx: driver for Intel Software Guard Extensions

In reply to Jarkko Sakkinen
Quoted 21 lines · Jarkko Sakkinen

Intel SGX is a set of CPU instructions that can be used by applications to set aside private regions of code and data.  The code outside the enclave is disallowed to access the memory inside the enclave by the CPU access control.

SGX driver provides a ioctl API for loading and initializing enclaves. Address range for enclaves is reserved with mmap() and they are destroyed with munmap(). Enclave construction, measurement and initialization is done with the provided the ioctl API.

The driver implements also a swapper thread ksgxswapd for EPC pages backed by a private shmem file. Currently it has a limitation of not swapping VA pages but there is nothing preventing to implement it later on. Now it was scoped out in order to keep the implementation simple.

The parameter struct for SGX_IOC_ENCLAVE_INIT does not contain a parameter to supply a launch token. Generating and using tokens is best to be kept in the control of the kernel because it has direct binding to the IA32_SGXPUBKEYHASHx MSRs (a core must have MSRs set to the same value as the signer of token).

Quoted 3 lines

By giving user space any role in the launch process is a risk for introducing bottlenecks as kernel must exhibit behavior that user space launch daemon depends on

What do you mean by bottlenecks?  Assuming you're referring to performance bottlenecks, this statement is flat out false.  Moving the launch enclave into the kernel introduces performance bottlenecks, e.g. as implemented, a single LE services all EINIT requests and is protected by a mutex.  That is the very definition of a bottleneck.

The kernel will never be as performant as userspace when it comes to EINIT tokens because userspace can make informed decisions based on its usage model, e.g. number of LEs (or LE threads) to spawn, LE and token lifecycles, LE and token thread safety, etc...

Quoted 2 lines

, properietary risks (closed launch daemons on
closed platforms)

This justifies the need for the kernel to be able to generate launch tokens, but I don't think allowing userspace to also provide its own tokens adds any proprietary risks.

Quoted 3 lines

and stability risks as there would be division of
semantics between user space and kernel.

What exactly are the stability risks?  The token itself is architecturally defined and isn't fundamentally different than e.g. the sigstruct.  Writing the LE hash MSRs as needed, e.g. for userspace LEs, isn't difficult.

Quoted 2 lines

---

#14 lore raw

Re: [intel-sgx-kernel-dev] [PATCH v5 06/11] intel_sgx: driver for Intel Software Guard Extensions

Quoted 5 lines · Sean Christopherson

What do you mean by bottlenecks?  Assuming you're referring to performance bottlenecks, this statement is flat out false.  Moving the launch enclave into the kernel introduces performance bottlenecks, e.g. as implemented, a single LE services all EINIT requests and is protected by a mutex.  That is the very definition of a bottleneck.

I guess the text does not do a good job describing what I meant. Maybe I should refine it? Your argument about mutex is correct.

The use of "bottleneck" does not specifically refer to performance. I'm worried about splitting the tasks needed to launch an enclave between kernel and user space. It could become difficult to manage when more SGX features are added. That is what I was referring when I used the word "bottleneck".

I suppose you think I should refine the commit message?

About the perf bottleneck. Given that all the data is already in sgx_le_ctx the driver could for example have own LE process for every opened /dev/sgx. Is your comment also suggesting to refine this or could it be postponed?

The driver architecture already allows to scale this but it is not nearly as bad issue as the one Dave pointed out.

/Jarkko

#15 lore raw

Re: [intel-sgx-kernel-dev] [PATCH v5 06/11] intel_sgx: driver for Intel Software Guard Extensions

Quoted 18 lines · Jarkko Sakkinen
Quoted 8 lines · Sean Christopherson

What do you mean by bottlenecks?  Assuming you're referring to performance bottlenecks, this statement is flat out false.  Moving the launch enclave into
the kernel introduces performance bottlenecks, e.g. as implemented, a single LE
services all EINIT requests and is protected by a mutex.  That is the very definition of a bottleneck.

I guess the text does not do a good job describing what I meant. Maybe I should refine it? Your argument about mutex is correct.

The use of "bottleneck" does not specifically refer to performance. I'm worried about splitting the tasks needed to launch an enclave between kernel and user space. It could become difficult to manage when more SGX features are added. That is what I was referring when I used the word "bottleneck".

Quoted 6 lines

I suppose you think I should refine the commit message?

About the perf bottleneck. Given that all the data is already in sgx_le_ctx the driver could for example have own LE process for every opened /dev/sgx. Is your comment also suggesting to refine this or could it be postponed?

More that I don't understand why the driver doesn't allow userspace to provide an EINIT token, and reciprocally, doesn't provide the token back to userspace. IMO, the act of generating an EINIT token is orthogonal to deciding whether or not to run the enclave.  Running code in a kernel-owned enclave is not specific to SGX, e.g. paranoid kernels could run other sensitive tasks in an enclave. Being forced to run an enclave to generate an EINIT token is an unfortunate speed bump that exists purely because hardware doesn't provide the option to disable launch control entirely.

In other words, accepting a token via the IOCTL doesn't mean the driver has to use it, e.g. it can always ignore the token, enforce periodic reverification, check that the token was created by the driver, etc...  And using the token doesn't preclude the driver from re-running its verification checks outside of the launch enclave.

Quoted 8 lines

The driver architecture already allows to scale this but it is not nearly as bad issue as the one Dave pointed out.

/Jarkko
_______________________________________________
intel-sgx-kernel-dev mailing list
[email protected]
https://lists.01.org/mailman/listinfo/intel-sgx-kernel-dev

#16 lore raw

RE: [intel-sgx-kernel-dev] [PATCH v5 06/11] intel_sgx: driver for Intel Software Guard Extensions

Quoted 35 lines · Sean Christopherson
Quoted 18 lines · Jarkko Sakkinen
Quoted 8 lines · Sean Christopherson

What do you mean by bottlenecks? Assuming you're referring to performance bottlenecks, this statement is flat out false. Moving the launch enclave into
the kernel introduces performance bottlenecks, e.g. as implemented, a single LE
services all EINIT requests and is protected by a mutex. That is the very definition of a bottleneck.

I guess the text does not do a good job describing what I meant. Maybe I should refine it? Your argument about mutex is correct.

The use of "bottleneck" does not specifically refer to performance. I'm worried about splitting the tasks needed to launch an enclave between kernel and user space. It could become difficult to manage when more SGX features are added. That is what I was referring when I used the word "bottleneck".

Quoted 6 lines

I suppose you think I should refine the commit message?

About the perf bottleneck. Given that all the data is already in sgx_le_ctx the driver could for example have own LE process for every opened /dev/sgx. Is your comment also suggesting to refine this or could it be postponed?

More that I don't understand why the driver doesn't allow userspace to provide an EINIT token, and reciprocally, doesn't provide the token back to userspace. IMO, the act of generating an EINIT token is orthogonal to deciding whether or not to run the enclave. Running code in a kernel-owned enclave is not specific to SGX, e.g. paranoid kernels could run other sensitive tasks in an enclave. Being forced to run an enclave to generate an EINIT token is an unfortunate speed bump that exists purely because hardware doesn't provide the option to disable launch control entirely.

Quoted 5 lines

In other words, accepting a token via the IOCTL doesn't mean the driver has to use it, e.g. it can always ignore the token, enforce periodic reverification, check that the token was created by the driver, etc... And using the token doesn't preclude the driver from re-running its verification checks outside of the launch enclave.

Resurrecting this thread now that I have a system with launch control and have been able to measure the performance impact...

Regenerating the EINIT token every time adds somewhere in the vicinity of ~5% overhead to creating an enclave, versus generating a token once and reusing it in each EINIT call. This isn't a huge issue since real world usage models likely won't be re-launching enclaves at a high rate, but it is measurable.

On top of my other arguments, the key of the token's signer must match the current value in the LE hash MSRs, so except for future theoretical scenarios where we want to "revoke" an existing token, the only way we can end up with a token we don't trust is if the kernel launch enclave already screwed up or userspace has access to the LE's private key.

#17 lore raw

Re: [intel-sgx-kernel-dev] [PATCH v5 06/11] intel_sgx: driver for Intel Software Guard Extensions

Quoted 8 lines · Sean J

Resurrecting this thread now that I have a system with launch control and have been able to measure the performance impact...

Regenerating the EINIT token every time adds somewhere in the vicinity of ~5% overhead to creating an enclave, versus generating a token once and reusing it in each EINIT call. This isn't a huge issue since real world usage models likely won't be re-launching enclaves at a high rate, but it is measurable.

We can cache tokens in future in the kernel space, can't we?

/Jarkko

#18 lore raw

RE: [intel-sgx-kernel-dev] [PATCH v5 06/11] intel_sgx: driver for Intel Software Guard Extensions

Quoted 10 lines · Jarkko Sakkinen
Quoted 8 lines

Resurrecting this thread now that I have a system with launch control and have been able to measure the performance impact...

Regenerating the EINIT token every time adds somewhere in the vicinity of ~5% overhead to creating an enclave, versus generating a token once and reusing it in each EINIT call. This isn't a huge issue since real world usage models likely won't be re-launching enclaves at a high rate, but it is measurable.

We can cache tokens in future in the kernel space, can't we?

Yes, but why? Deferring to userspace is less complex and likely more performant.

Tokens are large enough that there would need to be some form of limit on the number of tokens, which brings up questions about how to account tokens, the cache eviction scheme, whether or not the size of the cache should be controllable from userspace, etc...

Userspace caching can likely provide better performance because the user/application knows the usage model and life expectancy of its tokens, i.e. userspace can make informed decisions about when to discard a token, how much memory to dedicate to caching tokens, etc... And in the case of VMs, userspace can reuse tokens across reboots (of the VM), e.g. by saving tokens to disk.

#19 lore raw

Re: [intel-sgx-kernel-dev] [PATCH v5 06/11] intel_sgx: driver for Intel Software Guard Extensions

Quoted 4 lines · Sean J
Quoted 1 line

We can cache tokens in future in the kernel space, can't we?

Yes, but why? Deferring to userspace is less complex and likely more performant.

That's quite strong argument especially if you are making that for systems running multiple independent workloads and not just a single application.

Quoted 4 lines

Tokens are large enough that there would need to be some form of limit on the number of tokens, which brings up questions about how to account tokens, the cache eviction scheme, whether or not the size of the cache should be controllable from userspace, etc...

Leaving caching decisions to the kernel also gives more freedoms to do global decisions.

Quoted 6 lines

Userspace caching can likely provide better performance because the user/application knows the usage model and life expectancy of its tokens, i.e. userspace can make informed decisions about when to discard a token, how much memory to dedicate to caching tokens, etc... And in the case of VMs, userspace can reuse tokens across reboots (of the VM), e.g. by saving tokens to disk.

I'm not really convinced that your argument is sound if you consider the whole range of x86 systems that can run enclaves especially if the system is running multiple irrelated applications.

And you are ignoring everything else but the performance, which is does not make any sense. The current design governs the Linux kernel to have the ultimate power, which enclaves to run with the minimized proprietary risk. I think that is something worth of emphasizing too.

Whether the token caching is left to kernel or user space will most definitely introduce some non-trivial performance problems to solve with some unexpected workloads that we cannot imagine right now. That's why the governance should be the driver. Not the performance. Those issues can and must be sorted out in any case.

/Jarkko

#20 lore raw

RE: [intel-sgx-kernel-dev] [PATCH v5 06/11] intel_sgx: driver for Intel Software Guard Extensions

Quoted 25 lines · On Tuesday, December 19, 2017 Jarkko Sakkinen wrote:
Quoted 4 lines · Sean J
Quoted 1 line

We can cache tokens in future in the kernel space, can't we?

Yes, but why? Deferring to userspace is less complex and likely more performant.

That's quite strong argument especially if you are making that for systems running multiple independent workloads and not just a single application.

Quoted 4 lines

Tokens are large enough that there would need to be some form of limit on the number of tokens, which brings up questions about how to account tokens, the cache eviction scheme, whether or not the size of the cache should be controllable from userspace, etc...

Leaving caching decisions to the kernel also gives more freedoms to do global decisions.

Quoted 6 lines

Userspace caching can likely provide better performance because the user/application knows the usage model and life expectancy of its tokens, i.e. userspace can make informed decisions about when to discard a token, how much memory to dedicate to caching tokens, etc... And in the case of VMs, userspace can reuse tokens across reboots (of the VM), e.g. by saving tokens to disk.

Quoted 8 lines

I'm not really convinced that your argument is sound if you consider the whole range of x86 systems that can run enclaves especially if the system is running multiple irrelated applications.

And you are ignoring everything else but the performance, which is does not make any sense. The current design governs the Linux kernel to have the ultimate power, which enclaves to run with the minimized proprietary risk. I think that is something worth of emphasizing too.

Exposing the token generated by the in-kernel LE doesn't affect the kernel's power in the slightest, e.g. the kernel doesn't need a LE to refuse to run an enclave and a privileged user can always load an out-of-tree driver if they really want to circumvent the kernel's policies, which is probably easier than stealing the LE's private key.

Quoted 7 lines

Whether the token caching is left to kernel or user space will most definitely introduce some non-trivial performance problems to solve with some unexpected workloads that we cannot imagine right now. That's why the governance should be the driver. Not the performance. Those issues can and must be sorted out in any case.

#21 lore raw

Re: [intel-sgx-kernel-dev] [PATCH v5 06/11] intel_sgx: driver for Intel Software Guard Extensions

Quoted 5 lines · Sean J

Exposing the token generated by the in-kernel LE doesn't affect the kernel's power in the slightest, e.g. the kernel doesn't need a LE to refuse to run an enclave and a privileged user can always load an out-of-tree driver if they really want to circumvent the kernel's policies, which is probably easier than stealing the LE's private key.

If the MSRs are read-only, kernel does need an LE in order to launch enclaves if it only has the SIGSTRUCT.

User with abilities to load out-of-tree driver or otherwise modify the running kernel code does not really work as an argument in to any direction.

/Jarkko

#22 lore raw

PATCH v5 7/11 intel_sgx: ptrace() support

In reply to Jarkko Sakkinen

Implemented VMA callbacks in order to ptrace() debug enclaves.

---
 drivers/platform/x86/intel_sgx/sgx_vma.c | 115 +++++++++++++++++++++++++++++++
 1 file changed, 115 insertions(+)
drivers/platform/x86/intel_sgx/sgx_vma.c +115 -0
index 2b714daa6eb7..e447b5a9bb96 100644
@@ -110,8 +110,123 @@ static int sgx_vma_fault(struct vm_fault *vmf)
 		return VM_FAULT_SIGBUS;
 }
 
+static inline int sgx_vma_access_word(struct sgx_encl *encl,
+				      unsigned long addr,
+				      void *buf,
+				      int len,
+				      int write,
+				      struct sgx_encl_page *encl_page,
+				      int i)
+{
+	char data[sizeof(unsigned long)];
+	int align, cnt, offset;
+	void *vaddr;
+	int ret;
+
+	offset = ((addr + i) & (PAGE_SIZE - 1)) & ~(sizeof(unsigned long) - 1);
+	align = (addr + i) & (sizeof(unsigned long) - 1);
+	cnt = sizeof(unsigned long) - align;
+	cnt = min(cnt, len - i);
+
+	if (write) {
+		if (encl_page->flags & SGX_ENCL_PAGE_TCS &&
+		    (offset < 8 || (offset + (len - i)) > 16))
+			return -ECANCELED;
+
+		if (align || (cnt != sizeof(unsigned long))) {
+			vaddr = sgx_get_page(encl_page->epc_page);
+			ret = __edbgrd((void *)((unsigned long)vaddr + offset),
+				       (unsigned long *)data);
+			sgx_put_page(vaddr);
+			if (ret) {
+				sgx_dbg(encl, "EDBGRD returned %d\n", ret);
+				return -EFAULT;
+			}
+		}
+
+		memcpy(data + align, buf + i, cnt);
+		vaddr = sgx_get_page(encl_page->epc_page);
+		ret = __edbgwr((void *)((unsigned long)vaddr + offset),
+			       (unsigned long *)data);
+		sgx_put_page(vaddr);
+		if (ret) {
+			sgx_dbg(encl, "EDBGWR returned %d\n", ret);
+			return -EFAULT;
+		}
+	} else {
+		if (encl_page->flags & SGX_ENCL_PAGE_TCS &&
+		    (offset + (len - i)) > 72)
+			return -ECANCELED;
+
+		vaddr = sgx_get_page(encl_page->epc_page);
+		ret = __edbgrd((void *)((unsigned long)vaddr + offset),
+			       (unsigned long *)data);
+		sgx_put_page(vaddr);
+		if (ret) {
+			sgx_dbg(encl, "EDBGRD returned %d\n", ret);
+			return -EFAULT;
+		}
+
+		memcpy(buf + i, data + align, cnt);
+	}
+
+	return cnt;
+}
+
+static int sgx_vma_access(struct vm_area_struct *vma, unsigned long addr,
+			  void *buf, int len, int write)
+{
+	struct sgx_encl *encl = vma->vm_private_data;
+	struct sgx_encl_page *entry = NULL;
+	const char *op_str = write ? "EDBGWR" : "EDBGRD";
+	int ret = 0;
+	int i;
+
+	/* If process was forked, VMA is still there but vm_private_data is set
+	 * to NULL.
+	 */
+	if (!encl)
+		return -EFAULT;
+
+	if (!(encl->flags & SGX_ENCL_DEBUG) ||
+	    !(encl->flags & SGX_ENCL_INITIALIZED) ||
+	    (encl->flags & SGX_ENCL_DEAD))
+		return -EFAULT;
+
+	sgx_dbg(encl, "%s addr=0x%lx, len=%d\n", op_str, addr, len);
+
+	for (i = 0; i < len; i += ret) {
+		if (!entry || !((addr + i) & (PAGE_SIZE - 1))) {
+			if (entry)
+				entry->flags &= ~SGX_ENCL_PAGE_RESERVED;
+
+			entry = sgx_fault_page(vma, (addr + i) & PAGE_MASK,
+					       SGX_FAULT_RESERVE);
+			if (IS_ERR(entry)) {
+				ret = PTR_ERR(entry);
+				entry = NULL;
+				break;
+			}
+		}
+
+		/* No locks are needed because used fields are immutable after
+		 * initialization.
+		 */
+		ret = sgx_vma_access_word(encl, addr, buf, len, write,
+					  entry, i);
+		if (ret < 0)
+			break;
+	}
+
+	if (entry)
+		entry->flags &= ~SGX_ENCL_PAGE_RESERVED;
+
+	return (ret < 0 && ret != -ECANCELED) ? ret : i;
+}
+
 const struct vm_operations_struct sgx_vm_ops = {
 	.close = sgx_vma_close,
 	.open = sgx_vma_open,
 	.fault = sgx_vma_fault,
+	.access = sgx_vma_access,
 };
Signature · 1 line
2.14.1
#23 lore raw

Re: [PATCH v5 07/11] intel_sgx: ptrace() support

Quoted 1 line · Jarkko Sakkinen

Implemented VMA callbacks in order to ptrace() debug enclaves.

The amount of information in this changelog is really overwhelming.

Please explain WHY you need that and HOW its supposed to work.

drivers/platform/x86/intel_sgx/sgx_vma.c static int sgx_vma_fault(struct vm_fault *vmf) · line 119

Quoted 7 lines
drivers/platform/x86/intel_sgx/sgx_vma.c +7 -0
+static inline int sgx_vma_access_word(struct sgx_encl *encl,
+				      unsigned long addr,
+				      void *buf,
+				      int len,
+				      int write,
+				      struct sgx_encl_page *encl_page,
+				      int i)

Can you find a way to waste more lines for a function declaration?

Aside of that using 'i' as a argument is just broken. Arguments should be self explaining as far as possible and sure not using names which are commonly used in code for iterators etc.

drivers/platform/x86/intel_sgx/sgx_vma.c static int sgx_vma_fault(struct vm_fault *vmf) · line 127

Quoted 8 lines
drivers/platform/x86/intel_sgx/sgx_vma.c +8 -0
+{
+	char data[sizeof(unsigned long)];
+	int align, cnt, offset;
+	void *vaddr;
+	int ret;
+
+	offset = ((addr + i) & (PAGE_SIZE - 1)) & ~(sizeof(unsigned long) - 1);
+	align = (addr + i) & (sizeof(unsigned long) - 1);

The kernel has macros for this kind of operations.

drivers/platform/x86/intel_sgx/sgx_vma.c static int sgx_vma_fault(struct vm_fault *vmf) · line 133

Quoted 6 lines
drivers/platform/x86/intel_sgx/sgx_vma.c +6 -0
+	cnt = sizeof(unsigned long) - align;
+	cnt = min(cnt, len - i);
+
+	if (write) {
+		if (encl_page->flags & SGX_ENCL_PAGE_TCS &&
+		    (offset < 8 || (offset + (len - i)) > 16))

Hard coded numbers which are nowhere explained are a nono. Please use proper defines and explain the meaning so the code becomes understandable.

drivers/platform/x86/intel_sgx/sgx_vma.c static int sgx_vma_fault(struct vm_fault *vmf) · line 136

Quoted 3 lines
drivers/platform/x86/intel_sgx/sgx_vma.c +3 -0
+			return -ECANCELED;
+
+		if (align || (cnt != sizeof(unsigned long))) {

What the heck is this doing? The complete lack of any comment in this code makes review impossible.

drivers/platform/x86/intel_sgx/sgx_vma.c static int sgx_vma_fault(struct vm_fault *vmf) · line 139

Quoted 3 lines
drivers/platform/x86/intel_sgx/sgx_vma.c +3 -0
+			vaddr = sgx_get_page(encl_page->epc_page);
+			ret = __edbgrd((void *)((unsigned long)vaddr + offset),
+				       (unsigned long *)data);

This typecast mess all over the place is just wrong. You either use the wrong variable types or your functions have the wrong parameter type.

Thanks,

tglx

#24 lore raw

Re: [PATCH v5 07/11] intel_sgx: ptrace() support

Quoted 50 lines · Thomas Gleixner
Quoted 1 line · Jarkko Sakkinen

Implemented VMA callbacks in order to ptrace() debug enclaves.

The amount of information in this changelog is really overwhelming.

Please explain WHY you need that and HOW its supposed to work.

drivers/platform/x86/intel_sgx/sgx_vma.c static int sgx_vma_fault(struct vm_fault *vmf) · line 119

Quoted 7 lines
drivers/platform/x86/intel_sgx/sgx_vma.c +7 -0
+static inline int sgx_vma_access_word(struct sgx_encl *encl,
+				      unsigned long addr,
+				      void *buf,
+				      int len,
+				      int write,
+				      struct sgx_encl_page *encl_page,
+				      int i)

Can you find a way to waste more lines for a function declaration?

Aside of that using 'i' as a argument is just broken. Arguments should be self explaining as far as possible and sure not using names which are commonly used in code for iterators etc.

drivers/platform/x86/intel_sgx/sgx_vma.c static int sgx_vma_fault(struct vm_fault *vmf) · line 127

Quoted 8 lines
drivers/platform/x86/intel_sgx/sgx_vma.c +8 -0
+{
+	char data[sizeof(unsigned long)];
+	int align, cnt, offset;
+	void *vaddr;
+	int ret;
+
+	offset = ((addr + i) & (PAGE_SIZE - 1)) & ~(sizeof(unsigned long) - 1);
+	align = (addr + i) & (sizeof(unsigned long) - 1);

The kernel has macros for this kind of operations.

drivers/platform/x86/intel_sgx/sgx_vma.c static int sgx_vma_fault(struct vm_fault *vmf) · line 133

Quoted 6 lines
drivers/platform/x86/intel_sgx/sgx_vma.c +6 -0
+	cnt = sizeof(unsigned long) - align;
+	cnt = min(cnt, len - i);
+
+	if (write) {
+		if (encl_page->flags & SGX_ENCL_PAGE_TCS &&
+		    (offset < 8 || (offset + (len - i)) > 16))

Hard coded numbers which are nowhere explained are a nono. Please use proper defines and explain the meaning so the code becomes understandable.

drivers/platform/x86/intel_sgx/sgx_vma.c static int sgx_vma_fault(struct vm_fault *vmf) · line 136

Quoted 3 lines
drivers/platform/x86/intel_sgx/sgx_vma.c +3 -0
+			return -ECANCELED;
+
+		if (align || (cnt != sizeof(unsigned long))) {

What the heck is this doing? The complete lack of any comment in this code makes review impossible.

Quoted 6 lines

drivers/platform/x86/intel_sgx/sgx_vma.c static int sgx_vma_fault(struct vm_fault *vmf) · line 139

Quoted 3 lines
drivers/platform/x86/intel_sgx/sgx_vma.c +3 -0
+			vaddr = sgx_get_page(encl_page->epc_page);
+			ret = __edbgrd((void *)((unsigned long)vaddr + offset),
+				       (unsigned long *)data);

This typecast mess all over the place is just wrong. You either use the wrong variable types or your functions have the wrong parameter type.

Thank you for the feedback. I agree with all your comments.

I'll also split the function into sgx_vma_read_word() and sgx_vma_write_word(). Makes the code somewhat cleaner and nicer to trace.

I have to admit that I have overlooked this commit when preparing the patch set. Even though I've written code myself, it's been a while and I had hard time to get a grip what is going on when I read your response, which makes your feedback even more valid :-)

/Jarkko

#25 lore raw

PATCH v5 8/11 intel_sgx: in-kernel launch enclave

In reply to Jarkko Sakkinen

This commits implements the in-kernel launch enclave. It is wrapped into a user space program that reads SIGSTRUCT instances from stdin and outputs launch tokens to stdout.

The commit also adds enclave signing tool that is used by kbuild to measure and sign the launch enclave.

CONFIG_INTEL_SGX_SIGNING_KEY points to a PEM-file for the 3072-bit RSA key that is used as the LE public key pair. The default location is:

drivers/platform/x86/intel_sgx/intel_sgx_signing_key.pem

If the default key does not exist kbuild will generate a random key and place it to this location. KBUILD_SGX_SIGN_PIN can be used to specify the passphrase for the LE public key.

TinyCrypt (https://github.com/01org/tinycrypt) is used as AES implementation, which is not timing resistant. Eventually this needs to be replaced with AES-NI based implementation that could be either

- re-use existing AES-NI code in the kernel
- have its own hand written code

---
 drivers/platform/x86/intel_sgx/Kconfig             |  13 +
 drivers/platform/x86/intel_sgx/Makefile            |  18 +
 drivers/platform/x86/intel_sgx/le/Makefile         |  26 +
 drivers/platform/x86/intel_sgx/le/enclave/Makefile |  46 ++
 .../x86/intel_sgx/le/enclave/aes_encrypt.c         | 191 ++++++++
 .../platform/x86/intel_sgx/le/enclave/cmac_mode.c  | 254 ++++++++++
 .../x86/intel_sgx/le/enclave/encl_bootstrap.S      | 163 +++++++
 .../intel_sgx/le/enclave/include/tinycrypt/aes.h   | 133 +++++
 .../le/enclave/include/tinycrypt/cmac_mode.h       | 194 ++++++++
 .../le/enclave/include/tinycrypt/constants.h       |  59 +++
 .../intel_sgx/le/enclave/include/tinycrypt/utils.h |  95 ++++
 drivers/platform/x86/intel_sgx/le/enclave/main.c   | 203 ++++++++
 .../platform/x86/intel_sgx/le/enclave/sgx_le.lds   |  28 ++
 .../platform/x86/intel_sgx/le/enclave/sgxsign.c    | 538 +++++++++++++++++++++
 drivers/platform/x86/intel_sgx/le/enclave/utils.c  |  78 +++
 drivers/platform/x86/intel_sgx/le/entry.S          | 117 +++++
 .../platform/x86/intel_sgx/le/include/sgx_asm.h    |  64 +++
 .../platform/x86/intel_sgx/le/include/sgx_encl.h   | 110 +++++
 drivers/platform/x86/intel_sgx/le/main.c           | 214 ++++++++
 drivers/platform/x86/intel_sgx/le/sgx_le_piggy.S   |  15 +
 .../platform/x86/intel_sgx/sgx_le_proxy_piggy.S    |  11 +
 21 files changed, 2570 insertions(+)
 create mode 100644 drivers/platform/x86/intel_sgx/le/Makefile
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/Makefile
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/aes_encrypt.c
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/cmac_mode.c
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/encl_bootstrap.S
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/aes.h
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/cmac_mode.h
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/constants.h
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/utils.h
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/main.c
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/sgx_le.lds
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/sgxsign.c
 create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/utils.c
 create mode 100644 drivers/platform/x86/intel_sgx/le/entry.S
 create mode 100644 drivers/platform/x86/intel_sgx/le/include/sgx_asm.h
 create mode 100644 drivers/platform/x86/intel_sgx/le/include/sgx_encl.h
 create mode 100644 drivers/platform/x86/intel_sgx/le/main.c
 create mode 100644 drivers/platform/x86/intel_sgx/le/sgx_le_piggy.S
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_le_proxy_piggy.S
drivers/platform/x86/intel_sgx/Kconfig +13 -0
index 5c7e61ecb524..9c445e9f89fa 100644
@@ -2,6 +2,8 @@
 # Intel SGX
 #
 
+menu "Intel SGX"
+
 config INTEL_SGX
 	tristate "Intel(R) SGX Driver"
 	default n
@@ -17,3 +19,14 @@ config INTEL_SGX
 	called Enclave Page Cache (EPC). There is a hardware unit in the
 	processor called Memory Encryption Engine. The MEE encrypts and decrypts
 	the EPC pages as they enter and leave the processor package.
+
+
+config INTEL_SGX_SIGNING_KEY
+	string "Path to the Intel SGX LE signing key"
+	default "drivers/platform/x86/intel_sgx/intel_sgx_signing_key.pem"
+	depends on INTEL_SGX
+	---help---
+	Provide a path to a 3072-bit RSA private key that will be used to
+	sign the launch enclave.
+
+endmenu
drivers/platform/x86/intel_sgx/Makefile +18 -0
index 92af94668508..39c0a5ad242c 100644
@@ -11,3 +11,21 @@ intel_sgx-$(CONFIG_INTEL_SGX) += \
 	sgx_page_cache.o \
 	sgx_util.o \
 	sgx_vma.o \
+	sgx_le_proxy_piggy.o
+
+$(eval $(call config_filename,INTEL_SGX_SIGNING_KEY))
+
+INTEL_SGX_SIGNING_KEY_PATH := \
+	$(INTEL_SGX_SIGNING_KEY_SRCPREFIX)$(INTEL_SGX_SIGNING_KEY_FILENAME)
+
+ifeq ($(CONFIG_INTEL_SGX_SIGNING_KEY),"drivers/platform/x86/intel_sgx/intel_sgx_signing_key.pem")
+$(INTEL_SGX_SIGNING_KEY_PATH):
+	$(Q)openssl genrsa -3 -out $(INTEL_SGX_SIGNING_KEY_PATH) 3072
+endif
+
+$(obj)/sgx_le_proxy_piggy.o: $(INTEL_SGX_SIGNING_KEY_PATH) \
+			     $(obj)/le/sgx_le_proxy
+$(obj)/le/sgx_le_proxy: FORCE
+	$(Q)$(MAKE) $(build)=$(obj)/le $@
+
+export INTEL_SGX_SIGNING_KEY_PATH
drivers/platform/x86/intel_sgx/le/Makefile new file +26 -0
new file mode 100644
index 000000000000..50fe1849f6ca
@@ -0,0 +1,26 @@
+KASAN_SANITIZE := n
+OBJECT_FILES_NON_STANDARD := y
+KCOV_INSTRUMENT := n
+KBUILD_CFLAGS := -Wall -Werror -static -nostdlib -nostartfiles -I$(obj)/include
+KBUILD_AFLAGS += -I$(obj)/include
+
+subdir- := enclave
+
+always := sgx_le_proxy
+clean-files := sgx_le_proxy
+
+#
+# sgx_le_proxy
+#
+
+sgx_le_proxy-y += main.o entry.o sgx_le_piggy.o
+targets += $(sgx_le_proxy-y)
+SGX_LE_PROXY_OBJS = $(addprefix $(obj)/,$(sgx_le_proxy-y))
+
+$(obj)/sgx_le_piggy.o: $(obj)/enclave/sgx_le.bin $(obj)/enclave/sgx_le.ss
+$(obj)/enclave/sgx_le.bin $(obj)/enclave/sgx_le.ss: FORCE
+	$(Q)$(MAKE) $(build)=$(obj)/enclave $@
+
+targets += sgx_le_proxy
+$(obj)/sgx_le_proxy: $(SGX_LE_PROXY_OBJS)
+	$(call if_changed,ld)
drivers/platform/x86/intel_sgx/le/enclave/Makefile new file +46 -0

46 lines not on this page · raw message

drivers/platform/x86/intel_sgx/le/enclave/aes_encrypt.c new file +191 -0

191 lines not on this page · raw message

drivers/platform/x86/intel_sgx/le/enclave/cmac_mode.c new file +254 -0

254 lines not on this page · raw message

drivers/platform/x86/intel_sgx/le/enclave/encl_bootstrap.S new file +163 -0

163 lines not on this page · raw message

drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/aes.h new file +133 -0

133 lines not on this page · raw message

drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/cmac_mode.h new file +194 -0

194 lines not on this page · raw message

drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/constants.h new file +59 -0

59 lines not on this page · raw message

drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/utils.h new file +95 -0

95 lines not on this page · raw message

drivers/platform/x86/intel_sgx/le/enclave/main.c new file +203 -0

203 lines not on this page · raw message

drivers/platform/x86/intel_sgx/le/enclave/sgx_le.lds new file +28 -0
new file mode 100644
index 000000000000..56b694c15ebe
@@ -0,0 +1,28 @@
+OUTPUT_FORMAT(elf64-x86-64)
+
+SECTIONS
+{
+	. = 0;
+	.tcs : {
+		*(.tcs*)
+	}
+
+	. = ALIGN(4096);
+	.text : {
+		*(.text*)
+		*(.rodata*)
+	}
+
+	. = ALIGN(4096);
+	.data : {
+		*(.data*)
+	}
+
+	/DISCARD/ : {
+		*(.data*)
+		*(.comment*)
+		*(.note*)
+		*(.debug*)
+		*(.eh_frame*)
+	}
+}
drivers/platform/x86/intel_sgx/le/enclave/sgxsign.c new file +538 -0

538 lines not on this page · raw message

drivers/platform/x86/intel_sgx/le/enclave/utils.c new file +78 -0

78 lines not on this page · raw message

drivers/platform/x86/intel_sgx/le/entry.S new file +117 -0

117 lines not on this page · raw message

drivers/platform/x86/intel_sgx/le/include/sgx_asm.h new file +64 -0

64 lines not on this page · raw message

drivers/platform/x86/intel_sgx/le/include/sgx_encl.h new file +110 -0

110 lines not on this page · raw message

drivers/platform/x86/intel_sgx/le/main.c new file +214 -0

214 lines not on this page · raw message

drivers/platform/x86/intel_sgx/le/sgx_le_piggy.S new file +15 -0
new file mode 100644
index 000000000000..e5631e8a755d
@@ -0,0 +1,15 @@
+#include <linux/linkage.h>
+#include <asm/page_types.h>
+
+	.section ".rodata","a"
+	.balign PAGE_SIZE
+
+GLOBAL(sgx_le_blob)
+	.incbin	"drivers/platform/x86/intel_sgx/le/enclave/sgx_le.bin"
+END(sgx_le_blob)
+
+GLOBAL(sgx_le_blob_end);
+
+GLOBAL(sgx_le_ss)
+	.incbin	"drivers/platform/x86/intel_sgx/le/enclave/sgx_le.ss"
+END(sgx_le_ss)
drivers/platform/x86/intel_sgx/sgx_le_proxy_piggy.S new file +11 -0
new file mode 100644
index 000000000000..faced8a9a75a
@@ -0,0 +1,11 @@
+#include <linux/linkage.h>
+#include <asm/export.h>
+#include <asm/page_types.h>
+
+	.section ".rodata","a"
+
+GLOBAL(sgx_le_proxy)
+	.incbin	"drivers/platform/x86/intel_sgx/le/sgx_le_proxy"
+END(sgx_le_proxy)
+
+GLOBAL(sgx_le_proxy_end)
Signature · 1 line
2.14.1
#26 lore raw

Re: [intel-sgx-kernel-dev] [PATCH v5 08/11] intel_sgx: in-kernel launch enclave

Quoted 11 lines · Jarkko Sakkinen

This commits implements the in-kernel launch enclave. It is wrapped into a user space program that reads SIGSTRUCT instances from stdin and outputs launch tokens to stdout.

The commit also adds enclave signing tool that is used by kbuild to measure and sign the launch enclave.

CONFIG_INTEL_SGX_SIGNING_KEY points to a PEM-file for the 3072-bit RSA key that is used as the LE public key pair. The default location is:

  drivers/platform/x86/intel_sgx/intel_sgx_signing_key.pem

Unless there is some conflict you are worried about, "signing_key.pem" is preferable as the default name so that the key is ignored via the top-level .gitignore.  The intel_sgx dir should have also a .gitignore to exclude the other LE related output files:

drivers/platform/x86/intel_sgx/le/enclave/sgx_le.ss drivers/platform/x86/intel_sgx/le/enclave/sgxsign drivers/platform/x86/intel_sgx/le/sgx_le_proxy

Quoted 13 lines

If the default key does not exist kbuild will generate a random key and place it to this location. KBUILD_SGX_SIGN_PIN can be used to specify the passphrase for the LE public key.

TinyCrypt (https://github.com/01org/tinycrypt) is used as AES implementation, which is not timing resistant. Eventually this needs to be replaced with AES-NI based implementation that could be either

- re-use existing AES-NI code in the kernel
- have its own hand written code

---

#27 lore raw

Re: [intel-sgx-kernel-dev] [PATCH v5 08/11] intel_sgx: in-kernel launch enclave

Quoted 8 lines · Sean Christopherson

Unless there is some conflict you are worried about, "signing_key.pem" is preferable as the default name so that the key is ignored via the top-level .gitignore.  The intel_sgx dir should have also a .gitignore to exclude the other LE related output files:

drivers/platform/x86/intel_sgx/le/enclave/sgx_le.ss drivers/platform/x86/intel_sgx/le/enclave/sgxsign drivers/platform/x86/intel_sgx/le/sgx_le_proxy

OK, I'll add these to the .gitignore.

/Jarkko

#28 lore raw

Re: [intel-sgx-kernel-dev] [PATCH v5 08/11] intel_sgx: in-kernel launch enclave

Quoted 6 lines · Jarkko Sakkinen
Quoted 5 lines · Sean Christopherson

Unless there is some conflict you are worried about, "signing_key.pem" is preferable as the default name so that the key is ignored via the top-level .gitignore.  The intel_sgx dir should have also a .gitignore to exclude the other LE related output files:

Quoted 7 lines
Quoted 3 lines

drivers/platform/x86/intel_sgx/le/enclave/sgx_le.ss drivers/platform/x86/intel_sgx/le/enclave/sgxsign drivers/platform/x86/intel_sgx/le/sgx_le_proxy

OK, I'll add these to the .gitignore.

/Jarkko

I did the signing_key.pem change to v6. Not sure if I update .gitignore for that version.

/Jarkko

#29 lore raw

Re: [PATCH v5 08/11] intel_sgx: in-kernel launch enclave

In reply to Jarkko Sakkinen
Quoted 5 lines · Jarkko Sakkinen

TinyCrypt (https://github.com/01org/tinycrypt) is used as AES implementation, which is not timing resistant. Eventually this needs to be replaced with AES-NI based implementation that could be either

- re-use existing AES-NI code in the kernel

This. That is an absolute must. We're not going to merge custom AES implementations.

#30 lore raw

Re: [PATCH v5 08/11] intel_sgx: in-kernel launch enclave

Quoted 9 lines · Peter Zijlstra
Quoted 5 lines · Jarkko Sakkinen

TinyCrypt (https://github.com/01org/tinycrypt) is used as AES implementation, which is not timing resistant. Eventually this needs to be replaced with AES-NI based implementation that could be either

- re-use existing AES-NI code in the kernel

This. That is an absolute must. We're not going to merge custom AES implementations.

I'll post v6 update without update to this in order to get otherwise improved version out and work on it for v7.

My initial idea to sort this out would be to try to compile

arch/x86/crypto/aesni-intel_asm.S

as part of the enclave binary and use it to run AES-256.

/Jarkko

#31 lore raw

Re: [PATCH v5 08/11] intel_sgx: in-kernel launch enclave

Quoted 11 lines · Jarkko Sakkinen
Quoted 9 lines · Peter Zijlstra
Quoted 5 lines · Jarkko Sakkinen

TinyCrypt (https://github.com/01org/tinycrypt) is used as AES implementation, which is not timing resistant. Eventually this needs to be replaced with AES-NI based implementation that could be either

- re-use existing AES-NI code in the kernel

This. That is an absolute must. We're not going to merge custom AES implementations.

Quoted 8 lines

I'll post v6 update without update to this in order to get otherwise improved version out and work on it for v7.

My initial idea to sort this out would be to try to compile

arch/x86/crypto/aesni-intel_asm.S

as part of the enclave binary and use it to run AES-256.

No. The kernel has a crypto API.

Thanks,

tglx

#32 lore raw

Re: [PATCH v5 08/11] intel_sgx: in-kernel launch enclave

Quoted 15 lines · Thomas Gleixner
Quoted 14 lines · Jarkko Sakkinen
Quoted 9 lines · Peter Zijlstra
Quoted 5 lines · Jarkko Sakkinen

TinyCrypt (https://github.com/01org/tinycrypt) is used as AES implementation, which is not timing resistant. Eventually this needs to be replaced with AES-NI based implementation that could be either

- re-use existing AES-NI code in the kernel

This. That is an absolute must. We're not going to merge custom AES implementations.

I'll post v6 update without update to this in order to get otherwise improved version out and work on it for v7.

Quoted 7 lines
Quoted 5 lines

My initial idea to sort this out would be to try to compile

arch/x86/crypto/aesni-intel_asm.S

as part of the enclave binary and use it to run AES-256.

No. The kernel has a crypto API.

But you cannot call it from inside an enclave. A syscall will exit the enclave. The launch enclave requires AES-256 to be executed inside the enclave.

/Jarkko

#33 lore raw

Re: [PATCH v5 08/11] intel_sgx: in-kernel launch enclave

Quoted 20 lines · Jarkko Sakkinen
Quoted 19 lines · Thomas Gleixner
Quoted 18 lines · Jarkko Sakkinen
Quoted 9 lines · Peter Zijlstra
Quoted 5 lines · Jarkko Sakkinen

TinyCrypt (https://github.com/01org/tinycrypt) is used as AES implementation, which is not timing resistant. Eventually this needs to be replaced with AES-NI based implementation that could be either

- re-use existing AES-NI code in the kernel

This. That is an absolute must. We're not going to merge custom AES implementations.

I'll post v6 update without update to this in order to get otherwise improved version out and work on it for v7.

My initial idea to sort this out would be to try to compile

arch/x86/crypto/aesni-intel_asm.S

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

as part of the enclave binary and use it to run AES-256.

No. The kernel has a crypto API.

But you cannot call it from inside an enclave. A syscall will exit the enclave. The launch enclave requires AES-256 to be executed inside the enclave.

Color me confused.

The launch enclave is part of the kernel, at least that's what the subject line claims. So why and how would it do a syscall? The kernel has it's internal crypto API.

Thanks,

tglx

#34 lore raw

Re: [PATCH v5 08/11] intel_sgx: in-kernel launch enclave

Quoted 3 lines · Thomas Gleixner

The launch enclave is part of the kernel, at least that's what the subject line claims. So why and how would it do a syscall? The kernel has it's internal crypto API.

It's part of the kernel in the way as lets say code arch/x86/realmode is. It's hosted by kernel but it does not run in the same address space as the kernel.

These constraints apply for enclaves:

1. They only run in ring-3.
2. They can only execute code inside their address range.

/Jarkko

#35 lore raw

Re: [PATCH v5 08/11] intel_sgx: in-kernel launch enclave

Quoted 5 lines · Jarkko Sakkinen
Quoted 3 lines · Thomas Gleixner

The launch enclave is part of the kernel, at least that's what the subject line claims. So why and how would it do a syscall? The kernel has it's internal crypto API.

Quoted 8 lines

It's part of the kernel in the way as lets say code arch/x86/realmode is. It's hosted by kernel but it does not run in the same address space as the kernel.

These constraints apply for enclaves:

1. They only run in ring-3.
2. They can only execute code inside their address range.

Can you please explain the whole thing so people not fully familiar with the inner workings of your stuff can understand it? You are providing only tiny bits of cryptic information on each question. That's going to take ages ...

Thanks,

tglx

#36 lore raw

PATCH v5 9/11 fs/pipe.c: export create_pipe_files() and replace_fd()

In reply to Jarkko Sakkinen

Exported create_pipe_files() and replace_fd() because the SGX driver needs to be able to setup pipes in order to communicate with the helper process that hosts the Launch Enclave (LE). The pipe creation will be done in the init-callback supplied to call_usermodehelper_setup().

The driver will use two pipes for communication with the LE hosting process:

* One for writing SIGSTRUCT blobs.
* One for reading EINITTOKEN blobs.

---
 fs/file.c | 1 +
 fs/pipe.c | 1 +
 2 files changed, 2 insertions(+)
fs/file.c +1 -0
index 1fc7fbbb4510..b1fa28919b22 100644
@@ -871,6 +871,7 @@ int replace_fd(unsigned fd, struct file *file, unsigned flags)
 	spin_unlock(&files->file_lock);
 	return err;
 }
+EXPORT_SYMBOL_GPL(replace_fd);
 
 SYSCALL_DEFINE3(dup3, unsigned int, oldfd, unsigned int, newfd, int, flags)
 {
fs/pipe.c +1 -0
index 97e5be897753..ee33a84127e7 100644
@@ -784,6 +784,7 @@ int create_pipe_files(struct file **res, int flags)
 	iput(inode);
 	return err;
 }
+EXPORT_SYMBOL_GPL(create_pipe_files);
 
 static int __do_pipe_flags(int *fd, struct file **files, int flags)
 {
Signature · 1 line
2.14.1
#37 lore raw

Re: [PATCH v5 09/11] fs/pipe.c: export create_pipe_files() and replace_fd()

On Mon, 13 Nov 2017, Jarkko Sakkinen wrote:

+ Cc: Al

- Cc: that intel sgx list because it's moderated and spams my inbox with useless moderation mails.

Quoted 10 lines

Exported create_pipe_files() and replace_fd() because the SGX driver needs to be able to setup pipes in order to communicate with the helper process that hosts the Launch Enclave (LE). The pipe creation will be done in the init-callback supplied to call_usermodehelper_setup().

The driver will use two pipes for communication with the LE hosting process:

* One for writing SIGSTRUCT blobs.
* One for reading EINITTOKEN blobs.

Can you pretty please make sure that people who maintain the affected code are CC'd on your changes?

Quoted 34 lines
---
 fs/file.c | 1 +
 fs/pipe.c | 1 +
 2 files changed, 2 insertions(+)
fs/file.c +1 -0
index 1fc7fbbb4510..b1fa28919b22 100644
@@ -871,6 +871,7 @@ int replace_fd(unsigned fd, struct file *file, unsigned flags)
 	spin_unlock(&files->file_lock);
 	return err;
 }
+EXPORT_SYMBOL_GPL(replace_fd);
 
 SYSCALL_DEFINE3(dup3, unsigned int, oldfd, unsigned int, newfd, int, flags)
 {
fs/pipe.c +1 -0
index 97e5be897753..ee33a84127e7 100644
@@ -784,6 +784,7 @@ int create_pipe_files(struct file **res, int flags)
 	iput(inode);
 	return err;
 }
+EXPORT_SYMBOL_GPL(create_pipe_files);
 
 static int __do_pipe_flags(int *fd, struct file **files, int flags)
 {
Signature · 1 line
2.14.1
#38 lore raw

Re: [PATCH v5 09/11] fs/pipe.c: export create_pipe_files() and replace_fd()

Quoted 6 lines · Thomas Gleixner

On Mon, 13 Nov 2017, Jarkko Sakkinen wrote:

+ Cc: Al

- Cc: that intel sgx list because it's moderated and spams my inbox with useless moderation mails.

I guess I'll have to apply for vger.kernel.org list then.

Quoted 5 lines
Quoted 5 lines

Exported create_pipe_files() and replace_fd() because the SGX driver needs to be able to setup pipes in order to communicate with the helper process that hosts the Launch Enclave (LE). The pipe creation will be done in the init-callback supplied to call_usermodehelper_setup().

Quoted 8 lines
Quoted 5 lines

The driver will use two pipes for communication with the LE hosting process:

* One for writing SIGSTRUCT blobs.
* One for reading EINITTOKEN blobs.

Can you pretty please make sure that people who maintain the affected code are CC'd on your changes?

Yes. I actually have written a script to collect everything and pass to --cc-cmd command that I use with TPM stuff and I forgot it from my send-email command line when I've sent these series.

/Jarkko

#39 lore raw

PATCH v5 10/11 intel_sgx: glue code for in-kernel LE

In reply to Jarkko Sakkinen

Glue code for hosting in-kernel Launch Enclave (LE) by using the user space helper framework.

Tokens for launching enclaves are generated with by the following protocol:

1. The driver sends a SIGSTRUCT blob to the LE hosting process to the input pipe.
2. The LE hosting process reads the SIGSTRUCT blob from the input pipe.
3. After generating a EINITTOKEN blob, the LE hosting process writes it to the output pipe.
4. The driver reads the EINITTOKEN blob from the output pipe.

If IA32_SGXLEPUBKEYHASH* MSRs are writable and they don't have the public key hash of the LE they will be updated.

---
 drivers/platform/x86/intel_sgx/Kconfig             |   2 +
 drivers/platform/x86/intel_sgx/Makefile            |   1 +
 drivers/platform/x86/intel_sgx/sgx.h               |  23 ++
 drivers/platform/x86/intel_sgx/sgx_encl.c          |  19 ++
 drivers/platform/x86/intel_sgx/sgx_ioctl.c         |  10 +-
 drivers/platform/x86/intel_sgx/sgx_le.c            | 313 +++++++++++++++++++++
 .../platform/x86/intel_sgx/sgx_le_proxy_piggy.S    |   4 +
 drivers/platform/x86/intel_sgx/sgx_main.c          |  54 +++-
 drivers/platform/x86/intel_sgx/sgx_util.c          |  25 ++
 9 files changed, 444 insertions(+), 7 deletions(-)
 create mode 100644 drivers/platform/x86/intel_sgx/sgx_le.c
drivers/platform/x86/intel_sgx/Kconfig +2 -0
index 9c445e9f89fa..1a80ca0ddca3 100644
@@ -9,6 +9,8 @@ config INTEL_SGX
 	default n
 	depends on X86_64 && CPU_SUP_INTEL
 	select MMU_NOTIFIER
+	select CRYPTO
+	select CRYPTO_SHA256
 	---help---
 	Intel(R) SGX is a set of CPU instructions that can be used by
 	applications to set aside private regions of code and data.  The code
drivers/platform/x86/intel_sgx/Makefile +1 -0
index 39c0a5ad242c..34abcdd8eb72 100644
@@ -11,6 +11,7 @@ intel_sgx-$(CONFIG_INTEL_SGX) += \
 	sgx_page_cache.o \
 	sgx_util.o \
 	sgx_vma.o \
+	sgx_le.o \
 	sgx_le_proxy_piggy.o
 
 $(eval $(call config_filename,INTEL_SGX_SIGNING_KEY))
drivers/platform/x86/intel_sgx/sgx.h +23 -0
index 3f018fae7ba4..e397da3b4c72 100644
@@ -68,6 +68,7 @@
 #include <linux/workqueue.h>
 #include <linux/mmu_notifier.h>
 #include <linux/radix-tree.h>
+#include <crypto/hash.h>
 #include <asm/sgx.h>
 
 #define SGX_MAX_EPC_BANKS 8
@@ -170,6 +171,10 @@ struct sgx_epc_bank {
 	unsigned long size;
 };
 
+extern unsigned char sgx_le_proxy[];
+extern unsigned char sgx_le_proxy_end[];
+extern struct sgx_sigstruct sgx_le_ss;
+
 extern struct workqueue_struct *sgx_add_page_wq;
 extern struct sgx_epc_bank sgx_epc_banks[];
 extern int sgx_nr_epc_banks;
@@ -178,7 +183,10 @@ extern u64 sgx_encl_size_max_64;
 extern u64 sgx_xfrm_mask;
 extern u32 sgx_misc_reserved;
 extern u32 sgx_xsave_size_tbl[64];
+extern u64 sgx_le_pubkeyhash[4];
+extern bool sgx_unlocked_msrs;
 
+extern const struct file_operations sgx_fops;
 extern const struct vm_operations_struct sgx_vm_ops;
 
 #define sgx_pr_ratelimited(level, encl, fmt, ...)			  \
@@ -237,6 +245,9 @@ struct sgx_encl_page *sgx_fault_page(struct vm_area_struct *vma,
 				     unsigned int flags);
 
 
+int sgx_get_key_hash(struct crypto_shash *tfm, const void *modulus, void *hash);
+int sgx_get_key_hash_simple(const void *modulus, void *hash);
+
 extern struct mutex sgx_tgid_ctx_mutex;
 extern struct list_head sgx_tgid_ctx_list;
 extern atomic_t sgx_va_pages_cnt;
@@ -251,4 +262,16 @@ void sgx_put_page(void *epc_page_vaddr);
 void sgx_eblock(struct sgx_encl *encl, struct sgx_epc_page *epc_page);
 void sgx_etrack(struct sgx_encl *encl);
 
+extern struct sgx_le_ctx sgx_le_ctx;
+
+int sgx_le_init(struct sgx_le_ctx *ctx);
+void sgx_le_exit(struct sgx_le_ctx *ctx);
+void sgx_le_stop(struct sgx_le_ctx *ctx);
+int sgx_le_start(struct sgx_le_ctx *ctx);
+
+int sgx_le_get_token(struct sgx_le_ctx *ctx,
+		     const struct sgx_encl *encl,
+		     const struct sgx_sigstruct *sigstruct,
+		     struct sgx_einittoken *token);
+
 #endif /* __ARCH_X86_INTEL_SGX_H__ */
drivers/platform/x86/intel_sgx/sgx_encl.c +19 -0
index 5ed9ceb01f1f..c175b540e607 100644
@@ -68,6 +68,7 @@
 #include <linux/slab.h>
 #include <linux/hashtable.h>
 #include <linux/shmem_fs.h>
+#include <linux/percpu.h>
 
 struct sgx_add_page_req {
 	struct sgx_encl *encl;
@@ -873,6 +874,14 @@ static int sgx_einit(struct sgx_encl *encl, struct sgx_sigstruct *sigstruct,
 	return ret;
 }
 
+static void sgx_update_pubkeyhash(void)
+{
+	wrmsrl(MSR_IA32_SGXLEPUBKEYHASH0, sgx_le_pubkeyhash[0]);
+	wrmsrl(MSR_IA32_SGXLEPUBKEYHASH1, sgx_le_pubkeyhash[1]);
+	wrmsrl(MSR_IA32_SGXLEPUBKEYHASH2, sgx_le_pubkeyhash[2]);
+	wrmsrl(MSR_IA32_SGXLEPUBKEYHASH3, sgx_le_pubkeyhash[3]);
+}
+
 /**
  * sgx_encl_init - perform EINIT for the given enclave
  *
@@ -908,6 +917,16 @@ int sgx_encl_init(struct sgx_encl *encl, struct sgx_sigstruct *sigstruct,
 		for (j = 0; j < SGX_EINIT_SPIN_COUNT; j++) {
 			ret = sgx_einit(encl, sigstruct, token);
 
+			if (ret == SGX_INVALID_ATTRIBUTE ||
+			    ret == SGX_INVALID_EINITTOKEN) {
+				if (sgx_unlocked_msrs) {
+					preempt_disable();
+					sgx_update_pubkeyhash();
+					ret = sgx_einit(encl, sigstruct, token);
+					preempt_enable();
+				}
+			}
+
 			if (ret == SGX_UNMASKED_EVENT)
 				continue;
 			else
drivers/platform/x86/intel_sgx/sgx_ioctl.c +6 -4
index 3fcef914005f..1f14d31690ca 100644
@@ -69,7 +69,7 @@
 #include <linux/hashtable.h>
 #include <linux/shmem_fs.h>
 
-static int sgx_get_encl(unsigned long addr, struct sgx_encl **encl)
+static int sgx_encl_get(unsigned long addr, struct sgx_encl **encl)
 {
 	struct mm_struct *mm = current->mm;
 	struct vm_area_struct *vma;
@@ -156,7 +156,7 @@ static long sgx_ioc_enclave_add_page(struct file *filep, unsigned int cmd,
 	void *data;
 	int ret;
 
-	ret = sgx_get_encl(addp->addr, &encl);
+	ret = sgx_encl_get(addp->addr, &encl);
 	if (ret)
 		return ret;
 
@@ -227,11 +227,13 @@ static long sgx_ioc_enclave_init(struct file *filep, unsigned int cmd,
 	if (ret)
 		goto out;
 
-	ret = sgx_get_encl(encl_id, &encl);
+	ret = sgx_encl_get(encl_id, &encl);
 	if (ret)
 		goto out;
 
-	ret = sgx_encl_init(encl, sigstruct, einittoken);
+	ret = sgx_le_get_token(&sgx_le_ctx, encl, sigstruct, einittoken);
+	if (!ret)
+		ret = sgx_encl_init(encl, sigstruct, einittoken);
 
 	kref_put(&encl->refcount, sgx_encl_release);
 
drivers/platform/x86/intel_sgx/sgx_le.c new file +313 -0

313 lines not on this page · raw message

drivers/platform/x86/intel_sgx/sgx_le_proxy_piggy.S +4 -0
index faced8a9a75a..e1e3742a0c93 100644
@@ -9,3 +9,7 @@ GLOBAL(sgx_le_proxy)
 END(sgx_le_proxy)
 
 GLOBAL(sgx_le_proxy_end)
+
+GLOBAL(sgx_le_ss)
+	.incbin	"drivers/platform/x86/intel_sgx/le/enclave/sgx_le.ss"
+END(sgx_le_ss)
drivers/platform/x86/intel_sgx/sgx_main.c +51 -3

107 lines not on this page · raw message

drivers/platform/x86/intel_sgx/sgx_util.c +25 -0
index 6ef79499100f..6c1f06d1978a 100644
@@ -370,3 +370,28 @@ void sgx_etrack(struct sgx_encl *encl)
 		sgx_invalidate(encl, true);
 	}
 }
+
+int sgx_get_key_hash(struct crypto_shash *tfm, const void *modulus, void *hash)
+{
+	SHASH_DESC_ON_STACK(shash, tfm);
+
+	shash->tfm = tfm;
+	shash->flags = CRYPTO_TFM_REQ_MAY_SLEEP;
+
+	return crypto_shash_digest(shash, modulus, SGX_MODULUS_SIZE, hash);
+}
+
+int sgx_get_key_hash_simple(const void *modulus, void *hash)
+{
+	struct crypto_shash *tfm;
+	int ret;
+
+	tfm = crypto_alloc_shash("sha256", 0, CRYPTO_ALG_ASYNC);
+	if (IS_ERR(tfm))
+		return PTR_ERR(tfm);
+
+	ret = sgx_get_key_hash(tfm, modulus, hash);
+
+	crypto_free_shash(tfm);
+	return ret;
+}
Signature · 1 line
2.14.1
#40 lore raw

Re: [intel-sgx-kernel-dev] [PATCH v5 10/11] intel_sgx: glue code for in-kernel LE

Quoted 27 lines · Jarkko Sakkinen
drivers/platform/x86/intel_sgx/sgx_main.c +21 -0
@@ -88,6 +88,37 @@ u64 sgx_encl_size_max_64;
  u64 sgx_xfrm_mask = 0x3;
  u32 sgx_misc_reserved;
  u32 sgx_xsave_size_tbl[64];
+bool sgx_unlocked_msrs;
+u64 sgx_le_pubkeyhash[4];
+
+static DECLARE_RWSEM(sgx_file_sem);
+
+static int sgx_open(struct inode *inode, struct file *file)
+{
+	int ret;
+
+	down_read(&sgx_file_sem);
+
+	ret = sgx_le_start(&sgx_le_ctx);
+	if (ret) {
+		up_read(&sgx_file_sem);
+		return ret;
+	}
+
+	return 0;
+}
+
+static int sgx_release(struct inode *inode, struct file *file)

drivers/platform/x86/intel_sgx/sgx_main.c u64 sgx_encl_size_max_64; · line 121

Quoted 10 lines
drivers/platform/x86/intel_sgx/sgx_main.c +10 -0
+{
+	up_read(&sgx_file_sem);
+
+	if (down_write_trylock(&sgx_file_sem)) {
+		sgx_le_stop(&sgx_le_ctx);
+		up_write(&sgx_file_sem);
+	}
+
+	return 0;
+}

This semaphore approach is broken due to the LE process using an anon inode for /dev/sgx, which results in sgx_release being called without an accompanying call to sgx_open.  This causes deadlocks due to a semaphore underrun.

https://lists.01.org/pipermail/intel-sgx-kernel-dev/2017-November/000901.html

[  242.659272] INFO: task lsdt:9425 blocked for more than 120 seconds. [  242.659783]       Not tainted 4.14.0-rc4+ #18
[  242.660063] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this [  242.660558] lsdt            D    0  9425      1 0x00000004
[  242.660559] Call Trace:
[  242.660564]  __schedule+0x3c2/0x8b0
[  242.660567]  schedule+0x36/0x80
[  242.660568]  rwsem_down_read_failed+0x10a/0x170
[  242.660569]  call_rwsem_down_read_failed+0x18/0x30
[  242.660570]  ? call_rwsem_down_read_failed+0x18/0x30
[  242.660571]  down_read+0x20/0x40
[  242.660572]  sgx_open+0x19/0x40 [intel_sgx]
[  242.660574]  chrdev_open+0xbf/0x1b0
[  242.660576]  do_dentry_open+0x1f8/0x300
[  242.660577]  ? cdev_put+0x30/0x30
[  242.660578]  vfs_open+0x4f/0x70
[  242.660579]  path_openat+0x2ae/0x13a0
[  242.660581]  ? mem_cgroup_uncharge_swap+0x60/0x90
[  242.660582]  do_filp_open+0x99/0x110
[  242.660583]  ? __check_object_size+0xfc/0x1a0
[  242.660585]  ? __alloc_fd+0xb0/0x170
[  242.660586]  do_sys_open+0x124/0x210
[  242.660587]  ? do_sys_open+0x124/0x210
[  242.660588]  SyS_open+0x1e/0x20
[  242.660589]  entry_SYSCALL_64_fastpath+0x1e/0xa9
[  242.660590] RIP: 0033:0x7f426cf9ec7d
[  242.660591] RSP: 002b:00007f426b31ea60 EFLAGS: 00000293 ORIG_RAX:
[  242.660592] RAX: ffffffffffffffda RBX: 000000c4200ba000 RCX: 00007f426cf9ec7d [  242.660592] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 000000000068cca7 [  242.660593] RBP: 00007f426b31ec10 R08: 0000000000f6bc30 R09: 0000000000000000 [  242.660593] R10: 00007f4264000078 R11: 0000000000000293 R12: 0000000000000001 [  242.660594] R13: 0000000000000000 R14: 00007f426d31b13d R15: 00007f42640008c0

Quoted 16 lines
drivers/platform/x86/intel_sgx/sgx_main.c +0 -0
  #ifdef CONFIG_COMPAT

 long sgx_compat_ioctl(struct file *filep, unsigned int cmd, unsigned long arg)

drivers/platform/x86/intel_sgx/sgx_main.c +0 -0
@@ -141,8 +172,10 @@ static unsigned long sgx_get_unmapped_area(struct file
 *file,
  	return addr;
  }

drivers/platform/x86/intel_sgx/sgx_main.c +1 -1
-static const struct file_operations sgx_fops = {
+const struct file_operations sgx_fops = {

  .owner = THIS_MODULE,

drivers/platform/x86/intel_sgx/sgx_main.c +2 -0
+	.open			= sgx_open,
+	.release		= sgx_release,

  .unlocked_ioctl = sgx_ioctl,
 #ifdef CONFIG_COMPAT
  .compat_ioctl = sgx_compat_ioctl,

#41 lore raw

Re: [intel-sgx-kernel-dev] [PATCH v5 10/11] intel_sgx: glue code for in-kernel LE

Quoted 5 lines · Sean Christopherson

This semaphore approach is broken due to the LE process using an anon inode for /dev/sgx, which results in sgx_release being called without an accompanying call to sgx_open.  This causes deadlocks due to a semaphore underrun.

https://lists.01.org/pipermail/intel-sgx-kernel-dev/2017-November/000901.html

Thank you for catching this. I can only say that I forgot to fix it it for v5. It will be refined for v6.

/Jarkko

#42 lore raw

Re: [intel-sgx-kernel-dev] [PATCH v5 10/11] intel_sgx: glue code for in-kernel LE

Quoted 11 lines · Jarkko Sakkinen
Quoted 5 lines · Sean Christopherson

This semaphore approach is broken due to the LE process using an anon inode for /dev/sgx, which results in sgx_release being called without an accompanying call to sgx_open.  This causes deadlocks due to a semaphore underrun.

https://lists.01.org/pipermail/intel-sgx-kernel-dev/2017-November/000901.html

Thank you for catching this. I can only say that I forgot to fix it it for v5. It will be refined for v6.

/Jarkko

I added a fix to 'le' branch (will be squashed in v6).

/Jarkko

#43 lore raw

Re: [PATCH v5 10/11] intel_sgx: glue code for in-kernel LE

In reply to Jarkko Sakkinen
Quoted 14 lines · Jarkko Sakkinen

Glue code for hosting in-kernel Launch Enclave (LE) by using the user space helper framework.

Tokens for launching enclaves are generated with by the following protocol:

1. The driver sends a SIGSTRUCT blob to the LE hosting process to the input pipe.
2. The LE hosting process reads the SIGSTRUCT blob from the input pipe.
3. After generating a EINITTOKEN blob, the LE hosting process writes it to the output pipe.
4. The driver reads the EINITTOKEN blob from the output pipe.

Quoted 3 lines

If IA32_SGXLEPUBKEYHASH* MSRs are writable and they don't have the public key hash of the LE they will be updated.

A few nits throughout to keep in mind:

* #includes in alphabetical order in general
* function local variables declared in order of decreasing line length
* don't insert newlines where coding_style doesn't compel you to

Quoted 2 lines

-

...--

Quoted 6 lines
drivers/platform/x86/intel_sgx/sgx_le.c new file +0 -0
new file mode 100644
index 000000000000..d49c58f09db6
@@ -0,0 +1,313 @@

...

drivers/platform/x86/intel_sgx/sgx_le.c line 65

Quoted 9 lines
drivers/platform/x86/intel_sgx/sgx_le.c new file +9 -0
+#include <linux/file.h>
+#include <linux/fs.h>
+#include <linux/kmod.h>
+#include <linux/mutex.h>
+#include <linux/wait.h>
+#include <linux/pipe_fs_i.h>
+#include <linux/sched/signal.h>
+#include <linux/shmem_fs.h>
+#include <linux/anon_inodes.h>

alphabetical order
...

drivers/platform/x86/intel_sgx/sgx_le.c line 90

Quoted 9 lines
drivers/platform/x86/intel_sgx/sgx_le.c new file +9 -0
+static int sgx_le_create_pipe(struct sgx_le_ctx *ctx,
+			      unsigned int fd)
+{
+	struct file *files[2];
+	int ret;
+
+	ret = create_pipe_files(files, 0);
+	if (ret)
+		goto out;

Fairly inconsistent in the use of the goto out: model and returning inline where there is no cleanup to be done. Whatever you do, please be consistent within the file.

If there is no cleanup to do, a local return is fine.

drivers/platform/x86/intel_sgx/sgx_le.c line 103

Quoted 13 lines
drivers/platform/x86/intel_sgx/sgx_le.c new file +13 -0
+
+	ctx->pipes[fd] = files[fd ^ 1];
+	ret = replace_fd(fd, files[fd], 0);
+	fput(files[fd]);
+
+out:
+	return ret;
+}
+
+static int sgx_le_read(struct file *file, void *data, unsigned int len)
+{
+	ssize_t ret;
+	loff_t pos = 0;

Decreasing line length.

drivers/platform/x86/intel_sgx/sgx_le.c line 113

Quoted 9 lines
drivers/platform/x86/intel_sgx/sgx_le.c new file +9 -0
+	ret = kernel_read(file, data, len, &pos);
+
+	if (ret != len && ret >= 0)
+		return -ENOMEM;
+
+	if (ret < 0)
+		return ret;
+
+	return 0;

You save a 4 lines with:

	if (ret < 0)
		return ret;
	return ret == len ? 0 : -ENOMEM;

They're common, but ternary ops aren't the most legible things in the world, so your call.

drivers/platform/x86/intel_sgx/sgx_le.c line 120

Quoted 7 lines
drivers/platform/x86/intel_sgx/sgx_le.c new file +7 -0
+}
+
+static int sgx_le_write(struct file *file, const void *data,
+			unsigned int len)
+{
+	ssize_t ret;
+	loff_t pos = 0;

Decreasing line length.
...

drivers/platform/x86/intel_sgx/sgx_le.c line 136

Quoted 4 lines
drivers/platform/x86/intel_sgx/sgx_le.c new file +4 -0
+static int sgx_le_task_init(struct subprocess_info *subinfo, struct cred *new)
+{
+	struct sgx_le_ctx *ctx =
+		(struct sgx_le_ctx *)subinfo->data;

No wrap

Quoted 26 lines
drivers/platform/x86/intel_sgx/sgx_le.c new file +26 -0
+	unsigned long len;
+	struct file *tmp_filp;
+	int ret;
+
+	len = (unsigned long)&sgx_le_proxy_end - (unsigned long)&sgx_le_proxy;
+
+	tmp_filp = shmem_file_setup("[sgx_le_proxy]", len, 0);
+	if (IS_ERR(tmp_filp)) {
+		ret = PTR_ERR(tmp_filp);
+		return ret;
+	}
+	ret = replace_fd(SGX_LE_PROXY_FD, tmp_filp, 0);
+	fput(tmp_filp);
+	if (ret < 0)
+		return ret;
+
+	ret = sgx_le_write(tmp_filp, &sgx_le_proxy, len);
+	if (ret)
+		return ret;
+
+	tmp_filp = anon_inode_getfile("[/dev/sgx]", &sgx_fops, NULL, O_RDWR);
+	if (IS_ERR(tmp_filp))
+		return PTR_ERR(tmp_filp);
+	ret = replace_fd(SGX_LE_DEV_FD, tmp_filp, 0);
+	fput(tmp_filp);
+	if (ret < 0)

drivers/platform/x86/intel_sgx/sgx_le.c line 172

Quoted 10 lines
drivers/platform/x86/intel_sgx/sgx_le.c new file +10 -0
+		return ret;
+
+	ret = sgx_le_create_pipe(ctx, 0);
+	if (ret < 0)
+		return ret;
+
+	ret = sgx_le_create_pipe(ctx, 1);
+	if (ret < 0)
+		return ret;
+

No incremental cleanup here - appears to all be handled through sgx_le_stop - do I have that right?
...

Quoted 16 lines
drivers/platform/x86/intel_sgx/sgx_main.c +8 -0
index 636a583bad31..7931083651f5 100644
@@ -88,6 +88,37 @@ u64 sgx_encl_size_max_64;
 u64 sgx_xfrm_mask = 0x3;
 u32 sgx_misc_reserved;
 u32 sgx_xsave_size_tbl[64];
+bool sgx_unlocked_msrs;
+u64 sgx_le_pubkeyhash[4];
+
+static DECLARE_RWSEM(sgx_file_sem);
+
+static int sgx_open(struct inode *inode, struct file *file)
+{
+	int ret;

drivers/platform/x86/intel_sgx/sgx_main.c u64 sgx_encl_size_max_64; · line 108

Quoted 10 lines
drivers/platform/x86/intel_sgx/sgx_main.c +10 -0
+
+	down_read(&sgx_file_sem);
+
+	ret = sgx_le_start(&sgx_le_ctx);
+	if (ret) {
+		up_read(&sgx_file_sem);
+		return ret;
+	}
+
+	return 0;

We can simplify the return logic:

if (ret)

	up_read...

return ret;

Signature · 2 lines
Darren Hart
VMware Open Source Technology Center
#44 lore raw

Re: [PATCH v5 10/11] intel_sgx: glue code for in-kernel LE

Quoted 61 lines · Darren Hart
Quoted 14 lines · Jarkko Sakkinen

Glue code for hosting in-kernel Launch Enclave (LE) by using the user space helper framework.

Tokens for launching enclaves are generated with by the following protocol:

1. The driver sends a SIGSTRUCT blob to the LE hosting process to the input pipe.
2. The LE hosting process reads the SIGSTRUCT blob from the input pipe.
3. After generating a EINITTOKEN blob, the LE hosting process writes it to the output pipe.
4. The driver reads the EINITTOKEN blob from the output pipe.

Quoted 3 lines

If IA32_SGXLEPUBKEYHASH* MSRs are writable and they don't have the public key hash of the LE they will be updated.

A few nits throughout to keep in mind:

* #includes in alphabetical order in general
* function local variables declared in order of decreasing line length
* don't insert newlines where coding_style doesn't compel you to

Quoted 2 lines

-

...--

Quoted 6 lines
drivers/platform/x86/intel_sgx/sgx_le.c new file +0 -0
new file mode 100644
index 000000000000..d49c58f09db6
@@ -0,0 +1,313 @@

...

drivers/platform/x86/intel_sgx/sgx_le.c line 65

Quoted 9 lines
drivers/platform/x86/intel_sgx/sgx_le.c new file +9 -0
+#include <linux/file.h>
+#include <linux/fs.h>
+#include <linux/kmod.h>
+#include <linux/mutex.h>
+#include <linux/wait.h>
+#include <linux/pipe_fs_i.h>
+#include <linux/sched/signal.h>
+#include <linux/shmem_fs.h>
+#include <linux/anon_inodes.h>

alphabetical order
...

drivers/platform/x86/intel_sgx/sgx_le.c line 90

Quoted 9 lines
drivers/platform/x86/intel_sgx/sgx_le.c new file +9 -0
+static int sgx_le_create_pipe(struct sgx_le_ctx *ctx,
+			      unsigned int fd)
+{
+	struct file *files[2];
+	int ret;
+
+	ret = create_pipe_files(files, 0);
+	if (ret)
+		goto out;

Fairly inconsistent in the use of the goto out: model and returning inline where there is no cleanup to be done. Whatever you do, please be consistent within the file.

Quoted 1 line

If there is no cleanup to do, a local return is fine.

It is cruft that I haven't remembered to clean up eg there used to be clean up. Thanks for spotting that.

/Jarkko

#45 lore raw

Re: [PATCH v5 10/11] intel_sgx: glue code for in-kernel LE

In reply to Darren Hart
Quoted 2 lines · Darren Hart

No incremental cleanup here - appears to all be handled through sgx_le_stop - do I have that right?

Yes. This is correct.

/Jarkko

#46 lore raw

PATCH v5 11/11 intel_sgx: driver documentation

In reply to Jarkko Sakkinen
---
 Documentation/index.rst         |   1 +
 Documentation/x86/intel_sgx.rst | 131 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 132 insertions(+)
 create mode 100644 Documentation/x86/intel_sgx.rst
Documentation/index.rst +1 -0
index cb7f1ba5b3b1..ccfebc260e04 100644
@@ -86,6 +86,7 @@ implementation.
    :maxdepth: 2
 
    sh/index
+   x86/index
 
 Korean translations
 -------------------
Documentation/x86/intel_sgx.rst new file +131 -0

131 lines not on this page · raw message

Signature · 1 line
2.14.1
#47 lore raw

Re: [intel-sgx-kernel-dev] [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 52 lines · Jarkko Sakkinen
---
 Documentation/index.rst         |   1 +
 Documentation/x86/intel_sgx.rst | 131
++++++++++++++++++++++++++++++++++++++++
 2 files changed, 132 insertions(+)
 create mode 100644 Documentation/x86/intel_sgx.rst
Documentation/index.rst +1 -0
index cb7f1ba5b3b1..ccfebc260e04 100644
@@ -86,6 +86,7 @@ implementation.
    :maxdepth: 2
 
    sh/index
+   x86/index
 
 Korean translations
 -------------------
Documentation/x86/intel_sgx.rst new file +8 -0
new file mode 100644
index 000000000000..34bcf6a2a495
@@ -0,0 +1,131 @@
+===================
+Intel(R) SGX driver
+===================
+
+Introduction
+============
+
+Intel(R) SGX is a set of CPU instructions that can be used by

applications to
+set aside private regions of code and data. The code outside the enclave is
+disallowed to access the memory inside the enclave by the CPU access control.
+In a way you can think that SGX provides inverted sandbox. It protects the

Documentation/x86/intel_sgx.rst new file +3 -0
+application from a malicious host.
+
+There is a new hardware unit in the processor called Memory

Encryption Engine
+(MEE) starting from the Skylake microarchitecture. BIOS can define one or many
+MEE regions that can hold enclave data by configuring them with PRMRR registers.

Documentation/x86/intel_sgx.rst new file +2 -0
+
+The MEE automatically encrypts the data leaving the processor
Quoted 4 lines

package to the MEE
+regions. The data is encrypted using a random key whose life-time is exactly one
+power cycle.

Not sure whether you should talk about MEE staff here. They are not in SDM and (thus) may potentially be changed in the future.

Quoted 96 lines
Documentation/x86/intel_sgx.rst new file +2 -0
+
+You can tell if your CPU supports SGX by looking into

``/proc/cpuinfo``:

Documentation/x86/intel_sgx.rst new file +7 -0
+
+	``cat /proc/cpuinfo  | grep sgx``
+
+Enclave data types
+==================
+
+SGX defines new data types to maintain information about the

enclaves and their

Documentation/x86/intel_sgx.rst new file +5 -0
+security properties.
+
+The following data structures exist in MEE regions:
+
+* **Enclave Page Cache (EPC):** memory pages for protected code and

data

Documentation/x86/intel_sgx.rst new file +5 -0
+* **Enclave Page Cache Map (EPCM):** meta-data for each EPC page
+
+The Enclave Page Cache holds following types of pages:
+
+* **SGX Enclave Control Structure (SECS)**: meta-data defining the

global

Documentation/x86/intel_sgx.rst new file +3 -0
+  properties of an enclave such as range of addresses it can access.
+* **Regular (REG):** containing code and data for the enclave.
+* **Thread Control Structure (TCS):** defines an entry point for a

hardware
+ thread to enter into the enclave. The enclave can only be entered through

Documentation/x86/intel_sgx.rst new file +2 -0
+  these entry points.
+* **Version Array (VA)**: an EPC page receives a unique 8 byte

version number
+ when it is swapped, which is then stored into a VA page. A VA page can hold up

Documentation/x86/intel_sgx.rst new file +6 -0
+  to 512 version numbers.
+
+Launch control
+==============
+
+For launching an enclave, two structures must be provided for

ENCLS(EINIT):
+
+1. **SIGSTRUCT:** a signed measurement of the enclave binary. +2. **EINITTOKEN:** the measurement, the public key of the signer and various
+ enclave attributes. This structure contains a MAC of its contents
using

Documentation/x86/intel_sgx.rst new file +3 -0
+   hardware derived symmetric key called *launch key*.
+
+The hardware platform contains a root key pair for signing the

SIGTRUCT

Documentation/x86/intel_sgx.rst new file +3 -0
+for a *launch enclave* that is able to acquire the *launch key* for
+creating EINITTOKEN's for other enclaves.  For the launch enclave
+EINITTOKEN is not needed because it is signed with the private root

key.

Documentation/x86/intel_sgx.rst new file +4 -0
+
+There are two feature control bits associate with launch control
+
+* **IA32_FEATURE_CONTROL[0]**: locks down the feature control

register

Documentation/x86/intel_sgx.rst new file +2 -0
+* **IA32_FEATURE_CONTROL[17]**: allow runtime reconfiguration of
+  IA32_SGXLEPUBKEYHASHn MSRs that define MRSIGNER hash for the

launch
+ enclave. Essentially they define a signing key that does not require

Documentation/x86/intel_sgx.rst new file +3 -0
+  EINITTOKEN to be let run.
+
+The BIOS can configure IA32_SGXLEPUBKEYHASHn MSRs before feature

control

Documentation/x86/intel_sgx.rst new file +4 -0
+register is locked.
+
+It could be tempting to implement launch control by writing the MSRs
+every time when an enclave is launched. This does not scale because

for
+generic case because BIOS might lock down the MSRs before handover to

Documentation/x86/intel_sgx.rst new file +6 -0
+the OS.
+
+Debug enclaves
+--------------
+
+Enclave can be set as a *debug enclave* of which memory can be read

or written
+by using the ENCLS(EDBGRD) and ENCLS(EDBGWR) opcodes. The Intel provided launch
+enclave provides them always a valid EINITTOKEN and therefore they are a low

Documentation/x86/intel_sgx.rst new file +9 -0
+hanging fruit way to try out SGX.
+
+Virtualization
+==============
+
+Launch control
+--------------
+
+The values for IA32_SGXLEPUBKEYHASHn MSRs cannot be emulated for a
Quoted 6 lines

virtual
+machine guest. It would easily seem feasible to hold virtual values for these
+MSRs, trap ENCLS(EINIT) and use the host LE to generate a token when a guest LE
+is initialized.

IMHO, this statement doesn't make sense to me because of:

- Guest LE doesn't need a token. Guest LE is nothing different from Host LE from HW's point of view.
- Host LE won't (and shouldn't) generate token for any enclave from guest. Guset LE generates token for enclaves in the guest, just like host LE generates token for enclaves in the host (but not guest).
- In fact, theoretically KVM guest can still run LE and other enclaves without entire host SGX driver. There's no dependency between host LE and guest enclaves.

Quoted 8 lines
Documentation/x86/intel_sgx.rst new file +2 -0
+
+However, looking at the pseudo code of ENCLS(EINIT) from the SDM

there is a
+constraint that the instruction will fail if ATTRIBUTES.EINITTOKENKEY is set
+(the documentation does not tell the reason why the constraint exists but it
+exists).

Don't understand what you mean. This statement in SDM has nothing to do with virtualization. It's just description of hehavior of EINIT.

Quoted 4 lines
Documentation/x86/intel_sgx.rst new file +2 -0
+
+Thus, only when the MSRs are left unlocked before handover to the OS

the
+setting of these MSRs can be supported for VM guests.

IMHO you can just remove this "virtualization" section entirely as your first upstreaming driver won't consider virtualization at all.

Thanks,
-Kai

Quoted 36 lines
Documentation/x86/intel_sgx.rst new file +5 -0
+
+Suspend and resume
+------------------
+
+If the host suspends and resumes, the enclave memory for the VM

guest could

Documentation/x86/intel_sgx.rst new file +3 -0
+become invalid. This can make ENCLS leaf operations suddenly fail.
+
+The driver has a graceful fallback mechanism to manage this

situation. If any of
+the ENCLS leaf operations fail, the driver will fallback by kicking threads out
+of the enclave, removing the TCS entries and marking enclave as invalid. After
+this no new pages can be allocated for the enclave and no entry can be done.

Documentation/x86/intel_sgx.rst new file +15 -0
+
+SGX uapi
+========
+
+.. kernel-doc:: drivers/platform/x86/intel_sgx_ioctl.c
+   :functions: sgx_ioc_enclave_create
+               sgx_ioc_enclave_add_page
+               sgx_ioc_enclave_init
+
+.. kernel-doc:: arch/x86/include/uapi/asm/sgx.h
+
+References
+==========
+
+* System Programming Manual: 39.1.4 Intel® SGX Launch Control

Configuration
_______________________________________________
intel-sgx-kernel-dev mailing list
[email protected]
https://lists.01.org/mailman/listinfo/intel-sgx-kernel-dev

#48 lore raw

Re: [intel-sgx-kernel-dev] [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 2 lines · Kai Huang

Not sure whether you should talk about MEE staff here. They are not in SDM and (thus) may potentially be changed in the future.

The use of MEE is documented in many places like

https://eprint.iacr.org/2016/204.pdf

I see no reason not to talk about MEE.

Quoted 5 lines

IMHO you can just remove this "virtualization" section entirely as your first upstreaming driver won't consider virtualization at all.

Thanks,
-Kai

Makes sense.

/Jarkko

#49 lore raw

Re: [intel-sgx-kernel-dev] [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 10 lines · Jarkko Sakkinen
Quoted 3 lines · Kai Huang

Not sure whether you should talk about MEE staff here. They are not in
SDM and (thus) may potentially be changed in the future.

The use of MEE is documented in many places like

https://eprint.iacr.org/2016/204.pdf

I see no reason not to talk about MEE.

Sure. Thanks for pointing out the paper.

Quoted 9 lines
Quoted 7 lines

IMHO you can just remove this "virtualization" section entirely as your
first upstreaming driver won't consider virtualization at all.

Thanks,
-Kai

Makes sense.

Thanks.

Thanks,
-Kai

Quoted 2 lines

/Jarkko

maintainer
#50 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

In reply to Jarkko Sakkinen

On Mon, Nov 13, 2017 at 09:45:28PM +0200, Jarkko Sakkinen wrote:

<--- and yeah, all those patches without a commit message, need one.

Quoted 6 lines
---
 Documentation/index.rst         |   1 +
 Documentation/x86/intel_sgx.rst | 131 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 132 insertions(+)
 create mode 100644 Documentation/x86/intel_sgx.rst

...

Quoted 20 lines
Documentation/x86/intel_sgx.rst new file +20 -0
+Launch control
+==============
+
+For launching an enclave, two structures must be provided for ENCLS(EINIT):
+
+1. **SIGSTRUCT:** a signed measurement of the enclave binary.
+2. **EINITTOKEN:** the measurement, the public key of the signer and various
+   enclave attributes. This structure contains a MAC of its contents using
+   hardware derived symmetric key called *launch key*.
+
+The hardware platform contains a root key pair for signing the SIGTRUCT
+for a *launch enclave* that is able to acquire the *launch key* for
+creating EINITTOKEN's for other enclaves.  For the launch enclave
+EINITTOKEN is not needed because it is signed with the private root key.
+
+There are two feature control bits associate with launch control
+
+* **IA32_FEATURE_CONTROL[0]**: locks down the feature control register
+* **IA32_FEATURE_CONTROL[17]**: allow runtime reconfiguration of
+  IA32_SGXLEPUBKEYHASHn MSRs that define MRSIGNER hash for the launch

Documentation/x86/intel_sgx.rst line 78

Quoted 10 lines
Documentation/x86/intel_sgx.rst new file +10 -0
+  enclave. Essentially they define a signing key that does not require
+  EINITTOKEN to be let run.
+
+The BIOS can configure IA32_SGXLEPUBKEYHASHn MSRs before feature control
+register is locked.
+
+It could be tempting to implement launch control by writing the MSRs
+every time when an enclave is launched. This does not scale because for
+generic case because BIOS might lock down the MSRs before handover to
+the OS.

What does that mean exactly?

OEM vendor BIOS can control how many enclaves user can launch and what signing key is loaded and lock down the feature control register so that no other signing keys are loaded?

Or am I misreading this?

Signature · 4 lines
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.
#51 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 7 lines · Borislav Petkov
Quoted 5 lines
Documentation/x86/intel_sgx.rst new file +5 -0
+* **IA32_FEATURE_CONTROL[0]**: locks down the feature control register
+* **IA32_FEATURE_CONTROL[17]**: allow runtime reconfiguration of
+  IA32_SGXLEPUBKEYHASHn MSRs that define MRSIGNER hash for the launch
+  enclave. Essentially they define a signing key that does not require
+  EINITTOKEN to be let run.

[SNIP]

Quoted 7 lines

What does that mean exactly?

OEM vendor BIOS can control how many enclaves user can launch and what signing key is loaded and lock down the feature control register so that no other signing keys are loaded?

Or am I misreading this?

Pre-boot firmware could potentially configure the root key hash for the enclave that signs launch tokens for other enclaves i.e. the launch enclave that is built and signed during the kbuild.

VMM/Hypervisor could potentially simulate such enviroment for guests.

In these cases IA32_FEATURE_CONTROL[17] would be zeroed before locking the feature control, which would mean that the kernel could not write new values with wrmsr for the root key hash.

You could use this to write policy inside the LE. The current LE is really simple "pass-through everything" but it could be refined to be something more non-trivial.

The question is whether we want to allow this or not. If the answer is no, a check can be added to the driver initialization code whether 17 is set, and if not, it driver would fail to initialize.

I'm not too dilated to any direction concerning this question. I just wrote a driver that is robust. This is where I need your feedback to do the right thing.

Quoted 3 lines
Signature · 2 lines
Regards/Gruss,
    Boris.

PS. I'll add a proper commit message. Sorry for not doing that before.

/Jarkko

maintainer
#52 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 3 lines · Jarkko Sakkinen

Pre-boot firmware could potentially configure the root key hash for the enclave that signs launch tokens for other enclaves i.e. the launch enclave that is built and signed during the kbuild.

So how about firmware doesn't do anything and the machine owner decide what enclaves get launched and what key hashes to load for a change? I.e., let the owner really own the hardware she paid money for.

Or are we doing encrypted enclaves but then the firmware vendor can look inside too?

Signature · 4 lines
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.
#53 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 5 lines · Borislav Petkov
Quoted 3 lines · Jarkko Sakkinen

Pre-boot firmware could potentially configure the root key hash for the enclave that signs launch tokens for other enclaves i.e. the launch enclave that is built and signed during the kbuild.

Quoted 10 lines

So how about firmware doesn't do anything and the machine owner decide what enclaves get launched and what key hashes to load for a change? I.e., let the owner really own the hardware she paid money for.

Or are we doing encrypted enclaves but then the firmware vendor can look inside too?

Signature · 2 lines
Regards/Gruss,
    Boris.

Firmware cannot access the memory inside an enclave. CPU asserts every memory access coming outside the enclave.

/Jarkko

maintainer
#54 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 2 lines · Jarkko Sakkinen

Firmware cannot access the memory inside an enclave. CPU asserts every memory access coming outside the enclave.

But "firmware could potentially configure the root key hash for the enclave." How about the owner configures the root key hash instead? Along with deciding whether to lock down the feature control register or not...

Signature · 4 lines
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.
#55 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 7 lines · Borislav Petkov
Quoted 2 lines · Jarkko Sakkinen

Firmware cannot access the memory inside an enclave. CPU asserts every memory access coming outside the enclave.

But "firmware could potentially configure the root key hash for the enclave." How about the owner configures the root key hash instead? Along with deciding whether to lock down the feature control register or

In potential deployments of SGX, the owner could do this either in the firmware level or OS level depending whether the MSRs are configured as writable in the feature control.

One option would be to have a config flag to decide whether to require MSRs to be writable or not.

/Jarkko

maintainer
#56 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 6 lines · Jarkko Sakkinen

In potential deployments of SGX, the owner could do this either in the firmware level or OS level depending whether the MSRs are configured as writable in the feature control.

One option would be to have a config flag to decide whether to require MSRs to be writable or not.

"potential", "would", "could" - all carefully formulated. :-)

Realistically, though, I'm afraid OEMs would jump on the opportunity to control yet another arch aspect like wasps on honey. So having a way to override what the firmware decided for me - without even asking me - would be RealGood(tm).

Signature · 4 lines
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.
#57 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

In reply to Jarkko Sakkinen
Quoted 7 lines · Jarkko Sakkinen

In these cases IA32_FEATURE_CONTROL[17] would be zeroed before locking the feature control, which would mean that the kernel could not write new values with wrmsr for the root key hash.

The question is whether we want to allow this or not. If the answer is no, a check can be added to the driver initialization code whether 17 is set, and if not, it driver would fail to initialize.

That has my vote; I would not trust a firmware/BIOS key.

#58 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 10 lines · Peter Zijlstra
Quoted 7 lines · Jarkko Sakkinen

In these cases IA32_FEATURE_CONTROL[17] would be zeroed before locking the feature control, which would mean that the kernel could not write new values with wrmsr for the root key hash.

The question is whether we want to allow this or not. If the answer is no, a check can be added to the driver initialization code whether 17 is set, and if not, it driver would fail to initialize.

That has my vote; I would not trust a firmware/BIOS key.

Please note that it does not have a key to look inside the enclave. The enclave is protected by two means:

1. The CPU asserts the memory accesses to it.
2. The CPU encrypts/decrypts in L1 in order to protect from physical attacks and peripherals that have potential spy the bus.

/Jarkko

#59 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 10 lines · Jarkko Sakkinen
Quoted 9 lines · Peter Zijlstra
Quoted 7 lines · Jarkko Sakkinen

In these cases IA32_FEATURE_CONTROL[17] would be zeroed before locking the feature control, which would mean that the kernel could not write new values with wrmsr for the root key hash.

The question is whether we want to allow this or not. If the answer is no, a check can be added to the driver initialization code whether 17 is set, and if not, it driver would fail to initialize.

Quoted 8 lines
Quoted 1 line

That has my vote; I would not trust a firmware/BIOS key.

Please note that it does not have a key to look inside the enclave. The enclave is protected by two means:

1. The CPU asserts the memory accesses to it.
2. The CPU encrypts/decrypts in L1 in order to protect from physical attacks and peripherals that have potential spy the bus.

The encryption key is generated for every boot cycle and it is *never* leaked out of the CPU package.

After thinking what you and Borislav said I position myself to the point of view that even if the MSRs would be read-only the kernel could allow running enclaves *with a condition*.

I propose adding an additional check to the driver initialization:

Try to start LE. If it doesn't start i.e. is signed with a different root key than the one inside MSRs, then fail the initialization.

/Jarkko

maintainer
#60 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 2 lines · Jarkko Sakkinen

Try to start LE. If it doesn't start i.e. is signed with a different root key than the one inside MSRs, then fail the initialization.

But what if the one inside the MSRs is from the fw vendor and I don't trust it?

Signature · 4 lines
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.
#61 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 7 lines · Borislav Petkov
Quoted 2 lines · Jarkko Sakkinen

Try to start LE. If it doesn't start i.e. is signed with a different root key than the one inside MSRs, then fail the initialization.

But what if the one inside the MSRs is from the fw vendor and I don't trust it?

Boris & Peter: this key has nothing to do with "trust" or "security". As Sean mentioned in the other thread (PATCH v5 06/11) the kernel is fully capable of enforcing any security policy on its own without help of an enclave.

Quoted 1 line

I.e., let the owner really own the hardware she paid money for.

Yes, let's. Processors with SGX have been commercially available for over 2 years (and I have owned them for the same time). Why is it that I still can't use a mainline kernel to get access to all the cabapilities of my 2-year old hardware? Let's try not to put too much policy in the kernel and let userspace decide.

Jethro Beekman

maintainer
#62 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 1 line · Jethro Beekman

Boris & Peter: this key has nothing to do with "trust" or "security".

But with what? Why is the firmware at all involved then?

Signature · 4 lines
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.
#63 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 4 lines · Borislav Petkov
Quoted 1 line · Jethro Beekman

Boris & Peter: this key has nothing to do with "trust" or "security".

But with what? Why is the firmware at all involved then?

See http://www.spinics.net/lists/platform-driver-x86/msg13829.html under "Launch control". Essentially, firmware can make it so that user has no control over IA32_SGXLEPUBKEYHASHn value.

One comment on the documentation I linked:

Review

Quoted 2 lines
(quoted diff) +2 -0
+The BIOS can configure IA32_SGXLEPUBKEYHASHn MSRs before feature control
+register is locked.

This is not entirely accurate, hardware exists on the market today where IA32_SGXLEPUBKEYHASHn can't be configured, even by firmware. As mentioned in my previous email, I'd like to use said hardware.

Jethro Beekman

maintainer
#64 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 3 lines · Jethro Beekman

See http://www.spinics.net/lists/platform-driver-x86/msg13829.html under "Launch control". Essentially, firmware can make it so that user has no control over IA32_SGXLEPUBKEYHASHn value.

... and we're back full circle to my initial objection: firmware should not be doing anything here. The user should.

The mail you're quoting is the same subthread we are at.

Signature · 4 lines
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.
#65 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 2 lines · Borislav Petkov

... and we're back full circle to my initial objection: firmware should not be doing anything here. The user should.

Sure, and I agree. However, there is a difference between what it should be doing and what hardware/firmware that you can buy today **is** doing right now.

Jethro Beekman

#66 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

In reply to Jarkko Sakkinen

On Mon, Nov 13, 2017 at 09:45:28PM +0200, Jarkko Sakkinen wrote:

Please do not submit patches to LKML without a commit message. There is *always* something you can provide to give the review additional context to aid in their review of your code.

As Thomas has noted, the various maintainers have been omitted from this series. Always review the output of get_maintainers.pl and update the To and Cc list accordingly for every patch. See Documentation/process/submitting-patches.rst for details, e.g.

5) Select the recipients for your patch
12) When to use Acked-by: and Cc:

For example, for this patch:

$ scripts/get_maintainer.pl -f Documentation/x86
Thomas Gleixner <[email protected]> (maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT))
Ingo Molnar <[email protected]> (maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT))
"H. Peter Anvin" <[email protected]> (maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT))
[email protected] (maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT))
Jonathan Corbet <[email protected]> (maintainer:DOCUMENTATION)
[email protected] (open list:X86 ARCHITECTURE (32-BIT AND 64-BIT))
[email protected] (open list:DOCUMENTATION)

This series will need to be updated per the comments received so far, as well as with commit messages and a complete Cc list *per patch* giving all required parties an opportunity to review.

With respect to the obvious security nature of this series, who from the kernel security folks are going to be reviewing this? [email protected]?

Cc updated for this thread, and specifically the question regarding location below:

Quoted 7 lines
---
 Documentation/index.rst         |   1 +
 Documentation/x86/intel_sgx.rst | 131 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 132 insertions(+)
 create mode 100644 Documentation/x86/intel_sgx.rst

...

Quoted 20 lines
Documentation/x86/intel_sgx.rst new file +14 -0
new file mode 100644
index 000000000000..34bcf6a2a495
@@ -0,0 +1,131 @@
+===================
+Intel(R) SGX driver
+===================
+
+Introduction
+============
+
+Intel(R) SGX is a set of CPU instructions that can be used by applications to
+set aside private regions of code and data. The code outside the enclave is
+disallowed to access the memory inside the enclave by the CPU access control.
+In a way you can think that SGX provides inverted sandbox. It protects the
+application from a malicious host.
+
+There is a new hardware unit in the processor called Memory Encryption Engine

Documentation/x86/intel_sgx.rst line 24

Quoted 10 lines
Documentation/x86/intel_sgx.rst new file +10 -0
+(MEE) starting from the Skylake microarchitecture. BIOS can define one or many
+MEE regions that can hold enclave data by configuring them with PRMRR registers.
+
+The MEE automatically encrypts the data leaving the processor package to the MEE
+regions. The data is encrypted using a random key whose life-time is exactly one
+power cycle.
+
+You can tell if your CPU supports SGX by looking into ``/proc/cpuinfo``:
+
+	``cat /proc/cpuinfo  | grep sgx``

Is SGX considered architectural or not? A quick search of the SDM includes it in Volume 3:

Volume 3: Includes the full system programming guide, parts 1, 2, 3, and
4. Describes the operating-system support environment of Intel® 64 and IA-32 architectures, including: memory management, protection, task management, interrupt and exception handling, multi-processor support, thermal and power management features, debugging, performance monitoring, system management mode, virtual machine extensions (VMX) instructions, Intel® Virtualization Technology (Intel® VT), and Intel® Software Guard Extensions (Intel® SGX).

https://software.intel.com/en-us/articles/intel-sdm

Depending on the answer, this impacts whether this belongs in drivers/platform/x86 or arch/x86/platform per our recent agreement with Thomas.

Thomas, Mingo, HPA, do you wish to see this organized/located differently than it is here in v5?

Quoted 6 lines
Documentation/x86/intel_sgx.rst new file +6 -0
+Launch control
+==============
+
+For launching an enclave, two structures must be provided for ENCLS(EINIT):
+
+1. **SIGSTRUCT:** a signed measurement of the enclave binary.

Documentation/x86/intel_sgx.rst line 64

Quoted 10 lines
Documentation/x86/intel_sgx.rst new file +10 -0
+2. **EINITTOKEN:** the measurement, the public key of the signer and various
+   enclave attributes. This structure contains a MAC of its contents using
+   hardware derived symmetric key called *launch key*.
+
+The hardware platform contains a root key pair for signing the SIGTRUCT
+for a *launch enclave* that is able to acquire the *launch key* for
+creating EINITTOKEN's for other enclaves.  For the launch enclave
+EINITTOKEN is not needed because it is signed with the private root key.
+
+There are two feature control bits associate with launch control

Nit: missing colon at the end of the line above ^

Signature · 2 lines
Darren Hart
VMware Open Source Technology Center
#67 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

On Fri, 17 Nov 2017, Darren Hart wrote:

@intel: I removed [email protected] from CC because I can do without the silly moderation spam of that list. Please disable that nonsense.

Quoted 13 lines · Jarkko Sakkinen

Is SGX considered architectural or not? A quick search of the SDM includes it in Volume 3:

Volume 3: Includes the full system programming guide, parts 1, 2, 3, and
4. Describes the operating-system support environment of Intel® 64 and IA-32 architectures, including: memory management, protection, task management, interrupt and exception handling, multi-processor support, thermal and power management features, debugging, performance monitoring, system management mode, virtual machine extensions (VMX) instructions, Intel® Virtualization Technology (Intel® VT), and Intel® Software Guard Extensions (Intel® SGX).

Quoted 8 lines

https://software.intel.com/en-us/articles/intel-sdm

Depending on the answer, this impacts whether this belongs in drivers/platform/x86 or arch/x86/platform per our recent agreement with Thomas.

Thomas, Mingo, HPA, do you wish to see this organized/located differently than it is here in v5?

This is architecural. From the cursory read of that series it seems there are two parts to it:

1) The actual core handling, which should be in arch/x86 because that hardly qualifies as a 'platform' device driver.

2) The user space interface, which can be separated out perhaps.

I don't know how intertwingled they are, but that's hard to tell from the actual patches w/o doing a deep inspection. Jarkko should be able to answer that.

Thanks,

tglx

#68 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 25 lines · Thomas Gleixner

On Fri, 17 Nov 2017, Darren Hart wrote:

@intel: I removed [email protected] from CC because I can do without the silly moderation spam of that list. Please disable that nonsense.

Quoted 19 lines · Jarkko Sakkinen

Is SGX considered architectural or not? A quick search of the SDM includes it in Volume 3:

Volume 3: Includes the full system programming guide, parts 1, 2, 3, and
4. Describes the operating-system support environment of Intel® 64 and IA-32 architectures, including: memory management, protection, task management, interrupt and exception handling, multi-processor support, thermal and power management features, debugging, performance monitoring, system management mode, virtual machine extensions (VMX) instructions, Intel® Virtualization Technology (Intel® VT), and Intel® Software Guard Extensions (Intel® SGX).

https://software.intel.com/en-us/articles/intel-sdm

Depending on the answer, this impacts whether this belongs in drivers/platform/x86 or arch/x86/platform per our recent agreement with Thomas.

Quoted 9 lines
Quoted 2 lines

Thomas, Mingo, HPA, do you wish to see this organized/located differently than it is here in v5?

This is architecural. From the cursory read of that series it seems there are two parts to it:

1) The actual core handling, which should be in arch/x86 because that hardly qualifies as a 'platform' device driver.

I'm supportive of that.

Quoted 5 lines

2) The user space interface, which can be separated out perhaps.

I don't know how intertwingled they are, but that's hard to tell from the actual patches w/o doing a deep inspection. Jarkko should be able to answer that.

Jarkko, some additional context on your placement decisions would be helpful.

Thanks,

Signature · 2 lines
Darren Hart
VMware Open Source Technology Center
#69 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 38 lines · Darren Hart
Quoted 25 lines · Thomas Gleixner

On Fri, 17 Nov 2017, Darren Hart wrote:

@intel: I removed [email protected] from CC because I can do without the silly moderation spam of that list. Please disable that nonsense.

Quoted 19 lines · Jarkko Sakkinen

Is SGX considered architectural or not? A quick search of the SDM includes it in Volume 3:

Volume 3: Includes the full system programming guide, parts 1, 2, 3, and
4. Describes the operating-system support environment of Intel® 64 and IA-32 architectures, including: memory management, protection, task management, interrupt and exception handling, multi-processor support, thermal and power management features, debugging, performance monitoring, system management mode, virtual machine extensions (VMX) instructions, Intel® Virtualization Technology (Intel® VT), and Intel® Software Guard Extensions (Intel® SGX).

https://software.intel.com/en-us/articles/intel-sdm

Depending on the answer, this impacts whether this belongs in drivers/platform/x86 or arch/x86/platform per our recent agreement with Thomas.

Quoted 9 lines
Quoted 2 lines

Thomas, Mingo, HPA, do you wish to see this organized/located differently than it is here in v5?

This is architecural. From the cursory read of that series it seems there are two parts to it:

1) The actual core handling, which should be in arch/x86 because that hardly qualifies as a 'platform' device driver.

I'm supportive of that.

Quoted 7 lines
Quoted 5 lines

2) The user space interface, which can be separated out perhaps.

I don't know how intertwingled they are, but that's hard to tell from the actual patches w/o doing a deep inspection. Jarkko should be able to answer that.

Jarkko, some additional context on your placement decisions would be helpful.

tglx, Darren,

First thanks for your commit specific comments. I'll deal with them and appreciate the effort you've done reviewing the code. I'm just busy doing revamping sgx_page_cache.c based on the feedback I got from Dave Hansen. After I'm done with that I'll response to those emails. I'm sorry about the latency.

I gave some feedback to the tglx's original response. If something is lacking, please ping me.

/Jarkko

#70 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

In reply to Thomas Gleixner
Quoted 8 lines · Thomas Gleixner

This is architecural. From the cursory read of that series it seems there are two parts to it:

1) The actual core handling, which should be in arch/x86 because that hardly qualifies as a 'platform' device driver.

2) The user space interface, which can be separated out perhaps.

Quoted 7 lines

I don't know how intertwingled they are, but that's hard to tell from the actual patches w/o doing a deep inspection. Jarkko should be able to answer that.

Thanks,

tglx

Darren, tglx,

You can leave user space device as separate module as sgx_ioctl.c merely calls stuff that I have inside sgx_encl.c. VMA creation is bound to file operations.

My questions would be:

1. What is your recommendation on the deployment under arch/x86?
2. Which parts should be compilable as a LKM? Only the user interface or both parts?

/Jarkko

#71 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 23 lines · Jarkko Sakkinen
Quoted 9 lines · Thomas Gleixner

This is architecural. From the cursory read of that series it seems there are two parts to it:

  1) The actual core handling, which should be in arch/x86 because that hardly qualifies as a 'platform' device driver.

  2) The user space interface, which can be separated out perhaps.

Quoted 7 lines

I don't know how intertwingled they are, but that's hard to tell from the actual patches w/o doing a deep inspection. Jarkko should be able to answer that.

Thanks,

tglx

Darren, tglx,

You can leave user space device as separate module as sgx_ioctl.c merely calls stuff that I have inside sgx_encl.c. VMA creation is bound to file operations.

Quoted 7 lines

My questions would be:

1. What is your recommendation on the deployment under arch/x86?
2. Which parts should be compilable as a LKM? Only the user interface or both parts?

/Jarkko

To enable KVM and a cgroup for EPC accounting, at a minimum arch/x86 needs to manage the EPC pages (alloc/free/lrus/reclaim/etc...) and LE hash MSRs.  IMO, ideally everything else would be left in the device driver, e.g. anything involving ENCLS.  Keeping the majority of the driver out of arch/x86 minimizes the footprint in arch/x86 and thereby the size of KVM's dependency required to virtualize SGX, and allows the various SGX pieces, e.g. arch, driver and KVM, to evolve more independently.

Preferably the arch/x86 code would not be a loadable module, e.g. to simplify KVM support.

I have a branch based on Jarkko's patches (I believe it's up-to-date with v5) that implements what I described.  I'd be happy to send RFC patches if that would help.

Branches for those interested:

https://github.com/sean-jc/linux.git sgx/arch   - move core EPC to arch/x86
https://github.com/sean-jc/linux.git sgx/kvm    - KVM support for SGX
https://github.com/sean-jc/linux.git sgx/lc     - KVM support for Launch Control
https://github.com/sean-jc/linux.git sgx/cgroup - EPC cgroup

branch relationships:

    Jarkko's patches
            |
            |
         sgx/arch
        /        \
     sgx/kvm    sgx/cgroup
      /
   sgx/lc

#72 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

+ Cc: KVM, Paolo and Radim

Quoted 69 lines · Sean Christopherson
Quoted 26 lines · Jarkko Sakkinen
Quoted 10 lines · Thomas Gleixner

This is architecural. From the cursory read of that series it seems there are two parts to it:

  1) The actual core handling, which should be in arch/x86 because that hardly qualifies as a 'platform' device driver.

  2) The user space interface, which can be separated out perhaps.

Quoted 8 lines

I don't know how intertwingled they are, but that's hard to tell from the actual patches w/o doing a deep inspection. Jarkko should be able to answer
that.

Thanks,

tglx

Darren, tglx,

You can leave user space device as separate module as sgx_ioctl.c merely calls stuff that I have inside sgx_encl.c. VMA creation is bound to file operations.

Quoted 7 lines

My questions would be:

1. What is your recommendation on the deployment under arch/x86?
2. Which parts should be compilable as a LKM? Only the user interface or both parts?

/Jarkko

To enable KVM and a cgroup for EPC accounting, at a minimum arch/x86 needs to manage the EPC pages (alloc/free/lrus/reclaim/etc...) and LE hash MSRs.  IMO, ideally everything else would be left in the device driver, e.g. anything involving ENCLS.  Keeping the majority of the driver out of arch/x86 minimizes the footprint in arch/x86 and thereby the size of KVM's dependency required to virtualize SGX, and allows the various SGX pieces, e.g. arch, driver and KVM, to evolve more independently.

Preferably the arch/x86 code would not be a loadable module, e.g. to simplify KVM support.

I have a branch based on Jarkko's patches (I believe it's up-to-date with v5) that implements what I described.  I'd be happy to send RFC patches if that would help.

Branches for those interested:

https://github.com/sean-jc/linux.git sgx/arch   - move core EPC to arch/x86
https://github.com/sean-jc/linux.git sgx/kvm    - KVM support for SGX
https://github.com/sean-jc/linux.git sgx/lc     - KVM support for Launch Control
https://github.com/sean-jc/linux.git sgx/cgroup - EPC cgroup

branch relationships:

    Jarkko's patches
            |
            |
         sgx/arch
        /        \
     sgx/kvm    sgx/cgroup
      /
   sgx/lc

#73 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

In reply to Sean Christopherson
Quoted 3 lines · Sean Christopherson

I have a branch based on Jarkko's patches (I believe it's up-to-date with v5) that implements what I described.  I'd be happy to send RFC patches if that would help.

That would only slow things down. The code is easy to move around and I'm doing infrastructure changes as part of the review process. The latest version (v6) that I sent on Sat has struct sgx_epc_page removed just to name an example.

Rather than sending any deprecated patches it would be more useful to get input (in English) on directory layout.

I guess you missed v6 as it I had to drop the 01.org list temporarily. It will be back in v7 as I was able to retrieve the admin password and configure it in suitable way.

/Jarkko

#74 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

Quoted 10 lines · Jarkko Sakkinen
Quoted 3 lines · Sean Christopherson

I have a branch based on Jarkko's patches (I believe it's up-to-date with v5) that implements what I described.  I'd be happy to send RFC patches if that would help.

That would only slow things down. The code is easy to move around and I'm doing infrastructure changes as part of the review process. The latest version (v6) that I sent on Sat has struct sgx_epc_page removed just to name an example.

Quoted 6 lines

Rather than sending any deprecated patches it would be more useful to get input (in English) on directory layout.

I guess you missed v6 as it I had to drop the 01.org list temporarily. It will be back in v7 as I was able to retrieve the admin password and configure it in suitable way.

Once my series has been landed, you could drop a series for review. Then it does make sense. I won't of course add any exports needed by KVM etc. I'm only doing "lowest common denominator" groundwork.

/Jarkko

#75 lore raw

Re: [PATCH v5 11/11] intel_sgx: driver documentation

In reply to Darren Hart
Quoted 7 lines · Darren Hart

This series will need to be updated per the comments received so far, as well as with commit messages and a complete Cc list *per patch* giving all required parties an opportunity to review.

With respect to the obvious security nature of this series, who from the kernel security folks are going to be reviewing this? [email protected]?

I think it would make sense to CC this to linux-security module.

Quoted 56 lines

Cc updated for this thread, and specifically the question regarding location below:

Quoted 7 lines
---
 Documentation/index.rst         |   1 +
 Documentation/x86/intel_sgx.rst | 131 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 132 insertions(+)
 create mode 100644 Documentation/x86/intel_sgx.rst

...

Quoted 20 lines
Documentation/x86/intel_sgx.rst new file +14 -0
new file mode 100644
index 000000000000..34bcf6a2a495
@@ -0,0 +1,131 @@
+===================
+Intel(R) SGX driver
+===================
+
+Introduction
+============
+
+Intel(R) SGX is a set of CPU instructions that can be used by applications to
+set aside private regions of code and data. The code outside the enclave is
+disallowed to access the memory inside the enclave by the CPU access control.
+In a way you can think that SGX provides inverted sandbox. It protects the
+application from a malicious host.
+
+There is a new hardware unit in the processor called Memory Encryption Engine

Documentation/x86/intel_sgx.rst line 24

Quoted 10 lines
Documentation/x86/intel_sgx.rst new file +10 -0
+(MEE) starting from the Skylake microarchitecture. BIOS can define one or many
+MEE regions that can hold enclave data by configuring them with PRMRR registers.
+
+The MEE automatically encrypts the data leaving the processor package to the MEE
+regions. The data is encrypted using a random key whose life-time is exactly one
+power cycle.
+
+You can tell if your CPU supports SGX by looking into ``/proc/cpuinfo``:
+
+	``cat /proc/cpuinfo  | grep sgx``

Is SGX considered architectural or not? A quick search of the SDM includes it in Volume 3:

Volume 3: Includes the full system programming guide, parts 1, 2, 3, and
4. Describes the operating-system support environment of Intel® 64 and IA-32 architectures, including: memory management, protection, task management, interrupt and exception handling, multi-processor support, thermal and power management features, debugging, performance monitoring, system management mode, virtual machine extensions (VMX) instructions, Intel® Virtualization Technology (Intel® VT), and Intel® Software Guard Extensions (Intel® SGX).

Quoted 8 lines

https://software.intel.com/en-us/articles/intel-sdm

Depending on the answer, this impacts whether this belongs in drivers/platform/x86 or arch/x86/platform per our recent agreement with Thomas.

Thomas, Mingo, HPA, do you wish to see this organized/located differently than it is here in v5?

The code is made easily relocatable. I just wanted to keep it as an encapsulated driver up until I hear the maintainer feedback. I'll submit v6 with code otherwise fixed according to the feedback that I've heard up until that point and relocate it in v7 based on your feedback.

Quoted 10 lines
Quoted 10 lines
Documentation/x86/intel_sgx.rst new file +10 -0
+Launch control
+==============
+
+For launching an enclave, two structures must be provided for ENCLS(EINIT):
+
+1. **SIGSTRUCT:** a signed measurement of the enclave binary.
+2. **EINITTOKEN:** the measurement, the public key of the signer and various
+   enclave attributes. This structure contains a MAC of its contents using
+   hardware derived symmetric key called *launch key*.
+
Quoted 8 lines

Documentation/x86/intel_sgx.rst line 64

Quoted 6 lines
Documentation/x86/intel_sgx.rst new file +6 -0
+The hardware platform contains a root key pair for signing the SIGTRUCT
+for a *launch enclave* that is able to acquire the *launch key* for
+creating EINITTOKEN's for other enclaves.  For the launch enclave
+EINITTOKEN is not needed because it is signed with the private root key.
+
+There are two feature control bits associate with launch control

Nit: missing colon at the end of the line above ^

Yes. Thanks for spotting that out :-)

/Jarkko

#76 lore raw
In reply to Jarkko Sakkinen

On Mon, 13 Nov 2017, Jarkko Sakkinen wrote:

Any reason why you did not CC the x86 maintainers and other x86 developers who have had opinions on the first attempt to get SGX into the kernel?

This is not some driver for a random peripheral. This has substantional effects. I'm assuming you simply forgot or were not aware, but this is not the way this works.

Thanks,

tglx

#77 lore raw
Quoted 5 lines · Thomas Gleixner

On Mon, 13 Nov 2017, Jarkko Sakkinen wrote:

Any reason why you did not CC the x86 maintainers and other x86 developers who have had opinions on the first attempt to get SGX into the kernel?

Quoted 7 lines

This is not some driver for a random peripheral. This has substantional effects. I'm assuming you simply forgot or were not aware, but this is not the way this works.

Thanks,

tglx

I'll CC v6 to x86 maintainers. I'm sorry about this.

/Jarkko