Intel(R) SGX Driver
Subsystems (4)
v3 cover Intel(R) SGX Driver
Intel(R) SGX is a set of CPU instructions that can be used by applications to set aside private regions of code and data. The code outside the enclave is disallowed to access the memory inside the enclave by the CPU access control. In a way you can think that SGX provides inverted sandbox. It protects the application from a malicious host.
There is a new hardware unit in the processor called Memory Encryption Engine (MEE) starting from the Skylake microachitecture. BIOS can define one or many MEE regions that can hold enclave data by configuring them with PRMRR registers.
The MEE automatically encrypts the data leaving the processor package to the MEE regions. The data is encrypted using a random key whose life-time is exactly one power cycle.
You can tell if your CPU supports SGX by looking into ``/proc/cpuinfo``:
``cat /proc/cpuinfo | grep ' sgx '``
The GIT repositoy for SGX driver resides in
https://github.com/jsakkine-intel/linux-sgx.git
'le' branch contains the upstream candidate patches.
'master' branch contains the same patches with the following differences:
* top-level patch modifies the ioctl API to be SDK compatible
* does not use flexible launch control but instead relies on SDK provided Intel launch enclave.
'master' is just maintained to allow people to test the code.
v3:
* Check that FEATURE_CONTROL_LOCKED and FEATURE_CONTROL_SGX_ENABLE are set.
* Return -ERESTARTSYS in __sgx_encl_add_page() when sgx_alloc_page() fails.
* Use unused bits in epc_page->pa to store the bank number.
* Removed #ifdef for WQ_NONREENTRANT.
* If mmu_notifier_register() fails with -EINTR, return -ERESTARTSYS.
* Added --remove-section=.got.plt to objcopy flags in order to prevent a dummy .got.plt, which will cause an inconsistent size for the LE.
* Documented sgx_encl_* functions.
* Added remark about AES implementation used inside the LE.
* Removed redundant sgx_sys_exit() from le/main.c.
* Fixed struct sgx_secinfo alignment from 128 to 64 bytes.
* Validate miscselect in sgx_encl_create().
* Fixed SSA frame size calculation to take the misc region into account.
* Implemented consistent exception handling to __encls() and __encls_ret().
* Implemented a proper device model in order to allow sysfs attributes and in-kernel API.
* Cleaned up various "find enclave" implementations to the unified sgx_encl_find().
* Validate that vm_pgoff is zero.
* Discard backing pages with shmem_truncate_range() after EADD.
* Added missing EEXTEND operations to LE signing and launch.
* Fixed SSA size for GPRS region from 168 to 184 bytes.
* Fixed the checks for TCS flags. Now DBGOPTIN is allowed.
* Check that TCS addresses are in ELRANGE and not just page aligned.
* Require kernel to be compiled with X64_64 and CPU_SUP_INTEL.
* Fixed an incorrect value for SGX_ATTR_DEBUG from 0x01 to 0x02.
v2:
* get_rand_uint32() changed the value of the pointer instead of value where it is pointing at.
* Launch enclave incorrectly used sigstruct attributes-field instead of enclave attributes-field.
* Removed unused struct sgx_add_page_req from sgx_ioctl.c
* Removed unused sgx_has_sgx2.
* Updated arch/x86/include/asm/sgx.h so that it provides stub implementations when sgx in not enabled.
* Removed cruft rdmsr-calls from sgx_set_pubkeyhash_msrs().
* return -ENOMEM in sgx_alloc_page() when VA pages consume too much space
* removed unused global sgx_nr_pids
* moved sgx_encl_release to sgx_encl.c
* return -ERESTARTSYS instead of -EINTR in sgx_encl_init()
Haim Cohen (1):
x86: add SGX MSRs to msr-index.h
Jarkko Sakkinen (9):
intel_sgx: updated MAINTAINERS
x86: define the feature control MSR's SGX launch control bit fs/pipe.c: export create_pipe_files() and replace_fd()
intel_sgx: driver for Intel Software Guard Extensions
intel_sgx: ptrace() support
intel_sgx: driver documentation
intel_sgx: in-kernel launch enclave
intel_sgx: glue code for in-kernel LE
intel_sgx: update IA32_SGXLEPUBKEYHASH* MSRs
Kai Huang (1):
x86: add SGX definition to cpufeature
Sean Christopherson (1):
x86: define the feature control MSR's SGX enable bit
Documentation/index.rst | 1 + Documentation/x86/intel_sgx.rst | 131 +++ MAINTAINERS | 5 + arch/x86/include/asm/cpufeatures.h | 2 + arch/x86/include/asm/msr-index.h | 8 + arch/x86/include/asm/sgx.h | 233 +++++ arch/x86/include/asm/sgx_arch.h | 268 ++++++ arch/x86/include/uapi/asm/sgx.h | 147 +++ drivers/platform/x86/Kconfig | 2 + drivers/platform/x86/Makefile | 1 + drivers/platform/x86/intel_sgx/Kconfig | 34 + drivers/platform/x86/intel_sgx/Makefile | 32 + drivers/platform/x86/intel_sgx/le/Makefile | 26 + drivers/platform/x86/intel_sgx/le/enclave/Makefile | 46 + .../x86/intel_sgx/le/enclave/aes_encrypt.c | 191 ++++ .../platform/x86/intel_sgx/le/enclave/cmac_mode.c | 254 +++++ .../x86/intel_sgx/le/enclave/encl_bootstrap.S | 163 ++++ .../intel_sgx/le/enclave/include/tinycrypt/aes.h | 133 +++ .../le/enclave/include/tinycrypt/cmac_mode.h | 194 ++++ .../le/enclave/include/tinycrypt/constants.h | 59 ++ .../intel_sgx/le/enclave/include/tinycrypt/utils.h | 95 ++ drivers/platform/x86/intel_sgx/le/enclave/main.c | 203 ++++ .../platform/x86/intel_sgx/le/enclave/sgx_le.lds | 28 + .../platform/x86/intel_sgx/le/enclave/sgxsign.c | 537 +++++++++++ drivers/platform/x86/intel_sgx/le/enclave/utils.c | 78 ++ drivers/platform/x86/intel_sgx/le/entry.S | 117 +++ .../platform/x86/intel_sgx/le/include/sgx_asm.h | 64 ++ .../platform/x86/intel_sgx/le/include/sgx_encl.h | 105 ++ drivers/platform/x86/intel_sgx/le/main.c | 219 +++++ drivers/platform/x86/intel_sgx/le/sgx_le_piggy.S | 15 + drivers/platform/x86/intel_sgx/sgx.h | 264 +++++ drivers/platform/x86/intel_sgx/sgx_encl.c | 1009 ++++++++++++++++++++ drivers/platform/x86/intel_sgx/sgx_ioctl.c | 282 ++++++ drivers/platform/x86/intel_sgx/sgx_le.c | 290 ++++++ .../platform/x86/intel_sgx/sgx_le_proxy_piggy.S | 15 + drivers/platform/x86/intel_sgx/sgx_main.c | 475 +++++++++ drivers/platform/x86/intel_sgx/sgx_page_cache.c | 586 ++++++++++++ drivers/platform/x86/intel_sgx/sgx_util.c | 396 ++++++++ drivers/platform/x86/intel_sgx/sgx_vma.c | 230 +++++ fs/file.c | 1 + fs/pipe.c | 1 + 41 files changed, 6940 insertions(+) create mode 100644 Documentation/x86/intel_sgx.rst create mode 100644 arch/x86/include/asm/sgx.h create mode 100644 arch/x86/include/asm/sgx_arch.h create mode 100644 arch/x86/include/uapi/asm/sgx.h create mode 100644 drivers/platform/x86/intel_sgx/Kconfig create mode 100644 drivers/platform/x86/intel_sgx/Makefile create mode 100644 drivers/platform/x86/intel_sgx/le/Makefile create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/Makefile create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/aes_encrypt.c create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/cmac_mode.c create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/encl_bootstrap.S create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/aes.h create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/cmac_mode.h create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/constants.h create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/utils.h create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/main.c create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/sgx_le.lds create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/sgxsign.c create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/utils.c create mode 100644 drivers/platform/x86/intel_sgx/le/entry.S create mode 100644 drivers/platform/x86/intel_sgx/le/include/sgx_asm.h create mode 100644 drivers/platform/x86/intel_sgx/le/include/sgx_encl.h create mode 100644 drivers/platform/x86/intel_sgx/le/main.c create mode 100644 drivers/platform/x86/intel_sgx/le/sgx_le_piggy.S create mode 100644 drivers/platform/x86/intel_sgx/sgx.h create mode 100644 drivers/platform/x86/intel_sgx/sgx_encl.c create mode 100644 drivers/platform/x86/intel_sgx/sgx_ioctl.c create mode 100644 drivers/platform/x86/intel_sgx/sgx_le.c create mode 100644 drivers/platform/x86/intel_sgx/sgx_le_proxy_piggy.S create mode 100644 drivers/platform/x86/intel_sgx/sgx_main.c create mode 100644 drivers/platform/x86/intel_sgx/sgx_page_cache.c create mode 100644 drivers/platform/x86/intel_sgx/sgx_util.c create mode 100644 drivers/platform/x86/intel_sgx/sgx_vma.c
Signature · 1 line
2.14.1
RFC PATCH v3 1/12 intel_sgx: updated MAINTAINERS
--- MAINTAINERS | 5 +++++ 1 file changed, 5 insertions(+)
MAINTAINERS +5 -0
@@ -14932,6 +14932,11 @@ L: [email protected] S: Maintained F: mm/zswap.c +INTEL SGX +M: Jarkko Sakkinen <[email protected]> +L: [email protected] +Q: https://patchwork.kernel.org/project/intel-sgx/list/ + THE REST M: Linus Torvalds <[email protected]> L: [email protected]
Signature · 1 line
2.14.1
RFC PATCH v3 2/12 x86: add SGX definition to cpufeature
In reply to Jarkko SakkinenFrom: Kai Huang <[email protected]>
--- arch/x86/include/asm/cpufeatures.h | 1 + 1 file changed, 1 insertion(+)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -219,6 +219,7 @@/* Intel-defined CPU features, CPUID level 0x00000007:0 (ebx), word 9 */ #define X86_FEATURE_FSGSBASE ( 9*32+ 0)/* {RD/WR}{FS/GS}BASE instructions*/ #define X86_FEATURE_TSC_ADJUST ( 9*32+ 1)/* TSC adjustment MSR 0x3b */ +#define X86_FEATURE_SGX ( 9*32+ 2)/* Software Guard Extensions */ #define X86_FEATURE_BMI1 ( 9*32+ 3)/* 1st group bit manipulation extensions */ #define X86_FEATURE_HLE ( 9*32+ 4)/* Hardware Lock Elision */ #define X86_FEATURE_AVX2 ( 9*32+ 5)/* AVX2 instructions */
Signature · 1 line
2.14.1
RFC PATCH v3 3/12 x86: define the feature control MSR's SGX enable bit
In reply to Jarkko SakkinenFrom: Sean Christopherson <[email protected]>
--- arch/x86/include/asm/msr-index.h | 1 + 1 file changed, 1 insertion(+)
arch/x86/include/asm/msr-index.h +1 -0
@@ -435,6 +435,7 @@#define FEATURE_CONTROL_LOCKED (1<<0)#define FEATURE_CONTROL_VMXON_ENABLED_INSIDE_SMX (1<<1)#define FEATURE_CONTROL_VMXON_ENABLED_OUTSIDE_SMX (1<<2) +#define FEATURE_CONTROL_SGX_ENABLE (1<<18)#define FEATURE_CONTROL_LMCE (1<<20)#define MSR_IA32_APICBASE 0x0000001b
Signature · 1 line
2.14.1
RFC PATCH v3 4/12 x86: define the feature control MSR's SGX launch control bit
In reply to Jarkko Sakkinen--- arch/x86/include/asm/cpufeatures.h | 1 + 1 file changed, 1 insertion(+)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -298,6 +298,7 @@#define X86_FEATURE_AVX512_VPOPCNTDQ (16*32+14)/* POPCNT for vectors of DW/QW */ #define X86_FEATURE_LA57 (16*32+16)/* 5-level page tables */ #define X86_FEATURE_RDPID (16*32+22)/* RDPID instruction */ +#define X86_FEATURE_SGX_LC (16*32+30)/* supports SGX launch configuration */ /* AMD-defined CPU features, CPUID level 0x80000007 (ebx), word 17 */ #define X86_FEATURE_OVERFLOW_RECOV (17*32+0)/* MCA overflow recovery support */
Signature · 1 line
2.14.1
RFC PATCH v3 5/12 x86: add SGX MSRs to msr-index.h
In reply to Jarkko SakkinenFrom: Haim Cohen <[email protected]>
These MSRs hold the SHA256 checksum of the currently configured launch enclave public key.
--- arch/x86/include/asm/msr-index.h | 7 +++++++ 1 file changed, 7 insertions(+)
arch/x86/include/asm/msr-index.h +7 -0
@@ -436,6 +436,7 @@#define FEATURE_CONTROL_VMXON_ENABLED_INSIDE_SMX (1<<1)#define FEATURE_CONTROL_VMXON_ENABLED_OUTSIDE_SMX (1<<2)#define FEATURE_CONTROL_SGX_ENABLE (1<<18) +#define FEATURE_CONTROL_SGX_LAUNCH_CONTROL_ENABLE (1<<17)#define FEATURE_CONTROL_LMCE (1<<20)#define MSR_IA32_APICBASE 0x0000001b
@@ -502,6 +503,12 @@#define PACKAGE_THERM_INT_LOW_ENABLE (1 << 1)#define PACKAGE_THERM_INT_PLN_ENABLE (1 << 24) +/* Intel SGX MSRs */ +#define MSR_IA32_SGXLEPUBKEYHASH0 0x0000008C +#define MSR_IA32_SGXLEPUBKEYHASH1 0x0000008D +#define MSR_IA32_SGXLEPUBKEYHASH2 0x0000008E +#define MSR_IA32_SGXLEPUBKEYHASH3 0x0000008F +/* Thermal Thresholds Support */ #define THERM_INT_THRESHOLD0_ENABLE (1 << 15)#define THERM_SHIFT_THRESHOLD0 8
Signature · 1 line
2.14.1
RFC PATCH v3 6/12 fs/pipe.c: export create_pipe_files() and replace_fd()
In reply to Jarkko SakkinenExport create_pipe_files() and replace_fd() so that the SGX driver is able to create stdin and stdout pipes.
--- fs/file.c | 1 + fs/pipe.c | 1 + 2 files changed, 2 insertions(+)
fs/file.c +1 -0
@@ -871,6 +871,7 @@ int replace_fd(unsigned fd, struct file *file, unsigned flags)spin_unlock (&files->file_lock);return err; } +EXPORT_SYMBOL_GPL (replace_fd); SYSCALL_DEFINE3(dup3, unsigned int, oldfd, unsigned int, newfd, int, flags) {
fs/pipe.c +1 -0
@@ -784,6 +784,7 @@ int create_pipe_files(struct file **res, int flags)iput (inode);return err; } +EXPORT_SYMBOL_GPL (create_pipe_files);static int __do_pipe_flags (int *fd,struct file **files,int flags) {
Signature · 1 line
2.14.1
RFC PATCH v3 8/12 intel_sgx: ptrace() support
In reply to Jarkko SakkinenImplemented VMA callbacks in order to ptrace() debug enclaves.
--- drivers/platform/x86/intel_sgx/sgx_vma.c | 115 +++++++++++++++++++++++++++++++ 1 file changed, 115 insertions(+)
drivers/platform/x86/intel_sgx/sgx_vma.c +115 -0
@@ -108,8 +108,123 @@ static int sgx_vma_fault(struct vm_fault *vmf)return VM_FAULT_SIGBUS ; } +static inline int sgx_vma_access_word (struct sgx_encl *encl, +unsigned long addr, +void *buf, +int len, +int write, +struct sgx_encl_page *encl_page, +int i) +{ +char data[sizeof (unsigned long )]; +int align, cnt, offset; +void *vaddr; +int ret; + + offset = ((addr + i) & (PAGE_SIZE -1 )) & ~(sizeof (unsigned long ) -1 ); + align = (addr + i) & (sizeof (unsigned long ) -1 ); + cnt =sizeof (unsigned long ) - align; + cnt =min (cnt, len - i); + +if (write) { +if (encl_page->flags &SGX_ENCL_PAGE_TCS && + (offset <8 || (offset + (len - i)) >16 )) +return -ECANCELED ; + +if (align || (cnt !=sizeof (unsigned long ))) { + vaddr =sgx_get_page (encl_page->epc_page); + ret =__edbgrd ((void *)((unsigned long )vaddr + offset), + (unsigned long *)data); +sgx_put_page (vaddr); +if (ret) { +sgx_dbg (encl,"EDBGRD returned %d\n" , ret); +return -EFAULT ; + } + } + +memcpy (data + align, buf + i, cnt); + vaddr =sgx_get_page (encl_page->epc_page); + ret =__edbgwr ((void *)((unsigned long )vaddr + offset), + (unsigned long *)data); +sgx_put_page (vaddr); +if (ret) { +sgx_dbg (encl,"EDBGWR returned %d\n" , ret); +return -EFAULT ; + } + }else { +if (encl_page->flags &SGX_ENCL_PAGE_TCS && + (offset + (len - i)) >72 ) +return -ECANCELED ; + + vaddr =sgx_get_page (encl_page->epc_page); + ret =__edbgrd ((void *)((unsigned long )vaddr + offset), + (unsigned long *)data); +sgx_put_page (vaddr); +if (ret) { +sgx_dbg (encl,"EDBGRD returned %d\n" , ret); +return -EFAULT ; + } + +memcpy (buf + i, data + align, cnt); + } + +return cnt; +} + +static int sgx_vma_access (struct vm_area_struct *vma,unsigned long addr, +void *buf,int len,int write) +{ +struct sgx_encl *encl = vma->vm_private_data; +struct sgx_encl_page *entry =NULL ; +const char *op_str = write ?"EDBGWR" :"EDBGRD" ; +int ret =0 ; +int i; + +/* If process was forked, VMA is still there but vm_private_data is set +* to NULL. +*/ +if (!encl) +return -EFAULT ; + +if (!(encl->flags &SGX_ENCL_DEBUG ) || + !(encl->flags &SGX_ENCL_INITIALIZED ) || + (encl->flags &SGX_ENCL_DEAD )) +return -EFAULT ; + +sgx_dbg (encl,"%s addr=0x%lx, len=%d\n" , op_str, addr, len); + +for (i =0 ; i < len; i += ret) { +if (!entry || !((addr + i) & (PAGE_SIZE -1 ))) { +if (entry) + entry->flags &= ~SGX_ENCL_PAGE_RESERVED ; + + entry =sgx_fault_page (vma, (addr + i) &PAGE_MASK , +SGX_FAULT_RESERVE ); +if (IS_ERR (entry)) { + ret =PTR_ERR (entry); + entry =NULL ; +break ; + } + } + +/* No locks are needed because used fields are immutable after +* intialization. +*/ + ret =sgx_vma_access_word (encl, addr, buf, len, write, + entry, i); +if (ret <0 ) +break ; + } + +if (entry) + entry->flags &= ~SGX_ENCL_PAGE_RESERVED ; + +return (ret <0 && ret != -ECANCELED ) ? ret : i; +} +const struct vm_operations_struct sgx_vm_ops = { .close = sgx_vma_close, .open = sgx_vma_open, .fault = sgx_vma_fault, + .access = sgx_vma_access, };
Signature · 1 line
2.14.1
RFC PATCH v3 9/12 intel_sgx: driver documentation
In reply to Jarkko Sakkinen--- Documentation/index.rst | 1 + Documentation/x86/intel_sgx.rst | 131 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 132 insertions(+) create mode 100644 Documentation/x86/intel_sgx.rst
Documentation/index.rst +1 -0
@@ -86,6 +86,7 @@ implementation. :maxdepth: 2 sh/index + x86/index Korean translations -------------------
Documentation/x86/intel_sgx.rst new file +131 -0
131 lines not on this page · raw message
Signature · 1 line
2.14.1
RFC PATCH v3 10/12 intel_sgx: in-kernel launch enclave
In reply to Jarkko SakkinenThis commits implements the in-kernel launch enclave. It is wrapped into a user space program that reads SIGSTRUCT instances from stdin and outputs launch tokens to stdout.
The commit also adds enclave signing tool that is used by kbuild to measure and sign the lauch enclave.
CONFIG_INTEL_SGX_SIGNING_KEY points to a PEM-file for the 3072-bit RSA key that is used as the LE public key pair. The default location is:
drivers/platform/x86/intel_sgx/intel_sgx_signing_key.pem
If the default key does not exist kbuild will generate a random key and place it to this location. KBUILD_SGX_SIGN_PIN can be used to specify the passphrase for the LE public key.
TinyCrypt (https://github.com/01org/tinycrypt) is used as AES implementation, which is not timing resistant. Eventually this needs to be replaced with AES-NI based implementation that could be either
- re-use existing AES-NI code in the kernel
- have its own hand written code
--- drivers/platform/x86/intel_sgx/Kconfig | 13 + drivers/platform/x86/intel_sgx/Makefile | 18 + drivers/platform/x86/intel_sgx/le/Makefile | 26 + drivers/platform/x86/intel_sgx/le/enclave/Makefile | 46 ++ .../x86/intel_sgx/le/enclave/aes_encrypt.c | 191 ++++++++ .../platform/x86/intel_sgx/le/enclave/cmac_mode.c | 254 ++++++++++ .../x86/intel_sgx/le/enclave/encl_bootstrap.S | 163 +++++++ .../intel_sgx/le/enclave/include/tinycrypt/aes.h | 133 +++++ .../le/enclave/include/tinycrypt/cmac_mode.h | 194 ++++++++ .../le/enclave/include/tinycrypt/constants.h | 59 +++ .../intel_sgx/le/enclave/include/tinycrypt/utils.h | 95 ++++ drivers/platform/x86/intel_sgx/le/enclave/main.c | 203 ++++++++ .../platform/x86/intel_sgx/le/enclave/sgx_le.lds | 28 ++ .../platform/x86/intel_sgx/le/enclave/sgxsign.c | 537 +++++++++++++++++++++ drivers/platform/x86/intel_sgx/le/enclave/utils.c | 78 +++ drivers/platform/x86/intel_sgx/le/entry.S | 117 +++++ .../platform/x86/intel_sgx/le/include/sgx_asm.h | 64 +++ .../platform/x86/intel_sgx/le/include/sgx_encl.h | 105 ++++ drivers/platform/x86/intel_sgx/le/main.c | 219 +++++++++ drivers/platform/x86/intel_sgx/le/sgx_le_piggy.S | 15 + .../platform/x86/intel_sgx/sgx_le_proxy_piggy.S | 11 + 21 files changed, 2569 insertions(+) create mode 100644 drivers/platform/x86/intel_sgx/le/Makefile create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/Makefile create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/aes_encrypt.c create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/cmac_mode.c create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/encl_bootstrap.S create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/aes.h create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/cmac_mode.h create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/constants.h create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/utils.h create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/main.c create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/sgx_le.lds create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/sgxsign.c create mode 100644 drivers/platform/x86/intel_sgx/le/enclave/utils.c create mode 100644 drivers/platform/x86/intel_sgx/le/entry.S create mode 100644 drivers/platform/x86/intel_sgx/le/include/sgx_asm.h create mode 100644 drivers/platform/x86/intel_sgx/le/include/sgx_encl.h create mode 100644 drivers/platform/x86/intel_sgx/le/main.c create mode 100644 drivers/platform/x86/intel_sgx/le/sgx_le_piggy.S create mode 100644 drivers/platform/x86/intel_sgx/sgx_le_proxy_piggy.S
drivers/platform/x86/intel_sgx/Kconfig +13 -0
@@ -2,6 +2,8 @@ # Intel SGX # +menu "Intel SGX" + config INTEL_SGX tristate "Intel(R) SGX Driver" default n
@@ -17,3 +19,14 @@ config INTEL_SGX called Enclave Page Cache (EPC). There is a hardware unit in the processor called Memory Encryption Engine. The MEE encrypts and decrypts the EPC pages as they enter and leave the processor package. + + +config INTEL_SGX_SIGNING_KEY + string "Path to the Intel SGX LE signing key" + default "drivers/platform/x86/intel_sgx/intel_sgx_signing_key.pem" + depends on INTEL_SGX + ---help--- + Provide a path to a 3072-bit RSA private key that will be used to + sign the launch enclave. + +endmenu
drivers/platform/x86/intel_sgx/Makefile +18 -0
@@ -11,3 +11,21 @@ intel_sgx-$(CONFIG_INTEL_SGX) += \ sgx_page_cache.o \ sgx_util.o \ sgx_vma.o \ + sgx_le_proxy_piggy.o + +$(eval $(call config_filename,INTEL_SGX_SIGNING_KEY)) + +INTEL_SGX_SIGNING_KEY_PATH := \ + $(INTEL_SGX_SIGNING_KEY_SRCPREFIX)$(INTEL_SGX_SIGNING_KEY_FILENAME) + +ifeq ($(CONFIG_INTEL_SGX_SIGNING_KEY),"drivers/platform/x86/intel_sgx/intel_sgx_signing_key.pem") +$(INTEL_SGX_SIGNING_KEY_PATH): + $(Q)openssl genrsa -3 -out $(INTEL_SGX_SIGNING_KEY_PATH) 3072 +endif + +$(obj)/sgx_le_proxy_piggy.o: $(INTEL_SGX_SIGNING_KEY_PATH) \ + $(obj)/le/sgx_le_proxy +$(obj)/le/sgx_le_proxy: FORCE + $(Q)$(MAKE) $(build)=$(obj)/le $@ + +export INTEL_SGX_SIGNING_KEY_PATH
drivers/platform/x86/intel_sgx/le/Makefile new file +26 -0
@@ -0,0 +1,26 @@ +KASAN_SANITIZE := n +OBJECT_FILES_NON_STANDARD := y +KCOV_INSTRUMENT := n +KBUILD_CFLAGS := -Wall -Werror -static -nostdlib -nostartfiles -I$(obj)/include +KBUILD_AFLAGS += -I$(obj)/include + +subdir- := enclave + +always := sgx_le_proxy +clean-files := sgx_le_proxy + +# +# sgx_le_proxy +# + +sgx_le_proxy-y += main.o entry.o sgx_le_piggy.o +targets += $(sgx_le_proxy-y) +SGX_LE_PROXY_OBJS = $(addprefix $(obj)/,$(sgx_le_proxy-y)) + +$(obj)/sgx_le_piggy.o: $(obj)/enclave/sgx_le.bin $(obj)/enclave/sgx_le.ss +$(obj)/enclave/sgx_le.bin $(obj)/enclave/sgx_le.ss: FORCE + $(Q)$(MAKE) $(build)=$(obj)/enclave $@ + +targets += sgx_le_proxy +$(obj)/sgx_le_proxy: $(SGX_LE_PROXY_OBJS) + $(call if_changed,ld)
drivers/platform/x86/intel_sgx/le/enclave/Makefile new file +46 -0
46 lines not on this page · raw message
drivers/platform/x86/intel_sgx/le/enclave/aes_encrypt.c new file +191 -0
191 lines not on this page · raw message
drivers/platform/x86/intel_sgx/le/enclave/cmac_mode.c new file +254 -0
254 lines not on this page · raw message
drivers/platform/x86/intel_sgx/le/enclave/encl_bootstrap.S new file +163 -0
163 lines not on this page · raw message
drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/aes.h new file +133 -0
133 lines not on this page · raw message
drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/cmac_mode.h new file +194 -0
194 lines not on this page · raw message
drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/constants.h new file +59 -0
59 lines not on this page · raw message
drivers/platform/x86/intel_sgx/le/enclave/include/tinycrypt/utils.h new file +95 -0
95 lines not on this page · raw message
drivers/platform/x86/intel_sgx/le/enclave/main.c new file +203 -0
203 lines not on this page · raw message
drivers/platform/x86/intel_sgx/le/enclave/sgx_le.lds new file +28 -0
@@ -0,0 +1,28 @@ +OUTPUT_FORMAT(elf64-x86-64) + +SECTIONS +{ + . = 0; + .tcs : { + *(.tcs*) + } + + . = ALIGN(4096); + .text : { + *(.text*) + *(.rodata*) + } + + . = ALIGN(4096); + .data : { + *(.data*) + } + + /DISCARD/ : { + *(.data*) + *(.comment*) + *(.note*) + *(.debug*) + *(.eh_frame*) + } +}
drivers/platform/x86/intel_sgx/le/enclave/sgxsign.c new file +537 -0
537 lines not on this page · raw message
drivers/platform/x86/intel_sgx/le/enclave/utils.c new file +78 -0
78 lines not on this page · raw message
drivers/platform/x86/intel_sgx/le/entry.S new file +117 -0
117 lines not on this page · raw message
drivers/platform/x86/intel_sgx/le/include/sgx_asm.h new file +64 -0
64 lines not on this page · raw message
drivers/platform/x86/intel_sgx/le/include/sgx_encl.h new file +105 -0
105 lines not on this page · raw message
drivers/platform/x86/intel_sgx/le/main.c new file +219 -0
219 lines not on this page · raw message
drivers/platform/x86/intel_sgx/le/sgx_le_piggy.S new file +15 -0
@@ -0,0 +1,15 @@ +#include <linux/linkage.h> +#include <asm/page_types.h> + + .section ".rodata","a" + .balign PAGE_SIZE + +GLOBAL(sgx_le_blob) + .incbin "drivers/platform/x86/intel_sgx/le/enclave/sgx_le.bin" +END(sgx_le_blob) + +GLOBAL(sgx_le_blob_end); + +GLOBAL(sgx_le_ss) + .incbin "drivers/platform/x86/intel_sgx/le/enclave/sgx_le.ss" +END(sgx_le_ss)
drivers/platform/x86/intel_sgx/sgx_le_proxy_piggy.S new file +11 -0
@@ -0,0 +1,11 @@ +#include <linux/linkage.h> +#include <asm/export.h> +#include <asm/page_types.h> + + .section ".rodata","a" + +GLOBAL(sgx_le_proxy) + .incbin "drivers/platform/x86/intel_sgx/le/sgx_le_proxy" +END(sgx_le_proxy) + +GLOBAL(sgx_le_proxy_end)
Signature · 1 line
2.14.1
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 10/12] intel_sgx: in-kernel launch enclave
Quoted 6 lines · Jarkko Sakkinen
(quoted diff) +6
-0+static RSA *load_sign_key(const char *path) +{ + FILE *f; + RSA *key; + + f = fopen(path, "rb");
Review
Quoted 10 lines
(quoted diff) +10
-0+ if (!f) { + fprintf(stderr, "Unable to open %s\n", path); + return NULL; + } + key = RSA_new(); + if (!PEM_read_RSAPrivateKey(f, &key, pem_passwd_cb, NULL)) + return NULL; + fclose(f); + + if (BN_num_bytes(key->n) != SGX_MODULUS_SIZE) {
Dereferencing the RSA pointer (key) breaks on OpenSSL 1.1.0 as RSA is now an opaque object. It's relatively easy to fudge around the issue, patch below.
https://github.com/openssl/openssl/issues/1491
https://wiki.openssl.org/index.php/OpenSSL_1.1.0_Changes
Review
Quoted 9 lines
(quoted diff) +1
-0+ fprintf(stderr, "Invalid key size %d\n", BN_num_bytes(key-Quoted 1 line
n));
(quoted diff) +7
-0+ RSA_free(key); + return NULL; + } + + return key; +} +
(quoted diff) +0 -0
drivers/platform/x86/intel_sgx/le/enclave/sgxsign.c index 27e8c61d033c..e454dc95f438 100644
platform/x86/intel_sgx/le/enclave/sgxsign.c +0 -0
@@ -110,6 +110,17 @@ static int pem_passwd_cb(char *buf, int size, int rwflag,
void *u)
return strlen(buf) >= size ? size - 1 : strlen(buf);
}
+static inline const BIGNUM *get_modulus(RSA *key)
+{
+#if OPENSSL_VERSION_NUMBER < 0x10100000L
+ return key->n;
+#else
+ const BIGNUM *n;
+ RSA_get0_key(key, &n, NULL, NULL);
+ return n;
+#endif
+}
+
static RSA *load_sign_key(const char *path)
{
FILE *f;
@@ -125,8 +136,9 @@ static RSA *load_sign_key(const char *path) return NULL;
fclose(f);
- if (BN_num_bytes(key->n) != SGX_MODULUS_SIZE) {
- fprintf(stderr, "Invalid key size %d\n", BN_num_bytes(key->n)); + if (BN_num_bytes(get_modulus(key)) != SGX_MODULUS_SIZE) { + fprintf(stderr, "Invalid key size %d\n",
+ BN_num_bytes(get_modulus(key))); RSA_free(key);
return NULL;
}
@@ -511,7 +523,7 @@ int main(int argc, char **argv)
if (!sign_key)
goto out;
- BN_bn2bin(sign_key->n, ss.modulus);
+ BN_bn2bin(get_modulus(sign_key), ss.modulus);
if (!measure_encl(argv[1], ss.body.mrenclave))
goto out;
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 10/12] intel_sgx: in-kernel launch enclave
Quoted 73 lines · Sean Christopherson
Quoted 6 lines · Jarkko Sakkinen
(quoted diff) +6
-0+static RSA *load_sign_key(const char *path) +{ + FILE *f; + RSA *key; + + f = fopen(path, "rb");Review
Quoted 10 lines
(quoted diff) +10
-0+ if (!f) { + fprintf(stderr, "Unable to open %s\n", path); + return NULL; + } + key = RSA_new(); + if (!PEM_read_RSAPrivateKey(f, &key, pem_passwd_cb, NULL)) + return NULL; + fclose(f); + + if (BN_num_bytes(key->n) != SGX_MODULUS_SIZE) {Dereferencing the RSA pointer (key) breaks on OpenSSL 1.1.0 as RSA is now an opaque object. It's relatively easy to fudge around the issue, patch below.
https://github.com/openssl/openssl/issues/1491
https://wiki.openssl.org/index.php/OpenSSL_1.1.0_ChangesQuoted 9 lines
(quoted diff) +1
-0+ fprintf(stderr, "Invalid key size %d\n", BN_num_bytes(key-Quoted 1 line
n));
(quoted diff) +7
-0+ RSA_free(key); + return NULL; + } + + return key; +} +(quoted diff) +0
-0
drivers/platform/x86/intel_sgx/le/enclave/sgxsign.cindex27e8c61d033c..e454dc95f438100644platform/x86/intel_sgx/le/enclave/sgxsign.c +0
-0@@ -110,6 +110,17 @@ static int pem_passwd_cb(char *buf, int size, int rwflag, void *u) return strlen(buf) >= size ? size - 1 : strlen(buf); }platform/x86/intel_sgx/le/enclave/sgxsign.c +11
-0+static inline const BIGNUM *get_modulus (RSA *key) +{ +#if OPENSSL_VERSION_NUMBER <0x10100000L + return key->n; +#else + const BIGNUM *n; + RSA_get0_key(key, &n, NULL, NULL); + return n; +#endif +} +static RSA *load_sign_key(const char *path)
{
FILE *f;platform/x86/intel_sgx/le/enclave/sgxsign.c +0
-0@@ -125,8 +136,9 @@ static RSA *load_sign_key(const char *path) return NULL; fclose(f);
- if (BN_num_bytes(key->n)!=SGX_MODULUS_SIZE) {
- fprintf(stderr, "Invalid key size %d\n",BN_num_bytes(key->n)); + if (BN_num_bytes(get_modulus(key)) !=SGX_MODULUS_SIZE) { + fprintf(stderr, "Invalid key size %d\n",
+ BN_num_bytes(get_modulus(key)));RSA_free(key);
return NULL;
}platform/x86/intel_sgx/le/enclave/sgxsign.c +0
-0@@ -511,7 +523,7 @@ int main(int argc, char **argv) if (!sign_key) goto out;
Quoted 7 lines
- BN_bn2bin(sign_key->n, ss.modulus);
+ BN_bn2bin(get_modulus(sign_key), ss.modulus);
if (!measure_encl(argv[1], ss.body.mrenclave)) goto out;
Already sent v5 but I'll put this to v6. Thanks.
/Jarkko
RFC PATCH v3 11/12 intel_sgx: glue code for in-kernel LE
In reply to Jarkko SakkinenImplements a simple framework for hosting in-kernel LE by using the user space helper framework.
--- drivers/platform/x86/intel_sgx/Kconfig | 2 + drivers/platform/x86/intel_sgx/Makefile | 1 + drivers/platform/x86/intel_sgx/sgx.h | 15 ++ drivers/platform/x86/intel_sgx/sgx_ioctl.c | 13 +- drivers/platform/x86/intel_sgx/sgx_le.c | 290 +++++++++++++++++++++++++++++ drivers/platform/x86/intel_sgx/sgx_main.c | 12 +- drivers/platform/x86/intel_sgx/sgx_util.c | 25 +++ 7 files changed, 352 insertions(+), 6 deletions(-) create mode 100644 drivers/platform/x86/intel_sgx/sgx_le.c
drivers/platform/x86/intel_sgx/Kconfig +2 -0
@@ -9,6 +9,8 @@ config INTEL_SGX default n depends on X86_64 && CPU_SUP_INTEL select MMU_NOTIFIER + select CRYPTO + select CRYPTO_SHA256 ---help--- Intel(R) SGX is a set of CPU instructions that can be used by applications to set aside private regions of code and data. The code
drivers/platform/x86/intel_sgx/Makefile +1 -0
@@ -11,6 +11,7 @@ intel_sgx-$(CONFIG_INTEL_SGX) += \ sgx_page_cache.o \ sgx_util.o \ sgx_vma.o \ + sgx_le.o \ sgx_le_proxy_piggy.o $(eval $(call config_filename,INTEL_SGX_SIGNING_KEY))
drivers/platform/x86/intel_sgx/sgx.h +15 -0
@@ -68,6 +68,7 @@#include <linux/workqueue.h> #include <linux/mmu_notifier.h> #include <linux/radix-tree.h> +#include <crypto/hash.h> #include <asm/sgx.h> #define SGX_EINIT_SPIN_COUNT 20
@@ -178,6 +179,7 @@ extern u32 sgx_misc_reserved;extern u32 sgx_xsave_size_tbl[64 ];extern bool sgx_locked_msrs; +extern const struct file_operations sgx_fops;extern const struct vm_operations_struct sgx_vm_ops;#define sgx_pr_ratelimited (level, encl, fmt, ...) \
@@ -231,6 +233,9 @@ struct sgx_encl_page *sgx_fault_page(struct vm_area_struct *vma, unsigned int flags); +int sgx_get_key_hash (struct crypto_shash *tfm,const void *modulus,void *hash); +int sgx_get_key_hash_simple (const void *modulus,void *hash); +extern struct mutex sgx_tgid_ctx_mutex;extern struct list_head sgx_tgid_ctx_list;extern atomic_t sgx_va_pages_cnt;
@@ -245,4 +250,14 @@ void sgx_put_page(void *epc_page_vaddr);void sgx_eblock (struct sgx_encl *encl,struct sgx_epc_page *epc_page);void sgx_etrack (struct sgx_encl *encl); +extern struct sgx_le_ctx sgx_le_ctx; + +int sgx_le_init (struct sgx_le_ctx *ctx); +void sgx_le_exit (struct sgx_le_ctx *ctx); + +int sgx_le_get_token (struct sgx_le_ctx *ctx, +const struct sgx_encl *encl, +const struct sgx_sigstruct *sigstruct, +struct sgx_einittoken *token); +#endif /* __ARCH_X86_INTEL_SGX_H__ */
drivers/platform/x86/intel_sgx/sgx_ioctl.c +9 -4
@@ -69,7 +69,7 @@#include <linux/hashtable.h> #include <linux/shmem_fs.h> -static int sgx_get_encl (unsigned long addr,struct sgx_encl **encl) +static int sgx_encl_get (unsigned long addr,struct sgx_encl **encl) {struct mm_struct *mm = current->mm;struct vm_area_struct *vma;
@@ -156,7 +156,7 @@ static long sgx_ioc_enclave_add_page(struct file *filep, unsigned int cmd,void *data;int ret; - ret =sgx_get_encl (addp->addr, &encl); + ret =sgx_encl_get (addp->addr, &encl);if (ret)return ret;
@@ -227,11 +227,16 @@ static long sgx_ioc_enclave_init(struct file *filep, unsigned int cmd,if (ret) goto out; - ret =sgx_get_encl (encl_id, &encl); + ret =sgx_encl_get (encl_id, &encl);if (ret) goto out; - ret =sgx_encl_init (encl, sigstruct, einittoken); +if (!(initp->flags &&SGX_ENCLAVE_INIT_ARCH )) + ret =sgx_le_get_token (&sgx_le_ctx, encl, sigstruct, + einittoken); + +if (!ret) + ret =sgx_encl_init (encl, sigstruct, einittoken);kref_put (&encl->refcount, sgx_encl_release);
drivers/platform/x86/intel_sgx/sgx_le.c new file +290 -0
290 lines not on this page · raw message
drivers/platform/x86/intel_sgx/sgx_main.c +10 -2
@@ -143,7 +143,7 @@ static unsigned long sgx_get_unmapped_area(struct file *file,return addr; } -static const struct file_operations sgx_fops = { +const struct file_operations sgx_fops = { .owner =THIS_MODULE , .unlocked_ioctl = sgx_ioctl, #ifdef CONFIG_COMPAT
@@ -315,11 +315,17 @@ static int sgx_dev_init(struct device *parent, bool locked_msrs) goto out_iounmap; } - ret =cdev_device_add (&sgx_dev->cdev, &sgx_dev->dev); + ret =sgx_le_init (&sgx_le_ctx);if (ret) goto out_workqueue; + ret =cdev_device_add (&sgx_dev->cdev, &sgx_dev->dev); +if (ret) + goto out_le; +return 0 ; +out_le: +sgx_le_exit (&sgx_le_ctx); out_workqueue:destroy_workqueue (sgx_add_page_wq); out_iounmap:
@@ -387,6 +393,8 @@ static int sgx_drv_remove(struct platform_device *pdev)cdev_device_del (&ctx->cdev, &ctx->dev); +sgx_le_exit (&sgx_le_ctx); +destroy_workqueue (sgx_add_page_wq);#ifdef CONFIG_X86_64 for (i =0 ; i < sgx_nr_epc_banks; i++)
drivers/platform/x86/intel_sgx/sgx_util.c +25 -0
@@ -369,3 +369,28 @@ void sgx_etrack(struct sgx_encl *encl)sgx_invalidate (encl, true); } } + +int sgx_get_key_hash (struct crypto_shash *tfm,const void *modulus,void *hash) +{ +SHASH_DESC_ON_STACK (shash, tfm); + + shash->tfm = tfm; + shash->flags =CRYPTO_TFM_REQ_MAY_SLEEP ; + +return crypto_shash_digest (shash, modulus,SGX_MODULUS_SIZE , hash); +} + +int sgx_get_key_hash_simple (const void *modulus,void *hash) +{ +struct crypto_shash *tfm; +int ret; + + tfm =crypto_alloc_shash ("sha256" ,0 ,CRYPTO_ALG_ASYNC ); +if (IS_ERR (tfm)) +return PTR_ERR (tfm); + + ret =sgx_get_key_hash (tfm, modulus, hash); + +crypto_free_shash (tfm); +return ret; +}
Signature · 1 line
2.14.1
RFC PATCH v3 12/12 intel_sgx: update IA32_SGXLEPUBKEYHASH* MSRs
In reply to Jarkko SakkinenCheck if IA32_SGXLEPUBKEYHASH* MSRs match. If they do not match, allow the driver initialization to continue only if they are writable. In this case update them with the MRSIGNER of the launch enclave.
--- drivers/platform/x86/intel_sgx/sgx.h | 1 + drivers/platform/x86/intel_sgx/sgx_encl.c | 20 ++++++++++++++++++++ drivers/platform/x86/intel_sgx/sgx_le_proxy_piggy.S | 4 ++++ drivers/platform/x86/intel_sgx/sgx_main.c | 7 ++++++- 4 files changed, 31 insertions(+), 1 deletion(-)
drivers/platform/x86/intel_sgx/sgx.h +1 -0
@@ -178,6 +178,7 @@ extern u64 sgx_xfrm_mask;extern u32 sgx_misc_reserved;extern u32 sgx_xsave_size_tbl[64 ];extern bool sgx_locked_msrs; +extern u64 sgx_le_pubkeyhash[4 ];extern const struct file_operations sgx_fops;extern const struct vm_operations_struct sgx_vm_ops;
drivers/platform/x86/intel_sgx/sgx_encl.c +20 -0
@@ -68,6 +68,7 @@#include <linux/slab.h> #include <linux/hashtable.h> #include <linux/shmem_fs.h> +#include <linux/percpu.h> struct sgx_add_page_req {struct sgx_encl *encl;
@@ -874,6 +875,17 @@ static int sgx_einit(struct sgx_encl *encl, struct sgx_sigstruct *sigstruct,return ret; } +static void sgx_update_pubkeyhash (void ) +{ +if (sgx_locked_msrs) +return ; + +wrmsrl (MSR_IA32_SGXLEPUBKEYHASH0 , sgx_le_pubkeyhash[0 ]); +wrmsrl (MSR_IA32_SGXLEPUBKEYHASH1 , sgx_le_pubkeyhash[1 ]); +wrmsrl (MSR_IA32_SGXLEPUBKEYHASH2 , sgx_le_pubkeyhash[2 ]); +wrmsrl (MSR_IA32_SGXLEPUBKEYHASH3 , sgx_le_pubkeyhash[3 ]); +} +/** * sgx_encl_init - perform EINIT for the given enclave *
@@ -909,6 +921,14 @@ int sgx_encl_init(struct sgx_encl *encl, struct sgx_sigstruct *sigstruct,for (j =0 ; j <SGX_EINIT_SPIN_COUNT ; j++) { ret =sgx_einit (encl, sigstruct, token); +if (ret ==SGX_INVALID_ATTRIBUTE || + ret ==SGX_INVALID_EINITTOKEN ) { +preempt_disable (); +sgx_update_pubkeyhash (); + ret =sgx_einit (encl, sigstruct, token); +preempt_enable (); + } +if (ret ==SGX_UNMASKED_EVENT )continue ;else
drivers/platform/x86/intel_sgx/sgx_le_proxy_piggy.S +4 -0
@@ -9,3 +9,7 @@ GLOBAL(sgx_le_proxy) END(sgx_le_proxy) GLOBAL(sgx_le_proxy_end) + +GLOBAL(sgx_le_ss) + .incbin "drivers/platform/x86/intel_sgx/le/enclave/sgx_le.ss" +END(sgx_le_ss)
drivers/platform/x86/intel_sgx/sgx_main.c +6 -1
@@ -80,6 +80,7 @@ MODULE_VERSION(DRV_VERSION);* Global data. */ +extern struct sgx_sigstruct sgx_le_ss;struct workqueue_struct *sgx_add_page_wq;#define SGX_MAX_EPC_BANKS 8struct sgx_epc_bank sgx_epc_banks[SGX_MAX_EPC_BANKS ];
@@ -90,6 +91,7 @@ u64 sgx_xfrm_mask = 0x3;u32 sgx_misc_reserved;u32 sgx_xsave_size_tbl[64 ];bool sgx_locked_msrs; +u64 sgx_le_pubkeyhash[4 ];#ifdef CONFIG_COMPAT long sgx_compat_ioctl (struct file *filep,unsigned int cmd,unsigned long arg)
@@ -267,6 +269,10 @@ static int sgx_dev_init(struct device *parent, bool locked_msrs) } } + ret =sgx_get_key_hash_simple (sgx_le_ss.modulus, sgx_le_pubkeyhash); +if (ret) +return ret; +for (i =0 ; i <SGX_MAX_EPC_BANKS ; i++) {cpuid_count (SGX_CPUID , i +SGX_CPUID_EPC_BANKS , &eax, &ebx, &ecx, &edx);
@@ -356,7 +362,6 @@ static int sgx_drv_probe(struct platform_device *pdev) }rdmsrl (MSR_IA32_FEATURE_CONTROL , fc); -if (!(fc &FEATURE_CONTROL_LOCKED )) {pr_err ("intel_sgx: the feature control MSR is not locked\n" );return -ENODEV ;
Signature · 1 line
2.14.1
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
Quoted 42 lines · Jarkko Sakkinen
drivers/platform/x86/intel_sgx/sgx_page_cache.c new file +36
-0+/** +* sgx_alloc_page - allocate an EPC page +* @flags: allocation flags +* +* Try to grab a page from the free EPC page list. If there is a free page +* available, it is returned to the caller. If called with SGX_ALLOC_ATOMIC, +* the function will return immediately if the list is empty. Otherwise, it +* will swap pages up until there is a free page available. Before returning +* the low watermark is checked and ksgxswapd is waken up if we are below it. +* +* Return: an EPC page or a system error code +*/ +struct sgx_epc_page *sgx_alloc_page (unsigned int flags) +{ +struct sgx_epc_page *entry; + +for ( ; ; ) { + entry =sgx_alloc_page_fast (); +if (entry) +break ; + +/* We need at minimum two pages for the #PF handler. */ +if (atomic_read (&sgx_va_pages_cnt) > + (sgx_nr_total_epc_pages -2 )) +return ERR_PTR (-ENOMEM ); + +if (flags &SGX_ALLOC_ATOMIC ) { + entry =ERR_PTR (-EBUSY ); +break ; + } + +if (signal_pending (current)) { + entry =ERR_PTR (-ERESTARTSYS ); +break ; + } +
drivers/platform/x86/intel_sgx/sgx_page_cache.c
Quoted 10 lines
drivers/platform/x86/intel_sgx/sgx_page_cache.c new file +10
-0+sgx_swap_pages (SGX_NR_SWAP_CLUSTER_MAX ); +schedule (); + } + +if (sgx_nr_free_pages < sgx_nr_low_pages) +wake_up (&ksgxswapd_waitq); + +return entry; +} +EXPORT_SYMBOL (sgx_alloc_page);
I think it makes sense to remove the exports from sgx_page_cache.c for the initial upstreaming given that the only consumer is the pre-release/out-of-tree KVM module, which generally requires recompiling the entire kernel anyways.
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
Quoted 50 lines · Sean Christopherson
Quoted 49 lines · Jarkko Sakkinen
drivers/platform/x86/intel_sgx/sgx_page_cache.c new file +43
-0+/** +* sgx_alloc_page - allocate an EPC page +* @flags: allocation flags +* +* Try to grab a page from the free EPC page list. If there is a free page +* available, it is returned to the caller. If called with SGX_ALLOC_ATOMIC, +* the function will return immediately if the list is empty. Otherwise, it +* will swap pages up until there is a free page available. Before returning +* the low watermark is checked and ksgxswapd is waken up if we are below it. +* +* Return: an EPC page or a system error code +*/ +struct sgx_epc_page *sgx_alloc_page (unsigned int flags) +{ +struct sgx_epc_page *entry; + +for ( ; ; ) { + entry =sgx_alloc_page_fast (); +if (entry) +break ; + +/* We need at minimum two pages for the #PF handler. */ +if (atomic_read (&sgx_va_pages_cnt) > + (sgx_nr_total_epc_pages -2 )) +return ERR_PTR (-ENOMEM ); + +if (flags &SGX_ALLOC_ATOMIC ) { + entry =ERR_PTR (-EBUSY ); +break ; + } + +if (signal_pending (current)) { + entry =ERR_PTR (-ERESTARTSYS ); +break ; + } + +sgx_swap_pages (SGX_NR_SWAP_CLUSTER_MAX ); +schedule (); + } + +if (sgx_nr_free_pages < sgx_nr_low_pages) +wake_up (&ksgxswapd_waitq); +
Quoted 8 lines
drivers/platform/x86/intel_sgx/sgx_page_cache.c
Quoted 3 lines
drivers/platform/x86/intel_sgx/sgx_page_cache.c new file +3
-0+return entry; +} +EXPORT_SYMBOL (sgx_alloc_page);I think it makes sense to remove the exports from
sgx_page_cache.cfor the initial upstreaming given that the only consumer is the pre-release/out-of-tree KVM module, which generally requires recompiling the entire kernel anyways.
Forgot them. Thanks.
For the same reason as you described I removed them from arch/x86/include/asm/sgx.h
/Jarkko
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
Quoted 60 lines · Jarkko Sakkinen
Quoted 50 lines · Sean Christopherson
Quoted 49 lines · Jarkko Sakkinen
drivers/platform/x86/intel_sgx/sgx_page_cache.c new file +43
-0+/** +* sgx_alloc_page - allocate an EPC page +* @flags: allocation flags +* +* Try to grab a page from the free EPC page list. If there is a free page +* available, it is returned to the caller. If called with SGX_ALLOC_ATOMIC, +* the function will return immediately if the list is empty. Otherwise, it +* will swap pages up until there is a free page available. Before returning +* the low watermark is checked and ksgxswapd is waken up if we are below it. +* +* Return: an EPC page or a system error code +*/ +struct sgx_epc_page *sgx_alloc_page (unsigned int flags) +{ +struct sgx_epc_page *entry; + +for ( ; ; ) { + entry =sgx_alloc_page_fast (); +if (entry) +break ; + +/* We need at minimum two pages for the #PF handler. */ +if (atomic_read (&sgx_va_pages_cnt) > + (sgx_nr_total_epc_pages -2 )) +return ERR_PTR (-ENOMEM ); + +if (flags &SGX_ALLOC_ATOMIC ) { + entry =ERR_PTR (-EBUSY ); +break ; + } + +if (signal_pending (current)) { + entry =ERR_PTR (-ERESTARTSYS ); +break ; + } + +sgx_swap_pages (SGX_NR_SWAP_CLUSTER_MAX ); +schedule (); + } + +if (sgx_nr_free_pages < sgx_nr_low_pages) +wake_up (&ksgxswapd_waitq); +Quoted 8 lines
drivers/platform/x86/intel_sgx/sgx_page_cache.c
Quoted 3 lines
drivers/platform/x86/intel_sgx/sgx_page_cache.c new file +3
-0+return entry; +} +EXPORT_SYMBOL (sgx_alloc_page);I think it makes sense to remove the exports from
sgx_page_cache.cfor the initial upstreaming given that the only consumer is the pre-release/out-of-tree KVM module, which generally requires recompiling the entire kernel anyways.
Quoted 4 lines
Forgot them. Thanks.
For the same reason as you described I removed them from
arch/x86/include/asm/sgx.h
sgx_free_page can also be cleaned up a bit if it's no longer exported, proposed patch below.
intel_sgx: make encl a required param for sgx_free_page
sgx_free_page is no longer exported, and so encl is no longer an optional param as all driver usage must specify the encl. Making encl required also eliminates the path that returns a value, i.e. modify sgx_free_page to have a void return.
--- drivers/platform/x86/intel_sgx/sgx.h | 2 +- drivers/platform/x86/intel_sgx/sgx_page_cache.c | 29 ++++++------------------- 2 files changed, 8 insertions(+), 23 deletions(-)
drivers/platform/x86/intel_sgx/sgx.h +1 -1
@@ -245,7 +245,7 @@ int sgx_add_epc_bank(resource_size_t start, unsigned long size, int bank);int sgx_page_cache_init (void );void sgx_page_cache_teardown (void );struct sgx_epc_page *sgx_alloc_page (unsigned int flags); -int sgx_free_page (struct sgx_epc_page *entry,struct sgx_encl *encl); +void sgx_free_page (struct sgx_epc_page *entry,struct sgx_encl *encl);void *sgx_get_page (struct sgx_epc_page *entry);void sgx_put_page (void *epc_page_vaddr);void sgx_eblock (struct sgx_encl *encl,struct sgx_epc_page *epc_page);
drivers/platform/x86/intel_sgx/sgx_page_cache.c +7 -22
@@ -522,23 +522,14 @@ struct sgx_epc_page *sgx_alloc_page(unsigned int flags)/** * sgx_free_page - free an EPC page * -* EREMOVE an EPC page and insert it back to the list of free pages. Optionally, -* an enclave can be given as a parameter. If the enclave is given, the -* resulting error is printed out loud as a critical error. It is an indicator -* of a driver bug if that would happen. -* -* If the enclave is not given as a parameter (like in the case when VMM uses -* this function)), it is fully up to the caller to deal with the return value, -* including printing it to the klog if it wants to do such a thing. +* EREMOVE an EPC page and insert it back to the list of free pages. +* If EREMOVE fails, the error is printed out loud as a critical error. +* It is an indicator of a driver bug if that would happen. * * @entry: any EPC page -* @encl: enclave that owns the given EPC page (optional) -* -* Return: -* 0 on success, -* SGX error code if an enclave is not defined +* @encl: enclave that owns the given EPC page */ -int sgx_free_page (struct sgx_epc_page *entry,struct sgx_encl *encl) +void sgx_free_page (struct sgx_epc_page *entry,struct sgx_encl *encl) {void *epc;int ret;
@@ -547,19 +538,13 @@ int sgx_free_page(struct sgx_epc_page *entry, struct sgx_encl *encl) ret =__eremove (epc);sgx_put_page (epc); -if (ret) { -if (encl) -sgx_crit (encl,"EREMOVE returned %d\n" , ret); -else -return ret; - } +if (ret) +sgx_crit (encl,"EREMOVE returned %d\n" , ret);spin_lock (&sgx_free_list_lock);list_add (&entry->list, &sgx_free_list); sgx_nr_free_pages++;spin_unlock (&sgx_free_list_lock); - -return 0 ; }void *sgx_get_page (struct sgx_epc_page *entry)
Signature · 1 line
2.14.2
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
In reply to a message not in the archiveQuoted 29 lines · Jarkko Sakkinen
drivers/platform/x86/intel_sgx/sgx_encl.c new file +22
-0@@ -0,0 +1,989 @@ +/** +* sgx_encl_find - find an enclave +* @mm: mm struct of the current process +* @addr: address in the ELRANGE +* @created is the enclave already created? +* @vma: the resulting VMA +* +* Finds an enclave identified by the given address. Gives back the VMA, that is +* part of the enclave, located in that address. +* +* Return: +* 0 on success, +* -EINVAL if not found, +*/ +int sgx_encl_find (struct mm_struct *mm,unsigned long addr,bool created, +struct vm_area_struct **vma) +{ +struct vm_area_struct *result; +struct sgx_encl *encl; + + result =find_vma (mm, addr); +if (!result || result->vm_ops != &sgx_vm_ops || addr < result->vm_start)
drivers/platform/x86/intel_sgx/sgx_encl.c
Quoted 10 lines
drivers/platform/x86/intel_sgx/sgx_encl.c new file +10
-0+return -EINVAL ; + + encl = result->vm_private_data; +if (created) { +if (!encl) +return -EINVAL ; + }else { +if (encl) +return -EINVAL ; + }
What about removing @created and returning -ENOENT (or -ENXIO?) if result->vm_private_data is NULL? Removing @created will eliminate any potential confusion for the common case of @created=true. For @created=false, which should be limited to sgx_encl_create, I think that explicitly checking for "ret != -ENOENT" is more intuitive than checking whether or not sgx_encl_find succeeded, e.g. I knew the intent of the check in sgx_encl_create ahead of time and I still had to walk through sgx_encl_find to verify the behavior.
Quoted 5 lines
drivers/platform/x86/intel_sgx/sgx_encl.c new file +5
-0+ + *vma = result; +return 0 ; +} +
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
Quoted 41 lines · Sean Christopherson
Quoted 29 lines · Jarkko Sakkinen
drivers/platform/x86/intel_sgx/sgx_encl.c new file +22
-0@@ -0,0 +1,989 @@ +/** +* sgx_encl_find - find an enclave +* @mm: mm struct of the current process +* @addr: address in the ELRANGE +* @created is the enclave already created? +* @vma: the resulting VMA +* +* Finds an enclave identified by the given address. Gives back the VMA, that is +* part of the enclave, located in that address. +* +* Return: +* 0 on success, +* -EINVAL if not found, +*/ +int sgx_encl_find (struct mm_struct *mm,unsigned long addr,bool created, +struct vm_area_struct **vma) +{ +struct vm_area_struct *result; +struct sgx_encl *encl; + + result =find_vma (mm, addr); +if (!result || result->vm_ops != &sgx_vm_ops || addr < result->vm_start)drivers/platform/x86/intel_sgx/sgx_encl.c
Quoted 10 lines
drivers/platform/x86/intel_sgx/sgx_encl.c new file +10
-0+return -EINVAL ; + + encl = result->vm_private_data; +if (created) { +if (!encl) +return -EINVAL ; + }else { +if (encl) +return -EINVAL ; + }
Quoted 8 lines
What about removing @created and returning
-ENOENT(or-ENXIO?) ifresult->vm_private_datais NULL? Removing @created will eliminate any potential confusion for the common case of @created=true. For @created=false, which should be limited tosgx_encl_create, I think that explicitly checking for "ret !=-ENOENT" is more intuitive than checking whether or notsgx_encl_findsucceeded, e.g. I knew the intent of the check insgx_encl_createahead of time and I still had to walk throughsgx_encl_findto verify the behavior.
Would make sense. Thank you.
/Jarkko
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
Quoted 51 lines · Jarkko Sakkinen
Quoted 41 lines · Sean Christopherson
Quoted 29 lines · Jarkko Sakkinen
drivers/platform/x86/intel_sgx/sgx_encl.c new file +22
-0@@ -0,0 +1,989 @@ +/** +* sgx_encl_find - find an enclave +* @mm: mm struct of the current process +* @addr: address in the ELRANGE +* @created is the enclave already created? +* @vma: the resulting VMA +* +* Finds an enclave identified by the given address. Gives back the VMA, that is +* part of the enclave, located in that address. +* +* Return: +* 0 on success, +* -EINVAL if not found, +*/ +int sgx_encl_find (struct mm_struct *mm,unsigned long addr,bool created, +struct vm_area_struct **vma) +{ +struct vm_area_struct *result; +struct sgx_encl *encl; + + result =find_vma (mm, addr); +if (!result || result->vm_ops != &sgx_vm_ops || addr < result->vm_start)drivers/platform/x86/intel_sgx/sgx_encl.c
Quoted 10 lines
drivers/platform/x86/intel_sgx/sgx_encl.c new file +10
-0+return -EINVAL ; + + encl = result->vm_private_data; +if (created) { +if (!encl) +return -EINVAL ; + }else { +if (encl) +return -EINVAL ; + }Quoted 8 lines
What about removing @created and returning
-ENOENT(or-ENXIO?) ifresult->vm_private_datais NULL? Removing @created will eliminate any potential confusion for the common case of @created=true. For @created=false, which should be limited tosgx_encl_create, I think that explicitly checking for "ret !=-ENOENT" is more intuitive than checking whether or notsgx_encl_findsucceeded, e.g. I knew the intent of the check insgx_encl_createahead of time and I still had to walk throughsgx_encl_findto verify the behavior.
Quoted 3 lines
Would make sense. Thank you.
/Jarkko
And in the case of ioctls (sgx_encl_get()) it would probably make sense to deliver -ENOENT back to the user space instead of -EINVAL, wouldn't it?
/Jarkko
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
Quoted 52 lines · Jarkko Sakkinen
Quoted 51 lines · Jarkko Sakkinen
Quoted 49 lines · Sean Christopherson
Quoted 29 lines · Jarkko Sakkinen
drivers/platform/x86/intel_sgx/sgx_encl.c new file +22
-0@@ -0,0 +1,989 @@ +/** +* sgx_encl_find - find an enclave +* @mm: mm struct of the current process +* @addr: address in the ELRANGE +* @created is the enclave already created? +* @vma: the resulting VMA +* +* Finds an enclave identified by the given address. Gives back the VMA, that is +* part of the enclave, located in that address. +* +* Return: +* 0 on success, +* -EINVAL if not found, +*/ +int sgx_encl_find (struct mm_struct *mm,unsigned long addr,bool created, +struct vm_area_struct **vma) +{ +struct vm_area_struct *result; +struct sgx_encl *encl; + + result =find_vma (mm, addr); +if (!result || result->vm_ops != &sgx_vm_ops || addr < result->vm_start)drivers/platform/x86/intel_sgx/sgx_encl.c
Quoted 10 lines
drivers/platform/x86/intel_sgx/sgx_encl.c new file +10
-0+return -EINVAL ; + + encl = result->vm_private_data; +if (created) { +if (!encl) +return -EINVAL ; + }else { +if (encl) +return -EINVAL ; + }What about removing @created and returning
-ENOENT(or-ENXIO?) ifresult->vm_private_datais NULL? Removing @created will eliminate any potential confusion for the common case of @created=true. For @created=false, which should be limited tosgx_encl_create, I think that explicitly checking for "ret !=-ENOENT" is more intuitive than checking whether or notsgx_encl_findsucceeded, e.g. I knew the intent of the check insgx_encl_createahead of time and I still had to walk throughsgx_encl_findto verify the behavior.
Quoted 8 lines
Quoted 3 lines
Would make sense. Thank you.
/Jarkko
And in the case of ioctls (
sgx_encl_get()) it would probably make sense to deliver-ENOENTback to the user space instead of-EINVAL, wouldn't it?/Jarkko
Please ignore this response! :-)
/Jarkko
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
Quoted 57 lines · Jarkko Sakkinen
Quoted 56 lines · Jarkko Sakkinen
Quoted 54 lines · Jarkko Sakkinen
Quoted 41 lines · Sean Christopherson
Quoted 29 lines · Jarkko Sakkinen
drivers/platform/x86/intel_sgx/sgx_encl.c new file +22
-0@@ -0,0 +1,989 @@ +/** +* sgx_encl_find - find an enclave +* @mm: mm struct of the current process +* @addr: address in the ELRANGE +* @created is the enclave already created? +* @vma: the resulting VMA +* +* Finds an enclave identified by the given address. Gives back the VMA, that is +* part of the enclave, located in that address. +* +* Return: +* 0 on success, +* -EINVAL if not found, +*/ +int sgx_encl_find (struct mm_struct *mm,unsigned long addr,bool created, +struct vm_area_struct **vma) +{ +struct vm_area_struct *result; +struct sgx_encl *encl; + + result =find_vma (mm, addr); +if (!result || result->vm_ops != &sgx_vm_ops || addr < result->vm_start)drivers/platform/x86/intel_sgx/sgx_encl.c
Quoted 10 lines
drivers/platform/x86/intel_sgx/sgx_encl.c new file +10
-0+return -EINVAL ; + + encl = result->vm_private_data; +if (created) { +if (!encl) +return -EINVAL ; + }else { +if (encl) +return -EINVAL ; + }Quoted 8 lines
What about removing @created and returning
-ENOENT(or-ENXIO?) ifresult->vm_private_datais NULL? Removing @created will eliminate any potential confusion for the common case of @created=true. For @created=false, which should be limited tosgx_encl_create, I think that explicitly checking for "ret !=-ENOENT" is more intuitive than checking whether or notsgx_encl_findsucceeded, e.g. I knew the intent of the check insgx_encl_createahead of time and I still had to walk throughsgx_encl_findto verify the behavior.Would make sense. Thank you.
/Jarkko
Quoted 8 lines
Quoted 4 lines
And in the case of ioctls (
sgx_encl_get()) it would probably make sense to deliver-ENOENTback to the user space instead of-EINVAL, wouldn't it?/Jarkko
Please ignore this response! :-)
/Jarkko
Or actually why not. Does not harm to tell user space that it is an enclave VMA but not yet created.
/Jarkko
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
In reply to a message not in the archiveQuoted 20 lines · Jarkko Sakkinen
drivers/platform/x86/intel_sgx/sgx_main.c new file +14
-0+static dev_t sgx_devt; + +static void sgx_dev_release (struct device *dev) +{ + struct sgx_context *ctx = container_of(dev, struct sgx_context, dev); +kfree (ctx); +} + +static struct sgx_context *sgx_ctx_alloc (struct device *parent) +{ +struct sgx_context *ctx; + + ctx =kzalloc (sizeof (*ctx),GFP_KERNEL ); +if (!ctx)
drivers/platform/x86/intel_sgx/sgx_main.c
Quoted 10 lines
drivers/platform/x86/intel_sgx/sgx_main.c new file +10
-0+return ERR_PTR (-ENOMEM ); + +device_initialize (&ctx->dev); + + ctx->dev.bus = &sgx_subsys; + ctx->dev.parent = parent; + ctx->dev.devt =MKDEV (MAJOR (sgx_devt),0 ); + ctx->dev.release = sgx_dev_release; + +dev_set_name (&ctx->dev,"sgx" );
/dev/sgx is created with 0600 permissions, does access to SGX really need to be restricted to root by default?
Quoted 9 lines
drivers/platform/x86/intel_sgx/sgx_main.c new file +9
-0+ +cdev_init (&ctx->cdev, &sgx_fops); + ctx->cdev.owner =THIS_MODULE ; + +dev_set_drvdata (parent, ctx); + +return ctx; +} +
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
Quoted 26 lines · Sean Christopherson
Quoted 25 lines · Jarkko Sakkinen
drivers/platform/x86/intel_sgx/sgx_main.c new file +19
-0+static dev_t sgx_devt; + +static void sgx_dev_release (struct device *dev) +{ + struct sgx_context *ctx = container_of(dev, struct sgx_context, dev); +kfree (ctx); +} + +static struct sgx_context *sgx_ctx_alloc (struct device *parent) +{ +struct sgx_context *ctx; + + ctx =kzalloc (sizeof (*ctx),GFP_KERNEL ); +if (!ctx) +return ERR_PTR (-ENOMEM ); + +device_initialize (&ctx->dev); + + ctx->dev.bus = &sgx_subsys;
Quoted 8 lines
drivers/platform/x86/intel_sgx/sgx_main.c
Quoted 5 lines
drivers/platform/x86/intel_sgx/sgx_main.c new file +5
-0+ ctx->dev.parent = parent; + ctx->dev.devt =MKDEV (MAJOR (sgx_devt),0 ); + ctx->dev.release = sgx_dev_release; + +dev_set_name (&ctx->dev,"sgx" );
/dev/sgxis created with 0600 permissions, does access to SGX really need to be restricted to root by default?
I personally am just a fan of whitelisting. It's only matter of a udev rule.
/Jarkko
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
In reply to a message not in the archiveReview
Quoted 5 lines · Jarkko Sakkinen
(quoted diff) +5
-0+static int __init sgx_drv_init(void) +{ + int ret; + + ret = sgx_drv_subsys_init();
The return value of sgx_drv_subsys_init is being ignored.
Quoted 15 lines
(quoted diff) +15
-0+ + ret = platform_driver_register(&sgx_drv); + if (ret) + sgx_drv_subsys_exit(); + + return ret; +} +module_init(sgx_drv_init); + +static void __exit sgx_drv_exit(void) +{ + platform_driver_unregister(&sgx_drv); + sgx_drv_subsys_exit(); +} +module_exit(sgx_drv_exit);
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
Quoted 9 lines · Sean Christopherson
Review
Quoted 5 lines · Jarkko Sakkinen
(quoted diff) +5
-0+static int __init sgx_drv_init(void) +{ + int ret; + + ret = sgx_drv_subsys_init();The return value of sgx_drv_subsys_init is being ignored.
Quoted 15 lines
Quoted 5 lines
(quoted diff) +5
-0+ + ret = platform_driver_register(&sgx_drv); + if (ret) + sgx_drv_subsys_exit(); +Review
Quoted 10 lines
(quoted diff) +10
-0+ return ret; +} +module_init(sgx_drv_init); + +static void __exit sgx_drv_exit(void) +{ + platform_driver_unregister(&sgx_drv); + sgx_drv_subsys_exit(); +} +module_exit(sgx_drv_exit);
Holds still for v4. the latest version under review. Thanks.
/Jarkko
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
In reply to a message not in the archiveQuoted 84 lines · Jarkko Sakkinen
(quoted diff) +16
-0+static int sgx_dev_init(struct device *parent, bool locked_msrs) +{ + struct sgx_context *sgx_dev; + unsigned int eax, ebx, ecx, edx; + unsigned long pa; + unsigned long size; + int ret; + int i; + + pr_info("intel_sgx: " DRV_DESCRIPTION " v" DRV_VERSION "\n"); + + sgx_dev = sgxm_ctx_alloc(parent); + + sgx_locked_msrs = locked_msrs; + + cpuid_count(SGX_CPUID, SGX_CPUID_CAPABILITIES, &eax, &ebx, &ecx,&edx);
(quoted diff) +9
-0+ /* Only allow misc bits supported by the driver. */ + sgx_misc_reserved = ~ebx | SGX_MISC_RESERVED_MASK; +#ifdef CONFIG_X86_64 + sgx_encl_size_max_64 = 1ULL << ((edx >> 8) & 0xFF); +#endif + sgx_encl_size_max_32 = 1ULL << (edx & 0xFF); + + if (boot_cpu_has(X86_FEATURE_OSXSAVE)) { + cpuid_count(SGX_CPUID, SGX_CPUID_ATTRIBUTES, &eax, &ebx,&ecx,
(quoted diff) +18
-0+ &edx); + sgx_xfrm_mask = (((u64)edx) << 32) + (u64)ecx; + + for (i = 2; i < 64; i++) { + cpuid_count(0x0D, i, &eax, &ebx, &ecx, &edx); + if ((1 << i) & sgx_xfrm_mask) + sgx_xsave_size_tbl[i] = eax + ebx; + } + } + + for (i = 0; i < SGX_MAX_EPC_BANKS; i++) { + cpuid_count(SGX_CPUID, i + SGX_CPUID_EPC_BANKS, &eax, &ebx, + &ecx, &edx); + if (!(eax & 0xf)) + break; + + pa = ((u64)(ebx & 0xfffff) << 32) + (u64)(eax & 0xfffff000); + size = ((u64)(edx & 0xfffff) << 32) + (u64)(ecx &
0xfffff000);(quoted diff) +38
-0+ + dev_info(parent, "EPC bank 0x%lx-0x%lx\n", pa, pa + size); + + sgx_epc_banks[i].pa = pa; + sgx_epc_banks[i].size = size; + } + + sgx_nr_epc_banks = i; + + for (i = 0; i < sgx_nr_epc_banks; i++) { +#ifdef CONFIG_X86_64 + sgx_epc_banks[i].va = (unsigned long) + ioremap_cache(sgx_epc_banks[i].pa, + sgx_epc_banks[i].size); + if (!sgx_epc_banks[i].va) { + sgx_nr_epc_banks = i; + ret = -ENOMEM; + goto out_iounmap; + } +#endif + ret = sgx_add_epc_bank(sgx_epc_banks[i].pa, + sgx_epc_banks[i].size, i); + if (ret) { + sgx_nr_epc_banks = i + 1; + goto out_iounmap; + } + } + + ret = sgx_page_cache_init(); + if (ret) + goto out_iounmap; + + sgx_add_page_wq = alloc_workqueue("intel_sgx-add-page-wq", + WQ_UNBOUND | WQ_FREEZABLE, 1); + if (!sgx_add_page_wq) { + pr_err("intel_sgx: alloc_workqueue() failed\n"); + ret = -ENOMEM; + goto out_iounmap;
Review
Quoted 10 lines
(quoted diff) +10
-0+ } + + ret = cdev_device_add(&sgx_dev->cdev, &sgx_dev->dev); + if (ret) + goto out_workqueue; + + return 0; +out_workqueue: + destroy_workqueue(sgx_add_page_wq); +out_iounmap:
sgx_page_cache_teardown() should be called here, else ksgxswapd and the list of EPC pages will leak.
Quoted 6 lines
(quoted diff) +6
-0+#ifdef CONFIG_X86_64 + for (i = 0; i < sgx_nr_epc_banks; i++) + iounmap((void *)sgx_epc_banks[i].va); +#endif + return ret; +}
...
Quoted 20 lines
(quoted diff) +20
-0+int sgx_add_epc_bank(resource_size_t start, unsigned long size, int bank) +{ + unsigned long i; + struct sgx_epc_page *new_epc_page, *entry; + struct list_head *parser, *temp; + + for (i = 0; i < size; i += PAGE_SIZE) { + new_epc_page = kzalloc(sizeof(*new_epc_page), GFP_KERNEL); + if (!new_epc_page) + goto err_freelist; + new_epc_page->pa = (start + i) | bank; + + spin_lock(&sgx_free_list_lock); + list_add_tail(&new_epc_page->list, &sgx_free_list); + sgx_nr_total_epc_pages++; + sgx_nr_free_pages++; + spin_unlock(&sgx_free_list_lock); + } + + return 0;
Review
Quoted 10 lines
(quoted diff) +10
-0+err_freelist: + list_for_each_safe(parser, temp, &sgx_free_list) { + spin_lock(&sgx_free_list_lock); + entry = list_entry(parser, struct sgx_epc_page, list); + list_del(&entry->list); + spin_unlock(&sgx_free_list_lock); + kfree(entry); + } + return -ENOMEM; +}
Freeing the entire list on failure does not seem like the appropriate behavior for this helper func, e.g. the list should be purged by sgx_page_cache_teardown. Buffering the new pages into a local list and only splicing said list into the global list upon success is more inline with what I would expect from a helper func, and also only requires a single lock/unlock.
(quoted diff) +0 -0
drivers/platform/x86/intel_sgx/sgx_page_cache.c
index f8883d24692a..38496e6296f1 100644
platform/x86/intel_sgx/sgx_page_cache.c +0 -0
@@ -397,7 +397,8 @@ int sgx_add_epc_bank(resource_size_t start, unsigned long
size, int bank)
{
unsigned long i;
struct sgx_epc_page *new_epc_page, *entry;
- struct list_head *parser, *temp;
+ unsigned int nr_pages = 0;
+ LIST_HEAD(epc_pages);
for (i = 0; i < size; i += PAGE_SIZE) {
new_epc_page = kzalloc(sizeof(*new_epc_page), GFP_KERNEL); @@ -405,22 +406,19 @@ int sgx_add_epc_bank(resource_size_t start, unsigned long size, int bank)
goto err_freelist;
new_epc_page->pa = (start + i) | bank;
- spin_lock(&sgx_free_list_lock);
- list_add_tail(&new_epc_page->list, &sgx_free_list); - sgx_nr_total_epc_pages++;
- sgx_nr_free_pages++;
- spin_unlock(&sgx_free_list_lock);
+ list_add_tail(&new_epc_page->list, &epc_pages);
+ nr_pages++;
}
+ spin_lock(&sgx_free_list_lock);
+ list_splice_tail(&epc_pages, &sgx_free_list);
+ sgx_nr_total_epc_pages += nr_pages;
+ sgx_nr_free_pages += nr_pages;
+ spin_unlock(&sgx_free_list_lock);
return 0;
err_freelist:
- list_for_each_safe(parser, temp, &sgx_free_list) {
- spin_lock(&sgx_free_list_lock);
- entry = list_entry(parser, struct sgx_epc_page, list); - list_del(&entry->list);
- spin_unlock(&sgx_free_list_lock);
+ list_for_each_entry(entry, &sgx_free_list, list) kfree(entry);
- }
return -ENOMEM;
}
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
Quoted 90 lines · Sean Christopherson
Quoted 89 lines · Jarkko Sakkinen
(quoted diff) +16
-0+static int sgx_dev_init(struct device *parent, bool locked_msrs) +{ + struct sgx_context *sgx_dev; + unsigned int eax, ebx, ecx, edx; + unsigned long pa; + unsigned long size; + int ret; + int i; + + pr_info("intel_sgx: " DRV_DESCRIPTION " v" DRV_VERSION "\n"); + + sgx_dev = sgxm_ctx_alloc(parent); + + sgx_locked_msrs = locked_msrs; + + cpuid_count(SGX_CPUID, SGX_CPUID_CAPABILITIES, &eax, &ebx, &ecx,&edx);
(quoted diff) +9
-0+ /* Only allow misc bits supported by the driver. */ + sgx_misc_reserved = ~ebx | SGX_MISC_RESERVED_MASK; +#ifdef CONFIG_X86_64 + sgx_encl_size_max_64 = 1ULL << ((edx >> 8) & 0xFF); +#endif + sgx_encl_size_max_32 = 1ULL << (edx & 0xFF); + + if (boot_cpu_has(X86_FEATURE_OSXSAVE)) { + cpuid_count(SGX_CPUID, SGX_CPUID_ATTRIBUTES, &eax, &ebx,&ecx,
(quoted diff) +18
-0+ &edx); + sgx_xfrm_mask = (((u64)edx) << 32) + (u64)ecx; + + for (i = 2; i < 64; i++) { + cpuid_count(0x0D, i, &eax, &ebx, &ecx, &edx); + if ((1 << i) & sgx_xfrm_mask) + sgx_xsave_size_tbl[i] = eax + ebx; + } + } + + for (i = 0; i < SGX_MAX_EPC_BANKS; i++) { + cpuid_count(SGX_CPUID, i + SGX_CPUID_EPC_BANKS, &eax, &ebx, + &ecx, &edx); + if (!(eax & 0xf)) + break; + + pa = ((u64)(ebx & 0xfffff) << 32) + (u64)(eax & 0xfffff000); + size = ((u64)(edx & 0xfffff) << 32) + (u64)(ecx &
0xfffff000);(quoted diff) +43
-0+ + dev_info(parent, "EPC bank 0x%lx-0x%lx\n", pa, pa + size); + + sgx_epc_banks[i].pa = pa; + sgx_epc_banks[i].size = size; + } + + sgx_nr_epc_banks = i; + + for (i = 0; i < sgx_nr_epc_banks; i++) { +#ifdef CONFIG_X86_64 + sgx_epc_banks[i].va = (unsigned long) + ioremap_cache(sgx_epc_banks[i].pa, + sgx_epc_banks[i].size); + if (!sgx_epc_banks[i].va) { + sgx_nr_epc_banks = i; + ret = -ENOMEM; + goto out_iounmap; + } +#endif + ret = sgx_add_epc_bank(sgx_epc_banks[i].pa, + sgx_epc_banks[i].size, i); + if (ret) { + sgx_nr_epc_banks = i + 1; + goto out_iounmap; + } + } + + ret = sgx_page_cache_init(); + if (ret) + goto out_iounmap; + + sgx_add_page_wq = alloc_workqueue("intel_sgx-add-page-wq", + WQ_UNBOUND | WQ_FREEZABLE, 1); + if (!sgx_add_page_wq) { + pr_err("intel_sgx: alloc_workqueue() failed\n"); + ret = -ENOMEM; + goto out_iounmap; + } + + ret = cdev_device_add(&sgx_dev->cdev, &sgx_dev->dev); + if (ret) + goto out_workqueue;
Quoted 9 lines
Review
Quoted 5 lines
(quoted diff) +5
-0+ + return 0; +out_workqueue: + destroy_workqueue(sgx_add_page_wq); +out_iounmap:
sgx_page_cache_teardown()should be called here, else ksgxswapd and the list of EPC pages will leak.
Thanks.
Quoted 84 lines
Quoted 6 lines
(quoted diff) +6
-0+#ifdef CONFIG_X86_64 + for (i = 0; i < sgx_nr_epc_banks; i++) + iounmap((void *)sgx_epc_banks[i].va); +#endif + return ret; +}...
Quoted 20 lines
(quoted diff) +20
-0+int sgx_add_epc_bank(resource_size_t start, unsigned long size, int bank) +{ + unsigned long i; + struct sgx_epc_page *new_epc_page, *entry; + struct list_head *parser, *temp; + + for (i = 0; i < size; i += PAGE_SIZE) { + new_epc_page = kzalloc(sizeof(*new_epc_page), GFP_KERNEL); + if (!new_epc_page) + goto err_freelist; + new_epc_page->pa = (start + i) | bank; + + spin_lock(&sgx_free_list_lock); + list_add_tail(&new_epc_page->list, &sgx_free_list); + sgx_nr_total_epc_pages++; + sgx_nr_free_pages++; + spin_unlock(&sgx_free_list_lock); + } + + return 0;Review
Quoted 10 lines
(quoted diff) +10
-0+err_freelist: + list_for_each_safe(parser, temp, &sgx_free_list) { + spin_lock(&sgx_free_list_lock); + entry = list_entry(parser, struct sgx_epc_page, list); + list_del(&entry->list); + spin_unlock(&sgx_free_list_lock); + kfree(entry); + } + return -ENOMEM; +}Freeing the entire list on failure does not seem like the appropriate behavior for this helper func, e.g. the list should be purged by
sgx_page_cache_teardown. Buffering the new pages into a local list and only splicing said list into the global list upon success is more inline with what I would expect from a helper func, and also only requires a single lock/unlock.(quoted diff) +0
-0
drivers/platform/x86/intel_sgx/sgx_page_cache.c
indexf8883d24692a..38496e6296f1100644platform/x86/intel_sgx/sgx_page_cache.c +1
-1@@ -397,7 +397,8 @@ int sgx_add_epc_bank(resource_size_t start, unsigned long size, int bank) { unsigned long i; struct sgx_epc_page *new_epc_page, *entry; - struct list_head *parser, *temp; + LIST_HEAD(epc_pages);
for (i = 0; i < size; i +=PAGE_SIZE) {
new_epc_page =kzalloc(sizeof(*new_epc_page), GFP_KERNEL);platform/x86/intel_sgx/sgx_page_cache.c +0
-0@@ -405,22 +406,19 @@ int sgx_add_epc_bank(resource_size_t start, unsigned long size, int bank) goto err_freelist; new_epc_page->pa = (start + i) | bank;
- spin_lock(&sgx_free_list_lock);
- list_add_tail(&new_epc_page->list,&sgx_free_list); - sgx_nr_total_epc_pages++;
- sgx_nr_free_pages++;
- spin_unlock(&sgx_free_list_lock);
+ list_add_tail(&new_epc_page->list,&epc_pages); + nr_pages++;
}
+ spin_lock(&sgx_free_list_lock);
+ list_splice_tail(&epc_pages,&sgx_free_list);
+ sgx_nr_total_epc_pages +=nr_pages;
+ sgx_nr_free_pages +=nr_pages;
+ spin_unlock(&sgx_free_list_lock);
return 0;
err_freelist:
- list_for_each_safe(parser, temp,&sgx_free_list) { - spin_lock(&sgx_free_list_lock);
Quoted 8 lines
- entry =
list_entry(parser, struct sgx_epc_page, list); - list_del(&entry->list);
- spin_unlock(&sgx_free_list_lock);
+ list_for_each_entry(entry,&sgx_free_list, list) kfree(entry);
- }
return-ENOMEM;
}
This makes sense to me and I think it makes also sense to move bunch of stuff from sgx_dev_init() to sgx_page_cache_init().
/Jarkko
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
Quoted 199 lines · Jarkko Sakkinen
Quoted 90 lines · Sean Christopherson
Quoted 89 lines · Jarkko Sakkinen
(quoted diff) +16
-0+static int sgx_dev_init(struct device *parent, bool locked_msrs) +{ + struct sgx_context *sgx_dev; + unsigned int eax, ebx, ecx, edx; + unsigned long pa; + unsigned long size; + int ret; + int i; + + pr_info("intel_sgx: " DRV_DESCRIPTION " v" DRV_VERSION "\n"); + + sgx_dev = sgxm_ctx_alloc(parent); + + sgx_locked_msrs = locked_msrs; + + cpuid_count(SGX_CPUID, SGX_CPUID_CAPABILITIES, &eax, &ebx, &ecx,&edx);
(quoted diff) +9
-0+ /* Only allow misc bits supported by the driver. */ + sgx_misc_reserved = ~ebx | SGX_MISC_RESERVED_MASK; +#ifdef CONFIG_X86_64 + sgx_encl_size_max_64 = 1ULL << ((edx >> 8) & 0xFF); +#endif + sgx_encl_size_max_32 = 1ULL << (edx & 0xFF); + + if (boot_cpu_has(X86_FEATURE_OSXSAVE)) { + cpuid_count(SGX_CPUID, SGX_CPUID_ATTRIBUTES, &eax, &ebx,&ecx,
(quoted diff) +18
-0+ &edx); + sgx_xfrm_mask = (((u64)edx) << 32) + (u64)ecx; + + for (i = 2; i < 64; i++) { + cpuid_count(0x0D, i, &eax, &ebx, &ecx, &edx); + if ((1 << i) & sgx_xfrm_mask) + sgx_xsave_size_tbl[i] = eax + ebx; + } + } + + for (i = 0; i < SGX_MAX_EPC_BANKS; i++) { + cpuid_count(SGX_CPUID, i + SGX_CPUID_EPC_BANKS, &eax, &ebx, + &ecx, &edx); + if (!(eax & 0xf)) + break; + + pa = ((u64)(ebx & 0xfffff) << 32) + (u64)(eax & 0xfffff000); + size = ((u64)(edx & 0xfffff) << 32) + (u64)(ecx &
0xfffff000);(quoted diff) +43
-0+ + dev_info(parent, "EPC bank 0x%lx-0x%lx\n", pa, pa + size); + + sgx_epc_banks[i].pa = pa; + sgx_epc_banks[i].size = size; + } + + sgx_nr_epc_banks = i; + + for (i = 0; i < sgx_nr_epc_banks; i++) { +#ifdef CONFIG_X86_64 + sgx_epc_banks[i].va = (unsigned long) + ioremap_cache(sgx_epc_banks[i].pa, + sgx_epc_banks[i].size); + if (!sgx_epc_banks[i].va) { + sgx_nr_epc_banks = i; + ret = -ENOMEM; + goto out_iounmap; + } +#endif + ret = sgx_add_epc_bank(sgx_epc_banks[i].pa, + sgx_epc_banks[i].size, i); + if (ret) { + sgx_nr_epc_banks = i + 1; + goto out_iounmap; + } + } + + ret = sgx_page_cache_init(); + if (ret) + goto out_iounmap; + + sgx_add_page_wq = alloc_workqueue("intel_sgx-add-page-wq", + WQ_UNBOUND | WQ_FREEZABLE, 1); + if (!sgx_add_page_wq) { + pr_err("intel_sgx: alloc_workqueue() failed\n"); + ret = -ENOMEM; + goto out_iounmap; + } + + ret = cdev_device_add(&sgx_dev->cdev, &sgx_dev->dev); + if (ret) + goto out_workqueue;Quoted 9 lines
Review
Quoted 5 lines
(quoted diff) +5
-0+ + return 0; +out_workqueue: + destroy_workqueue(sgx_add_page_wq); +out_iounmap:
sgx_page_cache_teardown()should be called here, else ksgxswapd and the list of EPC pages will leak.Thanks.
Quoted 84 lines
Quoted 6 lines
(quoted diff) +6
-0+#ifdef CONFIG_X86_64 + for (i = 0; i < sgx_nr_epc_banks; i++) + iounmap((void *)sgx_epc_banks[i].va); +#endif + return ret; +}...
Quoted 20 lines
(quoted diff) +20
-0+int sgx_add_epc_bank(resource_size_t start, unsigned long size, int bank) +{ + unsigned long i; + struct sgx_epc_page *new_epc_page, *entry; + struct list_head *parser, *temp; + + for (i = 0; i < size; i += PAGE_SIZE) { + new_epc_page = kzalloc(sizeof(*new_epc_page), GFP_KERNEL); + if (!new_epc_page) + goto err_freelist; + new_epc_page->pa = (start + i) | bank; + + spin_lock(&sgx_free_list_lock); + list_add_tail(&new_epc_page->list, &sgx_free_list); + sgx_nr_total_epc_pages++; + sgx_nr_free_pages++; + spin_unlock(&sgx_free_list_lock); + } + + return 0;Review
Quoted 10 lines
(quoted diff) +10
-0+err_freelist: + list_for_each_safe(parser, temp, &sgx_free_list) { + spin_lock(&sgx_free_list_lock); + entry = list_entry(parser, struct sgx_epc_page, list); + list_del(&entry->list); + spin_unlock(&sgx_free_list_lock); + kfree(entry); + } + return -ENOMEM; +}Freeing the entire list on failure does not seem like the appropriate behavior for this helper func, e.g. the list should be purged by
sgx_page_cache_teardown. Buffering the new pages into a local list and only splicing said list into the global list upon success is more inline with what I would expect from a helper func, and also only requires a single lock/unlock.(quoted diff) +0
-0
drivers/platform/x86/intel_sgx/sgx_page_cache.c
indexf8883d24692a..38496e6296f1100644platform/x86/intel_sgx/sgx_page_cache.c +1
-1@@ -397,7 +397,8 @@ int sgx_add_epc_bank(resource_size_t start, unsigned long size, int bank) { unsigned long i; struct sgx_epc_page *new_epc_page, *entry; - struct list_head *parser, *temp; + LIST_HEAD(epc_pages);
for (i = 0; i < size; i +=PAGE_SIZE) {
new_epc_page =kzalloc(sizeof(*new_epc_page), GFP_KERNEL);platform/x86/intel_sgx/sgx_page_cache.c +0
-0@@ -405,22 +406,19 @@ int sgx_add_epc_bank(resource_size_t start, unsigned long size, int bank) goto err_freelist; new_epc_page->pa = (start + i) | bank;
- spin_lock(&sgx_free_list_lock);
- list_add_tail(&new_epc_page->list,&sgx_free_list); - sgx_nr_total_epc_pages++;
- sgx_nr_free_pages++;
- spin_unlock(&sgx_free_list_lock);
+ list_add_tail(&new_epc_page->list,&epc_pages); + nr_pages++;
}
+ spin_lock(&sgx_free_list_lock);
+ list_splice_tail(&epc_pages,&sgx_free_list);
+ sgx_nr_total_epc_pages +=nr_pages;
+ sgx_nr_free_pages +=nr_pages;
+ spin_unlock(&sgx_free_list_lock);
return 0;
err_freelist:
- list_for_each_safe(parser, temp,&sgx_free_list) { - spin_lock(&sgx_free_list_lock);Quoted 8 lines
- entry =
list_entry(parser, struct sgx_epc_page, list); - list_del(&entry->list);
- spin_unlock(&sgx_free_list_lock);
+ list_for_each_entry(entry,&sgx_free_list, list) kfree(entry);
- }
return-ENOMEM;
}This makes sense to me and I think it makes also sense to move bunch of stuff from
sgx_dev_init()tosgx_page_cache_init().
Quoted 1 line
/Jarkko
You can get away without nr_pages as you can get from i or am I misobserving something?
/Jarkko
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
Quoted 216 lines · Jarkko Sakkinen
Quoted 207 lines · Jarkko Sakkinen
Quoted 98 lines · Sean Christopherson
Quoted 96 lines · Jarkko Sakkinen
(quoted diff) +16
-0+static int sgx_dev_init(struct device *parent, bool locked_msrs) +{ + struct sgx_context *sgx_dev; + unsigned int eax, ebx, ecx, edx; + unsigned long pa; + unsigned long size; + int ret; + int i; + + pr_info("intel_sgx: " DRV_DESCRIPTION " v" DRV_VERSION "\n"); + + sgx_dev = sgxm_ctx_alloc(parent); + + sgx_locked_msrs = locked_msrs; + + cpuid_count(SGX_CPUID, SGX_CPUID_CAPABILITIES, &eax, &ebx,&ecx,
&edx);(quoted diff) +9
-0+ /* Only allow misc bits supported by the driver. */ + sgx_misc_reserved = ~ebx | SGX_MISC_RESERVED_MASK; +#ifdef CONFIG_X86_64 + sgx_encl_size_max_64 = 1ULL << ((edx >> 8) & 0xFF); +#endif + sgx_encl_size_max_32 = 1ULL << (edx & 0xFF); + + if (boot_cpu_has(X86_FEATURE_OSXSAVE)) { + cpuid_count(SGX_CPUID, SGX_CPUID_ATTRIBUTES, &eax,&ebx,
&ecx,(quoted diff) +12
-0+ &edx); + sgx_xfrm_mask = (((u64)edx) << 32) + (u64)ecx; + + for (i = 2; i < 64; i++) { + cpuid_count(0x0D, i, &eax, &ebx, &ecx, &edx); + if ((1 << i) & sgx_xfrm_mask) + sgx_xsave_size_tbl[i] = eax + ebx; + } + } + + for (i = 0; i < SGX_MAX_EPC_BANKS; i++) { + cpuid_count(SGX_CPUID, i + SGX_CPUID_EPC_BANKS, &eax,&ebx,
(quoted diff) +5
-0+ &ecx, &edx); + if (!(eax & 0xf)) + break; + + pa = ((u64)(ebx & 0xfffff) << 32) + (u64)(eax &
0xfffff000);(quoted diff) +1
-0+ size = ((u64)(edx & 0xfffff) << 32) + (u64)(ecx &
0xfffff000);(quoted diff) +2
-0+ + dev_info(parent, "EPC bank 0x%lx-0x%lx\n", pa, pa +size);
(quoted diff) +32
-0+ + sgx_epc_banks[i].pa = pa; + sgx_epc_banks[i].size = size; + } + + sgx_nr_epc_banks = i; + + for (i = 0; i < sgx_nr_epc_banks; i++) { +#ifdef CONFIG_X86_64 + sgx_epc_banks[i].va = (unsigned long) + ioremap_cache(sgx_epc_banks[i].pa, + sgx_epc_banks[i].size); + if (!sgx_epc_banks[i].va) { + sgx_nr_epc_banks = i; + ret = -ENOMEM; + goto out_iounmap; + } +#endif + ret = sgx_add_epc_bank(sgx_epc_banks[i].pa, + sgx_epc_banks[i].size, i); + if (ret) { + sgx_nr_epc_banks = i + 1; + goto out_iounmap; + } + } + + ret = sgx_page_cache_init(); + if (ret) + goto out_iounmap; + + sgx_add_page_wq = alloc_workqueue("intel_sgx-add-page-wq", + WQ_UNBOUND | WQ_FREEZABLE,1);
(quoted diff) +9
-0+ if (!sgx_add_page_wq) { + pr_err("intel_sgx: alloc_workqueue() failed\n"); + ret = -ENOMEM; + goto out_iounmap; + } + + ret = cdev_device_add(&sgx_dev->cdev, &sgx_dev->dev); + if (ret) + goto out_workqueue;Quoted 9 lines
Review
Quoted 5 lines
(quoted diff) +5
-0+ + return 0; +out_workqueue: + destroy_workqueue(sgx_add_page_wq); +out_iounmap:
sgx_page_cache_teardown()should be called here, else ksgxswapd and the list of
EPC pages will leak.Thanks.
Quoted 96 lines
Quoted 8 lines
(quoted diff) +6
-0+#ifdef CONFIG_X86_64 + for (i = 0; i < sgx_nr_epc_banks; i++) + iounmap((void *)sgx_epc_banks[i].va); +#endif + return ret; +}...
Quoted 23 lines
+int
sgx_add_epc_bank(resource_size_t start, unsigned long size, int bank)(quoted diff) +7
-0+{ + unsigned long i; + struct sgx_epc_page *new_epc_page, *entry; + struct list_head *parser, *temp; + + for (i = 0; i < size; i += PAGE_SIZE) { + new_epc_page = kzalloc(sizeof(*new_epc_page),
GFP_KERNEL);(quoted diff) +12
-0+ if (!new_epc_page) + goto err_freelist; + new_epc_page->pa = (start + i) | bank; + + spin_lock(&sgx_free_list_lock); + list_add_tail(&new_epc_page->list, &sgx_free_list); + sgx_nr_total_epc_pages++; + sgx_nr_free_pages++; + spin_unlock(&sgx_free_list_lock); + } + + return 0;Review
Quoted 10 lines
(quoted diff) +10
-0+err_freelist: + list_for_each_safe(parser, temp, &sgx_free_list) { + spin_lock(&sgx_free_list_lock); + entry = list_entry(parser, struct sgx_epc_page, list); + list_del(&entry->list); + spin_unlock(&sgx_free_list_lock); + kfree(entry); + } + return -ENOMEM; +}Freeing the entire list on failure does not seem like the appropriate behavior
for this helper func, e.g. the list should be purged bysgx_page_cache_teardown.
Buffering the new pages into a local list and only splicing said list into the
global list upon success is more inline with what I would expect from a helper
func, and also only requires a single lock/unlock.(quoted diff) +0
-0
drivers/platform/x86/intel_sgx/sgx_page_cache.c
indexf8883d24692a..38496e6296f1100644platform/x86/intel_sgx/sgx_page_cache.c +0
-0@@ -397,7 +397,8 @@ int sgx_add_epc_bank(resource_size_t start, unsignedlong
size, int bank)
{
unsigned long i;
structsgx_epc_page*new_epc_page, *entry;
- structlist_head*parser, *temp;
+ LIST_HEAD(epc_pages);
for (i = 0; i < size; i +=PAGE_SIZE) {
new_epc_page =kzalloc(sizeof(*new_epc_page), GFP_KERNEL);platform/x86/intel_sgx/sgx_page_cache.c +0
-0@@ -405,22 +406,19 @@ int sgx_add_epc_bank(resource_size_t start, unsignedlong
size, int bank)
gotoerr_freelist;
new_epc_page->pa = (start + i) | bank;
- spin_lock(&sgx_free_list_lock);
- list_add_tail(&new_epc_page->list,&sgx_free_list); - sgx_nr_total_epc_pages++;
- sgx_nr_free_pages++;
- spin_unlock(&sgx_free_list_lock);
+ list_add_tail(&new_epc_page->list,&epc_pages); + nr_pages++;
}
+ spin_lock(&sgx_free_list_lock);
+ list_splice_tail(&epc_pages,&sgx_free_list);
+ sgx_nr_total_epc_pages +=nr_pages;
+ sgx_nr_free_pages +=nr_pages;
+ spin_unlock(&sgx_free_list_lock);
return 0;
err_freelist:
- list_for_each_safe(parser, temp,&sgx_free_list) { - spin_lock(&sgx_free_list_lock);
- entry =list_entry(parser, struct sgx_epc_page, list); - list_del(&entry->list);
- spin_unlock(&sgx_free_list_lock);Quoted 8 lines
Quoted 5 lines
+ list_for_each_entry(entry,
&sgx_free_list, list) kfree(entry);
- }
return-ENOMEM;
}This makes sense to me and I think it makes also sense to move bunch of stuff from
sgx_dev_init()tosgx_page_cache_init().
/Jarkko
Quoted 2 lines
You can get away without
nr_pagesas you can get from i or am I misobserving something?
i is incremented by PAGE_SIZE. I suppose you could do "+= i / PAGE_SIZE", but I would opt for clarity in this case.
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
Quoted 217 lines · Sean Christopherson
Quoted 216 lines · Jarkko Sakkinen
Quoted 215 lines · Jarkko Sakkinen
Quoted 98 lines · Sean Christopherson
Quoted 96 lines · Jarkko Sakkinen
(quoted diff) +16
-0+static int sgx_dev_init(struct device *parent, bool locked_msrs) +{ + struct sgx_context *sgx_dev; + unsigned int eax, ebx, ecx, edx; + unsigned long pa; + unsigned long size; + int ret; + int i; + + pr_info("intel_sgx: " DRV_DESCRIPTION " v" DRV_VERSION "\n"); + + sgx_dev = sgxm_ctx_alloc(parent); + + sgx_locked_msrs = locked_msrs; + + cpuid_count(SGX_CPUID, SGX_CPUID_CAPABILITIES, &eax, &ebx,&ecx,
&edx);(quoted diff) +9
-0+ /* Only allow misc bits supported by the driver. */ + sgx_misc_reserved = ~ebx | SGX_MISC_RESERVED_MASK; +#ifdef CONFIG_X86_64 + sgx_encl_size_max_64 = 1ULL << ((edx >> 8) & 0xFF); +#endif + sgx_encl_size_max_32 = 1ULL << (edx & 0xFF); + + if (boot_cpu_has(X86_FEATURE_OSXSAVE)) { + cpuid_count(SGX_CPUID, SGX_CPUID_ATTRIBUTES, &eax,&ebx,
&ecx,(quoted diff) +12
-0+ &edx); + sgx_xfrm_mask = (((u64)edx) << 32) + (u64)ecx; + + for (i = 2; i < 64; i++) { + cpuid_count(0x0D, i, &eax, &ebx, &ecx, &edx); + if ((1 << i) & sgx_xfrm_mask) + sgx_xsave_size_tbl[i] = eax + ebx; + } + } + + for (i = 0; i < SGX_MAX_EPC_BANKS; i++) { + cpuid_count(SGX_CPUID, i + SGX_CPUID_EPC_BANKS, &eax,&ebx,
(quoted diff) +5
-0+ &ecx, &edx); + if (!(eax & 0xf)) + break; + + pa = ((u64)(ebx & 0xfffff) << 32) + (u64)(eax &
0xfffff000);(quoted diff) +1
-0+ size = ((u64)(edx & 0xfffff) << 32) + (u64)(ecx &
0xfffff000);(quoted diff) +2
-0+ + dev_info(parent, "EPC bank 0x%lx-0x%lx\n", pa, pa +size);
(quoted diff) +32
-0+ + sgx_epc_banks[i].pa = pa; + sgx_epc_banks[i].size = size; + } + + sgx_nr_epc_banks = i; + + for (i = 0; i < sgx_nr_epc_banks; i++) { +#ifdef CONFIG_X86_64 + sgx_epc_banks[i].va = (unsigned long) + ioremap_cache(sgx_epc_banks[i].pa, + sgx_epc_banks[i].size); + if (!sgx_epc_banks[i].va) { + sgx_nr_epc_banks = i; + ret = -ENOMEM; + goto out_iounmap; + } +#endif + ret = sgx_add_epc_bank(sgx_epc_banks[i].pa, + sgx_epc_banks[i].size, i); + if (ret) { + sgx_nr_epc_banks = i + 1; + goto out_iounmap; + } + } + + ret = sgx_page_cache_init(); + if (ret) + goto out_iounmap; + + sgx_add_page_wq = alloc_workqueue("intel_sgx-add-page-wq", + WQ_UNBOUND | WQ_FREEZABLE,1);
(quoted diff) +9
-0+ if (!sgx_add_page_wq) { + pr_err("intel_sgx: alloc_workqueue() failed\n"); + ret = -ENOMEM; + goto out_iounmap; + } + + ret = cdev_device_add(&sgx_dev->cdev, &sgx_dev->dev); + if (ret) + goto out_workqueue;Quoted 9 lines
Review
Quoted 5 lines
(quoted diff) +5
-0+ + return 0; +out_workqueue: + destroy_workqueue(sgx_add_page_wq); +out_iounmap:
sgx_page_cache_teardown()should be called here, else ksgxswapd and the list of
EPC pages will leak.Thanks.
Quoted 93 lines
Quoted 8 lines
(quoted diff) +6
-0+#ifdef CONFIG_X86_64 + for (i = 0; i < sgx_nr_epc_banks; i++) + iounmap((void *)sgx_epc_banks[i].va); +#endif + return ret; +}...
Quoted 23 lines
+int
sgx_add_epc_bank(resource_size_t start, unsigned long size, int bank)(quoted diff) +7
-0+{ + unsigned long i; + struct sgx_epc_page *new_epc_page, *entry; + struct list_head *parser, *temp; + + for (i = 0; i < size; i += PAGE_SIZE) { + new_epc_page = kzalloc(sizeof(*new_epc_page),
GFP_KERNEL);(quoted diff) +12
-0+ if (!new_epc_page) + goto err_freelist; + new_epc_page->pa = (start + i) | bank; + + spin_lock(&sgx_free_list_lock); + list_add_tail(&new_epc_page->list, &sgx_free_list); + sgx_nr_total_epc_pages++; + sgx_nr_free_pages++; + spin_unlock(&sgx_free_list_lock); + } + + return 0;Review
Quoted 10 lines
(quoted diff) +10
-0+err_freelist: + list_for_each_safe(parser, temp, &sgx_free_list) { + spin_lock(&sgx_free_list_lock); + entry = list_entry(parser, struct sgx_epc_page, list); + list_del(&entry->list); + spin_unlock(&sgx_free_list_lock); + kfree(entry); + } + return -ENOMEM; +}Freeing the entire list on failure does not seem like the appropriate behavior
for this helper func, e.g. the list should be purged bysgx_page_cache_teardown.
Buffering the new pages into a local list and only splicing said list into the
global list upon success is more inline with what I would expect from a helper
func, and also only requires a single lock/unlock.(quoted diff) +0
-0
drivers/platform/x86/intel_sgx/sgx_page_cache.c
indexf8883d24692a..38496e6296f1100644platform/x86/intel_sgx/sgx_page_cache.c +0
-0@@ -397,7 +397,8 @@ int sgx_add_epc_bank(resource_size_t start, unsignedlong
size, int bank)
{
unsigned long i;
structsgx_epc_page*new_epc_page, *entry;
- structlist_head*parser, *temp;
+ LIST_HEAD(epc_pages);
for (i = 0; i < size; i +=PAGE_SIZE) {
new_epc_page =kzalloc(sizeof(*new_epc_page), GFP_KERNEL);platform/x86/intel_sgx/sgx_page_cache.c +0
-0@@ -405,22 +406,19 @@ int sgx_add_epc_bank(resource_size_t start, unsignedlong
size, int bank)
gotoerr_freelist;new_epc_page->pa= (start + i) | bank;
- spin_lock(&sgx_free_list_lock);
- list_add_tail(&new_epc_page->list,&sgx_free_list); - sgx_nr_total_epc_pages++;
- sgx_nr_free_pages++;
- spin_unlock(&sgx_free_list_lock);
+ list_add_tail(&new_epc_page->list,&epc_pages); + nr_pages++;
}
+ spin_lock(&sgx_free_list_lock);
+ list_splice_tail(&epc_pages,&sgx_free_list);
+ sgx_nr_total_epc_pages +=nr_pages;
+ sgx_nr_free_pages +=nr_pages;
+ spin_unlock(&sgx_free_list_lock);
return 0;
err_freelist:
- list_for_each_safe(parser, temp,&sgx_free_list) { - spin_lock(&sgx_free_list_lock);Quoted 8 lines
- entry =
list_entry(parser, struct sgx_epc_page, list); - list_del(&entry->list);
- spin_unlock(&sgx_free_list_lock);
+ list_for_each_entry(entry,&sgx_free_list, list) kfree(entry);
- }
return-ENOMEM;
}This makes sense to me and I think it makes also sense to move bunch of stuff from
sgx_dev_init()tosgx_page_cache_init().
/Jarkko
Quoted 5 lines
Quoted 2 lines
You can get away without
nr_pagesas you can get from i or am I misobserving something?i is incremented by
PAGE_SIZE. I suppose you could do "+= i /PAGE_SIZE", but I would opt for clarity in this case.
Redundant calculations do not improve clarity.
/Jarkko
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
In reply to Sean ChristophersonQuoted 3 lines · Sean Christopherson
+ list_for_each_entry(entry,
&sgx_free_list, list) kfree(entry);
- }
Shouldn't you iterate 'epc_pages' in the rollback now?
/Jarkko
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
Quoted 6 lines · Jarkko Sakkinen
Quoted 4 lines · Sean Christopherson
+ list_for_each_entry(entry,
&sgx_free_list, list) kfree(entry);
- }Shouldn't you iterate '
epc_pages' in the rollback now?
Yes, that's the intent, nice catch.
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
Quoted 9 lines · Sean Christopherson
Quoted 6 lines · Jarkko Sakkinen
Quoted 4 lines · Sean Christopherson
+ list_for_each_entry(entry,
&sgx_free_list, list) kfree(entry);
- }Shouldn't you iterate '
epc_pages' in the rollback now?Yes, that's the intent, nice catch.
Yeah, just a sanity check that I'm misinterpreting what I'm seeing :-) Thanks.
/Jarkko
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
In reply to a message not in the archiveReview
Quoted 2 lines · Jarkko Sakkinen
(quoted diff) +2
-0+static LIST_HEAD(sgx_free_list); +static DEFINE_SPINLOCK(sgx_free_list_lock);
Is this a global list? Will this be a scalability problem on larger systems?
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
Quoted 6 lines · Dave Hansen
Review
Quoted 2 lines · Jarkko Sakkinen
(quoted diff) +2
-0+static LIST_HEAD(sgx_free_list); +static DEFINE_SPINLOCK(sgx_free_list_lock);Is this a global list? Will this be a scalability problem on larger systems?
It will be need to be refined for NUMA.
In addition, per-CPU caches would probably make sense.
For simplicity, I would keep it as it is up until the driver is in the mainline.
/Jarkko
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
Quoted 8 lines · Jarkko Sakkinen
Quoted 6 lines · Dave Hansen
Review
Quoted 2 lines · Jarkko Sakkinen
(quoted diff) +2
-0+static LIST_HEAD(sgx_free_list); +static DEFINE_SPINLOCK(sgx_free_list_lock);Is this a global list? Will this be a scalability problem on larger systems?
Quoted 6 lines
It will be need to be refined for NUMA.
In addition, per-CPU caches would probably make sense.
For simplicity, I would keep it as it is up until the driver is in the mainline.
FWIW, I don't think we should merge things that aren't performant. Global locks like this are just intolerable. You can add this as a later patch, but please don't merge stuff like this.
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
Quoted 11 lines · Dave Hansen
Quoted 10 lines · Jarkko Sakkinen
Quoted 6 lines · Dave Hansen
Review
Quoted 2 lines · Jarkko Sakkinen
(quoted diff) +2
-0+static LIST_HEAD(sgx_free_list); +static DEFINE_SPINLOCK(sgx_free_list_lock);Is this a global list? Will this be a scalability problem on larger systems?
It will be need to be refined for NUMA.
Quoted 8 lines
Quoted 4 lines
In addition, per-CPU caches would probably make sense.
For simplicity, I would keep it as it is up until the driver is in the mainline.
FWIW, I don't think we should merge things that aren't performant. Global locks like this are just intolerable. You can add this as a later patch, but please don't merge stuff like this.
The back pointer to struct sgx_encl_page from struct sgx_epc_page is useless. I've had this in backlog already long time ago but had forgot it as I've been mostly working with the launch infrastructure lately.
Your comment worked as kind of a reminder of that. Thank you.
Once that field is removed the whole struct is useless and EPC bank converges to an array. With an array the driver should be able reserve pages without a global lock.
I've started writing a patch to make all this happen and it is progressing really well. I'm planning to include this change to v6. As it simplifies code I'm going to squash it as part of the initial driver patch.
How does this sound?
/Jarkko
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
Quoted 31 lines · Jarkko Sakkinen
Quoted 11 lines · Dave Hansen
Quoted 10 lines · Jarkko Sakkinen
Quoted 6 lines · Dave Hansen
Review
Quoted 2 lines · Jarkko Sakkinen
(quoted diff) +2
-0+static LIST_HEAD(sgx_free_list); +static DEFINE_SPINLOCK(sgx_free_list_lock);Is this a global list? Will this be a scalability problem on larger systems?
It will be need to be refined for NUMA.
Quoted 8 lines
Quoted 4 lines
In addition, per-CPU caches would probably make sense.
For simplicity, I would keep it as it is up until the driver is in the mainline.
FWIW, I don't think we should merge things that aren't performant. Global locks like this are just intolerable. You can add this as a later patch, but please don't merge stuff like this.
The back pointer to struct
sgx_encl_pagefrom structsgx_epc_pageis useless. I've had this in backlog already long time ago but had forgot it as I've been mostly working with the launch infrastructure lately.Your comment worked as kind of a reminder of that. Thank you.
Once that field is removed the whole struct is useless and EPC bank converges to an array. With an array the driver should be able reserve pages without a global lock.
Quoted 6 lines
I've started writing a patch to make all this happen and it is progressing really well. I'm planning to include this change to v6. As it simplifies code I'm going to squash it as part of the initial driver patch.
How does this sound?
Every sgx_epc_bank will have a bitmap array for reserved EPC.
Every unswapped sgx_encl_page will have a pointer containing physical address of the EPC page in the upper bits and bank number in the lower bits (like sgx_epc_page has now in the 'pa' field).
This layout does not require a global lock.
/Jarkko
Re: [intel-sgx-kernel-dev] [PATCH RFC v3 07/12] intel_sgx: driver for Intel Software Guard Extensions
Quoted 6 lines · Jarkko Sakkinen
I've started writing a patch to make all this happen and it is progressing really well. I'm planning to include this change to v6. As it simplifies code I'm going to squash it as part of the initial driver patch.
How does this sound?
Sounds sane to me.