x86/sgx: Make vDSO callable from C
Subsystems (5)
v2 cover x86/sgx: Make vDSO callable from C
Nathaniel pointed out that __vdso_sgx_enter_enclave() is tantalizingly close to being callable from C (with caveats and a cooperative enclave). The missing pieces are preserving %rbx and taking @leaf as a standard parameter.
v2:
- Rebase to Jarkko's latest master, commit 402fb35a477a, "docs: ...")
- Add CFI directive for RBX. [Cedric]
v1:
- https://patchwork.kernel.org/cover/11446355/
Sean Christopherson (5):
x86/sgx: vdso: Make __vdso_sgx_enter_enclave() callable from C code x86/sgx: vdso: Define a typedef for __vdso_sgx_enter_enclave selftests/sgx: Pass EENTER to vDSO wrapper instead of hardcoding selftests/sgx: Stop clobbering non-volatile registers selftests/sgx: Add selftest to invoke __vsgx_enter_enclave() from C
arch/x86/entry/vdso/vsgx_enter_enclave.S | 64 ++----------------- arch/x86/include/uapi/asm/sgx.h | 61 ++++++++++++++++++ tools/testing/selftests/sgx/call.S | 1 - tools/testing/selftests/sgx/defines.h | 1 + tools/testing/selftests/sgx/main.c | 20 ++++-- tools/testing/selftests/sgx/main.h | 2 +- .../selftests/sgx/test_encl_bootstrap.S | 6 +- 7 files changed, 84 insertions(+), 71 deletions(-)
Signature · 1 line
2.24.1
PATCH v2 1/5 x86/sgx: vdso: Make __vdso_sgx_enter_enclave() callable from C code
Make __vdso_sgx_enter_enclave() callable from C by preserving %rbx and taking @leaf in %rcx instead of %rax. Being able to invoke the vDSO from C reduces the overhead of runtimes that are tightly coupled with their enclaves, e.g. that can rely on the enclave to save and restore non-volatile registers, as the runtime doesn't need an assembly wrapper to preserve non-volatile registers and/or shuffle stack arguments.
Note, both %rcx and %rbx are consumed by EENTER/ERESUME, i.e. consuming them doesn't violate the primary tenet of __vdso_sgx_enter_enclave() that "thou shalt not restrict how information is exchanged between an enclave and its host process".
--- arch/x86/entry/vdso/vsgx_enter_enclave.S | 30 ++++++++++++++---------- 1 file changed, 18 insertions(+), 12 deletions(-)
arch/x86/entry/vdso/vsgx_enter_enclave.S +18 -12
@@ -17,22 +17,22 @@ /** * __vdso_sgx_enter_enclave() - Enter an SGX enclave + * @rdi: Pass-through value for RDI + * @rsi: Pass-through value for RSI + * @rdx: Pass-through value for RDX * @leaf: ENCLU leaf, must be EENTER or ERESUME + * @r8: Pass-through value for R8 + * @r9: Pass-through value for R9 * @tcs: TCS, must be non-NULL * @e: Optional struct sgx_enclave_exception instance * @handler: Optional enclave exit handler * - * **Important!** __vdso_sgx_enter_enclave() is **NOT** compliant with the - * x86-64 ABI, i.e. cannot be called from standard C code. - * - * Input ABI: - * @leaf %eax - * @tcs 8(%rsp) - * @e 0x10(%rsp) - * @handler 0x18(%rsp) - * - * Output ABI: - * @ret %eax + * **Important!** __vdso_sgx_enter_enclave() does not ensure full compliance + * with the x86-64 ABI, e.g. doesn't explicitly clear EFLAGS.DF after EEXIT. + * Except for non-volatile general purpose registers, preserving/setting state + * in accordance with the x86-64 ABI is the responsibility of the enclave and + * its runtime, i.e. __vdso_sgx_enter_enclave() cannot be called from C code + * without careful consideration by both the enclave and its runtime. * * All general purpose registers except RAX, RBX and RCX are passed as-is to * the enclave. RAX, RBX and RCX are consumed by EENTER and ERESUME and are
@@ -71,7 +71,9 @@ */ #ifdef SGX_KERNEL_DOC /* C-style function prototype to coerce kernel-doc into parsing the comment. */ -int __vdso_sgx_enter_enclave(int leaf, void *tcs, +int __vdso_sgx_enter_enclave(unsigned long rdi, unsigned long rsi, + unsigned long rdx, unsigned int leaf, + unsigned long r8, unsigned long r9, void *tcs, struct sgx_enclave_exception *e, sgx_enclave_exit_handler_t handler); #endif
@@ -83,7 +85,10 @@ SYM_FUNC_START(__vdso_sgx_enter_enclave) .cfi_rel_offset %rbp, 0 mov %rsp, %rbp .cfi_def_cfa_register %rbp + push %rbx + .cfi_rel_offset %rbx, -8 + mov %ecx, %eax .Lenter_enclave: /* EENTER <= leaf <= ERESUME */ cmp $EENTER, %eax
@@ -109,6 +114,7 @@ SYM_FUNC_START(__vdso_sgx_enter_enclave) jne .Linvoke_userspace_handler .Lout: + pop %rbx leave .cfi_def_cfa %rsp, 8 ret
Signature · 1 line
2.24.1
Re: [PATCH for_v29 v2 1/5] x86/sgx: vdso: Make __vdso_sgx_enter_enclave() callable from C code
Quoted 42 lines · Sean Christopherson
Make
__vdso_sgx_enter_enclave()callable from C by preserving %rbx and taking @leaf in %rcx instead of %rax. Being able to invoke the vDSO from C reduces the overhead of runtimes that are tightly coupled with their enclaves, e.g. that can rely on the enclave to save and restore non-volatile registers, as the runtime doesn't need an assembly wrapper to preserve non-volatile registers and/or shuffle stack arguments.Note, both %rcx and %rbx are consumed by EENTER/ERESUME, i.e. consuming them doesn't violate the primary tenet of
__vdso_sgx_enter_enclave()that "thou shalt not restrict how information is exchanged between an enclave and its host process".Suggested-by Nathaniel McCallum <[email protected]>Cc Cedric Xing <[email protected]>Cc Jethro Beekman <[email protected]>Cc Andy Lutomirski <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/entry/vdso/vsgx_enter_enclave.S | 30 ++++++++++++++---------- 1 file changed, 18 insertions(+), 12 deletions(-)arch/x86/entry/vdso/vsgx_enter_enclave.S +5
-2@@ -17,22 +17,22 @@ /** * __vdso_sgx_enter_enclave() - Enter an SGX enclave + * @rdi: Pass-through value for RDI + * @rsi: Pass-through value for RSI + * @rdx: Pass-through value for RDX * @leaf: ENCLU leaf, must be EENTER or ERESUME + * @r8: Pass-through value for R8 + * @r9: Pass-through value for R9 * @tcs: TCS, must be non-NULL * @e: Optional struct sgx_enclave_exception instance * @handler: Optional enclave exit handler * - * **Important!** __vdso_sgx_enter_enclave() is **NOT** compliant with the - * x86-64 ABI, i.e. cannot be called from standard C code.
arch/x86/entry/vdso/vsgx_enter_enclave.S line 30
Quoted 10 lines
arch/x86/entry/vdso/vsgx_enter_enclave.S +1
-9- * - * Input ABI: - * @leaf %eax - * @tcs 8(%rsp) - * @e 0x10(%rsp) - * @handler 0x18(%rsp) - * - * Output ABI: - * @ret %eax + * **Important!** __vdso_sgx_enter_enclave() does not ensure full compliance
I'd simply put **NOTE** here instead of **Important!** as it is more common.
arch/x86/entry/vdso/vsgx_enter_enclave.S line 35
Quoted 5 lines
arch/x86/entry/vdso/vsgx_enter_enclave.S +5
-0+ * with the x86-64 ABI, e.g. doesn't explicitly clear EFLAGS.DF after EEXIT. + * Except for non-volatile general purpose registers, preserving/setting state + * in accordance with the x86-64 ABI is the responsibility of the enclave and + * its runtime, i.e. __vdso_sgx_enter_enclave() cannot be called from C code + * without careful consideration by both the enclave and its runtime.
Instead "e.g. doesn't explcitly clear EFLAGS.DF after EEXIT" (which is somewhat confusing statement) paragraph should be replaced with a simple enumerated list of differences.
Something might be left out but that's cool. Just do your best and it can refined over time to be more exact.
Quoted 36 lines
arch/x86/entry/vdso/vsgx_enter_enclave.S +7
-1* * All general purpose registers except RAX, RBX and RCX are passed as-is to * the enclave. RAX, RBX and RCX are consumed by EENTER and ERESUME and are@@ -71,7 +71,9 @@ */ #ifdef SGX_KERNEL_DOC /* C-style function prototype to coerce kernel-doc into parsing the comment. */ -int __vdso_sgx_enter_enclave(int leaf, void *tcs, +int __vdso_sgx_enter_enclave(unsigned long rdi, unsigned long rsi, + unsigned long rdx, unsigned int leaf, + unsigned long r8, unsigned long r9, void *tcs, struct sgx_enclave_exception *e, sgx_enclave_exit_handler_t handler); #endif@@ -83,7 +85,10 @@ SYM_FUNC_START(__vdso_sgx_enter_enclave) .cfi_rel_offset %rbp, 0 mov %rsp, %rbp .cfi_def_cfa_register %rbp + push %rbx + .cfi_rel_offset %rbx, -8 + mov %ecx, %eax .Lenter_enclave: /* EENTER <= leaf <= ERESUME */ cmp $EENTER, %eax@@ -109,6 +114,7 @@ SYM_FUNC_START(__vdso_sgx_enter_enclave) jne .Linvoke_userspace_handler .Lout: + pop %rbx leave .cfi_def_cfa %rsp, 8 retSignature · 1 line
2.24.1
Re: [PATCH for_v29 v2 1/5] x86/sgx: vdso: Make __vdso_sgx_enter_enclave() callable from C code
Quoted 63 lines · Jarkko Sakkinen
Quoted 42 lines · Sean Christopherson
Make
__vdso_sgx_enter_enclave()callable from C by preserving %rbx and taking @leaf in %rcx instead of %rax. Being able to invoke the vDSO from C reduces the overhead of runtimes that are tightly coupled with their enclaves, e.g. that can rely on the enclave to save and restore non-volatile registers, as the runtime doesn't need an assembly wrapper to preserve non-volatile registers and/or shuffle stack arguments.Note, both %rcx and %rbx are consumed by EENTER/ERESUME, i.e. consuming them doesn't violate the primary tenet of
__vdso_sgx_enter_enclave()that "thou shalt not restrict how information is exchanged between an enclave and its host process".Suggested-by Nathaniel McCallum <[email protected]>Cc Cedric Xing <[email protected]>Cc Jethro Beekman <[email protected]>Cc Andy Lutomirski <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/entry/vdso/vsgx_enter_enclave.S | 30 ++++++++++++++---------- 1 file changed, 18 insertions(+), 12 deletions(-)arch/x86/entry/vdso/vsgx_enter_enclave.S +5
-2@@ -17,22 +17,22 @@ /** * __vdso_sgx_enter_enclave() - Enter an SGX enclave + * @rdi: Pass-through value for RDI + * @rsi: Pass-through value for RSI + * @rdx: Pass-through value for RDX * @leaf: ENCLU leaf, must be EENTER or ERESUME + * @r8: Pass-through value for R8 + * @r9: Pass-through value for R9 * @tcs: TCS, must be non-NULL * @e: Optional struct sgx_enclave_exception instance * @handler: Optional enclave exit handler * - * **Important!** __vdso_sgx_enter_enclave() is **NOT** compliant with the - * x86-64 ABI, i.e. cannot be called from standard C code.arch/x86/entry/vdso/vsgx_enter_enclave.S line 30
Quoted 10 lines
arch/x86/entry/vdso/vsgx_enter_enclave.S +1
-9- * - * Input ABI: - * @leaf %eax - * @tcs 8(%rsp) - * @e 0x10(%rsp) - * @handler 0x18(%rsp) - * - * Output ABI: - * @ret %eax + * **Important!** __vdso_sgx_enter_enclave() does not ensure full complianceI'd simply put **NOTE** here instead of **Important!** as it is more common.
arch/x86/entry/vdso/vsgx_enter_enclave.S line 35
Quoted 5 lines
arch/x86/entry/vdso/vsgx_enter_enclave.S +5
-0+ * with the x86-64 ABI, e.g. doesn't explicitly clear EFLAGS.DF after EEXIT. + * Except for non-volatile general purpose registers, preserving/setting state + * in accordance with the x86-64 ABI is the responsibility of the enclave and + * its runtime, i.e. __vdso_sgx_enter_enclave() cannot be called from C code + * without careful consideration by both the enclave and its runtime.
Quoted 3 lines
Instead "e.g. doesn't explcitly clear EFLAGS.DF after EEXIT" (which is somewhat confusing statement) paragraph should be replaced with a simple enumerated list of differences.
I don't think the list is that simple, e.g. there is a lot of state that is defined by the ABI that isn't touched, IMO talking about that state will add confusion.
I also don't understand what's confusing about stating EFLAGS.DF isn't cleared.
Quoted 39 lines
Something might be left out but that's cool. Just do your best and it can refined over time to be more exact.
Quoted 36 lines
arch/x86/entry/vdso/vsgx_enter_enclave.S +7
-1* * All general purpose registers except RAX, RBX and RCX are passed as-is to * the enclave. RAX, RBX and RCX are consumed by EENTER and ERESUME and are@@ -71,7 +71,9 @@ */ #ifdef SGX_KERNEL_DOC /* C-style function prototype to coerce kernel-doc into parsing the comment. */ -int __vdso_sgx_enter_enclave(int leaf, void *tcs, +int __vdso_sgx_enter_enclave(unsigned long rdi, unsigned long rsi, + unsigned long rdx, unsigned int leaf, + unsigned long r8, unsigned long r9, void *tcs, struct sgx_enclave_exception *e, sgx_enclave_exit_handler_t handler); #endif@@ -83,7 +85,10 @@ SYM_FUNC_START(__vdso_sgx_enter_enclave) .cfi_rel_offset %rbp, 0 mov %rsp, %rbp .cfi_def_cfa_register %rbp + push %rbx + .cfi_rel_offset %rbx, -8 + mov %ecx, %eax .Lenter_enclave: /* EENTER <= leaf <= ERESUME */ cmp $EENTER, %eax@@ -109,6 +114,7 @@ SYM_FUNC_START(__vdso_sgx_enter_enclave) jne .Linvoke_userspace_handler .Lout: + pop %rbx leave .cfi_def_cfa %rsp, 8 retSignature · 1 line
2.24.1
Re: [PATCH for_v29 v2 1/5] x86/sgx: vdso: Make __vdso_sgx_enter_enclave() callable from C code
Quoted 68 lines · Sean Christopherson
Quoted 63 lines · Jarkko Sakkinen
Quoted 42 lines · Sean Christopherson
Make
__vdso_sgx_enter_enclave()callable from C by preserving %rbx and taking @leaf in %rcx instead of %rax. Being able to invoke the vDSO from C reduces the overhead of runtimes that are tightly coupled with their enclaves, e.g. that can rely on the enclave to save and restore non-volatile registers, as the runtime doesn't need an assembly wrapper to preserve non-volatile registers and/or shuffle stack arguments.Note, both %rcx and %rbx are consumed by EENTER/ERESUME, i.e. consuming them doesn't violate the primary tenet of
__vdso_sgx_enter_enclave()that "thou shalt not restrict how information is exchanged between an enclave and its host process".Suggested-by Nathaniel McCallum <[email protected]>Cc Cedric Xing <[email protected]>Cc Jethro Beekman <[email protected]>Cc Andy Lutomirski <[email protected]>Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/entry/vdso/vsgx_enter_enclave.S | 30 ++++++++++++++---------- 1 file changed, 18 insertions(+), 12 deletions(-)arch/x86/entry/vdso/vsgx_enter_enclave.S +5
-2@@ -17,22 +17,22 @@ /** * __vdso_sgx_enter_enclave() - Enter an SGX enclave + * @rdi: Pass-through value for RDI + * @rsi: Pass-through value for RSI + * @rdx: Pass-through value for RDX * @leaf: ENCLU leaf, must be EENTER or ERESUME + * @r8: Pass-through value for R8 + * @r9: Pass-through value for R9 * @tcs: TCS, must be non-NULL * @e: Optional struct sgx_enclave_exception instance * @handler: Optional enclave exit handler * - * **Important!** __vdso_sgx_enter_enclave() is **NOT** compliant with the - * x86-64 ABI, i.e. cannot be called from standard C code.arch/x86/entry/vdso/vsgx_enter_enclave.S line 30
Quoted 10 lines
arch/x86/entry/vdso/vsgx_enter_enclave.S +1
-9- * - * Input ABI: - * @leaf %eax - * @tcs 8(%rsp) - * @e 0x10(%rsp) - * @handler 0x18(%rsp) - * - * Output ABI: - * @ret %eax + * **Important!** __vdso_sgx_enter_enclave() does not ensure full complianceI'd simply put **NOTE** here instead of **Important!** as it is more common.
arch/x86/entry/vdso/vsgx_enter_enclave.S line 35
Quoted 5 lines
arch/x86/entry/vdso/vsgx_enter_enclave.S +5
-0+ * with the x86-64 ABI, e.g. doesn't explicitly clear EFLAGS.DF after EEXIT. + * Except for non-volatile general purpose registers, preserving/setting state + * in accordance with the x86-64 ABI is the responsibility of the enclave and + * its runtime, i.e. __vdso_sgx_enter_enclave() cannot be called from C code + * without careful consideration by both the enclave and its runtime.Quoted 3 lines
Instead "e.g. doesn't explcitly clear EFLAGS.DF after EEXIT" (which is somewhat confusing statement) paragraph should be replaced with a simple enumerated list of differences.
Quoted 6 lines
I don't think the list is that simple, e.g. there is a lot of state that is defined by the ABI that isn't touched, IMO talking about that state will add confusion.
I also don't understand what's confusing about stating EFLAGS.DF isn't cleared.
Too much time was has passed since the last version, and too many patches have been reviewed since.
Please just refresh the way you feel best and I will make conclusions based on that. It's faster that way in the end.
/Jarkko
PATCH v2 2/5 x86/sgx: vdso: Define a typedef for __vdso_sgx_enter_enclave
In reply to Sean ChristophersonDefine a typedef for the __vdso_sgx_enter_enclave() prototype and move the entire function comment from the assembly code to the uAPI header, dropping the kernel doc hack along the way.
--- arch/x86/entry/vdso/vsgx_enter_enclave.S | 62 ------------------------ arch/x86/include/uapi/asm/sgx.h | 61 +++++++++++++++++++++++ 2 files changed, 61 insertions(+), 62 deletions(-)
arch/x86/entry/vdso/vsgx_enter_enclave.S +0 -62
@@ -15,68 +15,6 @@ .code64 .section .text, "ax" -/** - * __vdso_sgx_enter_enclave() - Enter an SGX enclave - * @rdi: Pass-through value for RDI - * @rsi: Pass-through value for RSI - * @rdx: Pass-through value for RDX - * @leaf: ENCLU leaf, must be EENTER or ERESUME - * @r8: Pass-through value for R8 - * @r9: Pass-through value for R9 - * @tcs: TCS, must be non-NULL - * @e: Optional struct sgx_enclave_exception instance - * @handler: Optional enclave exit handler - * - * **Important!** __vdso_sgx_enter_enclave() does not ensure full compliance - * with the x86-64 ABI, e.g. doesn't explicitly clear EFLAGS.DF after EEXIT. - * Except for non-volatile general purpose registers, preserving/setting state - * in accordance with the x86-64 ABI is the responsibility of the enclave and - * its runtime, i.e. __vdso_sgx_enter_enclave() cannot be called from C code - * without careful consideration by both the enclave and its runtime. - * - * All general purpose registers except RAX, RBX and RCX are passed as-is to - * the enclave. RAX, RBX and RCX are consumed by EENTER and ERESUME and are - * loaded with @leaf, asynchronous exit pointer, and @tcs respectively. - * - * RBP and the stack are used to anchor __vdso_sgx_enter_enclave() to the - * pre-enclave state, e.g. to retrieve @e and @handler after an enclave exit. - * All other registers are available for use by the enclave and its runtime, - * e.g. an enclave can push additional data onto the stack (and modify RSP) to - * pass information to the optional exit handler (see below). - * - * Most exceptions reported on ENCLU, including those that occur within the - * enclave, are fixed up and reported synchronously instead of being delivered - * via a standard signal. Debug Exceptions (#DB) and Breakpoints (#BP) are - * never fixed up and are always delivered via standard signals. On synchrously - * reported exceptions, -EFAULT is returned and details about the exception are - * recorded in @e, the optional sgx_enclave_exception struct. - - * If an exit handler is provided, the handler will be invoked on synchronous - * exits from the enclave and for all synchronously reported exceptions. In - * latter case, @e is filled prior to invoking the handler. - * - * The exit handler's return value is interpreted as follows: - * >0: continue, restart __vdso_sgx_enter_enclave() with @ret as @leaf - * 0: success, return @ret to the caller - * <0: error, return @ret to the caller - * - * The exit handler may transfer control, e.g. via longjmp() or C++ exception, - * without returning to __vdso_sgx_enter_enclave(). - * - * Return: - * 0 on success, - * -EINVAL if ENCLU leaf is not allowed, - * -EFAULT if an exception occurs on ENCLU or within the enclave - * -errno for all other negative values returned by the userspace exit handler - */ -#ifdef SGX_KERNEL_DOC -/* C-style function prototype to coerce kernel-doc into parsing the comment. */ -int __vdso_sgx_enter_enclave(unsigned long rdi, unsigned long rsi, - unsigned long rdx, unsigned int leaf, - unsigned long r8, unsigned long r9, void *tcs, - struct sgx_enclave_exception *e, - sgx_enclave_exit_handler_t handler); -#endif SYM_FUNC_START(__vdso_sgx_enter_enclave) /* Prolog */ .cfi_startproc
arch/x86/include/uapi/asm/sgx.h +61 -0
65 lines not on this page · raw message
Signature · 1 line
2.24.1
Re: [PATCH for_v29 v2 2/5] x86/sgx: vdso: Define a typedef for __vdso_sgx_enter_enclave
Quoted 156 lines · Sean Christopherson
Define a typedef for the
__vdso_sgx_enter_enclave()prototype and move the entire function comment from the assembly code to the uAPI header, dropping the kernel doc hack along the way.Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/entry/vdso/vsgx_enter_enclave.S | 62 ------------------------ arch/x86/include/uapi/asm/sgx.h | 61 +++++++++++++++++++++++ 2 files changed, 61 insertions(+), 62 deletions(-)arch/x86/entry/vdso/vsgx_enter_enclave.S +0
-62@@ -15,68 +15,6 @@ .code64 .section .text, "ax" -/** - * __vdso_sgx_enter_enclave() - Enter an SGX enclave - * @rdi: Pass-through value for RDI - * @rsi: Pass-through value for RSI - * @rdx: Pass-through value for RDX - * @leaf: ENCLU leaf, must be EENTER or ERESUME - * @r8: Pass-through value for R8 - * @r9: Pass-through value for R9 - * @tcs: TCS, must be non-NULL - * @e: Optional struct sgx_enclave_exception instance - * @handler: Optional enclave exit handler - * - * **Important!** __vdso_sgx_enter_enclave() does not ensure full compliance - * with the x86-64 ABI, e.g. doesn't explicitly clear EFLAGS.DF after EEXIT. - * Except for non-volatile general purpose registers, preserving/setting state - * in accordance with the x86-64 ABI is the responsibility of the enclave and - * its runtime, i.e. __vdso_sgx_enter_enclave() cannot be called from C code - * without careful consideration by both the enclave and its runtime. - * - * All general purpose registers except RAX, RBX and RCX are passed as-is to - * the enclave. RAX, RBX and RCX are consumed by EENTER and ERESUME and are - * loaded with @leaf, asynchronous exit pointer, and @tcs respectively. - * - * RBP and the stack are used to anchor __vdso_sgx_enter_enclave() to the - * pre-enclave state, e.g. to retrieve @e and @handler after an enclave exit. - * All other registers are available for use by the enclave and its runtime, - * e.g. an enclave can push additional data onto the stack (and modify RSP) to - * pass information to the optional exit handler (see below). - * - * Most exceptions reported on ENCLU, including those that occur within the - * enclave, are fixed up and reported synchronously instead of being delivered - * via a standard signal. Debug Exceptions (#DB) and Breakpoints (#BP) are - * never fixed up and are always delivered via standard signals. On synchrously - * reported exceptions, -EFAULT is returned and details about the exception are - * recorded in @e, the optional sgx_enclave_exception struct. - - * If an exit handler is provided, the handler will be invoked on synchronous - * exits from the enclave and for all synchronously reported exceptions. In - * latter case, @e is filled prior to invoking the handler. - * - * The exit handler's return value is interpreted as follows: - * >0: continue, restart __vdso_sgx_enter_enclave() with @ret as @leaf - * 0: success, return @ret to the caller - * <0: error, return @ret to the caller - * - * The exit handler may transfer control, e.g. via longjmp() or C++ exception, - * without returning to __vdso_sgx_enter_enclave(). - * - * Return: - * 0 on success, - * -EINVAL if ENCLU leaf is not allowed, - * -EFAULT if an exception occurs on ENCLU or within the enclave - * -errno for all other negative values returned by the userspace exit handler - */ -#ifdef SGX_KERNEL_DOC -/* C-style function prototype to coerce kernel-doc into parsing the comment. */ -int __vdso_sgx_enter_enclave(unsigned long rdi, unsigned long rsi, - unsigned long rdx, unsigned int leaf, - unsigned long r8, unsigned long r9, void *tcs, - struct sgx_enclave_exception *e, - sgx_enclave_exit_handler_t handler); -#endif SYM_FUNC_START(__vdso_sgx_enter_enclave) /* Prolog */ .cfi_startprocarch/x86/include/uapi/asm/sgx.h +61
-0@@ -111,4 +111,65 @@ typedef int (*sgx_enclave_exit_handler_t)(long rdi, long rsi, long rdx, void *tcs, int ret, struct sgx_enclave_exception *e); +/** +* __vdso_sgx_enter_enclave() - Enter an SGX enclave +* @rdi: Pass-through value for RDI +* @rsi: Pass-through value for RSI +* @rdx: Pass-through value for RDX +* @leaf: ENCLU leaf, must be EENTER or ERESUME +* @r8: Pass-through value for R8 +* @r9: Pass-through value for R9 +* @tcs: TCS, must be non-NULL +* @e: Optional struct sgx_enclave_exception instance +* @handler: Optional enclave exit handler +* +* **Important!** __vdso_sgx_enter_enclave() does not ensure full compliance +* with the x86-64 ABI, e.g. doesn't explicitly clear EFLAGS.DF after EEXIT. +* Except for non-volatile general purpose registers, preserving/setting state +* in accordance with the x86-64 ABI is the responsibility of the enclave and +* its runtime, i.e. __vdso_sgx_enter_enclave() cannot be called from C code +* without careful consideration by both the enclave and its runtime. +* +* All general purpose registers except RAX, RBX and RCX are passed as-is to +* the enclave. RAX, RBX and RCX are consumed by EENTER and ERESUME and are +* loaded with @leaf, asynchronous exit pointer, and @tcs respectively. +* +* RBP and the stack are used to anchor __vdso_sgx_enter_enclave() to the +* pre-enclave state, e.g. to retrieve @e and @handler after an enclave exit. +* All other registers are available for use by the enclave and its runtime, +* e.g. an enclave can push additional data onto the stack (and modify RSP) to +* pass information to the optional exit handler (see below). +* +* Most exceptions reported on ENCLU, including those that occur within the +* enclave, are fixed up and reported synchronously instead of being delivered +* via a standard signal. Debug Exceptions (#DB) and Breakpoints (#BP) are +* never fixed up and are always delivered via standard signals. On synchrously +* reported exceptions, -EFAULT is returned and details about the exception are +* recorded in @e, the optional sgx_enclave_exception struct. + +* If an exit handler is provided, the handler will be invoked on synchronous +* exits from the enclave and for all synchronously reported exceptions. In +* latter case, @e is filled prior to invoking the handler. +* +* The exit handler's return value is interpreted as follows: +* >0: continue, restart __vdso_sgx_enter_enclave() with @ret as @leaf +* 0: success, return @ret to the caller +* <0: error, return @ret to the caller +* +* The exit handler may transfer control, e.g. via longjmp() or C++ exception, +* without returning to __vdso_sgx_enter_enclave(). +* +* Return: +* 0 on success, +* -EINVAL if ENCLU leaf is not allowed, +* -EFAULT if an exception occurs on ENCLU or within the enclave +* -errno for all other negative values returned by the userspace exit handler +*/ +typedef int (*vdso_sgx_enter_enclave_t )(unsigned long rdi,unsigned long rsi, +unsigned long rdx,unsigned int leaf, +unsigned long r8,unsigned long r9, +void *tcs, +struct sgx_enclave_exception *e, +sgx_enclave_exit_handler_t handler); +#endif /* _UAPI_ASM_X86_SGX_H */ Signature · 1 line
2.24.1
Most probaby agree with this.
/Jarkko
PATCH v2 3/5 selftests/sgx: Pass EENTER to vDSO wrapper instead of hardcoding
In reply to Sean ChristophersonPass EENTER to the vDSO wrapper via %ecx instead of hardcoding it to make the wrapper compatible with the new vDSO calling convention.
--- tools/testing/selftests/sgx/call.S | 1 - tools/testing/selftests/sgx/defines.h | 1 + tools/testing/selftests/sgx/main.c | 2 +- tools/testing/selftests/sgx/main.h | 2 +- 4 files changed, 3 insertions(+), 3 deletions(-)
tools/testing/selftests/sgx/call.S +0 -1
@@ -37,7 +37,6 @@ sgx_call_vdso: .cfi_adjust_cfa_offset 8 push 0x48(%rsp) .cfi_adjust_cfa_offset 8 - mov $2, %eax call *eenter(%rip) add $0x20, %rsp .cfi_adjust_cfa_offset -0x20
tools/testing/selftests/sgx/defines.h +1 -0
@@ -15,6 +15,7 @@#define __packed __attribute__((packed))#include "../../../../arch/x86/kernel/cpu/sgx/arch.h" +#include "../../../../arch/x86/include/asm/enclu.h" #include "../../../../arch/x86/include/uapi/asm/sgx.h" #endif /* DEFINES_H */
tools/testing/selftests/sgx/main.c +1 -1
@@ -171,7 +171,7 @@ int main(int argc, char *argv[], char *envp[]) eenter = addr + eenter_sym->st_value; -sgx_call_vdso ((void *)&MAGIC , &result,0 ,NULL ,NULL ,NULL , +sgx_call_vdso ((void *)&MAGIC , &result,0 ,EENTER ,NULL ,NULL , (void *)encl.encl_base, &exception,NULL );if (result !=MAGIC ) goto err;
tools/testing/selftests/sgx/main.h +1 -1
@@ -32,7 +32,7 @@ bool encl_load(const char *path, struct encl *encl);bool encl_measure (struct encl *encl);bool encl_build (struct encl *encl); -int sgx_call_vdso (void *rdi,void *rsi,long rdx,void *rcx,void *r8,void *r9, +int sgx_call_vdso (void *rdi,void *rsi,long rdx,u32 leaf,void *r8,void *r9,void *tcs,struct sgx_enclave_exception *ei,void *cb);#endif /* MAIN_H */
Signature · 1 line
2.24.1
Re: [PATCH for_v29 v2 3/5] selftests/sgx: Pass EENTER to vDSO wrapper instead of hardcoding
Quoted 4 lines · Sean Christopherson
Pass EENTER to the vDSO wrapper via %ecx instead of hardcoding it to make the wrapper compatible with the new vDSO calling convention.
Signed-off-by Sean Christopherson <[email protected]>
I think it'd be good if you merge 1/5-3/5 and 4/5-55 to have two patches.
Easier to review given the strong bounds, easier for you to update and easier for me to merge later on.
/Jarko
Re: [PATCH for_v29 v2 3/5] selftests/sgx: Pass EENTER to vDSO wrapper instead of hardcoding
Quoted 10 lines · Jarkko Sakkinen
Quoted 4 lines · Sean Christopherson
Pass EENTER to the vDSO wrapper via %ecx instead of hardcoding it to make the wrapper compatible with the new vDSO calling convention.
Signed-off-by Sean Christopherson <[email protected]>I think it'd be good if you merge 1/5-3/5 and 4/5-55 to have two patches.
Easier to review given the strong bounds, easier for you to update and easier for me to merge later on.
I mean {1, 2, 3} and {4, 5}
/Jarkko
PATCH v2 4/5 selftests/sgx: Stop clobbering non-volatile registers
In reply to Sean ChristophersonStop clearing non-volatile registers in the enclave's trampoline code, there are no secrets to preserve, and not clobbering the registers makes the enclave compatible with calling the vDSO from C.
--- tools/testing/selftests/sgx/test_encl_bootstrap.S | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-)
tools/testing/selftests/sgx/test_encl_bootstrap.S +1 -5
@@ -54,7 +54,7 @@ encl_entry: pop %rbx # pop the enclave base address - # Clear GPRs. + /* Clear volatile GPRs, except RAX (EEXIT leaf). */ xor %rcx, %rcx xor %rdx, %rdx xor %rdi, %rdi
@@ -63,10 +63,6 @@ encl_entry: xor %r9, %r9 xor %r10, %r10 xor %r11, %r11 - xor %r12, %r12 - xor %r13, %r13 - xor %r14, %r14 - xor %r15, %r15 # Reset status flags. add %rdx, %rdx # OF = SF = AF = CF = 0; ZF = PF = 1
Signature · 1 line
2.24.1
PATCH v2 5/5 selftests/sgx: Add selftest to invoke __vsgx_enter_enclave() from C
In reply to Sean ChristophersonAdd a selftest to call __vsgx_enter_enclave() from C.
--- tools/testing/selftests/sgx/main.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)
tools/testing/selftests/sgx/main.c +15 -3
@@ -19,7 +19,7 @@#include "main.h" static const uint64_t MAGIC =0x1122334455667788ULL ; -void *eenter; +vdso_sgx_enter_enclave_t eenter;struct vdso_symtab {Elf64_Sym *elf_symtab;
@@ -173,15 +173,27 @@ int main(int argc, char *argv[], char *envp[])sgx_call_vdso ((void *)&MAGIC , &result,0 ,EENTER ,NULL ,NULL , (void *)encl.encl_base, &exception,NULL ); -if (result !=MAGIC ) +if (result !=MAGIC ) { +printf ("FAIL: sgx_call_vdso(), expected: 0x%lx, got: 0x%lx\n" , +MAGIC , result); goto err; + } + +/* Invoke the vDSO directly. */ + result =0 ; +eenter ((unsigned long )&MAGIC , (unsigned long )&result,0 ,EENTER ,0 ,0 , + (void *)encl.encl_base, &exception,NULL ); +if (result !=MAGIC ) { +printf ("FAIL: eenter(), expected: 0x%lx, got: 0x%lx\n" , +MAGIC , result); + goto err; + }printf ("SUCCESS\n" );encl_delete (&encl);exit (0 ); err: -printf ("FAILURE\n" );encl_delete (&encl);exit (1 ); }
Signature · 1 line
2.24.1
Quoted 8 lines · Sean Christopherson
Nathaniel pointed out that
__vdso_sgx_enter_enclave()is tantalizingly close to being callable from C (with caveats and a cooperative enclave). The missing pieces are preserving %rbx and taking @leaf as a standard parameter.v2:
- Rebase to Jarkko's latest master, commit402fb35a477a, "docs: ...")
- Add CFI directive for RBX. [Cedric]
I'm sorry for throwing stick's constantly but I think having a real ELF loader is for better.
/Jarkko
Quoted 7 lines · Jarkko Sakkinen
Quoted 5 lines · Sean Christopherson
Nathaniel pointed out that
__vdso_sgx_enter_enclave()is tantalizingly close to being callable from C (with caveats and a cooperative enclave). The missing pieces are preserving %rbx and taking @leaf as a standard parameter.
Quoted 6 lines
Quoted 3 lines
v2:
- Rebase to Jarkko's latest master, commit402fb35a477a, "docs: ...")
- Add CFI directive for RBX. [Cedric]I'm sorry for throwing stick's constantly but I think having a real ELF loader is for better.
Why is it one or the other?
Quoted 9 lines · Nathaniel McCallum
Quoted 7 lines · Jarkko Sakkinen
Quoted 5 lines · Sean Christopherson
Nathaniel pointed out that
__vdso_sgx_enter_enclave()is tantalizingly close to being callable from C (with caveats and a cooperative enclave). The missing pieces are preserving %rbx and taking @leaf as a standard parameter.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
v2:
- Rebase to Jarkko's latest master, commit402fb35a477a, "docs: ...")
- Add CFI directive for RBX. [Cedric]I'm sorry for throwing stick's constantly but I think having a real ELF loader is for better.
Why is it one or the other?
Hmm... Not sure I understand what you want to know.
A raw binary requires a hardcoded way to interpret it. Obviously ELF is for better.
/Jarkko
Quoted 11 lines · Jarkko Sakkinen
Quoted 9 lines · Nathaniel McCallum
Quoted 7 lines · Jarkko Sakkinen
Quoted 5 lines · Sean Christopherson
Nathaniel pointed out that
__vdso_sgx_enter_enclave()is tantalizingly close to being callable from C (with caveats and a cooperative enclave). The missing pieces are preserving %rbx and taking @leaf as a standard parameter.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
v2:
- Rebase to Jarkko's latest master, commit402fb35a477a, "docs: ...")
- Add CFI directive for RBX. [Cedric]I'm sorry for throwing stick's constantly but I think having a real ELF loader is for better.
This statement seems like you are juxtaposing having __vdso_sgx_enter_enclave() be potentially C-compatible with having an ELF-loader. These are not incompabile. __vdso_sgx_enter_enclave() can be C-callable *and* you can have an ELF loader.
Quoted 6 lines
Quoted 1 line
Why is it one or the other?
Hmm... Not sure I understand what you want to know.
A raw binary requires a hardcoded way to interpret it. Obviously ELF is for better.
We (Enarx) implement an ELF loader.
Quoted 17 lines · Nathaniel McCallum
Quoted 15 lines · Jarkko Sakkinen
Quoted 13 lines · Nathaniel McCallum
Quoted 11 lines · Jarkko Sakkinen
Quoted 8 lines · Sean Christopherson
Nathaniel pointed out that
__vdso_sgx_enter_enclave()is tantalizingly close to being callable from C (with caveats and a cooperative enclave). The missing pieces are preserving %rbx and taking @leaf as a standard parameter.v2:
- Rebase to Jarkko's latest master, commit402fb35a477a, "docs: ...")
- Add CFI directive for RBX. [Cedric]
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
I'm sorry for throwing stick's constantly but I think having a real ELF loader is for better.
This statement seems like you are juxtaposing having
__vdso_sgx_enter_enclave()be potentially C-compatible with having an ELF-loader. These are not incompabile.__vdso_sgx_enter_enclave()can be C-callable *and* you can have an ELF loader.
I'm not honestly sure what this is about but my comment was about heavy rebasing of the GIT tree as I rewrote the selftest last week.
/Jarkko
Quoted 22 lines · Jarkko Sakkinen
Quoted 20 lines · Nathaniel McCallum
Quoted 17 lines · Jarkko Sakkinen
Quoted 15 lines · Nathaniel McCallum
Quoted 13 lines · Jarkko Sakkinen
Quoted 8 lines · Sean Christopherson
Nathaniel pointed out that
__vdso_sgx_enter_enclave()is tantalizingly close to being callable from C (with caveats and a cooperative enclave). The missing pieces are preserving %rbx and taking @leaf as a standard parameter.v2:
- Rebase to Jarkko's latest master, commit402fb35a477a, "docs: ...")
- Add CFI directive for RBX. [Cedric]I'm sorry for throwing stick's constantly but I think having a real ELF loader is for better.
Quoted 7 lines
Quoted 4 lines
This statement seems like you are juxtaposing having
__vdso_sgx_enter_enclave()be potentially C-compatible with having an ELF-loader. These are not incompabile.__vdso_sgx_enter_enclave()can be C-callable *and* you can have an ELF loader.I'm not honestly sure what this is about but my comment was about heavy rebasing of the GIT tree as I rewrote the selftest last week.
Okay. Let's chalk it up to miscommunication then. :)
Quoted 29 lines · Nathaniel McCallum
Quoted 27 lines · Jarkko Sakkinen
Quoted 24 lines · Nathaniel McCallum
Quoted 11 lines · Jarkko Sakkinen
Quoted 9 lines · Nathaniel McCallum
Quoted 7 lines · Jarkko Sakkinen
Quoted 5 lines · Sean Christopherson
Nathaniel pointed out that
__vdso_sgx_enter_enclave()is tantalizingly close to being callable from C (with caveats and a cooperative enclave). The missing pieces are preserving %rbx and taking @leaf as a standard parameter.Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
v2:
- Rebase to Jarkko's latest master, commit402fb35a477a, "docs: ...")
- Add CFI directive for RBX. [Cedric]I'm sorry for throwing stick's constantly but I think having a real ELF loader is for better.
This statement seems like you are juxtaposing having
__vdso_sgx_enter_enclave()be potentially C-compatible with having an ELF-loader. These are not incompabile.__vdso_sgx_enter_enclave()can be C-callable *and* you can have an ELF loader.
Quoted 4 lines
Quoted 2 lines
I'm not honestly sure what this is about but my comment was about heavy rebasing of the GIT tree as I rewrote the selftest last week.
Okay. Let's chalk it up to miscommunication then. :)
Ha, I was in the same boat as Nathaniel. We thought the "having a real ELF loader comment" was a comment on the patch itself, i.e. that you disagreed with it in some way because it didn't support an ELF loader, hence our confusion.
Now I realize you were refering to the rebase needed due to rewriting the selftest to use an ELF loader. Crisis aborted :-)
Quoted 35 lines · Sean Christopherson
Quoted 29 lines · Nathaniel McCallum
Quoted 27 lines · Jarkko Sakkinen
Quoted 24 lines · Nathaniel McCallum
Quoted 11 lines · Jarkko Sakkinen
Quoted 9 lines · Nathaniel McCallum
Quoted 7 lines · Jarkko Sakkinen
Quoted 5 lines · Sean Christopherson
Nathaniel pointed out that
__vdso_sgx_enter_enclave()is tantalizingly close to being callable from C (with caveats and a cooperative enclave). The missing pieces are preserving %rbx and taking @leaf as a standard parameter.Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
v2:
- Rebase to Jarkko's latest master, commit402fb35a477a, "docs: ...")
- Add CFI directive for RBX. [Cedric]I'm sorry for throwing stick's constantly but I think having a real ELF loader is for better.
This statement seems like you are juxtaposing having
__vdso_sgx_enter_enclave()be potentially C-compatible with having an ELF-loader. These are not incompabile.__vdso_sgx_enter_enclave()can be C-callable *and* you can have an ELF loader.Quoted 4 lines
Quoted 2 lines
I'm not honestly sure what this is about but my comment was about heavy rebasing of the GIT tree as I rewrote the selftest last week.
Okay. Let's chalk it up to miscommunication then. :)
Quoted 7 lines
Ha, I was in the same boat as Nathaniel. We thought the "having a real ELF loader comment" was a comment on the patch itself, i.e. that you disagreed with it in some way because it didn't support an ELF loader, hence our confusion.
Now I realize you were refering to the rebase needed due to rewriting the selftest to use an ELF loader. Crisis aborted :-)
Awesome :-)
/Jarkko
Quoted 29 lines · Nathaniel McCallum
Quoted 27 lines · Jarkko Sakkinen
Quoted 24 lines · Nathaniel McCallum
Quoted 11 lines · Jarkko Sakkinen
Quoted 9 lines · Nathaniel McCallum
Quoted 7 lines · Jarkko Sakkinen
Quoted 5 lines · Sean Christopherson
Nathaniel pointed out that
__vdso_sgx_enter_enclave()is tantalizingly close to being callable from C (with caveats and a cooperative enclave). The missing pieces are preserving %rbx and taking @leaf as a standard parameter.Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
v2:
- Rebase to Jarkko's latest master, commit402fb35a477a, "docs: ...")
- Add CFI directive for RBX. [Cedric]I'm sorry for throwing stick's constantly but I think having a real ELF loader is for better.
This statement seems like you are juxtaposing having
__vdso_sgx_enter_enclave()be potentially C-compatible with having an ELF-loader. These are not incompabile.__vdso_sgx_enter_enclave()can be C-callable *and* you can have an ELF loader.
Quoted 4 lines
Quoted 2 lines
I'm not honestly sure what this is about but my comment was about heavy rebasing of the GIT tree as I rewrote the selftest last week.
Okay. Let's chalk it up to miscommunication then. :)
Yeah, lets bring a background :-)
Last week basically rewrote the self-test. It now directly loads ELF and dynamically signs that. Thus, the codebase has been kind of moving target for a while. I'm absolutely for these changes as soon as the form fits to the existing codebase.
/Jarkko