x86/sgx: Allow enclaves to use Asynchrounous Exit Notification
Dave Hansen
maintainerPATCH 1/1 x86/sgx: Allow enclaves to use Asynchrounous Exit Notification
Short Version:
Allow enclaves to use the new Asynchronous EXit (AEX) notification mechanism. This mechanism lets enclaves run a handler after an AEX event. These handlers can run mitigations for things like SGX-Step[1].
AEX Notify will be made available both on upcoming processors and on some older processors through microcode updates.
Long Version:
== SGX Attribute Background ==
The SGX architecture includes a list of SGX "attributes". These attributes ensure consistency and transparency around specific enclave features.
As a simple example, the "DEBUG" attribute allows an enclave to be debugged, but also destroys virtually all of SGX security. Using attributes, enclaves can know that they are being debugged. Attributes also affect enclave attestation so an enclave can, for instance, be denied access to secrets while it is being debugged.
The kernel keeps a list of known attributes and will only initialize enclaves that use a known set of attributes. This kernel policy eliminates the chance that a new SGX attribute could cause undesired effects.
For example, imagine a new attribute was added called "PROVISIONKEY2" that provided similar functionality to "PROVISIIONKEY". A kernel policy that allowed indiscriminate use of unknown attributes and thus PROVISIONKEY2 would undermine the existing kernel policy which limits use of PROVISIONKEY enclaves.
== AEX Notify Background ==
"Intel Architecture Instruction Set Extensions and Future Features - Version 45" is out[2]. There is a new chapter:
Asynchronous Enclave Exit Notify and the EDECCSSA User Leaf Function.
Enclaves exit can be either synchronous and consensual (EEXIT for instance) or asynchronous (on an interrupt or fault). The asynchronous ones can evidently be exploited to single step enclaves[1], on top of which other naughty things can be built.
AEX Notify will be made available both on upcoming processors and on some older processors through microcode updates.
== The Problem ==
These attacks are currently entirely opaque to the enclave since the hardware does the save/restore under the covers. The Asynchronous Enclave Exit Notify (AEX Notify) mechanism provides enclaves an ability to detect and mitigate potential exposure to these kinds of attacks.
== The Solution ==
Define the new attribute value for AEX Notification. Ensure the attribute is cleared from the list reserved attributes which allows it to be used in enclaves.
I just built this and ran it to make sure there were no obvious regressions since I do not have the hardware to test it handy. Tested-by's would be much appreciated.
1. https://github.com/jovanbulck/sgx-step
2. https://cdrdv2.intel.com/v1/dl/getContent/671368?explicitVersion=true
--- arch/x86/include/asm/sgx.h | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-)
arch/x86/include/asm/sgx.h +18 -7
@@ -110,17 +110,28 @@ enum sgx_miscselect {* %SGX_ATTR_EINITTOKENKEY: Allow to use token signing key that is used to * sign cryptographic tokens that can be passed to * EINIT as an authorization to run an enclave. +* %SGX_ATTR_ASYNC_EXIT_NOTIFY: Allow enclaves to be notified after an +* asynchronous exit has occurred. */ enum sgx_attribute { -SGX_ATTR_INIT =BIT (0 ), -SGX_ATTR_DEBUG =BIT (1 ), -SGX_ATTR_MODE64BIT =BIT (2 ), -SGX_ATTR_PROVISIONKEY =BIT (4 ), -SGX_ATTR_EINITTOKENKEY =BIT (5 ), -SGX_ATTR_KSS =BIT (7 ), +SGX_ATTR_INIT =BIT (0 ), +SGX_ATTR_DEBUG =BIT (1 ), +SGX_ATTR_MODE64BIT =BIT (2 ), +/* BIT(3) is reserved */ +SGX_ATTR_PROVISIONKEY =BIT (4 ), +SGX_ATTR_EINITTOKENKEY =BIT (5 ), +/* BIT(6) is for CET */ +SGX_ATTR_KSS =BIT (7 ), +/* BIT(8) is reserved */ +/* BIT(9) is reserved */ +SGX_ATTR_ASYNC_EXIT_NOTIFY =BIT (10 ), }; -#define SGX_ATTR_RESERVED_MASK (BIT_ULL(3) | BIT_ULL(6) | GENMASK_ULL(63, 8)) +#define SGX_ATTR_RESERVED_MASK (BIT_ULL(3) | \ + BIT_ULL(6) | \ + BIT_ULL(8) | \ + BIT_ULL(9) | \ + GENMASK_ULL(63, 11))/** * struct sgx_secs - SGX Enclave Control Structure (SECS)
Signature · 1 line
2.34.1
Jarkko Sakkinen
maintainerQuoted 60 lines · Dave Hansen
Short Version:
Allow enclaves to use the new Asynchronous EXit (AEX) notification mechanism. This mechanism lets enclaves run a handler after an AEX event. These handlers can run mitigations for things like SGX-Step[1].
AEX Notify will be made available both on upcoming processors and on some older processors through microcode updates.
Long Version:
== SGX Attribute Background ==
The SGX architecture includes a list of SGX "attributes". These attributes ensure consistency and transparency around specific enclave features.
As a simple example, the "DEBUG" attribute allows an enclave to be debugged, but also destroys virtually all of SGX security. Using attributes, enclaves can know that they are being debugged. Attributes also affect enclave attestation so an enclave can, for instance, be denied access to secrets while it is being debugged.
The kernel keeps a list of known attributes and will only initialize enclaves that use a known set of attributes. This kernel policy eliminates the chance that a new SGX attribute could cause undesired effects.
For example, imagine a new attribute was added called "PROVISIONKEY2" that provided similar functionality to "PROVISIIONKEY". A kernel policy that allowed indiscriminate use of unknown attributes and thus PROVISIONKEY2 would undermine the existing kernel policy which limits use of PROVISIONKEY enclaves.
== AEX Notify Background ==
"Intel Architecture Instruction Set Extensions and Future Features - Version 45" is out[2]. There is a new chapter:
Asynchronous Enclave Exit Notify and the EDECCSSA User Leaf Function.
Enclaves exit can be either synchronous and consensual (EEXIT for instance) or asynchronous (on an interrupt or fault). The asynchronous ones can evidently be exploited to single step enclaves[1], on top of which other naughty things can be built.
AEX Notify will be made available both on upcoming processors and on some older processors through microcode updates.
== The Problem ==
These attacks are currently entirely opaque to the enclave since the hardware does the save/restore under the covers. The Asynchronous Enclave Exit Notify (AEX Notify) mechanism provides enclaves an ability to detect and mitigate potential exposure to these kinds of attacks.
== The Solution ==
Quoted 7 lines
Define the new attribute value for AEX Notification. Ensure the attribute is cleared from the list reserved attributes which allows it to be used in enclaves.
I just built this and ran it to make sure there were no obvious regressions since I do not have the hardware to test it handy. Tested-by's would be much appreciated.
Is this available on recent ucode updates e.g. for Icelake or Geminilake?
Quoted 61 lines
1. https://github.com/jovanbulck/sgx-step
2. https://cdrdv2.intel.com/v1/dl/getContent/671368?explicitVersion=trueSigned-off-by Dave Hansen <[email protected]>Cc Jarkko Sakkinen <[email protected]>Cc Andy Lutomirski <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc Borislav Petkov <[email protected]>Cc "H. Peter Anvin" <[email protected]>--- arch/x86/include/asm/sgx.h | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-)arch/x86/include/asm/sgx.h +18
-7@@ -110,17 +110,28 @@ enum sgx_miscselect {* %SGX_ATTR_EINITTOKENKEY: Allow to use token signing key that is used to * sign cryptographic tokens that can be passed to * EINIT as an authorization to run an enclave. +* %SGX_ATTR_ASYNC_EXIT_NOTIFY: Allow enclaves to be notified after an +* asynchronous exit has occurred. */ enum sgx_attribute { -SGX_ATTR_INIT =BIT (0 ), -SGX_ATTR_DEBUG =BIT (1 ), -SGX_ATTR_MODE64BIT =BIT (2 ), -SGX_ATTR_PROVISIONKEY =BIT (4 ), -SGX_ATTR_EINITTOKENKEY =BIT (5 ), -SGX_ATTR_KSS =BIT (7 ), +SGX_ATTR_INIT =BIT (0 ), +SGX_ATTR_DEBUG =BIT (1 ), +SGX_ATTR_MODE64BIT =BIT (2 ), +/* BIT(3) is reserved */ +SGX_ATTR_PROVISIONKEY =BIT (4 ), +SGX_ATTR_EINITTOKENKEY =BIT (5 ), +/* BIT(6) is for CET */ +SGX_ATTR_KSS =BIT (7 ), +/* BIT(8) is reserved */ +/* BIT(9) is reserved */ +SGX_ATTR_ASYNC_EXIT_NOTIFY =BIT (10 ), }; -#define SGX_ATTR_RESERVED_MASK (BIT_ULL(3) | BIT_ULL(6) | GENMASK_ULL(63, 8)) +#define SGX_ATTR_RESERVED_MASK (BIT_ULL(3) | \ + BIT_ULL(6) | \ + BIT_ULL(8) | \ + BIT_ULL(9) | \ + GENMASK_ULL(63, 11))/** * struct sgx_secs - SGX Enclave Control Structure (SECS) Signature · 1 line
2.34.1
BR, Jarkko
Jarkko Sakkinen
maintainerQuoted 65 lines · Jarkko Sakkinen
Quoted 64 lines · Dave Hansen
Short Version:
Allow enclaves to use the new Asynchronous EXit (AEX) notification mechanism. This mechanism lets enclaves run a handler after an AEX event. These handlers can run mitigations for things like SGX-Step[1].
AEX Notify will be made available both on upcoming processors and on some older processors through microcode updates.
Long Version:
== SGX Attribute Background ==
The SGX architecture includes a list of SGX "attributes". These attributes ensure consistency and transparency around specific enclave features.
As a simple example, the "DEBUG" attribute allows an enclave to be debugged, but also destroys virtually all of SGX security. Using attributes, enclaves can know that they are being debugged. Attributes also affect enclave attestation so an enclave can, for instance, be denied access to secrets while it is being debugged.
The kernel keeps a list of known attributes and will only initialize enclaves that use a known set of attributes. This kernel policy eliminates the chance that a new SGX attribute could cause undesired effects.
For example, imagine a new attribute was added called "PROVISIONKEY2" that provided similar functionality to "PROVISIIONKEY". A kernel policy that allowed indiscriminate use of unknown attributes and thus PROVISIONKEY2 would undermine the existing kernel policy which limits use of PROVISIONKEY enclaves.
== AEX Notify Background ==
"Intel Architecture Instruction Set Extensions and Future Features - Version 45" is out[2]. There is a new chapter:
Asynchronous Enclave Exit Notify and the EDECCSSA User Leaf Function.
Enclaves exit can be either synchronous and consensual (EEXIT for instance) or asynchronous (on an interrupt or fault). The asynchronous ones can evidently be exploited to single step enclaves[1], on top of which other naughty things can be built.
AEX Notify will be made available both on upcoming processors and on some older processors through microcode updates.
== The Problem ==
These attacks are currently entirely opaque to the enclave since the hardware does the save/restore under the covers. The Asynchronous Enclave Exit Notify (AEX Notify) mechanism provides enclaves an ability to detect and mitigate potential exposure to these kinds of attacks.
== The Solution ==
Define the new attribute value for AEX Notification. Ensure the attribute is cleared from the list reserved attributes which allows it to be used in enclaves.
Quoted 6 lines
Quoted 3 lines
I just built this and ran it to make sure there were no obvious regressions since I do not have the hardware to test it handy. Tested-by's would be much appreciated.
Is this available on recent ucode updates e.g. for Icelake or Geminilake?
I mean it would not take me long to upgrade our exception handling flow to this. Then I can run our full test suite on it. But this will of course require ucode update for Icelake.
AEX Notify will actually just simplify everything. We kind of simulate "AEX Notify" already with EENTER to execute in-enclave exception handler before doing actual ERESUME.
BR, Jarkko
Quoted 2 lines · Jarkko Sakkinen
Is this available on recent ucode updates e.g. for Icelake or Geminilake?
I do not believe it is currently publicly available.
It *really* needs some @intel.com Tested-by's.
Hi Dave
I think you need add this change in arch/x86/kernel/cpu/sgx/ioctl.c, inside sgx_encl_create function:
- encl->attributes_mask = SGX_ATTR_DEBUG | SGX_ATTR_MODE64BIT | SGX_ATTR_KSS; + encl->attributes_mask = SGX_ATTR_DEBUG | SGX_ATTR_MODE64BIT | SGX_ATTR_KSS | SGX_ATTR_ASYNC_EXIT_NOTIFY;
Otherwise, EINIT ioctl fails with this new attribute set in enclave secs.
Thanks
Haitao
Quoted 125 lines · Dave Hansen
Short Version:
Allow enclaves to use the new Asynchronous EXit (AEX) notification mechanism. This mechanism lets enclaves run a handler after an AEX event. These handlers can run mitigations for things like SGX-Step[1].
AEX Notify will be made available both on upcoming processors and on some older processors through microcode updates.
Long Version:
== SGX Attribute Background ==
The SGX architecture includes a list of SGX "attributes". These attributes ensure consistency and transparency around specific enclave features.
As a simple example, the "DEBUG" attribute allows an enclave to be debugged, but also destroys virtually all of SGX security. Using attributes, enclaves can know that they are being debugged. Attributes also affect enclave attestation so an enclave can, for instance, be denied access to secrets while it is being debugged.
The kernel keeps a list of known attributes and will only initialize enclaves that use a known set of attributes. This kernel policy eliminates the chance that a new SGX attribute could cause undesired effects.
For example, imagine a new attribute was added called "PROVISIONKEY2" that provided similar functionality to "PROVISIIONKEY". A kernel policy that allowed indiscriminate use of unknown attributes and thus PROVISIONKEY2 would undermine the existing kernel policy which limits use of PROVISIONKEY enclaves.
== AEX Notify Background ==
"Intel Architecture Instruction Set Extensions and Future Features - Version 45" is out[2]. There is a new chapter:
Asynchronous Enclave Exit Notify and the EDECCSSA User Leaf Function.
Enclaves exit can be either synchronous and consensual (EEXIT for instance) or asynchronous (on an interrupt or fault). The asynchronous ones can evidently be exploited to single step enclaves[1], on top of which other naughty things can be built.
AEX Notify will be made available both on upcoming processors and on some older processors through microcode updates.
== The Problem ==
These attacks are currently entirely opaque to the enclave since the hardware does the save/restore under the covers. The Asynchronous Enclave Exit Notify (AEX Notify) mechanism provides enclaves an ability to detect and mitigate potential exposure to these kinds of attacks.
== The Solution ==
Define the new attribute value for AEX Notification. Ensure the attribute is cleared from the list reserved attributes which allows it to be used in enclaves.
I just built this and ran it to make sure there were no obvious regressions since I do not have the hardware to test it handy. Tested-by's would be much appreciated.
1. https://github.com/jovanbulck/sgx-step
2. https://cdrdv2.intel.com/v1/dl/getContent/671368?explicitVersion=trueSigned-off-by Dave Hansen <[email protected]>Cc Jarkko Sakkinen <[email protected]>Cc Andy Lutomirski <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Ingo Molnar <[email protected]>Cc Borislav Petkov <[email protected]>Cc "H. Peter Anvin" <[email protected]>--- arch/x86/include/asm/sgx.h | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-)arch/x86/include/asm/sgx.h +0
-0@@ -110,17 +110,28 @@ enum sgx_miscselect {* %SGX_ATTR_EINITTOKENKEY: Allow to use token signing key that is used to
* sign cryptographic tokens that can be passed to
* EINIT as an authorization to run an enclave.
+ * %SGX_ATTR_ASYNC_EXIT_NOTIFY: Allow enclaves to be notified after an
+ * asynchronous exit has occurred.
*/
enumsgx_attribute{arch/x86/include/asm/sgx.h +11
-7-SGX_ATTR_INIT =BIT (0 ), -SGX_ATTR_DEBUG =BIT (1 ), -SGX_ATTR_MODE64BIT =BIT (2 ), -SGX_ATTR_PROVISIONKEY =BIT (4 ), -SGX_ATTR_EINITTOKENKEY =BIT (5 ), - SGX_ATTR_KSS = BIT(7), +SGX_ATTR_INIT =BIT (0 ), +SGX_ATTR_DEBUG =BIT (1 ), +SGX_ATTR_MODE64BIT =BIT (2 ), +/* BIT(3) is reserved */ +SGX_ATTR_PROVISIONKEY =BIT (4 ), +SGX_ATTR_EINITTOKENKEY =BIT (5 ), +/* BIT(6) is for CET */ +SGX_ATTR_KSS =BIT (7 ), +/* BIT(8) is reserved */ +/* BIT(9) is reserved */ + SGX_ATTR_ASYNC_EXIT_NOTIFY = BIT(10), }; -#define SGX_ATTR_RESERVED_MASK (BIT_ULL(3) | BIT_ULL(6) |
GENMASK_ULL(63, 8))arch/x86/include/asm/sgx.h +5
-0+#define SGX_ATTR_RESERVED_MASK (BIT_ULL(3) | \ + BIT_ULL(6) | \ + BIT_ULL(8) | \ + BIT_ULL(9) | \ + GENMASK_ULL(63, 11))/**
* structsgx_secs- SGX Enclave Control Structure (SECS)
Quoted 10 lines · Haitao Huang
I think you need add this change in
arch/x86/kernel/cpu/sgx/ioctl.c, insidesgx_encl_createfunction:-
encl->attributes_mask=SGX_ATTR_DEBUG|SGX_ATTR_MODE64BIT|SGX_ATTR_KSS;
+encl->attributes_mask=SGX_ATTR_DEBUG|SGX_ATTR_MODE64BIT|SGX_ATTR_KSS|SGX_ATTR_ASYNC_EXIT_NOTIFY;Otherwise, EINIT ioctl fails with this new attribute set in enclave secs.
Makes sense. I'll add it there.
I also just realized we need to update arch/x86/kvm/cpuid.c as well. It keeps a complete list of guest-available SGX attributes.
Oh, and, btw... I don't have any hardware that can actually test this patch. It can't be applied until I have an actual tested-by from someone with the hardware.
Quoted 3 lines · Dave Hansen
Oh, and, btw... I don't have any hardware that can actually test this patch. It can't be applied until I have an actual tested-by from someone with the hardware.
I can ask someone in our team to test and report back results with Tested-by when you post updated patch. Thanks
Haitao
Jarkko Sakkinen
maintainerQuoted 3 lines · Dave Hansen
Oh, and, btw... I don't have any hardware that can actually test this patch. It can't be applied until I have an actual tested-by from someone with the hardware.
... I bet that NUC7's will never get a patch
BR, Jarkko