Multi-Key Total Memory Encryption API (MKTME)

Document the API's used for MKTME on Intel platforms.
MKTME: Multi-KEY Total Memory Encryption

Signed-off-by Alison Schofield <[email protected]>

---
 Documentation/x86/mktme-keys.txt | 153 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 153 insertions(+)
 create mode 100644 Documentation/x86/mktme-keys.txt

Documentation/x86/mktme-keys.txt new file +15 -0

new file mode 100644
index 000000000000..2dea7acd2a17
@@ -0,0 +1,153 @@
+MKTME (Multi-Key Total Memory Encryption) is a technology that allows
+memory encryption on Intel platforms. Whereas TME (Total Memory Encryption)
+allows encryption of the entire system memory using a single key, MKTME
+allows multiple encryption domains, each having their own key. The main use
+case for the feature is virtual machine isolation. The API's introduced here
+are intended to offer flexibility to work in a wide range of uses.
+
+The externally available Intel Architecture Spec:
+https://software.intel.com/sites/default/files/managed/a5/16/Multi-Key-Total-Memory-Encryption-Spec.pdf
+
+==============  API Overview  ==============
+
+There are 2 MKTME specific API's that enable userspace to create and use
+the memory encryption keys:
+

Documentation/x86/mktme-keys.txt new file +20 -0

+
+   o MKTME keys are a limited resource. There is a single pool of
+     MKTME keys for a system and that pool can be from 3 to 63 keys.
+     With that in mind, userspace may take advantage of the kernel
+     key services sharing and permissions model for userspace keys.
+     One key can be shared as long as each user has the permission
+     of "KEY_NEED_VIEW" to use it.
+
+   o MKTME key type uses capabilities to restrict the allocation
+     of keys. It only requires CAP_SYS_RESOURCE, but will accept
+     the broader capability of CAP_SYS_ADMIN.  See capabilities(7).
+
+   o The MKTME key service blocks kernel key service commands that
+     could lead to reprogramming of in use keys, or loss of keys from
+     the pool. This means MKTME does not allow a key to be invalidated,
+     unlinked, or timed out. These operations are blocked by MKTME as
+     it creates all keys with the internal flag KEY_FLAG_KEEP.
+
+   o MKTME does not support the keyctl option of UPDATE. Userspace
+     may change the programming of a key by revoking it and adding

Documentation/x86/mktme-keys.txt new file +31 -0

+
+   o The range of memory that is to be protected must be mapped as
+     ANONYMOUS. If it is not, the entire encrypt_mprotect() request
+     fails with EINVAL.
+
+   o As an extension to the existing mprotect() system call,
+     encrypt_mprotect() supports the legacy mprotect behavior plus
+     the enabling of memory encryption. That means that in addition
+     to encrypting the memory, the protection flags will be updated
+     as requested in the call.
+
+   o Additional mprotect() calls to memory already protected with
+     MKTME will not alter the MKTME status.
+
+===========  Usage: MKTME Key Service  ===========
+
+MKTME is enabled on supported Intel platforms by selecting
+CONFIG_X86_INTEL_MKTME which selects CONFIG_MKTME_KEYS.
+
+Allocating MKTME Keys via command line or system call:
+    keyctl add mktme name "[options]" ring
+
+    key_serial_t add_key(const char *type, const char *description,
+                         const void *payload, size_t plen,
+                         key_serial_t keyring);
+
+Revoking MKTME Keys via command line or system call::
+   keyctl revoke <key>
+
+   long keyctl(KEYCTL_REVOKE, key_serial_t key);
+

Documentation/x86/mktme-keys.txt new file +7 -0

+		  the tweak key when CPU generated key is requested.
+
+Algorithm Dependencies:
+    AES-XTS 128 is the only supported algorithm.
+    There are only 2 ways that AES-XTS 128 may be used:
+
+    1) User specified encryption key

Documentation/x86/mktme-keys.txt new file +16 -0

+
+  Map memory:
+        ptr = mmap(NULL, size, prot, MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
+
+  Protect memory:
+        ret = syscall(sys_encrypt_mprotect, ptr, size, prot, keyid);
+
+  Use protected memory:
+        ................
+
+  Free memory:
+        ret = munmap(ptr, size);
+
+  Revoke key:
+        ret = keyctl(KEYCTL_REVOKE, key);
+

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Alison Schofield
Sent: Saturday, September 8, 2018 10:34 AM
To: [email protected]; [email protected]
Cc: Huang, Kai <[email protected]>; Nakajima, Jun <[email protected]>; Shutemov, Kirill <[email protected]>; Hansen, Dave <[email protected]>; Sakkinen, Jarkko <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [RFC 01/12] docs/x86: Document the Multi-Key Total Memory Encryption API

Document the API's used for MKTME on Intel platforms.
MKTME: Multi-KEY Total Memory Encryption

Signed-off-by Alison Schofield <[email protected]>

---
 Documentation/x86/mktme-keys.txt | 153
+++++++++++++++++++++++++++++++++++++++
 1 file changed, 153 insertions(+)
 create mode 100644 Documentation/x86/mktme-keys.txt

Documentation/x86/mktme- renamed from Documentation/x86/mktme-keys.txt +0 -0

keys.txt
new file mode 100644
index 000000000000..2dea7acd2a17

Documentation/x86/mktme-keys.txt new file +6 -0

@@ -0,0 +1,153 @@
+MKTME (Multi-Key Total Memory Encryption) is a technology that allows
+memory encryption on Intel platforms. Whereas TME (Total Memory
+Encryption) allows encryption of the entire system memory using a
+single key, MKTME allows multiple encryption domains, each having their
+own key. The main use case for the feature is virtual machine
+isolation. The API's introduced here are intended to offer flexibility to work in a

wide range of uses.

Documentation/x86/mktme-keys.txt new file +14 -0

+
+The externally available Intel Architecture Spec:
+https://software.intel.com/sites/default/files/managed/a5/16/Multi-Key-
+Total-Memory-Encryption-Spec.pdf
+
+==============  API Overview
+==============
+
+There are 2 MKTME specific API's that enable userspace to create and
+use the memory encryption keys:
+
+1) Kernel Key Service: MKTME Type
+
+   MKTME is a new key type added to the existing Kernel Key Services

Documentation/x86/mktme-keys.txt new file +30 -0

+     With that in mind, userspace may take advantage of the kernel
+     key services sharing and permissions model for userspace keys.
+     One key can be shared as long as each user has the permission
+     of "KEY_NEED_VIEW" to use it.
+
+   o MKTME key type uses capabilities to restrict the allocation
+     of keys. It only requires CAP_SYS_RESOURCE, but will accept
+     the broader capability of CAP_SYS_ADMIN.  See capabilities(7).
+
+   o The MKTME key service blocks kernel key service commands that
+     could lead to reprogramming of in use keys, or loss of keys from
+     the pool. This means MKTME does not allow a key to be invalidated,
+     unlinked, or timed out. These operations are blocked by MKTME as
+     it creates all keys with the internal flag KEY_FLAG_KEEP.
+
+   o MKTME does not support the keyctl option of UPDATE. Userspace
+     may change the programming of a key by revoking it and adding
+     a new key with the updated encryption options (or vice-versa).
+
+2) System Call: encrypt_mprotect()
+
+   MKTME encryption is requested by calling encrypt_mprotect(). The
+   caller passes the serial number to a previously allocated and
+   programmed encryption key. That handle was created with the MKTME
+   Key Service.
+
+   o The caller must have KEY_NEED_VIEW permission on the key
+
+   o The range of memory that is to be protected must be mapped as
+     ANONYMOUS. If it is not, the entire encrypt_mprotect() request

Documentation/x86/mktme-keys.txt new file +42 -0

+
+===========  Usage: MKTME Key Service
+===========
+
+MKTME is enabled on supported Intel platforms by selecting
+CONFIG_X86_INTEL_MKTME which selects CONFIG_MKTME_KEYS.
+
+Allocating MKTME Keys via command line or system call:
+    keyctl add mktme name "[options]" ring
+
+    key_serial_t add_key(const char *type, const char *description,
+                         const void *payload, size_t plen,
+                         key_serial_t keyring);
+
+Revoking MKTME Keys via command line or system call::
+   keyctl revoke <key>
+
+   long keyctl(KEYCTL_REVOKE, key_serial_t key);
+
+Options Field Definition:
+    userkey=      ASCII HEX value encryption key. Defaults to a CPU
+		  generated key if a userkey is not defined here.
+
+    algorithm=    Encryption algorithm name as a string.
+		  Valid algorithm: "aes-xts-128"
+
+    tweak=        ASCII HEX value tweak key. Tweak key will be added to the
+                  userkey...  (need to be clear here that this is being sent
+                  to the hardware - kernel not messing w it)
+
+    entropy=      ascii hex value entropy.
+                  This entropy will be used to generated the CPU key and
+		  the tweak key when CPU generated key is requested.
+
+Algorithm Dependencies:
+    AES-XTS 128 is the only supported algorithm.
+    There are only 2 ways that AES-XTS 128 may be used:
+
+    1) User specified encryption key
+	- The user specified encryption key must be exactly
+	  16 ASCII Hex bytes (128 bits).
+	- A tweak key must be specified and it must be exactly

Documentation/x86/mktme-keys.txt new file +14 -0

+
+===========  Usage: encrypt_mprotect()
+===========
+
+System Call encrypt_mprotect()::
+
+    This system call is an extension of the existing mprotect() system
+    call. It requires the same parameters as legary mprotect() plus
+    one additional parameter, the keyid. Userspace must provide the
+    key serial number assigned through the kernel key service.
+
+    int encrypt_mprotect(void *addr, size_t len, int prot, int keyid);
+
+===========  Usage: Sample Roundtrip

===========

Documentation/x86/mktme-keys.txt new file +21 -1

+
+Sample usage of MKTME Key Service API with encrypt_mprotect() API:
+
+  Add a key:
+        key = add_key(mktme, name, options, strlen(option), keyring);
+
+  Map memory:
+        ptr = mmap(NULL, size, prot, MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
+
+  Protect memory:
+        ret = syscall(sys_encrypt_mprotect, ptr, size, prot, keyid);
+
+  Use protected memory:
+        ................
+
+  Free memory:
+        ret = munmap(ptr, size);
+
+  Revoke key:
+        ret = keyctl(KEYCTL_REVOKE, key);
+
--

2.14.1

Quoted 57 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Alison Schofield
Sent: Saturday, September 8, 2018 10:34 AM
To: [email protected]; [email protected]
Cc: Huang, Kai <[email protected]>; Nakajima, Jun <[email protected]>; Shutemov, Kirill <[email protected]>; Hansen, Dave <[email protected]>; Sakkinen, Jarkko <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [RFC 01/12] docs/x86: Document the Multi-Key Total Memory Encryption API

Document the API's used for MKTME on Intel platforms.
MKTME: Multi-KEY Total Memory Encryption

Signed-off-by Alison Schofield <[email protected]>

---
 Documentation/x86/mktme-keys.txt | 153
+++++++++++++++++++++++++++++++++++++++
 1 file changed, 153 insertions(+)
 create mode 100644 Documentation/x86/mktme-keys.txt

Documentation/x86/mktme- renamed from Documentation/x86/mktme-keys.txt +0 -0

keys.txt
new file mode 100644
index 000000000000..2dea7acd2a17

Documentation/x86/mktme-keys.txt new file +6 -0

@@ -0,0 +1,153 @@
+MKTME (Multi-Key Total Memory Encryption) is a technology that allows
+memory encryption on Intel platforms. Whereas TME (Total Memory
+Encryption) allows encryption of the entire system memory using a
+single key, MKTME allows multiple encryption domains, each having their
+own key. The main use case for the feature is virtual machine
+isolation. The API's introduced here are intended to offer flexibility to work in a

wide range of uses.

Documentation/x86/mktme-keys.txt new file +18 -0

+
+The externally available Intel Architecture Spec:
+https://software.intel.com/sites/default/files/managed/a5/16/Multi-Key-
+Total-Memory-Encryption-Spec.pdf
+
+============================  API Overview
+============================
+
+There are 2 MKTME specific API's that enable userspace to create and
+use the memory encryption keys:
+
+1) Kernel Key Service: MKTME Type
+
+   MKTME is a new key type added to the existing Kernel Key Services
+   to support the memory encryption keys. The MKTME service manages
+   the addition and removal of MKTME keys. It maps userspace keys
+   to hardware keyids and programs the hardware with user requested
+   encryption parameters.

Documentation/x86/mktme-keys.txt line 28

Quoted 6 lines

Documentation/x86/mktme-keys.txt new file +6 -0

+
+   o An understanding of the Kernel Key Service is required in order
+     to use the MKTME key type as it is a subset of that service.
+
+   o MKTME keys are a limited resource. There is a single pool of
+     MKTME keys for a system and that pool can be from 3 to 63 keys.

Why 3 to 63 keys? Architecturally we are able to support up to 15-bit keyID, although in the first generation server we only support 6-bit keyID, which is 63 key/keyIDs (excluding keyID 0, which is TME's keyID).

Quoted 35 lines

Documentation/x86/mktme-keys.txt new file +34 -0

+     With that in mind, userspace may take advantage of the kernel
+     key services sharing and permissions model for userspace keys.
+     One key can be shared as long as each user has the permission
+     of "KEY_NEED_VIEW" to use it.
+
+   o MKTME key type uses capabilities to restrict the allocation
+     of keys. It only requires CAP_SYS_RESOURCE, but will accept
+     the broader capability of CAP_SYS_ADMIN.  See capabilities(7).
+
+   o The MKTME key service blocks kernel key service commands that
+     could lead to reprogramming of in use keys, or loss of keys from
+     the pool. This means MKTME does not allow a key to be invalidated,
+     unlinked, or timed out. These operations are blocked by MKTME as
+     it creates all keys with the internal flag KEY_FLAG_KEEP.
+
+   o MKTME does not support the keyctl option of UPDATE. Userspace
+     may change the programming of a key by revoking it and adding
+     a new key with the updated encryption options (or vice-versa).
+
+2) System Call: encrypt_mprotect()
+
+   MKTME encryption is requested by calling encrypt_mprotect(). The
+   caller passes the serial number to a previously allocated and
+   programmed encryption key. That handle was created with the MKTME
+   Key Service.
+
+   o The caller must have KEY_NEED_VIEW permission on the key
+
+   o The range of memory that is to be protected must be mapped as
+     ANONYMOUS. If it is not, the entire encrypt_mprotect() request
+     fails with EINVAL.
+
+   o As an extension to the existing mprotect() system call,
+     encrypt_mprotect() supports the legacy mprotect behavior plus

Documentation/x86/mktme-keys.txt line 68

Quoted 6 lines

Documentation/x86/mktme-keys.txt new file +6 -0

+     the enabling of memory encryption. That means that in addition
+     to encrypting the memory, the protection flags will be updated
+     as requested in the call.
+
+   o Additional mprotect() calls to memory already protected with
+     MKTME will not alter the MKTME status.

I think it's better to separate encrypt_mprotect() into another doc so both parts can be reviewed easier.

Quoted 45 lines

Documentation/x86/mktme-keys.txt new file +44 -0

+
+======================  Usage: MKTME Key Service
+======================
+
+MKTME is enabled on supported Intel platforms by selecting
+CONFIG_X86_INTEL_MKTME which selects CONFIG_MKTME_KEYS.
+
+Allocating MKTME Keys via command line or system call:
+    keyctl add mktme name "[options]" ring
+
+    key_serial_t add_key(const char *type, const char *description,
+                         const void *payload, size_t plen,
+                         key_serial_t keyring);
+
+Revoking MKTME Keys via command line or system call::
+   keyctl revoke <key>
+
+   long keyctl(KEYCTL_REVOKE, key_serial_t key);
+
+Options Field Definition:
+    userkey=      ASCII HEX value encryption key. Defaults to a CPU
+		  generated key if a userkey is not defined here.
+
+    algorithm=    Encryption algorithm name as a string.
+		  Valid algorithm: "aes-xts-128"
+
+    tweak=        ASCII HEX value tweak key. Tweak key will be added to the
+                  userkey...  (need to be clear here that this is being sent
+                  to the hardware - kernel not messing w it)
+
+    entropy=      ascii hex value entropy.
+                  This entropy will be used to generated the CPU key and
+		  the tweak key when CPU generated key is requested.
+
+Algorithm Dependencies:
+    AES-XTS 128 is the only supported algorithm.
+    There are only 2 ways that AES-XTS 128 may be used:
+
+    1) User specified encryption key
+	- The user specified encryption key must be exactly
+	  16 ASCII Hex bytes (128 bits).
+	- A tweak key must be specified and it must be exactly
+	  16 ASCII Hex bytes (128 bits).
+	- No entropy field is accepted.

Documentation/x86/mktme-keys.txt line 119

Quoted 8 lines

Documentation/x86/mktme-keys.txt new file +8 -0

+
+    2) CPU generated encryption key
+	- When no user specified encryption key is provided, the
+	  default encryption key will be CPU generated.
+	- User must specify 16 ASCII Hex bytes of entropy. This
+	  entropy will be used by the CPU to generate both the
+	  encryption key and the tweak key.
+	- No entropy field is accepted.

This is not true. The spec says in CPU generated random mode, both 'key' and 'tweak' part are used to generate the final key and tweak respectively.

Actually, simple 'XOR' is used to generate the final key:

case KEYID_SET_KEY_RANDOM:

	......
	(* Mix user supplied entropy to the data key and tweak key *)
	TMP_RND_DATA_KEY = TMP_RND_KEY XOR
		TMP_KEY_PROGRAM_STRUCT.KEY_FIELD_1.BYTES[15:0];
	TMP_RND_TWEAK_KEY = TMP_RND_TWEAK_KEY XOR
		TMP_KEY_PROGRAM_STRUCT.KEY_FIELD_2.BYTES[15:0];

So I think we can either just remove 'entropy' parameter, since we can use both 'userkey' and 'tweak' even for random key mode.

In fact, which might be better IMHO, we can simply disallow or ignore 'userkey' and 'tweak' part for random key mode, since if we allow user to specify both entropies, and if user passes value with all 1, we are effectively making the key and tweak to be all 1, which is not random anymore.

Instead, kernel can generate random for both entropies, or we can simply uses 0, ignoring user input.

Thanks,
-Kai

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Alison Schofield
Sent: Tuesday, September 11, 2018 12:13 PM
To: Huang, Kai <[email protected]>
Cc: [email protected]; [email protected]; Nakajima, Jun <[email protected]>; Shutemov, Kirill <[email protected]>; Hansen, Dave <[email protected]>; Sakkinen, Jarkko <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC 01/12] docs/x86: Document the Multi-Key Total Memory Encryption API

Quoted 12 lines · Kai

Quoted 12 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Alison Schofield
Sent: Saturday, September 8, 2018 10:34 AM
To: [email protected]; [email protected]
Cc: Huang, Kai <[email protected]>; Nakajima, Jun <[email protected]>; Shutemov, Kirill <[email protected]>; Hansen, Dave <[email protected]>; Sakkinen, Jarkko <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux-

[email protected]

Quoted 47 lines

Quoted 47 lines

Subject: [RFC 01/12] docs/x86: Document the Multi-Key Total Memory Encryption API

Document the API's used for MKTME on Intel platforms.
MKTME: Multi-KEY Total Memory Encryption

Signed-off-by Alison Schofield <[email protected]>

---
 Documentation/x86/mktme-keys.txt | 153
+++++++++++++++++++++++++++++++++++++++
 1 file changed, 153 insertions(+)
 create mode 100644 Documentation/x86/mktme-keys.txt

(quoted diff) +0 -0

b/Documentation/x86/mktme- keys.txt new file mode 100644 index 000000000000..2dea7acd2a17

Documentation/x86/mktme-keys.txt new file +8 -0

@@ -0,0 +1,153 @@
+MKTME (Multi-Key Total Memory Encryption) is a technology that
+allows memory encryption on Intel platforms. Whereas TME (Total
+Memory
+Encryption) allows encryption of the entire system memory using a
+single key, MKTME allows multiple encryption domains, each having
+their own key. The main use case for the feature is virtual machine
+isolation. The API's introduced here are intended to offer
+flexibility to work in a

wide range of uses.

Documentation/x86/mktme-keys.txt new file +19 -0

+
+The externally available Intel Architecture Spec:
+https://software.intel.com/sites/default/files/managed/a5/16/Multi-
+Key-
+Total-Memory-Encryption-Spec.pdf
+
+==============  API Overview
+==============
+
+There are 2 MKTME specific API's that enable userspace to create
+and use the memory encryption keys:
+
+1) Kernel Key Service: MKTME Type
+
+   MKTME is a new key type added to the existing Kernel Key Services
+   to support the memory encryption keys. The MKTME service manages
+   the addition and removal of MKTME keys. It maps userspace keys
+   to hardware keyids and programs the hardware with user requested
+   encryption parameters.

Quoted 8 lines

Documentation/x86/mktme-keys.txt line 28

Quoted 6 lines

Documentation/x86/mktme-keys.txt new file +6 -0

+
+   o An understanding of the Kernel Key Service is required in order
+     to use the MKTME key type as it is a subset of that service.
+
+   o MKTME keys are a limited resource. There is a single pool of
+     MKTME keys for a system and that pool can be from 3 to 63 keys.

Why 3 to 63 keys? Architecturally we are able to support up to 15-bit keyID,

although in the first generation server we only support 6-bit keyID, which is 63 key/keyIDs (excluding keyID 0, which is TME's keyID).

My understanding is that low level SKU's could have as few as 3 bits available to hold the keyid, and that the max is 6 bits, hence 64.
I probably don't need to be stating that level of detail here, but rather just iterate the important point that the resource is limited!

Quoted 35 lines

Quoted 35 lines

Documentation/x86/mktme-keys.txt new file +34 -0

+     With that in mind, userspace may take advantage of the kernel
+     key services sharing and permissions model for userspace keys.
+     One key can be shared as long as each user has the permission
+     of "KEY_NEED_VIEW" to use it.
+
+   o MKTME key type uses capabilities to restrict the allocation
+     of keys. It only requires CAP_SYS_RESOURCE, but will accept
+     the broader capability of CAP_SYS_ADMIN.  See capabilities(7).
+
+   o The MKTME key service blocks kernel key service commands that
+     could lead to reprogramming of in use keys, or loss of keys from
+     the pool. This means MKTME does not allow a key to be invalidated,
+     unlinked, or timed out. These operations are blocked by MKTME as
+     it creates all keys with the internal flag KEY_FLAG_KEEP.
+
+   o MKTME does not support the keyctl option of UPDATE. Userspace
+     may change the programming of a key by revoking it and adding
+     a new key with the updated encryption options (or vice-versa).
+
+2) System Call: encrypt_mprotect()
+
+   MKTME encryption is requested by calling encrypt_mprotect(). The
+   caller passes the serial number to a previously allocated and
+   programmed encryption key. That handle was created with the MKTME
+   Key Service.
+
+   o The caller must have KEY_NEED_VIEW permission on the key
+
+   o The range of memory that is to be protected must be mapped as
+     ANONYMOUS. If it is not, the entire encrypt_mprotect() request
+     fails with EINVAL.
+
+   o As an extension to the existing mprotect() system call,
+     encrypt_mprotect() supports the legacy mprotect behavior plus

Quoted 8 lines

Documentation/x86/mktme-keys.txt line 68

Quoted 6 lines

Documentation/x86/mktme-keys.txt new file +6 -0

+     the enabling of memory encryption. That means that in addition
+     to encrypting the memory, the protection flags will be updated
+     as requested in the call.
+
+   o Additional mprotect() calls to memory already protected with
+     MKTME will not alter the MKTME status.

I think it's better to separate encrypt_mprotect() into another doc so both

parts can be reviewed easier.

I can do that.
Also, I do know I need man page for that too.

Quoted 45 lines

Quoted 45 lines

Documentation/x86/mktme-keys.txt new file +44 -0

+
+===========  Usage: MKTME Key Service
+===========
+
+MKTME is enabled on supported Intel platforms by selecting
+CONFIG_X86_INTEL_MKTME which selects CONFIG_MKTME_KEYS.
+
+Allocating MKTME Keys via command line or system call:
+    keyctl add mktme name "[options]" ring
+
+    key_serial_t add_key(const char *type, const char *description,
+                         const void *payload, size_t plen,
+                         key_serial_t keyring);
+
+Revoking MKTME Keys via command line or system call::
+   keyctl revoke <key>
+
+   long keyctl(KEYCTL_REVOKE, key_serial_t key);
+
+Options Field Definition:
+    userkey=      ASCII HEX value encryption key. Defaults to a CPU
+		  generated key if a userkey is not defined here.
+
+    algorithm=    Encryption algorithm name as a string.
+		  Valid algorithm: "aes-xts-128"
+
+    tweak=        ASCII HEX value tweak key. Tweak key will be added to the
+                  userkey...  (need to be clear here that this is being sent
+                  to the hardware - kernel not messing w it)
+
+    entropy=      ascii hex value entropy.
+                  This entropy will be used to generated the CPU key and
+		  the tweak key when CPU generated key is requested.
+
+Algorithm Dependencies:
+    AES-XTS 128 is the only supported algorithm.
+    There are only 2 ways that AES-XTS 128 may be used:
+
+    1) User specified encryption key
+	- The user specified encryption key must be exactly
+	  16 ASCII Hex bytes (128 bits).
+	- A tweak key must be specified and it must be exactly
+	  16 ASCII Hex bytes (128 bits).
+	- No entropy field is accepted.

Quoted 8 lines

Documentation/x86/mktme-keys.txt line 119

Quoted 8 lines

Documentation/x86/mktme-keys.txt new file +8 -0

+
+    2) CPU generated encryption key
+	- When no user specified encryption key is provided, the
+	  default encryption key will be CPU generated.
+	- User must specify 16 ASCII Hex bytes of entropy. This
+	  entropy will be used by the CPU to generate both the
+	  encryption key and the tweak key.
+	- No entropy field is accepted.

             ^^^^^^^ should be tweak

Quoted 2 lines

This is not true. The spec says in CPU generated random mode, both 'key' and

'tweak' part are used to generate the final key and tweak respectively.

Quoted 4 lines

Actually, simple 'XOR' is used to generate the final key:

case KEYID_SET_KEY_RANDOM:

Quoted 8 lines

	......
	(* Mix user supplied entropy to the data key and tweak key *)
	TMP_RND_DATA_KEY = TMP_RND_KEY XOR
		TMP_KEY_PROGRAM_STRUCT.KEY_FIELD_1.BYTES[15:0];
	TMP_RND_TWEAK_KEY = TMP_RND_TWEAK_KEY XOR
		TMP_KEY_PROGRAM_STRUCT.KEY_FIELD_2.BYTES[15:0];

So I think we can either just remove 'entropy' parameter, since we can use

both 'userkey' and 'tweak' even for random key mode.

Quoted 2 lines

In fact, which might be better IMHO, we can simply disallow or ignore

'userkey' and 'tweak' part for random key mode, since if we allow user to specify both entropies, and if user passes value with all 1, we are effectively making the key and tweak to be all 1, which is not random anymore.

Quoted 2 lines

Instead, kernel can generate random for both entropies, or we can simply uses

0, ignoring user input.

Kai,
I think my typo above, threw you off. We have the same understanding of the key fields.

Is this the structure you are suggesting?

Options

key_type= "user" or "CPU"

key= If key_type = user
key= is the data key
If key_type = CPU
key= is not required
if key= is present
it is entropy to be mixed with CPU generated data key

tweak= If key_type = user
tweak= is the tweak key
If key_type = CPU
tweak= is not required
if tweak= is present
it is entropy to be mixed with CPU generated tweak key

Alison

Quoted 3 lines

Thanks,
-Kai

........snip...........

Quoted 236 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Alison Schofield
Sent: Tuesday, September 11, 2018 12:13 PM
To: Huang, Kai <[email protected]>
Cc: [email protected]; [email protected]; Nakajima, Jun <[email protected]>; Shutemov, Kirill <[email protected]>; Hansen, Dave <[email protected]>; Sakkinen, Jarkko <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [RFC 01/12] docs/x86: Document the Multi-Key Total Memory Encryption API

Quoted 12 lines · Kai

Quoted 12 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Alison Schofield
Sent: Saturday, September 8, 2018 10:34 AM
To: [email protected]; [email protected]
Cc: Huang, Kai <[email protected]>; Nakajima, Jun <[email protected]>; Shutemov, Kirill <[email protected]>; Hansen, Dave <[email protected]>; Sakkinen, Jarkko <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux-

[email protected]

Quoted 47 lines

Quoted 47 lines

Subject: [RFC 01/12] docs/x86: Document the Multi-Key Total Memory Encryption API

Document the API's used for MKTME on Intel platforms.
MKTME: Multi-KEY Total Memory Encryption

Signed-off-by Alison Schofield <[email protected]>

---
 Documentation/x86/mktme-keys.txt | 153
+++++++++++++++++++++++++++++++++++++++
 1 file changed, 153 insertions(+)
 create mode 100644 Documentation/x86/mktme-keys.txt

(quoted diff) +0 -0

b/Documentation/x86/mktme- keys.txt new file mode 100644 index 000000000000..2dea7acd2a17

Documentation/x86/mktme-keys.txt new file +8 -0

@@ -0,0 +1,153 @@
+MKTME (Multi-Key Total Memory Encryption) is a technology that
+allows memory encryption on Intel platforms. Whereas TME (Total
+Memory
+Encryption) allows encryption of the entire system memory using a
+single key, MKTME allows multiple encryption domains, each having
+their own key. The main use case for the feature is virtual machine
+isolation. The API's introduced here are intended to offer
+flexibility to work in a

wide range of uses.

Documentation/x86/mktme-keys.txt new file +19 -0

+
+The externally available Intel Architecture Spec:
+https://software.intel.com/sites/default/files/managed/a5/16/Multi-
+Key-
+Total-Memory-Encryption-Spec.pdf
+
+============================  API Overview
+============================
+
+There are 2 MKTME specific API's that enable userspace to create
+and use the memory encryption keys:
+
+1) Kernel Key Service: MKTME Type
+
+   MKTME is a new key type added to the existing Kernel Key Services
+   to support the memory encryption keys. The MKTME service manages
+   the addition and removal of MKTME keys. It maps userspace keys
+   to hardware keyids and programs the hardware with user requested
+   encryption parameters.

Quoted 8 lines

Documentation/x86/mktme-keys.txt line 28

Quoted 6 lines

Documentation/x86/mktme-keys.txt new file +6 -0

+
+   o An understanding of the Kernel Key Service is required in order
+     to use the MKTME key type as it is a subset of that service.
+
+   o MKTME keys are a limited resource. There is a single pool of
+     MKTME keys for a system and that pool can be from 3 to 63 keys.

Why 3 to 63 keys? Architecturally we are able to support up to 15-bit keyID,

although in the first generation server we only support 6-bit keyID, which is 63 key/keyIDs (excluding keyID 0, which is TME's keyID).

My understanding is that low level SKU's could have as few as 3 bits available to hold the keyid, and that the max is 6 bits, hence 64.
I probably don't need to be stating that level of detail here, but rather just iterate the important point that the resource is limited!

Quoted 35 lines

Quoted 35 lines

Documentation/x86/mktme-keys.txt new file +34 -0

+     With that in mind, userspace may take advantage of the kernel
+     key services sharing and permissions model for userspace keys.
+     One key can be shared as long as each user has the permission
+     of "KEY_NEED_VIEW" to use it.
+
+   o MKTME key type uses capabilities to restrict the allocation
+     of keys. It only requires CAP_SYS_RESOURCE, but will accept
+     the broader capability of CAP_SYS_ADMIN.  See capabilities(7).
+
+   o The MKTME key service blocks kernel key service commands that
+     could lead to reprogramming of in use keys, or loss of keys from
+     the pool. This means MKTME does not allow a key to be invalidated,
+     unlinked, or timed out. These operations are blocked by MKTME as
+     it creates all keys with the internal flag KEY_FLAG_KEEP.
+
+   o MKTME does not support the keyctl option of UPDATE. Userspace
+     may change the programming of a key by revoking it and adding
+     a new key with the updated encryption options (or vice-versa).
+
+2) System Call: encrypt_mprotect()
+
+   MKTME encryption is requested by calling encrypt_mprotect(). The
+   caller passes the serial number to a previously allocated and
+   programmed encryption key. That handle was created with the MKTME
+   Key Service.
+
+   o The caller must have KEY_NEED_VIEW permission on the key
+
+   o The range of memory that is to be protected must be mapped as
+     ANONYMOUS. If it is not, the entire encrypt_mprotect() request
+     fails with EINVAL.
+
+   o As an extension to the existing mprotect() system call,
+     encrypt_mprotect() supports the legacy mprotect behavior plus

Quoted 8 lines

Documentation/x86/mktme-keys.txt line 68

Quoted 6 lines

Documentation/x86/mktme-keys.txt new file +6 -0

+     the enabling of memory encryption. That means that in addition
+     to encrypting the memory, the protection flags will be updated
+     as requested in the call.
+
+   o Additional mprotect() calls to memory already protected with
+     MKTME will not alter the MKTME status.

I think it's better to separate encrypt_mprotect() into another doc so both

parts can be reviewed easier.

I can do that.
Also, I do know I need man page for that too.

Quoted 45 lines

Quoted 45 lines

Documentation/x86/mktme-keys.txt new file +44 -0

+
+======================  Usage: MKTME Key Service
+======================
+
+MKTME is enabled on supported Intel platforms by selecting
+CONFIG_X86_INTEL_MKTME which selects CONFIG_MKTME_KEYS.
+
+Allocating MKTME Keys via command line or system call:
+    keyctl add mktme name "[options]" ring
+
+    key_serial_t add_key(const char *type, const char *description,
+                         const void *payload, size_t plen,
+                         key_serial_t keyring);
+
+Revoking MKTME Keys via command line or system call::
+   keyctl revoke <key>
+
+   long keyctl(KEYCTL_REVOKE, key_serial_t key);
+
+Options Field Definition:
+    userkey=      ASCII HEX value encryption key. Defaults to a CPU
+		  generated key if a userkey is not defined here.
+
+    algorithm=    Encryption algorithm name as a string.
+		  Valid algorithm: "aes-xts-128"
+
+    tweak=        ASCII HEX value tweak key. Tweak key will be added to the
+                  userkey...  (need to be clear here that this is being sent
+                  to the hardware - kernel not messing w it)
+
+    entropy=      ascii hex value entropy.
+                  This entropy will be used to generated the CPU key and
+		  the tweak key when CPU generated key is requested.
+
+Algorithm Dependencies:
+    AES-XTS 128 is the only supported algorithm.
+    There are only 2 ways that AES-XTS 128 may be used:
+
+    1) User specified encryption key
+	- The user specified encryption key must be exactly
+	  16 ASCII Hex bytes (128 bits).
+	- A tweak key must be specified and it must be exactly
+	  16 ASCII Hex bytes (128 bits).
+	- No entropy field is accepted.

Quoted 8 lines

Documentation/x86/mktme-keys.txt line 119

Quoted 8 lines

Documentation/x86/mktme-keys.txt new file +8 -0

+
+    2) CPU generated encryption key
+	- When no user specified encryption key is provided, the
+	  default encryption key will be CPU generated.
+	- User must specify 16 ASCII Hex bytes of entropy. This
+	  entropy will be used by the CPU to generate both the
+	  encryption key and the tweak key.
+	- No entropy field is accepted.

             ^^^^^^^ should be tweak

Quoted 2 lines

This is not true. The spec says in CPU generated random mode, both 'key' and

'tweak' part are used to generate the final key and tweak respectively.

Quoted 4 lines

Actually, simple 'XOR' is used to generate the final key:

case KEYID_SET_KEY_RANDOM:

Quoted 8 lines

	......
	(* Mix user supplied entropy to the data key and tweak key *)
	TMP_RND_DATA_KEY = TMP_RND_KEY XOR
		TMP_KEY_PROGRAM_STRUCT.KEY_FIELD_1.BYTES[15:0];
	TMP_RND_TWEAK_KEY = TMP_RND_TWEAK_KEY XOR
		TMP_KEY_PROGRAM_STRUCT.KEY_FIELD_2.BYTES[15:0];

So I think we can either just remove 'entropy' parameter, since we can use

both 'userkey' and 'tweak' even for random key mode.

Quoted 2 lines

In fact, which might be better IMHO, we can simply disallow or ignore

'userkey' and 'tweak' part for random key mode, since if we allow user to specify both entropies, and if user passes value with all 1, we are effectively making the key and tweak to be all 1, which is not random anymore.

Quoted 2 lines

Instead, kernel can generate random for both entropies, or we can simply uses

0, ignoring user input.

Kai,
I think my typo above, threw you off. We have the same understanding of the key fields.

Is this the structure you are suggesting?

Options

key_type= "user" or "CPU"

key= If key_type == user
key= is the data key
If key_type == CPU
key= is not required
if key= is present
it is entropy to be mixed with CPU generated data key

Quoted 7 lines

tweak= If key_type == user
tweak= is the tweak key
If key_type == CPU
tweak= is not required
if tweak= is present
it is entropy to be mixed with CPU generated tweak key

Exactly.

Although I am not sure whether we should support other 2 modes: Clear key and no encryption;

Thanks,
-Kai

Quoted 8 lines

Alison

Quoted 3 lines

Thanks,
-Kai

........snip...........

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Huang, Kai
Sent: Monday, September 10, 2018 1:28 PM
To: Schofield, Alison <[email protected]>; [email protected]; [email protected]
Cc: Nakajima, Jun <[email protected]>; Shutemov, Kirill <[email protected]>; Hansen, Dave <[email protected]>; Sakkinen, Jarkko <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: RE: [RFC 01/12] docs/x86: Document the Multi-Key Total Memory Encryption API

Quoted 53 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Alison Schofield
Sent: Saturday, September 8, 2018 10:34 AM
To: [email protected]; [email protected]
Cc: Huang, Kai <[email protected]>; Nakajima, Jun <[email protected]>; Shutemov, Kirill <[email protected]>; Hansen, Dave <[email protected]>; Sakkinen, Jarkko <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [RFC 01/12] docs/x86: Document the Multi-Key Total Memory Encryption API

Document the API's used for MKTME on Intel platforms.
MKTME: Multi-KEY Total Memory Encryption

Signed-off-by Alison Schofield <[email protected]>

---
 Documentation/x86/mktme-keys.txt | 153
+++++++++++++++++++++++++++++++++++++++
 1 file changed, 153 insertions(+)
 create mode 100644 Documentation/x86/mktme-keys.txt

(quoted diff) +0 -0

b/Documentation/x86/mktme- keys.txt new file mode 100644 index 000000000000..2dea7acd2a17

Documentation/x86/mktme-keys.txt new file +7 -0

@@ -0,0 +1,153 @@
+MKTME (Multi-Key Total Memory Encryption) is a technology that allows
+memory encryption on Intel platforms. Whereas TME (Total Memory
+Encryption) allows encryption of the entire system memory using a
+single key, MKTME allows multiple encryption domains, each having
+their own key. The main use case for the feature is virtual machine
+isolation. The API's introduced here are intended to offer
+flexibility to work in a

wide range of uses.

Documentation/x86/mktme-keys.txt new file +15 -0

+
+The externally available Intel Architecture Spec:
+https://software.intel.com/sites/default/files/managed/a5/16/Multi-Ke
+y-
+Total-Memory-Encryption-Spec.pdf
+
+==============  API Overview
+==============
+
+There are 2 MKTME specific API's that enable userspace to create and
+use the memory encryption keys:
+
+1) Kernel Key Service: MKTME Type
+
+   MKTME is a new key type added to the existing Kernel Key Services

Documentation/x86/mktme-keys.txt line 28

Quoted 10 lines

Documentation/x86/mktme-keys.txt new file +10 -0

+   to support the memory encryption keys. The MKTME service manages
+   the addition and removal of MKTME keys. It maps userspace keys
+   to hardware keyids and programs the hardware with user requested
+   encryption parameters.
+
+   o An understanding of the Kernel Key Service is required in order
+     to use the MKTME key type as it is a subset of that service.
+
+   o MKTME keys are a limited resource. There is a single pool of
+     MKTME keys for a system and that pool can be from 3 to 63 keys.

Why 3 to 63 keys? Architecturally we are able to support up to 15-bit keyID, although in the first generation server we only support 6-bit keyID, which is 63 key/keyIDs (excluding keyID 0, which is TME's keyID).

Quoted 30 lines

Documentation/x86/mktme-keys.txt new file +30 -0

+     With that in mind, userspace may take advantage of the kernel
+     key services sharing and permissions model for userspace keys.
+     One key can be shared as long as each user has the permission
+     of "KEY_NEED_VIEW" to use it.
+
+   o MKTME key type uses capabilities to restrict the allocation
+     of keys. It only requires CAP_SYS_RESOURCE, but will accept
+     the broader capability of CAP_SYS_ADMIN.  See capabilities(7).
+
+   o The MKTME key service blocks kernel key service commands that
+     could lead to reprogramming of in use keys, or loss of keys from
+     the pool. This means MKTME does not allow a key to be invalidated,
+     unlinked, or timed out. These operations are blocked by MKTME as
+     it creates all keys with the internal flag KEY_FLAG_KEEP.
+
+   o MKTME does not support the keyctl option of UPDATE. Userspace
+     may change the programming of a key by revoking it and adding
+     a new key with the updated encryption options (or vice-versa).
+
+2) System Call: encrypt_mprotect()
+
+   MKTME encryption is requested by calling encrypt_mprotect(). The
+   caller passes the serial number to a previously allocated and
+   programmed encryption key. That handle was created with the MKTME
+   Key Service.
+
+   o The caller must have KEY_NEED_VIEW permission on the key
+
+   o The range of memory that is to be protected must be mapped as
+     ANONYMOUS. If it is not, the entire encrypt_mprotect() request

Documentation/x86/mktme-keys.txt line 68

Quoted 10 lines

Documentation/x86/mktme-keys.txt new file +10 -0

+     fails with EINVAL.
+
+   o As an extension to the existing mprotect() system call,
+     encrypt_mprotect() supports the legacy mprotect behavior plus
+     the enabling of memory encryption. That means that in addition
+     to encrypting the memory, the protection flags will be updated
+     as requested in the call.
+
+   o Additional mprotect() calls to memory already protected with
+     MKTME will not alter the MKTME status.

I think it's better to separate encrypt_mprotect() into another doc so both parts can be reviewed easier.

Quoted 42 lines

Documentation/x86/mktme-keys.txt new file +42 -0

+
+===========  Usage: MKTME Key Service
+===========
+
+MKTME is enabled on supported Intel platforms by selecting
+CONFIG_X86_INTEL_MKTME which selects CONFIG_MKTME_KEYS.
+
+Allocating MKTME Keys via command line or system call:
+    keyctl add mktme name "[options]" ring
+
+    key_serial_t add_key(const char *type, const char *description,
+                         const void *payload, size_t plen,
+                         key_serial_t keyring);
+
+Revoking MKTME Keys via command line or system call::
+   keyctl revoke <key>
+
+   long keyctl(KEYCTL_REVOKE, key_serial_t key);
+
+Options Field Definition:
+    userkey=      ASCII HEX value encryption key. Defaults to a CPU
+		  generated key if a userkey is not defined here.
+
+    algorithm=    Encryption algorithm name as a string.
+		  Valid algorithm: "aes-xts-128"
+
+    tweak=        ASCII HEX value tweak key. Tweak key will be added to the
+                  userkey...  (need to be clear here that this is being sent
+                  to the hardware - kernel not messing w it)
+
+    entropy=      ascii hex value entropy.
+                  This entropy will be used to generated the CPU key and
+		  the tweak key when CPU generated key is requested.
+
+Algorithm Dependencies:
+    AES-XTS 128 is the only supported algorithm.
+    There are only 2 ways that AES-XTS 128 may be used:
+
+    1) User specified encryption key
+	- The user specified encryption key must be exactly
+	  16 ASCII Hex bytes (128 bits).
+	- A tweak key must be specified and it must be exactly

Documentation/x86/mktme-keys.txt line 119

Quoted 10 lines

Documentation/x86/mktme-keys.txt new file +10 -0

+	  16 ASCII Hex bytes (128 bits).
+	- No entropy field is accepted.
+
+    2) CPU generated encryption key
+	- When no user specified encryption key is provided, the
+	  default encryption key will be CPU generated.
+	- User must specify 16 ASCII Hex bytes of entropy. This
+	  entropy will be used by the CPU to generate both the
+	  encryption key and the tweak key.
+	- No entropy field is accepted.

This is not true. The spec says in CPU generated random mode, both 'key' and 'tweak' part are used to generate the final key and tweak respectively.

Actually, simple 'XOR' is used to generate the final key:

case KEYID_SET_KEY_RANDOM:

	......
	(* Mix user supplied entropy to the data key and tweak key *)
	TMP_RND_DATA_KEY = TMP_RND_KEY XOR
		TMP_KEY_PROGRAM_STRUCT.KEY_FIELD_1.BYTES[15:0];
	TMP_RND_TWEAK_KEY = TMP_RND_TWEAK_KEY XOR
		TMP_KEY_PROGRAM_STRUCT.KEY_FIELD_2.BYTES[15:0];

So I think we can either just remove 'entropy' parameter, since we can use both 'userkey' and 'tweak' even for random key mode.

In fact, which might be better IMHO, we can simply disallow or ignore 'userkey' and 'tweak' part for random key mode, since if we allow user to specify both entropies, and if user passes value with all 1, we are effectively making the key and tweak to be all 1, which is not random anymore.

Instead, kernel can generate random for both entropies, or we can simply uses 0, ignoring user input.

Thanks,
-Kai

Quoted 40 lines

Documentation/x86/mktme-keys.txt new file +15 -0

+
+===========  Usage: encrypt_mprotect()
+===========
+
+System Call encrypt_mprotect()::
+
+    This system call is an extension of the existing mprotect() system
+    call. It requires the same parameters as legary mprotect() plus
+    one additional parameter, the keyid. Userspace must provide the
+    key serial number assigned through the kernel key service.
+
+    int encrypt_mprotect(void *addr, size_t len, int prot, int
+ keyid);
+
+===========  Usage: Sample Roundtrip

===========

Documentation/x86/mktme-keys.txt new file +22 -1

+
+Sample usage of MKTME Key Service API with encrypt_mprotect() API:
+
+  Add a key:
+        key = add_key(mktme, name, options, strlen(option), keyring);
+
+  Map memory:
+        ptr = mmap(NULL, size, prot, MAP_ANONYMOUS|MAP_PRIVATE, -1,
+ 0);
+
+  Protect memory:
+        ret = syscall(sys_encrypt_mprotect, ptr, size, prot, keyid);
+
+  Use protected memory:
+        ................
+
+  Free memory:
+        ret = munmap(ptr, size);
+
+  Revoke key:
+        ret = keyctl(KEYCTL_REVOKE, key);
+
--

2.14.1

Quoted 27 lines · Alison Schofield

Document the API's used for MKTME on Intel platforms.
MKTME: Multi-KEY Total Memory Encryption

Signed-off-by Alison Schofield <[email protected]>

---
 Documentation/x86/mktme-keys.txt | 153
+++++++++++++++++++++++++++++++++++++++
 1 file changed, 153 insertions(+)
 create mode 100644 Documentation/x86/mktme-keys.txt

Documentation/x86/mktme- renamed from Documentation/x86/mktme-keys.txt +0 -0

keys.txt
new file mode 100644
index 000000000000..2dea7acd2a17

Documentation/x86/mktme-keys.txt new file +9 -0

@@ -0,0 +1,153 @@
+MKTME (Multi-Key Total Memory Encryption) is a technology that allows
+memory encryption on Intel platforms. Whereas TME (Total Memory Encryption)
+allows encryption of the entire system memory using a single key, MKTME
+allows multiple encryption domains, each having their own key. The main use
+case for the feature is virtual machine isolation. The API's introduced here
+are intended to offer flexibility to work in a wide range of uses.
+
+The externally available Intel Architecture Spec:
+https://software.intel.com/sites/default/files/managed/a5/16/Multi-Key-Total-

Memory-Encryption-Spec.pdf

Documentation/x86/mktme-keys.txt line 14

Quoted 5 lines

Documentation/x86/mktme-keys.txt new file +5 -0

+
+============================  API Overview  ============================
+
+There are 2 MKTME specific API's that enable userspace to create and use
+the memory encryption keys:

This is like saying that they are different APIs to do semantically the same exact thing. Is that so?

/Jarkko

Today mprotect is implemented to support legacy mprotect behavior plus an extension for memory protection keys. Make it more generic so that it can support additional extensions in the future.

This is done is preparation for adding a new system call for memory encyption keys. The intent is that the new encrypted mprotect will be another extension to legacy mprotect.

Signed-off-by Alison Schofield <[email protected]>

---
 mm/mprotect.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

mm/mprotect.c +0 -0

index 68dc476310c0..56e64ef7931e 100644

Quoted 19 lines · Alison Schofield

Today mprotect is implemented to support legacy mprotect behavior plus an extension for memory protection keys. Make it more generic so that it can support additional extensions in the future.

This is done is preparation for adding a new system call for memory encyption keys. The intent is that the new encrypted mprotect will be another extension to legacy mprotect.

Signed-off-by Alison Schofield <[email protected]>

---
 mm/mprotect.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

mm/mprotect.c +0 -0

index 68dc476310c0..56e64ef7931e 100644
@@ -35,6 +35,8 @@
 

mm/mprotect.c line 38

Quoted 3 lines

mm/mprotect.c +1 -0

 #include "internal.h"
 
+#define NO_PKEY  -1

This commit does not make anything more generic but it does take away a magic number. The code change is senseful. The commit message is nonsense.

PS. Please use @linux.intel.com for LKML.

/Jarkko

-----Original Message-----
From: [email protected] [mailto:owner-linux- [email protected]] On Behalf Of Alison Schofield
Sent: Saturday, September 8, 2018 10:36 AM
To: [email protected]; [email protected]
Cc: Huang, Kai <[email protected]>; Nakajima, Jun <[email protected]>; Shutemov, Kirill <[email protected]>; Hansen, Dave <[email protected]>; Sakkinen, Jarkko <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [RFC 04/12] x86/mm: Add helper functions to manage memory encryption keys

Define a global mapping structure to track the mapping of userspace keys to hardware keyids in MKTME (Multi-Key Total Memory Encryption).
This data will be used for the memory encryption system call and the kernel key service API.

Implement helper functions to access this mapping structure and make them visible to the MKTME Kernel Key Service: security/keys/mktme_keys

Signed-off-by Alison Schofield <[email protected]>

---
 arch/x86/include/asm/mktme.h | 11 ++++++
 arch/x86/mm/mktme.c          | 85
++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 96 insertions(+)

arch/x86/include/asm/mktme.h +9 -0

index dbfbd955da98..f6acd551457f 100644
@@ -13,6 +13,17 @@ extern phys_addr_t mktme_keyid_mask;  extern int
 mktme_nr_keyids;  extern int mktme_keyid_shift;
 
+/* Manage mappings between hardware keyids and userspace keys */ extern
+int mktme_map_alloc(void); extern void mktme_map_free(void); extern
+void mktme_map_lock(void); extern void mktme_map_unlock(void); extern
+int mktme_map_get_free_keyid(void); extern void
+mktme_map_clear_keyid(int keyid); extern void mktme_map_set_keyid(int
+keyid, unsigned int serial); extern int
+mktme_map_keyid_from_serial(unsigned int serial); extern unsigned int
+mktme_map_serial_from_keyid(int keyid);
+
 extern struct page_ext_operations page_mktme_ops;
 
 #define page_keyid page_keyid

arch/x86/mm/mktme.c index renamed from arch/x86/mm/mktme.c +0 -0

660caf6a5ce1..5246d8323359 100644

arch/x86/mm/mktme.c +82 -1

@@ -63,6 +63,91 @@ int vma_keyid(struct vm_area_struct *vma)
 	return (prot & mktme_keyid_mask) >> mktme_keyid_shift;  }
 
+/*
+ * struct mktme_mapping and the mktme_map_* functions manage the
+mapping
+ * of userspace keys to hardware keyids in MKTME. They are used by the
+ * the encrypt_mprotect system call and the MKTME Key Service API.
+ */
+struct mktme_mapping {
+	struct mutex	lock;		/* protect this map & HW state */
+	unsigned int	mapped_keyids;
+	unsigned int	serial[];
+};
+
+struct mktme_mapping *mktme_map;
+
+static inline long mktme_map_size(void) {
+	long size = 0;
+
+	size += sizeof(mktme_map);
+	size += sizeof(mktme_map->serial[0]) * mktme_nr_keyids;
+	return size;
+}
+
+int mktme_map_alloc(void)
+{
+	mktme_map = kzalloc(mktme_map_size(), GFP_KERNEL);
+	if (!mktme_map)
+		return 0;
+	mutex_init(&mktme_map->lock);
+	return 1;
+}
+
+void mktme_map_free(void)
+{
+	kfree(mktme_map);
+}
+
+void mktme_map_lock(void)
+{
+	mutex_lock(&mktme_map->lock);
+}
+
+void mktme_map_unlock(void)
+{
+	mutex_unlock(&mktme_map->lock);
+}
+
+void mktme_map_set_keyid(int keyid, unsigned int serial) {
+	mktme_map->serial[keyid] = serial;
+	mktme_map->mapped_keyids++;
+}
+
+void mktme_map_clear_keyid(int keyid)
+{
+	mktme_map->serial[keyid] = 0;
+	mktme_map->mapped_keyids--;
+}
+
+unsigned int mktme_map_serial_from_keyid(int keyid) {
+	return mktme_map->serial[keyid];
+}
+
+int mktme_map_keyid_from_serial(unsigned int serial) {
+	int i;
+
+	for (i = 1; i < mktme_nr_keyids; i++)
+		if (mktme_map->serial[i] = serial)
+			return i;
+	return 0;
+}
+
+int mktme_map_get_free_keyid(void)
+{
+	int i;
+
+	if (mktme_map->mapped_keyids < mktme_nr_keyids) {
+		for (i = 1; i < mktme_nr_keyids; i++)
+			if (mktme_map->serial[i] = 0)
+				return i;
+	}
+	return 0;
+}
+
 void prep_encrypted_page(struct page *page, int order, int keyid, bool zero)  {
 	int i;
--

2.14.1

-----Original Message-----
From: [email protected] [mailto:owner-linux- [email protected]] On Behalf Of Huang, Kai
Sent: Monday, September 10, 2018 2:57 PM
To: Schofield, Alison <[email protected]>; [email protected]; [email protected]
Cc: Nakajima, Jun <[email protected]>; Shutemov, Kirill <[email protected]>; Hansen, Dave <[email protected]>; Sakkinen, Jarkko <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: RE: [RFC 04/12] x86/mm: Add helper functions to manage memory encryption keys

Quoted 22 lines

-----Original Message-----
From: [email protected] [mailto:owner-linux- [email protected]] On Behalf Of Alison Schofield
Sent: Saturday, September 8, 2018 10:36 AM
To: [email protected]; [email protected]
Cc: Huang, Kai <[email protected]>; Nakajima, Jun <[email protected]>; Shutemov, Kirill <[email protected]>; Hansen, Dave <[email protected]>; Sakkinen, Jarkko <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [RFC 04/12] x86/mm: Add helper functions to manage memory encryption keys

Define a global mapping structure to track the mapping of userspace keys to hardware keyids in MKTME (Multi-Key Total Memory Encryption). This data will be used for the memory encryption system call and the kernel key service API.

Implement helper functions to access this mapping structure and make them visible to the MKTME Kernel Key Service: security/keys/mktme_keys

Quoted 6 lines

Signed-off-by Alison Schofield <[email protected]>

---
 arch/x86/include/asm/mktme.h | 11 ++++++
 arch/x86/mm/mktme.c          | 85
++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 96 insertions(+)

Maybe it's better to put those changes to include/keys/mktme-type.h, and security/keys/mktme_key.c? It seems you don't have to involve linux-mm and x86 guys by doing so?

Thanks,
-Kai

Quoted 27 lines

(quoted diff) +0 -0

b/arch/x86/include/asm/mktme.h index dbfbd955da98..f6acd551457f 100644

arch/x86/include/asm/mktme.h +9 -0

@@ -13,6 +13,17 @@ extern phys_addr_t mktme_keyid_mask;  extern int
 mktme_nr_keyids;  extern int mktme_keyid_shift;
 
+/* Manage mappings between hardware keyids and userspace keys */
+extern int mktme_map_alloc(void); extern void mktme_map_free(void);
+extern void mktme_map_lock(void); extern void mktme_map_unlock(void);
+extern int mktme_map_get_free_keyid(void); extern void
+mktme_map_clear_keyid(int keyid); extern void mktme_map_set_keyid(int
+keyid, unsigned int serial); extern int
+mktme_map_keyid_from_serial(unsigned int serial); extern unsigned int
+mktme_map_serial_from_keyid(int keyid);
+
 extern struct page_ext_operations page_mktme_ops;
 
 #define page_keyid page_keyid

arch/x86/mm/mktme.c index renamed from arch/x86/mm/mktme.c +0 -0

660caf6a5ce1..5246d8323359 100644

arch/x86/mm/mktme.c +0 -0

@@ -63,6 +63,91 @@ int vma_keyid(struct vm_area_struct *vma)
 	return (prot & mktme_keyid_mask) >> mktme_keyid_shift;  }
 

arch/x86/mm/mktme.c int vma_keyid(struct vm_area_struct *vma) · line 75

Quoted 12 lines

arch/x86/mm/mktme.c +12 -0

+/*
+ * struct mktme_mapping and the mktme_map_* functions manage the
+mapping
+ * of userspace keys to hardware keyids in MKTME. They are used by
+the
+ * the encrypt_mprotect system call and the MKTME Key Service API.
+ */
+struct mktme_mapping {
+	struct mutex	lock;		/* protect this map & HW state */
+	unsigned int	mapped_keyids;
+	unsigned int	serial[];
+};

Quoted 74 lines

arch/x86/mm/mktme.c +70 -1

+
+struct mktme_mapping *mktme_map;
+
+static inline long mktme_map_size(void) {
+	long size = 0;
+
+	size += sizeof(mktme_map);
+	size += sizeof(mktme_map->serial[0]) * mktme_nr_keyids;
+	return size;
+}
+
+int mktme_map_alloc(void)
+{
+	mktme_map = kzalloc(mktme_map_size(), GFP_KERNEL);
+	if (!mktme_map)
+		return 0;
+	mutex_init(&mktme_map->lock);
+	return 1;
+}
+
+void mktme_map_free(void)
+{
+	kfree(mktme_map);
+}
+
+void mktme_map_lock(void)
+{
+	mutex_lock(&mktme_map->lock);
+}
+
+void mktme_map_unlock(void)
+{
+	mutex_unlock(&mktme_map->lock);
+}
+
+void mktme_map_set_keyid(int keyid, unsigned int serial) {
+	mktme_map->serial[keyid] = serial;
+	mktme_map->mapped_keyids++;
+}
+
+void mktme_map_clear_keyid(int keyid) {
+	mktme_map->serial[keyid] = 0;
+	mktme_map->mapped_keyids--;
+}
+
+unsigned int mktme_map_serial_from_keyid(int keyid) {
+	return mktme_map->serial[keyid];
+}
+
+int mktme_map_keyid_from_serial(unsigned int serial) {
+	int i;
+
+	for (i = 1; i < mktme_nr_keyids; i++)
+		if (mktme_map->serial[i] = serial)
+			return i;
+	return 0;
+}
+
+int mktme_map_get_free_keyid(void)
+{
+	int i;
+
+	if (mktme_map->mapped_keyids < mktme_nr_keyids) {
+		for (i = 1; i < mktme_nr_keyids; i++)
+			if (mktme_map->serial[i] = 0)
+				return i;
+	}
+	return 0;
+}
+
 void prep_encrypted_page(struct page *page, int order, int keyid, bool zero)  {
 	int i;
--

2.14.1

Quoted 77 lines

-----Original Message-----
From: [email protected] [mailto:owner-linux- [email protected]] On Behalf Of Huang, Kai
Sent: Monday, September 10, 2018 2:57 PM
To: Schofield, Alison <[email protected]>; [email protected]; [email protected]
Cc: Nakajima, Jun <[email protected]>; Shutemov, Kirill <[email protected]>; Hansen, Dave <[email protected]>; Sakkinen, Jarkko <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: RE: [RFC 04/12] x86/mm: Add helper functions to manage memory encryption keys

Quoted 22 lines

-----Original Message-----
From: [email protected] [mailto:owner-linux- [email protected]] On Behalf Of Alison Schofield
Sent: Saturday, September 8, 2018 10:36 AM
To: [email protected]; [email protected]
Cc: Huang, Kai <[email protected]>; Nakajima, Jun <[email protected]>; Shutemov, Kirill <[email protected]>; Hansen, Dave <[email protected]>; Sakkinen, Jarkko <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [RFC 04/12] x86/mm: Add helper functions to manage memory encryption keys

Define a global mapping structure to track the mapping of userspace keys to hardware keyids in MKTME (Multi-Key Total Memory Encryption). This data will be used for the memory encryption system call and the kernel key service API.

Implement helper functions to access this mapping structure and make them visible to the MKTME Kernel Key Service: security/keys/mktme_keys

Quoted 6 lines

Signed-off-by Alison Schofield <[email protected]>

---
 arch/x86/include/asm/mktme.h | 11 ++++++
 arch/x86/mm/mktme.c          | 85
++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 96 insertions(+)

Maybe it's better to put those changes to include/keys/mktme-type.h, and security/keys/mktme_key.c? It seems you don't have to involve linux-mm and x86 guys by doing so?

Thanks,
-Kai

Quoted 27 lines

(quoted diff) +0 -0

b/arch/x86/include/asm/mktme.h index dbfbd955da98..f6acd551457f 100644

arch/x86/include/asm/mktme.h +9 -0

@@ -13,6 +13,17 @@ extern phys_addr_t mktme_keyid_mask;  extern int
 mktme_nr_keyids;  extern int mktme_keyid_shift;
 
+/* Manage mappings between hardware keyids and userspace keys */
+extern int mktme_map_alloc(void); extern void mktme_map_free(void);
+extern void mktme_map_lock(void); extern void mktme_map_unlock(void);
+extern int mktme_map_get_free_keyid(void); extern void
+mktme_map_clear_keyid(int keyid); extern void mktme_map_set_keyid(int
+keyid, unsigned int serial); extern int
+mktme_map_keyid_from_serial(unsigned int serial); extern unsigned int
+mktme_map_serial_from_keyid(int keyid);
+
 extern struct page_ext_operations page_mktme_ops;
 
 #define page_keyid page_keyid

arch/x86/mm/mktme.c index renamed from arch/x86/mm/mktme.c +0 -0

660caf6a5ce1..5246d8323359 100644

arch/x86/mm/mktme.c +0 -0

@@ -63,6 +63,91 @@ int vma_keyid(struct vm_area_struct *vma)
 	return (prot & mktme_keyid_mask) >> mktme_keyid_shift;  }
 

Quoted 12 lines

arch/x86/mm/mktme.c int vma_keyid(struct vm_area_struct *vma) · line 75

Quoted 12 lines

arch/x86/mm/mktme.c +12 -0

+/*
+ * struct mktme_mapping and the mktme_map_* functions manage the
+mapping
+ * of userspace keys to hardware keyids in MKTME. They are used by
+the
+ * the encrypt_mprotect system call and the MKTME Key Service API.
+ */
+struct mktme_mapping {
+	struct mutex	lock;		/* protect this map & HW state */
+	unsigned int	mapped_keyids;
+	unsigned int	serial[];
+};

Sorry one more comment that I missed yesterday:

I think 'key_serial_t' should be used as type of serial throughout this patch, but not 'unsigned int'.

Thanks,
-Kai

Define a global mapping structure to track the mapping of userspace keys to hardware keyids in MKTME (Multi-Key Total Memory Encryption). This data will be used for the memory encryption system call and the kernel key service API.

Implement helper functions to access this mapping structure and make them visible to the MKTME Kernel Key Service: security/keys/mktme_keys

Signed-off-by Alison Schofield <[email protected]>

---
 arch/x86/include/asm/mktme.h | 11 ++++++
 arch/x86/mm/mktme.c          | 85
++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 96 insertions(+)

arch/x86/include/asm/mktme.h +11 -0

index dbfbd955da98..f6acd551457f 100644
@@ -13,6 +13,17 @@ extern phys_addr_t mktme_keyid_mask;
 extern int mktme_nr_keyids;
 extern int mktme_keyid_shift;
 
+/* Manage mappings between hardware keyids and userspace keys */
+extern int mktme_map_alloc(void);
+extern void mktme_map_free(void);
+extern void mktme_map_lock(void);
+extern void mktme_map_unlock(void);
+extern int mktme_map_get_free_keyid(void);
+extern void mktme_map_clear_keyid(int keyid);
+extern void mktme_map_set_keyid(int keyid, unsigned int serial);
+extern int mktme_map_keyid_from_serial(unsigned int serial);
+extern unsigned int mktme_map_serial_from_keyid(int keyid);
+
 extern struct page_ext_operations page_mktme_ops;
 
 #define page_keyid page_keyid

arch/x86/mm/mktme.c +7 -0

index 660caf6a5ce1..5246d8323359 100644
@@ -63,6 +63,91 @@ int vma_keyid(struct vm_area_struct *vma)
 	return (prot & mktme_keyid_mask) >> mktme_keyid_shift;
 }
 
+/*
+ * struct mktme_mapping and the mktme_map_* functions manage the mapping
+ * of userspace keys to hardware keyids in MKTME. They are used by the
+ * the encrypt_mprotect system call and the MKTME Key Service API.
+ */
+struct mktme_mapping {
+	struct mutex	lock;		/* protect this map & HW

state */

arch/x86/mm/mktme.c +67 -0

+	unsigned int	mapped_keyids;
+	unsigned int	serial[];
+};
+
+struct mktme_mapping *mktme_map;
+
+static inline long mktme_map_size(void)
+{
+	long size = 0;
+
+	size += sizeof(mktme_map);
+	size += sizeof(mktme_map->serial[0]) * mktme_nr_keyids;
+	return size;
+}
+
+int mktme_map_alloc(void)
+{
+	mktme_map = kzalloc(mktme_map_size(), GFP_KERNEL);
+	if (!mktme_map)
+		return 0;
+	mutex_init(&mktme_map->lock);
+	return 1;
+}
+
+void mktme_map_free(void)
+{
+	kfree(mktme_map);
+}
+
+void mktme_map_lock(void)
+{
+	mutex_lock(&mktme_map->lock);
+}
+
+void mktme_map_unlock(void)
+{
+	mutex_unlock(&mktme_map->lock);
+}
+
+void mktme_map_set_keyid(int keyid, unsigned int serial)
+{
+	mktme_map->serial[keyid] = serial;
+	mktme_map->mapped_keyids++;
+}
+
+void mktme_map_clear_keyid(int keyid)
+{
+	mktme_map->serial[keyid] = 0;
+	mktme_map->mapped_keyids--;
+}
+
+unsigned int mktme_map_serial_from_keyid(int keyid)
+{
+	return mktme_map->serial[keyid];
+}
+
+int mktme_map_keyid_from_serial(unsigned int serial)
+{
+	int i;
+
+	for (i = 1; i < mktme_nr_keyids; i++)
+		if (mktme_map->serial[i] == serial)
+			return i;
+	return 0;
+}
+
+int mktme_map_get_free_keyid(void)

Implement memory encryption with a new system call that is an extension of the legacy mprotect() system call.

In encrypt_mprotect the caller must pass a handle to a previously allocated and programmed encryption key. Validate the key and store the keyid bits in the vm_page_prot for each VMA in the protection range.

Signed-off-by Alison Schofield <[email protected]>

---
 fs/exec.c           |  4 ++--
 include/linux/key.h |  2 ++
 include/linux/mm.h  |  3 ++-
 mm/mprotect.c       | 67 ++++++++++++++++++++++++++++++++++++++++++++++++--
---
 4 files changed, 67 insertions(+), 9 deletions(-)

fs/exec.c +0 -0

index a1a246062561..b681a413db9c 100644

fs/exec.c +0 -0

 	if (ret)
 		goto out_unlock;

fs/exec.c +0 -0

 	BUG_ON(prev != vma);

include/linux/key.h +2 -0

index e58ee10f6e58..fb8a7d5f6149 100644
@@ -346,6 +346,8 @@ static inline key_serial_t key_serial(const struct key
 *key)
 
 extern void key_set_timeout(struct key *, unsigned);
 
+extern key_ref_t lookup_user_key(key_serial_t id, unsigned long lflags,
+				 key_perm_t perm);
 /*
  * The permissions required on a key that we're looking up.

*/

include/linux/mm.h +0 -0

index ac85c0805761..0f9422c7841e 100644
@@ -1579,7 +1579,8 @@ extern unsigned long change_protection(struct

vm_area_struct *vma, unsigned long

include/linux/mm.h +2 -1

 			      int dirty_accountable, int prot_numa);
 extern int mprotect_fixup(struct vm_area_struct *vma,
 			  struct vm_area_struct **pprev, unsigned long start,
-			  unsigned long end, unsigned long newflags);
+			  unsigned long end, unsigned long newflags,
+			  int newkeyid);
 
 /*
  * doesn't attempt to fault and will return short.

mm/mprotect.c +1 -0

index 56e64ef7931e..6c2e1106525c 100644
@@ -28,14 +28,17 @@
 #include <linux/ksm.h>
 #include <linux/uaccess.h>
 #include <linux/mm_inline.h>
+#include <linux/key.h>

mm/mprotect.c +2 -1

 static unsigned long change_pte_range(struct vm_area_struct *vma, pmd_t *pmd,
 		unsigned long addr, unsigned long end, pgprot_t newprot,

@@ -310,7 +313,8 @@ unsigned long change_protection(struct vm_area_struct
 *vma, unsigned long start,
 
 int
 mprotect_fixup(struct vm_area_struct *vma, struct vm_area_struct **pprev,
-	unsigned long start, unsigned long end, unsigned long newflags)
+	       unsigned long start, unsigned long end, unsigned long
 newflags,
+	       int newkeyid)
 {

mm/mprotect.c +14 -0

 	struct mm_struct *mm = vma->vm_mm;
 	unsigned long oldflags = vma->vm_flags;

@@ -320,10 +324,24 @@ mprotect_fixup(struct vm_area_struct *vma, struct
 vm_area_struct **pprev,
 	int error;
 	int dirty_accountable = 0;
 
+	/*
+	 * Flags match and Keyids match or we have NO_KEYID.
+	 * This _fixup is usually called from do_mprotect_ext() except
+	 * for one special case: caller fs/exec.c/setup_arg_pages()
+	 * In that case, newkeyid is passed as -1 (NO_KEYID).
+	 */
+	if (newflags = oldflags &&
+	    (newkeyid = vma_keyid(vma) || newkeyid = NO_KEYID)) {
+		*pprev = vma;
+		return 0;
+	}
+	/* Flags match and Keyid changes */
 	if (newflags = oldflags) {
+		mprotect_set_encrypt(vma, newkeyid);
 		*pprev = vma;
 		return 0;
 	}
+	/* Flags and Keyids both change, continue. */
 
 	/*

mm/mprotect.c +2 -0

 	 * If we make a private mapping writable we increase our commit;

@@ -373,6 +391,8 @@ mprotect_fixup(struct vm_area_struct *vma, struct
 vm_area_struct **pprev,
 	}
 
 success:
+	if (newkeyid != NO_KEYID)
+		mprotect_set_encrypt(vma, newkeyid);
 	/*
 	 * vm_flags and vm_page_prot are protected by the mmap_sem

mm/mprotect.c +0 -0

 	 * held in write mode.

@@ -404,10 +424,15 @@ mprotect_fixup(struct vm_area_struct *vma, struct
 vm_area_struct **pprev,
 }

mm/mprotect.c +1 -1

 static int do_mprotect_ext(unsigned long start, size_t len,
-		unsigned long prot, int pkey)
+			   unsigned long prot, int pkey, int keyid)
 {
 	unsigned long nstart, end, tmp, reqprot;

mm/mprotect.c +2 -1

 	struct vm_area_struct *vma, *prev;

@@ -505,7 +530,8 @@ static int do_mprotect_ext(unsigned long start, size_t
 len,
 		tmp = vma->vm_end;
 		if (tmp > end)
 			tmp = end;
-		error = mprotect_fixup(vma, &prev, nstart, tmp, newflags);
+		error = mprotect_fixup(vma, &prev, nstart, tmp, newflags,
+				       keyid);
 		if (error)
 			goto out;

mm/mprotect.c +1 -1

 		nstart = tmp;

@@ -530,7 +556,7 @@ static int do_mprotect_ext(unsigned long start, size_t
 len,
 SYSCALL_DEFINE3(mprotect, unsigned long, start, size_t, len,
 		unsigned long, prot)
 {
-	return do_mprotect_ext(start, len, prot, NO_PKEY);
+	return do_mprotect_ext(start, len, prot, NO_PKEY, NO_KEYID);
 }
 

#ifdef CONFIG_ARCH_HAS_PKEYS

mm/mprotect.c +1 -1

@@ -538,7 +564,7 @@ SYSCALL_DEFINE3(mprotect, unsigned long, start, size_t,
 len,
 SYSCALL_DEFINE4(pkey_mprotect, unsigned long, start, size_t, len,
 		unsigned long, prot, int, pkey)
 {
-	return do_mprotect_ext(start, len, prot, pkey);
+	return do_mprotect_ext(start, len, prot, pkey, NO_KEYID);
 }
 

SYSCALL_DEFINE2(pkey_alloc, unsigned long, flags, unsigned long, init_val)

mm/mprotect.c +19 -0

@@ -587,3 +613,32 @@ SYSCALL_DEFINE1(pkey_free, int, pkey)
 }
 
 #endif /* CONFIG_ARCH_HAS_PKEYS */
+
+#ifdef CONFIG_X86_INTEL_MKTME
+
+SYSCALL_DEFINE4(encrypt_mprotect, unsigned long, start, size_t, len,
+		unsigned long, prot, key_serial_t, serial)
+{
+	key_ref_t key_ref;
+	int ret, keyid;
+
+	/* TODO MKTME key service must be initialized */
+
+	key_ref = lookup_user_key(serial, 0, KEY_NEED_VIEW);
+	if (IS_ERR(key_ref))
+		return PTR_ERR(key_ref);
+
+	mktme_map_lock();
+	keyid = mktme_map_keyid_from_serial(serial);
+	if (!keyid) {
+		mktme_map_unlock();

Quoted 25 lines · Alison Schofield

Implement memory encryption with a new system call that is an extension of the legacy mprotect() system call.

In encrypt_mprotect the caller must pass a handle to a previously allocated and programmed encryption key. Validate the key and store the keyid bits in the vm_page_prot for each VMA in the protection range.

Signed-off-by Alison Schofield <[email protected]>

---
 fs/exec.c           |  4 ++--
 include/linux/key.h |  2 ++
 include/linux/mm.h  |  3 ++-
 mm/mprotect.c       | 67 ++++++++++++++++++++++++++++++++++++++++++++++++--
---
 4 files changed, 67 insertions(+), 9 deletions(-)

fs/exec.c +0 -0

index a1a246062561..b681a413db9c 100644
@@ -754,8 +754,8 @@ int setup_arg_pages(struct linux_binprm *bprm,
 	vm_flags |= mm->def_flags;
 	vm_flags |= VM_STACK_INCOMPLETE_SETUP;
 

fs/exec.c int setup_arg_pages(struct linux_binprm *bprm, · line 758

Quoted 5 lines

fs/exec.c +2 -2

-	ret = mprotect_fixup(vma, &prev, vma->vm_start, vma->vm_end,
-			vm_flags);
+	ret = mprotect_fixup(vma, &prev, vma->vm_start, vma->vm_end,
 vm_flags,
+			     -1);

Why you pass a magic number here when you went the trouble having a named constant?

Quoted 48 lines

fs/exec.c +0 -0

 	if (ret)
 		goto out_unlock;

fs/exec.c +0 -0

 	BUG_ON(prev != vma);

include/linux/key.h +2 -0

index e58ee10f6e58..fb8a7d5f6149 100644
@@ -346,6 +346,8 @@ static inline key_serial_t key_serial(const struct key
 *key)
 
 extern void key_set_timeout(struct key *, unsigned);
 
+extern key_ref_t lookup_user_key(key_serial_t id, unsigned long lflags,
+				 key_perm_t perm);
 /*
  * The permissions required on a key that we're looking up.

*/

include/linux/mm.h +0 -0

index ac85c0805761..0f9422c7841e 100644
@@ -1579,7 +1579,8 @@ extern unsigned long change_protection(struct

vm_area_struct *vma, unsigned long

include/linux/mm.h +2 -1

 			      int dirty_accountable, int prot_numa);
 extern int mprotect_fixup(struct vm_area_struct *vma,
 			  struct vm_area_struct **pprev, unsigned long start,
-			  unsigned long end, unsigned long newflags);
+			  unsigned long end, unsigned long newflags,
+			  int newkeyid);
 
 /*
  * doesn't attempt to fault and will return short.

mm/mprotect.c +2 -0

index 56e64ef7931e..6c2e1106525c 100644
@@ -28,14 +28,17 @@
 #include <linux/ksm.h>
 #include <linux/uaccess.h>
 #include <linux/mm_inline.h>
+#include <linux/key.h>
 #include <asm/pgtable.h>
 #include <asm/cacheflush.h>
 #include <asm/mmu_context.h>
 #include <asm/tlbflush.h>
+#include <asm/mktme.h>
 

mm/mprotect.c line 41

Quoted 4 lines

mm/mprotect.c +1 -0

 #include "internal.h"
 
 #define NO_PKEY  -1
+#define NO_KEYID -1

Should have only single named constant IMHO. This ambiguity is worse than some reasonable constant name for both cases. Maybe NO_KEYID would be adequate?

Quoted 21 lines

mm/mprotect.c +2 -1

 static unsigned long change_pte_range(struct vm_area_struct *vma, pmd_t *pmd,
 		unsigned long addr, unsigned long end, pgprot_t newprot,

@@ -310,7 +313,8 @@ unsigned long change_protection(struct vm_area_struct
 *vma, unsigned long start,
 
 int
 mprotect_fixup(struct vm_area_struct *vma, struct vm_area_struct **pprev,
-	unsigned long start, unsigned long end, unsigned long newflags)
+	       unsigned long start, unsigned long end, unsigned long
 newflags,
+	       int newkeyid)
 {

mm/mprotect.c +0 -0

 	struct mm_struct *mm = vma->vm_mm;
 	unsigned long oldflags = vma->vm_flags;

@@ -320,10 +324,24 @@ mprotect_fixup(struct vm_area_struct *vma, struct
 vm_area_struct **pprev,
 	int error;
 	int dirty_accountable = 0;
 

mm/mprotect.c mprotect_fixup(struct vm_area_struct *vma, struct vm_area_struct **pprev, · line 332

Quoted 6 lines

mm/mprotect.c +6 -0

+	/*
+	 * Flags match and Keyids match or we have NO_KEYID.
+	 * This _fixup is usually called from do_mprotect_ext() except
+	 * for one special case: caller fs/exec.c/setup_arg_pages()
+	 * In that case, newkeyid is passed as -1 (NO_KEYID).
+	 */

Quoted 28 lines

mm/mprotect.c +8 -0

+	if (newflags = oldflags &&
+	    (newkeyid = vma_keyid(vma) || newkeyid = NO_KEYID)) {
+		*pprev = vma;
+		return 0;
+	}
+	/* Flags match and Keyid changes */
 	if (newflags = oldflags) {
+		mprotect_set_encrypt(vma, newkeyid);
 		*pprev = vma;
 		return 0;
 	}
+	/* Flags and Keyids both change, continue. */
 
 	/*

mm/mprotect.c +2 -0

 	 * If we make a private mapping writable we increase our commit;

@@ -373,6 +391,8 @@ mprotect_fixup(struct vm_area_struct *vma, struct
 vm_area_struct **pprev,
 	}
 
 success:
+	if (newkeyid != NO_KEYID)
+		mprotect_set_encrypt(vma, newkeyid);
 	/*
 	 * vm_flags and vm_page_prot are protected by the mmap_sem

mm/mprotect.c +0 -0

 	 * held in write mode.

@@ -404,10 +424,15 @@ mprotect_fixup(struct vm_area_struct *vma, struct
 vm_area_struct **pprev,
 }

mm/mprotect.c mprotect_fixup(struct vm_area_struct *vma, struct · line 434

Quoted 10 lines

mm/mprotect.c +6 -1

 
 /*
- * When pkey=NO_PKEY we get legacy mprotect behavior here.
+ * do_mprotect_ext() supports the legacy mprotect behavior plus extensions
+ * for protection keys and memory encryption keys. These extensions are
+ * mutually exclusive and the behavior is:
+ *	(pkey=NO_PKEY && keyid=NO_KEYID) => legacy mprotect
+ *	(pkey is valid)  => legacy mprotect plus protection key extensions
+ *	(keyid is valid) => legacy mprotect plus encryption key extensions
  */

The header does not follow

https://www.kernel.org/doc/Documentation/kernel-doc-nano-HOWTO.txt

Quoted 61 lines

mm/mprotect.c +1 -1

 static int do_mprotect_ext(unsigned long start, size_t len,
-		unsigned long prot, int pkey)
+			   unsigned long prot, int pkey, int keyid)
 {
 	unsigned long nstart, end, tmp, reqprot;

mm/mprotect.c +2 -1

 	struct vm_area_struct *vma, *prev;

@@ -505,7 +530,8 @@ static int do_mprotect_ext(unsigned long start, size_t
 len,
 		tmp = vma->vm_end;
 		if (tmp > end)
 			tmp = end;
-		error = mprotect_fixup(vma, &prev, nstart, tmp, newflags);
+		error = mprotect_fixup(vma, &prev, nstart, tmp, newflags,
+				       keyid);
 		if (error)
 			goto out;

mm/mprotect.c +1 -1

 		nstart = tmp;

@@ -530,7 +556,7 @@ static int do_mprotect_ext(unsigned long start, size_t
 len,
 SYSCALL_DEFINE3(mprotect, unsigned long, start, size_t, len,
 		unsigned long, prot)
 {
-	return do_mprotect_ext(start, len, prot, NO_PKEY);
+	return do_mprotect_ext(start, len, prot, NO_PKEY, NO_KEYID);
 }
 

#ifdef CONFIG_ARCH_HAS_PKEYS

mm/mprotect.c +1 -1

@@ -538,7 +564,7 @@ SYSCALL_DEFINE3(mprotect, unsigned long, start, size_t,
 len,
 SYSCALL_DEFINE4(pkey_mprotect, unsigned long, start, size_t, len,
 		unsigned long, prot, int, pkey)
 {
-	return do_mprotect_ext(start, len, prot, pkey);
+	return do_mprotect_ext(start, len, prot, pkey, NO_KEYID);
 }
 

SYSCALL_DEFINE2(pkey_alloc, unsigned long, flags, unsigned long, init_val)

mm/mprotect.c +19 -0

@@ -587,3 +613,32 @@ SYSCALL_DEFINE1(pkey_free, int, pkey)
 }
 
 #endif /* CONFIG_ARCH_HAS_PKEYS */
+
+#ifdef CONFIG_X86_INTEL_MKTME
+
+SYSCALL_DEFINE4(encrypt_mprotect, unsigned long, start, size_t, len,
+		unsigned long, prot, key_serial_t, serial)
+{
+	key_ref_t key_ref;
+	int ret, keyid;
+
+	/* TODO MKTME key service must be initialized */
+
+	key_ref = lookup_user_key(serial, 0, KEY_NEED_VIEW);
+	if (IS_ERR(key_ref))
+		return PTR_ERR(key_ref);
+
+	mktme_map_lock();
+	keyid = mktme_map_keyid_from_serial(serial);
+	if (!keyid) {
+		mktme_map_unlock();

mm/mprotect.c SYSCALL_DEFINE1(pkey_free, int, pkey) · line 644

Quoted 10 lines

mm/mprotect.c +10 -0

+		key_ref_put(key_ref);
+		return -EINVAL;
+	}
+	ret = do_mprotect_ext(start, len, prot, NO_PKEY, keyid);
+	mktme_map_unlock();
+	key_ref_put(key_ref);
+	return ret;
+}
+
+#endif /* CONFIG_X86_INTEL_MKTME */

/Jarkko

-----Original Message-----
From: [email protected] [mailto:keyrings- [email protected]] On Behalf Of Alison Schofield
Sent: Saturday, September 8, 2018 10:37 AM
To: [email protected]; [email protected]
Cc: Huang, Kai <[email protected]>; Nakajima, Jun <[email protected]>; Shutemov, Kirill <[email protected]>; Hansen, Dave <[email protected]>; Sakkinen, Jarkko <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [RFC 07/12] x86/mm: Add helper functions to track encrypted VMA's

In order to safely manage the usage of memory encryption keys, VMA's using each keyid need to be tracked. This tracking allows the Kernel Key Service to know when the keyid resource is actually in use, or when it is idle and may be considered for reuse.

Define a global atomic encrypt_count array to track the number of VMA's oustanding for each encryption keyid.

Implement helper functions to manipulate this encrypt_count array.

Signed-off-by Alison Schofield <[email protected]>

---
 arch/x86/include/asm/mktme.h |  7 +++++++
 arch/x86/mm/mktme.c          | 39
+++++++++++++++++++++++++++++++++++++++
 include/linux/mm.h           |  2 ++
 3 files changed, 48 insertions(+)

arch/x86/include/asm/mktme.h +6 -0

index b707f800b68f..5f3fa0c39c1c 100644
@@ -16,6 +16,13 @@ extern int mktme_keyid_shift;
 /* Set the encryption keyid bits in a VMA */  extern void
 mprotect_set_encrypt(struct vm_area_struct *vma, int newkeyid);
 
+/* Manage the references to outstanding VMA's per encryption key */
+extern int vma_alloc_encrypt_array(void); extern void
+vma_free_encrypt_array(void); extern int vma_read_encrypt_ref(int
+keyid); extern void vma_get_encrypt_ref(struct vm_area_struct *vma);
+extern void vma_put_encrypt_ref(struct vm_area_struct *vma);
+
 /* Manage mappings between hardware keyids and userspace keys */  extern

int mktme_map_alloc(void); extern void mktme_map_free(void); diff --git a/arch/x86/mm/mktme.c b/arch/x86/mm/mktme.c index 5ee7f37e9cd0..5690ef51a79a 100644

arch/x86/mm/mktme.c +16 -0

@@ -163,6 +163,45 @@ int mktme_map_get_free_keyid(void)
 	return 0;
 }
 
+/*
+ *  Helper functions manage the encrypt_count[] array that tracks the
+ *  VMA's outstanding for each encryption keyid. The gets & puts are
+ *  used in core mm code that allocates and free's VMA's. The alloc,
+ *  free, and read functions are used by the MKTME key service to
+ *  manage key allocation and programming.
+ */
+atomic_t *encrypt_count;
+
+int vma_alloc_encrypt_array(void)
+{
+	encrypt_count = kcalloc(mktme_nr_keyids, sizeof(atomic_t),
 GFP_KERNEL);
+	if (!encrypt_count)
+		return -ENOMEM;
+	return 0;
+}

arch/x86/mm/mktme.c +11 -0

+
+void vma_get_encrypt_ref(struct vm_area_struct *vma) {
+	if (vma_keyid(vma))
+		atomic_inc(&encrypt_count[vma_keyid(vma)]);
+}
+
+void vma_put_encrypt_ref(struct vm_area_struct *vma) {
+	if (vma_keyid(vma))
+		atomic_dec(&encrypt_count[vma_keyid(vma)]);
+}
+
 void prep_encrypted_page(struct page *page, int order, int keyid, bool zero)  {
 	int i;

include/linux/mm.h index renamed from include/linux/mm.h +0 -0

0f9422c7841e..b217c699dbab 100644

include/linux/mm.h +0 -0

@@ -2803,6 +2803,8 @@ static inline void setup_nr_node_ids(void) {}  #ifndef

CONFIG_X86_INTEL_MKTME static inline void mprotect_set_encrypt(struct vm_area_struct *vma,

include/linux/mm.h +2 -1

 					int newkeyid) {}
+static inline void vma_get_encrypt_ref(struct vm_area_struct *vma) {}
+static inline void vma_put_encrypt_ref(struct vm_area_struct *vma) {}
 #endif /* CONFIG_X86_INTEL_MKTME */
 #endif /* __KERNEL__ */
 #endif /* _LINUX_MM_H */
--

2.14.1

Keep track of the VMA's oustanding for each memory encryption keyid. The count is used by the MKTME (Multi-Key Total Memory Encryption) Key Service to determine when it is safe to reprogram a hardware encryption key.

Approach here is to do gets and puts on the encryption reference wherever kmem_cache_alloc/free's of vma_area_cachep's are executed. A couple of these locations will not be hit until cgroup support is added. One of these locations should never hit, so use a VM_WARN_ON.

Signed-off-by Alison Schofield <[email protected]>

---
 arch/x86/mm/mktme.c |  2 ++
 kernel/fork.c       |  2 ++
 mm/mmap.c           | 12 ++++++++++++
 mm/nommu.c          |  4 ++++
 4 files changed, 20 insertions(+)

arch/x86/mm/mktme.c +2 -0

index 5690ef51a79a..8a7c326d4546 100644
@@ -72,10 +72,12 @@ void mprotect_set_encrypt(struct vm_area_struct *vma, int
 newkeyid)
 	if (newkeyid = oldkeyid)
 		return;
 
+	vma_put_encrypt_ref(vma);
 	newprot = pgprot_val(vma->vm_page_prot);
 	newprot &= ~mktme_keyid_mask;
 	newprot |= (unsigned long)newkeyid << mktme_keyid_shift;
 	vma->vm_page_prot = __pgprot(newprot);
+	vma_get_encrypt_ref(vma);
 }
 

/*

kernel/fork.c +2 -0

index e5e7a220a124..2d0e507bde7c 100644
@@ -459,6 +459,7 @@ static __latent_entropy int dup_mmap(struct mm_struct *mm,
 		if (!tmp)
 			goto fail_nomem;
 		*tmp = *mpnt;
+		vma_get_encrypt_ref(tmp);	/* Track encrypted vma's */
 		INIT_LIST_HEAD(&tmp->anon_vma_chain);
 		retval = vma_dup_policy(mpnt, tmp);
 		if (retval)

@@ -539,6 +540,7 @@ static __latent_entropy int dup_mmap(struct mm_struct *mm,
 fail_nomem_anon_vma_fork:
 	mpol_put(vma_policy(tmp));
 fail_nomem_policy:
+	vma_put_encrypt_ref(tmp);		/* Track encrypted vma's */
 	kmem_cache_free(vm_area_cachep, tmp);
 fail_nomem:
 	retval = -ENOMEM;

mm/mmap.c +1 -0

index 4c604eb644b4..7390b8b69fd6 100644
@@ -182,6 +182,7 @@ static struct vm_area_struct *remove_vma(struct
 vm_area_struct *vma)
 	if (vma->vm_file)
 		fput(vma->vm_file);
 	mpol_put(vma_policy(vma));
+	vma_put_encrypt_ref(vma);
 	kmem_cache_free(vm_area_cachep, vma);
 	return next;

}

mm/mmap.c +1 -0

@@ -913,6 +914,7 @@ int __vma_adjust(struct vm_area_struct *vma, unsigned long
 start,
 			anon_vma_merge(vma, next);
 		mm->map_count--;
 		mpol_put(vma_policy(next));
+		vma_put_encrypt_ref(next);
 		kmem_cache_free(vm_area_cachep, next);
 		/*

mm/mmap.c +1 -0

 		 * In mprotect's case 6 (see comments on vma_merge),

@@ -1744,6 +1746,7 @@ unsigned long mmap_region(struct file *file, unsigned
 long addr,
 		goto unacct_error;
 	}
 
+	vma_get_encrypt_ref(vma);
 	vma->vm_mm = mm;
 	vma->vm_start = addr;

mm/mmap.c +1 -0

 	vma->vm_end = addr + len;

@@ -1839,6 +1842,7 @@ unsigned long mmap_region(struct file *file, unsigned
 long addr,
 unmap_and_free_vma:
 	vma->vm_file = NULL;
 	fput(file);
+	vma_put_encrypt_ref(vma);
 
 	/* Undo any partial mapping done by a device driver. */

mm/mmap.c +1 -0

 	unmap_region(mm, vma, prev, vma->vm_start, vma->vm_end);

@@ -2653,6 +2657,7 @@ int __split_vma(struct mm_struct *mm, struct
 vm_area_struct *vma,
 		new->vm_pgoff += ((addr - vma->vm_start) >> PAGE_SHIFT);
 	}
 
+	vma_get_encrypt_ref(new);
 	err = vma_dup_policy(vma, new);
 	if (err)

mm/mmap.c +1 -0

 		goto out_free_vma;

@@ -2686,6 +2691,7 @@ int __split_vma(struct mm_struct *mm, struct
 vm_area_struct *vma,
  out_free_mpol:
 	mpol_put(vma_policy(new));
  out_free_vma:
+	vma_put_encrypt_ref(new);
 	kmem_cache_free(vm_area_cachep, new);
 	return err;

}

mm/mmap.c +0 -0

@@ -3007,6 +3013,7 @@ static int do_brk_flags(unsigned long addr, unsigned

long len, unsigned long fla

mm/mmap.c +2 -0

 		return -ENOMEM;
 	}
 
+	vma_get_encrypt_ref(vma);
 	INIT_LIST_HEAD(&vma->anon_vma_chain);
 	vma->vm_mm = mm;
 	vma->vm_ops = &anon_vm_ops;

@@ -3229,6 +3236,7 @@ struct vm_area_struct *copy_vma(struct vm_area_struct
 **vmap,
 		new_vma->vm_pgoff = pgoff;
 		if (vma_dup_policy(vma, new_vma))
 			goto out_free_vma;
+		vma_get_encrypt_ref(new_vma);
 		INIT_LIST_HEAD(&new_vma->anon_vma_chain);
 		if (anon_vma_clone(new_vma, vma))

mm/mmap.c +1 -0

 			goto out_free_mempol;

@@ -3243,6 +3251,7 @@ struct vm_area_struct *copy_vma(struct vm_area_struct
 **vmap,
 
 out_free_mempol:
 	mpol_put(vma_policy(new_vma));
+	vma_put_encrypt_ref(new_vma);
 out_free_vma:
 	kmem_cache_free(vm_area_cachep, new_vma);

out:

mm/mmap.c +3 -0

@@ -3372,6 +3381,9 @@ static struct vm_area_struct *__install_special_mapping(
 	if (unlikely(vma = NULL))
 		return ERR_PTR(-ENOMEM);
 
+	/* Do not expect a memory encrypted vma here */
+	VM_WARN_ON(vma_keyid(vma));
+
 	INIT_LIST_HEAD(&vma->anon_vma_chain);
 	vma->vm_mm = mm;
 	vma->vm_start = addr;

mm/nommu.c +4 -0

index 73f66e81cfb0..85f04c174638 100644
@@ -769,6 +769,7 @@ static void delete_vma(struct mm_struct *mm, struct
 vm_area_struct *vma)
 	if (vma->vm_file)
 		fput(vma->vm_file);
 	put_nommu_region(vma->vm_region);
+	vma_put_encrypt_ref(vma);
 	kmem_cache_free(vm_area_cachep, vma);
 }

@@ -1215,6 +1216,7 @@ unsigned long do_mmap(struct file *file,
 	if (!vma)
 		goto error_getting_vma;
 
+	vma_get_encrypt_ref(vma);
 	region->vm_usage = 1;
 	region->vm_flags = vm_flags;
 	region->vm_pgoff = pgoff;

@@ -1375,6 +1377,7 @@ unsigned long do_mmap(struct file *file,
 	kmem_cache_free(vm_region_jar, region);
 	if (vma->vm_file)
 		fput(vma->vm_file);
+	vma_put_encrypt_ref(vma);
 	kmem_cache_free(vm_area_cachep, vma);
 	return ret;
 

@@ -1486,6 +1489,7 @@ int split_vma(struct mm_struct *mm, struct
 vm_area_struct *vma,
 	*new = *vma;
 	*region = *vma->vm_region;
 	new->vm_region = region;
+	vma_get_encrypt_ref(new);
 
 	npages = (addr - vma->vm_start) >> PAGE_SHIFT;

Quoted 4 lines · Alison Schofield

Keep track of the VMA's oustanding for each memory encryption keyid. The count is used by the MKTME (Multi-Key Total Memory Encryption) Key Service to determine when it is safe to reprogram a hardware encryption key.

Maybe a stupid question but why they are tracked and what do you mean by tracking?

/Jarkko

Quoted 14 lines

Approach here is to do gets and puts on the encryption reference wherever kmem_cache_alloc/free's of vma_area_cachep's are executed. A couple of these locations will not be hit until cgroup support is added. One of these locations should never hit, so use a VM_WARN_ON.

Signed-off-by Alison Schofield <[email protected]>

---
 arch/x86/mm/mktme.c |  2 ++
 kernel/fork.c       |  2 ++
 mm/mmap.c           | 12 ++++++++++++
 mm/nommu.c          |  4 ++++
 4 files changed, 20 insertions(+)

Quoted 5 lines

+/*
+ * Encrypted mprotect is only supported on anonymous mappings.
+ * All VMA's in the requested range must be anonymous. If this
+ * test fails on any single VMA, the entire mprotect request fails.
+ */

kdoc

Quoted 6 lines · Jarkko

Quoted 5 lines

+/*
+ * Encrypted mprotect is only supported on anonymous mappings.
+ * All VMA's in the requested range must be anonymous. If this
+ * test fails on any single VMA, the entire mprotect request fails.
+ */

kdoc

kdoc what? You want this comment in kdoc format? Why?

Quoted 8 lines · Dave Hansen

Quoted 6 lines · Jarkko

Quoted 5 lines

+/*
+ * Encrypted mprotect is only supported on anonymous mappings.
+ * All VMA's in the requested range must be anonymous. If this
+ * test fails on any single VMA, the entire mprotect request fails.
+ */

kdoc

kdoc what? You want this comment in kdoc format? Why?

If there is a header comment for a function anyway, why wouldn't you put it to kdoc-format?

-----Original Message-----
From: Schofield, Alison
Sent: Saturday, September 8, 2018 10:38 AM
To: [email protected]; [email protected]
Cc: Huang, Kai <[email protected]>; Nakajima, Jun <[email protected]>; Shutemov, Kirill <[email protected]>; Hansen, Dave <[email protected]>; Sakkinen, Jarkko <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: [RFC 10/12] x86/pconfig: Program memory encryption keys on a system-wide basis

The kernel manages the MKTME (Multi-Key Total Memory Encryption) Keys as a system wide single pool of keys. The hardware, however, manages the keys on a per physical package basis. Each physical package maintains a key table that all CPU's in that package share.

In order to maintain the consistent, system wide view that the kernel requires, program all physical packages during a key program request.

Signed-off-by Alison Schofield <[email protected]>

---
 arch/x86/include/asm/intel_pconfig.h | 42
++++++++++++++++++++++++++++++------
 1 file changed, 36 insertions(+), 6 deletions(-)

arch/x86/include/asm/intel_pconfig.h +3 -1

index 3cb002b1d0f9..d3bf0a297e89 100644
@@ -3,6 +3,7 @@
 
 #include <asm/asm.h>
 #include <asm/processor.h>
+#include <linux/cpu.h>
 
 enum pconfig_target {
 	INVALID_TARGET	= 0,

@@ -47,19 +48,48 @@ struct mktme_key_program {
 	u8 key_field_2[64];
 } __packed __aligned(256);
 
-static inline int mktme_key_program(struct mktme_key_program
 *key_program)
+struct mktme_key_program_info {
+	struct mktme_key_program *key_program;

arch/x86/include/asm/intel_pconfig.h +21 -0

+		: "0" (rax), "1" (info->key_program)
+		: "memory", "cc");
+
+	if (rax != MKTME_PROG_SUCCESS)
+		WRITE_ONCE(info->status, rax);
+}
+
+/*
+ * MKTME keys are managed as a system-wide single pool of keys.
+ * In the hardware, each physical package maintains a separate key
+ * table. Program all physical packages with the same key info to
+ * maintain that system-wide kernel view.
+ */
+static inline int mktme_key_program(struct mktme_key_program
 *key_program,
+				    cpumask_var_t mktme_cpumask)
+{
+	struct mktme_key_program_info info = {
+		.key_program = key_program,
+		.status = MKTME_PROG_SUCCESS,
+	};
+
 	if (!pconfig_target_supported(MKTME_TARGET))

arch/x86/include/asm/intel_pconfig.h +1 -1

-	return rax;
+	return info.status;
 }
 

#endif /* _ASM_X86_INTEL_PCONFIG_H */
--
2.14.1