Skip to contentSkip to conversation

x86/sgx: Add SGX_MemTotal to /proc/meminfo

PATCH In review Jarkko Sakkinen 4 participants 7 messages Aug 18, 2021 – Aug 24, 2021

Subsystems (4)
lore
Share & tools
lore

Jarkko Sakkinen

author maintainer
#0 lore raw

PATCH 1/1 x86/sgx: Add SGX_MemTotal to /proc/meminfo

The amount of SGX memory on the system is determined by the BIOS and it varies wildly between systems. It can be from dozens of MB's on desktops or VM's, up to many GB's on servers. Just like for regular memory, it is sometimes useful to know the amount of usable SGX memory in the system.

Add SGX_MemTotal field to /proc/meminfo, which shows the total amount of usable SGX memory in the system. E.g. with 32 MB reserved for SGX from BIOS, the printout would be:

SGX_MemTotal: 22528 kB

It is less than 32 MB because some of the space is reserved for Enclave Page Cache Metadata (EPCM), which contains state variables for all the pages in the Enclave Page Cache (EPC). The latter contains the pages, which applications can use to create enclaves.

---
 Documentation/x86/sgx.rst      |  6 ++++++
 arch/x86/include/asm/sgx.h     | 10 +++++++---
 arch/x86/kernel/cpu/sgx/main.c |  7 ++++++-
 arch/x86/mm/pat/set_memory.c   |  5 +++++
 4 files changed, 24 insertions(+), 4 deletions(-)
Documentation/x86/sgx.rst +6 -0
index dd0ac96ff9ef..68ee171e1d8f 100644
@@ -250,3 +250,9 @@ user wants to deploy SGX applications both on the host and in guests
 on the same machine, the user should reserve enough EPC (by taking out
 total virtual EPC size of all SGX VMs from the physical EPC size) for
 host SGX applications so they can run with acceptable performance.
+
+Supplemental fields for /proc/meminfo
+=====================================
+
+SGX_MemTotal
+	The total usable SGX protected memory in kilobytes.
arch/x86/include/asm/sgx.h +7 -3
index 05f3e21f01a7..2ae9dc8c9411 100644
@@ -365,6 +365,13 @@ struct sgx_sigstruct {
  * comment!
  */
 
+#if defined(CONFIG_X86_SGX) || defined(CONFIG_X86_SGX_KVM)
+extern unsigned long sgx_nr_all_pages;
+
+int sgx_set_attribute(unsigned long *allowed_attributes,
+		      unsigned int attribute_fd);
+#endif
+
 #ifdef CONFIG_X86_SGX_KVM
 int sgx_virt_ecreate(struct sgx_pageinfo *pageinfo, void __user *secs,
 		     int *trapnr);
@@ -372,7 +379,4 @@ int sgx_virt_einit(void __user *sigstruct, void __user *token,
 		   void __user *secs, u64 *lepubkeyhash, int *trapnr);
 #endif
 
-int sgx_set_attribute(unsigned long *allowed_attributes,
-		      unsigned int attribute_fd);
-
 #endif /* _ASM_X86_SGX_H */
arch/x86/kernel/cpu/sgx/main.c +6 -1
index 63d3de02bbcc..1fe26a8e80dc 100644
@@ -28,7 +28,10 @@ static DECLARE_WAIT_QUEUE_HEAD(ksgxd_waitq);
 static LIST_HEAD(sgx_active_page_list);
 static DEFINE_SPINLOCK(sgx_reclaimer_lock);
 
-/* The free page list lock protected variables prepend the lock. */
+/* The number of usable EPC pages in the system. */
+unsigned long sgx_nr_all_pages;
+
+/* The number of free EPC pages in all nodes. */
 static unsigned long sgx_nr_free_pages;
 
 /* Nodes with one or more EPC sections. */
@@ -656,6 +659,8 @@ static bool __init sgx_setup_epc_section(u64 phys_addr, u64 size,
 		list_add_tail(&section->pages[i].list, &sgx_dirty_page_list);
 	}
 
+	sgx_nr_all_pages += nr_pages;
+
 	return true;
 }
 
arch/x86/mm/pat/set_memory.c +5 -0
index ad8a5c586a35..82bb09c298de 100644
@@ -29,6 +29,7 @@
 #include <asm/proto.h>
 #include <asm/memtype.h>
 #include <asm/set_memory.h>
+#include <asm/sgx.h>
 
 #include "../mm_internal.h"
 
@@ -116,6 +117,10 @@ void arch_report_meminfo(struct seq_file *m)
 	if (direct_gbpages)
 		seq_printf(m, "DirectMap1G:    %8lu kB\n",
 			direct_pages_count[PG_LEVEL_1G] << 20);
+
+#if defined(CONFIG_X86_SGX) || defined(CONFIG_X86_SGX_KVM)
+	seq_printf(m, "SGX_MemTotal:   %8lu kB\n", sgx_nr_all_pages << 2);
+#endif
 }
 #else
 static inline void split_page_count(int level) { }
Signature · 1 line
2.25.1
#1 lore raw
Quoted 57 lines · Jarkko Sakkinen

The amount of SGX memory on the system is determined by the BIOS and it varies wildly between systems. It can be from dozens of MB's on desktops or VM's, up to many GB's on servers. Just like for regular memory, it is sometimes useful to know the amount of usable SGX memory in the system.

Add SGX_MemTotal field to /proc/meminfo, which shows the total amount of usable SGX memory in the system. E.g. with 32 MB reserved for SGX from BIOS, the printout would be:

SGX_MemTotal: 22528 kB

It is less than 32 MB because some of the space is reserved for Enclave Page Cache Metadata (EPCM), which contains state variables for all the pages in the Enclave Page Cache (EPC). The latter contains the pages, which applications can use to create enclaves.

---
 Documentation/x86/sgx.rst      |  6 ++++++
 arch/x86/include/asm/sgx.h     | 10 +++++++---
 arch/x86/kernel/cpu/sgx/main.c |  7 ++++++-
 arch/x86/mm/pat/set_memory.c   |  5 +++++
 4 files changed, 24 insertions(+), 4 deletions(-)
Documentation/x86/sgx.rst +6 -0
index dd0ac96ff9ef..68ee171e1d8f 100644
@@ -250,3 +250,9 @@ user wants to deploy SGX applications both on the host and in guests
 on the same machine, the user should reserve enough EPC (by taking out
 total virtual EPC size of all SGX VMs from the physical EPC size) for
 host SGX applications so they can run with acceptable performance.
+
+Supplemental fields for /proc/meminfo
+=====================================
+
+SGX_MemTotal
+	The total usable SGX protected memory in kilobytes.
arch/x86/include/asm/sgx.h +7 -0
index 05f3e21f01a7..2ae9dc8c9411 100644
@@ -365,6 +365,13 @@ struct sgx_sigstruct {
  * comment!
  */
 
+#if defined(CONFIG_X86_SGX) || defined(CONFIG_X86_SGX_KVM)
+extern unsigned long sgx_nr_all_pages;
+
+int sgx_set_attribute(unsigned long *allowed_attributes,
+		      unsigned int attribute_fd);
+#endif
+
 #ifdef CONFIG_X86_SGX_KVM
 int sgx_virt_ecreate(struct sgx_pageinfo *pageinfo, void __user *secs,
 		     int *trapnr);

arch/x86/include/asm/sgx.h int sgx_virt_einit(void __user *sigstruct, void __user *token, · line 377

Quoted 6 lines
arch/x86/include/asm/sgx.h +0 -3
@@ -372,7 +379,4 @@ int sgx_virt_einit(void __user *sigstruct, void __user *token,
 		   void __user *secs, u64 *lepubkeyhash, int *trapnr);
 #endif
 
-int sgx_set_attribute(unsigned long *allowed_attributes,
-		      unsigned int attribute_fd);
-

This change seems unrelated?

Quoted 50 lines
arch/x86/include/asm/sgx.h +0 -0
 #endif /* _ASM_X86_SGX_H */
arch/x86/kernel/cpu/sgx/main.c +6 -1
index 63d3de02bbcc..1fe26a8e80dc 100644
@@ -28,7 +28,10 @@ static DECLARE_WAIT_QUEUE_HEAD(ksgxd_waitq);
 static LIST_HEAD(sgx_active_page_list);
 static DEFINE_SPINLOCK(sgx_reclaimer_lock);
 
-/* The free page list lock protected variables prepend the lock. */
+/* The number of usable EPC pages in the system. */
+unsigned long sgx_nr_all_pages;
+
+/* The number of free EPC pages in all nodes. */
 static unsigned long sgx_nr_free_pages;
 
 /* Nodes with one or more EPC sections. */
@@ -656,6 +659,8 @@ static bool __init sgx_setup_epc_section(u64 phys_addr, u64 size,
 		list_add_tail(&section->pages[i].list, &sgx_dirty_page_list);
 	}
 
+	sgx_nr_all_pages += nr_pages;
+
 	return true;
 }
 
arch/x86/mm/pat/set_memory.c +5 -0
index ad8a5c586a35..82bb09c298de 100644
@@ -29,6 +29,7 @@
 #include <asm/proto.h>
 #include <asm/memtype.h>
 #include <asm/set_memory.h>
+#include <asm/sgx.h>
 
 #include "../mm_internal.h"
 
@@ -116,6 +117,10 @@ void arch_report_meminfo(struct seq_file *m)
 	if (direct_gbpages)
 		seq_printf(m, "DirectMap1G:    %8lu kB\n",
 			direct_pages_count[PG_LEVEL_1G] << 20);
+
+#if defined(CONFIG_X86_SGX) || defined(CONFIG_X86_SGX_KVM)
+	seq_printf(m, "SGX_MemTotal:   %8lu kB\n", sgx_nr_all_pages << 2);
+#endif
 }
 #else
 static inline void split_page_count(int level) { }
author maintainer
#2 lore raw
Quoted 61 lines · Jethro Beekman
Quoted 60 lines · Jarkko Sakkinen

The amount of SGX memory on the system is determined by the BIOS and it varies wildly between systems. It can be from dozens of MB's on desktops or VM's, up to many GB's on servers. Just like for regular memory, it is sometimes useful to know the amount of usable SGX memory in the system.

Add SGX_MemTotal field to /proc/meminfo, which shows the total amount of usable SGX memory in the system. E.g. with 32 MB reserved for SGX from BIOS, the printout would be:

SGX_MemTotal: 22528 kB

It is less than 32 MB because some of the space is reserved for Enclave Page Cache Metadata (EPCM), which contains state variables for all the pages in the Enclave Page Cache (EPC). The latter contains the pages, which applications can use to create enclaves.

---
 Documentation/x86/sgx.rst      |  6 ++++++
 arch/x86/include/asm/sgx.h     | 10 +++++++---
 arch/x86/kernel/cpu/sgx/main.c |  7 ++++++-
 arch/x86/mm/pat/set_memory.c   |  5 +++++
 4 files changed, 24 insertions(+), 4 deletions(-)
Documentation/x86/sgx.rst +6 -0
index dd0ac96ff9ef..68ee171e1d8f 100644
@@ -250,3 +250,9 @@ user wants to deploy SGX applications both on the host and in guests
 on the same machine, the user should reserve enough EPC (by taking out
 total virtual EPC size of all SGX VMs from the physical EPC size) for
 host SGX applications so they can run with acceptable performance.
+
+Supplemental fields for /proc/meminfo
+=====================================
+
+SGX_MemTotal
+	The total usable SGX protected memory in kilobytes.
arch/x86/include/asm/sgx.h +7 -0
index 05f3e21f01a7..2ae9dc8c9411 100644
@@ -365,6 +365,13 @@ struct sgx_sigstruct {
  * comment!
  */
 
+#if defined(CONFIG_X86_SGX) || defined(CONFIG_X86_SGX_KVM)
+extern unsigned long sgx_nr_all_pages;
+
+int sgx_set_attribute(unsigned long *allowed_attributes,
+		      unsigned int attribute_fd);
+#endif
+
 #ifdef CONFIG_X86_SGX_KVM
 int sgx_virt_ecreate(struct sgx_pageinfo *pageinfo, void __user *secs,
 		     int *trapnr);
@@ -372,7 +379,4 @@ int sgx_virt_einit(void __user *sigstruct, void __user *token,
 		   void __user *secs, u64 *lepubkeyhash, int *trapnr);
 #endif
 
Quoted 5 lines

arch/x86/include/asm/sgx.h int sgx_virt_einit(void __user *sigstruct, void __user *token, · line 377

Quoted 3 lines
arch/x86/include/asm/sgx.h +0 -3
-int sgx_set_attribute(unsigned long *allowed_attributes,
-		      unsigned int attribute_fd);
-

This change seems unrelated?

It's just a good practice not to define symbols that do not exist, so that if the symbol is ever used, we get a compilation error, not linking error.

Since this is included to set_memory.c, based on this conclusion, I added the check.

/Jarkko

reviewer
#3 lore raw
Quoted 68 lines · Jarkko Sakkinen
Quoted 61 lines · Jethro Beekman
Quoted 60 lines · Jarkko Sakkinen

The amount of SGX memory on the system is determined by the BIOS and it varies wildly between systems. It can be from dozens of MB's on desktops or VM's, up to many GB's on servers. Just like for regular memory, it is sometimes useful to know the amount of usable SGX memory in the system.

Add SGX_MemTotal field to /proc/meminfo, which shows the total amount of usable SGX memory in the system. E.g. with 32 MB reserved for SGX from BIOS, the printout would be:

SGX_MemTotal: 22528 kB

It is less than 32 MB because some of the space is reserved for Enclave Page Cache Metadata (EPCM), which contains state variables for all the pages in the Enclave Page Cache (EPC). The latter contains the pages, which applications can use to create enclaves.

---
  Documentation/x86/sgx.rst      |  6 ++++++
  arch/x86/include/asm/sgx.h     | 10 +++++++---
  arch/x86/kernel/cpu/sgx/main.c |  7 ++++++-
  arch/x86/mm/pat/set_memory.c   |  5 +++++
  4 files changed, 24 insertions(+), 4 deletions(-)
Documentation/x86/sgx.rst +6 -0
index dd0ac96ff9ef..68ee171e1d8f 100644
@@ -250,3 +250,9 @@ user wants to deploy SGX applications both on the host and in guests
  on the same machine, the user should reserve enough EPC (by taking out
  total virtual EPC size of all SGX VMs from the physical EPC size) for
  host SGX applications so they can run with acceptable performance.
+
+Supplemental fields for /proc/meminfo
+=====================================
+
+SGX_MemTotal
+	The total usable SGX protected memory in kilobytes.
arch/x86/include/asm/sgx.h +7 -0
index 05f3e21f01a7..2ae9dc8c9411 100644
@@ -365,6 +365,13 @@ struct sgx_sigstruct {
   * comment!
   */
 
+#if defined(CONFIG_X86_SGX) || defined(CONFIG_X86_SGX_KVM)
+extern unsigned long sgx_nr_all_pages;
+
+int sgx_set_attribute(unsigned long *allowed_attributes,
+		      unsigned int attribute_fd);
+#endif
+
  #ifdef CONFIG_X86_SGX_KVM
  int sgx_virt_ecreate(struct sgx_pageinfo *pageinfo, void __user *secs,
  		     int *trapnr);
@@ -372,7 +379,4 @@ int sgx_virt_einit(void __user *sigstruct, void __user *token,
  		   void __user *secs, u64 *lepubkeyhash, int *trapnr);
  #endif
 
Quoted 5 lines

arch/x86/include/asm/sgx.h int sgx_virt_einit(void __user *sigstruct, void __user *token, · line 377

Quoted 3 lines
arch/x86/include/asm/sgx.h +0 -3
-int sgx_set_attribute(unsigned long *allowed_attributes,
-		      unsigned int attribute_fd);
-

This change seems unrelated?

Quoted 6 lines

It's just a good practice not to define symbols that do not exist, so that if the symbol is ever used, we get a compilation error, not linking error.

Since this is included to set_memory.c, based on this conclusion, I added the check.

It would make sense to make this change in a separate patch since.

thanks,
-- Shuah

author maintainer
#4 lore raw
Quoted 97 lines · Shuah Khan
Quoted 86 lines · Jarkko Sakkinen
Quoted 79 lines · Jethro Beekman
Quoted 78 lines · Jarkko Sakkinen

The amount of SGX memory on the system is determined by the BIOS and it
varies wildly between systems. It can be from dozens of MB's on desktops
or VM's, up to many GB's on servers. Just like for regular memory, it is
sometimes useful to know the amount of usable SGX memory in the system.

Add SGX_MemTotal field to /proc/meminfo, which shows the total amount of
usable SGX memory in the system. E.g. with 32 MB reserved for SGX from
BIOS, the printout would be:

SGX_MemTotal: 22528 kB

It is less than 32 MB because some of the space is reserved for Enclave
Page Cache Metadata (EPCM), which contains state variables for all the
pages in the Enclave Page Cache (EPC). The latter contains the pages,
which applications can use to create enclaves.

---
  Documentation/x86/sgx.rst      |  6 ++++++
  arch/x86/include/asm/sgx.h     | 10 +++++++---
  arch/x86/kernel/cpu/sgx/main.c |  7 ++++++-
  arch/x86/mm/pat/set_memory.c   |  5 +++++
  4 files changed, 24 insertions(+), 4 deletions(-)
Documentation/x86/sgx.rst +0 -0
index dd0ac96ff9ef..68ee171e1d8f 100644
@@ -250,3 +250,9 @@ user wants to deploy SGX applications both

on the host and in guests
on the same machine, the user should reserve enough EPC (by
taking out
total virtual EPC size of all SGX VMs from the physical EPC
size) for
host SGX applications so they can run with acceptable
performance.

Documentation/x86/sgx.rst +6 -0
+
+Supplemental fields for /proc/meminfo
+=====================================
+
+SGX_MemTotal
+	The total usable SGX protected memory in kilobytes.
arch/x86/include/asm/sgx.h +7 -0
index 05f3e21f01a7..2ae9dc8c9411 100644
@@ -365,6 +365,13 @@ struct sgx_sigstruct {
   * comment!
   */
 
+#if defined(CONFIG_X86_SGX) || defined(CONFIG_X86_SGX_KVM)
+extern unsigned long sgx_nr_all_pages;
+
+int sgx_set_attribute(unsigned long *allowed_attributes,
+		      unsigned int attribute_fd);
+#endif
+
  #ifdef CONFIG_X86_SGX_KVM
  int sgx_virt_ecreate(struct sgx_pageinfo *pageinfo, void
 __user *secs,

int *trapnr);

arch/x86/include/asm/sgx.h +0 -0
@@ -372,7 +379,4 @@ int sgx_virt_einit(void __user *sigstruct,
 void __user *token,
  		   void __user *secs, u64 *lepubkeyhash, int
 *trapnr);
  #endif
Quoted 5 lines

arch/x86/include/asm/sgx.h int sgx_virt_einit(void __user *sigstruct, void __user *token, · line 377

Quoted 3 lines
arch/x86/include/asm/sgx.h +0 -3
-int sgx_set_attribute(unsigned long *allowed_attributes,
-		      unsigned int attribute_fd);
-

This change seems unrelated?

Quoted 9 lines

It's just a good practice not to define symbols that do not exist, so that
if the symbol is ever used, we get a compilation error, not linking error.

Since this is included to set_memory.c, based on this conclusion, I added
the check.

Quoted 4 lines

It would make sense to make this change in a separate patch since.

thanks,
-- Shuah

NP, I can split it.

/Jarkko

#5 lore raw
In reply to Jarkko Sakkinen
Quoted 10 lines · Jarkko Sakkinen

The amount of SGX memory on the system is determined by the BIOS and it varies wildly between systems. It can be from dozens of MB's on desktops or VM's, up to many GB's on servers. Just like for regular memory, it is sometimes useful to know the amount of usable SGX memory in the system.

Add SGX_MemTotal field to /proc/meminfo, which shows the total amount of usable SGX memory in the system. E.g. with 32 MB reserved for SGX from BIOS, the printout would be:

SGX_MemTotal: 22528 kB

The big question here: Do we want to put purely architecture-specific entries in (the currently quite arch-independent) /proc/meminfo?

The current "DirectMap4k/2M/1G" entries from arch_report_meminfo() are arch-specific in their sizes, of course, but the concept is at least quite arch-independent.

author maintainer
#6 lore raw
Quoted 10 lines · Dave Hansen
Quoted 9 lines · Jarkko Sakkinen

The amount of SGX memory on the system is determined by the BIOS and it varies wildly between systems. It can be from dozens of MB's on desktops or VM's, up to many GB's on servers. Just like for regular memory, it is sometimes useful to know the amount of usable SGX memory in the system.

Add SGX_MemTotal field to /proc/meminfo, which shows the total amount of usable SGX memory in the system. E.g. with 32 MB reserved for SGX from BIOS, the printout would be:

Quoted 8 lines
Quoted 1 line

SGX_MemTotal: 22528 kB

The big question here: Do we want to put purely architecture-specific entries in (the currently quite arch-independent) /proc/meminfo?

The current "DirectMap4k/2M/1G" entries from arch_report_meminfo() are arch-specific in their sizes, of course, but the concept is at least quite arch-independent.

I started with /proc/meminfo instead of /sys/devices/system/node/*/meminfo because there is pre-existing arch callback, and secondary because it's also better fit for my kselftest code.

The same discussion still applies to both files, they probably should follow a matching pattern.

/Jarkko