Add an oversubscription test
Subsystems (4)
Jarkko Sakkinen
maintainerv4 cover Add an oversubscription test
The goal of these patches is to add a test case for a SGX reserved memory oversubscription, i.e. make sure that the page reclaimer and and the page fault handler are working correctly.
Change Log
==========
v3:
* Reorganized the patch set into smaller pieces, and refactored the code so that the test enclave can be created inside each test case. Added a new test case unclobbered_vdso_oversubscribed that creates a large enough heap to fill all of the available SGX reserved memory (EPC).
Jarkko Sakkinen (8):
x86/sgx: Add /sys/kernel/debug/x86/sgx_total_mem selftests/sgx: Assign source for each segment selftests/sgx: Make data measurement for an enclave segment optional selftests/sgx: Create a heap for the test enclave selftests/sgx: Dump segments and /proc/self/maps only on failure selftests/sgx: Encpsulate the test enclave creation selftests/sgx: Move setup_test_encl() to each TEST_F() selftests/sgx: Add a new kselftest: unclobbered_vdso_oversubscribed
Documentation/x86/sgx.rst | 6 ++ arch/x86/kernel/cpu/sgx/main.c | 10 +- tools/testing/selftests/sgx/load.c | 40 ++++++-- tools/testing/selftests/sgx/main.c | 123 +++++++++++++++++++----- tools/testing/selftests/sgx/main.h | 7 +- tools/testing/selftests/sgx/sigstruct.c | 12 ++- 6 files changed, 159 insertions(+), 39 deletions(-)
Signature · 1 line
2.32.0
Jarkko Sakkinen
maintainerPATCH v4 1/8 x86/sgx: Add /sys/kernel/debug/x86/sgx_total_mem
Just like normal memory, SGX memory can be overcommitted. SGX has its own reclaim mechanism which kicks in when physical SGX memory (Enclave Page Cache / EPC) is exhausted. That reclaim mechanism is relatively rarely exercised and needs selftests to poke at it.
The amount of EPC on the system is determined by the BIOS and it varies wildly between systems. It can be dozens of MB on desktops, or many GB on servers.
To run in a reasonable amount of time, the selftest needs to know how much EPC there is in the system.
Introduce a new debugfs file to export that information.
---
v3:
* Describe the units of sgx_total_mem in Dcumentation/x86/sgx.rst.
* Rewrite of the commit message (suggested by Dave):
https://lore.kernel.org/linux-sgx/[email protected]/
v2:
* sgx_nr_all_pages -> sgx_total_mem
Documentation/x86/sgx.rst | 6 ++++++ arch/x86/kernel/cpu/sgx/main.c | 10 +++++++++- 2 files changed, 15 insertions(+), 1 deletion(-)
Documentation/x86/sgx.rst +6 -0
@@ -250,3 +250,9 @@ user wants to deploy SGX applications both on the host and in guests on the same machine, the user should reserve enough EPC (by taking out total virtual EPC size of all SGX VMs from the physical EPC size) for host SGX applications so they can run with acceptable performance. + +Debugging +========= + +*/sys/kernel/debug/x86/sgx_total_mem* contains an integer describing +the total SGX reserved memory in bytes, available in the system.
arch/x86/kernel/cpu/sgx/main.c +9 -1
@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0 /* Copyright(c) 2016-20 Intel Corporation. */ +#include <linux/debugfs.h> #include <linux/file.h> #include <linux/freezer.h> #include <linux/highmem.h>
@@ -28,7 +29,10 @@ static DECLARE_WAIT_QUEUE_HEAD(ksgxd_waitq);static LIST_HEAD (sgx_active_page_list);static DEFINE_SPINLOCK (sgx_reclaimer_lock); -/* The free page list lock protected variables prepend the lock. */ +/* Total EPC memory available in bytes. */ +static unsigned long sgx_total_mem; + +/* The number of free EPC pages in all nodes. */ static unsigned long sgx_nr_free_pages;/* Nodes with one or more EPC sections. */
@@ -656,6 +660,8 @@ static bool __init sgx_setup_epc_section(u64 phys_addr, u64 size,list_add_tail (§ion->pages[i].list, &sgx_dirty_page_list); } + sgx_total_mem += nr_pages *PAGE_SIZE ; +return true; }
@@ -823,6 +829,8 @@ static int __init sgx_init(void)if (sgx_vepc_init () && ret) goto err_provision; +debugfs_create_ulong ("sgx_total_mem" ,0444 , arch_debugfs_dir, &sgx_total_mem); +return 0 ; err_provision:
Signature · 1 line
2.32.0
Shuah Khan
maintainerRe: [PATCH v4 1/8] x86/sgx: Add /sys/kernel/debug/x86/sgx_total_mem
Quoted 35 lines · Jarkko Sakkinen
Just like normal memory, SGX memory can be overcommitted. SGX has its own reclaim mechanism which kicks in when physical SGX memory (Enclave Page Cache / EPC) is exhausted. That reclaim mechanism is relatively rarely exercised and needs selftests to poke at it.
The amount of EPC on the system is determined by the BIOS and it varies wildly between systems. It can be dozens of MB on desktops, or many GB on servers.
To run in a reasonable amount of time, the selftest needs to know how much EPC there is in the system.
Introduce a new debugfs file to export that information.
Acked-by Dave Hansen <[email protected]>Signed-off-by Jarkko Sakkinen <[email protected]>---
v3:
* Describe the units ofsgx_total_meminDcumentation/x86/sgx.rst.
* Rewrite of the commit message (suggested by Dave):https://lore.kernel.org/linux-sgx/[email protected]/v2:
*sgx_nr_all_pages->sgx_total_memDocumentation/x86/sgx.rst | 6 ++++++ arch/x86/kernel/cpu/sgx/main.c | 10 +++++++++- 2 files changed, 15 insertions(+), 1 deletion(-)Documentation/x86/sgx.rst +0
-0
Documentation/x86/sgx.rst user wants to deploy SGX applications both on the host and in guests · line 258
Quoted 9 lines
Documentation/x86/sgx.rst +6
-0@@ -250,3 +250,9 @@ user wants to deploy SGX applications both on the host and in guests on the same machine, the user should reserve enough EPC (by taking out total virtual EPC size of all SGX VMs from the physical EPC size) for host SGX applications so they can run with acceptable performance. + +Debugging +========= + +*/sys/kernel/debug/x86/sgx_total_mem* contains an integer describing +the total SGX reserved memory in bytes, available in the system.
Why not add the details you have in the commit log to the document? Also add more details on how to debug/test.
Quoted 35 lines
arch/x86/kernel/cpu/sgx/main.c +7
-1@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0 /* Copyright(c) 2016-20 Intel Corporation. */ +#include <linux/debugfs.h> #include <linux/file.h> #include <linux/freezer.h> #include <linux/highmem.h> @@ -28,7 +29,10 @@ static DECLARE_WAIT_QUEUE_HEAD(ksgxd_waitq);static LIST_HEAD (sgx_active_page_list);static DEFINE_SPINLOCK (sgx_reclaimer_lock); -/* The free page list lock protected variables prepend the lock. */ +/* Total EPC memory available in bytes. */ +static unsigned long sgx_total_mem; + +/* The number of free EPC pages in all nodes. */ static unsigned long sgx_nr_free_pages;/* Nodes with one or more EPC sections. */ @@ -656,6 +660,8 @@ static bool __init sgx_setup_epc_section(u64 phys_addr, u64 size,list_add_tail (§ion->pages[i].list, &sgx_dirty_page_list); } + sgx_total_mem += nr_pages *PAGE_SIZE ; +return true; }
arch/x86/kernel/cpu/sgx/main.c static int __init sgx_init(void) · line 836
Quoted 8 lines
arch/x86/kernel/cpu/sgx/main.c +2
-0@@ -823,6 +829,8 @@ static int __init sgx_init(void)if (sgx_vepc_init () && ret) goto err_provision; +debugfs_create_ulong ("sgx_total_mem" ,0444 , arch_debugfs_dir, &sgx_total_mem); +return 0 ; err_provision:
thanks,
-- Shuah
Jarkko Sakkinen
maintainerRe: [PATCH v4 1/8] x86/sgx: Add /sys/kernel/debug/x86/sgx_total_mem
Quoted 2 lines · Shuah Khan
Why not add the details you have in the commit log to the document? Also add more details on how to debug/test.
Agreed, we can re-use it, thanks for the remark. I'll extend it, but it remains to be seen where exactly it will be documented.
I've started to manage this patch as a separate cycle:
https://lore.kernel.org/linux-sgx/[email protected]/T/#u
I'll CC you the 1st non-RFC version, and hold on with the patch set under discussion up until we get the kernel change first to the mainline.
Should happen within couple of days as I think we got quite well sorted out with Dave last week, what to do next.
/Jarkko
Shuah Khan
maintainerRe: [PATCH v4 1/8] x86/sgx: Add /sys/kernel/debug/x86/sgx_total_mem
In reply to Jarkko SakkinenQuoted 78 lines · Jarkko Sakkinen
Just like normal memory, SGX memory can be overcommitted. SGX has its own reclaim mechanism which kicks in when physical SGX memory (Enclave Page Cache / EPC) is exhausted. That reclaim mechanism is relatively rarely exercised and needs selftests to poke at it.
The amount of EPC on the system is determined by the BIOS and it varies wildly between systems. It can be dozens of MB on desktops, or many GB on servers.
To run in a reasonable amount of time, the selftest needs to know how much EPC there is in the system.
Introduce a new debugfs file to export that information.
Acked-by Dave Hansen <[email protected]>Signed-off-by Jarkko Sakkinen <[email protected]>---
v3:
* Describe the units ofsgx_total_meminDcumentation/x86/sgx.rst.
* Rewrite of the commit message (suggested by Dave):https://lore.kernel.org/linux-sgx/[email protected]/v2:
*sgx_nr_all_pages->sgx_total_memDocumentation/x86/sgx.rst | 6 ++++++ arch/x86/kernel/cpu/sgx/main.c | 10 +++++++++- 2 files changed, 15 insertions(+), 1 deletion(-)Documentation/x86/sgx.rst +6
-0@@ -250,3 +250,9 @@ user wants to deploy SGX applications both on the host and in guests on the same machine, the user should reserve enough EPC (by taking out total virtual EPC size of all SGX VMs from the physical EPC size) for host SGX applications so they can run with acceptable performance. + +Debugging +========= + +*/sys/kernel/debug/x86/sgx_total_mem* contains an integer describing +the total SGX reserved memory in bytes, available in the system.arch/x86/kernel/cpu/sgx/main.c +7
-1@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0 /* Copyright(c) 2016-20 Intel Corporation. */ +#include <linux/debugfs.h> #include <linux/file.h> #include <linux/freezer.h> #include <linux/highmem.h> @@ -28,7 +29,10 @@ static DECLARE_WAIT_QUEUE_HEAD(ksgxd_waitq);static LIST_HEAD (sgx_active_page_list);static DEFINE_SPINLOCK (sgx_reclaimer_lock); -/* The free page list lock protected variables prepend the lock. */ +/* Total EPC memory available in bytes. */ +static unsigned long sgx_total_mem; + +/* The number of free EPC pages in all nodes. */ static unsigned long sgx_nr_free_pages;/* Nodes with one or more EPC sections. */ @@ -656,6 +660,8 @@ static bool __init sgx_setup_epc_section(u64 phys_addr, u64 size,list_add_tail (§ion->pages[i].list, &sgx_dirty_page_list); } + sgx_total_mem += nr_pages *PAGE_SIZE ; +return true; }
arch/x86/kernel/cpu/sgx/main.c static int __init sgx_init(void) · line 832
Quoted 4 lines
arch/x86/kernel/cpu/sgx/main.c +1
-0@@ -823,6 +829,8 @@ static int __init sgx_init(void)if (sgx_vepc_init () && ret) goto err_provision; +debugfs_create_ulong ("sgx_total_mem" ,0444 , arch_debugfs_dir, &sgx_total_mem);
dbugfs init dependency would be met I assume by this time?
Quoted 5 lines
arch/x86/kernel/cpu/sgx/main.c +1
-0+return 0 ; err_provision:
Jarkko Sakkinen
maintainerPATCH v4 2/8 selftests/sgx: Assign source for each segment
In reply to Jarkko SakkinenDefine source per segment so that enclave pages can be added from different sources, e.g. anonymous VMA for zero pages. In other words, add 'src' field to struct encl_segment, and assign it to 'encl->src' for pages inherited from the enclave binary.
--- tools/testing/selftests/sgx/load.c | 5 +++-- tools/testing/selftests/sgx/main.h | 1 + tools/testing/selftests/sgx/sigstruct.c | 8 ++++---- 3 files changed, 8 insertions(+), 6 deletions(-)
tools/testing/selftests/sgx/load.c +3 -2
@@ -107,7 +107,7 @@ static bool encl_ioc_add_pages(struct encl *encl, struct encl_segment *seg)memset (&secinfo,0 ,sizeof (secinfo)); secinfo.flags = seg->flags; - ioc.src = (uint64_t )encl->src + seg->offset; + ioc.src = (uint64_t )seg->src; ioc.offset = seg->offset; ioc.length = seg->size; ioc.secinfo = (unsigned long )&secinfo;
@@ -216,6 +216,7 @@ bool encl_load(const char *path, struct encl *encl)if (j ==0 ) { src_offset = phdr->p_offset &PAGE_MASK ; + encl->src = encl->bin + src_offset; seg->prot =PROT_READ |PROT_WRITE ; seg->flags =SGX_PAGE_TYPE_TCS <<8 ;
@@ -228,13 +229,13 @@ bool encl_load(const char *path, struct encl *encl) seg->offset = (phdr->p_offset &PAGE_MASK ) - src_offset; seg->size = (phdr->p_filesz +PAGE_SIZE -1 ) &PAGE_MASK ; + seg->src = encl->src + seg->offset; j++; }assert (j == encl->nr_segments); - encl->src = encl->bin + src_offset; encl->src_size = encl->segment_tbl[j -1 ].offset + encl->segment_tbl[j -1 ].size;
tools/testing/selftests/sgx/main.h +1 -0
@@ -7,6 +7,7 @@#define MAIN_H struct encl_segment { +void *src;off_t offset;size_t size;unsigned int prot;
tools/testing/selftests/sgx/sigstruct.c +4 -4
@@ -289,14 +289,14 @@ static bool mrenclave_eextend(EVP_MD_CTX *ctx, uint64_t offset,static bool mrenclave_segment (EVP_MD_CTX *ctx,struct encl *encl,struct encl_segment *seg) { -uint64_t end = seg->offset + seg->size; +uint64_t end = seg->size;uint64_t offset; -for (offset = seg->offset; offset < end; offset +=PAGE_SIZE ) { -if (!mrenclave_eadd (ctx, offset, seg->flags)) +for (offset =0 ; offset < end; offset +=PAGE_SIZE ) { +if (!mrenclave_eadd (ctx, seg->offset + offset, seg->flags))return false; -if (!mrenclave_eextend (ctx, offset, encl->src + offset)) +if (!mrenclave_eextend (ctx, seg->offset + offset, seg->src + offset))return false; }
Signature · 1 line
2.32.0
Jarkko Sakkinen
maintainerPATCH v4 3/8 selftests/sgx: Make data measurement for an enclave segment optional
In reply to Jarkko SakkinenFor a heap makes sense to leave its contents "unmeasured" in the SGX enclave build process, meaning that they won't contribute to the cryptographic signature (a RSA-3072 signed SHA56 hash) of the enclave.
Enclaves are signed blobs where the signature is calculated both from page data and also from "structural properties" of the pages. For instance a page offset of *every* page added to the enclave is hashed.
For data, this is optional, not least because hashing a page has a significant contribution to the enclave load time. Thus, where there is no reason to hash, do not. The SGX ioctl interface supports this with SGX_PAGE_MEASURE flag. Only when the flag is *set*, data is measured.
Add seg->measure boolean flag to struct encl_segment. Only when the flag is set, include the segment data to the signature (represented by SIGSTRUCT architectural structure).
--- tools/testing/selftests/sgx/load.c | 6 +++++- tools/testing/selftests/sgx/main.h | 1 + tools/testing/selftests/sgx/sigstruct.c | 6 ++++-- 3 files changed, 10 insertions(+), 3 deletions(-)
tools/testing/selftests/sgx/load.c +5 -1
@@ -111,7 +111,10 @@ static bool encl_ioc_add_pages(struct encl *encl, struct encl_segment *seg) ioc.offset = seg->offset; ioc.length = seg->size; ioc.secinfo = (unsigned long )&secinfo; - ioc.flags =SGX_PAGE_MEASURE ; +if (seg->measure) + ioc.flags =SGX_PAGE_MEASURE ; +else + ioc.flags =0 ; rc =ioctl (encl->fd,SGX_IOC_ENCLAVE_ADD_PAGES , &ioc);if (rc <0 ) {
@@ -230,6 +233,7 @@ bool encl_load(const char *path, struct encl *encl) seg->offset = (phdr->p_offset &PAGE_MASK ) - src_offset; seg->size = (phdr->p_filesz +PAGE_SIZE -1 ) &PAGE_MASK ; seg->src = encl->src + seg->offset; + seg->measure = true; j++; }
tools/testing/selftests/sgx/main.h +1 -0
@@ -12,6 +12,7 @@ struct encl_segment {size_t size;unsigned int prot;unsigned int flags; +bool measure; };struct encl {
tools/testing/selftests/sgx/sigstruct.c +4 -2
@@ -296,8 +296,10 @@ static bool mrenclave_segment(EVP_MD_CTX *ctx, struct encl *encl,if (!mrenclave_eadd (ctx, seg->offset + offset, seg->flags))return false; -if (!mrenclave_eextend (ctx, seg->offset + offset, seg->src + offset)) -return false; +if (seg->measure) { +if (!mrenclave_eextend (ctx, seg->offset + offset, seg->src + offset)) +return false; + } }return true;
Signature · 1 line
2.32.0
Jarkko Sakkinen
maintainerPATCH v4 4/8 selftests/sgx: Create a heap for the test enclave
In reply to Jarkko SakkinenCreate a heap for the test enclave, which is allocated from /dev/null, and left unmeasured. This is beneficial by its own because it verifies that an enclave built from multiple choices, works properly. If LSM hooks are added for SGX some day, a multi source enclave has higher probability to trigger bugs on access control checks.
The immediate need comes from the need to implement page reclaim tests. In order to trigger the page reclaimer, one can just set the size of the heap to high enough.
--- tools/testing/selftests/sgx/load.c | 29 ++++++++++++++++++++++------- tools/testing/selftests/sgx/main.c | 2 +- tools/testing/selftests/sgx/main.h | 4 +++- 3 files changed, 26 insertions(+), 9 deletions(-)
tools/testing/selftests/sgx/load.c +22 -7
@@ -21,6 +21,8 @@void encl_delete (struct encl *encl) { +struct encl_segment *heap_seg = &encl->segment_tbl[encl->nr_segments -1 ]; +if (encl->encl_base)munmap ((void *)encl->encl_base, encl->encl_size);
@@ -30,6 +32,8 @@ void encl_delete(struct encl *encl)if (encl->fd)close (encl->fd); +munmap (heap_seg->src, heap_seg->size); +if (encl->segment_tbl)free (encl->segment_tbl);
@@ -125,11 +129,10 @@ static bool encl_ioc_add_pages(struct encl *encl, struct encl_segment *seg)return true; } - - -bool encl_load (const char *path,struct encl *encl) +bool encl_load (const char *path,struct encl *encl,unsigned long heap_size) {const char device_path[] ="/dev/sgx_enclave" ; +struct encl_segment *seg;Elf64_Phdr *phdr_tbl;off_t src_offset;Elf64_Ehdr *ehdr;
@@ -181,6 +184,8 @@ bool encl_load(const char *path, struct encl *encl) ehdr = encl->bin; phdr_tbl = encl->bin + ehdr->e_phoff; + encl->nr_segments =1 ;/* one for the heap */ +for (i =0 ; i < ehdr->e_phnum; i++) {Elf64_Phdr *phdr = &phdr_tbl[i];
@@ -196,7 +201,6 @@ bool encl_load(const char *path, struct encl *encl)for (i =0 , j =0 ; i < ehdr->e_phnum; i++) {Elf64_Phdr *phdr = &phdr_tbl[i];unsigned int flags = phdr->p_flags; -struct encl_segment *seg;if (phdr->p_type !=PT_LOAD )continue ;
@@ -238,10 +242,21 @@ bool encl_load(const char *path, struct encl *encl) j++; } -assert (j == encl->nr_segments); +assert (j == encl->nr_segments -1 ); + + seg = &encl->segment_tbl[j]; + seg->offset = encl->segment_tbl[j -1 ].offset + encl->segment_tbl[j -1 ].size; + seg->size = heap_size; + seg->src =mmap (NULL , heap_size,PROT_READ |PROT_WRITE , +MAP_ANONYMOUS |MAP_PRIVATE ,-1 ,0 ); + seg->prot =PROT_READ |PROT_WRITE ; + seg->flags = (SGX_PAGE_TYPE_REG <<8 ) | seg->prot; + seg->measure = false; + +if (seg->src ==MAP_FAILED ) + goto err; - encl->src_size = encl->segment_tbl[j -1 ].offset + - encl->segment_tbl[j -1 ].size; + encl->src_size = encl->segment_tbl[j].offset + encl->segment_tbl[j].size;for (encl->encl_size =4096 ; encl->encl_size < encl->src_size; ) encl->encl_size <<=1 ;
tools/testing/selftests/sgx/main.c +1 -1
@@ -122,7 +122,7 @@ FIXTURE_SETUP(enclave)unsigned int i;void *addr; -if (!encl_load ("test_encl.elf" , &self->encl)) { +if (!encl_load ("test_encl.elf" , &self->encl,ENCL_HEAP_SIZE_DEFAULT )) {encl_delete (&self->encl);ksft_exit_skip ("cannot load enclaves\n" ); }
tools/testing/selftests/sgx/main.h +3 -1
@@ -6,6 +6,8 @@ #ifndef MAIN_H #define MAIN_H +#define ENCL_HEAP_SIZE_DEFAULT 4096 + struct encl_segment { void *src; off_t offset;
@@ -33,7 +35,7 @@ extern unsigned char sign_key[];extern unsigned char sign_key_end[];void encl_delete (struct encl *ctx); -bool encl_load (const char *path,struct encl *encl); +bool encl_load (const char *path,struct encl *encl,unsigned long heap_size);bool encl_measure (struct encl *encl);bool encl_build (struct encl *encl);
Signature · 1 line
2.32.0
Jarkko Sakkinen
maintainerPATCH v4 5/8 selftests/sgx: Dump segments and /proc/self/maps only on failure
In reply to Jarkko SakkinenLogging is always a compromise between clarity and detail. The main use case for dumping VMA's is when FIXTURE_SETUP() fails, and is less important for enclaves that do initialize correctly. Therefore, print the segments and /proc/self/maps only in the error case.
Finally, if a single test ever creates multiple enclaves, the amount of log lines would become enormous.
--- tools/testing/selftests/sgx/main.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-)
tools/testing/selftests/sgx/main.c +12 -11
@@ -127,12 +127,6 @@ FIXTURE_SETUP(enclave)ksft_exit_skip ("cannot load enclaves\n" ); } -for (i =0 ; i < self->encl.nr_segments; i++) { - seg = &self->encl.segment_tbl[i]; - -TH_LOG ("0x%016lx 0x%016lx 0x%02x" , seg->offset, seg->size, seg->prot); - } -if (!encl_measure (&self->encl)) goto err;
@@ -169,6 +163,17 @@ FIXTURE_SETUP(enclave)memset (&self->run,0 ,sizeof (self->run)); self->run.tcs = self->encl.encl_base; +return ; + +err: +encl_delete (&self->encl); + +for (i =0 ; i < self->encl.nr_segments; i++) { + seg = &self->encl.segment_tbl[i]; + +TH_LOG ("0x%016lx 0x%016lx 0x%02x" , seg->offset, seg->size, seg->prot); + } + maps_file =fopen ("/proc/self/maps" ,"r" );if (maps_file !=NULL ) {while (fgets (maps_line,sizeof (maps_line), maps_file) !=NULL ) {
@@ -181,11 +186,7 @@ FIXTURE_SETUP(enclave)fclose (maps_file); } -err: -if (!sgx_enter_enclave_sym) -encl_delete (&self->encl); - -ASSERT_NE (sgx_enter_enclave_sym,NULL ); +ASSERT_TRUE (false); }FIXTURE_TEARDOWN (enclave)
Signature · 1 line
2.32.0
Jarkko Sakkinen
maintainerPATCH v4 6/8 selftests/sgx: Encpsulate the test enclave creation
In reply to Jarkko SakkinenIntroduce setup_test_encl() so that the enclave creation can be moved to TEST_F()'s. This is required for a reclaimer test where the heap size needs to be set large enough to triger the page reclaimer.
---
v4:
* Wrap setup_test_encl() with ASSERT_TRUE().
* Update commit message.
tools/testing/selftests/sgx/main.c | 44 ++++++++++++++++++------------ 1 file changed, 26 insertions(+), 18 deletions(-)
tools/testing/selftests/sgx/main.c +26 -18
@@ -112,7 +112,8 @@ FIXTURE(enclave) {struct sgx_enclave_run run; }; -FIXTURE_SETUP (enclave) +static bool setup_test_encl (unsigned long heap_size,struct encl *encl, +struct __test_metadata *_metadata) {Elf64_Sym *sgx_enter_enclave_sym =NULL ;struct vdso_symtab symtab;
@@ -122,25 +123,25 @@ FIXTURE_SETUP(enclave)unsigned int i;void *addr; -if (!encl_load ("test_encl.elf" , &self->encl,ENCL_HEAP_SIZE_DEFAULT )) { -encl_delete (&self->encl); -ksft_exit_skip ("cannot load enclaves\n" ); +if (!encl_load ("test_encl.elf" , encl, heap_size)) { +encl_delete (encl); +TH_LOG ("Failed to load the test enclave.\n" ); } -if (!encl_measure (&self->encl)) +if (!encl_measure (encl)) goto err; -if (!encl_build (&self->encl)) +if (!encl_build (encl)) goto err;/* * An enclave consumer only must do this. */ -for (i =0 ; i < self->encl.nr_segments; i++) { -struct encl_segment *seg = &self->encl.segment_tbl[i]; +for (i =0 ; i < encl->nr_segments; i++) { +struct encl_segment *seg = &encl->segment_tbl[i]; - addr =mmap ((void *)self->encl.encl_base + seg->offset, seg->size, - seg->prot,MAP_SHARED |MAP_FIXED , self->encl.fd,0 ); + addr =mmap ((void *)encl->encl_base + seg->offset, seg->size, + seg->prot,MAP_SHARED |MAP_FIXED , encl->fd,0 );EXPECT_NE (addr,MAP_FAILED );if (addr ==MAP_FAILED ) goto err;
@@ -160,16 +161,13 @@ FIXTURE_SETUP(enclave) vdso_sgx_enter_enclave = addr + sgx_enter_enclave_sym->st_value; -memset (&self->run,0 ,sizeof (self->run)); - self->run.tcs = self->encl.encl_base; - -return ; +return true; err: -encl_delete (&self->encl); +encl_delete (encl); -for (i =0 ; i < self->encl.nr_segments; i++) { - seg = &self->encl.segment_tbl[i]; +for (i =0 ; i < encl->nr_segments; i++) { + seg = &encl->segment_tbl[i];TH_LOG ("0x%016lx 0x%016lx 0x%02x" , seg->offset, seg->size, seg->prot); }
@@ -186,7 +184,17 @@ FIXTURE_SETUP(enclave)fclose (maps_file); } -ASSERT_TRUE (false); +TH_LOG ("Failed to initialize the test enclave.\n" ); + +return false; +} + +FIXTURE_SETUP (enclave) +{ +ASSERT_TRUE (setup_test_encl (ENCL_HEAP_SIZE_DEFAULT , &self->encl, _metadata)); + +memset (&self->run,0 ,sizeof (self->run)); + self->run.tcs = self->encl.encl_base; }FIXTURE_TEARDOWN (enclave)
Signature · 1 line
2.32.0
Jarkko Sakkinen
maintainerPATCH v4 7/8 selftests/sgx: Move setup_test_encl() to each TEST_F()
In reply to Jarkko SakkinenCreate the test enclave inside each TEST_F(), instead of FIXTURE_SETUP(), so that the heap size can be defined per test.
---
v4:
* Wrap setup_test_encl() with ASSERT_TRUE().
* Update commit message.
tools/testing/selftests/sgx/main.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)
tools/testing/selftests/sgx/main.c +15 -4
@@ -191,10 +191,6 @@ static bool setup_test_encl(unsigned long heap_size, struct encl *encl,FIXTURE_SETUP (enclave) { -ASSERT_TRUE (setup_test_encl (ENCL_HEAP_SIZE_DEFAULT , &self->encl, _metadata)); - -memset (&self->run,0 ,sizeof (self->run)); - self->run.tcs = self->encl.encl_base; }FIXTURE_TEARDOWN (enclave)
@@ -226,6 +222,11 @@ TEST_F(enclave, unclobbered_vdso) {struct encl_op op; +ASSERT_TRUE (setup_test_encl (ENCL_HEAP_SIZE_DEFAULT , &self->encl, _metadata)); + +memset (&self->run,0 ,sizeof (self->run)); + self->run.tcs = self->encl.encl_base; + op.type =ENCL_OP_PUT ; op.buffer =MAGIC ;
@@ -248,6 +249,11 @@ TEST_F(enclave, clobbered_vdso) {struct encl_op op; +ASSERT_TRUE (setup_test_encl (ENCL_HEAP_SIZE_DEFAULT , &self->encl, _metadata)); + +memset (&self->run,0 ,sizeof (self->run)); + self->run.tcs = self->encl.encl_base; + op.type =ENCL_OP_PUT ; op.buffer =MAGIC ;
@@ -278,6 +284,11 @@ TEST_F(enclave, clobbered_vdso_and_user_function) {struct encl_op op; +ASSERT_TRUE (setup_test_encl (ENCL_HEAP_SIZE_DEFAULT , &self->encl, _metadata)); + +memset (&self->run,0 ,sizeof (self->run)); + self->run.tcs = self->encl.encl_base; + self->run.user_handler = (__u64 )test_handler; self->run.user_data =0xdeadbeef ;
Signature · 1 line
2.32.0
Jarkko Sakkinen
maintainerPATCH v4 8/8 selftests/sgx: Add a new kselftest: unclobbered_vdso_oversubscribed
In reply to Jarkko SakkinenAdd a variation of the unclobbered_vdso test.
In the new test, create a heap for the test enclave, which has the same size as all available Enclave Page Cache (EPC) pages in the system. This will guarantee that all test_encl.elf pages *and* SGX Enclave Control Structure (SECS) have been swapped out by the page reclaimer during the load time..
This test will trigger both the page reclaimer and the page fault handler. The page reclaimer triggered, while the heap is being created during the load time. The page fault handler is triggered for all the required pages, while the test case is executing.
---
v4:
* Wrap setup_test_encl() and get_sysfs_long() with ASSERT_TRUE().
tools/testing/selftests/sgx/main.c | 59 ++++++++++++++++++++++++++++++ tools/testing/selftests/sgx/main.h | 1 + 2 files changed, 60 insertions(+)
tools/testing/selftests/sgx/main.c +59 -0
@@ -245,6 +245,65 @@ TEST_F(enclave, unclobbered_vdso)EXPECT_EQ (self->run.user_data,0 ); } +static bool sysfs_get_ulong (const char *path,unsigned long *value) +{ +struct stat sbuf; +char buf[128 ]; +ssize_t ret; +int fd; + + ret =stat (path, &sbuf); +if (ret) +return false; + + fd =open (path,O_RDONLY ); +if (fd <0 ) +return false; + + ret =read (fd, buf,sizeof (buf)); +if (ret <0 ) { +close (fd); +return false; + } + + errno =0 ; + *value =strtoul (buf,NULL ,0 ); + +close (fd); + +return errno ? false : true; +} + +TEST_F(enclave, unclobbered_vdso_oversubscribed) +{ +unsigned long total_mem; +struct encl_op op; + +ASSERT_TRUE (sysfs_get_ulong (SGX_TOTAL_MEM_PATH , &total_mem)); +ASSERT_TRUE (setup_test_encl (total_mem, &self->encl, _metadata)); + +memset (&self->run,0 ,sizeof (self->run)); + self->run.tcs = self->encl.encl_base; + + op.type =ENCL_OP_PUT ; + op.buffer =MAGIC ; + +EXPECT_EQ (ENCL_CALL (&op, &self->run, false),0 ); + +EXPECT_EEXIT (&self->run); +EXPECT_EQ (self->run.user_data,0 ); + + op.type =ENCL_OP_GET ; + op.buffer =0 ; + +EXPECT_EQ (ENCL_CALL (&op, &self->run, false),0 ); + +EXPECT_EQ (op.buffer,MAGIC ); +EXPECT_EEXIT (&self->run); +EXPECT_EQ (self->run.user_data,0 ); + +} + TEST_F(enclave, clobbered_vdso) {struct encl_op op;
tools/testing/selftests/sgx/main.h +1 -0
@@ -7,6 +7,7 @@#define MAIN_H #define ENCL_HEAP_SIZE_DEFAULT 4096 +#define SGX_TOTAL_MEM_PATH "/sys/kernel/debug/x86/sgx_total_mem"struct encl_segment {void *src;
Signature · 1 line
2.32.0