Take away EREMOVE from sgx_free_epc_page()
[email protected]
maintainerv2 cover Take away EREMOVE from sgx_free_epc_page()
From: "[email protected]" <[email protected]>
Refactor EREMOVE out of sgx_free_epc_page() since it is only used in one function.
I re-organized my Git a bit in order to keep track of things:
- 'master' bleading edge and 'next' ready PR ready branch
- 'numa' has nearly finished stuff, including the patches I'm posting now.
[email protected] (2):
x86/sgx: Remove a warn from sgx_free_epc_page() x86/sgx: Wipe out EREMOVE from sgx_free_epc_page()
arch/x86/kernel/cpu/sgx/encl.c | 15 ++++++++++++++- arch/x86/kernel/cpu/sgx/main.c | 12 ++++-------- 2 files changed, 18 insertions(+), 9 deletions(-)
Signature · 1 line
2.29.2
[email protected]
maintainerPATCH v2 1/2 x86/sgx: Remove a warn from sgx_free_epc_page()
From: "[email protected]" <[email protected]>
Remove SGX_EPC_PAGE_RECLAIMER_TRACKED check and warning. This cannot happen, as enclave pages are freed only at the time when encl->refcount triggers, i.e. when both VFS and the page reclaimer have given up on their references.
--- arch/x86/kernel/cpu/sgx/main.c | 2 -- 1 file changed, 2 deletions(-)
arch/x86/kernel/cpu/sgx/main.c +0 -2
@@ -636,8 +636,6 @@ void sgx_free_epc_page(struct sgx_epc_page *page)struct sgx_epc_section *section = &sgx_epc_sections[page->section];int ret; -WARN_ON_ONCE (page->flags &SGX_EPC_PAGE_RECLAIMER_TRACKED ); - ret =__eremove (sgx_get_epc_virt_addr (page));if (WARN_ONCE (ret,"EREMOVE returned %d (0x%x)" , ret, ret))return ;
Signature · 1 line
2.29.2
[email protected]
maintainerPATCH v2 2/2 x86/sgx: Wipe out EREMOVE from sgx_free_epc_page()
In reply to [email protected]From: "[email protected]" <[email protected]>
Encapsulate the snippet in sgx_free_epc_page() concerning EREMOVE to sgx_reset_epc_page(), which is a static helper function for sgx_encl_release(). It's the only function existing, which deals with initialized pages.
---
v2: Fixed a compilation error s/page/epc_page/ change not amended
to the patches.
arch/x86/kernel/cpu/sgx/encl.c | 15 ++++++++++++++- arch/x86/kernel/cpu/sgx/main.c | 10 ++++------ 2 files changed, 18 insertions(+), 7 deletions(-)
arch/x86/kernel/cpu/sgx/encl.c +14 -1
@@ -389,6 +389,16 @@ const struct vm_operations_struct sgx_vm_ops = { .access = sgx_vma_access, }; + +static void sgx_reset_epc_page (struct sgx_epc_page *epc_page) +{ +int ret; + + ret =__eremove (sgx_get_epc_virt_addr (epc_page)); +if (WARN_ONCE (ret,"EREMOVE returned %d (0x%x)" , ret, ret)) +return ; +} +/** * sgx_encl_release - Destroy an enclave instance * @kref: address of a kref inside &sgx_encl
@@ -412,6 +422,7 @@ void sgx_encl_release(struct kref *ref)if (sgx_unmark_page_reclaimable (entry->epc_page))continue ; +sgx_reset_epc_page (entry->epc_page);sgx_free_epc_page (entry->epc_page); encl->secs_child_cnt--; entry->epc_page =NULL ;
@@ -423,6 +434,7 @@ void sgx_encl_release(struct kref *ref)xa_destroy (&encl->page_array);if (!encl->secs_child_cnt && encl->secs.epc_page) { +sgx_reset_epc_page (encl->secs.epc_page);sgx_free_epc_page (encl->secs.epc_page); encl->secs.epc_page =NULL ; }
@@ -431,7 +443,8 @@ void sgx_encl_release(struct kref *ref) va_page = list_first_entry(&encl->va_pages, struct sgx_va_page, list);list_del (&va_page->list); -sgx_free_epc_page (va_page->epc_page); +sgx_reset_epc_page (entry->epc_page); +sgx_free_epc_page (entry->epc_page);kfree (va_page); }
arch/x86/kernel/cpu/sgx/main.c +4 -6
@@ -629,16 +629,14 @@ struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim)* sgx_free_epc_page() - Free an EPC page * @page: an EPC page * -* Call EREMOVE for an EPC page and insert it back to the list of free pages. +* Put the EPC page back to the list of free pages. It's the callers +* responsibility to make sure that the page is in uninitialized state In other +* words, do EREMOVE, EWB or whatever operation is necessary before calling +* this function. */ void sgx_free_epc_page (struct sgx_epc_page *page) {struct sgx_epc_section *section = &sgx_epc_sections[page->section]; -int ret; - - ret =__eremove (sgx_get_epc_virt_addr (page)); -if (WARN_ONCE (ret,"EREMOVE returned %d (0x%x)" , ret, ret)) -return ;spin_lock (§ion->lock);list_add_tail (&page->list, §ion->page_list);
Signature · 1 line
2.29.2
Re: [PATCH v2 2/2] x86/sgx: Wipe out EREMOVE from sgx_free_epc_page()
Quoted 53 lines · On Wed, 13 Jan 2021 01:33:51 +0200 [email protected] wrote:
From: "[email protected]" <[email protected]>
Encapsulate the snippet in
sgx_free_epc_page()concerning EREMOVE tosgx_reset_epc_page(), which is a static helper function forsgx_encl_release(). It's the only function existing, which deals with initialized pages.Signed-off-by [email protected] <[email protected]>---
v2: Fixed a compilation error s/page/epc_page/ change not amended
to the patches.arch/x86/kernel/cpu/sgx/encl.c | 15 ++++++++++++++- arch/x86/kernel/cpu/sgx/main.c | 10 ++++------ 2 files changed, 18 insertions(+), 7 deletions(-)arch/x86/kernel/cpu/sgx/encl.c +12
-0@@ -389,6 +389,16 @@ const struct vm_operations_struct sgx_vm_ops = { .access = sgx_vma_access, }; + +static void sgx_reset_epc_page (struct sgx_epc_page *epc_page) +{ +int ret; + + ret =__eremove (sgx_get_epc_virt_addr (epc_page)); +if (WARN_ONCE (ret,"EREMOVE returned %d (0x%x)" , ret, ret)) +return ; +} +/** * sgx_encl_release - Destroy an enclave instance * @kref: address of a kref inside &sgx_encl @@ -412,6 +422,7 @@ void sgx_encl_release(struct kref *ref)if (sgx_unmark_page_reclaimable (entry->epc_page))continue ; +sgx_reset_epc_page (entry->epc_page);sgx_free_epc_page (entry->epc_page); encl->secs_child_cnt--; entry->epc_page =NULL ;@@ -423,6 +434,7 @@ void sgx_encl_release(struct kref *ref)xa_destroy (&encl->page_array);if (!encl->secs_child_cnt && encl->secs.epc_page) { +sgx_reset_epc_page (encl->secs.epc_page);sgx_free_epc_page (encl->secs.epc_page); encl->secs.epc_page =NULL ; }
arch/x86/kernel/cpu/sgx/encl.c void sgx_encl_release(struct kref *ref) · line 447
Quoted 6 lines
arch/x86/kernel/cpu/sgx/encl.c +2
-1@@ -431,7 +443,8 @@ void sgx_encl_release(struct kref *ref) va_page = list_first_entry(&encl->va_pages, struct sgx_va_page, list);list_del (&va_page->list); -sgx_free_epc_page (va_page->epc_page); +sgx_reset_epc_page (entry->epc_page); +sgx_free_epc_page (entry->epc_page);
Reply so that Sean can see, since he just subscribed to linux-sgx list.
As Sean pointed out in KVM SGX virtualization thread, entry should be va_page.
Quoted 31 lines
arch/x86/kernel/cpu/sgx/encl.c +0
-0kfree (va_page); }arch/x86/kernel/cpu/sgx/main.c +4
-6@@ -629,16 +629,14 @@ struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim)* sgx_free_epc_page() - Free an EPC page * @page: an EPC page * -* Call EREMOVE for an EPC page and insert it back to the list of free pages. +* Put the EPC page back to the list of free pages. It's the callers +* responsibility to make sure that the page is in uninitialized state In other +* words, do EREMOVE, EWB or whatever operation is necessary before calling +* this function. */ void sgx_free_epc_page (struct sgx_epc_page *page) {struct sgx_epc_section *section = &sgx_epc_sections[page->section]; -int ret; - - ret =__eremove (sgx_get_epc_virt_addr (page)); -if (WARN_ONCE (ret,"EREMOVE returned %d (0x%x)" , ret, ret)) -return ;spin_lock (§ion->lock);list_add_tail (&page->list, §ion->page_list);Signature · 1 line
2.29.2
Jarkko Sakkinen
maintainerRe: [PATCH v2 2/2] x86/sgx: Wipe out EREMOVE from sgx_free_epc_page()
Quoted 63 lines · Kai Huang
Quoted 53 lines · On Wed, 13 Jan 2021 01:33:51 +0200 [email protected] wrote:
From: "[email protected]" <[email protected]>
Encapsulate the snippet in
sgx_free_epc_page()concerning EREMOVE tosgx_reset_epc_page(), which is a static helper function forsgx_encl_release(). It's the only function existing, which deals with initialized pages.Signed-off-by [email protected] <[email protected]>---
v2: Fixed a compilation error s/page/epc_page/ change not amended
to the patches.arch/x86/kernel/cpu/sgx/encl.c | 15 ++++++++++++++- arch/x86/kernel/cpu/sgx/main.c | 10 ++++------ 2 files changed, 18 insertions(+), 7 deletions(-)arch/x86/kernel/cpu/sgx/encl.c +12
-0@@ -389,6 +389,16 @@ const struct vm_operations_struct sgx_vm_ops = { .access = sgx_vma_access, }; + +static void sgx_reset_epc_page (struct sgx_epc_page *epc_page) +{ +int ret; + + ret =__eremove (sgx_get_epc_virt_addr (epc_page)); +if (WARN_ONCE (ret,"EREMOVE returned %d (0x%x)" , ret, ret)) +return ; +} +/** * sgx_encl_release - Destroy an enclave instance * @kref: address of a kref inside &sgx_encl @@ -412,6 +422,7 @@ void sgx_encl_release(struct kref *ref)if (sgx_unmark_page_reclaimable (entry->epc_page))continue ; +sgx_reset_epc_page (entry->epc_page);sgx_free_epc_page (entry->epc_page); encl->secs_child_cnt--; entry->epc_page =NULL ;@@ -423,6 +434,7 @@ void sgx_encl_release(struct kref *ref)xa_destroy (&encl->page_array);if (!encl->secs_child_cnt && encl->secs.epc_page) { +sgx_reset_epc_page (encl->secs.epc_page);sgx_free_epc_page (encl->secs.epc_page); encl->secs.epc_page =NULL ; }arch/x86/kernel/cpu/sgx/encl.c void sgx_encl_release(struct kref *ref) · line 447
Quoted 6 lines
arch/x86/kernel/cpu/sgx/encl.c +2
-1@@ -431,7 +443,8 @@ void sgx_encl_release(struct kref *ref) va_page = list_first_entry(&encl->va_pages, struct sgx_va_page, list);list_del (&va_page->list); -sgx_free_epc_page (va_page->epc_page); +sgx_reset_epc_page (entry->epc_page); +sgx_free_epc_page (entry->epc_page);Reply so that Sean can see, since he just subscribed to linux-sgx list.
Quoted 1 line
As Sean pointed out in KVM SGX virtualization thread, entry should be
va_page.
Thank, noticed, and responded. I'm sending v3. I think it would be best to then include that version to KVM patch set, as there is no such a rush to get my patches to upstream. I already dropped them from my bleeding edge numa branch.
/Jarkko
Re: [PATCH v2 2/2] x86/sgx: Wipe out EREMOVE from sgx_free_epc_page()
Quoted 64 lines · On Wed, 13 Jan 2021 23:08:08 +0200 Jarkko Sakkinen wrote:
Quoted 63 lines · Kai Huang
Quoted 53 lines · On Wed, 13 Jan 2021 01:33:51 +0200 [email protected] wrote:
From: "[email protected]" <[email protected]>
Encapsulate the snippet in
sgx_free_epc_page()concerning EREMOVE tosgx_reset_epc_page(), which is a static helper function forsgx_encl_release(). It's the only function existing, which deals with initialized pages.Signed-off-by [email protected] <[email protected]>---
v2: Fixed a compilation error s/page/epc_page/ change not amended
to the patches.arch/x86/kernel/cpu/sgx/encl.c | 15 ++++++++++++++- arch/x86/kernel/cpu/sgx/main.c | 10 ++++------ 2 files changed, 18 insertions(+), 7 deletions(-)arch/x86/kernel/cpu/sgx/encl.c +12
-0@@ -389,6 +389,16 @@ const struct vm_operations_struct sgx_vm_ops = { .access = sgx_vma_access, }; + +static void sgx_reset_epc_page (struct sgx_epc_page *epc_page) +{ +int ret; + + ret =__eremove (sgx_get_epc_virt_addr (epc_page)); +if (WARN_ONCE (ret,"EREMOVE returned %d (0x%x)" , ret, ret)) +return ; +} +/** * sgx_encl_release - Destroy an enclave instance * @kref: address of a kref inside &sgx_encl @@ -412,6 +422,7 @@ void sgx_encl_release(struct kref *ref)if (sgx_unmark_page_reclaimable (entry->epc_page))continue ; +sgx_reset_epc_page (entry->epc_page);sgx_free_epc_page (entry->epc_page); encl->secs_child_cnt--; entry->epc_page =NULL ;@@ -423,6 +434,7 @@ void sgx_encl_release(struct kref *ref)xa_destroy (&encl->page_array);if (!encl->secs_child_cnt && encl->secs.epc_page) { +sgx_reset_epc_page (encl->secs.epc_page);sgx_free_epc_page (encl->secs.epc_page); encl->secs.epc_page =NULL ; }arch/x86/kernel/cpu/sgx/encl.c void sgx_encl_release(struct kref *ref) · line 447
Quoted 6 lines
arch/x86/kernel/cpu/sgx/encl.c +2
-1@@ -431,7 +443,8 @@ void sgx_encl_release(struct kref *ref) va_page = list_first_entry(&encl->va_pages, struct sgx_va_page, list);list_del (&va_page->list); -sgx_free_epc_page (va_page->epc_page); +sgx_reset_epc_page (entry->epc_page); +sgx_free_epc_page (entry->epc_page);Reply so that Sean can see, since he just subscribed to linux-sgx list.
Quoted 6 lines
Quoted 1 line
As Sean pointed out in KVM SGX virtualization thread, entry should be
va_page.Thank, noticed, and responded. I'm sending v3. I think it would be best to then include that version to KVM patch set, as there is no such a rush to get my patches to upstream. I already dropped them from my bleeding edge numa branch.
Sure. I'll take your patch once I see it. Thanks Jarkko.