x86/sgx: Fix sgx_encl_may_map locking
PATCH v3 1/1 x86/sgx: Fix sgx_encl_may_map locking
Fix the issue further discussed in:
1. https://lore.kernel.org/linux-sgx/[email protected]/
2. https://lore.kernel.org/linux-sgx/[email protected]/
---
v3:
* Added the missing unlock pointed out by Matthew.
* Tested with the correct patch (last time had v1 applied)
* I don't know what happened to v2 changelog, checked from patchwork and it wasn't there. Hope this is not scraped.
arch/x86/kernel/cpu/sgx/encl.c | 29 +++++++++++++++++++++++++---- 1 file changed, 25 insertions(+), 4 deletions(-)
arch/x86/kernel/cpu/sgx/encl.c +25 -4
@@ -307,6 +307,8 @@ int sgx_encl_may_map(struct sgx_encl *encl, unsigned long start,unsigned long idx_start =PFN_DOWN (start);unsigned long idx_end =PFN_DOWN (end -1 );struct sgx_encl_page *page; +unsigned long count =0 ; +int ret =0 ;XA_STATE (xas, &encl->page_array, idx_start);
@@ -317,11 +319,30 @@ int sgx_encl_may_map(struct sgx_encl *encl, unsigned long start,if (current->personality &READ_IMPLIES_EXEC )return -EACCES ; -xas_for_each (&xas, page, idx_end) -if (!page || (~page->vm_max_prot_bits & vm_prot_bits)) -return -EACCES ; +/* +* No need to hold encl->lock: +* 1. None of the page->* get written. +* 2. page->vm_max_prot_bits is set in sgx_encl_page_alloc(). This +* is before calling xa_insert(). After that it is never modified. +*/ +xas_lock (&xas); +xas_for_each (&xas, page, idx_end) { +if (++count %XA_CHECK_SCHED ) +continue ; -return 0 ; +if (!page || (~page->vm_max_prot_bits & vm_prot_bits)) { + ret = -EACCES ; +break ; + } + +xas_pause (&xas); +xas_unlock (&xas); +cond_resched (); +xas_lock (&xas); + } +xas_unlock (&xas); + +return ret; } static int sgx_vma_mprotect(struct vm_area_struct *vma,
Signature · 1 line
2.25.1
Quoted 1 line · Jarkko Sakkinen
Fix the issue further discussed in:
No, this is still utter crap. Just use the version I sent.
Quoted 69 lines
1. https://lore.kernel.org/linux-sgx/[email protected]/
2. https://lore.kernel.org/linux-sgx/[email protected]/Reported-by Haitao Huang <[email protected]>Suggested-by Matthew Wilcox <[email protected]>Cc Sean Christopherson <[email protected]>Cc Jethro Beekman <[email protected]>Cc Dave Hansen <[email protected]>Signed-off-by Jarkko Sakkinen <[email protected]>---
v3:
* Added the missing unlock pointed out by Matthew.
* Tested with the correct patch (last time had v1 applied)
* I don't know what happened to v2 changelog, checked from patchwork and it wasn't there. Hope this is not scraped.arch/x86/kernel/cpu/sgx/encl.c | 29 +++++++++++++++++++++++++---- 1 file changed, 25 insertions(+), 4 deletions(-)arch/x86/kernel/cpu/sgx/encl.c +25
-4@@ -307,6 +307,8 @@ int sgx_encl_may_map(struct sgx_encl *encl, unsigned long start,unsigned long idx_start =PFN_DOWN (start);unsigned long idx_end =PFN_DOWN (end -1 );struct sgx_encl_page *page; +unsigned long count =0 ; +int ret =0 ;XA_STATE (xas, &encl->page_array, idx_start);@@ -317,11 +319,30 @@ int sgx_encl_may_map(struct sgx_encl *encl, unsigned long start,if (current->personality &READ_IMPLIES_EXEC )return -EACCES ; -xas_for_each (&xas, page, idx_end) -if (!page || (~page->vm_max_prot_bits & vm_prot_bits)) -return -EACCES ; +/* +* No need to hold encl->lock: +* 1. None of the page->* get written. +* 2. page->vm_max_prot_bits is set in sgx_encl_page_alloc(). This +* is before calling xa_insert(). After that it is never modified. +*/ +xas_lock (&xas); +xas_for_each (&xas, page, idx_end) { +if (++count %XA_CHECK_SCHED ) +continue ; -return 0 ; +if (!page || (~page->vm_max_prot_bits & vm_prot_bits)) { + ret = -EACCES ; +break ; + } + +xas_pause (&xas); +xas_unlock (&xas); +cond_resched (); +xas_lock (&xas); + } +xas_unlock (&xas); + +return ret; } static int sgx_vma_mprotect(struct vm_area_struct *vma,Signature · 1 line
2.25.1
Quoted 4 lines · Matthew Wilcox
Quoted 1 line · Jarkko Sakkinen
Fix the issue further discussed in:
No, this is still utter crap. Just use the version I sent.
OK, just a quick recap to fully understand it.
Here's your response from the original thread:
https://lore.kernel.org/linux-sgx/[email protected]/
And here's your snippet from v2:
https://lore.kernel.org/linux-sgx/[email protected]/
This is what confused me.
Compared to (v3) patch and your version, the differences I spot are:
A. 'count' is checked first.
B. Both this and the snippet from the original thread (i.e.
page-writeback.c) use the same loop construct.
Right, and snippet from page-writeback.c is not compatible with this because looking at documentation xas_find() continued with xas_next_entry() sequence will jump through the existing entries and skip the holes? On the other hand, xas_next() does not.
Of the A part I'm not sure how the order there semantically matter.
/Jarkko
Quoted 24 lines · Jarkko Sakkinen
Quoted 4 lines · Matthew Wilcox
Quoted 1 line · Jarkko Sakkinen
Fix the issue further discussed in:
No, this is still utter crap. Just use the version I sent.
OK, just a quick recap to fully understand it.
Here's your response from the original thread:
https://lore.kernel.org/linux-sgx/[email protected]/
And here's your snippet from v2:
https://lore.kernel.org/linux-sgx/[email protected]/
This is what confused me.
Compared to (v3) patch and your version, the differences I spot are:
A. 'count' is checked first.
B. Both this and the snippet from the original thread (i.e.
page-writeback.c) use the same loop construct.
Quoted 6 lines
Right, and snippet from
page-writeback.cis not compatible with this because looking at documentationxas_find()continued withxas_next_entry()sequence will jump through the existing entries and skip the holes? On the other hand,xas_next()does not.Of the A part I'm not sure how the order there semantically matter.
Right, how blind I was with B. Read Dave's response.
/Jarkko
Quoted 1 line · Jarkko Sakkinen
arch/x86/kernel/cpu/sgx/encl.c +1
-0+unsigned long count =0 ;
...
arch/x86/kernel/cpu/sgx/encl.c int sgx_encl_may_map(struct sgx_encl *encl, unsigned long start, · line 331
Quoted 4 lines
arch/x86/kernel/cpu/sgx/encl.c +4
-0+xas_lock (&xas); +xas_for_each (&xas, page, idx_end) { +if (++count %XA_CHECK_SCHED ) +continue ;
Let's slow down and think through the loop, please.
First time through the loop, count=0, XA_CHECK_SCHED=4096, it will do a ++count, and now count=1. (count % XA_CHECK_SCHED) == 1. It will continue. It skips the page->vm_max_prot_bits checks.
Next time through the loop, count=1, XA_CHECK_SCHED=4096, it will do a ++count, and now count=2. (count % XA_CHECK_SCHED) == 2. It will continue. It skips the page->vm_max_prot_bits checks.
...
It will do this until it hits count=4095 where it will actually fall into the rest of the loop, doing the page->vm_max_prot_bits checks.
So, in the end the loop only does what it's supposed to be doing 1/4096 times. Not great. Don't we have tests that will notice breakage like this?
The XA_CHECK_SCHED needs to be stuck near the *end* of the loop, just before the lock dropping and resched stuff.
Quoted 14 lines · Dave Hansen
Quoted 1 line · Jarkko Sakkinen
arch/x86/kernel/cpu/sgx/encl.c +1
-0+unsigned long count =0 ;...
arch/x86/kernel/cpu/sgx/encl.c int sgx_encl_may_map(struct sgx_encl *encl, unsigned long start, · line 331
Quoted 4 lines
arch/x86/kernel/cpu/sgx/encl.c +4
-0+xas_lock (&xas); +xas_for_each (&xas, page, idx_end) { +if (++count %XA_CHECK_SCHED ) +continue ;Let's slow down and think through the loop, please.
First time through the loop, count=0, XA_CHECK_SCHED=4096, it will do a ++count, and now count=1. (count %
XA_CHECK_SCHED) == 1. It will continue. It skips thepage->vm_max_prot_bitschecks.
Quoted 8 lines
Next time through the loop, count=1, XA_CHECK_SCHED=4096, it will do a ++count, and now count=2. (count %
XA_CHECK_SCHED) == 2. It will continue. It skips thepage->vm_max_prot_bitschecks....
It will do this until it hits count=4095 where it will actually fall into the rest of the loop, doing the
page->vm_max_prot_bitschecks.
Uh oh, how stupid of me.
Quoted 2 lines
So, in the end the loop only does what it's supposed to be doing 1/4096 times. Not great. Don't we have tests that will notice breakage like this?
It would not be hard to have two counts and WARN_ON in the end to compare for mismatch.
Quoted 2 lines
The
XA_CHECK_SCHEDneeds to be stuck near the *end* of the loop, just before the lock dropping and resched stuff.
Referring to Matthew's suggestion:
https://lore.kernel.org/linux-sgx/[email protected]/
I think that the order is just right.
It takes the page, does the processing, and in the tail does check before rescheduling.
The only thing I'd do for clarity would be to change the tail in that like this:
/* Move to the next iteration. */
count++;
/* Surpassed the modulus, reschedule. */
if (!(count % XA_CHECK_SCHED)) {
xas_pause(&xas);
xas_unlock(&xas);
cond_resched();
xas_lock(&xas);
}
}
xas_unlock(&xas);I think this is just a bit more readable way to express the same thing. Matthew, can you agree with this small twist?
/Jarkko
arch/x86/kernel/cpu/sgx/encl.c int sgx_encl_may_map(struct sgx_encl *encl, unsigned long start, · line 327
Quoted 13 lines · Jarkko Sakkinen
arch/x86/kernel/cpu/sgx/encl.c +6
-3@@ -317,11 +319,30 @@ int sgx_encl_may_map(struct sgx_encl *encl, unsigned long start,if (current->personality &READ_IMPLIES_EXEC )return -EACCES ; -xas_for_each (&xas, page, idx_end) -if (!page || (~page->vm_max_prot_bits & vm_prot_bits)) -return -EACCES ; +/* +* No need to hold encl->lock: +* 1. None of the page->* get written. +* 2. page->vm_max_prot_bits is set in sgx_encl_page_alloc(). This +* is before calling xa_insert(). After that it is never modified. +*/
You forgot to cover racing with insertion, e.g. below is the snippet from my original patch[*], which did the lookup without protection from encl->lock.`
+ /*
+ * No need to take encl->lock, vm_prot_bits is set prior to
+ * insertion and never changes, and racing with adding pages is
+ * a userspace bug.
+ */
+ rcu_read_lock();
+ page = radix_tree_lookup(&encl->page_tree, idx);
+ rcu_read_unlock();
Quoted 28 lines · Sean Christopherson
arch/x86/kernel/cpu/sgx/encl.c int sgx_encl_may_map(struct sgx_encl *encl, unsigned long start, · line 327
Quoted 13 lines · Jarkko Sakkinen
arch/x86/kernel/cpu/sgx/encl.c +6
-3@@ -317,11 +319,30 @@ int sgx_encl_may_map(struct sgx_encl *encl, unsigned long start,if (current->personality &READ_IMPLIES_EXEC )return -EACCES ; -xas_for_each (&xas, page, idx_end) -if (!page || (~page->vm_max_prot_bits & vm_prot_bits)) -return -EACCES ; +/* +* No need to hold encl->lock: +* 1. None of the page->* get written. +* 2. page->vm_max_prot_bits is set in sgx_encl_page_alloc(). This +* is before calling xa_insert(). After that it is never modified. +*/ You forgot to cover racing with insertion, e.g. below is the snippet from my original patch[*], which did the lookup without protection from encl->lock.`
(quoted diff) +8
-0+ /* + * No need to take encl->lock, vm_prot_bits is set prior to + * insertion and never changes, and racing with adding pages is + * a userspace bug. + */ + rcu_read_lock(); + page = radix_tree_lookup(&encl->page_tree, idx); + rcu_read_unlock();
Quoted 1 line
I'm not sure why that was merged as it was but it probably was not because of that snippet. It had encl->lock before, so it was by all practical means covered then. I would have replaced encl->lock with that if I ever had received a patch with just that, i.e. that particular snippet gone to noise. That's why it was not broken before v36.
/Jarkko
Quoted 29 lines · Jarkko Sakkinen
Quoted 28 lines · Sean Christopherson
arch/x86/kernel/cpu/sgx/encl.c int sgx_encl_may_map(struct sgx_encl *encl, unsigned long start, · line 327
Quoted 13 lines · Jarkko Sakkinen
arch/x86/kernel/cpu/sgx/encl.c +6
-3@@ -317,11 +319,30 @@ int sgx_encl_may_map(struct sgx_encl *encl, unsigned long start,if (current->personality &READ_IMPLIES_EXEC )return -EACCES ; -xas_for_each (&xas, page, idx_end) -if (!page || (~page->vm_max_prot_bits & vm_prot_bits)) -return -EACCES ; +/* +* No need to hold encl->lock: +* 1. None of the page->* get written. +* 2. page->vm_max_prot_bits is set in sgx_encl_page_alloc(). This +* is before calling xa_insert(). After that it is never modified. +*/ You forgot to cover racing with insertion, e.g. below is the snippet from my original patch[*], which did the lookup without protection from encl->lock.`
(quoted diff) +8
-0+ /* + * No need to take encl->lock, vm_prot_bits is set prior to + * insertion and never changes, and racing with adding pages is + * a userspace bug. + */ + rcu_read_lock(); + page = radix_tree_lookup(&encl->page_tree, idx); + rcu_read_unlock();
Quoted 7 lines
Quoted 1 line
I'm not sure why that was merged as it was but it probably was not because of that snippet. It had
encl->lockbefore, so it was by all practical means covered then. I would have replacedencl->lockwith that if I ever had received a patch with just that, i.e. that particular snippet gone to noise. That's why it was not broken before v36.
Or even if that patch was squashed (can't recall that far), it was not intentionally not replaced. I have no idea why encl->lock was not replaced but the code was not broken.
/Jarkko