tee subsystem pin_user_pages for v5.8
Subsystems (2)
Hello arm-soc maintainers,
Please pull this small patch converting the tee subsystem to use pin_user_pages() instead of get_user_pages().
Thanks,
Jens
The following changes since commit ae83d0b416db002fe95601e7f97f64b59514d936:
Linux 5.7-rc2 (2020-04-19 14:35:30 -0700)
are available in the Git repository at:
git://git.linaro.org:/people/jens.wiklander/linux-tee.git tags/tee-pin-user-pages-for-5.8
for you to fetch changes up to 37f6b4d5f47b600ec4ab6682c005a44a1bfca530:
tee: convert get_user_pages() --> pin_user_pages() (2020-05-26 10:42:41 +0200)
----------------------------------------------------------------
Converts tee subsystem to use pin_user_pages() instead of get_user_pages()
----------------------------------------------------------------
John Hubbard (1):
tee: convert get_user_pages() --> pin_user_pages()
drivers/tee/tee_shm.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-)
Quoted 12 lines · Jens Wiklander
Hello arm-soc maintainers,
Please pull this small patch converting the tee subsystem to use
pin_user_pages()instead ofget_user_pages().Thanks,
JensThe following changes since commit
ae83d0b416db002fe95601e7f97f64b59514d936:Linux 5.7-rc2 (2020-04-19 14:35:30 -0700)
Quoted 7 lines
are available in the Git repository at:
git://git.linaro.org:/people/jens.wiklander/linux-tee.gittags/tee-pin-user-pages-for-5.8for you to fetch changes up to
37f6b4d5f47b600ec4ab6682c005a44a1bfca530:tee: convert
get_user_pages()-->pin_user_pages()(2020-05-26 10:42:41 +0200)
Hi, I noticed this never got merged, but I don't see any follow-up here that retracts it. Is it still pending merge such that I should queue it for v5.10?
-Olof
Quoted 15 lines · Olof Johansson
Quoted 14 lines · Jens Wiklander
Hello arm-soc maintainers,
Please pull this small patch converting the tee subsystem to use
pin_user_pages()instead ofget_user_pages().Thanks,
JensThe following changes since commit
ae83d0b416db002fe95601e7f97f64b59514d936:Linux 5.7-rc2 (2020-04-19 14:35:30 -0700)are available in the Git repository at:
Quoted 9 lines
Quoted 5 lines
git://git.linaro.org:/people/jens.wiklander/linux-tee.gittags/tee-pin-user-pages-for-5.8for you to fetch changes up to
37f6b4d5f47b600ec4ab6682c005a44a1bfca530:tee: convert get_user_pages() --> pin_user_pages() (2020-05-26 10:42:41 +0200)Hi, I noticed this never got merged, but I don't see any follow-up here that retracts it. Is it still pending merge such that I should queue it for v5.10?
I think so. I had marked it in my notes as "accepted, and the maintainer will eventually merge it", and I left it at that. It's still desirable.
thanks,
Signature · 2 lines
John Hubbard NVIDIA
Quoted 21 lines · John Hubbard
Quoted 19 lines · Olof Johansson
Quoted 18 lines · Jens Wiklander
Hello arm-soc maintainers,
Please pull this small patch converting the tee subsystem to use
pin_user_pages()instead ofget_user_pages().Thanks,
JensThe following changes since commit
ae83d0b416db002fe95601e7f97f64b59514d936:Linux 5.7-rc2 (2020-04-19 14:35:30 -0700)are available in the Git repository at:
git://git.linaro.org:/people/jens.wiklander/linux-tee.gittags/tee-pin-user-pages-for-5.8for you to fetch changes up to
37f6b4d5f47b600ec4ab6682c005a44a1bfca530:
Quoted 9 lines
Quoted 5 lines
Quoted 1 line
tee: convert get_user_pages() --> pin_user_pages() (2020-05-26 10:42:41 +0200)Hi, I noticed this never got merged, but I don't see any follow-up here that retracts it. Is it still pending merge such that I should queue it for v5.10?
I think so. I had marked it in my notes as "accepted, and the maintainer will eventually merge it", and I left it at that. It's still desirable.
Looks like it conflicts with some of the later work. Jens, given the timelines here it's probably easiest all around if you rebase/respin and send a fresh pull request. I could fix it up but you'd still need to review that so the amount of work is probably less if you do it directly.
-Olof
Quoted 33 lines · Olof Johansson
Quoted 21 lines · John Hubbard
Quoted 19 lines · Olof Johansson
Quoted 18 lines · Jens Wiklander
Hello arm-soc maintainers,
Please pull this small patch converting the tee subsystem to use
pin_user_pages()instead ofget_user_pages().Thanks,
JensThe following changes since commit
ae83d0b416db002fe95601e7f97f64b59514d936:Linux 5.7-rc2 (2020-04-19 14:35:30 -0700)are available in the Git repository at:
git://git.linaro.org:/people/jens.wiklander/linux-tee.gittags/tee-pin-user-pages-for-5.8for you to fetch changes up to
37f6b4d5f47b600ec4ab6682c005a44a1bfca530:Quoted 9 lines
Quoted 5 lines
Quoted 1 line
tee: convert get_user_pages() --> pin_user_pages() (2020-05-26 10:42:41 +0200)Hi, I noticed this never got merged, but I don't see any follow-up here that retracts it. Is it still pending merge such that I should queue it for v5.10?
I think so. I had marked it in my notes as "accepted, and the maintainer will eventually merge it", and I left it at that. It's still desirable.
Quoted 5 lines
Looks like it conflicts with some of the later work. Jens, given the timelines here it's probably easiest all around if you rebase/respin and send a fresh pull request. I could fix it up but you'd still need to review that so the amount of work is probably less if you do it directly.
Agree, I'll send a fresh pull request once we have this rebased. The conflict is with the recently added call to get_kernel_pages() when kernel memory is shared.
The conflict isn't trivial, I guess we need to handle the different types of pages differently when releasing them.
John, would you mind rebasing and posting the patch again?
Thanks,
Jens
Quoted 40 lines · Jens Wiklander
Quoted 33 lines · Olof Johansson
Quoted 21 lines · John Hubbard
Quoted 19 lines · Olof Johansson
Quoted 18 lines · Jens Wiklander
Hello arm-soc maintainers,
Please pull this small patch converting the tee subsystem to use
pin_user_pages()instead ofget_user_pages().Thanks,
JensThe following changes since commit
ae83d0b416db002fe95601e7f97f64b59514d936:Linux 5.7-rc2 (2020-04-19 14:35:30 -0700)are available in the Git repository at:
git://git.linaro.org:/people/jens.wiklander/linux-tee.gittags/tee-pin-user-pages-for-5.8for you to fetch changes up to
37f6b4d5f47b600ec4ab6682c005a44a1bfca530:Quoted 9 lines
Quoted 5 lines
Quoted 1 line
tee: convert get_user_pages() --> pin_user_pages() (2020-05-26 10:42:41 +0200)Hi, I noticed this never got merged, but I don't see any follow-up here that retracts it. Is it still pending merge such that I should queue it for v5.10?
I think so. I had marked it in my notes as "accepted, and the maintainer will eventually merge it", and I left it at that. It's still desirable.
Quoted 5 lines
Looks like it conflicts with some of the later work. Jens, given the timelines here it's probably easiest all around if you rebase/respin and send a fresh pull request. I could fix it up but you'd still need to review that so the amount of work is probably less if you do it directly.
Quoted 7 lines
Agree, I'll send a fresh pull request once we have this rebased. The conflict is with the recently added call to
get_kernel_pages()when kernel memory is shared.
The conflict isn't trivial, I guess we need to handle the different types of pages differently when releasing them.
John, would you mind rebasing and posting the patch again?
Sure. Should it be against 5.9-rc2, or something else? I can do this in the morning, about 10 hrs from now.
thanks,
Signature · 2 lines
John Hubbard NVIDIA
Quoted 50 lines · John Hubbard
Quoted 40 lines · Jens Wiklander
Quoted 33 lines · Olof Johansson
Quoted 21 lines · John Hubbard
Quoted 19 lines · Olof Johansson
Quoted 18 lines · Jens Wiklander
Hello arm-soc maintainers,
Please pull this small patch converting the tee subsystem to use
pin_user_pages()instead ofget_user_pages().Thanks,
JensThe following changes since commit
ae83d0b416db002fe95601e7f97f64b59514d936:Linux 5.7-rc2 (2020-04-19 14:35:30 -0700)are available in the Git repository at:
git://git.linaro.org:/people/jens.wiklander/linux-tee.gittags/tee-pin-user-pages-for-5.8for you to fetch changes up to
37f6b4d5f47b600ec4ab6682c005a44a1bfca530:Quoted 9 lines
Quoted 5 lines
Quoted 1 line
tee: convert get_user_pages() --> pin_user_pages() (2020-05-26 10:42:41 +0200)Hi, I noticed this never got merged, but I don't see any follow-up here that retracts it. Is it still pending merge such that I should queue it for v5.10?
I think so. I had marked it in my notes as "accepted, and the maintainer will eventually merge it", and I left it at that. It's still desirable.
Quoted 5 lines
Looks like it conflicts with some of the later work. Jens, given the timelines here it's probably easiest all around if you rebase/respin and send a fresh pull request. I could fix it up but you'd still need to review that so the amount of work is probably less if you do it directly.
Quoted 7 lines
Agree, I'll send a fresh pull request once we have this rebased. The conflict is with the recently added call to
get_kernel_pages()when kernel memory is shared.
The conflict isn't trivial, I guess we need to handle the different types of pages differently when releasing them.
John, would you mind rebasing and posting the patch again?
Quoted 2 lines
Sure. Should it be against 5.9-rc2, or something else? I can do this in the morning, about 10 hrs from now.
5.9-rc2 sounds good.
Thanks,
Jens
PATCH v2 1/1 tee: convert convert get_user_pages() --> pin_user_pages()
This code was using get_user_pages*(), in a "Case 2" scenario (DMA/RDMA), using the categorization from [1]. That means that it's time to convert the get_user_pages*() + put_page() calls to pin_user_pages*() + unpin_user_pages() calls.
There is some helpful background in [2]: basically, this is a small part of fixing a long-standing disconnect between pinning pages, and file systems' use of those pages.
[1] Documentation/core-api/pin_user_pages.rst
[2] "Explicit pinning of user-space pages":
https://lwn.net/Articles/807108/
---
OK, this should be indentical to v1 [1], but now rebased against Linux 5.9-rc2.
As before, I've compile-tested it again with a cross compiler, but that's the only testing I'm set up for with CONFIG_TEE.
[1] https://lore.kernel.org/r/[email protected]
thanks,
John Hubbard
NVIDIA
drivers/tee/tee_shm.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-)
drivers/tee/tee_shm.c +3 -9
@@ -32,16 +32,13 @@ static void tee_shm_release(struct tee_shm *shm) poolm->ops->free (poolm, shm); }else if (shm->flags &TEE_SHM_REGISTER ) { -size_t n;int rc = teedev->desc->ops->shm_unregister (shm->ctx, shm);if (rc)dev_err (teedev->dev.parent,"unregister shm %p failed: %d" , shm, rc); -for (n =0 ; n < shm->num_pages; n++) -put_page (shm->pages[n]); - +unpin_user_pages (shm->pages, shm->num_pages);kfree (shm->pages); }
@@ -228,7 +225,7 @@ struct tee_shm *tee_shm_register(struct tee_context *ctx, unsigned long addr, }if (flags &TEE_SHM_USER_MAPPED ) { - rc =get_user_pages_fast (start, num_pages,FOLL_WRITE , + rc =pin_user_pages_fast (start, num_pages,FOLL_WRITE , shm->pages); }else {struct kvec *kiov;
@@ -292,16 +289,13 @@ struct tee_shm *tee_shm_register(struct tee_context *ctx, unsigned long addr,return shm; err:if (shm) { -size_t n; -if (shm->id >=0 ) {mutex_lock (&teedev->mutex);idr_remove (&teedev->idr, shm->id);mutex_unlock (&teedev->mutex); }if (shm->pages) { -for (n =0 ; n < shm->num_pages; n++) -put_page (shm->pages[n]); +unpin_user_pages (shm->pages, shm->num_pages);kfree (shm->pages); } }
Signature · 6 lines
2.28.0 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH v2] tee: convert convert get_user_pages() --> pin_user_pages()
Quoted 21 lines · John Hubbard
This code was using get_user_pages*(), in a "Case 2" scenario (DMA/RDMA), using the categorization from [1]. That means that it's time to convert the get_user_pages*() +
put_page()calls to pin_user_pages*() +unpin_user_pages()calls.There is some helpful background in [2]: basically, this is a small part of fixing a long-standing disconnect between pinning pages, and file systems' use of those pages.
[1]
Documentation/core-api/pin_user_pages.rst[2] "Explicit pinning of user-space pages":
https://lwn.net/Articles/807108/Cc Jens Wiklander <[email protected]>Cc Sumit Semwal <[email protected]>Signed-off-by John Hubbard <[email protected]>
Quoted 5 lines
---
OK, this should be indentical to v1 [1], but now rebased against Linux 5.9-rc2.
...ohhh, wait, I should have read the earlier message from Jens more carefully:
"The conflict isn't trivial, I guess we need to handle the different types of pages differently when releasing them."
So it's not good to have a logically identical patch. argghhh. Let me see how hard it is to track these memory types separately and handle the release accordingly, just a sec.
Sorry about the false move here.
thanks,
Signature · 2 lines
John Hubbard NVIDIA
Quoted 52 lines
As before, I've compile-tested it again with a cross compiler, but that's the only testing I'm set up for with
CONFIG_TEE.[1] https://lore.kernel.org/r/[email protected]
thanks,
John Hubbard
NVIDIAdrivers/tee/tee_shm.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-)drivers/tee/tee_shm.c +2
-7@@ -32,16 +32,13 @@ static void tee_shm_release(struct tee_shm *shm) poolm->ops->free (poolm, shm); }else if (shm->flags &TEE_SHM_REGISTER ) { -size_t n;int rc = teedev->desc->ops->shm_unregister (shm->ctx, shm);if (rc)dev_err (teedev->dev.parent,"unregister shm %p failed: %d" , shm, rc); -for (n =0 ; n < shm->num_pages; n++) -put_page (shm->pages[n]); - +unpin_user_pages (shm->pages, shm->num_pages);kfree (shm->pages); }@@ -228,7 +225,7 @@ struct tee_shm *tee_shm_register(struct tee_context *ctx, unsigned long addr, }if (flags &TEE_SHM_USER_MAPPED ) { - rc =get_user_pages_fast (start, num_pages,FOLL_WRITE , + rc =pin_user_pages_fast (start, num_pages,FOLL_WRITE , shm->pages); }else {struct kvec *kiov;@@ -292,16 +289,13 @@ struct tee_shm *tee_shm_register(struct tee_context *ctx, unsigned long addr,return shm; err:if (shm) { -size_t n; -if (shm->id >=0 ) {mutex_lock (&teedev->mutex);
drivers/tee/tee_shm.c struct tee_shm *tee_shm_register(struct tee_context *ctx, unsigned long addr, · line 301
Quoted 10 lines
drivers/tee/tee_shm.c +1
-2idr_remove (&teedev->idr, shm->id);mutex_unlock (&teedev->mutex); }if (shm->pages) { -for (n =0 ; n < shm->num_pages; n++) -put_page (shm->pages[n]); +unpin_user_pages (shm->pages, shm->num_pages);kfree (shm->pages); } }
v
PATCH v3 1/1 tee: convert convert get_user_pages() --> pin_user_pages()
In reply to Jens WiklanderThis code was using get_user_pages*(), in a "Case 2" scenario (DMA/RDMA), using the categorization from [1]. That means that it's time to convert the get_user_pages*() + put_page() calls to pin_user_pages*() + unpin_user_pages() calls.
Factor out a new, small release_registered_pages() function, in order to consolidate the logic for discerning between TEE_SHM_USER_MAPPED and TEE_SHM_KERNEL_MAPPED pages. This also absorbs the kfree() call that is also required there.
There is some helpful background in [2]: basically, this is a small part of fixing a long-standing disconnect between pinning pages, and file systems' use of those pages.
[1] Documentation/core-api/pin_user_pages.rst
[2] "Explicit pinning of user-space pages":
https://lwn.net/Articles/807108/
---
OK, one more try, this time actually handling the _USER_MAPPED vs. _KERNEL_MAPPED pages!
thanks,
John Hubbard
NVIDIA
drivers/tee/tee_shm.c | 32 +++++++++++++++++++------------- 1 file changed, 19 insertions(+), 13 deletions(-)
drivers/tee/tee_shm.c +19 -13
@@ -12,6 +12,22 @@#include <linux/uio.h> #include "tee_private.h" +static void release_registered_pages (struct tee_shm *shm) +{ +if (shm->pages) { +if (shm->flags &TEE_SHM_USER_MAPPED ) { +unpin_user_pages (shm->pages, shm->num_pages); + }else { +size_t n; + +for (n =0 ; n < shm->num_pages; n++) +put_page (shm->pages[n]); + } + +kfree (shm->pages); + } +} +static void tee_shm_release (struct tee_shm *shm) {struct tee_device *teedev = shm->ctx->teedev;
@@ -32,17 +48,13 @@ static void tee_shm_release(struct tee_shm *shm) poolm->ops->free (poolm, shm); }else if (shm->flags &TEE_SHM_REGISTER ) { -size_t n;int rc = teedev->desc->ops->shm_unregister (shm->ctx, shm);if (rc)dev_err (teedev->dev.parent,"unregister shm %p failed: %d" , shm, rc); -for (n =0 ; n < shm->num_pages; n++) -put_page (shm->pages[n]); - -kfree (shm->pages); +release_registered_pages (shm); }teedev_ctx_put (shm->ctx);
@@ -228,7 +240,7 @@ struct tee_shm *tee_shm_register(struct tee_context *ctx, unsigned long addr, }if (flags &TEE_SHM_USER_MAPPED ) { - rc =get_user_pages_fast (start, num_pages,FOLL_WRITE , + rc =pin_user_pages_fast (start, num_pages,FOLL_WRITE , shm->pages); }else {struct kvec *kiov;
@@ -292,18 +304,12 @@ struct tee_shm *tee_shm_register(struct tee_context *ctx, unsigned long addr,return shm; err:if (shm) { -size_t n; -if (shm->id >=0 ) {mutex_lock (&teedev->mutex);idr_remove (&teedev->idr, shm->id);mutex_unlock (&teedev->mutex); } -if (shm->pages) { -for (n =0 ; n < shm->num_pages; n++) -put_page (shm->pages[n]); -kfree (shm->pages); - } +release_registered_pages (shm); }kfree (shm);teedev_ctx_put (ctx);
Signature · 6 lines
2.28.0 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH v3] tee: convert convert get_user_pages() --> pin_user_pages()
Quoted 26 lines · John Hubbard
This code was using get_user_pages*(), in a "Case 2" scenario (DMA/RDMA), using the categorization from [1]. That means that it's time to convert the get_user_pages*() +
put_page()calls to pin_user_pages*() +unpin_user_pages()calls.Factor out a new, small
release_registered_pages()function, in order to consolidate the logic for discerning betweenTEE_SHM_USER_MAPPEDandTEE_SHM_KERNEL_MAPPEDpages. This also absorbs thekfree()call that is also required there.There is some helpful background in [2]: basically, this is a small part of fixing a long-standing disconnect between pinning pages, and file systems' use of those pages.
[1]
Documentation/core-api/pin_user_pages.rst[2] "Explicit pinning of user-space pages":
https://lwn.net/Articles/807108/Cc Jens Wiklander <[email protected]>Cc Sumit Semwal <[email protected]>Signed-off-by John Hubbard <[email protected]>
Quoted 8 lines
---
OK, one more try, this time actually handling the
_USER_MAPPEDvs._KERNEL_MAPPEDpages!thanks,
John Hubbard
NVIDIA
Looks good and it works too! :-) I've tested it on my Hikey board with the OP-TEE test suite.
I'm picking this up.
Thanks,
Jens
Re: [PATCH v3] tee: convert convert get_user_pages() --> pin_user_pages()
Quoted 1 line · Jens Wiklander
On Mon, Aug 24, 2020 at 02:11:25PM -0700, John Hubbard wrote:
...
Quoted 11 lines
Quoted 6 lines
OK, one more try, this time actually handling the
_USER_MAPPEDvs._KERNEL_MAPPEDpages!thanks,
John Hubbard
NVIDIALooks good and it works too! :-) I've tested it on my Hikey board with the OP-TEE test suite.
I'm picking this up.
Great! I see that I have, once again, somehow doubled up on the subject line:
"tee: convert convert ...". This particular typo just seems to stick to me. :)
If you get a chance to fix that up by changing it to just a single "convert" I'd appreciate it.
thanks,
Signature · 6 lines
John Hubbard NVIDIA _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Re: [PATCH v3] tee: convert convert get_user_pages() --> pin_user_pages()
Quoted 16 lines · John Hubbard
Quoted 1 line · Jens Wiklander
On Mon, Aug 24, 2020 at 02:11:25PM -0700, John Hubbard wrote:
...
Quoted 11 lines
Quoted 6 lines
OK, one more try, this time actually handling the
_USER_MAPPEDvs._KERNEL_MAPPEDpages!thanks,
John Hubbard
NVIDIALooks good and it works too! :-) I've tested it on my Hikey board with the OP-TEE test suite.
I'm picking this up.
Quoted 5 lines
Great! I see that I have, once again, somehow doubled up on the subject line:
"tee: convert convert ...". This particular typo just seems to stick to me. :)If you get a chance to fix that up by changing it to just a single "convert" I'd appreciate it.
Sure, no problem.
Cheers,
Jens