more get_user_pages mmap_sem cleanups
Subsystems (5)
cover more get_user_pages mmap_sem cleanups
Hi,
Here are two more patchlets that cleanup mmap_sem and gup abusers. The second is also a fixlet.
Compile-tested only. Please consider for v5.1
Thanks!
Davidlohr Bueso (2):
xsk: do not use mmap_sem
MIPS/c-r4k: do no use mmap_sem for gup_fast()
arch/mips/mm/c-r4k.c | 6 +----- net/xdp/xdp_umem.c | 6 ++---- 2 files changed, 3 insertions(+), 9 deletions(-)
Signature · 1 line
2.16.4
PATCH 1/2 xsk: do not use mmap_sem
Holding mmap_sem exclusively for a gup() is an overkill. Lets replace the call for gup_fast() and let the mm take it if necessary.
--- net/xdp/xdp_umem.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
net/xdp/xdp_umem.c +2 -4
@@ -265,10 +265,8 @@ static int xdp_umem_pin_pages(struct xdp_umem *umem)if (!umem->pgs)return -ENOMEM ; -down_write (¤t->mm->mmap_sem); - npgs =get_user_pages (umem->address, umem->npgs, - gup_flags, &umem->pgs[0 ],NULL ); -up_write (¤t->mm->mmap_sem); + npgs =get_user_pages_fast (umem->address, umem->npgs, + gup_flags, &umem->pgs[0 ]);if (npgs != umem->npgs) {if (npgs >=0 ) {
Signature · 1 line
2.16.4
Re: [PATCH 1/2] xsk: do not use mmap_sem
Den tors 7 feb. 2019 kl 06:38 skrev Davidlohr Bueso <[email protected]>:
Quoted 19 lines
Holding
mmap_semexclusively for agup()is an overkill. Lets replace the call forgup_fast()and let the mm take it if necessary.Cc David S. Miller <[email protected]>Cc Bjorn Topel <[email protected]>Cc Magnus Karlsson <[email protected]>Signed-off-by Davidlohr Bueso <[email protected]>--- net/xdp/xdp_umem.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)net/xdp/xdp_umem.c +0
-0
net/xdp/xdp_umem.c static int xdp_umem_pin_pages(struct xdp_umem *umem) · line 270
Quoted 10 lines
net/xdp/xdp_umem.c +2
-4@@ -265,10 +265,8 @@ static int xdp_umem_pin_pages(struct xdp_umem *umem)if (!umem->pgs)return -ENOMEM ; -down_write (¤t->mm->mmap_sem); - npgs =get_user_pages (umem->address, umem->npgs, - gup_flags, &umem->pgs[0 ],NULL ); -up_write (¤t->mm->mmap_sem); + npgs =get_user_pages_fast (umem->address, umem->npgs, + gup_flags, &umem->pgs[0 ]);
Thanks for the patch!
The lifetime of the pinning is similar to RDMA umem mapping, so isn't gup_longterm preferred?
Björn
Quoted 5 lines
net/xdp/xdp_umem.c +0
-1if (npgs != umem->npgs) {if (npgs >=0 ) { --2.16.4
Re: [PATCH 1/2] xsk: do not use mmap_sem
[ +Dan ]
Quoted 33 lines · Björn Töpel
Den tors 7 feb. 2019 kl 06:38 skrev Davidlohr Bueso <[email protected]>:
Quoted 19 lines
Holding
mmap_semexclusively for agup()is an overkill. Lets replace the call forgup_fast()and let the mm take it if necessary.Cc David S. Miller <[email protected]>Cc Bjorn Topel <[email protected]>Cc Magnus Karlsson <[email protected]>Signed-off-by Davidlohr Bueso <[email protected]>--- net/xdp/xdp_umem.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)net/xdp/xdp_umem.c +0
-0net/xdp/xdp_umem.c static int xdp_umem_pin_pages(struct xdp_umem *umem) · line 270
Quoted 10 lines
net/xdp/xdp_umem.c +2
-4@@ -265,10 +265,8 @@ static int xdp_umem_pin_pages(struct xdp_umem *umem)if (!umem->pgs)return -ENOMEM ; -down_write (¤t->mm->mmap_sem); - npgs =get_user_pages (umem->address, umem->npgs, - gup_flags, &umem->pgs[0 ],NULL ); -up_write (¤t->mm->mmap_sem); + npgs =get_user_pages_fast (umem->address, umem->npgs, + gup_flags, &umem->pgs[0 ]);Thanks for the patch!
Quoted 2 lines
The lifetime of the pinning is similar to RDMA umem mapping, so isn't
gup_longtermpreferred?
Seems reasonable from reading what gup_longterm seems to do. Davidlohr or Dan, any thoughts on the above?
Thanks,
Daniel
Re: [PATCH 1/2] xsk: do not use mmap_sem
[ add Ira ]
Quoted 37 lines · Daniel Borkmann
[ +Dan ]
Quoted 33 lines · Björn Töpel
Den tors 7 feb. 2019 kl 06:38 skrev Davidlohr Bueso <[email protected]>:
Quoted 19 lines
Holding
mmap_semexclusively for agup()is an overkill. Lets replace the call forgup_fast()and let the mm take it if necessary.Cc David S. Miller <[email protected]>Cc Bjorn Topel <[email protected]>Cc Magnus Karlsson <[email protected]>Signed-off-by Davidlohr Bueso <[email protected]>--- net/xdp/xdp_umem.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)net/xdp/xdp_umem.c +0
-0net/xdp/xdp_umem.c static int xdp_umem_pin_pages(struct xdp_umem *umem) · line 270
Quoted 10 lines
net/xdp/xdp_umem.c +2
-4@@ -265,10 +265,8 @@ static int xdp_umem_pin_pages(struct xdp_umem *umem)if (!umem->pgs)return -ENOMEM ; -down_write (¤t->mm->mmap_sem); - npgs =get_user_pages (umem->address, umem->npgs, - gup_flags, &umem->pgs[0 ],NULL ); -up_write (¤t->mm->mmap_sem); + npgs =get_user_pages_fast (umem->address, umem->npgs, + gup_flags, &umem->pgs[0 ]);Thanks for the patch!
Quoted 5 lines
Quoted 2 lines
The lifetime of the pinning is similar to RDMA umem mapping, so isn't
gup_longtermpreferred?Seems reasonable from reading what
gup_longtermseems to do. Davidlohr or Dan, any thoughts on the above?
Yes, any gup where the lifetime of the corresponding put_page() is under direct control of userspace should be using the _longterm flavor to coordinate be careful in the presence of dax. In the dax case there is no page cache indirection to coordinate truncate vs page pins. Ira is looking to add a "fast + longterm" flavor for cases like this.
RE: [PATCH 1/2] xsk: do not use mmap_sem
Quoted 26 lines
Quoted 9 lines
Quoted 9 lines
net/xdp/xdp_umem.c static int xdp_umem_pin_pages(struct
Quoted 9 lines
--- net/xdp/xdp_umem.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)net/xdp/xdp_umem.c index renamed from net/xdp/xdp_umem.c +0
-0
5ab236c5c9a5..25e1e76654a8100644net/xdp/xdp_umem.c +0
-0@@ -265,10 +265,8 @@ static int xdp_umem_pin_pages(struct
xdp_umem*umem)Quoted 16 lines
Quoted 15 lines
net/xdp/xdp_umem.c static int xdp_umem_pin_pages(struct xdp_umem *umem) · line 270
Quoted 10 lines
net/xdp/xdp_umem.c +2
-4if (!umem->pgs)return -ENOMEM ; -down_write (¤t->mm->mmap_sem); - npgs =get_user_pages (umem->address, umem->npgs, - gup_flags, &umem->pgs[0 ],NULL ); -up_write (¤t->mm->mmap_sem); + npgs =get_user_pages_fast (umem->address, umem->npgs, + gup_flags, &umem->pgs[0 ]);Thanks for the patch!
The lifetime of the pinning is similar to RDMA umem mapping, so isn't
gup_longtermpreferred?
Quoted 8 lines
Quoted 2 lines
Seems reasonable from reading what
gup_longtermseems to do. Davidlohr or Dan, any thoughts on the above?Yes, any gup where the lifetime of the corresponding
put_page()is under direct control of userspace should be using the_longtermflavor to coordinate be careful in the presence of dax. In the dax case there is no page cache indirection to coordinate truncate vs page pins. Ira is looking to add a "fast + longterm" flavor for cases like this.
Yes I'm just about ready with a small patch set to add a GUP fast longterm.
I think this should wait for that series.
Ira
PATCH v2 1/1 xsk: share the mmap_sem for page pinning
In reply to Davidlohr BuesoHolding mmap_sem exclusively for a gup() is an overkill. Lets share the lock and replace the gup call for gup_longterm(), as it is better suited for the lifetime of the pinning.
---net/xdp/xdp_umem.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
net/xdp/xdp_umem.c +0 -0
@@ -265,10 +265,10 @@ static int xdp_umem_pin_pages(struct xdp_umem *umem)
if (!umem->pgs) return -ENOMEM;
- down_write(¤t->mm->mmap_sem);
- npgs = get_user_pages(umem->address, umem->npgs,
+ down_read(¤t->mm->mmap_sem);
+ npgs = get_user_pages_longterm(umem->address, umem->npgs,
gup_flags, &umem->pgs[0], NULL);
- up_write(¤t->mm->mmap_sem);
+ up_read(¤t->mm->mmap_sem);
if (npgs != umem->npgs) {
if (npgs >= 0) {Signature · 1 line
2.16.4
Re: [PATCH v2] xsk: share the mmap_sem for page pinning
Den mån 11 feb. 2019 kl 17:15 skrev Davidlohr Bueso <[email protected]>:
Quoted 5 lines
Holding
mmap_semexclusively for agup()is an overkill. Lets share the lock and replace the gup call forgup_longterm(), as it is better suited for the lifetime of the pinning.
Thanks for the cleanup!
Quoted 30 lines
Cc David S. Miller <[email protected]>Cc Bjorn Topel <[email protected]>Cc Magnus Karlsson <[email protected]>Signed-off-by Davidlohr Bueso <[email protected]>--- net/xdp/xdp_umem.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)net/xdp/xdp_umem.c +3
-4@@ -265,10 +265,10 @@ static int xdp_umem_pin_pages(struct xdp_umem *umem)if (!umem->pgs)return -ENOMEM ; -down_write (¤t->mm->mmap_sem); - npgs =get_user_pages (umem->address, umem->npgs, +down_read (¤t->mm->mmap_sem); + npgs =get_user_pages_longterm (umem->address, umem->npgs, gup_flags, &umem->pgs[0 ],NULL ); -up_write (¤t->mm->mmap_sem); +up_read (¤t->mm->mmap_sem);if (npgs != umem->npgs) {if (npgs >=0 ) { --2.16.4
Daniel Borkmann
AppliedRe: [PATCH v2] xsk: share the mmap_sem for page pinning
In reply to Davidlohr BuesoQuoted 9 lines · Davidlohr Bueso
Holding
mmap_semexclusively for agup()is an overkill. Lets share the lock and replace the gup call forgup_longterm(), as it is better suited for the lifetime of the pinning.Cc David S. Miller <[email protected]>Cc Bjorn Topel <[email protected]>Cc Magnus Karlsson <[email protected]>Signed-off-by Davidlohr Bueso <[email protected]>
Applied, thanks!
PATCH 2/2 MIPS/c-r4k: do no use mmap_sem for gup_fast()
In reply to Davidlohr BuesoIt is well known that because the mm can internally call the regular gup_unlocked if the lockless approach fails and take the sem there, the caller must not hold the mmap_sem already.
--- arch/mips/mm/c-r4k.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-)
arch/mips/mm/c-r4k.c +1 -5
@@ -1034,11 +1034,9 @@ static void r4k_flush_cache_sigtramp(unsigned long addr)struct flush_cache_sigtramp_args args;int npages; -down_read (¤t->mm->mmap_sem); - npages =get_user_pages_fast (addr,1 ,0 , &args.page);if (npages <1 ) - goto out; +return ; args.mm = current->mm; args.addr = addr;
@@ -1046,8 +1044,6 @@ static void r4k_flush_cache_sigtramp(unsigned long addr)r4k_on_each_cpu (R4K_HIT , local_r4k_flush_cache_sigtramp, &args);put_page (args.page); -out: -up_read (¤t->mm->mmap_sem); }static void r4k_flush_icache_all (void )
Signature · 1 line
2.16.4
Re: [PATCH 2/2] MIPS/c-r4k: do no use mmap_sem for gup_fast()
Hi Davidlohr,
Quoted 11 lines · Davidlohr Bueso
It is well known that because the mm can internally call the regular
gup_unlockedif the lockless approach fails and take the sem there, the caller must not hold themmap_semalready.Fixes e523f289fe4d (MIPS: c-r4k: Fix sigtramp SMP call to use kmap)Cc Ralf Baechle <[email protected]>Cc Paul Burton <[email protected]>Cc James Hogan <[email protected]>Signed-off-by Davidlohr Bueso <[email protected]>
Thanks - this looks good, but:
1) The problem it fixes was introduced in v4.8.
2) Commit adcc81f148d7 ("MIPS: math-emu: Write-protect delay slot emulation pages") actually left flush_cache_sigtramp unused, and has been backported to stable kernels also as far as v4.8.
Therefore this will just fix code that never gets called, and I'll go delete the whole thing instead.
Thanks,
Paul
Re: [PATCH 2/2] MIPS/c-r4k: do no use mmap_sem for gup_fast()
Quoted 17 lines · Paul Burton
Hi Davidlohr,
Quoted 11 lines · Davidlohr Bueso
It is well known that because the mm can internally call the regular
gup_unlockedif the lockless approach fails and take the sem there, the caller must not hold themmap_semalready.Fixes e523f289fe4d (MIPS: c-r4k: Fix sigtramp SMP call to use kmap)Cc Ralf Baechle <[email protected]>Cc Paul Burton <[email protected]>Cc James Hogan <[email protected]>Signed-off-by Davidlohr Bueso <[email protected]>Thanks - this looks good, but:
Quoted 8 lines
1) The problem it fixes was introduced in v4.8.
2) Commit
adcc81f148d7("MIPS: math-emu: Write-protect delay slot emulation pages") actually leftflush_cache_sigtrampunused, and has been backported to stable kernels also as far as v4.8.Therefore this will just fix code that never gets called, and I'll go delete the whole thing instead.
Even better.
Thanks,
Davidlohr
Unlike what the subject says, this is not against -tip, it applies on today's -next.
Quoted 20 lines · Davidlohr Bueso
Hi,
Here are two more patchlets that cleanup
mmap_semand gup abusers. The second is also a fixlet.Compile-tested only. Please consider for v5.1
Thanks!
Davidlohr Bueso (2):
xsk: do not usemmap_sem
MIPS/c-r4k: do no usemmap_semforgup_fast()
arch/mips/mm/c-r4k.c| 6 +-----net/xdp/xdp_umem.c| 6 ++----2 files changed, 3 insertions(+), 9 deletions(-)Signature · 1 line
2.16.4