Skip to contentSkip to conversation

more get_user_pages mmap_sem cleanups

PATCH In review Davidlohr Bueso 6 participants 13 messages Feb 7, 2019 – Feb 11, 2019

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

cover more get_user_pages mmap_sem cleanups

Hi,

Here are two more patchlets that cleanup mmap_sem and gup abusers. The second is also a fixlet.

Compile-tested only. Please consider for v5.1

Thanks!

Davidlohr Bueso (2):
xsk: do not use mmap_sem
MIPS/c-r4k: do no use mmap_sem for gup_fast()

 arch/mips/mm/c-r4k.c | 6 +-----
 net/xdp/xdp_umem.c   | 6 ++----
 2 files changed, 3 insertions(+), 9 deletions(-)
Signature · 1 line
2.16.4
#1 lore raw

PATCH 1/2 xsk: do not use mmap_sem

Holding mmap_sem exclusively for a gup() is an overkill. Lets replace the call for gup_fast() and let the mm take it if necessary.

Cc David S. Miller <[email protected]>
Cc Bjorn Topel <[email protected]>
Cc Magnus Karlsson <[email protected]>
---
 net/xdp/xdp_umem.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)
net/xdp/xdp_umem.c +2 -4
index 5ab236c5c9a5..25e1e76654a8 100644
@@ -265,10 +265,8 @@ static int xdp_umem_pin_pages(struct xdp_umem *umem)
 	if (!umem->pgs)
 		return -ENOMEM;
 
-	down_write(&current->mm->mmap_sem);
-	npgs = get_user_pages(umem->address, umem->npgs,
-			      gup_flags, &umem->pgs[0], NULL);
-	up_write(&current->mm->mmap_sem);
+	npgs = get_user_pages_fast(umem->address, umem->npgs,
+				   gup_flags, &umem->pgs[0]);
 
 	if (npgs != umem->npgs) {
 		if (npgs >= 0) {
Signature · 1 line
2.16.4
#2 lore raw

Re: [PATCH 1/2] xsk: do not use mmap_sem

Den tors 7 feb. 2019 kl 06:38 skrev Davidlohr Bueso <[email protected]>:

Quoted 19 lines

Holding mmap_sem exclusively for a gup() is an overkill. Lets replace the call for gup_fast() and let the mm take it if necessary.

Cc David S. Miller <[email protected]>
Cc Bjorn Topel <[email protected]>
Cc Magnus Karlsson <[email protected]>
---
 net/xdp/xdp_umem.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)
net/xdp/xdp_umem.c +0 -0
index 5ab236c5c9a5..25e1e76654a8 100644

net/xdp/xdp_umem.c static int xdp_umem_pin_pages(struct xdp_umem *umem) · line 270

Quoted 10 lines
net/xdp/xdp_umem.c +2 -4
@@ -265,10 +265,8 @@ static int xdp_umem_pin_pages(struct xdp_umem *umem)
        if (!umem->pgs)
                return -ENOMEM;
 
-       down_write(&current->mm->mmap_sem);
-       npgs = get_user_pages(umem->address, umem->npgs,
-                             gup_flags, &umem->pgs[0], NULL);
-       up_write(&current->mm->mmap_sem);
+       npgs = get_user_pages_fast(umem->address, umem->npgs,
+                                  gup_flags, &umem->pgs[0]);
 

Thanks for the patch!

The lifetime of the pinning is similar to RDMA umem mapping, so isn't gup_longterm preferred?

Björn

Quoted 5 lines
net/xdp/xdp_umem.c +0 -1
        if (npgs != umem->npgs) {
                if (npgs >= 0) {
--

2.16.4

#3 lore raw

Re: [PATCH 1/2] xsk: do not use mmap_sem

[ +Dan ]

Quoted 33 lines · Björn Töpel

Den tors 7 feb. 2019 kl 06:38 skrev Davidlohr Bueso <[email protected]>:

Quoted 19 lines

Holding mmap_sem exclusively for a gup() is an overkill. Lets replace the call for gup_fast() and let the mm take it if necessary.

Cc David S. Miller <[email protected]>
Cc Bjorn Topel <[email protected]>
Cc Magnus Karlsson <[email protected]>
---
 net/xdp/xdp_umem.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)
net/xdp/xdp_umem.c +0 -0
index 5ab236c5c9a5..25e1e76654a8 100644

net/xdp/xdp_umem.c static int xdp_umem_pin_pages(struct xdp_umem *umem) · line 270

Quoted 10 lines
net/xdp/xdp_umem.c +2 -4
@@ -265,10 +265,8 @@ static int xdp_umem_pin_pages(struct xdp_umem *umem)
        if (!umem->pgs)
                return -ENOMEM;
 
-       down_write(&current->mm->mmap_sem);
-       npgs = get_user_pages(umem->address, umem->npgs,
-                             gup_flags, &umem->pgs[0], NULL);
-       up_write(&current->mm->mmap_sem);
+       npgs = get_user_pages_fast(umem->address, umem->npgs,
+                                  gup_flags, &umem->pgs[0]);
 

Thanks for the patch!

Quoted 2 lines

The lifetime of the pinning is similar to RDMA umem mapping, so isn't gup_longterm preferred?

Seems reasonable from reading what gup_longterm seems to do. Davidlohr or Dan, any thoughts on the above?

Thanks,
Daniel

#4 lore raw

Re: [PATCH 1/2] xsk: do not use mmap_sem

[ add Ira ]

Quoted 37 lines · Daniel Borkmann

[ +Dan ]

Quoted 33 lines · Björn Töpel

Den tors 7 feb. 2019 kl 06:38 skrev Davidlohr Bueso <[email protected]>:

Quoted 19 lines

Holding mmap_sem exclusively for a gup() is an overkill. Lets replace the call for gup_fast() and let the mm take it if necessary.

Cc David S. Miller <[email protected]>
Cc Bjorn Topel <[email protected]>
Cc Magnus Karlsson <[email protected]>
---
 net/xdp/xdp_umem.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)
net/xdp/xdp_umem.c +0 -0
index 5ab236c5c9a5..25e1e76654a8 100644

net/xdp/xdp_umem.c static int xdp_umem_pin_pages(struct xdp_umem *umem) · line 270

Quoted 10 lines
net/xdp/xdp_umem.c +2 -4
@@ -265,10 +265,8 @@ static int xdp_umem_pin_pages(struct xdp_umem *umem)
        if (!umem->pgs)
                return -ENOMEM;
 
-       down_write(&current->mm->mmap_sem);
-       npgs = get_user_pages(umem->address, umem->npgs,
-                             gup_flags, &umem->pgs[0], NULL);
-       up_write(&current->mm->mmap_sem);
+       npgs = get_user_pages_fast(umem->address, umem->npgs,
+                                  gup_flags, &umem->pgs[0]);
 

Thanks for the patch!

Quoted 5 lines
Quoted 2 lines

The lifetime of the pinning is similar to RDMA umem mapping, so isn't gup_longterm preferred?

Seems reasonable from reading what gup_longterm seems to do. Davidlohr or Dan, any thoughts on the above?

Yes, any gup where the lifetime of the corresponding put_page() is under direct control of userspace should be using the _longterm flavor to coordinate be careful in the presence of dax. In the dax case there is no page cache indirection to coordinate truncate vs page pins. Ira is looking to add a "fast + longterm" flavor for cases like this.

#5 lore raw

RE: [PATCH 1/2] xsk: do not use mmap_sem

Quoted 26 lines
Quoted 9 lines
Quoted 9 lines

net/xdp/xdp_umem.c static int xdp_umem_pin_pages(struct

Quoted 9 lines
---
 net/xdp/xdp_umem.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)
net/xdp/xdp_umem.c index renamed from net/xdp/xdp_umem.c +0 -0

5ab236c5c9a5..25e1e76654a8 100644

net/xdp/xdp_umem.c +0 -0
@@ -265,10 +265,8 @@ static int xdp_umem_pin_pages(struct

xdp_umem *umem)

Quoted 16 lines
Quoted 15 lines

net/xdp/xdp_umem.c static int xdp_umem_pin_pages(struct xdp_umem *umem) · line 270

Quoted 10 lines
net/xdp/xdp_umem.c +2 -4
        if (!umem->pgs)
                return -ENOMEM;
 
-       down_write(&current->mm->mmap_sem);
-       npgs = get_user_pages(umem->address, umem->npgs,
-                             gup_flags, &umem->pgs[0], NULL);
-       up_write(&current->mm->mmap_sem);
+       npgs = get_user_pages_fast(umem->address, umem->npgs,
+                                  gup_flags, &umem->pgs[0]);
 

Thanks for the patch!

The lifetime of the pinning is similar to RDMA umem mapping, so isn't gup_longterm preferred?

Quoted 8 lines
Quoted 2 lines

Seems reasonable from reading what gup_longterm seems to do. Davidlohr or Dan, any thoughts on the above?

Yes, any gup where the lifetime of the corresponding put_page() is under direct control of userspace should be using the _longterm flavor to coordinate be careful in the presence of dax. In the dax case there is no page cache indirection to coordinate truncate vs page pins. Ira is looking to add a "fast + longterm" flavor for cases like this.

Yes I'm just about ready with a small patch set to add a GUP fast longterm.

I think this should wait for that series.

Ira

#6 lore raw

PATCH v2 1/1 xsk: share the mmap_sem for page pinning

In reply to Davidlohr Bueso

Holding mmap_sem exclusively for a gup() is an overkill. Lets share the lock and replace the gup call for gup_longterm(), as it is better suited for the lifetime of the pinning.

Cc David S. Miller <[email protected]>
Cc Bjorn Topel <[email protected]>
Cc Magnus Karlsson <[email protected]>

---
net/xdp/xdp_umem.c | 6 +++---

1 file changed, 3 insertions(+), 3 deletions(-)
net/xdp/xdp_umem.c +0 -0
index 5ab236c5c9a5..e7fa8d0d7090 100644
@@ -265,10 +265,10 @@ static int xdp_umem_pin_pages(struct xdp_umem *umem)
	if (!umem->pgs)
		return -ENOMEM;

- down_write(&current->mm->mmap_sem);
- npgs = get_user_pages(umem->address, umem->npgs,
+ down_read(&current->mm->mmap_sem);
+ npgs = get_user_pages_longterm(umem->address, umem->npgs,

			      gup_flags, &umem->pgs[0], NULL);

- up_write(&current->mm->mmap_sem);
+ up_read(&current->mm->mmap_sem);

	if (npgs != umem->npgs) {
		if (npgs >= 0) {
Signature · 1 line
2.16.4
Acked-by on v2 1/1
#7 lore raw

Re: [PATCH v2] xsk: share the mmap_sem for page pinning

Den mån 11 feb. 2019 kl 17:15 skrev Davidlohr Bueso <[email protected]>:

Quoted 5 lines

Holding mmap_sem exclusively for a gup() is an overkill. Lets share the lock and replace the gup call for gup_longterm(), as it is better suited for the lifetime of the pinning.

Thanks for the cleanup!

Quoted 30 lines
Cc David S. Miller <[email protected]>
Cc Bjorn Topel <[email protected]>
Cc Magnus Karlsson <[email protected]>
---
 net/xdp/xdp_umem.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
net/xdp/xdp_umem.c +3 -4
index 5ab236c5c9a5..e7fa8d0d7090 100644
@@ -265,10 +265,10 @@ static int xdp_umem_pin_pages(struct xdp_umem *umem)
        if (!umem->pgs)
                return -ENOMEM;
 
-       down_write(&current->mm->mmap_sem);
-       npgs = get_user_pages(umem->address, umem->npgs,
+       down_read(&current->mm->mmap_sem);
+       npgs = get_user_pages_longterm(umem->address, umem->npgs,
                              gup_flags, &umem->pgs[0], NULL);
-       up_write(&current->mm->mmap_sem);
+       up_read(&current->mm->mmap_sem);
 
        if (npgs != umem->npgs) {
                if (npgs >= 0) {
--

2.16.4

#8 lore raw

Re: [PATCH v2] xsk: share the mmap_sem for page pinning

In reply to Davidlohr Bueso
Quoted 9 lines · Davidlohr Bueso

Holding mmap_sem exclusively for a gup() is an overkill. Lets share the lock and replace the gup call for gup_longterm(), as it is better suited for the lifetime of the pinning.

Cc David S. Miller <[email protected]>
Cc Bjorn Topel <[email protected]>
Cc Magnus Karlsson <[email protected]>

Applied, thanks!

#9 lore raw

PATCH 2/2 MIPS/c-r4k: do no use mmap_sem for gup_fast()

In reply to Davidlohr Bueso

It is well known that because the mm can internally call the regular gup_unlocked if the lockless approach fails and take the sem there, the caller must not hold the mmap_sem already.

Fixes e523f289fe4d (MIPS: c-r4k: Fix sigtramp SMP call to use kmap)
Cc Ralf Baechle <[email protected]>
Cc Paul Burton <[email protected]>
Cc James Hogan <[email protected]>
---
 arch/mips/mm/c-r4k.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)
arch/mips/mm/c-r4k.c +1 -5
index cc4e17caeb26..38fe86928837 100644
@@ -1034,11 +1034,9 @@ static void r4k_flush_cache_sigtramp(unsigned long addr)
 	struct flush_cache_sigtramp_args args;
 	int npages;
 
-	down_read(&current->mm->mmap_sem);
-
 	npages = get_user_pages_fast(addr, 1, 0, &args.page);
 	if (npages < 1)
-		goto out;
+		return;
 
 	args.mm = current->mm;
 	args.addr = addr;
@@ -1046,8 +1044,6 @@ static void r4k_flush_cache_sigtramp(unsigned long addr)
 	r4k_on_each_cpu(R4K_HIT, local_r4k_flush_cache_sigtramp, &args);
 
 	put_page(args.page);
-out:
-	up_read(&current->mm->mmap_sem);
 }
 
 static void r4k_flush_icache_all(void)
Signature · 1 line
2.16.4
#10 lore raw

Re: [PATCH 2/2] MIPS/c-r4k: do no use mmap_sem for gup_fast()

Hi Davidlohr,

Quoted 11 lines · Davidlohr Bueso

It is well known that because the mm can internally call the regular gup_unlocked if the lockless approach fails and take the sem there, the caller must not hold the mmap_sem already.

Fixes e523f289fe4d (MIPS: c-r4k: Fix sigtramp SMP call to use kmap)
Cc Ralf Baechle <[email protected]>
Cc Paul Burton <[email protected]>
Cc James Hogan <[email protected]>

Thanks - this looks good, but:

1) The problem it fixes was introduced in v4.8.

2) Commit adcc81f148d7 ("MIPS: math-emu: Write-protect delay slot emulation pages") actually left flush_cache_sigtramp unused, and has been backported to stable kernels also as far as v4.8.

Therefore this will just fix code that never gets called, and I'll go delete the whole thing instead.

Thanks,
Paul

#11 lore raw

Re: [PATCH 2/2] MIPS/c-r4k: do no use mmap_sem for gup_fast()

Quoted 17 lines · Paul Burton

Hi Davidlohr,

Quoted 11 lines · Davidlohr Bueso

It is well known that because the mm can internally call the regular gup_unlocked if the lockless approach fails and take the sem there, the caller must not hold the mmap_sem already.

Fixes e523f289fe4d (MIPS: c-r4k: Fix sigtramp SMP call to use kmap)
Cc Ralf Baechle <[email protected]>
Cc Paul Burton <[email protected]>
Cc James Hogan <[email protected]>

Thanks - this looks good, but:

Quoted 8 lines

1) The problem it fixes was introduced in v4.8.

2) Commit adcc81f148d7 ("MIPS: math-emu: Write-protect delay slot emulation pages") actually left flush_cache_sigtramp unused, and has been backported to stable kernels also as far as v4.8.

Therefore this will just fix code that never gets called, and I'll go delete the whole thing instead.

Even better.

Thanks,
Davidlohr

#12 lore raw
In reply to Davidlohr Bueso

Unlike what the subject says, this is not against -tip, it applies on today's -next.

Quoted 20 lines · Davidlohr Bueso

Hi,

Here are two more patchlets that cleanup mmap_sem and gup abusers. The second is also a fixlet.

Compile-tested only. Please consider for v5.1

Thanks!

Davidlohr Bueso (2):
xsk: do not use mmap_sem
MIPS/c-r4k: do no use mmap_sem for gup_fast()

arch/mips/mm/c-r4k.c | 6 +-----
net/xdp/xdp_umem.c | 6 ++----

2 files changed, 3 insertions(+), 9 deletions(-)
Signature · 1 line
2.16.4