Skip to contentSkip to conversation

xen/privcmd: Convert get_user_pages*() to pin_user_pages*()

RFC PATCH Superseded Souptick Joarder 4 participants 9 messages Jun 17, 2020 – Jun 22, 2020 Xen hypervisor interface

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 xen/privcmd: Convert get_user_pages*() to pin_user_pages*()

In 2019, we introduced pin_user_pages*() and now we are converting get_user_pages*() to the new API as appropriate. [1] & [2] could be referred for more information.

[1] Documentation/core-api/pin_user_pages.rst

[2] "Explicit pinning of user-space pages":

        https://lwn.net/Articles/807108/

---
Hi,

I have compile tested this patch but unable to run-time test, so any testing help is much appriciated.

Also have a question, why the existing code is not marking the pages dirty (since it did FOLL_WRITE) ?

 drivers/xen/privcmd.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)
drivers/xen/privcmd.c +2 -5
index a250d11..543739e 100644
@@ -594,7 +594,7 @@ static int lock_pages(
 		if (requested > nr_pages)
 			return -ENOSPC;
 
-		pinned = get_user_pages_fast(
+		pinned = pin_user_pages_fast(
 			(unsigned long) kbufs[i].uptr,
 			requested, FOLL_WRITE, pages);
 		if (pinned < 0)
@@ -614,10 +614,7 @@ static void unlock_pages(struct page *pages[], unsigned int nr_pages)
 	if (!pages)
 		return;
 
-	for (i = 0; i < nr_pages; i++) {
-		if (pages[i])
-			put_page(pages[i]);
-	}
+	unpin_user_pages(pages, nr_pages);
 }
 
 static long privcmd_ioctl_dm_op(struct file *file, void __user *udata)
Signature · 1 line
1.9.1
#1 lore raw
Quoted 11 lines · Souptick Joarder

In 2019, we introduced pin_user_pages*() and now we are converting get_user_pages*() to the new API as appropriate. [1] & [2] could be referred for more information.

[1] Documentation/core-api/pin_user_pages.rst

[2] "Explicit pinning of user-space pages":

        https://lwn.net/Articles/807108/
Quoted 8 lines

---
Hi,

I have compile tested this patch but unable to run-time test, so any testing help is much appriciated.

Also have a question, why the existing code is not marking the pages dirty (since it did FOLL_WRITE) ?

Indeed, seems to me it should. Paul?

Quoted 18 lines
 drivers/xen/privcmd.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)
drivers/xen/privcmd.c +1 -1
index a250d11..543739e 100644
@@ -594,7 +594,7 @@ static int lock_pages(
 		if (requested > nr_pages)
 			return -ENOSPC;
 
-		pinned = get_user_pages_fast(
+		pinned = pin_user_pages_fast(
 			(unsigned long) kbufs[i].uptr,
 			requested, FOLL_WRITE, pages);
 		if (pinned < 0)

drivers/xen/privcmd.c static void unlock_pages(struct page *pages[], unsigned int nr_pages) · line 617

Quoted 8 lines
drivers/xen/privcmd.c +1 -4
@@ -614,10 +614,7 @@ static void unlock_pages(struct page *pages[], unsigned int nr_pages)
 	if (!pages)
 		return;
 
-	for (i = 0; i < nr_pages; i++) {
-		if (pages[i])
-			put_page(pages[i]);
-	}
+	unpin_user_pages(pages, nr_pages);

Why are you no longer checking for valid pages?

-boris

#2 lore raw
Quoted 47 lines · Boris Ostrovsky
Quoted 11 lines · Souptick Joarder

In 2019, we introduced pin_user_pages*() and now we are converting get_user_pages*() to the new API as appropriate. [1] & [2] could be referred for more information.

[1] Documentation/core-api/pin_user_pages.rst

[2] "Explicit pinning of user-space pages":

        https://lwn.net/Articles/807108/
Quoted 8 lines

---
Hi,

I have compile tested this patch but unable to run-time test, so any testing help is much appriciated.

Also have a question, why the existing code is not marking the pages dirty (since it did FOLL_WRITE) ?

Indeed, seems to me it should. Paul?

Quoted 21 lines
 drivers/xen/privcmd.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)
drivers/xen/privcmd.c +1 -1
index a250d11..543739e 100644
@@ -594,7 +594,7 @@ static int lock_pages(
              if (requested > nr_pages)
                      return -ENOSPC;
 
-             pinned = get_user_pages_fast(
+             pinned = pin_user_pages_fast(
                      (unsigned long) kbufs[i].uptr,
                      requested, FOLL_WRITE, pages);
              if (pinned < 0)
@@ -614,10 +614,7 @@ static void unlock_pages(struct page *pages[], unsigned int nr_pages)
      if (!pages)
              return;
 
Quoted 8 lines

drivers/xen/privcmd.c static void unlock_pages(struct page *pages[], unsigned int nr_pages) · line 617

Quoted 5 lines
drivers/xen/privcmd.c +1 -4
-     for (i = 0; i < nr_pages; i++) {
-             if (pages[i])
-                     put_page(pages[i]);
-     }
+     unpin_user_pages(pages, nr_pages);

Why are you no longer checking for valid pages?

My understanding is, in case of lock_pages() end up returning partial mapped pages,
we should pass no. of partial mapped pages to unlock_pages(), not nr_pages. This will avoid checking extra check to validate the pages[i].

and if lock_pages() returns 0 in success, anyway we have all the pages[i] valid. I will try to correct it in v2.

But I agree, there is no harm to check for pages[i] and I believe, unpin_user_pages()
is the right place to do so.

John any thought ?

#3 lore raw
Quoted 7 lines · Souptick Joarder
Quoted 5 lines · Boris Ostrovsky
Quoted 3 lines · Souptick Joarder

In 2019, we introduced pin_user_pages*() and now we are converting get_user_pages*() to the new API as appropriate. [1] & [2] could be referred for more information.

Ideally, the commit description should say which case, in pin_user_pages.rst, that this is.

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

[1] Documentation/core-api/pin_user_pages.rst

[2] "Explicit pinning of user-space pages":

         https://lwn.net/Articles/807108/

---
Hi,

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

I have compile tested this patch but unable to run-time test, so any testing help is much appriciated.

Also have a question, why the existing code is not marking the pages dirty (since it did FOLL_WRITE) ?

Indeed, seems to me it should. Paul?

Definitely good to get an answer from an expert in this code, but meanwhile, it's reasonable to just mark them dirty. Below...

Quoted 23 lines
Quoted 23 lines
Quoted 23 lines


  drivers/xen/privcmd.c | 7 ++-----
  1 file changed, 2 insertions(+), 5 deletions(-)
drivers/xen/privcmd.c +1 -1
index a250d11..543739e 100644
@@ -594,7 +594,7 @@ static int lock_pages(
               if (requested > nr_pages)
                       return -ENOSPC;
 
-             pinned = get_user_pages_fast(
+             pinned = pin_user_pages_fast(
                       (unsigned long) kbufs[i].uptr,
                       requested, FOLL_WRITE, pages);
               if (pinned < 0)
@@ -614,10 +614,7 @@ static void unlock_pages(struct page *pages[], unsigned int nr_pages)
       if (!pages)
               return;
 
Quoted 5 lines
Quoted 5 lines

drivers/xen/privcmd.c static void unlock_pages(struct page *pages[], unsigned int nr_pages) · line 617

Quoted 5 lines
drivers/xen/privcmd.c +1 -4
-     for (i = 0; i < nr_pages; i++) {
-             if (pages[i])
-                     put_page(pages[i]);
-     }
+     unpin_user_pages(pages, nr_pages);

...so just use unpin_user_pages_dirty_lock() here, I think.

Quoted 9 lines
Quoted 3 lines

Why are you no longer checking for valid pages?

My understanding is, in case of lock_pages() end up returning partial mapped pages,
we should pass no. of partial mapped pages to unlock_pages(), not nr_pages. This will avoid checking extra check to validate the pages[i].

Quoted 4 lines

and if lock_pages() returns 0 in success, anyway we have all the pages[i] valid. I will try to correct it in v2.

But I agree, there is no harm to check for pages[i] and I believe,

Generally, it *is* harmful to do unnecessary checks, in most code, but especially in most kernel code. If you can convince yourself that the check for null pages is redundant here, then please let's remove that check. The code becomes then becomes shorter, simpler, and faster.

Quoted 4 lines

unpin_user_pages()
is the right place to do so.

John any thought ?

So far I haven't seen any cases to justify changing the implementation of unpin_user_pages().

thanks,

Signature · 2 lines
John Hubbard
NVIDIA
#8 lore raw
In reply to Boris Ostrovsky
Quoted 23 lines

-----Original Message-----
From: Boris Ostrovsky <[email protected]>
Sent: 17 June 2020 18:57
To: Souptick Joarder <[email protected]>; [email protected]; [email protected]
Cc: [email protected]; [email protected]; John Hubbard <[email protected]>; [email protected]
Subject: Re: [RFC PATCH] xen/privcmd: Convert get_user_pages*() to pin_user_pages*()

Quoted 14 lines · Souptick Joarder

In 2019, we introduced pin_user_pages*() and now we are converting get_user_pages*() to the new API as appropriate. [1] & [2] could be referred for more information.

[1] Documentation/core-api/pin_user_pages.rst

[2] "Explicit pinning of user-space pages":

        https://lwn.net/Articles/807108/

---
Hi,

Quoted 9 lines
Quoted 5 lines

I have compile tested this patch but unable to run-time test, so any testing help is much appriciated.

Also have a question, why the existing code is not marking the pages dirty (since it did FOLL_WRITE) ?

Indeed, seems to me it should. Paul?

Yes, it looks like that was an oversight. The hypercall may well result in data being copied back into the buffers so the whole pages array should be considered dirty.

Paul

Quoted 36 lines
Quoted 19 lines

 drivers/xen/privcmd.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)
drivers/xen/privcmd.c +1 -1
index a250d11..543739e 100644
@@ -594,7 +594,7 @@ static int lock_pages(
 		if (requested > nr_pages)
 			return -ENOSPC;
 
-		pinned = get_user_pages_fast(
+		pinned = pin_user_pages_fast(
 			(unsigned long) kbufs[i].uptr,
 			requested, FOLL_WRITE, pages);
 		if (pinned < 0)

drivers/xen/privcmd.c static void unlock_pages(struct page *pages[], unsigned int nr_pages) · line 617

Quoted 8 lines
drivers/xen/privcmd.c +1 -4
@@ -614,10 +614,7 @@ static void unlock_pages(struct page *pages[], unsigned int nr_pages)
 	if (!pages)
 		return;
 
-	for (i = 0; i < nr_pages; i++) {
-		if (pages[i])
-			put_page(pages[i]);
-	}
+	unpin_user_pages(pages, nr_pages);

Why are you no longer checking for valid pages?

-boris

#4 lore raw
In reply to John Hubbard
Quoted 5 lines · John Hubbard
Quoted 3 lines · Souptick Joarder
Quoted 1 line · Boris Ostrovsky
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines · Souptick Joarder
Quoted 3 lines

In 2019, we introduced pin_user_pages*() and now we are converting get_user_pages*() to the new API as appropriate. [1] & [2] could be referred for more information.

Ideally, the commit description should say which case, in pin_user_pages.rst, that this is.

Ok.

Quoted 69 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines

[1] Documentation/core-api/pin_user_pages.rst

[2] "Explicit pinning of user-space pages":

         https://lwn.net/Articles/807108/

---
Hi,

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

I have compile tested this patch but unable to run-time test, so any testing help is much appriciated.

Also have a question, why the existing code is not marking the pages dirty (since it did FOLL_WRITE) ?

Indeed, seems to me it should. Paul?

Definitely good to get an answer from an expert in this code, but meanwhile, it's reasonable to just mark them dirty. Below...

Quoted 23 lines
Quoted 23 lines
Quoted 23 lines


  drivers/xen/privcmd.c | 7 ++-----
  1 file changed, 2 insertions(+), 5 deletions(-)
drivers/xen/privcmd.c +1 -1
index a250d11..543739e 100644
@@ -594,7 +594,7 @@ static int lock_pages(
               if (requested > nr_pages)
                       return -ENOSPC;
 
-             pinned = get_user_pages_fast(
+             pinned = pin_user_pages_fast(
                       (unsigned long) kbufs[i].uptr,
                       requested, FOLL_WRITE, pages);
               if (pinned < 0)
@@ -614,10 +614,7 @@ static void unlock_pages(struct page *pages[], unsigned int nr_pages)
       if (!pages)
               return;
 
Quoted 5 lines
Quoted 5 lines

drivers/xen/privcmd.c static void unlock_pages(struct page *pages[], unsigned int nr_pages) · line 617

Quoted 5 lines
drivers/xen/privcmd.c +1 -4
-     for (i = 0; i < nr_pages; i++) {
-             if (pages[i])
-                     put_page(pages[i]);
-     }
+     unpin_user_pages(pages, nr_pages);

...so just use unpin_user_pages_dirty_lock() here, I think.

Quoted 12 lines
Quoted 3 lines

Why are you no longer checking for valid pages?

My understanding is, in case of lock_pages() end up returning partial mapped pages,
we should pass no. of partial mapped pages to unlock_pages(), not nr_pages. This will avoid checking extra check to validate the pages[i].

and if lock_pages() returns 0 in success, anyway we have all the pages[i] valid. I will try to correct it in v2.

Quoted 7 lines
Quoted 1 line

But I agree, there is no harm to check for pages[i] and I believe,

Generally, it *is* harmful to do unnecessary checks, in most code, but especially in most kernel code. If you can convince yourself that the check for null pages is redundant here, then please let's remove that check. The code becomes then becomes shorter, simpler, and faster.

I read the code again. I think, this check is needed to handle a scenario when lock_pages() return -ENOSPC. Better to keep this check. Let me post v2 of this RFC for a clear view.

Quoted 16 lines
Quoted 6 lines

unpin_user_pages()
is the right place to do so.

John any thought ?

So far I haven't seen any cases to justify changing the implementation of unpin_user_pages().

thanks,
--
John Hubbard
NVIDIA

#5 lore raw
Quoted 4 lines · Souptick Joarder

I read the code again. I think, this check is needed to handle a scenario when lock_pages() return -ENOSPC. Better to keep this check. Let me post v2 of this RFC for a clear view.

Actually, error handling seems to be somewhat broken here. If lock_pages() returns number of pinned pages then that's what we end up returning from privcmd_ioctl_dm_op(), all the way to user ioctl(). Which I don't think is right, we should return proper (negative) error.

Do you mind fixing that we well? Then you should be able to avoid testing pages in a loop.

-boris

#6 lore raw
Quoted 8 lines · Boris Ostrovsky
Quoted 4 lines · Souptick Joarder

I read the code again. I think, this check is needed to handle a scenario when lock_pages() return -ENOSPC. Better to keep this check. Let me post v2 of this RFC for a clear view.

Quoted 5 lines

Actually, error handling seems to be somewhat broken here. If lock_pages() returns number of pinned pages then that's what we end up returning from privcmd_ioctl_dm_op(), all the way to user ioctl(). Which I don't think is right, we should return proper (negative) error.

What -ERRNO is more appropriate here ? -ENOSPC ?

Quoted 3 lines

Do you mind fixing that we well? Then you should be able to avoid testing pages in a loop.

Ok, let me try to fix it.

Quoted 4 lines

-boris

#7 lore raw
Quoted 7 lines · Souptick Joarder
Quoted 5 lines · Boris Ostrovsky
Quoted 3 lines · Souptick Joarder

I read the code again. I think, this check is needed to handle a scenario when lock_pages() return -ENOSPC. Better to keep this check. Let me post v2 of this RFC for a clear view.

Quoted 6 lines
Quoted 5 lines

Actually, error handling seems to be somewhat broken here. If lock_pages() returns number of pinned pages then that's what we end up returning from privcmd_ioctl_dm_op(), all the way to user ioctl(). Which I don't think is right, we should return proper (negative) error.

What -ERRNO is more appropriate here ? -ENOSPC ?

You can simply pass along error code that get_user_pages_fast() returned.

-boris