Skip to contentSkip to conversation

move_pages.2: not return ENOENT if the page are already on the target nodes

PATCH In review Yang Shi 7 participants 16 messages Dec 6, 2019 – Jan 2, 2020

Subsystems (3)
lore
Share & tools
lore

Yang Shi

author
#0 lore raw

PATCH 1/1 move_pages.2: not return ENOENT if the page are already on the target nodes

Since commit e78bbfa82624 ("mm: stop returning -ENOENT
from sys_move_pages() if nothing got migrated"), move_pages doesn't return -ENOENT anymore if the pages are already on the target nodes, but this change is never reflected in manpage.

Cc Michael Kerrisk <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc John Hubbard <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Qian Cai <[email protected]>
---
 man2/move_pages.2 | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
man2/move_pages.2 +2 -3
index 2d96468..2a2f3cd 100644
@@ -192,9 +192,8 @@ was specified or an attempt was made to migrate pages of a kernel thread.
 One of the target nodes is not online.
 .TP
 .B ENOENT
-No pages were found that require moving.
-All pages are either already
-on the target node, not present, had an invalid address or could not be
+No pages were found.
+All pages are either not present, had an invalid address or could not be
 moved because they were mapped by multiple processes.
 .TP
 .B EPERM
Signature · 1 line
1.8.3.1
#1 lore raw

On 12/5/19 5:34 PM, Yang Shi wrote:
...

Quoted 6 lines
man2/move_pages.2 +0 -0
index 2d96468..2a2f3cd 100644

man2/move_pages.2 was specified or an attempt was made to migrate pages of a kernel thread. · line 197

Quoted 9 lines
man2/move_pages.2 +2 -3
@@ -192,9 +192,8 @@ was specified or an attempt was made to migrate pages of a kernel thread.
 One of the target nodes is not online.
 .TP
 .B ENOENT
-No pages were found that require moving.
-All pages are either already
-on the target node, not present, had an invalid address or could not be
+No pages were found.
+All pages are either not present, had an invalid address or could not be
 moved because they were mapped by multiple processes.

How about this wording (ignoring man formatting for the moment):

No pages were moved, because all requested pages fell into one or more of the following cases:

* Page not present.
* Page has an invalid address.
* Page is mapped by multiple processes.

Reasoning: I don't like the "no pages were found" all by itself, because it blindly rewords the meaning of ENOENT. ENOENT is merely the closest symbol we have. So we use ENOENT and that's fine, but the descriptive text should describe what really happened, which is "no pages were moved". If we had an ENOPAGESMOVED then we'd use that. :)

thanks,

Signature · 2 lines
John Hubbard
NVIDIA
#2 lore raw
In reply to Yang Shi
Quoted 20 lines · Yang Shi

Since commit e78bbfa82624 ("mm: stop returning -ENOENT
from sys_move_pages() if nothing got migrated"), move_pages doesn't return -ENOENT anymore if the pages are already on the target nodes, but this change is never reflected in manpage.

Cc Michael Kerrisk <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc John Hubbard <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Qian Cai <[email protected]>
---
 man2/move_pages.2 | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
man2/move_pages.2 +0 -0
index 2d96468..2a2f3cd 100644

man2/move_pages.2 was specified or an attempt was made to migrate pages of a kernel thread. · line 197

Quoted 9 lines
man2/move_pages.2 +2 -3
@@ -192,9 +192,8 @@ was specified or an attempt was made to migrate pages of a kernel thread.
 One of the target nodes is not online.
 .TP
 .B ENOENT
-No pages were found that require moving.
-All pages are either already
-on the target node, not present, had an invalid address or could not be
+No pages were found.
+All pages are either not present, had an invalid address or could not be
 moved because they were mapped by multiple processes.

I would rather not put any specifics here because those reasons might differ in future. I would simply go with "No pages were found that require or could be moved."

Signature · 2 lines
Michal Hocko
SUSE Labs
#3 lore raw
In reply to Yang Shi
Quoted 22 lines · Yang Shi

Since commit e78bbfa82624 ("mm: stop returning -ENOENT
from sys_move_pages() if nothing got migrated"), move_pages doesn't return -ENOENT anymore if the pages are already on the target nodes, but this change is never reflected in manpage.

Cc Michael Kerrisk <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc John Hubbard <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Qian Cai <[email protected]>
---
  man2/move_pages.2 | 5 ++---
  1 file changed, 2 insertions(+), 3 deletions(-)
man2/move_pages.2 +0 -0
index 2d96468..2a2f3cd 100644
@@ -192,9 +192,8 @@ was specified or an attempt was made to migrate pages of a kernel thread.
  One of the target nodes is not online.

man2/move_pages.2 was specified or an attempt was made to migrate pages of a kernel thread. · line 199

Quoted 10 lines
man2/move_pages.2 +2 -3
  .TP
  .B ENOENT
-No pages were found that require moving.
-All pages are either already
-on the target node, not present, had an invalid address or could not be
+No pages were found.
+All pages are either not present, had an invalid address or could not be
  moved because they were mapped by multiple processes.
  .TP
  .B EPERM

whoa, hold on. If I'm reading through the various error paths correctly, then this code is *never* going to return ENOENT for the whole function. It can fill in that value per-page, in the status array, but that's all. Did I get that right?

If so, we need to redo this part of the man page.

thanks,

Signature · 2 lines
John Hubbard
NVIDIA
#4 lore raw
Quoted 29 lines · John Hubbard
Quoted 28 lines · Yang Shi

Since commit e78bbfa82624 ("mm: stop returning -ENOENT
from sys_move_pages() if nothing got migrated"), move_pages doesn't return -ENOENT anymore if the pages are already on the target nodes, but this change is never reflected in manpage.

Cc Michael Kerrisk <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc John Hubbard <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Qian Cai <[email protected]>
---
  man2/move_pages.2 | 5 ++---
  1 file changed, 2 insertions(+), 3 deletions(-)
man2/move_pages.2 +2 -3
index 2d96468..2a2f3cd 100644
@@ -192,9 +192,8 @@ was specified or an attempt was made to migrate pages of a kernel thread.
  One of the target nodes is not online.
  .TP
  .B ENOENT
-No pages were found that require moving.
-All pages are either already
-on the target node, not present, had an invalid address or could not be
+No pages were found.
+All pages are either not present, had an invalid address or could not be
Quoted 8 lines

man2/move_pages.2 was specified or an attempt was made to migrate pages of a kernel thread. · line 199

Quoted 4 lines
man2/move_pages.2 +0 -0
  moved because they were mapped by multiple processes.
  .TP
  .B EPERM

whoa, hold on. If I'm reading through the various error paths correctly, then this code is *never* going to return ENOENT for the whole function. It can fill in that value per-page, in the status array, but that's all. Did I get that right?

You are right. Both store_status and do_move_pages_to_node do overwrite the error code. So you are right that ENOENT return value is not possible. I haven't checked since when this is the case. This whole syscall is a disaster from the API and documentation POV.

Btw. Page states error codes could see some refinements as well.

Signature · 2 lines
Michal Hocko
SUSE Labs
author
#5 lore raw
Quoted 37 lines · Michal Hocko
Quoted 28 lines · John Hubbard
Quoted 27 lines · Yang Shi

Since commit e78bbfa82624 ("mm: stop returning -ENOENT
from sys_move_pages() if nothing got migrated"), move_pages doesn't return -ENOENT anymore if the pages are already on the target nodes, but this change is never reflected in manpage.

Cc Michael Kerrisk <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc John Hubbard <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Qian Cai <[email protected]>
---
   man2/move_pages.2 | 5 ++---
   1 file changed, 2 insertions(+), 3 deletions(-)
man2/move_pages.2 +1 -3
index 2d96468..2a2f3cd 100644
@@ -192,9 +192,8 @@ was specified or an attempt was made to migrate pages of a kernel thread.
   One of the target nodes is not online.
   .TP
   .B ENOENT
-No pages were found that require moving.
-All pages are either already
-on the target node, not present, had an invalid address or could not be
+No pages were found.
Quoted 8 lines

man2/move_pages.2 was specified or an attempt was made to migrate pages of a kernel thread. · line 199

Quoted 5 lines
man2/move_pages.2 +1 -0
+All pages are either not present, had an invalid address or could not be
   moved because they were mapped by multiple processes.
   .TP
   .B EPERM

whoa, hold on. If I'm reading through the various error paths correctly, then this code is *never* going to return ENOENT for the whole function. It can fill in that value per-page, in the status array, but that's all. Did I get that right?

Quoted 4 lines

You are right. Both store_status and do_move_pages_to_node do overwrite the error code. So you are right that ENOENT return value is not possible. I haven't checked since when this is the case. This whole syscall is a disaster from the API and documentation POV.

It looks since commit e78bbfa82624 ("mm: stop returning -ENOENT from sys_move_pages() if nothing got migrated") too, which reset err to 0 unconditionally. It seems it is on purpose by that commit the syscall caller should check status for the details according to the commit log.

Quoted 2 lines

Btw. Page states error codes could see some refinements as well.

#6 lore raw
Quoted 5 lines · Yang Shi

It looks since commit e78bbfa82624 ("mm: stop returning -ENOENT from sys_move_pages() if nothing got migrated") too, which reset err to 0 unconditionally. It seems it is on purpose by that commit the syscall caller should check status for the details according to the commit log.

I don’t read it on purpose. “There is no point in returning -ENOENT from sys_move_pages() if all pages
were already on the right node”, so this is only taking about the pages in the desired node. Anyway, but now it is probably the best time to think outside the box redesigning this syscalls and nuke this whole mess.

#7 lore raw
Quoted 9 lines · Qian Cai
Quoted 3 lines · Yang Shi

It looks since commit e78bbfa82624 ("mm: stop returning -ENOENT from sys_move_pages() if nothing got migrated") too, which reset err to 0 unconditionally. It seems it is on purpose by that commit the syscall caller should check status for the details according to the commit log.

I don’t read it on purpose. “There is no point in returning -ENOENT from sys_move_pages() if all pages were already on the right node”, so this is only taking about the pages in the desired node. Anyway, but now it is probably the best time to think outside the box redesigning this syscalls and nuke this whole mess.

The nature of the beast is that moving pages is not a deterministic process. The ability to move depends on pages being pinned and locked by other kernel subsystem. Other system components may also move the page independently.

If the user calls this system call and wants to move some pages then he has presumably figured out somehow that pages are misplaced. If no pages can be moved then the system call did nothing which could indicate that some other process is interfering with the desire to move pages to certain nodes.

This could be important to know (maybe the other system components already moved the page indepently or another user is also migrating pages).

author
#8 lore raw
In reply to John Hubbard
Quoted 31 lines · John Hubbard
Quoted 30 lines · Yang Shi

Since commit e78bbfa82624 ("mm: stop returning -ENOENT
from sys_move_pages() if nothing got migrated"), move_pages doesn't return -ENOENT anymore if the pages are already on the target nodes, but this change is never reflected in manpage.

Cc Michael Kerrisk <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc John Hubbard <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Qian Cai <[email protected]>

---
  man2/move_pages.2 | 5 ++---

  1 file changed, 2 insertions(+), 3 deletions(-)
man2/move_pages.2 +0 -0
index 2d96468..2a2f3cd 100644
@@ -192,9 +192,8 @@ was specified or an attempt was made to migrate 

pages of a kernel thread.
  One of the target nodes is not online.
  .TP
  .B ENOENT

man2/move_pages.2 +2 -3
-No pages were found that require moving.
-All pages are either already
-on the target node, not present, had an invalid address or could not be
+No pages were found.
+All pages are either not present, had an invalid address or could 

not be

Quoted 8 lines
Quoted 4 lines

  moved because they were mapped by multiple processes. .TP
  .B EPERM

whoa, hold on. If I'm reading through the various error paths correctly, then this code is *never* going to return ENOENT for the whole function. It can fill in that value per-page, in the status array, but that's all. Did I get that right?

Nice catch. Yes, you are right.

Quoted 2 lines

If so, we need to redo this part of the man page.

Yes.

Quoted 3 lines

thanks,

#9 lore raw
Quoted 46 lines · Yang Shi
Quoted 34 lines · John Hubbard
Quoted 33 lines · Yang Shi

Since commit e78bbfa82624 ("mm: stop returning -ENOENT
from sys_move_pages() if nothing got migrated"), move_pages doesn't return -ENOENT anymore if the pages are already on the target nodes, but this change is never reflected in manpage.

Cc Michael Kerrisk <[email protected]>
Cc Christoph Lameter <[email protected]>
Cc John Hubbard <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Qian Cai <[email protected]>

---
  man2/move_pages.2 | 5 ++---

  1 file changed, 2 insertions(+), 3 deletions(-)
man2/move_pages.2 +0 -0
index 2d96468..2a2f3cd 100644
@@ -192,9 +192,8 @@ was specified or an attempt was made to migrate 

pages of a kernel thread.
  One of the target nodes is not online.
  .TP
  .B ENOENT

man2/move_pages.2 +2 -3
-No pages were found that require moving.
-All pages are either already
-on the target node, not present, had an invalid address or could not be
+No pages were found.
+All pages are either not present, had an invalid address or could 

not be
  moved because they were mapped by multiple processes. .TP
  .B EPERM

Quoted 8 lines
Quoted 1 line

whoa, hold on. If I'm reading through the various error paths correctly, then this
code is *never* going to return ENOENT for the whole function. It can fill in that
value per-page, in the status array, but that's all. Did I get that right?

Quoted 6 lines

Nice catch. Yes, you are right.

Quoted 2 lines

If so, we need to redo this part of the man page.

Yes.

So where are things at with this? Is an improved man-pages patch on the way, or is some other action (on the API) planned?

Thanks,

Michael

Signature · 3 lines
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
#10 lore raw

On 12/13/19 5:55 PM, Michael Kerrisk (man-pages) wrote:
...

Quoted 10 lines
Quoted 10 lines
Quoted 6 lines

whoa, hold on. If I'm reading through the various error paths correctly, then this
code is *never* going to return ENOENT for the whole function. It can fill in that
value per-page, in the status array, but that's all. Did I get that right?

Nice catch. Yes, you are right.

Quoted 1 line
Quoted 7 lines
Quoted 3 lines
Quoted 1 line

If so, we need to redo this part of the man page.

Yes.

So where are things at with this? Is an improved man-pages patch on the way, or is some other action (on the API) planned?

I was waiting to see if Yang was going to respond...anyway, I think we're looking at approximately this sort of change:

man2/move_pages.2 +0 -6
index 2d96468fa..1bf1053f2 100644
@@ -191,12 +191,6 @@ was specified or an attempt was made to migrate pages of a kernel thread.
 .B ENODEV
 One of the target nodes is not online.
 .TP
-.B ENOENT
-No pages were found that require moving.
-All pages are either already
-on the target node, not present, had an invalid address or could not be
-moved because they were mapped by multiple processes.
-.TP
 .B EPERM
 The caller specified
 .B MPOL_MF_MOVE_ALL

...But I'm not sure if we should change the implementation, instead, so that it *can* return ENOENT. That's the main question to resolve before creating any more patches, I think.

In addition, Michal mentioned that the page states in the status array also need updated documentation.

thanks,

Signature · 2 lines
John Hubbard
NVIDIA
Acked-by
#11 lore raw

On Tue 17-12-19 23:36:09, John Hubbard wrote:
[...]

Quoted 18 lines
man2/move_pages.2 +0 -6
index 2d96468fa..1bf1053f2 100644
@@ -191,12 +191,6 @@ was specified or an attempt was made to migrate pages of a kernel thread.
 .B ENODEV
 One of the target nodes is not online.
 .TP
-.B ENOENT
-No pages were found that require moving.
-All pages are either already
-on the target node, not present, had an invalid address or could not be
-moved because they were mapped by multiple processes.
-.TP
 .B EPERM
 The caller specified
 .B MPOL_MF_MOVE_ALL
Quoted 3 lines

...But I'm not sure if we should change the implementation, instead, so that it *can* return ENOENT. That's the main question to resolve before creating any more patches, I think.

I would start by dropping any note about ENOENT first. I am not really sure there is a reasonable usecase for it but maybe somebody comes up with something and only then we should consider it.

Feel free to add

ideally with a kernel commit which removed the ENOENT.

Signature · 2 lines
Michal Hocko
SUSE Labs
author
#12 lore raw
Quoted 27 lines · Michal Hocko

On Tue 17-12-19 23:36:09, John Hubbard wrote:
[...]

Quoted 18 lines
man2/move_pages.2 +0 -6
index 2d96468fa..1bf1053f2 100644
@@ -191,12 +191,6 @@ was specified or an attempt was made to migrate pages of a kernel thread.
  .B ENODEV
  One of the target nodes is not online.
  .TP
-.B ENOENT
-No pages were found that require moving.
-All pages are either already
-on the target node, not present, had an invalid address or could not be
-moved because they were mapped by multiple processes.
-.TP
  .B EPERM
  The caller specified
  .B MPOL_MF_MOVE_ALL
Quoted 3 lines

...But I'm not sure if we should change the implementation, instead, so that it *can* return ENOENT. That's the main question to resolve before creating any more patches, I think.

I would start by dropping any note about ENOENT first. I am not really sure there is a reasonable usecase for it but maybe somebody comes up with something and only then we should consider it.

Quoted 4 lines

Feel free to add

ideally with a kernel commit which removed the ENOENT.

A quick audit doesn't show kernel code or comment notes about ENOENT wrongly. The status could be set as ENOENT if the page is not present (follow_page() returns NULL), and man page does match what kernel does.

#13 lore raw
Quoted 31 lines · Yang Shi
Quoted 30 lines · Michal Hocko

On Tue 17-12-19 23:36:09, John Hubbard wrote:
[...]

Quoted 18 lines
man2/move_pages.2 +0 -6
index 2d96468fa..1bf1053f2 100644
@@ -191,12 +191,6 @@ was specified or an attempt was made to migrate pages of a kernel thread.
  .B ENODEV
  One of the target nodes is not online.
  .TP
-.B ENOENT
-No pages were found that require moving.
-All pages are either already
-on the target node, not present, had an invalid address or could not be
-moved because they were mapped by multiple processes.
-.TP
  .B EPERM
  The caller specified
  .B MPOL_MF_MOVE_ALL
Quoted 3 lines

...But I'm not sure if we should change the implementation, instead, so that it *can* return ENOENT. That's the main question to resolve before creating any more patches, I think.

I would start by dropping any note about ENOENT first. I am not really sure there is a reasonable usecase for it but maybe somebody comes up with something and only then we should consider it.

Feel free to add

Quoted 6 lines
Quoted 1 line

ideally with a kernel commit which removed the ENOENT.

A quick audit doesn't show kernel code or comment notes about ENOENT wrongly. The status could be set as ENOENT if the page is not present (follow_page() returns NULL), and man page does match what kernel does.

Doesn't the function one layer up then consume the ENOENT?

Eric

author
#14 lore raw
Quoted 33 lines · Eric W. Biederman
Quoted 31 lines · Yang Shi
Quoted 30 lines · Michal Hocko

On Tue 17-12-19 23:36:09, John Hubbard wrote:
[...]

Quoted 18 lines
man2/move_pages.2 +0 -6
index 2d96468fa..1bf1053f2 100644
@@ -191,12 +191,6 @@ was specified or an attempt was made to migrate pages of a kernel thread.
   .B ENODEV
   One of the target nodes is not online.
   .TP
-.B ENOENT
-No pages were found that require moving.
-All pages are either already
-on the target node, not present, had an invalid address or could not be
-moved because they were mapped by multiple processes.
-.TP
   .B EPERM
   The caller specified
   .B MPOL_MF_MOVE_ALL
Quoted 3 lines

...But I'm not sure if we should change the implementation, instead, so that it *can* return ENOENT. That's the main question to resolve before creating any more patches, I think.

I would start by dropping any note about ENOENT first. I am not really sure there is a reasonable usecase for it but maybe somebody comes up with something and only then we should consider it.

Feel free to add

Quoted 6 lines
Quoted 5 lines
Quoted 1 line

ideally with a kernel commit which removed the ENOENT.

A quick audit doesn't show kernel code or comment notes about ENOENT wrongly. The status could be set as ENOENT if the page is not present (follow_page() returns NULL), and man page does match what kernel does.

Doesn't the function one layer up then consume the ENOENT?

No, it doesn't. The return value would be reset unconditionally by store_status(). This is what the man page patch tries to correct.

Quoted 2 lines

Eric

author
#15 lore raw
In reply to John Hubbard
Quoted 14 lines · John Hubbard

On 12/13/19 5:55 PM, Michael Kerrisk (man-pages) wrote:
...

Quoted 12 lines
Quoted 12 lines
Quoted 6 lines

whoa, hold on. If I'm reading through the various error paths correctly, then this
code is *never* going to return ENOENT for the whole function. It can fill in that
value per-page, in the status array, but that's all. Did I get that right?

Nice catch. Yes, you are right.

Quoted 2 lines

If so, we need to redo this part of the man page.

Quoted 9 lines
Quoted 5 lines
Quoted 1 line

Yes.

So where are things at with this? Is an improved man-pages patch on the way, or is some other action (on the API) planned?

I was waiting to see if Yang was going to respond...anyway, I think we're looking at approximately this sort of change:

Hi John,

I apologize for the delay, just came back from vacation. Thanks for taking care of the patch.

Quoted 28 lines
man2/move_pages.2 +0 -0
index 2d96468fa..1bf1053f2 100644
@@ -191,12 +191,6 @@ was specified or an attempt was made to migrate 

pages of a kernel thread.
 .B ENODEV
 One of the target nodes is not online.
 .TP
-.B ENOENT

man2/move_pages.2 +0 -5
-No pages were found that require moving.
-All pages are either already
-on the target node, not present, had an invalid address or could not be
-moved because they were mapped by multiple processes.
-.TP

 .B EPERM
 The caller specified
 .B MPOL_MF_MOVE_ALL

...But I'm not sure if we should change the implementation, instead, so that it *can* return ENOENT. That's the main question to resolve before creating any more patches, I think.

In addition, Michal mentioned that the page states in the status array also need updated documentation.

thanks,