mm: move_pages: fix the return value if there are not-migrated pages
Subsystems (3)
PATCH 1/1 mm: move_pages: fix the return value if there are not-migrated pages
The do_move_pages_to_node() might return > 0 value, the number of pages that are not migrated, then the value will be returned to userspace directly. But, move_pages() syscall would just return 0 or errno. So, we need reset the return value to 0 for such case as what pre-v4.17 did.
--- mm/migrate.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
mm/migrate.c +4 -1
@@ -1659,8 +1659,11 @@ static int do_pages_move(struct mm_struct *mm, nodemask_t task_nodes, goto out_flush; err =do_move_pages_to_node (mm, &pagelist, current_node); -if (err) +if (err) { +if (err >0 ) + err =0 ; goto out; + }if (i > start) { err =store_status (status, start, current_node, i - start);if (err)
Signature · 1 line
1.8.3.1
Quoted 33 lines · Yang Shi
The
do_move_pages_to_node()might return > 0 value, the number of pages that are not migrated, then the value will be returned to userspace directly. But,move_pages()syscall would just return 0 or errno. So, we need reset the return value to 0 for such case as what pre-v4.17 did.Fixes a49bd4d71637 ("mm, numa: rework do_pages_move")Cc Michal Hocko <[email protected]>Cc Wei Yang <[email protected]>Cc <[email protected]> [4.17+]Signed-off-by Yang Shi <[email protected]>---
mm/migrate.c| 5 ++++-1 file changed, 4 insertions(+), 1 deletion(-)mm/migrate.c +4
-1@@ -1659,8 +1659,11 @@ static int do_pages_move(struct mm_struct *mm, nodemask_t task_nodes, goto out_flush; err =do_move_pages_to_node (mm, &pagelist, current_node); -if (err) +if (err) { +if (err >0 ) + err =0 ; goto out; + }if (i > start) { err =store_status (status, start, current_node, i - start);if (err)Signature · 1 line
1.8.3.1
Hey, I am afraid you missed something. There are three calls of do_move_pages_to_node() in do_pages_move(). Why you just handle one return value? How about the other two?
Signature · 2 lines
Wei Yang Help you, Help me
Quoted 31 lines · Wei Yang
Quoted 30 lines · Yang Shi
The
do_move_pages_to_node()might return > 0 value, the number of pages that are not migrated, then the value will be returned to userspace directly. But,move_pages()syscall would just return 0 or errno. So, we need reset the return value to 0 for such case as what pre-v4.17 did.Fixes a49bd4d71637 ("mm, numa: rework do_pages_move")Cc Michal Hocko <[email protected]>Cc Wei Yang <[email protected]>Cc <[email protected]> [4.17+]Signed-off-by Yang Shi <[email protected]>---
mm/migrate.c| 5 ++++-1 file changed, 4 insertions(+), 1 deletion(-)mm/migrate.c +4
-1@@ -1659,8 +1659,11 @@ static int do_pages_move(struct mm_struct *mm, nodemask_t task_nodes, goto out_flush; err =do_move_pages_to_node (mm, &pagelist, current_node); -if (err) +if (err) { +if (err >0 ) + err =0 ; goto out; + }if (i > start) { err =store_status (status, start, current_node, i - start);
Quoted 9 lines
Quoted 3 lines
mm/migrate.c +0
-0if (err)Signature · 1 line
1.8.3.1Hey, I am afraid you missed something. There are three calls of
do_move_pages_to_node()indo_pages_move(). Why you just handle one return value? How about the other two?
Well, current logic in do_pages_move() is a little complicated to read.
I did a cleanup to make it easy to read and also friendly to do this fix.
If they look good to you, you could rebase your fix on top of them.
Quoted 3 lines
Signature · 2 lines
Wei Yang Help you, Help me
Signature · 2 lines
Wei Yang Help you, Help me
Quoted 40 lines · Wei Yang
Quoted 22 lines · Wei Yang
Quoted 21 lines · Yang Shi
The
do_move_pages_to_node()might return > 0 value, the number of pages that are not migrated, then the value will be returned to userspace directly. But,move_pages()syscall would just return 0 or errno. So, we need reset the return value to 0 for such case as what pre-v4.17 did.Fixes a49bd4d71637 ("mm, numa: rework do_pages_move")Cc Michal Hocko <[email protected]>Cc Wei Yang <[email protected]>Cc <[email protected]> [4.17+]Signed-off-by Yang Shi <[email protected]>---
mm/migrate.c| 5 ++++-1 file changed, 4 insertions(+), 1 deletion(-)mm/migrate.c +0
-0@@ -1659,8 +1659,11 @@ static int do_pages_move(struct mm_struct *mm, nodemask_t task_nodes, goto out_flush;Quoted 17 lines
Quoted 12 lines
mm/migrate.c +4
-1err =do_move_pages_to_node (mm, &pagelist, current_node); -if (err) +if (err) { +if (err >0 ) + err =0 ; goto out; + }if (i > start) { err =store_status (status, start, current_node, i - start);if (err)Signature · 1 line
1.8.3.1Hey, I am afraid you missed something. There are three calls of
do_move_pages_to_node()indo_pages_move(). Why you just handle one return value? How about the other two?
Quoted 5 lines
Well, current logic in
do_pages_move()is a little complicated to read.I did a cleanup to make it easy to read and also friendly to do this fix.
If they look good to you, you could rebase your fix on top of them.
Regression fix typically has higher priority. Since we already spotted the regressions and proposed fixes, I'd suggest we fix them in 5.5-rc, then aim any cleanup for 5.6 or 5.7. This should also make review easier IMHO.
Quoted 4 lines
Quoted 4 lines
Signature · 3 lines
-- Wei Yang Help you, Help me
Quoted 22 lines · Wei Yang
Quoted 21 lines · Yang Shi
The
do_move_pages_to_node()might return > 0 value, the number of pages that are not migrated, then the value will be returned to userspace directly. But,move_pages()syscall would just return 0 or errno. So, we need reset the return value to 0 for such case as what pre-v4.17 did.Fixes a49bd4d71637 ("mm, numa: rework do_pages_move")Cc Michal Hocko <[email protected]>Cc Wei Yang <[email protected]>Cc <[email protected]> [4.17+]Signed-off-by Yang Shi <[email protected]>---
mm/migrate.c| 5 ++++-1 file changed, 4 insertions(+), 1 deletion(-)mm/migrate.c +0
-0@@ -1659,8 +1659,11 @@ static int do_pages_move(struct mm_struct *mm, nodemask_t task_nodes, goto out_flush;
Quoted 16 lines
Quoted 12 lines
mm/migrate.c +4
-1err =do_move_pages_to_node (mm, &pagelist, current_node); -if (err) +if (err) { +if (err >0 ) + err =0 ; goto out; + }if (i > start) { err =store_status (status, start, current_node, i - start);if (err)Signature · 1 line
1.8.3.1Hey, I am afraid you missed something. There are three calls of
do_move_pages_to_node()indo_pages_move(). Why you just handle one return value? How about the other two?
A late night patch... I just simply missed those two. Thanks for catching it, will resolve in v2.
Quoted 1 line
Quoted 4 lines · Yang Shi
The
do_move_pages_to_node()might return > 0 value, the number of pages that are not migrated, then the value will be returned to userspace directly. But,move_pages()syscall would just return 0 or errno. So, we need reset the return value to 0 for such case as what pre-v4.17 did.
The patch is wrong. migrate_pages returns the number of pages it _hasn't_ migrated or -errno. Yeah that semantic sucks but... So err != 0 is always an error. Except err > 0 doesn't really provide any useful information to the userspace. I cannot really remember what was the actual behavior before my rework because there were some gotchas hidden there.
If you want to fix this properly then you have to query node status of each page unmigrated when migrate_pages fails with > 0. This would be easier if the fix is done on the latest cleanup posted to the list which consolidates all do_move_pages_to_node and store_status calls to a single function.
Quoted 28 lines
Fixes a49bd4d71637 ("mm, numa: rework do_pages_move")Cc Michal Hocko <[email protected]>Cc Wei Yang <[email protected]>Cc <[email protected]> [4.17+]Signed-off-by Yang Shi <[email protected]>--- mm/migrate.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)mm/migrate.c +4
-1@@ -1659,8 +1659,11 @@ static int do_pages_move(struct mm_struct *mm, nodemask_t task_nodes, goto out_flush; err =do_move_pages_to_node (mm, &pagelist, current_node); -if (err) +if (err) { +if (err >0 ) + err =0 ; goto out; + }if (i > start) { err =store_status (status, start, current_node, i - start);if (err)Signature · 1 line
1.8.3.1
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 6 lines · Michal Hocko
Quoted 4 lines · Yang Shi
The
do_move_pages_to_node()might return > 0 value, the number of pages that are not migrated, then the value will be returned to userspace directly. But,move_pages()syscall would just return 0 or errno. So, we need reset the return value to 0 for such case as what pre-v4.17 did.
Quoted 6 lines
The patch is wrong.
migrate_pagesreturns the number of pages it _hasn't_ migrated or -errno. Yeah that semantic sucks but... So err != 0 is always an error. Except err > 0 doesn't really provide any useful information to the userspace. I cannot really remember what was the actual behavior before my rework because there were some gotchas hidden there.
OK, so I've double checked. do_move_page_to_node_array would carry the
error code over to do_pages_move and it would store the status stored
in the pm array. It contains page_to_nid(page) so the resulting code
indeed behaves properly before my change and this is a regression. I
have a very vague recollection that this has been brought up already. <...looks in notes...>
Found it! The report is
http://lkml.kernel.org/r/0329efa0984b9b0252ef166abb4498c0795fab36.1535113317.git.jstancek@redhat.com
and my proposed workaround was http://lkml.kernel.org/r/[email protected]
Quoted 5 lines
If you want to fix this properly then you have to query node status of each page unmigrated when
migrate_pagesfails with > 0. This would be easier if the fix is done on the latest cleanup posted to the list which consolidates alldo_move_pages_to_nodeandstore_statuscalls to a single function.
Sorry forgot to put a reference to the patch: http://lkml.kernel.org/r/[email protected]
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 14 lines · Michal Hocko
Quoted 6 lines · Michal Hocko
Quoted 4 lines · Yang Shi
The
do_move_pages_to_node()might return > 0 value, the number of pages that are not migrated, then the value will be returned to userspace directly. But,move_pages()syscall would just return 0 or errno. So, we need reset the return value to 0 for such case as what pre-v4.17 did.Quoted 6 lines
The patch is wrong.
migrate_pagesreturns the number of pages it _hasn't_ migrated or -errno. Yeah that semantic sucks but... So err != 0 is always an error. Except err > 0 doesn't really provide any useful information to the userspace. I cannot really remember what was the actual behavior before my rework because there were some gotchas hidden there.
Quoted 4 lines
OK, so I've double checked.
do_move_page_to_node_arraywould carry the error code over todo_pages_moveand it would store the status stored in the pm array. It containspage_to_nid(page)so the resulting code indeed behaves properly before my change and this is a regression. I
Thanks, I see the change.
Quoted 5 lines
have a very vague recollection that this has been brought up already. <...looks in notes...>
Found it! The report is
http://lkml.kernel.org/r/0329efa0984b9b0252ef166abb4498c0795fab36.1535113317.git.jstancek@redhat.com
and my proposed workaround was http://lkml.kernel.org/r/[email protected]
Well, the above two links return 404.
Quoted 12 lines
Quoted 6 lines
If you want to fix this properly then you have to query node status of each page unmigrated when
migrate_pagesfails with > 0. This would be easier if the fix is done on the latest cleanup posted to the list which consolidates alldo_move_pages_to_nodeandstore_statuscalls to a single function.Sorry forgot to put a reference to the patch: http://lkml.kernel.org/r/[email protected]
Signature · 2 lines
Michal Hocko SUSE Labs
Signature · 2 lines
Wei Yang Help you, Help me
Quoted 22 lines · Wei Yang
Quoted 14 lines · Michal Hocko
Quoted 6 lines · Michal Hocko
Quoted 4 lines · Yang Shi
The
do_move_pages_to_node()might return > 0 value, the number of pages that are not migrated, then the value will be returned to userspace directly. But,move_pages()syscall would just return 0 or errno. So, we need reset the return value to 0 for such case as what pre-v4.17 did.Quoted 6 lines
The patch is wrong.
migrate_pagesreturns the number of pages it _hasn't_ migrated or -errno. Yeah that semantic sucks but... So err != 0 is always an error. Except err > 0 doesn't really provide any useful information to the userspace. I cannot really remember what was the actual behavior before my rework because there were some gotchas hidden there.Quoted 4 lines
OK, so I've double checked.
do_move_page_to_node_arraywould carry the error code over todo_pages_moveand it would store the status stored in the pm array. It containspage_to_nid(page)so the resulting code indeed behaves properly before my change and this is a regression. IThanks, I see the change.
Quoted 7 lines
Quoted 5 lines
have a very vague recollection that this has been brought up already. <...looks in notes...>
Found it! The report is
http://lkml.kernel.org/r/0329efa0984b9b0252ef166abb4498c0795fab36.1535113317.git.jstancek@redhat.com
and my proposed workaround was http://lkml.kernel.org/r/[email protected]Well, the above two links return 404.
You are right. They are not archived for some reason. Anyway, the patch I was proposing back then is below:
commit cfb88c266b645197135cde2905c2bfc82f6d82a9
Author: Michal Hocko <[email protected]> Date: Wed Nov 14 12:19:09 2018 +0100
mm: fix do_pages_move error reporting
a49bd4d71637 ("mm, numa: rework do_pages_move") has changed the way how we report error to layers above. As the changelog mentioned the semantic was quite unclear previously because the return 0 could mean both success and failure.
The above mentioned commit didn't get all the way down to fix this completely because it doesn't report pages that we even haven't attempted to migrate and therefore we cannot simply say that the semantic is:
- err < 0 - errno
- err >= 0 number of non-migrated pages.
mm/migrate.c +9 -1
@@ -1615,8 +1615,16 @@ static int do_pages_move(struct mm_struct *mm, nodemask_t task_nodes, goto out_flush; err =do_move_pages_to_node (mm, &pagelist, current_node); -if (err) +if (err) { +/* +* Possitive err means the number of failed pages to +* migrate. Make sure to report the rest of the +* nr_pages is not migrated as well. +*/ +if (err >0 ) + err += nr_pages - i -1 ; goto out; + }if (i > start) { err =store_status (status, start, current_node, i - start);if (err)
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 47 lines · Michal Hocko
Quoted 17 lines · Wei Yang
Quoted 12 lines · Michal Hocko
Quoted 11 lines · Michal Hocko
Quoted 4 lines · Yang Shi
The
do_move_pages_to_node()might return > 0 value, the number of pages that are not migrated, then the value will be returned to userspace directly. But,move_pages()syscall would just return 0 or errno. So, we need reset the return value to 0 for such case as what pre-v4.17 did.The patch is wrong.
migrate_pagesreturns the number of pages it _hasn't_ migrated or -errno. Yeah that semantic sucks but... So err != 0 is always an error. Except err > 0 doesn't really provide any useful information to the userspace. I cannot really remember what was the actual behavior before my rework because there were some gotchas hidden there.Quoted 4 lines
OK, so I've double checked.
do_move_page_to_node_arraywould carry the error code over todo_pages_moveand it would store the status stored in the pm array. It containspage_to_nid(page)so the resulting code indeed behaves properly before my change and this is a regression. IQuoted 8 lines
Thanks, I see the change.
Quoted 5 lines
have a very vague recollection that this has been brought up already. <...looks in notes...>
Found it! The report is
http://lkml.kernel.org/r/0329efa0984b9b0252ef166abb4498c0795fab36.1535113317.git.jstancek@redhat.com
and my proposed workaround was http://lkml.kernel.org/r/[email protected]Well, the above two links return 404.
You are right. They are not archived for some reason. Anyway, the patch I was proposing back then is below:
commit
cfb88c266b645197135cde2905c2bfc82f6d82a9Author: Michal Hocko <[email protected]> Date: Wed Nov 14 12:19:09 2018 +0100mm: fix
do_pages_moveerror reporting
a49bd4d71637("mm, numa: reworkdo_pages_move") has changed the way how we report error to layers above. As the changelog mentioned the semantic was quite unclear previously because the return 0 could mean both success and failure.The above mentioned commit didn't get all the way down to fix this completely because it doesn't report pages that we even haven't attempted to migrate and therefore we cannot simply say that the semantic is:
- err < 0 - errno
- err >= 0 number of non-migrated pages.
Quoted 2 lines
Fixes a49bd4d71637 ("mm, numa: rework do_pages_move")Signed-off-by Michal Hocko <[email protected]>
Thanks, Michal. But, it looks this patch still could return > 0 value (the total number of non-migrated pages, including not even attempted pages) too, but the problem we are trying to fix is to make do_pages_move() return <= 0 value only since the man page of move_pages() doesn't allow return > 0 value.
And, by looking into the old code (v4.16), I spotted another problem. The migrate_pages() would store the migration failure error code into page_to_node->status. So, When do_move_page_to_node_array() returns > 0 value, the return value would be reset to 0 and the migration error codes for non-migrated pages would be stored into status to return to userspace. But, the rework removed this.
I didn't dig into the intention of the rework, is it expected?
Quoted 23 lines
mm/migrate.c +9
-1@@ -1615,8 +1615,16 @@ static int do_pages_move(struct mm_struct *mm, nodemask_t task_nodes, goto out_flush; err =do_move_pages_to_node (mm, &pagelist, current_node); -if (err) +if (err) { +/* +* Possitive err means the number of failed pages to +* migrate. Make sure to report the rest of the +* nr_pages is not migrated as well. +*/ +if (err >0 ) + err += nr_pages - i -1 ; goto out; + }if (i > start) { err =store_status (status, start, current_node, i - start);if (err)
Quoted 50 lines · Yang Shi
Quoted 44 lines · Michal Hocko
Quoted 17 lines · Wei Yang
Quoted 12 lines · Michal Hocko
Quoted 11 lines · Michal Hocko
Quoted 4 lines · Yang Shi
The
do_move_pages_to_node()might return > 0 value, the number of pages that are not migrated, then the value will be returned to userspace directly. But,move_pages()syscall would just return 0 or errno. So, we need reset the return value to 0 for such case as what pre-v4.17 did.The patch is wrong.
migrate_pagesreturns the number of pages it _hasn't_ migrated or -errno. Yeah that semantic sucks but... So err != 0 is always an error. Except err > 0 doesn't really provide any useful information to the userspace. I cannot really remember what was the actual behavior before my rework because there were some gotchas hidden there.Quoted 4 lines
OK, so I've double checked.
do_move_page_to_node_arraywould carry the error code over todo_pages_moveand it would store the status stored in the pm array. It containspage_to_nid(page)so the resulting code indeed behaves properly before my change and this is a regression. IQuoted 8 lines
Thanks, I see the change.
Quoted 5 lines
have a very vague recollection that this has been brought up already. <...looks in notes...>
Found it! The report is
http://lkml.kernel.org/r/0329efa0984b9b0252ef166abb4498c0795fab36.1535113317.git.jstancek@redhat.com
and my proposed workaround was http://lkml.kernel.org/r/[email protected]Well, the above two links return 404.
You are right. They are not archived for some reason. Anyway, the patch I was proposing back then is below:
commit
cfb88c266b645197135cde2905c2bfc82f6d82a9Author: Michal Hocko <[email protected]> Date: Wed Nov 14 12:19:09 2018 +0100mm: fix
do_pages_moveerror reporting
a49bd4d71637("mm, numa: reworkdo_pages_move") has changed the way how we report error to layers above. As the changelog mentioned the semantic was quite unclear previously because the return 0 could mean both success and failure.
The above mentioned commit didn't get all the way down to fix this completely because it doesn't report pages that we even haven't attempted to migrate and therefore we cannot simply say that the semantic is:
- err < 0 - errno
- err >= 0 number of non-migrated pages.Quoted 2 lines
Fixes a49bd4d71637 ("mm, numa: rework do_pages_move")Signed-off-by Michal Hocko <[email protected]>
Quoted 5 lines
Thanks, Michal. But, it looks this patch still could return > 0 value (the total number of non-migrated pages, including not even attempted pages) too, but the problem we are trying to fix is to make
do_pages_move()return <= 0 value only since the man page ofmove_pages()doesn't allow return > 0 value.
Yes this patch just lives with the changed semantic and tries to make it sensible. So if some page cannot be migrated then we just stop and return the number of non migrated pages at the tail of the given array. This would make error handling slightly easier because you know that count - ret pages of the array can be skipped if ret >= 0.
Quoted 8 lines
And, by looking into the old code (v4.16), I spotted another problem. The
migrate_pages()would store the migration failure error code intopage_to_node->status. So, Whendo_move_page_to_node_array()returns > 0 value, the return value would be reset to 0 and the migration error codes for non-migrated pages would be stored into status to return to userspace. But, the rework removed this.I didn't dig into the intention of the rework, is it expected?
I have tried to preserve the original semantic as possible. As explained in the changelog there were quite some discrepancies even before. This new one was not really intentional. We have effectively two options here. Either somebody really depend on the former semantic and we have to fix this or we can relax the semantic as the above patch attempts.
I would be more inclined for the second option as nobody has complained about the new semantic except for few ltp tests which do not represent real workload. If you have a real usecase then speak up please.
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 54 lines · Michal Hocko
Quoted 48 lines · Yang Shi
Quoted 44 lines · Michal Hocko
Quoted 17 lines · Wei Yang
Quoted 12 lines · Michal Hocko
Quoted 11 lines · Michal Hocko
Quoted 4 lines · Yang Shi
The
do_move_pages_to_node()might return > 0 value, the number of pages that are not migrated, then the value will be returned to userspace directly. But,move_pages()syscall would just return 0 or errno. So, we need reset the return value to 0 for such case as what pre-v4.17 did.The patch is wrong.
migrate_pagesreturns the number of pages it _hasn't_ migrated or -errno. Yeah that semantic sucks but... So err != 0 is always an error. Except err > 0 doesn't really provide any useful information to the userspace. I cannot really remember what was the actual behavior before my rework because there were some gotchas hidden there.Quoted 4 lines
OK, so I've double checked.
do_move_page_to_node_arraywould carry the error code over todo_pages_moveand it would store the status stored in the pm array. It containspage_to_nid(page)so the resulting code indeed behaves properly before my change and this is a regression. IQuoted 8 lines
Thanks, I see the change.
Quoted 5 lines
have a very vague recollection that this has been brought up already. <...looks in notes...>
Found it! The report is
http://lkml.kernel.org/r/0329efa0984b9b0252ef166abb4498c0795fab36.1535113317.git.jstancek@redhat.com
and my proposed workaround was http://lkml.kernel.org/r/[email protected]Well, the above two links return 404.
You are right. They are not archived for some reason. Anyway, the patch I was proposing back then is below:
commit
cfb88c266b645197135cde2905c2bfc82f6d82a9Author: Michal Hocko <[email protected]> Date: Wed Nov 14 12:19:09 2018 +0100mm: fix
do_pages_moveerror reporting
a49bd4d71637("mm, numa: reworkdo_pages_move") has changed the way how we report error to layers above. As the changelog mentioned the semantic was quite unclear previously because the return 0 could mean both success and failure.
The above mentioned commit didn't get all the way down to fix this completely because it doesn't report pages that we even haven't attempted to migrate and therefore we cannot simply say that the semantic is:
- err < 0 - errno
- err >= 0 number of non-migrated pages.Quoted 2 lines
Fixes a49bd4d71637 ("mm, numa: rework do_pages_move")Signed-off-by Michal Hocko <[email protected]>Quoted 5 lines
Thanks, Michal. But, it looks this patch still could return > 0 value (the total number of non-migrated pages, including not even attempted pages) too, but the problem we are trying to fix is to make
do_pages_move()return <= 0 value only since the man page ofmove_pages()doesn't allow return > 0 value.
Quoted 5 lines
Yes this patch just lives with the changed semantic and tries to make it sensible. So if some page cannot be migrated then we just stop and return the number of non migrated pages at the tail of the given array. This would make error handling slightly easier because you know that count - ret pages of the array can be skipped if ret >= 0.
OK, I see. Returning > 0 value sounds more straightforward for userspace error handling.
BTW, we should update manpage to reflect the semantic change to indicate > 0 return value as an error case.
Quoted 15 lines
Quoted 9 lines
And, by looking into the old code (v4.16), I spotted another problem. The
migrate_pages()would store the migration failure error code intopage_to_node->status. So, Whendo_move_page_to_node_array()returns > 0 value, the return value would be reset to 0 and the migration error codes for non-migrated pages would be stored into status to return to userspace. But, the rework removed this.I didn't dig into the intention of the rework, is it expected?
I have tried to preserve the original semantic as possible. As explained in the changelog there were quite some discrepancies even before. This new one was not really intentional. We have effectively two options here. Either somebody really depend on the former semantic and we have to fix this or we can relax the semantic as the above patch attempts.
Quoted 3 lines
I would be more inclined for the second option as nobody has complained about the new semantic except for few ltp tests which do not represent real workload. If you have a real usecase then speak up please.
No, I don't have any real usecase. And, I tend to agree the most users may not care the reason of migration failure at all. Returning the number of non-migrated pages seems more straightforward.
I agree we could stick with the new semantic and fix the return value as what your patch did. I'm going to rebase your patch on top of Wei Yang's cleanup if you don't mind.
Quoted 63 lines · Yang Shi
Quoted 54 lines · Michal Hocko
Quoted 48 lines · Yang Shi
Quoted 44 lines · Michal Hocko
Quoted 17 lines · Wei Yang
Quoted 12 lines · Michal Hocko
Quoted 11 lines · Michal Hocko
Quoted 4 lines · Yang Shi
The
do_move_pages_to_node()might return > 0 value, the number of pages that are not migrated, then the value will be returned to userspace directly. But,move_pages()syscall would just return 0 or errno. So, we need reset the return value to 0 for such case as what pre-v4.17 did.The patch is wrong.
migrate_pagesreturns the number of pages it _hasn't_ migrated or -errno. Yeah that semantic sucks but... So err != 0 is always an error. Except err > 0 doesn't really provide any useful information to the userspace. I cannot really remember what was the actual behavior before my rework because there were some gotchas hidden there.Quoted 4 lines
OK, so I've double checked.
do_move_page_to_node_arraywould carry the error code over todo_pages_moveand it would store the status stored in the pm array. It containspage_to_nid(page)so the resulting code indeed behaves properly before my change and this is a regression. IQuoted 8 lines
Thanks, I see the change.
Quoted 5 lines
have a very vague recollection that this has been brought up already. <...looks in notes...>
Found it! The report is
http://lkml.kernel.org/r/0329efa0984b9b0252ef166abb4498c0795fab36.1535113317.git.jstancek@redhat.com
and my proposed workaround was http://lkml.kernel.org/r/[email protected]Well, the above two links return 404.
You are right. They are not archived for some reason. Anyway, the patch I was proposing back then is below:
commit
cfb88c266b645197135cde2905c2bfc82f6d82a9Author: Michal Hocko <[email protected]> Date: Wed Nov 14 12:19:09 2018 +0100mm: fix
do_pages_moveerror reporting
a49bd4d71637("mm, numa: reworkdo_pages_move") has changed the way how we report error to layers above. As the changelog mentioned the semantic was quite unclear previously because the return 0 could mean both success and failure.
The above mentioned commit didn't get all the way down to fix this completely because it doesn't report pages that we even haven't attempted to migrate and therefore we cannot simply say that the semantic is:
- err < 0 - errno
- err >= 0 number of non-migrated pages.Quoted 2 lines
Fixes a49bd4d71637 ("mm, numa: rework do_pages_move")Signed-off-by Michal Hocko <[email protected]>Quoted 5 lines
Thanks, Michal. But, it looks this patch still could return > 0 value (the total number of non-migrated pages, including not even attempted pages) too, but the problem we are trying to fix is to make
do_pages_move()return <= 0 value only since the man page ofmove_pages()doesn't allow return > 0 value.Quoted 5 lines
Yes this patch just lives with the changed semantic and tries to make it sensible. So if some page cannot be migrated then we just stop and return the number of non migrated pages at the tail of the given array. This would make error handling slightly easier because you know that count - ret pages of the array can be skipped if ret >= 0.
Quoted 5 lines
OK, I see. Returning > 0 value sounds more straightforward for userspace error handling.
BTW, we should update manpage to reflect the semantic change to indicate > 0 return value as an error case.
Absolutely.
Quoted 18 lines
Quoted 14 lines
Quoted 8 lines
And, by looking into the old code (v4.16), I spotted another problem. The
migrate_pages()would store the migration failure error code intopage_to_node->status. So, Whendo_move_page_to_node_array()returns > 0 value, the return value would be reset to 0 and the migration error codes for non-migrated pages would be stored into status to return to userspace. But, the rework removed this.I didn't dig into the intention of the rework, is it expected?
I have tried to preserve the original semantic as possible. As explained in the changelog there were quite some discrepancies even before. This new one was not really intentional. We have effectively two options here. Either somebody really depend on the former semantic and we have to fix this or we can relax the semantic as the above patch attempts.
Quoted 3 lines
I would be more inclined for the second option as nobody has complained about the new semantic except for few ltp tests which do not represent real workload. If you have a real usecase then speak up please.
Quoted 7 lines
No, I don't have any real usecase. And, I tend to agree the most users may not care the reason of migration failure at all. Returning the number of non-migrated pages seems more straightforward.
I agree we could stick with the new semantic and fix the return value as what your patch did. I'm going to rebase your patch on top of Wei Yang's cleanup if you don't mind.
Go ahead. Thanks a lot!
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 57 lines · Michal Hocko
Quoted 50 lines · Yang Shi
Quoted 44 lines · Michal Hocko
Quoted 17 lines · Wei Yang
Quoted 12 lines · Michal Hocko
Quoted 11 lines · Michal Hocko
Quoted 4 lines · Yang Shi
The
do_move_pages_to_node()might return > 0 value, the number of pages that are not migrated, then the value will be returned to userspace directly. But,move_pages()syscall would just return 0 or errno. So, we need reset the return value to 0 for such case as what pre-v4.17 did.The patch is wrong.
migrate_pagesreturns the number of pages it _hasn't_ migrated or -errno. Yeah that semantic sucks but... So err != 0 is always an error. Except err > 0 doesn't really provide any useful information to the userspace. I cannot really remember what was the actual behavior before my rework because there were some gotchas hidden there.Quoted 4 lines
OK, so I've double checked.
do_move_page_to_node_arraywould carry the error code over todo_pages_moveand it would store the status stored in the pm array. It containspage_to_nid(page)so the resulting code indeed behaves properly before my change and this is a regression. IQuoted 8 lines
Thanks, I see the change.
Quoted 5 lines
have a very vague recollection that this has been brought up already. <...looks in notes...>
Found it! The report is
http://lkml.kernel.org/r/0329efa0984b9b0252ef166abb4498c0795fab36.1535113317.git.jstancek@redhat.com
and my proposed workaround was http://lkml.kernel.org/r/[email protected]Well, the above two links return 404.
You are right. They are not archived for some reason. Anyway, the patch I was proposing back then is below:
commit
cfb88c266b645197135cde2905c2bfc82f6d82a9Author: Michal Hocko <[email protected]> Date: Wed Nov 14 12:19:09 2018 +0100mm: fix
do_pages_moveerror reporting
a49bd4d71637("mm, numa: reworkdo_pages_move") has changed the way how we report error to layers above. As the changelog mentioned the semantic was quite unclear previously because the return 0 could mean both success and failure.
The above mentioned commit didn't get all the way down to fix this completely because it doesn't report pages that we even haven't attempted to migrate and therefore we cannot simply say that the semantic is:
- err < 0 - errno
- err >= 0 number of non-migrated pages.Quoted 2 lines
Fixes a49bd4d71637 ("mm, numa: rework do_pages_move")Signed-off-by Michal Hocko <[email protected]>Quoted 5 lines
Thanks, Michal. But, it looks this patch still could return > 0 value (the total number of non-migrated pages, including not even attempted pages) too, but the problem we are trying to fix is to make
do_pages_move()return <= 0 value only since the man page ofmove_pages()doesn't allow return > 0 value.
Quoted 6 lines
Yes this patch just lives with the changed semantic and tries to make it sensible. So if some page cannot be migrated then we just stop and return the number of non migrated pages at the tail of the given array. This would make error handling slightly easier because you know that count - ret pages of the array can be skipped if ret >= 0.
Got some different idea for this.
Replied in the patch thread.
Quoted 21 lines
Quoted 8 lines
And, by looking into the old code (v4.16), I spotted another problem. The
migrate_pages()would store the migration failure error code intopage_to_node->status. So, Whendo_move_page_to_node_array()returns > 0 value, the return value would be reset to 0 and the migration error codes for non-migrated pages would be stored into status to return to userspace. But, the rework removed this.I didn't dig into the intention of the rework, is it expected?
I have tried to preserve the original semantic as possible. As explained in the changelog there were quite some discrepancies even before. This new one was not really intentional. We have effectively two options here. Either somebody really depend on the former semantic and we have to fix this or we can relax the semantic as the above patch attempts.
I would be more inclined for the second option as nobody has complained about the new semantic except for few ltp tests which do not represent real workload. If you have a real usecase then speak up please.
Signature · 2 lines
Michal Hocko SUSE Labs
Signature · 2 lines
Wei Yang Help you, Help me