PM: hibernate: Restore GFP mask in power_down() for HIBERNATION_PLATFORM
Subsystems (2)
PATCH 1/1 PM: hibernate: Restore GFP mask in power_down() for HIBERNATION_PLATFORM
commit 449c9c02537a1 ("PM: hibernate: Restrict GFP mask in hibernation_snapshot()") added a restrict GFP mask call that leads to mismatch when using the platform for hibernation. As part of calling hibernation_platform_enter() the mask will be restricted when calling dpm_suspend_start().
This is a similar problem as occurred with hybrid sleep that was fixed by commit 469d80a3712c6 ("PM: hibernate: Fix hybrid-sleep").
Restore GFP maks as part of power_down() in HIBERNATION_PLATFORM case to fix the mismatch.
--- kernel/power/hibernate.c | 1 + 1 file changed, 1 insertion(+)
kernel/power/hibernate.c +1 -0
@@ -721,6 +721,7 @@ static void power_down(void)kernel_restart (NULL );break ;case HIBERNATION_PLATFORM : +pm_restore_gfp_mask (); error =hibernation_platform_enter ();if (error == -EAGAIN || error == -EBUSY ) { events_check_enabled = false;
Signature · 1 line
2.43.0
Quoted 5 lines · Mario Limonciello (AMD)
commit
449c9c02537a1("PM: hibernate: Restrict GFP mask inhibernation_snapshot()") added a restrict GFP mask call that leads to mismatch when using the platform for hibernation. As part of callinghibernation_platform_enter()the mask will be restricted when callingdpm_suspend_start().
I tested the fix in Qemu. It indeed works.
Signature · 1 line
Askar Safin
Quoted 26 lines · Mario Limonciello (AMD)
commit
449c9c02537a1("PM: hibernate: Restrict GFP mask inhibernation_snapshot()") added a restrict GFP mask call that leads to mismatch when using the platform for hibernation. As part of callinghibernation_platform_enter()the mask will be restricted when callingdpm_suspend_start().This is a similar problem as occurred with hybrid sleep that was fixed by commit
469d80a3712c6("PM: hibernate: Fix hybrid-sleep").Restore GFP maks as part of
power_down()inHIBERNATION_PLATFORMcase to fix the mismatch.Reported-by Askar Safin <[email protected]>Fixes 449c9c02537a1 ("PM: hibernate: Restrict GFP mask in hibernation_snapshot()")Signed-off-by Mario Limonciello (AMD) <[email protected]>--- kernel/power/hibernate.c | 1 + 1 file changed, 1 insertion(+)kernel/power/hibernate.c +0
-0
kernel/power/hibernate.c static void power_down(void)
Quoted 8 lines
kernel/power/hibernate.c +1
-1@@ -721,6 +721,7 @@ static void power_down(void)kernel_restart (NULL );break ;case HIBERNATION_PLATFORM : +pm_restore_gfp_mask (); error =hibernation_platform_enter ();if (error == -EAGAIN || error == -EBUSY ) { events_check_enabled = false; --
Applied as 6.18-rc material, thanks!
"Mario Limonciello (AMD)" <[email protected]>:
Quoted 5 lines
commit
449c9c02537a1("PM: hibernate: Restrict GFP mask inhibernation_snapshot()") added a restrict GFP mask call that leads to mismatch when using the platform for hibernation. As part of callinghibernation_platform_enter()the mask will be restricted when callingdpm_suspend_start().
Are you sure this is proper solution?
As well as I understand, pm_restore_gfp_mask will make pm_suspended_storage to return false. And this will enable swapping. Thus it is possible that we will write some pages to swap after this pm_restore_gfp_mask call, and thus we will damage swap.
Note: I'm not sure that my explanation is correct. Anyway I think you should explain in commit message or comment why we will not damage swap here.
Signature · 1 line
Askar Safin
Rafael J. Wysocki
maintainerQuoted 8 lines · Askar Safin
"Mario Limonciello (AMD)" <[email protected]>:
Quoted 5 lines
commit
449c9c02537a1("PM: hibernate: Restrict GFP mask inhibernation_snapshot()") added a restrict GFP mask call that leads to mismatch when using the platform for hibernation. As part of callinghibernation_platform_enter()the mask will be restricted when callingdpm_suspend_start().
Quoted 6 lines
Are you sure this is proper solution?
As well as I understand,
pm_restore_gfp_maskwill makepm_suspended_storageto return false. And this will enable swapping. Thus it is possible that we will write some pages to swap after thispm_restore_gfp_maskcall, and thus we will damage swap.
What kind of damage are you talking about, specifically?
Quoted 3 lines
Note: I'm not sure that my explanation is correct. Anyway I think you should explain in commit message or comment why we will not damage swap here.
Well, if you have a specific failing scenario in mind, you need to say what it is. Otherwise you are effectively saying something like "there is a bug in your code, but you need to find it yourself".
Quoted 1 line · Rafael J. Wysocki
What kind of damage are you talking about, specifically?
Again: "pm_restore_gfp_mask" will restore "gfp_allowed_mask" to its normal value,
which will contain __GFP_IO and __GFP_FS. Thus "pm_suspended_storage" will start to return false.
But "pm_suspended_storage" is called here:
https://elixir.bootlin.com/linux/v6.18-rc3/source/mm/swapfile.c#L1895
(Also, please, read that big comment at this link. Well, I have to admit I don't understand it in full.)
This check is needed to prevent swapping out pages during hibernation.
Call chain is so:swap_writeout -> folio_free_swap -> folio_swapcache_freeable -> pm_suspended_storage
So by calling "pm_restore_gfp_mask" we allow pages to be swapped out.
But we already wrote hibernation image by that point!
So swapping pages will make our swap partition inconsistent.
Moreover, as well as I understand, whole reason why we deal with GFP mask in hibernation code is to prevent swapping out pages.
We restrict GFP before creating hibernation image here:
https://elixir.bootlin.com/linux/v6.18-rc3/source/kernel/power/hibernate.c#L463
.
We do this (as well as I understand) to prevent pages from swapping out.
And, starting from that moment, as well as I understand, we should not restore GFP mask until either:
- we resume
- hibernation will abort for some reason (for example, "wake up event detected during hibernation")
Again: I'm not trying to insult anybody. I'm just trying to help. And to understand PM code.
I'm not sure that my explanations are correct.
In any case I think it will be good idea to add comment explaining why restoring GFP mask
is safe there.
I'm attempting to write code to fix this bug:
https://lore.kernel.org/linux-pm/[email protected]/
And this is why I'm trying to understand PM code.
Signature · 1 line
Askar Safin
Rafael J. Wysocki
maintainerQuoted 9 lines · Askar Safin
Quoted 1 line · Rafael J. Wysocki
What kind of damage are you talking about, specifically?
Again: "
pm_restore_gfp_mask" will restore "gfp_allowed_mask" to its normal value,
which will contain__GFP_IOand__GFP_FS. Thus "pm_suspended_storage" will start to return false.
Quoted 7 lines
But "
pm_suspended_storage" is called here:
https://elixir.bootlin.com/linux/v6.18-rc3/source/mm/swapfile.c#L1895(Also, please, read that big comment at this link. Well, I have to admit I don't understand it in full.)
This check is needed to prevent swapping out pages during hibernation.
It is actually more complicated, but fair enough.
Basically, the concern is that a swap page can be freed and then re-used for storing another page of memory after the image has been created, so the swap metadata in the image would not match the reality any more.
Quoted 4 lines
Call chain is so:
swap_writeout->folio_free_swap->folio_swapcache_freeable->pm_suspended_storage
Quoted 8 lines
So by calling "
pm_restore_gfp_mask" we allow pages to be swapped out.But we already wrote hibernation image by that point!
So swapping pages will make our swap partition inconsistent.
Moreover, as well as I understand, whole reason why we deal with GFP mask in hibernation code is to prevent swapping out pages.
No, it is not the whole reason. It is also done to avoid accessing swap devices while they may not be (fully) operational.
Quoted 11 lines
We restrict GFP before creating hibernation image here:
https://elixir.bootlin.com/linux/v6.18-rc3/source/kernel/power/hibernate.c#L463
.We do this (as well as I understand) to prevent pages from swapping out.
And, starting from that moment, as well as I understand, we should not restore GFP mask until either:
- we resume
- hibernation will abort for some reason (for example, "wake up event detected during hibernation")
Again, this is slightly more complicated because the GFP mask is going to be restricted again shortly after being restored temporarily by the $subject patch and if no memory allocations happen in the meantime, nothing bad will happen. Also, if those memory allocations don't trigger swapping, nothing bad will happen either. Now, it is quite unlikely that anyone will attempt to allocate a lot of memory during the "prepare" phase of a suspend or power-off transition, so all of this is not super-worrisome, but I agree that avoiding all of this GFP mask restrict/restore dance would be better overall.
I'm going to remove the pm_restrict_gfp_mask() and pm_restore_gfp_mask() calls from dpm_suspend_start() and dpm_suspend_end(), respectively, but that's not a change for 6.18.
Thanks!
Rafael J. Wysocki
maintainerQuoted 64 lines · Rafael J. Wysocki
Quoted 9 lines · Askar Safin
Quoted 1 line · Rafael J. Wysocki
What kind of damage are you talking about, specifically?
Again: "
pm_restore_gfp_mask" will restore "gfp_allowed_mask" to its normal value,
which will contain__GFP_IOand__GFP_FS. Thus "pm_suspended_storage" will start to return false.Quoted 7 lines
But "
pm_suspended_storage" is called here:
https://elixir.bootlin.com/linux/v6.18-rc3/source/mm/swapfile.c#L1895(Also, please, read that big comment at this link. Well, I have to admit I don't understand it in full.)
This check is needed to prevent swapping out pages during hibernation.
It is actually more complicated, but fair enough.
Basically, the concern is that a swap page can be freed and then re-used for storing another page of memory after the image has been created, so the swap metadata in the image would not match the reality any more.
Quoted 4 lines
Call chain is so:
swap_writeout->folio_free_swap->folio_swapcache_freeable->pm_suspended_storageQuoted 8 lines
So by calling "
pm_restore_gfp_mask" we allow pages to be swapped out.But we already wrote hibernation image by that point!
So swapping pages will make our swap partition inconsistent.
Moreover, as well as I understand, whole reason why we deal with GFP mask in hibernation code is to prevent swapping out pages.
No, it is not the whole reason. It is also done to avoid accessing swap devices while they may not be (fully) operational.
Quoted 11 lines
We restrict GFP before creating hibernation image here:
https://elixir.bootlin.com/linux/v6.18-rc3/source/kernel/power/hibernate.c#L463
.We do this (as well as I understand) to prevent pages from swapping out.
And, starting from that moment, as well as I understand, we should not restore GFP mask until either:
- we resume
- hibernation will abort for some reason (for example, "wake up event detected during hibernation")Again, this is slightly more complicated because the GFP mask is going to be restricted again shortly after being restored temporarily by the $subject patch and if no memory allocations happen in the meantime, nothing bad will happen. Also, if those memory allocations don't trigger swapping, nothing bad will happen either. Now, it is quite unlikely that anyone will attempt to allocate a lot of memory during the "prepare" phase of a suspend or power-off transition, so all of this is not super-worrisome, but I agree that avoiding all of this GFP mask restrict/restore dance would be better overall.
Quoted 3 lines
I'm going to remove the
pm_restrict_gfp_mask()andpm_restore_gfp_mask()calls fromdpm_suspend_start()anddpm_suspend_end(), respectively, but that's not a change for 6.18.
And I've decided to do something else:
https://lore.kernel.org/linux-pm/[email protected]/
because having those calls in dpm_suspend_start() and dpm_suspend_end(), respectively, makes sense in general as the GFP mask needs to be restricted after the "prepare" phase of suspend.