r8152: use GFP_NOIO during system suspend
Subsystems (4)
PATCH 1/1 r8152: use GFP_NOIO during system suspend
During system suspend, memory allocation with GFP_KERNEL can block waiting for I/O to complete. If that I/O depends on a device that is itself suspended, a deadlock results.
Introduce RTL8152_SYSTEM_SUSPEND flag to track when the driver is operating in the system suspend/resume context. Set the flag at the start of rtl8152_system_suspend() and clear it at the end of rtl8152_system_resume(), following the same pattern used by SELECTIVE_SUSPEND for runtime suspend.
In get_registers() and set_registers(), select GFP_NOIO when the flag is set so that the kmalloc and kmemdup calls in those paths do not trigger I/O reclaim.
--- drivers/net/usb/r8152.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-)
drivers/net/usb/r8152.c +15 -2
@@ -792,6 +792,7 @@ enum rtl8152_flags {IN_PRE_RESET ,PROBED_WITH_NO_ERRORS ,PROBE_SHOULD_RETRY , + RTL8152_SYSTEM_SUSPEND, };#define DEVICE_ID_LENOVO_USB_C_TRAVEL_HUB 0x721e
@@ -1370,9 +1371,12 @@ staticint get_registers (struct r8152 *tp,u16 value,u16 index,u16 size,void *data) {int ret; +gfp_t gfp;void *tmp; - tmp =kmalloc (size,GFP_KERNEL ); + gfp =test_bit (RTL8152_SYSTEM_SUSPEND , &tp->flags) ?GFP_NOIO : +GFP_KERNEL ; + tmp =kmalloc (size, gfp);if (!tmp)return -ENOMEM ;
@@ -1394,9 +1398,12 @@ staticint set_registers (struct r8152 *tp,u16 value,u16 index,u16 size,void *data) {int ret; +gfp_t gfp;void *tmp; - tmp =kmemdup (data, size,GFP_KERNEL ); + gfp =test_bit (RTL8152_SYSTEM_SUSPEND , &tp->flags) ?GFP_NOIO : +GFP_KERNEL ; + tmp =kmemdup (data, size, gfp);if (!tmp)return -ENOMEM ;
@@ -8693,6 +8700,9 @@ static int rtl8152_system_resume(struct r8152 *tp)usb_submit_urb (tp->intr_urb,GFP_NOIO ); } +clear_bit (RTL8152_SYSTEM_SUSPEND , &tp->flags); +smp_mb__after_atomic (); +return 0 ; }
@@ -8758,6 +8768,9 @@ static int rtl8152_system_suspend(struct r8152 *tp) {struct net_device *netdev = tp->netdev; +set_bit (RTL8152_SYSTEM_SUSPEND , &tp->flags); +smp_mb__after_atomic (); +netif_device_detach (netdev);if (netif_running (netdev) &&test_bit (WORK_ENABLE , &tp->flags)) {
Signature · 1 line
2.34.1
Quoted 10 lines · Chih Kai Hsu
During system suspend, memory allocation with
GFP_KERNELcan block waiting for I/O to complete. If that I/O depends on a device that is itself suspended, a deadlock results.Introduce
RTL8152_SYSTEM_SUSPENDflag to track when the driver is operating in the system suspend/resume context. Set the flag at the start ofrtl8152_system_suspend()and clear it at the end ofrtl8152_system_resume(), following the same pattern used bySELECTIVE_SUSPENDfor runtime suspend.
Quoted 3 lines
In
get_registers()andset_registers(), selectGFP_NOIOwhen the flag is set so that the kmalloc and kmemdup calls in those paths do not trigger I/O reclaim.
Do you have a reproducer for this? Or is it theoretical?
Quoted 11 lines
drivers/net/usb/r8152.c +3
-0@@ -8693,6 +8700,9 @@ static int rtl8152_system_resume(struct r8152 *tp)usb_submit_urb (tp->intr_urb,GFP_NOIO ); } +clear_bit (RTL8152_SYSTEM_SUSPEND , &tp->flags); +smp_mb__after_atomic (); +return 0 ; }
drivers/net/usb/r8152.c static int rtl8152_system_suspend(struct r8152 *tp) · line 8776
Quoted 9 lines
drivers/net/usb/r8152.c +3
-0@@ -8758,6 +8768,9 @@ static int rtl8152_system_suspend(struct r8152 *tp) {struct net_device *netdev = tp->netdev; +set_bit (RTL8152_SYSTEM_SUSPEND , &tp->flags); +smp_mb__after_atomic (); +netif_device_detach (netdev);if (netif_running (netdev) &&test_bit (WORK_ENABLE , &tp->flags)) {
It seems odd to me that every driver needs to track if it is in suspend/resume or not. Doesn't the struct device tell you? Is there no global state somewhere?
Andrew
Quoted 11 lines · Andrew Lunn
Quoted 10 lines · Chih Kai Hsu
During system suspend, memory allocation with
GFP_KERNELcan block waiting for I/O to complete. If that I/O depends on a device that is itself suspended, a deadlock results.Introduce
RTL8152_SYSTEM_SUSPENDflag to track when the driver is operating in the system suspend/resume context. Set the flag at the start ofrtl8152_system_suspend()and clear it at the end ofrtl8152_system_resume(), following the same pattern used bySELECTIVE_SUSPENDfor runtime suspend.
Quoted 6 lines
Quoted 3 lines
In
get_registers()andset_registers(), selectGFP_NOIOwhen the flag is set so that the kmalloc and kmemdup calls in those paths do not trigger I/O reclaim.Do you have a reproducer for this? Or is it theoretical?
It is theoretical.
Quoted 19 lines
Quoted 19 lines
drivers/net/usb/r8152.c +6
-0@@ -8693,6 +8700,9 @@ static int rtl8152_system_resume(struct r8152 *tp)usb_submit_urb (tp->intr_urb,GFP_NOIO ); } +clear_bit (RTL8152_SYSTEM_SUSPEND , &tp->flags); +smp_mb__after_atomic (); +return 0 ; }@@ -8758,6 +8768,9 @@ static int rtl8152_system_suspend(struct r8152 *tp) {struct net_device *netdev = tp->netdev; +set_bit (RTL8152_SYSTEM_SUSPEND , &tp->flags); +smp_mb__after_atomic (); +netif_device_detach (netdev);
Quoted 5 lines
Quoted 2 lines
(quoted diff) +0
-0if (netif_running(netdev) && test_bit(WORK_ENABLE, &tp->flags)){
It seems odd to me that every driver needs to track if it is in suspend/resume or not. Doesn't the struct device tell you? Is there no global state somewhere?
I found https://elixir.bootlin.com/linux/v7.2-rc6/source/drivers/base/power/main.c#L2343
in the Linux source code.
pm_restrict_gfp_mask() is called before dpm_suspend(state). It removes __GFP_IO and __GFP_FS from the global gfp_t gfp_allowed_mask. Therefore, I think the driver can use GFP_KERNEL safely. Is this correct? Could you please provide any advice?
Quoted 2 lines
Andrew
Best,
Chih-Kai
Quoted 60 lines · Andrew Lunn
Quoted 11 lines · Chih Kai Hsu
Quoted 10 lines
During system suspend, memory allocation with
GFP_KERNELcan block waiting for I/O to complete. If that I/O depends on a device that is itself suspended, a deadlock results.Introduce
RTL8152_SYSTEM_SUSPENDflag to track when the driver is operating in the system suspend/resume context. Set the flag at the start ofrtl8152_system_suspend()and clear it at the end ofrtl8152_system_resume(), following the same pattern used bySELECTIVE_SUSPENDfor runtime suspend.Quoted 6 lines
Quoted 3 lines
In
get_registers()andset_registers(), selectGFP_NOIOwhen the flag is set so that the kmalloc and kmemdup calls in those paths do not trigger I/O reclaim.Do you have a reproducer for this? Or is it theoretical?
It is theoretical.
Quoted 1 line
Review static int rtl8152_system_resume(struct r8152
Quoted 1 line
(quoted diff) +0
-0@@ -8693,6 +8700,9 @@ static int rtl8152_system_resume(struct r8152*tp)
Quoted 18 lines
Quoted 18 lines
drivers/net/usb/r8152.c +6
-0usb_submit_urb (tp->intr_urb,GFP_NOIO ); } +clear_bit (RTL8152_SYSTEM_SUSPEND , &tp->flags); +smp_mb__after_atomic (); +return 0 ; }@@ -8758,6 +8768,9 @@ static int rtl8152_system_suspend(struct r8152 *tp) {struct net_device *netdev = tp->netdev; +set_bit (RTL8152_SYSTEM_SUSPEND , &tp->flags); +smp_mb__after_atomic (); +netif_device_detach (netdev);Quoted 5 lines
Quoted 2 lines
(quoted diff) +0
-0if (netif_running(netdev) && test_bit(WORK_ENABLE,
&tp->flags)) {It seems odd to me that every driver needs to track if it is in suspend/resume or not. Doesn't the struct device tell you? Is there no global
state somewhere?
I found
https://elixir.bootlin.com/linux/v7.2-rc6/source/drivers/base/power/main.c#L
2343
in the Linux source code.
pm_restrict_gfp_mask()is called beforedpm_suspend(state). It removes__GFP_IOand__GFP_FSfrom the globalgfp_tgfp_allowed_mask. Therefore, I think the driver can useGFP_KERNELsafely. Is this correct? Could you please provide any advice?Quoted 1 line
Quoted 4 lines
Quoted 1 line
Andrew
Best,
Chih-Kai
Hi Andrew,
Just a gentle ping on this patch.
Based on my previous observation, should the driver use GFP_NOIO explicitly?
Thank you for your time.
Best,
Chih-Kai
Quoted 62 lines · Chih Kai Hsu
Quoted 62 lines · Andrew Lunn
Quoted 11 lines · Chih Kai Hsu
Quoted 10 lines
During system suspend, memory allocation with
GFP_KERNELcan block waiting for I/O to complete. If that I/O depends on a device that is itself suspended, a deadlock results.Introduce
RTL8152_SYSTEM_SUSPENDflag to track when the driver is operating in the system suspend/resume context. Set the flag at the start ofrtl8152_system_suspend()and clear it at the end ofrtl8152_system_resume(), following the same pattern used bySELECTIVE_SUSPENDfor runtime suspend.Quoted 6 lines
Quoted 3 lines
In
get_registers()andset_registers(), selectGFP_NOIOwhen the flag is set so that the kmalloc and kmemdup calls in those paths do not trigger I/O reclaim.Do you have a reproducer for this? Or is it theoretical?
It is theoretical.
Quoted 1 line
Review static int rtl8152_system_resume(struct r8152
Quoted 1 line
(quoted diff) +0
-0@@ -8693,6 +8700,9 @@ static int rtl8152_system_resume(struct r8152*tp)
Quoted 18 lines
Quoted 18 lines
drivers/net/usb/r8152.c +6
-0usb_submit_urb (tp->intr_urb,GFP_NOIO ); } +clear_bit (RTL8152_SYSTEM_SUSPEND , &tp->flags); +smp_mb__after_atomic (); +return 0 ; }@@ -8758,6 +8768,9 @@ static int rtl8152_system_suspend(struct r8152 *tp) {struct net_device *netdev = tp->netdev; +set_bit (RTL8152_SYSTEM_SUSPEND , &tp->flags); +smp_mb__after_atomic (); +netif_device_detach (netdev);Quoted 5 lines
Quoted 2 lines
(quoted diff) +0
-0if (netif_running(netdev) && test_bit(WORK_ENABLE,
&tp->flags)) {It seems odd to me that every driver needs to track if it is in suspend/resume or not. Doesn't the struct device tell you? Is there no global
state somewhere?
I found
https://elixir.bootlin.com/linux/v7.2-rc6/source/drivers/base/power/main.c#L
2343
in the Linux source code.
pm_restrict_gfp_mask()is called beforedpm_suspend(state). It removes__GFP_IOand__GFP_FSfrom the globalgfp_tgfp_allowed_mask. Therefore, I think the driver can useGFP_KERNELsafely. Is this correct? Could you please provide any advice?Quoted 2 lines
Andrew
Quoted 7 lines
Quoted 2 lines
Best,
Chih-KaiHi Andrew,
Just a gentle ping on this patch.
Based on my previous observation, should the driver useGFP_NOIOexplicitly?
Please try to answer the question yourself. There are lots of drivers out there. What do they do? What does the suspend/resume documentation say?
Andrew
Quoted 44 lines · Andrew
Quoted 43 lines · Chih Kai Hsu
Quoted 43 lines · Andrew Lunn
Quoted 11 lines · Chih Kai Hsu
Quoted 10 lines
During system suspend, memory allocation with
GFP_KERNELcan block waiting for I/O to complete. If that I/O depends on a device that is itself suspended, a deadlock results.Introduce
RTL8152_SYSTEM_SUSPENDflag to track when the driver is operating in the system suspend/resume context. Set the flag at the start ofrtl8152_system_suspend()and clear it at the end ofrtl8152_system_resume(), following the same pattern used bySELECTIVE_SUSPENDfor runtime suspend.Quoted 6 lines
Quoted 3 lines
In
get_registers()andset_registers(), selectGFP_NOIOwhen the flag is set so that the kmalloc and kmemdup calls in those paths do not trigger I/O reclaim.Do you have a reproducer for this? Or is it theoretical?
It is theoretical.
Quoted 2 lines
Quoted 2 lines
(quoted diff) +0
-0@@ -8693,6 +8700,9 @@ static int rtl8152_system_resume(structr8152
*tp)
Quoted 19 lines
Quoted 19 lines
usb_submit_urb(tp->intr_urb, GFP_NOIO); }drivers/net/usb/r8152.c +3
-0+clear_bit (RTL8152_SYSTEM_SUSPEND , &tp->flags); +smp_mb__after_atomic (); +return 0 ; }@@ -8758,6 +8768,9 @@ static int rtl8152_system_suspend(structr8152
*tp) { struct net_device *netdev = tp->netdev;drivers/net/usb/r8152.c +3
-0+set_bit (RTL8152_SYSTEM_SUSPEND , &tp->flags); +smp_mb__after_atomic (); +netif_device_detach (netdev);
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 2 lines
if (netif_running(netdev) && test_bit(WORK_ENABLE,
&tp->flags)) {It seems odd to me that every driver needs to track if it is in suspend/resume or not. Doesn't the struct device tell you? Is there no global
state somewhere?
The function hibernate_quiet_exec() used by the driver at drivers/nvdimm/core.c calls dpm_suspend() directly, so the driver cannot use functions such as pm_suspend_in_progress() or system_entering_hibernation() to determine whether it is in the suspend/hibernate path now.
The ax88179 driver at drivers/net/usb/ax88179_178a.c uses a private variable, in_pm, which is set when entering the suspend callback. After that, the driver uses ax88179_in_pm() to determine whether the driver is in the suspend/hibernate path now. Therefore, using a per-driver flag makes sense.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
I found
https://elixir.bootlin.com/linux/v7.2-rc6/source/drivers/base/power/
main.c#L
2343
in the Linux source code.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
pm_restrict_gfp_mask()is called beforedpm_suspend(state). It removes__GFP_IOand__GFP_FSfrom the globalgfp_tgfp_allowed_mask. Therefore, I think the driver can useGFP_KERNELsafely. Is this correct? Could you please provide any advice?
For the same reason, hibernate_quiet_exec() calls dpm_suspend() directly, so pm_restrict_gfp_mask() is never called. The driver should use GFP_NOIO in this case.
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Andrew
Best,
Chih-KaiHi Andrew,
Quoted 8 lines
Quoted 2 lines
Just a gentle ping on this patch.
Based on my previous observation, should the driver useGFP_NOIOexplicitly?
Please try to answer the question yourself. There are lots of drivers out there. What do they do? What does the suspend/resume documentation say?
Andrew
Best Regards,
Chih-Kai