Skip to contentSkip to conversation

i2c: designware: Fix unbalanced suspended flag

PATCH Superseded Richard Fitzgerald 3 participants 8 messages Dec 9, 2022

Subsystems (3)

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 i2c: designware: Fix unbalanced suspended flag

Ensure that i2c_mark_adapter_suspended() is always balanced by a call to i2c_mark_adapter_resumed().

Don't set DPM_FLAG_MAY_SKIP_RESUME to skip system early_resume stage if the driver was runtime-suspended. Instead, always call dw_i2c_plat_resume() and use pm_runtime_suspended() to determine whether we need to power up the hardware.

The unbalanced suspended flag was introduced by
commit c57813b8b288 ("i2c: designware: Lock the adapter while setting the suspended flag")

Before that commit, the system and runtime PM used the same functions. The DPM_FLAG_MAY_SKIP_RESUME was used to skip the system resume if the driver had been in runtime-suspend. If system resume was skipped, the suspended flag would be cleared by the next runtime resume. The check of the suspended flag was _after_ the call to pm_runtime_get_sync() in i2c_dw_xfer(). So either a system resume or a runtime resume would clear the flag before it was checked.

Having introduced the unbalanced suspended flag with that commit, a further commit 80704a84a9f8
("i2c: designware: Use the i2c_mark_adapter_suspended/resumed() helpers")

changed from using a local suspended flag to using the i2c_mark_adapter_suspended/resumed() functions. These use a flag that is checked by I2C core code before issuing the transfer to the bus driver, so there was no opportunity for the bus driver to runtime resume itself before the flag check.

Fixes c57813b8b288 ("i2c: designware: Lock the adapter while setting the suspended flag")

---
Apologies if you get this message multiple times. I'm having trouble with my SMTP server.

---
 drivers/i2c/busses/i2c-designware-platdrv.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
drivers/i2c/busses/i2c-designware-platdrv.c +5 -5
index ba043b547393..d805b8c7e797 100644
@@ -351,13 +351,11 @@ static int dw_i2c_plat_probe(struct platform_device *pdev)
 
 	if (dev->flags & ACCESS_NO_IRQ_SUSPEND) {
 		dev_pm_set_driver_flags(&pdev->dev,
-					DPM_FLAG_SMART_PREPARE |
-					DPM_FLAG_MAY_SKIP_RESUME);
+					DPM_FLAG_SMART_PREPARE);
 	} else {
 		dev_pm_set_driver_flags(&pdev->dev,
 					DPM_FLAG_SMART_PREPARE |
-					DPM_FLAG_SMART_SUSPEND |
-					DPM_FLAG_MAY_SKIP_RESUME);
+					DPM_FLAG_SMART_SUSPEND);
 	}
 
 	device_enable_async_suspend(&pdev->dev);
@@ -475,7 +473,9 @@ static int __maybe_unused dw_i2c_plat_resume(struct device *dev)
 {
 	struct dw_i2c_dev *i_dev = dev_get_drvdata(dev);
 
-	dw_i2c_plat_runtime_resume(dev);
+	if (!pm_runtime_suspended(dev))
+		dw_i2c_plat_runtime_resume(dev);
+
 	i2c_mark_adapter_resumed(&i_dev->adapter);
 
 	return 0;
Signature · 1 line
2.30.2
#1 lore raw

Hi Richard,

Quoted 24 lines · Richard Fitzgerald

Ensure that i2c_mark_adapter_suspended() is always balanced by a call to i2c_mark_adapter_resumed().

Don't set DPM_FLAG_MAY_SKIP_RESUME to skip system early_resume stage if the driver was runtime-suspended. Instead, always call dw_i2c_plat_resume() and use pm_runtime_suspended() to determine whether we need to power up the hardware.

The unbalanced suspended flag was introduced by
commit c57813b8b288 ("i2c: designware: Lock the adapter while setting the suspended flag")

Before that commit, the system and runtime PM used the same functions. The DPM_FLAG_MAY_SKIP_RESUME was used to skip the system resume if the driver had been in runtime-suspend. If system resume was skipped, the suspended flag would be cleared by the next runtime resume. The check of the suspended flag was _after_ the call to pm_runtime_get_sync() in i2c_dw_xfer(). So either a system resume or a runtime resume would clear the flag before it was checked.

Having introduced the unbalanced suspended flag with that commit, a further commit 80704a84a9f8
("i2c: designware: Use the i2c_mark_adapter_suspended/resumed() helpers")

Quoted 8 lines

changed from using a local suspended flag to using the i2c_mark_adapter_suspended/resumed() functions. These use a flag that is checked by I2C core code before issuing the transfer to the bus driver, so there was no opportunity for the bus driver to runtime resume itself before the flag check.

Fixes c57813b8b288 ("i2c: designware: Lock the adapter while setting the suspended flag")

It is not entirely clear to me where the unbalance you claim to see comes from? When runtime-suspended SMART_SUSPEND should keep it suspended at which point the system suspend callback will never run ?

Are you sure that you are not maybe seeing a suspend/resume ordering issue?

Did you add printk messages to the suspend/resume callbacks of i2c-designware-platdrv.c which show the system suspend callback being called but not the system resume one ?

I guess that is possible with DPM_FLAG_MAY_SKIP_RESUME, but since we also use SMART_SUSPEND I would expect the system-suspend callback to also always be skipped for runtime-suspended controllers.

Quoted 28 lines

---
Apologies if you get this message multiple times. I'm having trouble with my SMTP server.

---
 drivers/i2c/busses/i2c-designware-platdrv.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
drivers/i2c/busses/i2c-designware-platdrv.c +2 -4
index ba043b547393..d805b8c7e797 100644
@@ -351,13 +351,11 @@ static int dw_i2c_plat_probe(struct platform_device *pdev)
 
 	if (dev->flags & ACCESS_NO_IRQ_SUSPEND) {
 		dev_pm_set_driver_flags(&pdev->dev,
-					DPM_FLAG_SMART_PREPARE |
-					DPM_FLAG_MAY_SKIP_RESUME);
+					DPM_FLAG_SMART_PREPARE);
 	} else {
 		dev_pm_set_driver_flags(&pdev->dev,
 					DPM_FLAG_SMART_PREPARE |
-					DPM_FLAG_SMART_SUSPEND |
-					DPM_FLAG_MAY_SKIP_RESUME);
+					DPM_FLAG_SMART_SUSPEND);
 	}
 
 	device_enable_async_suspend(&pdev->dev);

drivers/i2c/busses/i2c-designware-platdrv.c static int __maybe_unused dw_i2c_plat_resume(struct device *dev) · line 478

Quoted 7 lines
drivers/i2c/busses/i2c-designware-platdrv.c +3 -1
@@ -475,7 +473,9 @@ static int __maybe_unused dw_i2c_plat_resume(struct device *dev)
 {
 	struct dw_i2c_dev *i_dev = dev_get_drvdata(dev);
 
-	dw_i2c_plat_runtime_resume(dev);
+	if (!pm_runtime_suspended(dev))
+		dw_i2c_plat_runtime_resume(dev);
+

I'm afraid that this is always going to run now, before this callback gets called drivers/base/power/main.c: device_resume_noirq() does:

        skip_resume = dev_pm_skip_resume(dev);

        if (skip_resume)
                pm_runtime_set_suspended(dev);
        else if (dev_pm_skip_suspend(dev))
                pm_runtime_set_active(dev);

Where skip_resume now is false since you dropped the DPM_FLAG_MAY_SKIP_RESUME flag and dev_pm_skip_suspend(dev) will return true (see below) for runtime-suspended controllers, so they will be marked active here. and then your !pm_runtime_suspended(dev) will always be false.

Did you add a printk to both the if + else paths
and have you ever seen the controller not get
resumed with this test added ?

Regards,

Hans

bool dev_pm_skip_suspend(struct device *dev)

{
        return dev_pm_test_driver_flags(dev, DPM_FLAG_SMART_SUSPEND) &&
                pm_runtime_status_suspended(dev);
}
Quoted 3 lines
drivers/i2c/busses/i2c-designware-platdrv.c +0 -0
 	i2c_mark_adapter_resumed(&i_dev->adapter);
 
 	return 0;
#2 lore raw
Quoted 33 lines · Hans de Goede

Hi Richard,

Quoted 30 lines · Richard Fitzgerald

Ensure that i2c_mark_adapter_suspended() is always balanced by a call to i2c_mark_adapter_resumed().

Don't set DPM_FLAG_MAY_SKIP_RESUME to skip system early_resume stage if the driver was runtime-suspended. Instead, always call dw_i2c_plat_resume() and use pm_runtime_suspended() to determine whether we need to power up the hardware.

The unbalanced suspended flag was introduced by
commit c57813b8b288 ("i2c: designware: Lock the adapter while setting the suspended flag")

Before that commit, the system and runtime PM used the same functions. The DPM_FLAG_MAY_SKIP_RESUME was used to skip the system resume if the driver had been in runtime-suspend. If system resume was skipped, the suspended flag would be cleared by the next runtime resume. The check of the suspended flag was _after_ the call to pm_runtime_get_sync() in i2c_dw_xfer(). So either a system resume or a runtime resume would clear the flag before it was checked.

Having introduced the unbalanced suspended flag with that commit, a further commit 80704a84a9f8
("i2c: designware: Use the i2c_mark_adapter_suspended/resumed() helpers")

changed from using a local suspended flag to using the i2c_mark_adapter_suspended/resumed() functions. These use a flag that is checked by I2C core code before issuing the transfer to the bus driver, so there was no opportunity for the bus driver to runtime resume itself before the flag check.

Quoted 6 lines
Quoted 2 lines
Fixes c57813b8b288 ("i2c: designware: Lock the adapter while setting the suspended flag")

It is not entirely clear to me where the unbalance you claim to see comes from? When runtime-suspended SMART_SUSPEND should keep it suspended at which point the system suspend callback will never run ?

system suspend callback is called, which calls
i2c_mark_adapter_suspended().

system resume is NOT called so i2c_mark_adapter_resumed() is NOT called.

A subsequent audio playback using an I2C audio amp then does a runtime resume but the amp driver then gets a "Transfer while suspended" error when it tried to access the part over I2C during its own runtime resume.

Tested on Aaeon UpXstreme WHL (Intel Whiskylake chipset)

Looking in __device_suspend_late() (drivers/base/power/main.c) there are cases which will skip the SMART_SUSPEND check.

Quoted 7 lines

Are you sure that you are not maybe seeing a suspend/resume ordering issue?

Did you add printk messages to the suspend/resume callbacks of i2c-designware-platdrv.c which show the system suspend callback being called but not the system resume one ?

Yes, that's what I did.
system suspend callback is called. System resume callback isn't.

Quoted 86 lines

I guess that is possible with DPM_FLAG_MAY_SKIP_RESUME, but since we also use SMART_SUSPEND I would expect the system-suspend callback to also always be skipped for runtime-suspended controllers.

Quoted 28 lines

---
Apologies if you get this message multiple times. I'm having trouble with my SMTP server.

---
  drivers/i2c/busses/i2c-designware-platdrv.c | 10 +++++-----
  1 file changed, 5 insertions(+), 5 deletions(-)
drivers/i2c/busses/i2c-designware-platdrv.c +2 -4
index ba043b547393..d805b8c7e797 100644
@@ -351,13 +351,11 @@ static int dw_i2c_plat_probe(struct platform_device *pdev)
 
  	if (dev->flags & ACCESS_NO_IRQ_SUSPEND) {
  		dev_pm_set_driver_flags(&pdev->dev,
-					DPM_FLAG_SMART_PREPARE |
-					DPM_FLAG_MAY_SKIP_RESUME);
+					DPM_FLAG_SMART_PREPARE);
  	} else {
  		dev_pm_set_driver_flags(&pdev->dev,
  					DPM_FLAG_SMART_PREPARE |
-					DPM_FLAG_SMART_SUSPEND |
-					DPM_FLAG_MAY_SKIP_RESUME);
+					DPM_FLAG_SMART_SUSPEND);
  	}
 
  	device_enable_async_suspend(&pdev->dev);

drivers/i2c/busses/i2c-designware-platdrv.c static int __maybe_unused dw_i2c_plat_resume(struct device *dev) · line 478

Quoted 7 lines
drivers/i2c/busses/i2c-designware-platdrv.c +3 -1
@@ -475,7 +473,9 @@ static int __maybe_unused dw_i2c_plat_resume(struct device *dev)
  {
  	struct dw_i2c_dev *i_dev = dev_get_drvdata(dev);
 
-	dw_i2c_plat_runtime_resume(dev);
+	if (!pm_runtime_suspended(dev))
+		dw_i2c_plat_runtime_resume(dev);
+

I'm afraid that this is always going to run now, before this callback gets called drivers/base/power/main.c: device_resume_noirq() does:

         skip_resume = dev_pm_skip_resume(dev);

         if (skip_resume)
                 pm_runtime_set_suspended(dev);
         else if (dev_pm_skip_suspend(dev))
                 pm_runtime_set_active(dev);

Where skip_resume now is false since you dropped the DPM_FLAG_MAY_SKIP_RESUME flag and dev_pm_skip_suspend(dev) will return true (see below) for runtime-suspended controllers, so they will be marked active here. and then your !pm_runtime_suspended(dev) will always be false.

Did you add a printk to both the if + else paths
and have you ever seen the controller not get
resumed with this test added ?

Regards,

Hans

bool dev_pm_skip_suspend(struct device *dev)

{
         return dev_pm_test_driver_flags(dev, DPM_FLAG_SMART_SUSPEND) &&
                 pm_runtime_status_suspended(dev);
}
Quoted 3 lines
drivers/i2c/busses/i2c-designware-platdrv.c +0 -0
  	i2c_mark_adapter_resumed(&i_dev->adapter);
 
  	return 0;
#3 lore raw
In reply to Hans de Goede
Quoted 5 lines · Hans de Goede

Hi Richard,

Quoted 2 lines · Richard Fitzgerald

Ensure that i2c_mark_adapter_suspended() is always balanced by a call to i2c_mark_adapter_resumed().

<snip>

Quoted 11 lines

It is not entirely clear to me where the unbalance you claim to see comes from? When runtime-suspended SMART_SUSPEND should keep it suspended at which point the system suspend callback will never run ?

Are you sure that you are not maybe seeing a suspend/resume ordering issue?

Did you add printk messages to the suspend/resume callbacks of i2c-designware-platdrv.c which show the system suspend callback being called but not the system resume one ?

With messages in strategic places.

[  169.607358] i2c_designware i2c_designware.2: PM: dev_pm_skip_suspend: SMART_SUSPEND=0 pm_runtime_status_suspended=1
[  169.607361] i2c_designware i2c_designware.2: PM: __device_suspend_late: dev_pm_skip_suspend:false
[  169.607364] i2c_designware i2c_designware.2: dw_i2c_plat_suspend
...
[  169.702511] i2c_designware i2c_designware.2: PM: dev_pm_skip_resume: 1 because !power.must_resume
[  169.706241] i2c_designware i2c_designware.2: PM: dev_pm_skip_resume: 1 because !power.must_resume
[  169.706244] i2c_designware i2c_designware.2: PM: device_resume_early: dev_pm_skip_resume:true
...
[  175.254832] i2c i2c-2: Transfer while suspended

(Just to prove my logging isn't lying, for i2c3 it reports SMART_SUSPEND=1)

So it can skip the resume even if it didn't skip the suspend. The SMART_SUSPEND flag is not set on i2c2 and the driver core can skip resume even if it didn't skip suspend.

#4 lore raw
Quoted 33 lines · Richard Fitzgerald
Quoted 5 lines · Hans de Goede

Hi Richard,

Quoted 2 lines · Richard Fitzgerald

Ensure that i2c_mark_adapter_suspended() is always balanced by a call to i2c_mark_adapter_resumed().

<snip>

Quoted 11 lines

It is not entirely clear to me where the unbalance you claim to see comes from? When runtime-suspended SMART_SUSPEND should keep it suspended at which point the system suspend callback will never run ?

Are you sure that you are not maybe seeing a suspend/resume ordering issue?

Did you add printk messages to the suspend/resume callbacks of i2c-designware-platdrv.c which show the system suspend callback being called but not the system resume one ?

With messages in strategic places.

[  169.607358] i2c_designware i2c_designware.2: PM: dev_pm_skip_suspend: SMART_SUSPEND=0 pm_runtime_status_suspended=1 [  169.607361] i2c_designware i2c_designware.2: PM: __device_suspend_late: dev_pm_skip_suspend:false [  169.607364] i2c_designware i2c_designware.2: dw_i2c_plat_suspend
...
[  169.702511] i2c_designware i2c_designware.2: PM: dev_pm_skip_resume: 1 because !power.must_resume [  169.706241] i2c_designware i2c_designware.2: PM: dev_pm_skip_resume: 1 because !power.must_resume [  169.706244] i2c_designware i2c_designware.2: PM: device_resume_early: dev_pm_skip_resume:true
...
[  175.254832] i2c i2c-2: Transfer while suspended

Quoted 3 lines

(Just to prove my logging isn't lying, for i2c3 it reports SMART_SUSPEND=1)

Oh, that's embarrassing. After confidently telling you my logging is perfect, actually there was a bug in it...

New log summary:

[  162.253431] i2c_designware i2c_designware.2: PM: dev_pm_skip_suspend: SMART_SUSPEND=1 pm_runtime_status_suspended=0
[  162.253438] i2c_designware i2c_designware.2: PM: __device_suspend_late: dev_pm_skip_suspend:false
[  162.253445] i2c_designware i2c_designware.2: dw_i2c_plat_suspend
[  162.273115] i2c_designware i2c_designware.2: PM: dev_pm_skip_suspend: SMART_SUSPEND=1 pm_runtime_status_suspended=0
[  162.362547] i2c_designware i2c_designware.2: PM: dev_pm_skip_resume: 1 because !power.must_resume
[  162.369216] i2c_designware i2c_designware.2: PM: dev_pm_skip_resume: 1 because !power.must_resume
[  162.369220] i2c_designware i2c_designware.2: PM: device_resume_early: dev_pm_skip_resume:true
[  167.901269] i2c i2c-2: Transfer while suspended

Same result that it doesn't skip suspend but does skip resume.

#5 lore raw

Hi,

Quoted 38 lines · Richard Fitzgerald
Quoted 33 lines · Richard Fitzgerald
Quoted 5 lines · Hans de Goede

Hi Richard,

Quoted 2 lines · Richard Fitzgerald

Ensure that i2c_mark_adapter_suspended() is always balanced by a call to i2c_mark_adapter_resumed().

<snip>

Quoted 11 lines

It is not entirely clear to me where the unbalance you claim to see comes from? When runtime-suspended SMART_SUSPEND should keep it suspended at which point the system suspend callback will never run ?

Are you sure that you are not maybe seeing a suspend/resume ordering issue?

Did you add printk messages to the suspend/resume callbacks of i2c-designware-platdrv.c which show the system suspend callback being called but not the system resume one ?

With messages in strategic places.

[  169.607358] i2c_designware i2c_designware.2: PM: dev_pm_skip_suspend: SMART_SUSPEND=0 pm_runtime_status_suspended=1 [  169.607361] i2c_designware i2c_designware.2: PM: __device_suspend_late: dev_pm_skip_suspend:false [  169.607364] i2c_designware i2c_designware.2: dw_i2c_plat_suspend
...
[  169.702511] i2c_designware i2c_designware.2: PM: dev_pm_skip_resume: 1 because !power.must_resume [  169.706241] i2c_designware i2c_designware.2: PM: dev_pm_skip_resume: 1 because !power.must_resume [  169.706244] i2c_designware i2c_designware.2: PM: device_resume_early: dev_pm_skip_resume:true
...
[  175.254832] i2c i2c-2: Transfer while suspended

Quoted 3 lines

(Just to prove my logging isn't lying, for i2c3 it reports SMART_SUSPEND=1)

Quoted 6 lines

Oh, that's embarrassing. After confidently telling you my logging is perfect, actually there was a bug in it...

New log summary:

[  162.253431] i2c_designware i2c_designware.2: PM: dev_pm_skip_suspend: SMART_SUSPEND=1 pm_runtime_status_suspended=0

Ok, so the device's pm_runtime_get() count is 0 here (otherwise must_resume should be 1 later on) but the device is not run-time suspended yet. Probably because of some timeout; or because of runtime pm getting disabled durig suspend before the count dropped to 0.

And this scenario will indeed cause the system-level suspend callback to get called, but not the resume one ...

Quoted 9 lines

[  162.253438] i2c_designware i2c_designware.2: PM: __device_suspend_late: dev_pm_skip_suspend:false [  162.253445] i2c_designware i2c_designware.2: dw_i2c_plat_suspend
[  162.273115] i2c_designware i2c_designware.2: PM: dev_pm_skip_suspend: SMART_SUSPEND=1 pm_runtime_status_suspended=0 [  162.362547] i2c_designware i2c_designware.2: PM: dev_pm_skip_resume: 1 because !power.must_resume [  162.369216] i2c_designware i2c_designware.2: PM: dev_pm_skip_resume: 1 because !power.must_resume [  162.369220] i2c_designware i2c_designware.2: PM: device_resume_early: dev_pm_skip_resume:true
[  167.901269] i2c i2c-2: Transfer while suspended

Same result that it doesn't skip suspend but does skip resume.

From your other email:

Quoted 2 lines

Ok, what do you suggest as the fix?
If you post an alternate fix I can test it.

I don't really see a better solution, so lets go with your solution, but then:

1. Simply drop the flag but don't add the if (!pm_runtime_suspended(dev)) check. The runtime status is always going to be set to active at this point so the check does not do anything.

2. Drop the dw_i2c_plat_complete() callback since we now always resume the controller on system resume.

Regards,

Hans

#6 lore raw
In reply to Hans de Goede
Quoted 10 lines · Hans de Goede

Hi Richard,

Quoted 3 lines · Richard Fitzgerald

Ensure that i2c_mark_adapter_suspended() is always balanced by a call to i2c_mark_adapter_resumed().

I'm afraid that this is always going to run now, before this callback gets called drivers/base/power/main.c: device_resume_noirq() does:

Ok, what do you suggest as the fix?
If you post an alternate fix I can test it.

#7 lore raw
In reply to Hans de Goede
Quoted 1 line · Hans de Goede

On 12/9/22 12:40, Richard Fitzgerald wrote:

...

Quoted 3 lines

Did you add printk messages to the suspend/resume callbacks of i2c-designware-platdrv.c which show the system suspend callback being called but not the system resume one ?

Side note: It's better to use ftrace for that, less invasive time-wise.

...

Quoted 3 lines

Did you add a printk to both the if + else paths
and have you ever seen the controller not get
resumed with this test added ?

Signature · 2 lines
With Best Regards,
Andy Shevchenko