i2c: mv64xxx: Fix random system lock caused by runtime PM
PATCH 1/1 i2c: mv64xxx: Fix random system lock caused by runtime PM
I noticed a weird bug with this driver on Marvell CN9130 Customer Reference Board.
Sometime after boot, the system locks with the following message:
[104.071363] i2c i2c-0: mv64xxx: I2C bus locked, block: 1, time_left: 0
The system does not respond afterwards, only warns about RCU stalls.
This first appeared with commit e5c02cf54154 ("i2c: mv64xxx: Add runtime PM support").
With further experimentation I discovered that adding a delay into mv64xxx_i2c_hw_init() fixes this issue. This function is called before every xfer, due to how runtime PM works in this driver. It seems that in order to work correctly, a delay is needed after the bus is reset in this function.
Since there already is a known erratum with this controller needing a delay, I assume that this is just another place this needs to be applied. Therefore I apply the delay only if errata_delay is true.
--- drivers/i2c/busses/i2c-mv64xxx.c | 4 ++++ 1 file changed, 4 insertions(+)
drivers/i2c/busses/i2c-mv64xxx.c +4 -0
@@ -221,6 +221,10 @@ mv64xxx_i2c_hw_init(struct mv64xxx_i2c_data *drv_data)writel (0 , drv_data->reg_base + drv_data->reg_offsets.ext_addr);writel (MV64XXX_I2C_REG_CONTROL_TWSIEN |MV64XXX_I2C_REG_CONTROL_STOP , drv_data->reg_base + drv_data->reg_offsets.control); + +if (drv_data->errata_delay) +udelay (5 ); + drv_data->state =MV64XXX_I2C_STATE_IDLE ; }
Signature · 1 line
2.26.2
Quoted 2 lines · On Thu, 8 Apr 2021 04:00:00 +0200 Marek Behún <[email protected]> wrote:
e5c02cf54154("i2c: mv64xxx: Add runtime
PM support").
This commit should also contain a Fixes tag:
Quoted 17 lines · Marek Behún
I noticed a weird bug with this driver on Marvell CN9130 Customer Reference Board.
Sometime after boot, the system locks with the following message:
[104.071363] i2c i2c-0: mv64xxx: I2C bus locked, block: 1, time_left: 0The system does not respond afterwards, only warns about RCU stalls.
This first appeared with commit
e5c02cf54154("i2c: mv64xxx: Add runtime PM support").With further experimentation I discovered that adding a delay into
mv64xxx_i2c_hw_init()fixes this issue. This function is called before every xfer, due to how runtime PM works in this driver. It seems that in order to work correctly, a delay is needed after the bus is reset in this function.
Quoted 5 lines
Since there already is a known erratum with this controller needing a delay, I assume that this is just another place this needs to be applied. Therefore I apply the delay only if
errata_delayis true.Signed-off-by Marek Behún <[email protected]>
Gregory? Looks reasonable to me and if so, we should have this in 5.12 already. Comments from others are welcome, too, of course.
Quoted 22 lines
--- drivers/i2c/busses/i2c-mv64xxx.c | 4 ++++ 1 file changed, 4 insertions(+)drivers/i2c/busses/i2c-mv64xxx.c +4
-0@@ -221,6 +221,10 @@ mv64xxx_i2c_hw_init(struct mv64xxx_i2c_data *drv_data)writel (0 , drv_data->reg_base + drv_data->reg_offsets.ext_addr);writel (MV64XXX_I2C_REG_CONTROL_TWSIEN |MV64XXX_I2C_REG_CONTROL_STOP , drv_data->reg_base + drv_data->reg_offsets.control); + +if (drv_data->errata_delay) +udelay (5 ); + drv_data->state =MV64XXX_I2C_STATE_IDLE ; }Signature · 1 line
2.26.2
Quoted 9 lines · Marek Behún
Quoted 8 lines
I noticed a weird bug with this driver on Marvell CN9130 Customer Reference Board.
Sometime after boot, the system locks with the following message:
[104.071363] i2c i2c-0: mv64xxx: I2C bus locked, block: 1, time_left: 0The system does not respond afterwards, only warns about RCU stalls.
Quoted 8 lines
Quoted 8 lines
This first appeared with commit
e5c02cf54154("i2c: mv64xxx: Add runtime PM support").With further experimentation I discovered that adding a delay into
mv64xxx_i2c_hw_init()fixes this issue. This function is called before every xfer, due to how runtime PM works in this driver. It seems that in order to work correctly, a delay is needed after the bus is reset in this function.
Marek,
As you mentioned it was related to reset and the issue occurred with the support of runtime PM. Did you try to add the delay only in the function mv64xxx_i2c_runtime_resume(), just after the mv64xxx_i2c_hw_init() call ?
Quoted 9 lines
Quoted 6 lines
Since there already is a known erratum with this controller needing a delay, I assume that this is just another place this needs to be applied. Therefore I apply the delay only if
errata_delayis true.Signed-off-by Marek Behún <[email protected]>Gregory? Looks reasonable to me and if so, we should have this in 5.12 already. Comments from others are welcome, too, of course.
Hello Wolfram,
I don't have this specific platform. However, as you said it looks reasonable and as it fixes an issue. And even if I had a pending question, it is just an optimisation so you can add my
Gregory
Quoted 23 lines
Quoted 23 lines
--- drivers/i2c/busses/i2c-mv64xxx.c | 4 ++++ 1 file changed, 4 insertions(+)drivers/i2c/busses/i2c-mv64xxx.c +4
-0@@ -221,6 +221,10 @@ mv64xxx_i2c_hw_init(struct mv64xxx_i2c_data *drv_data)writel (0 , drv_data->reg_base + drv_data->reg_offsets.ext_addr);writel (MV64XXX_I2C_REG_CONTROL_TWSIEN |MV64XXX_I2C_REG_CONTROL_STOP , drv_data->reg_base + drv_data->reg_offsets.control); + +if (drv_data->errata_delay) +udelay (5 ); + drv_data->state =MV64XXX_I2C_STATE_IDLE ; }Signature · 1 line
2.26.2
Signature · 3 lines
Gregory Clement, Bootlin Embedded Linux and Kernel engineering http://bootlin.com
Samuel Holland
Reviewed-byQuoted 18 lines · Wolfram Sang
Quoted 17 lines · Marek Behún
I noticed a weird bug with this driver on Marvell CN9130 Customer Reference Board.
Sometime after boot, the system locks with the following message:
[104.071363] i2c i2c-0: mv64xxx: I2C bus locked, block: 1, time_left: 0The system does not respond afterwards, only warns about RCU stalls.
This first appeared with commit
e5c02cf54154("i2c: mv64xxx: Add runtime PM support").With further experimentation I discovered that adding a delay into
mv64xxx_i2c_hw_init()fixes this issue. This function is called before every xfer, due to how runtime PM works in this driver. It seems that in order to work correctly, a delay is needed after the bus is reset in this function.
Quoted 8 lines
Quoted 5 lines
Since there already is a known erratum with this controller needing a delay, I assume that this is just another place this needs to be applied. Therefore I apply the delay only if
errata_delayis true.Signed-off-by Marek Behún <[email protected]>Gregory? Looks reasonable to me and if so, we should have this in 5.12 already. Comments from others are welcome, too, of course.
I also don't have an affected platform, so I did not notice this when adding runtime PM. The change makes sense to me as well.
Cheers,
Samuel
Quoted 22 lines
Quoted 22 lines
--- drivers/i2c/busses/i2c-mv64xxx.c | 4 ++++ 1 file changed, 4 insertions(+)drivers/i2c/busses/i2c-mv64xxx.c +4
-0@@ -221,6 +221,10 @@ mv64xxx_i2c_hw_init(struct mv64xxx_i2c_data *drv_data)writel (0 , drv_data->reg_base + drv_data->reg_offsets.ext_addr);writel (MV64XXX_I2C_REG_CONTROL_TWSIEN |MV64XXX_I2C_REG_CONTROL_STOP , drv_data->reg_base + drv_data->reg_offsets.control); + +if (drv_data->errata_delay) +udelay (5 ); + drv_data->state =MV64XXX_I2C_STATE_IDLE ; }Signature · 1 line
2.26.2
Quoted 18 lines · On Wed, 14 Apr 2021 15:28:18 +0200 Gregory CLEMENT…
Quoted 9 lines · Marek Behún
Quoted 8 lines
I noticed a weird bug with this driver on Marvell CN9130 Customer Reference Board.
Sometime after boot, the system locks with the following message:
[104.071363] i2c i2c-0: mv64xxx: I2C bus locked, block: 1, time_left: 0The system does not respond afterwards, only warns about RCU stalls.
Quoted 8 lines
Quoted 8 lines
This first appeared with commit
e5c02cf54154("i2c: mv64xxx: Add runtime PM support").With further experimentation I discovered that adding a delay into
mv64xxx_i2c_hw_init()fixes this issue. This function is called before every xfer, due to how runtime PM works in this driver. It seems that in order to work correctly, a delay is needed after the bus is reset in this function.
Quoted 6 lines
Marek,
As you mentioned it was related to reset and the issue occurred with the support of runtime PM. Did you try to add the delay only in the function
mv64xxx_i2c_runtime_resume(), just after themv64xxx_i2c_hw_init()call ?
I did indeed discover this when I added this delay into the resume function. In fact I discovered this when I added printf()s into suspend and resume when debugging. The problem disappeared with these printf()s (UART is slow so printf() counted as the necessary delay it seems).
I then moved the delay into the hw_init() function because that is what made sense to me, that the delay is necessary after the reset, not only when resuming, but always. We just did not notice because a xfer was never done immediately after reset before the PM patch. (But maybe I am wrong, maybe it is not needed in the reset. It just makes the most sense to me...)
Marek
Wolfram Sang
AppliedQuoted 17 lines · Marek Behún
I noticed a weird bug with this driver on Marvell CN9130 Customer Reference Board.
Sometime after boot, the system locks with the following message:
[104.071363] i2c i2c-0: mv64xxx: I2C bus locked, block: 1, time_left: 0The system does not respond afterwards, only warns about RCU stalls.
This first appeared with commit
e5c02cf54154("i2c: mv64xxx: Add runtime PM support").With further experimentation I discovered that adding a delay into
mv64xxx_i2c_hw_init()fixes this issue. This function is called before every xfer, due to how runtime PM works in this driver. It seems that in order to work correctly, a delay is needed after the bus is reset in this function.
Quoted 5 lines
Since there already is a known erratum with this controller needing a delay, I assume that this is just another place this needs to be applied. Therefore I apply the delay only if
errata_delayis true.Signed-off-by Marek Behún <[email protected]>
Applied to for-current, thanks!