i2c: aspeed: Nullify bus messages after timeout
PATCH 1/1 i2c: aspeed: Nullify bus messages after timeout
For multimaster case, it's conceivable that an interrupt comes in after the transfer times out and attempts to use bus messages that have already been freed by the i2c core.
--- drivers/i2c/busses/i2c-aspeed.c | 1 + 1 file changed, 1 insertion(+)
drivers/i2c/busses/i2c-aspeed.c +1 -0
@@ -731,6 +731,7 @@ static int aspeed_i2c_master_xfer(struct i2c_adapter *adap,* master command. */ spin_lock_irqsave (&bus->lock, flags); + bus->msgs =NULL ;if (bus->master_state ==ASPEED_I2C_MASTER_PENDING ) bus->master_state =ASPEED_I2C_MASTER_INACTIVE ;spin_unlock_irqrestore (&bus->lock, flags);
Signature · 1 line
2.43.5
Andrew Jeffery
maintainerQuoted 3 lines · Eddie James
For multimaster case, it's conceivable that an interrupt comes in after the transfer times out and attempts to use bus messages that have already been freed by the i2c core.
This description seems a little vague. Did you hit this case in practice?
Quoted 12 lines
Signed-off-by Eddie James <[email protected]>---
drivers/i2c/busses/i2c-aspeed.c | 1 +1 file changed, 1 insertion(+)drivers/i2c/busses/i2c-aspeed.c +0
-0
drivers/i2c/busses/i2c-aspeed.c static int aspeed_i2c_master_xfer(struct · line 735
Quoted 5 lines
drivers/i2c/busses/i2c-aspeed.c +1
-0@@ -731,6 +731,7 @@ static int aspeed_i2c_master_xfer(structi2c_adapter *adap, * master command. */ spin_lock_irqsave(&bus->lock, flags); + bus->msgs = NULL;
It feels like there's buggy code elsewhere in the driver that this is protecting (broken state machine)? I've had a look at the aspeed_i2c_master_irq() implementation and can't see what though, as we take an early-exit (before indexing into bus->msgs) if bus-
Quoted 1 line
master_stateis INACTIVE or PENDING.
Can you be a bit more specific about where the issue lies?
Andrew
Quoted 4 lines
if (
bus->master_state==ASPEED_I2C_MASTER_PENDING)bus->master_state=ASPEED_I2C_MASTER_INACTIVE;spin_unlock_irqrestore(&bus->lock, flags);
Quoted 6 lines · Andrew Jeffery
Quoted 3 lines · Eddie James
For multimaster case, it's conceivable that an interrupt comes in after the transfer times out and attempts to use bus messages that have already been freed by the i2c core.
This description seems a little vague. Did you hit this case in practice?
Yes. I no longer have the back trace but it's a null pointer access in the interrupt handler. We had a certain system that would hit this under certain conditions and this patch fixed it.
I can update the commit message with some more detail.
Quoted 21 lines
Quoted 12 lines
Signed-off-by Eddie James <[email protected]>--- drivers/i2c/busses/i2c-aspeed.c | 1 + 1 file changed, 1 insertion(+)drivers/i2c/busses/i2c-aspeed.c +0
-0drivers/i2c/busses/i2c-aspeed.c static int aspeed_i2c_master_xfer(struct · line 735
Quoted 5 lines
drivers/i2c/busses/i2c-aspeed.c +1
-0@@ -731,6 +731,7 @@ static int aspeed_i2c_master_xfer(structi2c_adapter *adap, * master command. */ spin_lock_irqsave(&bus->lock, flags); + bus->msgs = NULL;It feels like there's buggy code elsewhere in the driver that this is protecting (broken state machine)? I've had a look at the
aspeed_i2c_master_irq()implementation and can't see what though, as we take an early-exit (before indexing intobus->msgs) if bus-
Quoted 2 lines
Quoted 1 line
master_stateis INACTIVE or PENDING.Can you be a bit more specific about where the issue lies?
I'm sure the state machine isn't perfect, yea. The bad access can happen like this: if a transfer times out while waiting for an interrupt, the aspeed_i2c_master_xfer function will either reset the engine or recover the bus, and exit ETIMEDOUT. Resetting the engine will turn off interrupts, so we're safe. But recovering the bus doesn't turn off interrupts, so after the function exits ETIMEDOUT, I assume what happens is we get the interrupt for the previous transfer and try and access the messages pointer, which the i2c core has already freed.
Thanks for looking!
Eddie
Quoted 8 lines
Andrew
Quoted 4 lines
if (
bus->master_state==ASPEED_I2C_MASTER_PENDING)bus->master_state=ASPEED_I2C_MASTER_INACTIVE;
spin_unlock_irqrestore(&bus->lock, flags);
Andrew Jeffery
maintainerQuoted 10 lines · Eddie James
Quoted 6 lines · Andrew Jeffery
Quoted 3 lines · Eddie James
For multimaster case, it's conceivable that an interrupt comes in after the transfer times out and attempts to use bus messages that have already been freed by the i2c core.
This description seems a little vague. Did you hit this case in practice?
Quoted 6 lines
Yes. I no longer have the back trace but it's a null pointer access in the interrupt handler. We had a certain system that would hit this under certain conditions and this patch fixed it.
I can update the commit message with some more detail.
Thanks.
Quoted 23 lines
Quoted 23 lines
Quoted 14 lines
Signed-off-by Eddie James <[email protected]>---
drivers/i2c/busses/i2c-aspeed.c | 1 +1 file changed, 1 insertion(+)drivers/i2c/busses/i2c-aspeed.c +0
-0drivers/i2c/busses/i2c-aspeed.c static int aspeed_i2c_master_xfer(struct · line 735
Quoted 5 lines
drivers/i2c/busses/i2c-aspeed.c +1
-0@@ -731,6 +731,7 @@ static int aspeed_i2c_master_xfer(structi2c_adapter *adap, * master command. */ spin_lock_irqsave(&bus->lock, flags); + bus->msgs = NULL;It feels like there's buggy code elsewhere in the driver that this is protecting (broken state machine)? I've had a look at the
aspeed_i2c_master_irq()implementation and can't see what though, as we take an early-exit (before indexing intobus->msgs) if bus-
Quoted 6 lines
Quoted 2 lines
Quoted 1 line
master_stateis INACTIVE or PENDING.Can you be a bit more specific about where the issue lies?
I'm sure the state machine isn't perfect, yea.
Right, so I think that's what should be fixed; the explicit states define possible invariants in the implementation. We shouldn't need to test msgs to know its value (whether its value is correct should be defined by the current state).
Quoted 8 lines
The bad access can happen
like this: if a transfer times out while waiting for an interrupt, theaspeed_i2c_master_xferfunction will either reset the engine or recover the bus, and exit ETIMEDOUT. Resetting the engine will turn off interrupts, so we're safe. But recovering the bus doesn't turn off interrupts, so after the function exits ETIMEDOUT, I assume what happens is we get the interrupt for the previous transfer and try and access the messages pointer, which the i2c core has already freed.
So what immediately concerns me is there's no RECOVER state in enum aspeed_i2c_master_state or the rest of the implementation. We do have the PENDING state, which we don't have in hardware, so there's no reason for RECOVER to be missing, especially since we have the RECOVER state in hardware (I2CD14[22:19] = 0b0011).
What do you think of adding that, and testing for it in the interrupt handler?
Andrew