Skip to contentSkip to conversation

Revert "clocksource/drivers/riscv: Events are stopped during CPU suspend"

PATCH Applied Conor Dooley 3 participants 5 messages Oct 23, 2022 – Dec 2, 2022

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Revert "clocksource/drivers/riscv: Events are stopped during CPU suspend"

From: Conor Dooley <[email protected]>

This reverts commit 232ccac1bd9b5bfe73895f527c08623e7fa0752d.
If an AXI read to the PCIe controller on PolarFire SoC times out, the system will stall, with an expected:
io scheduler mq-deadline registered
io scheduler kyber registered
microchip-pcie 2000000000.pcie: host bridge /soc/pcie@2000000000 ranges:
microchip-pcie 2000000000.pcie: MEM 0x2008000000..0x2087ffffff -> 0x0008000000
microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: axi read request error microchip-pcie 2000000000.pcie: axi read timeout microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer Freeing initrd memory: 7336K
mc_event_handler: 667402 callbacks suppressed microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer
mc_event_handler: 666588 callbacks suppressed
<truncated>
microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer
mc_event_handler: 666748 callbacks suppressed
microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 0-...0: (1 GPs behind) idle=19f/1/0x4000000000000002 softirq=34/36 fqs=2626
(detected by 1, t=5256 jiffies, g=-1151, q=1143 ncpus=4)
Task dump for CPU 0:
task:swapper/0 state:R running task stack: 0 pid: 1 ppid: 0 flags:0x00000008
Call Trace:
mc_event_handler: 666648 callbacks suppressed

With this patch applied, the system just locks up without RCU stalling:
io scheduler mq-deadline registered
io scheduler kyber registered
microchip-pcie 2000000000.pcie: host bridge /soc/pcie@2000000000 ranges:
microchip-pcie 2000000000.pcie: MEM 0x2008000000..0x2087ffffff -> 0x0008000000
microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: axi read request error microchip-pcie 2000000000.pcie: axi read timeout microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer Freeing initrd memory: 7332K

Fixes 232ccac1bd9b ("clocksource/drivers/riscv: Events are stopped during CPU suspend")

---
I don't really want to post a revert, but it's been nearly a month since I posted about my issue initially & 2 weeks without a reply to Palmer's comments.
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]

---
 drivers/clocksource/timer-riscv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/clocksource/timer-riscv.c +1 -1
index 969a552da8d2..a0d66fabf073 100644
@@ -51,7 +51,7 @@ static int riscv_clock_next_event(unsigned long delta,
 static unsigned int riscv_clock_event_irq;
 static DEFINE_PER_CPU(struct clock_event_device, riscv_clock_event) = {
 	.name			= "riscv_timer_clockevent",
-	.features		= CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_C3STOP,
+	.features		= CLOCK_EVT_FEAT_ONESHOT,
 	.rating			= 100,
 	.set_next_event		= riscv_clock_next_event,
 };
Signature · 1 line
2.38.0
maintainer Reviewed-by Acked-by on v1 1/1
#1 lore raw
Quoted 96 lines · Conor Dooley

From: Conor Dooley <[email protected]>

This reverts commit 232ccac1bd9b5bfe73895f527c08623e7fa0752d.
If an AXI read to the PCIe controller on PolarFire SoC times out, the system will stall, with an expected:
io scheduler mq-deadline registered
io scheduler kyber registered
microchip-pcie 2000000000.pcie: host bridge /soc/pcie@2000000000 ranges:
microchip-pcie 2000000000.pcie: MEM 0x2008000000..0x2087ffffff -> 0x0008000000
microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: axi read request error microchip-pcie 2000000000.pcie: axi read timeout microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer Freeing initrd memory: 7336K
mc_event_handler: 667402 callbacks suppressed microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer
mc_event_handler: 666588 callbacks suppressed
<truncated>
microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer
mc_event_handler: 666748 callbacks suppressed
microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 0-...0: (1 GPs behind) idle=19f/1/0x4000000000000002 softirq=34/36 fqs=2626
(detected by 1, t=5256 jiffies, g=-1151, q=1143 ncpus=4)
Task dump for CPU 0:
task:swapper/0 state:R running task stack: 0 pid: 1 ppid: 0 flags:0x00000008
Call Trace:
mc_event_handler: 666648 callbacks suppressed

With this patch applied, the system just locks up without RCU stalling:
io scheduler mq-deadline registered
io scheduler kyber registered
microchip-pcie 2000000000.pcie: host bridge /soc/pcie@2000000000 ranges:
microchip-pcie 2000000000.pcie: MEM 0x2008000000..0x2087ffffff -> 0x0008000000
microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: axi read request error microchip-pcie 2000000000.pcie: axi read timeout microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer Freeing initrd memory: 7332K

Fixes 232ccac1bd9b ("clocksource/drivers/riscv: Events are stopped during CPU suspend")

---
I don't really want to post a revert, but it's been nearly a month since I posted about my issue initially & 2 weeks without a reply to Palmer's comments.
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]

---
 drivers/clocksource/timer-riscv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/clocksource/timer-riscv.c +0 -0
index 969a552da8d2..a0d66fabf073 100644

drivers/clocksource/timer-riscv.c static int riscv_clock_next_event(unsigned long delta, · line 57

Quoted 8 lines
drivers/clocksource/timer-riscv.c +1 -1
@@ -51,7 +51,7 @@ static int riscv_clock_next_event(unsigned long delta,
 static unsigned int riscv_clock_event_irq;
 static DEFINE_PER_CPU(struct clock_event_device, riscv_clock_event) = {
 	.name			= "riscv_timer_clockevent",
-	.features		= CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_C3STOP,
+	.features		= CLOCK_EVT_FEAT_ONESHOT,
 	.rating			= 100,
 	.set_next_event		= riscv_clock_next_event,
 };

There's some discussion on that linked patch and we don't really have a fix yet, but IMO we're better off reverting this as it breaks the common case and it's not clear this is even a sane way to fix the bug.

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#2 lore raw
In reply to Conor Dooley
Quoted 5 lines · Conor Dooley

From: Conor Dooley <[email protected]>

This reverts commit 232ccac1bd9b5bfe73895f527c08623e7fa0752d. If an AXI read to the PCIe controller on PolarFire SoC times out, the system will stall,

Applied, thanks

Signature · 11 lines
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog


_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#3 lore raw
Quoted 8 lines · Daniel Lezcano
Quoted 5 lines · Conor Dooley

From: Conor Dooley <[email protected]>

This reverts commit 232ccac1bd9b5bfe73895f527c08623e7fa0752d. If an AXI read to the PCIe controller on PolarFire SoC times out, the system will stall,

Applied, thanks

Hey Daniel,

Looks like Thomas already took the v2 of this patch:
https://lore.kernel.org/all/166989319052.4906.3934360150862233210.tip-bot2@tip-bot2/

Thanks,
Conor.

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#4 lore raw
Quoted 8 lines · Conor Dooley
Quoted 7 lines · Daniel Lezcano
Quoted 5 lines · Conor Dooley

From: Conor Dooley <[email protected]>

This reverts commit 232ccac1bd9b5bfe73895f527c08623e7fa0752d. If an AXI read to the PCIe controller on PolarFire SoC times out, the system will stall,

Quoted 6 lines
Quoted 1 line

Applied, thanks

Hey Daniel,

Looks like Thomas already took the v2 of this patch:
https://lore.kernel.org/all/166989319052.4906.3934360150862233210.tip-bot2@tip-bot2/

Ok, thanks for pointing this out. I'll drop it

Signature · 5 lines
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog