Improve CLOCK_EVT_FEAT_C3STOP feature setting
Subsystems (3)
v3 cover Improve CLOCK_EVT_FEAT_C3STOP feature setting
This series improves the RISC-V timer driver to set CLOCK_EVT_FEAT_C3STOP feature based on RISC-V platform capabilities.
These patches can also be found in riscv_timer_dt_imp_v3 branch at:
https://github.com/avpatel/linux.git
Changes since v2:
- Include Conor's revert patch as the first patch and rebased other patches
- Update PATCH2 to document bindings for separate RISC-V timer DT node
- Update PATCH3 based on RISC-V timer DT node bindings
Changes since v1:
- Rebased on Linux-5.19-rc8
- Renamed "riscv,always-on" DT property to "riscv,timer-can-wake-cpu"
Anup Patel (2):
dt-bindings: timer: Add bindings for the RISC-V timer device
clocksource: timer-riscv: Set CLOCK_EVT_FEAT_C3STOP based on DT
Conor Dooley (1):
Revert "clocksource/drivers/riscv: Events are stopped during CPU
suspend"
.../bindings/timer/riscv,timer.yaml | 52 +++++++++++++++++++ drivers/clocksource/timer-riscv.c | 12 ++++- 2 files changed, 63 insertions(+), 1 deletion(-) create mode 100644 Documentation/devicetree/bindings/timer/riscv,timer.yaml
Signature · 1 line
2.34.1
PATCH v3 1/3 Revert "clocksource/drivers/riscv: Events are stopped during CPU suspend"
From: Conor Dooley <[email protected]>
This reverts commit 232ccac1bd9b5bfe73895f527c08623e7fa0752d.
On the subject of suspend, the RISC-V SBI spec states:
Quoted 5 lines
Request the SBI implementation to put the calling hart in a platform specific suspend (or low power) state specified by the
suspend_typeparameter. The hart will automatically come out of suspended state and resume normal execution when it receives an interrupt or platform specific hardware event.
This does not cover whether any given events actually reach the hart or not, just what the hart will do if it receives an event. On PolarFire SoC, and potentially other SiFive based implementations, events from the RISC-V timer do reach a hart during suspend. This is not the case for the implementation on the Allwinner D1 - there timer events are not received during suspend.
To fix this, the C3STOP feature was enabled for the timer driver - but this has broken both RCU stall detection and timers generally on PolarFire SoC (and potentially other SiFive based implementations).
If an AXI read to the PCIe controller on PolarFire SoC times out, the system will stall, however, with this patch applied, the system just locks up without RCU stalling:
io scheduler mq-deadline registered
io scheduler kyber registered
microchip-pcie 2000000000.pcie: host bridge /soc/pcie@2000000000 ranges:
microchip-pcie 2000000000.pcie: MEM 0x2008000000..0x2087ffffff -> 0x0008000000
microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: axi read request error microchip-pcie 2000000000.pcie: axi read timeout microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer microchip-pcie 2000000000.pcie: sec error in pcie2axi buffer microchip-pcie 2000000000.pcie: ded error in pcie2axi buffer Freeing initrd memory: 7332K
Similarly issues were reported with clock_nanosleep() - with a test app that sleeps each cpu for 6, 5, 4, 3 ms respectively, HZ=250 & the blamed commit in place, the sleep times are rounded up to the next jiffy:
== CPU: 1 == == CPU: 2 == == CPU: 3 == == CPU: 4 == Mean: 7.974992 Mean: 7.976534 Mean: 7.962591 Mean: 3.952179
Std Dev: 0.154374 Std Dev: 0.156082 Std Dev: 0.171018 Std Dev: 0.076193
Hi: 9.472000 Hi: 10.495000 Hi: 8.864000 Hi: 4.736000
Lo: 6.087000 Lo: 6.380000 Lo: 4.872000 Lo: 3.403000
Samples: 521 Samples: 521 Samples: 521 Samples: 521
Fortunately, the D1 has a second timer, which is "currently used in preference to the RISC-V/SBI timer driver" so a revert here does not hurt operation of D1 in its current form.
Ultimately, a DeviceTree property (or node) will be added to encode the behaviour of the timers, but until then revert the addition of CLOCK_EVT_FEAT_C3STOP.
--- drivers/clocksource/timer-riscv.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/clocksource/timer-riscv.c +1 -1
@@ -51,7 +51,7 @@ static int riscv_clock_next_event(unsigned long delta,static unsigned int riscv_clock_event_irq; static DEFINE_PER_CPU(struct clock_event_device, riscv_clock_event) = { .name ="riscv_timer_clockevent" , - .features =CLOCK_EVT_FEAT_ONESHOT |CLOCK_EVT_FEAT_C3STOP , + .features =CLOCK_EVT_FEAT_ONESHOT , .rating =100 , .set_next_event = riscv_clock_next_event, };
Signature · 1 line
2.34.1
PATCH v3 2/3 dt-bindings: timer: Add bindings for the RISC-V timer device
In reply to Anup PatelWe add DT bindings for a separate RISC-V timer DT node which can be used to describe implementation specific behaviour (such as timer interrupt not triggered during non-retentive suspend).
--- .../bindings/timer/riscv,timer.yaml | 52 +++++++++++++++++++ 1 file changed, 52 insertions(+) create mode 100644 Documentation/devicetree/bindings/timer/riscv,timer.yaml
Documentation/devicetree/bindings/timer/riscv,timer.yaml new file +52 -0
@@ -0,0 +1,52 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/timer/riscv,timer.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: RISC-V timer + +maintainers: + - Anup Patel <[email protected]> + +description: |+ + RISC-V platforms always have a RISC-V timer device for the supervisor-mode + based on the time CSR defined by the RISC-V privileged specification. The + timer interrupts of this device are configured using the RISC-V SBI Time + extension or the RISC-V Sstc extension. + + The clock frequency of RISC-V timer device is specified via the + "timebase-frequency" DT property of "/cpus" DT node which is described + in Documentation/devicetree/bindings/riscv/cpus.yaml + +properties: + compatible: + enum: + - riscv,timer + + interrupts-extended: + minItems: 1 + maxItems: 4096 # Should be enough? + + riscv,timer-cant-wake-cpu: + type: boolean + description: + If present, the timer interrupt can't wake up the CPU from + suspend/idle state. + +additionalProperties: false + +required: + - compatible + - interrupts-extended + +examples: + - | + timer { + compatible = "riscv,timer"; + interrupts-extended = <&cpu1intc 5>, + <&cpu2intc 5>, + <&cpu3intc 5>, + <&cpu4intc 5>; + }; +...
Signature · 1 line
2.34.1
Re: [PATCH v3 2/3] dt-bindings: timer: Add bindings for the RISC-V timer device
Hey Anup,
For the future, could you please CC me on all patches in a series that I have previously reviewed?
Quoted 41 lines · Anup Patel
We add DT bindings for a separate RISC-V timer DT node which can be used to describe implementation specific behaviour (such as timer interrupt not triggered during non-retentive suspend).
Signed-off-by Anup Patel <[email protected]>--- .../bindings/timer/riscv,timer.yaml | 52 +++++++++++++++++++ 1 file changed, 52 insertions(+) create mode 100644 Documentation/devicetree/bindings/timer/riscv,timer.yamlDocumentation/devicetree/bindings/timer/riscv,timer.yaml new file +25
-0@@ -0,0 +1,52 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/timer/riscv,timer.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: RISC-V timer + +maintainers: + - Anup Patel <[email protected]> + +description: |+ + RISC-V platforms always have a RISC-V timer device for the supervisor-mode + based on the time CSR defined by the RISC-V privileged specification. The + timer interrupts of this device are configured using the RISC-V SBI Time + extension or the RISC-V Sstc extension. + + The clock frequency of RISC-V timer device is specified via the + "timebase-frequency" DT property of "/cpus" DT node which is described + in Documentation/devicetree/bindings/riscv/cpus.yaml + +properties: + compatible: + enum: + - riscv,timer
Documentation/devicetree/bindings/timer/riscv,timer.yaml line 35
Quoted 10 lines
Documentation/devicetree/bindings/timer/riscv,timer.yaml new file +10
-0+ + interrupts-extended: + minItems: 1 + maxItems: 4096 # Should be enough? + + riscv,timer-cant-wake-cpu: + type: boolean + description: + If present, the timer interrupt can't wake up the CPU from + suspend/idle state.
I'm really not sure about this... I would be inclined to think that if someone does not specify then we should assume that they took the scroogiest view of the spec and so do not get events during suspend.
I suppose you could then argue that their DT is wrong & it's their fault though. Plus the existing platforms behave this way & we avoid having to retrofit stuff here.
Quoted 20 lines
Documentation/devicetree/bindings/timer/riscv,timer.yaml new file +17
-0+ +additionalProperties: false + +required: + - compatible + - interrupts-extended + +examples: + - | + timer { + compatible = "riscv,timer"; + interrupts-extended = <&cpu1intc 5>, + <&cpu2intc 5>, + <&cpu3intc 5>, + <&cpu4intc 5>; + }; +...Signature · 1 line
2.34.1
Re: [PATCH v3 2/3] dt-bindings: timer: Add bindings for the RISC-V timer device
Quoted 5 lines · Conor Dooley
Hey Anup,
For the future, could you please CC me on all patches in a series that I have previously reviewed?
Okay.
Quoted 58 lines · Anup Patel
Quoted 41 lines
We add DT bindings for a separate RISC-V timer DT node which can be used to describe implementation specific behaviour (such as timer interrupt not triggered during non-retentive suspend).
Signed-off-by Anup Patel <[email protected]>--- .../bindings/timer/riscv,timer.yaml | 52 +++++++++++++++++++ 1 file changed, 52 insertions(+) create mode 100644 Documentation/devicetree/bindings/timer/riscv,timer.yamlDocumentation/devicetree/bindings/timer/riscv,timer.yaml new file +25
-0@@ -0,0 +1,52 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/timer/riscv,timer.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: RISC-V timer + +maintainers: + - Anup Patel <[email protected]> + +description: |+ + RISC-V platforms always have a RISC-V timer device for the supervisor-mode + based on the time CSR defined by the RISC-V privileged specification. The + timer interrupts of this device are configured using the RISC-V SBI Time + extension or the RISC-V Sstc extension. + + The clock frequency of RISC-V timer device is specified via the + "timebase-frequency" DT property of "/cpus" DT node which is described + in Documentation/devicetree/bindings/riscv/cpus.yaml + +properties: + compatible: + enum: + - riscv,timerDocumentation/devicetree/bindings/timer/riscv,timer.yaml line 35
Quoted 10 lines
Documentation/devicetree/bindings/timer/riscv,timer.yaml new file +10
-0+ + interrupts-extended: + minItems: 1 + maxItems: 4096 # Should be enough? + + riscv,timer-cant-wake-cpu: + type: boolean + description: + If present, the timer interrupt can't wake up the CPU from + suspend/idle state.I'm really not sure about this... I would be inclined to think that if someone does not specify then we should assume that they took the scroogiest view of the spec and so do not get events during suspend.
Quoted 3 lines
I suppose you could then argue that their DT is wrong & it's their fault though. Plus the existing platforms behave this way & we avoid having to retrofit stuff here.
Yes, the DT property is defined to keep things working for existing platforms.
IMO, people should always read the DT bindings document at time of creating DT for their platform. If there are queries then they can always shoot email to the maintainers on LKML.
Quoted 13 lines
Quoted 13 lines
Documentation/devicetree/bindings/timer/riscv,timer.yaml new file +12
-0+ +additionalProperties: false + +required: + - compatible + - interrupts-extended + +examples: + - | + timer { + compatible = "riscv,timer"; + interrupts-extended = <&cpu1intc 5>,
Quoted 8 lines
Quoted 8 lines
Documentation/devicetree/bindings/timer/riscv,timer.yaml new file +5
-1+ <&cpu2intc 5>, + <&cpu3intc 5>, + <&cpu4intc 5>; + }; +... --2.34.1
Regards,
Anup
Conor Dooley
Reviewed-byRe: [PATCH v3 2/3] dt-bindings: timer: Add bindings for the RISC-V timer device
Quoted 71 lines · Anup Patel
Quoted 5 lines · Conor Dooley
Hey Anup,
For the future, could you please CC me on all patches in a series that I have previously reviewed?
Okay.
Quoted 58 lines · Anup Patel
Quoted 41 lines
We add DT bindings for a separate RISC-V timer DT node which can be used to describe implementation specific behaviour (such as timer interrupt not triggered during non-retentive suspend).
Signed-off-by Anup Patel <[email protected]>--- .../bindings/timer/riscv,timer.yaml | 52 +++++++++++++++++++ 1 file changed, 52 insertions(+) create mode 100644 Documentation/devicetree/bindings/timer/riscv,timer.yamlDocumentation/devicetree/bindings/timer/riscv,timer.yaml new file +25
-0@@ -0,0 +1,52 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/timer/riscv,timer.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: RISC-V timer + +maintainers: + - Anup Patel <[email protected]> + +description: |+ + RISC-V platforms always have a RISC-V timer device for the supervisor-mode + based on the time CSR defined by the RISC-V privileged specification. The + timer interrupts of this device are configured using the RISC-V SBI Time + extension or the RISC-V Sstc extension. + + The clock frequency of RISC-V timer device is specified via the + "timebase-frequency" DT property of "/cpus" DT node which is described + in Documentation/devicetree/bindings/riscv/cpus.yaml + +properties: + compatible: + enum: + - riscv,timerDocumentation/devicetree/bindings/timer/riscv,timer.yaml line 35
Quoted 10 lines
Documentation/devicetree/bindings/timer/riscv,timer.yaml new file +10
-0+ + interrupts-extended: + minItems: 1 + maxItems: 4096 # Should be enough? + + riscv,timer-cant-wake-cpu: + type: boolean + description: + If present, the timer interrupt can't wake up the CPU from + suspend/idle state.I'm really not sure about this... I would be inclined to think that if someone does not specify then we should assume that they took the scroogiest view of the spec and so do not get events during suspend.
Quoted 3 lines
I suppose you could then argue that their DT is wrong & it's their fault though. Plus the existing platforms behave this way & we avoid having to retrofit stuff here.
Quoted 6 lines
Yes, the DT property is defined to keep things working for existing platforms.
IMO, people should always read the DT bindings document at time of creating DT for their platform. If there are queries then they can always shoot email to the maintainers on LKML.
Aye, I suppose so. For every platform that may exist that this change hurts, there's likely another one that it fixes a timer for... /shrug
Binding itself looks grand though & we are in lessor of two evils territory, so:
Thanks for turning around a v3 promptly Anup!
Conor.
Quoted 23 lines
Quoted 12 lines
Quoted 12 lines
Documentation/devicetree/bindings/timer/riscv,timer.yaml new file +12
-0+ +additionalProperties: false + +required: + - compatible + - interrupts-extended + +examples: + - | + timer { + compatible = "riscv,timer"; + interrupts-extended = <&cpu1intc 5>,Quoted 8 lines
Quoted 8 lines
Documentation/devicetree/bindings/timer/riscv,timer.yaml new file +5
-1+ <&cpu2intc 5>, + <&cpu3intc 5>, + <&cpu4intc 5>; + }; +... --2.34.1
Regards,
Anup
PATCH v3 3/3 clocksource: timer-riscv: Set CLOCK_EVT_FEAT_C3STOP based on DT
In reply to Anup PatelWe should set CLOCK_EVT_FEAT_C3STOP for a clock_event_device only when riscv,timer-cant-wake-up DT property is present in the RISC-V timer DT node.
This way CLOCK_EVT_FEAT_C3STOP feature is set for clock_event_device based on RISC-V platform capabilities rather than having it set for all RISC-V platforms.
--- drivers/clocksource/timer-riscv.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
drivers/clocksource/timer-riscv.c +10 -0
@@ -28,6 +28,7 @@#include <asm/timex.h> static DEFINE_STATIC_KEY_FALSE (riscv_sstc_available); +static bool riscv_timer_cant_wake_cpu;static int riscv_clock_next_event (unsigned long delta,struct clock_event_device *ce)
@@ -85,6 +86,8 @@ static int riscv_timer_starting_cpu(unsigned int cpu) ce->cpumask =cpumask_of (cpu); ce->irq = riscv_clock_event_irq; +if (riscv_timer_cant_wake_cpu) + ce->features |=CLOCK_EVT_FEAT_C3STOP ;clockevents_config_and_register (ce, riscv_timebase,100 ,0x7fffffff ); enable_percpu_irq(riscv_clock_event_irq,
@@ -139,6 +142,13 @@ static int __init riscv_timer_init_dt(struct device_node *n)if (cpuid !=smp_processor_id ())return 0 ; + child =of_find_compatible_node (NULL ,NULL ,"riscv,timer" ); +if (child) { + riscv_timer_cant_wake_cpu =of_property_read_bool (child, +"riscv,timer-cant-wake-cpu" ); +of_node_put (child); + } + domain =NULL ; child =of_get_compatible_child (n,"riscv,cpu-intc" );if (!child) {
Signature · 1 line
2.34.1
Re: [PATCH v3 3/3] clocksource: timer-riscv: Set CLOCK_EVT_FEAT_C3STOP based on DT
Quoted 7 lines · Anup Patel
We should set
CLOCK_EVT_FEAT_C3STOPfor aclock_event_deviceonly when riscv,timer-cant-wake-up DT property is present in the RISC-V timer DT node.This way
CLOCK_EVT_FEAT_C3STOPfeature is set forclock_event_devicebased on RISC-V platform capabilities rather than having it set for all RISC-V platforms.
I need to go do some testing on what setting the C3STOP flag does on platforms other than PolarFire SoC. I'm not sure that we should be enabling this flag *at all* until we know that it does not break on other platforms too.
Hopefully I'll get to it tonight or tomorrow..
Quoted 43 lines
Signed-off-by Anup Patel <[email protected]>--- drivers/clocksource/timer-riscv.c | 10 ++++++++++ 1 file changed, 10 insertions(+)drivers/clocksource/timer-riscv.c +10
-0@@ -28,6 +28,7 @@#include <asm/timex.h> static DEFINE_STATIC_KEY_FALSE (riscv_sstc_available); +static bool riscv_timer_cant_wake_cpu;static int riscv_clock_next_event (unsigned long delta,struct clock_event_device *ce)@@ -85,6 +86,8 @@ static int riscv_timer_starting_cpu(unsigned int cpu) ce->cpumask =cpumask_of (cpu); ce->irq = riscv_clock_event_irq; +if (riscv_timer_cant_wake_cpu) + ce->features |=CLOCK_EVT_FEAT_C3STOP ;clockevents_config_and_register (ce, riscv_timebase,100 ,0x7fffffff ); enable_percpu_irq(riscv_clock_event_irq,@@ -139,6 +142,13 @@ static int __init riscv_timer_init_dt(struct device_node *n)if (cpuid !=smp_processor_id ())return 0 ; + child =of_find_compatible_node (NULL ,NULL ,"riscv,timer" ); +if (child) { + riscv_timer_cant_wake_cpu =of_property_read_bool (child, +"riscv,timer-cant-wake-cpu" ); +of_node_put (child); + } + domain =NULL ; child =of_get_compatible_child (n,"riscv,cpu-intc" );if (!child) {Signature · 1 line
2.34.1
Conor Dooley
Reviewed-byRe: [PATCH v3 3/3] clocksource: timer-riscv: Set CLOCK_EVT_FEAT_C3STOP based on DT
Quoted 5 lines · Conor Dooley
Quoted 4 lines · Anup Patel
We should set
CLOCK_EVT_FEAT_C3STOPfor aclock_event_deviceonly when riscv,timer-cant-wake-up DT property is present in the RISC-V timer DT node.
Quoted 8 lines
Quoted 3 lines
This way
CLOCK_EVT_FEAT_C3STOPfeature is set forclock_event_devicebased on RISC-V platform capabilities rather than having it set for all RISC-V platforms.I need to go do some testing on what setting the C3STOP flag does on platforms other than PolarFire SoC. I'm not sure that we should be enabling this flag *at all* until we know that it does not break on other platforms too.
I tried my fu540 & fu740 - both of those seem to exhibit broken timer behaviour with C3STOP set. Ethernet doesn't work upstream on the VisionFive, so I didn't go through the hassle of testing that - but I would imagine it is the same as the fu740. Whenever I get a VisionFive 2 I'll give that a try too.
I did try the D1 (thanks for fielding my dumb questions Samuel) but I was not able to get the thing to boot if I disabled the sunxi timer :/ Ethernet would not come up in U-Boot, clearly I did something not right..
Obviously we need to fix things & get it backported etc, so taking a pragmatic approach: I think that it is better to merge this stuff even though it there's a pretty good chance I think that it'll break the SBI timer on a D1, since it is not intended that it will be used.
It does make me worried about some of the other platforms though, like that Bouffalolabs SoC that Jisheng sent in a DT for. It's also using thead stuff so I wonder if it needs C3STOP too. I've added Jisheng to CC :)
Quoted 30 lines
Quoted 30 lines
Signed-off-by Anup Patel <[email protected]>--- drivers/clocksource/timer-riscv.c | 10 ++++++++++ 1 file changed, 10 insertions(+)drivers/clocksource/timer-riscv.c +3
-0@@ -28,6 +28,7 @@#include <asm/timex.h> static DEFINE_STATIC_KEY_FALSE (riscv_sstc_available); +static bool riscv_timer_cant_wake_cpu;static int riscv_clock_next_event (unsigned long delta,struct clock_event_device *ce)@@ -85,6 +86,8 @@ static int riscv_timer_starting_cpu(unsigned int cpu) ce->cpumask =cpumask_of (cpu); ce->irq = riscv_clock_event_irq; +if (riscv_timer_cant_wake_cpu) + ce->features |=CLOCK_EVT_FEAT_C3STOP ;clockevents_config_and_register (ce, riscv_timebase,100 ,0x7fffffff ); enable_percpu_irq(riscv_clock_event_irq,@@ -139,6 +142,13 @@ static int __init riscv_timer_init_dt(struct device_node *n)if (cpuid !=smp_processor_id ())return 0 ;
Quoted 10 lines
drivers/clocksource/timer-riscv.c static int __init riscv_timer_init_dt(struct device_node *n) · line 154
Quoted 10 lines
drivers/clocksource/timer-riscv.c +7
-0+ child =of_find_compatible_node (NULL ,NULL ,"riscv,timer" ); +if (child) { + riscv_timer_cant_wake_cpu =of_property_read_bool (child, +"riscv,timer-cant-wake-cpu" ); +of_node_put (child); + } + domain =NULL ; child =of_get_compatible_child (n,"riscv,cpu-intc" );if (!child) {
Anyway, the mechanics of the change here look good to me. The re-use of child is understandable but a little odd though, since riscv,timer /is not/ actually a child. That's relatively minor thing to change though.
I'm still not happy about turning on C3STOP when we have not figured out why it's breaking timer behaviour, but I think that's the lessor of two evils. Somewhat reluctantly:
I'll try to spend some time looking into why it's broken.
Thanks,
Conor.
Re: [PATCH v3 3/3] clocksource: timer-riscv: Set CLOCK_EVT_FEAT_C3STOP based on DT
Hey all,
Quoted 81 lines · Conor Dooley
Quoted 5 lines · Conor Dooley
Quoted 4 lines · Anup Patel
We should set
CLOCK_EVT_FEAT_C3STOPfor aclock_event_deviceonly when riscv,timer-cant-wake-up DT property is present in the RISC-V timer DT node.Quoted 8 lines
Quoted 3 lines
This way
CLOCK_EVT_FEAT_C3STOPfeature is set forclock_event_devicebased on RISC-V platform capabilities rather than having it set for all RISC-V platforms.I need to go do some testing on what setting the C3STOP flag does on platforms other than PolarFire SoC. I'm not sure that we should be enabling this flag *at all* until we know that it does not break on other platforms too.
I tried my fu540 & fu740 - both of those seem to exhibit broken timer behaviour with C3STOP set. Ethernet doesn't work upstream on the VisionFive, so I didn't go through the hassle of testing that - but I would imagine it is the same as the fu740. Whenever I get a VisionFive 2 I'll give that a try too.
I did try the D1 (thanks for fielding my dumb questions Samuel) but I was not able to get the thing to boot if I disabled the sunxi timer :/ Ethernet would not come up in U-Boot, clearly I did something not right..
Obviously we need to fix things & get it backported etc, so taking a pragmatic approach: I think that it is better to merge this stuff even though it there's a pretty good chance I think that it'll break the SBI timer on a D1, since it is not intended that it will be used.
It does make me worried about some of the other platforms though, like that Bouffalolabs SoC that Jisheng sent in a DT for. It's also using thead stuff so I wonder if it needs C3STOP too. I've added Jisheng to CC :)
Quoted 30 lines
Quoted 30 lines
Signed-off-by Anup Patel <[email protected]>--- drivers/clocksource/timer-riscv.c | 10 ++++++++++ 1 file changed, 10 insertions(+)drivers/clocksource/timer-riscv.c +3
-0@@ -28,6 +28,7 @@#include <asm/timex.h> static DEFINE_STATIC_KEY_FALSE (riscv_sstc_available); +static bool riscv_timer_cant_wake_cpu;static int riscv_clock_next_event (unsigned long delta,struct clock_event_device *ce)@@ -85,6 +86,8 @@ static int riscv_timer_starting_cpu(unsigned int cpu) ce->cpumask =cpumask_of (cpu); ce->irq = riscv_clock_event_irq; +if (riscv_timer_cant_wake_cpu) + ce->features |=CLOCK_EVT_FEAT_C3STOP ;clockevents_config_and_register (ce, riscv_timebase,100 ,0x7fffffff ); enable_percpu_irq(riscv_clock_event_irq,@@ -139,6 +142,13 @@ static int __init riscv_timer_init_dt(struct device_node *n)if (cpuid !=smp_processor_id ())return 0 ;Quoted 10 lines
drivers/clocksource/timer-riscv.c static int __init riscv_timer_init_dt(struct device_node *n) · line 154
Quoted 10 lines
drivers/clocksource/timer-riscv.c +7
-0+ child =of_find_compatible_node (NULL ,NULL ,"riscv,timer" ); +if (child) { + riscv_timer_cant_wake_cpu =of_property_read_bool (child, +"riscv,timer-cant-wake-cpu" ); +of_node_put (child); + } + domain =NULL ; child =of_get_compatible_child (n,"riscv,cpu-intc" );if (!child) {Anyway, the mechanics of the change here look good to me. The re-use of child is understandable but a little odd though, since riscv,timer /is not/ actually a child. That's relatively minor thing to change though.
Quoted 6 lines
I'm still not happy about turning on C3STOP when we have not figured out why it's breaking timer behaviour, but I think that's the lessor of two evils. Somewhat reluctantly:
Reviewed-by Conor Dooley <[email protected]>I'll try to spend some time looking into why it's broken.
Right, so some good news! After Samuel provided me with an openSBI setup to actually test that timer & C3STOP is currently breaking the timers on the D1 too! IOW the same timer durations are rounded up to the next jiffy. He then suggested the fix for it too, see below the scissors :)
I think the revert in patch 1 is still needed (to preserve suspend functionality for existing platforms) but the commit message needs to be changed.
Perhaps, it should become:
Quoted 31 lines
From: Conor Dooley <[email protected]>
This reverts commit
232ccac1bd9b5bfe73895f527c08623e7fa0752d.On the subject of suspend, the RISC-V SBI spec states:
Quoted 5 lines
Request the SBI implementation to put the calling hart in a platform specific suspend (or low power) state specified by the
suspend_typeparameter. The hart will automatically come out of suspended state and resume normal execution when it receives an interrupt or platform specific hardware event.This does not cover whether any given events actually reach the hart or not, just what the hart will do if it receives an event. On PolarFire SoC, and potentially other SiFive based implementations, events from the RISC-V timer do reach a hart during suspend. This is not the case for the implementation on the Allwinner D1 - there timer events are not received during suspend.
To prevent a device from entering an unrecoverable sleep state, the C3STOP feature was enabled unconditionally for the RISC-V timer driver. Unfortunately, this will have disabled sleep states used by existing platforms.
Fortunately, the D1 has a second timer, which is "currently used in preference to the RISC-V/SBI timer driver" so a revert here does not hurt operation of D1 in its current form.
Ultimately, a DeviceTree property (or node) will be added to encode the behaviour of the timers, but until then revert the addition of
CLOCK_EVT_FEAT_C3STOP.
Quoted 11 lines
Fixes 232ccac1bd9b ("clocksource/drivers/riscv: Events are stopped during CPU suspend")CC Samuel Holland <[email protected]>CC Anup Patel <[email protected]>CC Palmer Dabbelt <[email protected]>Reviewed-by Palmer Dabbelt <[email protected]>Acked-by Palmer Dabbelt <[email protected]>Acked-by Samuel Holland <[email protected]>Signed-off-by Conor Dooley <[email protected]>Signed-off-by Anup Patel <[email protected]>
Anyways, I think the new order of the patchset would have the below as patch 1 & the current series on top of that. With those changes, I am happy with the series & thanks for your (plural) help in figuring all of this out!
Thanks,
Conor.
-- >8 --
From aaf20926a7645394eab4c4ad934e7f8c55e25981 Mon Sep 17 00:00:00 2001
From: Conor Dooley <[email protected]>
Date: Sat, 26 Nov 2022 14:19:44 +0000
Subject: [PATCH] RISC-V: time: initialize broadcast hrtimer based clock event
device
Similarly to commit 022eb8ae8b5e ("ARM: 8938/1: kernel: initialize broadcast hrtimer based clock event device"), RISC-V needs to initiate hrtimers before C3STOP can be used. Otherwise, the introduction of C3STOP for the RISC-V arch timer in commit 232ccac1bd9b ("clocksource/drivers/riscv: Events are stopped during CPU suspend") breaks timer behaviour, for example clock_nanosleep().
A test app that sleeps each cpu for 6, 5, 4, 3 ms respectively, HZ=250 & C3STOP enabled, the sleep times are rounded up to the next jiffy:
== CPU: 1 == == CPU: 2 == == CPU: 3 == == CPU: 4 == Mean: 7.974992 Mean: 7.976534 Mean: 7.962591 Mean: 3.952179
Std Dev: 0.154374 Std Dev: 0.156082 Std Dev: 0.171018 Std Dev: 0.076193
Hi: 9.472000 Hi: 10.495000 Hi: 8.864000 Hi: 4.736000
Lo: 6.087000 Lo: 6.380000 Lo: 4.872000 Lo: 3.403000
Samples: 521 Samples: 521 Samples: 521 Samples: 521
--- arch/riscv/kernel/time.c | 3 +++ 1 file changed, 3 insertions(+)
arch/riscv/kernel/time.c +3 -0
@@ -5,6 +5,7 @@*/ #include <linux/of_clk.h> +#include <linux/clockchips.h> #include <linux/clocksource.h> #include <linux/delay.h> #include <asm/sbi.h>
@@ -29,6 +30,8 @@ void __init time_init(void)of_clk_init (NULL );timer_probe (); + +tick_setup_hrtimer_broadcast (); }void clocksource_arch_init (struct clocksource *cs)
Signature · 1 line
2.38.1