Improve CLOCK_EVT_FEAT_C3STOP feature setting

PATCH Superseded Anup Patel 6 participants 13 messages Dec 1, 2022 – Dec 28, 2022

Subsystems (3)

Device tree bindings RISC-V Clocksource, clockevent drivers

lore

Share & tools

lore

Anup Patel

author

2022-12-01 12:40 UTC #0 lore raw

v5 cover Improve CLOCK_EVT_FEAT_C3STOP feature setting

This series improves the RISC-V timer driver to set CLOCK_EVT_FEAT_C3STOP feature based on RISC-V platform capabilities.

These patches can also be found in riscv_timer_dt_imp_v5 branch at:
https://github.com/avpatel/linux.git

Changes since v4:
- Update commit text of PATCH1 based on Samuel's comments
- Renamed DT property "riscv,timer-can-wake-cpu" to "riscv,timer-cannot-wake-cpu" in PATCH2 and PATCH3
- Updated description of DT property "riscv,timer-cannot-wake-cpu" in PATCH2

Changes since v3:
- Rebased on Linux-6.1-rc7
- Replaced PATCH1 with a patch to initialize broadcast timer

Changes since v2:
- Include Conor's revert patch as the first patch and rebased other patches
- Update PATCH2 to document bindings for separate RISC-V timer DT node
- Update PATCH3 based on RISC-V timer DT node bindings

Changes since v1:
- Rebased on Linux-5.19-rc8
- Renamed "riscv,always-on" DT property to "riscv,timer-can-wake-cpu"

Anup Patel (2):
dt-bindings: timer: Add bindings for the RISC-V timer device
clocksource: timer-riscv: Set CLOCK_EVT_FEAT_C3STOP based on DT

Conor Dooley (1):
RISC-V: time: initialize hrtimer based broadcast clock event device

 .../bindings/timer/riscv,timer.yaml           | 52 +++++++++++++++++++
 arch/riscv/kernel/time.c                      |  3 ++
 drivers/clocksource/timer-riscv.c             | 12 ++++-
 3 files changed, 66 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/devicetree/bindings/timer/riscv,timer.yaml

Signature · 1 line

2.34.1

Anup Patel

author

2022-12-01 12:40 UTC #1 lore raw

PATCH v5 1/3 RISC-V: time: initialize hrtimer based broadcast clock event device

From: Conor Dooley <[email protected]>

Similarly to commit 022eb8ae8b5e ("ARM: 8938/1: kernel: initialize broadcast hrtimer based clock event device"), RISC-V needs to initiate hrtimer based broadcast clock event device before C3STOP can be used. Otherwise, the introduction of C3STOP for the RISC-V arch timer in commit 232ccac1bd9b ("clocksource/drivers/riscv: Events are stopped during CPU suspend") leaves us without any broadcast timer registered. This prevents the kernel from entering oneshot mode, which breaks timer behaviour, for example clock_nanosleep().

A test app that sleeps each cpu for 6, 5, 4, 3 ms respectively, HZ=250 & C3STOP enabled, the sleep times are rounded up to the next jiffy:

== CPU: 1 ==      == CPU: 2 ==      == CPU: 3 ==      == CPU: 4 ==
Mean: 7.974992    Mean: 7.976534    Mean: 7.962591    Mean: 3.952179

Std Dev: 0.154374 Std Dev: 0.156082 Std Dev: 0.171018 Std Dev: 0.076193
Hi: 9.472000 Hi: 10.495000 Hi: 8.864000 Hi: 4.736000
Lo: 6.087000 Lo: 6.380000 Lo: 4.872000 Lo: 3.403000
Samples: 521 Samples: 521 Samples: 521 Samples: 521

Link https://lore.kernel.org/linux-riscv/YzYTNQRxLr7Q9JR0@spud/ open here

Fixes 232ccac1bd9b ("clocksource/drivers/riscv: Events are stopped during CPU suspend")

Signed-off-by Conor Dooley <[email protected]>

Reviewed-by Samuel Holland <[email protected]>

Acked-by Palmer Dabbelt <[email protected]>

---
 arch/riscv/kernel/time.c | 3 +++
 1 file changed, 3 insertions(+)

arch/riscv/kernel/time.c +3 -0

index 8217b0f67c6c..1cf21db4fcc7 100644
@@ -5,6 +5,7 @@
  */
 
 #include <linux/of_clk.h>
+#include <linux/clockchips.h>
 #include <linux/clocksource.h>
 #include <linux/delay.h>
 #include <asm/sbi.h>

@@ -29,6 +30,8 @@ void __init time_init(void)
 
 	of_clk_init(NULL);
 	timer_probe();
+
+	tick_setup_hrtimer_broadcast();
 }
 
 void clocksource_arch_init(struct clocksource *cs)

Signature · 1 line

2.34.1

Conor Dooley

2022-12-05 00:10 UTC #2 lore raw

Re: [PATCH v5 1/3] RISC-V: time: initialize hrtimer based broadcast clock event device

Quoted 20 lines · Anup Patel

From: Conor Dooley <[email protected]>
Similarly to commit 022eb8ae8b5e ("ARM: 8938/1: kernel: initialize broadcast hrtimer based clock event device"), RISC-V needs to initiate hrtimer based broadcast clock event device before C3STOP can be used. Otherwise, the introduction of C3STOP for the RISC-V arch timer in commit 232ccac1bd9b ("clocksource/drivers/riscv: Events are stopped during CPU suspend") leaves us without any broadcast timer registered. This prevents the kernel from entering oneshot mode, which breaks timer behaviour, for example clock_nanosleep().
A test app that sleeps each cpu for 6, 5, 4, 3 ms respectively, HZ=250 & C3STOP enabled, the sleep times are rounded up to the next jiffy:
== CPU: 1 ==      == CPU: 2 ==      == CPU: 3 ==      == CPU: 4 ==
Mean: 7.974992    Mean: 7.976534    Mean: 7.962591    Mean: 3.952179
Std Dev: 0.154374 Std Dev: 0.156082 Std Dev: 0.171018 Std Dev: 0.076193
Hi: 9.472000 Hi: 10.495000 Hi: 8.864000 Hi: 4.736000
Lo: 6.087000 Lo: 6.380000 Lo: 4.872000 Lo: 3.403000
Samples: 521 Samples: 521 Samples: 521 Samples: 521

Quoted 6 lines

Link https://lore.kernel.org/linux-riscv/YzYTNQRxLr7Q9JR0@spud/ open here
Fixes 232ccac1bd9b ("clocksource/drivers/riscv: Events are stopped during CPU suspend")
Suggested-by Samuel Holland <[email protected]>
Signed-off-by Conor Dooley <[email protected]>
Reviewed-by Samuel Holland <[email protected]>
Acked-by Palmer Dabbelt <[email protected]>

Huh, thought I replied already but I just have forgotten to... Since you've added this patch to your series, it needs your SoB appended.

Thanks,
Conor.

Quoted 25 lines

---
arch/riscv/kernel/time.c | 3 +++

1 file changed, 3 insertions(+)

arch/riscv/kernel/time.c +3 -0

index 8217b0f67c6c..1cf21db4fcc7 100644
@@ -5,6 +5,7 @@
 */
 
 #include <linux/of_clk.h>
+#include <linux/clockchips.h>
 #include <linux/clocksource.h>
 #include <linux/delay.h>
 #include <asm/sbi.h>

@@ -29,6 +30,8 @@ void __init time_init(void)
 
 	of_clk_init(NULL);
 	timer_probe();
+
+	tick_setup_hrtimer_broadcast();
 }
 
 void clocksource_arch_init(struct clocksource *cs)

Anup Patel

author

2022-12-01 12:40 UTC #3 lore raw

PATCH v5 2/3 dt-bindings: timer: Add bindings for the RISC-V timer device

In reply to Anup Patel

We add DT bindings for a separate RISC-V timer DT node which can be used to describe implementation specific behaviour (such as timer interrupt not triggered during non-retentive suspend).

Signed-off-by Anup Patel <[email protected]>

Reviewed-by Conor Dooley <[email protected]>

Acked-by Palmer Dabbelt <[email protected]>

---
 .../bindings/timer/riscv,timer.yaml           | 52 +++++++++++++++++++
 1 file changed, 52 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/timer/riscv,timer.yaml

Documentation/devicetree/bindings/timer/riscv,timer.yaml new file +52 -0

new file mode 100644
index 000000000000..38d67e1a5a79
@@ -0,0 +1,52 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/timer/riscv,timer.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: RISC-V timer
+
+maintainers:
+  - Anup Patel <[email protected]>
+
+description: |+
+  RISC-V platforms always have a RISC-V timer device for the supervisor-mode
+  based on the time CSR defined by the RISC-V privileged specification. The
+  timer interrupts of this device are configured using the RISC-V SBI Time
+  extension or the RISC-V Sstc extension.
+
+  The clock frequency of RISC-V timer device is specified via the
+  "timebase-frequency" DT property of "/cpus" DT node which is described
+  in Documentation/devicetree/bindings/riscv/cpus.yaml
+
+properties:
+  compatible:
+    enum:
+      - riscv,timer
+
+  interrupts-extended:
+    minItems: 1
+    maxItems: 4096   # Should be enough?
+
+  riscv,timer-cannot-wake-cpu:
+    type: boolean
+    description:
+      If present, the timer interrupt cannot wake up the CPU from one or
+      more suspend/idle states.
+
+additionalProperties: false
+
+required:
+  - compatible
+  - interrupts-extended
+
+examples:
+  - |
+    timer {
+      compatible = "riscv,timer";
+      interrupts-extended = <&cpu1intc 5>,
+                            <&cpu2intc 5>,
+                            <&cpu3intc 5>,
+                            <&cpu4intc 5>;
+    };
+...

Signature · 1 line

2.34.1

Rob Herring

maintainer Reviewed-by on v5 2/3

2022-12-05 16:17 UTC #4 lore raw

Re: [PATCH v5 2/3] dt-bindings: timer: Add bindings for the RISC-V timer device

Quoted 4 lines · Anup Patel

We add DT bindings for a separate RISC-V timer DT node which can be used to describe implementation specific behaviour (such as timer interrupt not triggered during non-retentive suspend).

Quoted 8 lines

Signed-off-by Anup Patel <[email protected]>
Reviewed-by Conor Dooley <[email protected]>
Acked-by Palmer Dabbelt <[email protected]>
---
 .../bindings/timer/riscv,timer.yaml           | 52 +++++++++++++++++++
 1 file changed, 52 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/timer/riscv,timer.yaml

Reviewed-by Rob Herring <[email protected]>

Anup Patel

author

2022-12-01 12:40 UTC #5 lore raw

PATCH v5 3/3 clocksource: timer-riscv: Set CLOCK_EVT_FEAT_C3STOP based on DT

In reply to Anup Patel

We should set CLOCK_EVT_FEAT_C3STOP for a clock_event_device only when riscv,timer-cannot-wake-cpu DT property is present in the RISC-V timer DT node.

This way CLOCK_EVT_FEAT_C3STOP feature is set for clock_event_device based on RISC-V platform capabilities rather than having it set for all RISC-V platforms.

Signed-off-by Anup Patel <[email protected]>

Reviewed-by Conor Dooley <[email protected]>

Acked-by Palmer Dabbelt <[email protected]>

---
 drivers/clocksource/timer-riscv.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

drivers/clocksource/timer-riscv.c +11 -1

index 969a552da8d2..1b4b36df5484 100644
@@ -28,6 +28,7 @@
 #include <asm/timex.h>
 
 static DEFINE_STATIC_KEY_FALSE(riscv_sstc_available);
+static bool riscv_timer_cannot_wake_cpu;
 
 static int riscv_clock_next_event(unsigned long delta,
 		struct clock_event_device *ce)

@@ -51,7 +52,7 @@ static int riscv_clock_next_event(unsigned long delta,
 static unsigned int riscv_clock_event_irq;
 static DEFINE_PER_CPU(struct clock_event_device, riscv_clock_event) = {
 	.name			= "riscv_timer_clockevent",
-	.features		= CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_C3STOP,
+	.features		= CLOCK_EVT_FEAT_ONESHOT,
 	.rating			= 100,
 	.set_next_event		= riscv_clock_next_event,
 };

@@ -85,6 +86,8 @@ static int riscv_timer_starting_cpu(unsigned int cpu)
 
 	ce->cpumask = cpumask_of(cpu);
 	ce->irq = riscv_clock_event_irq;
+	if (riscv_timer_cannot_wake_cpu)
+		ce->features |= CLOCK_EVT_FEAT_C3STOP;
 	clockevents_config_and_register(ce, riscv_timebase, 100, 0x7fffffff);
 
 	enable_percpu_irq(riscv_clock_event_irq,

@@ -139,6 +142,13 @@ static int __init riscv_timer_init_dt(struct device_node *n)
 	if (cpuid != smp_processor_id())
 		return 0;
 
+	child = of_find_compatible_node(NULL, NULL, "riscv,timer");
+	if (child) {
+		riscv_timer_cannot_wake_cpu = of_property_read_bool(child,
+					"riscv,timer-cannot-wake-cpu");
+		of_node_put(child);
+	}
+
 	domain = NULL;
 	child = of_get_compatible_child(n, "riscv,cpu-intc");
 	if (!child) {

Signature · 1 line

2.34.1

Rob Herring

maintainer

2022-12-02 00:06 UTC #6 lore raw

Re: [PATCH v5 3/3] clocksource: timer-riscv: Set CLOCK_EVT_FEAT_C3STOP based on DT

Quoted 28 lines · Anup Patel

We should set CLOCK_EVT_FEAT_C3STOP for a clock_event_device only when riscv,timer-cannot-wake-cpu DT property is present in the RISC-V timer DT node.
This way CLOCK_EVT_FEAT_C3STOP feature is set for clock_event_device based on RISC-V platform capabilities rather than having it set for all RISC-V platforms.
Signed-off-by Anup Patel <[email protected]>
Reviewed-by Conor Dooley <[email protected]>
Acked-by Palmer Dabbelt <[email protected]>
---
 drivers/clocksource/timer-riscv.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
drivers/clocksource/timer-riscv.c +1 -0
index 969a552da8d2..1b4b36df5484 100644
@@ -28,6 +28,7 @@
 #include <asm/timex.h>
 
 static DEFINE_STATIC_KEY_FALSE(riscv_sstc_available);
+static bool riscv_timer_cannot_wake_cpu;
 
 static int riscv_clock_next_event(unsigned long delta,
 		struct clock_event_device *ce)

drivers/clocksource/timer-riscv.c static int riscv_clock_next_event(unsigned long delta, · line 55

Quoted 5 lines

drivers/clocksource/timer-riscv.c +1 -1

@@ -51,7 +52,7 @@ static int riscv_clock_next_event(unsigned long delta,
 static unsigned int riscv_clock_event_irq;
 static DEFINE_PER_CPU(struct clock_event_device, riscv_clock_event) = {
 	.name			= "riscv_timer_clockevent",
-	.features		= CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_C3STOP,
+	.features		= CLOCK_EVT_FEAT_ONESHOT,

A platform that depended on CLOCK_EVT_FEAT_C3STOP being set will break with this change because its existing DT will not have the new property.

It needs to be the other way around which would effectively be the existing 'always-on' property.

Quoted 30 lines

drivers/clocksource/timer-riscv.c +9 -0

 	.rating			= 100,
 	.set_next_event		= riscv_clock_next_event,
 };

@@ -85,6 +86,8 @@ static int riscv_timer_starting_cpu(unsigned int cpu)
 
 	ce->cpumask = cpumask_of(cpu);
 	ce->irq = riscv_clock_event_irq;
+	if (riscv_timer_cannot_wake_cpu)
+		ce->features |= CLOCK_EVT_FEAT_C3STOP;
 	clockevents_config_and_register(ce, riscv_timebase, 100, 0x7fffffff);
 
 	enable_percpu_irq(riscv_clock_event_irq,

@@ -139,6 +142,13 @@ static int __init riscv_timer_init_dt(struct device_node *n)
 	if (cpuid != smp_processor_id())
 		return 0;
 
+	child = of_find_compatible_node(NULL, NULL, "riscv,timer");
+	if (child) {
+		riscv_timer_cannot_wake_cpu = of_property_read_bool(child,
+					"riscv,timer-cannot-wake-cpu");
+		of_node_put(child);
+	}
+
 	domain = NULL;
 	child = of_get_compatible_child(n, "riscv,cpu-intc");
 	if (!child) {

Signature · 1 line

2.34.1

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

Anup Patel

author

2022-12-02 06:33 UTC #7 lore raw

Re: [PATCH v5 3/3] clocksource: timer-riscv: Set CLOCK_EVT_FEAT_C3STOP based on DT

Quoted 39 lines · Rob Herring

Quoted 28 lines · Anup Patel
We should set CLOCK_EVT_FEAT_C3STOP for a clock_event_device only when riscv,timer-cannot-wake-cpu DT property is present in the RISC-V timer DT node.
This way CLOCK_EVT_FEAT_C3STOP feature is set for clock_event_device based on RISC-V platform capabilities rather than having it set for all RISC-V platforms.
Signed-off-by Anup Patel <[email protected]>
Reviewed-by Conor Dooley <[email protected]>
Acked-by Palmer Dabbelt <[email protected]>
---
 drivers/clocksource/timer-riscv.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
drivers/clocksource/timer-riscv.c +1 -0
index 969a552da8d2..1b4b36df5484 100644
@@ -28,6 +28,7 @@
 #include <asm/timex.h>
 
 static DEFINE_STATIC_KEY_FALSE(riscv_sstc_available);
+static bool riscv_timer_cannot_wake_cpu;
 
 static int riscv_clock_next_event(unsigned long delta,
              struct clock_event_device *ce)
drivers/clocksource/timer-riscv.c static int riscv_clock_next_event(unsigned long delta, · line 55
Quoted 5 lines
drivers/clocksource/timer-riscv.c +1 -1
@@ -51,7 +52,7 @@ static int riscv_clock_next_event(unsigned long delta,
 static unsigned int riscv_clock_event_irq;
 static DEFINE_PER_CPU(struct clock_event_device, riscv_clock_event) = {
      .name                   = "riscv_timer_clockevent",
-     .features               = CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_C3STOP,
+     .features               = CLOCK_EVT_FEAT_ONESHOT,
A platform that depended on CLOCK_EVT_FEAT_C3STOP being set will break with this change because its existing DT will not have the new property.

Quoted 2 lines

It needs to be the other way around which would effectively be the existing 'always-on' property.

There are no RISC-V platforms using C3STOP. The patch which
added C3STOP has been reverted.
(Refer, https://lore.kernel.org/lkml/[email protected]/T/)

I just need to rebase this patch upon the C3STOP revert patch.

Quoted 17 lines

drivers/clocksource/timer-riscv.c +2 -0

      .rating                 = 100,
      .set_next_event         = riscv_clock_next_event,
 };

@@ -85,6 +86,8 @@ static int riscv_timer_starting_cpu(unsigned int cpu)
 
      ce->cpumask = cpumask_of(cpu);
      ce->irq = riscv_clock_event_irq;
+     if (riscv_timer_cannot_wake_cpu)
+             ce->features |= CLOCK_EVT_FEAT_C3STOP;
      clockevents_config_and_register(ce, riscv_timebase, 100, 0x7fffffff);
 
      enable_percpu_irq(riscv_clock_event_irq,

@@ -139,6 +142,13 @@ static int __init riscv_timer_init_dt(struct device_node *n)
      if (cpuid != smp_processor_id())
              return 0;

Quoted 14 lines

Quoted 11 lines

drivers/clocksource/timer-riscv.c +7 -1

+     child = of_find_compatible_node(NULL, NULL, "riscv,timer");
+     if (child) {
+             riscv_timer_cannot_wake_cpu = of_property_read_bool(child,
+                                     "riscv,timer-cannot-wake-cpu");
+             of_node_put(child);
+     }
+
      domain = NULL;
      child = of_get_compatible_child(n, "riscv,cpu-intc");
      if (!child) {
--

Quoted 3 lines

2.34.1

Regards,
Anup

Conor Dooley

2022-12-04 23:34 UTC #8 lore raw

Re: [PATCH v5 3/3] clocksource: timer-riscv: Set CLOCK_EVT_FEAT_C3STOP based on DT

Hey Rob, Anup, Prabhakar,

Quoted 40 lines · Anup Patel

Quoted 39 lines · Rob Herring
Quoted 28 lines · Anup Patel
We should set CLOCK_EVT_FEAT_C3STOP for a clock_event_device only when riscv,timer-cannot-wake-cpu DT property is present in the RISC-V timer DT node.
This way CLOCK_EVT_FEAT_C3STOP feature is set for clock_event_device based on RISC-V platform capabilities rather than having it set for all RISC-V platforms.
Signed-off-by Anup Patel <[email protected]>
Reviewed-by Conor Dooley <[email protected]>
Acked-by Palmer Dabbelt <[email protected]>
---
 drivers/clocksource/timer-riscv.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
drivers/clocksource/timer-riscv.c +1 -0
index 969a552da8d2..1b4b36df5484 100644
@@ -28,6 +28,7 @@
 #include <asm/timex.h>
 
 static DEFINE_STATIC_KEY_FALSE(riscv_sstc_available);
+static bool riscv_timer_cannot_wake_cpu;
 
 static int riscv_clock_next_event(unsigned long delta,
              struct clock_event_device *ce)
drivers/clocksource/timer-riscv.c static int riscv_clock_next_event(unsigned long delta, · line 55
Quoted 5 lines
drivers/clocksource/timer-riscv.c +1 -1
@@ -51,7 +52,7 @@ static int riscv_clock_next_event(unsigned long delta,
 static unsigned int riscv_clock_event_irq;
 static DEFINE_PER_CPU(struct clock_event_device, riscv_clock_event) = {
      .name                   = "riscv_timer_clockevent",
-     .features               = CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_C3STOP,
+     .features               = CLOCK_EVT_FEAT_ONESHOT,
A platform that depended on CLOCK_EVT_FEAT_C3STOP being set will break with this change because its existing DT will not have the new property.

Quoted 8 lines

Quoted 2 lines
It needs to be the other way around which would effectively be the existing 'always-on' property.
There are no RISC-V platforms using C3STOP. The patch which
added C3STOP has been reverted.
(Refer, https://lore.kernel.org/lkml/[email protected]/T/)
I just need to rebase this patch upon the C3STOP revert patch.

I guess you could say that the C3STOP addition was done spec-ulatively*, as the platform that actually exhibits that behaviour does not use the riscv-timer & its maintainer acked the revert (allwinner d1 family).

*The spec does not make any guarantees about whether events arrive during suspend, only the behaviour *if* they arrive.

Switching the property to "always-on" would require retrofitting that property to every other existing platform (and therefore regressing some behaviour there, no?).

Most of the existing platforms are "toys" or demo platforms though, so it would not, I guess, be the end of the world to do so. Doubly so since none of them actually implement any sleep states that making it an "always-on" property.

I've said since the start that defaulting to C3STOP is the "safer" thing to do, and although we disagreed on this last time Anup, I think the better outcome of someone missing a DT property is inaccessible sleep states rather than going into sleep states they cannot get out of.

For PolarFire SoC, which I guess is one of the few "commerical" platforms, I'd be willing to accept retrofitting, since we have not yet implemented such sleep states yet.

Maybe Prabhakar knows whether the RZ/Five has either a) implemented sleep states and b) which side of the "timer events arrive in suspend" divide their platform lies on.
I'm particular interested here since that is not a SiFive core complex.

I would like to get DT maintainer approval of an approach here soon-ish so that we can something sorted for the jh7110 stuff and for the bouffalolabs SoC - the latter of which may very well be in the "no events in suspend" camp as it also uses thead stuff.

Sorry for kinda rowing back on my previous acceptance of the approach, but I am really between two minds on this.

Thanks,
Conor.

Samuel Holland

2022-12-05 08:17 UTC #9 lore raw

Re: [PATCH v5 3/3] clocksource: timer-riscv: Set CLOCK_EVT_FEAT_C3STOP based on DT

Quoted 50 lines · Conor Dooley

Hey Rob, Anup, Prabhakar,
Quoted 47 lines · Anup Patel
Quoted 39 lines · Rob Herring
Quoted 28 lines · Anup Patel
We should set CLOCK_EVT_FEAT_C3STOP for a clock_event_device only when riscv,timer-cannot-wake-cpu DT property is present in the RISC-V timer DT node.
This way CLOCK_EVT_FEAT_C3STOP feature is set for clock_event_device based on RISC-V platform capabilities rather than having it set for all RISC-V platforms.
Signed-off-by Anup Patel <[email protected]>
Reviewed-by Conor Dooley <[email protected]>
Acked-by Palmer Dabbelt <[email protected]>
---
 drivers/clocksource/timer-riscv.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
drivers/clocksource/timer-riscv.c +1 -0
index 969a552da8d2..1b4b36df5484 100644
@@ -28,6 +28,7 @@
 #include <asm/timex.h>
 
 static DEFINE_STATIC_KEY_FALSE(riscv_sstc_available);
+static bool riscv_timer_cannot_wake_cpu;
 
 static int riscv_clock_next_event(unsigned long delta,
              struct clock_event_device *ce)
drivers/clocksource/timer-riscv.c static int riscv_clock_next_event(unsigned long delta, · line 55
Quoted 5 lines
drivers/clocksource/timer-riscv.c +1 -1
@@ -51,7 +52,7 @@ static int riscv_clock_next_event(unsigned long delta,
 static unsigned int riscv_clock_event_irq;
 static DEFINE_PER_CPU(struct clock_event_device, riscv_clock_event) = {
      .name                   = "riscv_timer_clockevent",
-     .features               = CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_C3STOP,
+     .features               = CLOCK_EVT_FEAT_ONESHOT,
A platform that depended on CLOCK_EVT_FEAT_C3STOP being set will break with this change because its existing DT will not have the new property.
Quoted 2 lines
It needs to be the other way around which would effectively be the existing 'always-on' property.
There are no RISC-V platforms using C3STOP. The patch which
added C3STOP has been reverted.
(Refer, https://lore.kernel.org/lkml/[email protected]/T/)

Quoted 5 lines

Quoted 1 line
I just need to rebase this patch upon the C3STOP revert patch.
I guess you could say that the C3STOP addition was done spec-ulatively*, as the platform that actually exhibits that behaviour does not use the riscv-timer & its maintainer acked the revert (allwinner d1 family).

For clarity: that doesn't mean the platform will _never_ use the SBI timer facility, just that Linux happens to not use it right now.

Quoted 11 lines

*The spec does not make any guarantees about whether events arrive during suspend, only the behaviour *if* they arrive.
Switching the property to "always-on" would require retrofitting that property to every other existing platform (and therefore regressing some behaviour there, no?).
Most of the existing platforms are "toys" or demo platforms though, so it would not, I guess, be the end of the world to do so. Doubly so since none of them actually implement any sleep states that making it an "always-on" property.

Specifically, only sleep states with a "local-timer-stop" property would be inhibited by the C3STOP flag, so there is only possibility of a regression if some DT declaring such a sleep state exists anywhere.

Regards,
Samuel

Quoted 25 lines

I've said since the start that defaulting to C3STOP is the "safer" thing to do, and although we disagreed on this last time Anup, I think the better outcome of someone missing a DT property is inaccessible sleep states rather than going into sleep states they cannot get out of.
For PolarFire SoC, which I guess is one of the few "commerical" platforms, I'd be willing to accept retrofitting, since we have not yet implemented such sleep states yet.
Maybe Prabhakar knows whether the RZ/Five has either a) implemented sleep states and b) which side of the "timer events arrive in suspend" divide their platform lies on.
I'm particular interested here since that is not a SiFive core complex.
I would like to get DT maintainer approval of an approach here soon-ish so that we can something sorted for the jh7110 stuff and for the bouffalolabs SoC - the latter of which may very well be in the "no events in suspend" camp as it also uses thead stuff.
Sorry for kinda rowing back on my previous acceptance of the approach, but I am really between two minds on this.
Thanks,
Conor.

Conor Dooley

2022-12-05 08:34 UTC #10 lore raw

Re: [PATCH v5 3/3] clocksource: timer-riscv: Set CLOCK_EVT_FEAT_C3STOP based on DT

Quoted 51 lines · Samuel Holland

Quoted 50 lines · Conor Dooley
Hey Rob, Anup, Prabhakar,
Quoted 47 lines · Anup Patel
Quoted 39 lines · Rob Herring
Quoted 28 lines · Anup Patel
We should set CLOCK_EVT_FEAT_C3STOP for a clock_event_device only when riscv,timer-cannot-wake-cpu DT property is present in the RISC-V timer DT node.
This way CLOCK_EVT_FEAT_C3STOP feature is set for clock_event_device based on RISC-V platform capabilities rather than having it set for all RISC-V platforms.
Signed-off-by Anup Patel <[email protected]>
Reviewed-by Conor Dooley <[email protected]>
Acked-by Palmer Dabbelt <[email protected]>
---
 drivers/clocksource/timer-riscv.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
drivers/clocksource/timer-riscv.c +1 -0
index 969a552da8d2..1b4b36df5484 100644
@@ -28,6 +28,7 @@
 #include <asm/timex.h>
 
 static DEFINE_STATIC_KEY_FALSE(riscv_sstc_available);
+static bool riscv_timer_cannot_wake_cpu;
 
 static int riscv_clock_next_event(unsigned long delta,
              struct clock_event_device *ce)
drivers/clocksource/timer-riscv.c static int riscv_clock_next_event(unsigned long delta, · line 55
Quoted 5 lines
drivers/clocksource/timer-riscv.c +1 -1
@@ -51,7 +52,7 @@ static int riscv_clock_next_event(unsigned long delta,
 static unsigned int riscv_clock_event_irq;
 static DEFINE_PER_CPU(struct clock_event_device, riscv_clock_event) = {
      .name                   = "riscv_timer_clockevent",
-     .features               = CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_C3STOP,
+     .features               = CLOCK_EVT_FEAT_ONESHOT,
A platform that depended on CLOCK_EVT_FEAT_C3STOP being set will break with this change because its existing DT will not have the new property.
Quoted 2 lines
It needs to be the other way around which would effectively be the existing 'always-on' property.
There are no RISC-V platforms using C3STOP. The patch which
added C3STOP has been reverted.
(Refer, https://lore.kernel.org/lkml/[email protected]/T/)

Quoted 8 lines

Quoted 5 lines
Quoted 1 line
I just need to rebase this patch upon the C3STOP revert patch.
I guess you could say that the C3STOP addition was done spec-ulatively*, as the platform that actually exhibits that behaviour does not use the riscv-timer & its maintainer acked the revert (allwinner d1 family).
For clarity: that doesn't mean the platform will _never_ use the SBI timer facility, just that Linux happens to not use it right now.

Yeah sorry - should have been a bit clearer there. There's a few other SoCs about that are using the thead cores, so I'd be "worried" that they share the timer behaviour but do not have an alternative like you do on the D1. That's part of what's kinda given me cold feet on the current approach.

Quoted 50 lines

Quoted 11 lines
*The spec does not make any guarantees about whether events arrive during suspend, only the behaviour *if* they arrive.
Switching the property to "always-on" would require retrofitting that property to every other existing platform (and therefore regressing some behaviour there, no?).
Most of the existing platforms are "toys" or demo platforms though, so it would not, I guess, be the end of the world to do so. Doubly so since none of them actually implement any sleep states that making it an "always-on" property.
Specifically, only sleep states with a "local-timer-stop" property would be inhibited by the C3STOP flag, so there is only possibility of a regression if some DT declaring such a sleep state exists anywhere.
Regards,
Samuel
Quoted 19 lines
I've said since the start that defaulting to C3STOP is the "safer" thing to do, and although we disagreed on this last time Anup, I think the better outcome of someone missing a DT property is inaccessible sleep states rather than going into sleep states they cannot get out of.
For PolarFire SoC, which I guess is one of the few "commerical" platforms, I'd be willing to accept retrofitting, since we have not yet implemented such sleep states yet.
Maybe Prabhakar knows whether the RZ/Five has either a) implemented sleep states and b) which side of the "timer events arrive in suspend" divide their platform lies on.
I'm particular interested here since that is not a SiFive core complex.
I would like to get DT maintainer approval of an approach here soon-ish so that we can something sorted for the jh7110 stuff and for the bouffalolabs SoC - the latter of which may very well be in the "no events in suspend" camp as it also uses thead stuff.
Quoted 6 lines
Sorry for kinda rowing back on my previous acceptance of the approach, but I am really between two minds on this.
Thanks,
Conor.
_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

Conor Dooley

2022-12-28 12:00 UTC #11 lore raw

Re: [PATCH v5 3/3] clocksource: timer-riscv: Set CLOCK_EVT_FEAT_C3STOP based on DT

Hey Anup (& Daniel++),

Quoted 58 lines · Conor Dooley

Quoted 57 lines · Samuel Holland
Quoted 55 lines · Conor Dooley
Hey Rob, Anup, Prabhakar,
Quoted 40 lines · Anup Patel
Quoted 39 lines · Rob Herring
Quoted 28 lines · Anup Patel
We should set CLOCK_EVT_FEAT_C3STOP for a clock_event_device only when riscv,timer-cannot-wake-cpu DT property is present in the RISC-V timer DT node.
This way CLOCK_EVT_FEAT_C3STOP feature is set for clock_event_device based on RISC-V platform capabilities rather than having it set for all RISC-V platforms.
Signed-off-by Anup Patel <[email protected]>
Reviewed-by Conor Dooley <[email protected]>
Acked-by Palmer Dabbelt <[email protected]>
---
 drivers/clocksource/timer-riscv.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
drivers/clocksource/timer-riscv.c +1 -0
index 969a552da8d2..1b4b36df5484 100644
@@ -28,6 +28,7 @@
 #include <asm/timex.h>
 
 static DEFINE_STATIC_KEY_FALSE(riscv_sstc_available);
+static bool riscv_timer_cannot_wake_cpu;
 
 static int riscv_clock_next_event(unsigned long delta,
              struct clock_event_device *ce)
drivers/clocksource/timer-riscv.c static int riscv_clock_next_event(unsigned long delta, · line 55
Quoted 5 lines
drivers/clocksource/timer-riscv.c +1 -1
@@ -51,7 +52,7 @@ static int riscv_clock_next_event(unsigned long delta,
 static unsigned int riscv_clock_event_irq;
 static DEFINE_PER_CPU(struct clock_event_device, riscv_clock_event) = {
      .name                   = "riscv_timer_clockevent",
-     .features               = CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_C3STOP,
+     .features               = CLOCK_EVT_FEAT_ONESHOT,
A platform that depended on CLOCK_EVT_FEAT_C3STOP being set will break with this change because its existing DT will not have the new property.
Quoted 8 lines
Quoted 2 lines
It needs to be the other way around which would effectively be the existing 'always-on' property.
There are no RISC-V platforms using C3STOP. The patch which
added C3STOP has been reverted.
(Refer, https://lore.kernel.org/lkml/[email protected]/T/)
I just need to rebase this patch upon the C3STOP revert patch.
I guess you could say that the C3STOP addition was done spec-ulatively*, as the platform that actually exhibits that behaviour does not use the riscv-timer & its maintainer acked the revert (allwinner d1 family).

Quoted 24 lines

Quoted 2 lines
For clarity: that doesn't mean the platform will _never_ use the SBI timer facility, just that Linux happens to not use it right now.
Yeah sorry - should have been a bit clearer there. There's a few other SoCs about that are using the thead cores, so I'd be "worried" that they share the timer behaviour but do not have an alternative like you do on the D1. That's part of what's kinda given me cold feet on the current approach.
Quoted 15 lines
Quoted 11 lines
*The spec does not make any guarantees about whether events arrive during suspend, only the behaviour *if* they arrive.
Switching the property to "always-on" would require retrofitting that property to every other existing platform (and therefore regressing some behaviour there, no?).
Most of the existing platforms are "toys" or demo platforms though, so it would not, I guess, be the end of the world to do so. Doubly so since none of them actually implement any sleep states that making it an "always-on" property.
Specifically, only sleep states with a "local-timer-stop" property would be inhibited by the C3STOP flag, so there is only possibility of a regression if some DT declaring such a sleep state exists anywhere.

What is the plan for this series? IIRC, a rebase was required on top of the revert?
I'm not overly pushed either way at this point, I'd like to get this C3STOP issue sorted out before more thead powered SoCs start showing up with the same implementation issues.

I know it is the Christmas period so not expecting anything to actually happen right away.

Thanks,
Conor.

Quoted 14 lines

Quoted 14 lines
Quoted 14 lines
I've said since the start that defaulting to C3STOP is the "safer" thing to do, and although we disagreed on this last time Anup, I think the better outcome of someone missing a DT property is inaccessible sleep states rather than going into sleep states they cannot get out of.
For PolarFire SoC, which I guess is one of the few "commerical" platforms, I'd be willing to accept retrofitting, since we have not yet implemented such sleep states yet.
Maybe Prabhakar knows whether the RZ/Five has either a) implemented sleep states and b) which side of the "timer events arrive in suspend" divide their platform lies on.
I'm particular interested here since that is not a SiFive core complex.

Quoted 7 lines

Quoted 7 lines
Quoted 7 lines
I would like to get DT maintainer approval of an approach here soon-ish so that we can something sorted for the jh7110 stuff and for the bouffalolabs SoC - the latter of which may very well be in the "no events in suspend" camp as it also uses thead stuff.
Sorry for kinda rowing back on my previous acceptance of the approach, but I am really between two minds on this.

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

Lad, Prabhakar

2022-12-06 22:03 UTC #12 lore raw

Re: [PATCH v5 3/3] clocksource: timer-riscv: Set CLOCK_EVT_FEAT_C3STOP based on DT

In reply to Conor Dooley

Hi Conor,

Quoted 74 lines · Conor Dooley

Hey Rob, Anup, Prabhakar,
Quoted 40 lines · Anup Patel
Quoted 39 lines · Rob Herring
Quoted 28 lines · Anup Patel
We should set CLOCK_EVT_FEAT_C3STOP for a clock_event_device only when riscv,timer-cannot-wake-cpu DT property is present in the RISC-V timer DT node.
This way CLOCK_EVT_FEAT_C3STOP feature is set for clock_event_device based on RISC-V platform capabilities rather than having it set for all RISC-V platforms.
Signed-off-by Anup Patel <[email protected]>
Reviewed-by Conor Dooley <[email protected]>
Acked-by Palmer Dabbelt <[email protected]>
---
 drivers/clocksource/timer-riscv.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
drivers/clocksource/timer-riscv.c +1 -0
index 969a552da8d2..1b4b36df5484 100644
@@ -28,6 +28,7 @@
 #include <asm/timex.h>
 
 static DEFINE_STATIC_KEY_FALSE(riscv_sstc_available);
+static bool riscv_timer_cannot_wake_cpu;
 
 static int riscv_clock_next_event(unsigned long delta,
              struct clock_event_device *ce)
drivers/clocksource/timer-riscv.c static int riscv_clock_next_event(unsigned long delta, · line 55
Quoted 5 lines
drivers/clocksource/timer-riscv.c +1 -1
@@ -51,7 +52,7 @@ static int riscv_clock_next_event(unsigned long delta,
 static unsigned int riscv_clock_event_irq;
 static DEFINE_PER_CPU(struct clock_event_device, riscv_clock_event) = {
      .name                   = "riscv_timer_clockevent",
-     .features               = CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_C3STOP,
+     .features               = CLOCK_EVT_FEAT_ONESHOT,
A platform that depended on CLOCK_EVT_FEAT_C3STOP being set will break with this change because its existing DT will not have the new property.
Quoted 8 lines
Quoted 2 lines
It needs to be the other way around which would effectively be the existing 'always-on' property.
There are no RISC-V platforms using C3STOP. The patch which
added C3STOP has been reverted.
(Refer, https://lore.kernel.org/lkml/[email protected]/T/)
I just need to rebase this patch upon the C3STOP revert patch.
I guess you could say that the C3STOP addition was done spec-ulatively*, as the platform that actually exhibits that behaviour does not use the riscv-timer & its maintainer acked the revert (allwinner d1 family).
*The spec does not make any guarantees about whether events arrive during suspend, only the behaviour *if* they arrive.
Switching the property to "always-on" would require retrofitting that property to every other existing platform (and therefore regressing some behaviour there, no?).
Most of the existing platforms are "toys" or demo platforms though, so it would not, I guess, be the end of the world to do so. Doubly so since none of them actually implement any sleep states that making it an "always-on" property.
I've said since the start that defaulting to C3STOP is the "safer" thing to do, and although we disagreed on this last time Anup, I think the better outcome of someone missing a DT property is inaccessible sleep states rather than going into sleep states they cannot get out of.

Quoted 9 lines

For PolarFire SoC, which I guess is one of the few "commerical" platforms, I'd be willing to accept retrofitting, since we have not yet implemented such sleep states yet.
Maybe Prabhakar knows whether the RZ/Five has either a) implemented sleep states and b) which side of the "timer events arrive in suspend" divide their platform lies on.
I'm particular interested here since that is not a SiFive core complex.

On RZ/Five we haven't implemented the sleep states yet.

Cheers,
Prabhakar