Skip to contentSkip to conversation

arm64: dts: renesas: Re-add voltages to OPP tables

PATCH In review Geert Uytterhoeven 4 participants 14 messages Oct 8, 2024 – Feb 18, 2025

Subsystems (2)
lore
Share & tools
lore

Geert Uytterhoeven

author maintainer
#0 lore raw

cover arm64: dts: renesas: Re-add voltages to OPP tables

Hi all,

When CONFIG_ENERGY_MODEL=y, an error is printed on RZ/G2E and R-Car E3:

cpu cpu0: EM: invalid perf. state: -22

This happens because the Operating Points Parameters tables do not list voltages, as they are all identical. Previously, it was assumed they were optional, and unused, when none of the CPU nodes is tied to a regulator using the "cpu-supply" property. This assumption turned out to be incorrect, causing the reported error message.

This RFC patch series fixes this by adding the missing voltages.

Note that the Energy Model calculates energy efficiency by dividing the (estimated) CPU power consumption by CPU core clock frequency. When all voltages have the same value, the former is proportional to clock frequency, and energy efficiency becomes a constant. Hence all operating points are considered to have the same efficiency, and the Energy Model always picks the one with the highest clock rate (see also [1]).

Alternatively, the Energy Model could be changed to silently ignore OPP tables with missing frequencies. IMHO this is not an unusual case.

Which approach should be taken?
Thanks for your comments!

[1] "PM: EM: Question Potential Issue with EM and OPP Table in cpufreq

     ondemand Governor"
    https://lore.kernel.org/all/[email protected]

Geert Uytterhoeven (2):
arm64: dts: renesas: r8a774c0: Re-add voltages to OPP table
arm64: dts: renesas: r8a77990: Re-add voltages to OPP table

 arch/arm64/boot/dts/renesas/r8a774c0.dtsi | 3 +++
 arch/arm64/boot/dts/renesas/r8a77990.dtsi | 3 +++
 2 files changed, 6 insertions(+)
Signature · 12 lines
2.34.1

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds
author maintainer
#1 lore raw

PATCH 1/2 arm64: dts: renesas: r8a774c0: Re-add voltages to OPP table

When CONFIG_ENERGY_MODEL=y:

cpu cpu0: EM: invalid perf. state: -22

When removing the (incorrect) voltages from the Operating Points Parameters tables, it was assumed they were optional, and unused, when none of the CPU nodes is tied to a regulator using the "cpu-supply" property. This assumption turned out to be incorrect, causing the reported error message.

Fix this by re-adding the (correct) voltages. Note that because all voltages are identical, all operating points are considered to have the same efficiency, and the energy model always picks the one with the highest clock rate.

Fixes 554edc3e9239bb81 ("arm64: dts: renesas: r8a774c0: Remove bogus voltages from OPP table")
---
 arch/arm64/boot/dts/renesas/r8a774c0.dtsi | 3 +++
 1 file changed, 3 insertions(+)
arch/arm64/boot/dts/renesas/r8a774c0.dtsi +3 -0
index 7655d5e3a034166e..5d4db20033212218 100644
@@ -49,14 +49,17 @@ cluster1_opp: opp-table-1 {
 		opp-shared;
 		opp-800000000 {
 			opp-hz = /bits/ 64 <800000000>;
+			opp-microvolt = <1030000>;
 			clock-latency-ns = <300000>;
 		};
 		opp-1000000000 {
 			opp-hz = /bits/ 64 <1000000000>;
+			opp-microvolt = <1030000>;
 			clock-latency-ns = <300000>;
 		};
 		opp-1200000000 {
 			opp-hz = /bits/ 64 <1200000000>;
+			opp-microvolt = <1030000>;
 			clock-latency-ns = <300000>;
 			opp-suspend;
 		};
Signature · 1 line
2.34.1
author maintainer
#2 lore raw

PATCH 2/2 arm64: dts: renesas: r8a77990: Re-add voltages to OPP table

In reply to Geert Uytterhoeven

When CONFIG_ENERGY_MODEL=y:

cpu cpu0: EM: invalid perf. state: -22

When removing the (incorrect) voltages from the Operating Points Parameters tables, it was assumed they were optional, and unused, when none of the CPU nodes is tied to a regulator using the "cpu-supply" property. This assumption turned out to be incorrect, causing the reported error message.

Fix this by re-adding the (correct) voltages. Note that because all voltages are identical, all operating points are considered to have the same efficiency, and the energy model always picks the one with the highest clock rate.

Fixes fb76b0fae3ca8803 ("arm64: dts: renesas: r8a77990: Remove bogus voltages from OPP table")
---
 arch/arm64/boot/dts/renesas/r8a77990.dtsi | 3 +++
 1 file changed, 3 insertions(+)
arch/arm64/boot/dts/renesas/r8a77990.dtsi +3 -0
index 233af3081e84a407..26d08f2cbcb6417b 100644
@@ -49,14 +49,17 @@ cluster1_opp: opp-table-1 {
 		opp-shared;
 		opp-800000000 {
 			opp-hz = /bits/ 64 <800000000>;
+			opp-microvolt = <1030000>;
 			clock-latency-ns = <300000>;
 		};
 		opp-1000000000 {
 			opp-hz = /bits/ 64 <1000000000>;
+			opp-microvolt = <1030000>;
 			clock-latency-ns = <300000>;
 		};
 		opp-1200000000 {
 			opp-hz = /bits/ 64 <1200000000>;
+			opp-microvolt = <1030000>;
 			clock-latency-ns = <300000>;
 			opp-suspend;
 		};
Signature · 1 line
2.34.1
#3 lore raw
In reply to Geert Uytterhoeven

Hi Geert,

Quoted 17 lines

-----Original Message-----
From: Geert Uytterhoeven <[email protected]>
Subject: [PATCH/RFC 0/2] arm64: dts: renesas: Re-add voltages to OPP tables

Hi all,

When CONFIG_ENERGY_MODEL=y, an error is printed on RZ/G2E and R-Car E3:

cpu cpu0: EM: invalid perf. state: -22

This happens because the Operating Points Parameters tables do not list voltages, as they are all identical. Previously, it was assumed they were optional, and unused, when none of the CPU nodes is tied to a regulator using the "cpu-supply" property. This assumption turned out to be incorrect, causing the reported error message.

This RFC patch series fixes this by adding the missing voltages.

Quoted 8 lines

Note that the Energy Model calculates energy efficiency by dividing the
(estimated) CPU power consumption by CPU core clock frequency. When all voltages have the same value, the former is proportional to clock frequency, and energy efficiency becomes a constant. Hence all operating points are considered to have the same efficiency, and the Energy Model always picks the one with the highest clock rate (see also [1]).

Alternatively, the Energy Model could be changed to silently ignore OPP tables with missing frequencies. IMHO this is not an unusual case.

I guess cooling uses OPP table for frequency down scaling at higher temp. So, we need OPP tables for thermal governors??

Cheers,
Biju

Quoted 28 lines

Which approach should be taken?
Thanks for your comments!

[1] "PM: EM: Question Potential Issue with EM and OPP Table in cpufreq

     ondemand Governor"
    https://lore.kernel.org/all/[email protected]

Geert Uytterhoeven (2):
arm64: dts: renesas: r8a774c0: Re-add voltages to OPP table
arm64: dts: renesas: r8a77990: Re-add voltages to OPP table

 arch/arm64/boot/dts/renesas/r8a774c0.dtsi | 3 +++  arch/arm64/boot/dts/renesas/r8a77990.dtsi | 3 +++
 2 files changed, 6 insertions(+)

--
2.34.1

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#4 lore raw

Hi Biju,

Quoted 21 lines · Biju Das
Quoted 21 lines

-----Original Message-----
From: Geert Uytterhoeven <[email protected]>
Subject: [PATCH/RFC 0/2] arm64: dts: renesas: Re-add voltages to OPP tables

When CONFIG_ENERGY_MODEL=y, an error is printed on RZ/G2E and R-Car E3:

cpu cpu0: EM: invalid perf. state: -22

This happens because the Operating Points Parameters tables do not list voltages, as they are all identical. Previously, it was assumed they were optional, and unused, when none of the CPU nodes is tied to a regulator using the "cpu-supply" property. This assumption turned out to be incorrect, causing the reported error message.

This RFC patch series fixes this by adding the missing voltages.

Note that the Energy Model calculates energy efficiency by dividing the
(estimated) CPU power consumption by CPU core clock frequency. When all voltages have the same value, the former is proportional to clock frequency, and energy efficiency becomes a constant. Hence all operating points are considered to have the same efficiency, and the Energy Model always picks the one with the highest clock rate (see also [1]).

Quoted 5 lines
Quoted 2 lines

Alternatively, the Energy Model could be changed to silently ignore OPP tables with missing frequencies. IMHO this is not an unusual case.

I guess cooling uses OPP table for frequency down scaling at higher temp. So, we need OPP tables for thermal governors??

Yes, you still need OPP tables for thermal governance.
But the Energy Model cannot really use it when the voltages are identical.

Gr{oetje,eeting}s,

Geert

Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
#5 lore raw
In reply to Geert Uytterhoeven
Quoted 20 lines · Geert Uytterhoeven

When CONFIG_ENERGY_MODEL=y, an error is printed on RZ/G2E and R-Car E3:

cpu cpu0: EM: invalid perf. state: -22

This happens because the Operating Points Parameters tables do not list voltages, as they are all identical. Previously, it was assumed they were optional, and unused, when none of the CPU nodes is tied to a regulator using the "cpu-supply" property. This assumption turned out to be incorrect, causing the reported error message.

This RFC patch series fixes this by adding the missing voltages.

Note that the Energy Model calculates energy efficiency by dividing the (estimated) CPU power consumption by CPU core clock frequency. When all voltages have the same value, the former is proportional to clock frequency, and energy efficiency becomes a constant. Hence all operating points are considered to have the same efficiency, and the Energy Model always picks the one with the highest clock rate (see also [1]).

Quoted 5 lines

Alternatively, the Energy Model could be changed to silently ignore OPP tables with missing frequencies. IMHO this is not an unusual case.

Which approach should be taken?
Thanks for your comments!

Any comments from the Energy Model and PM people?
Thanks!

Quoted 7 lines

[1] "PM: EM: Question Potential Issue with EM and OPP Table in cpufreq

     ondemand Governor"
    https://lore.kernel.org/all/[email protected]

Geert Uytterhoeven (2):
arm64: dts: renesas: r8a774c0: Re-add voltages to OPP table
arm64: dts: renesas: r8a77990: Re-add voltages to OPP table

Gr{oetje,eeting}s,

Geert

Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
#6 lore raw

Hi Geert,

Quoted 22 lines · Geert Uytterhoeven
Quoted 20 lines · Geert Uytterhoeven

When CONFIG_ENERGY_MODEL=y, an error is printed on RZ/G2E and R-Car E3:

cpu cpu0: EM: invalid perf. state: -22

This happens because the Operating Points Parameters tables do not list voltages, as they are all identical. Previously, it was assumed they were optional, and unused, when none of the CPU nodes is tied to a regulator using the "cpu-supply" property. This assumption turned out to be incorrect, causing the reported error message.

This RFC patch series fixes this by adding the missing voltages.

Note that the Energy Model calculates energy efficiency by dividing the (estimated) CPU power consumption by CPU core clock frequency. When all voltages have the same value, the former is proportional to clock frequency, and energy efficiency becomes a constant. Hence all operating points are considered to have the same efficiency, and the Energy Model always picks the one with the highest clock rate (see also [1]).

Quoted 7 lines
Quoted 5 lines

Alternatively, the Energy Model could be changed to silently ignore OPP tables with missing frequencies. IMHO this is not an unusual case.

Which approach should be taken?
Thanks for your comments!

Any comments from the Energy Model and PM people?

My apologies for delay.

So you had issue with bogus Voltage values and removed them.

There is another way to setup EM properly, via DT:
"opp-microwatt" [1].

That micro watt value won't confuse other subsystems, like your regulator fwk. It will only be used by the EM fwk.

This would be an alternative to your voltage values. Sounds better to you?

Do you know from /sys/kernel/debug/energy_model/ the current power values?

Regards,
Lukasz

[1] https://www.kernel.org/doc/html/v6.11/power/energy-model.html#registration-of-em-using-dt

#7 lore raw

Hi Lukasz,

Quoted 35 lines · Lukasz Luba
Quoted 22 lines · Geert Uytterhoeven
Quoted 20 lines · Geert Uytterhoeven

When CONFIG_ENERGY_MODEL=y, an error is printed on RZ/G2E and R-Car E3:

cpu cpu0: EM: invalid perf. state: -22

This happens because the Operating Points Parameters tables do not list voltages, as they are all identical. Previously, it was assumed they were optional, and unused, when none of the CPU nodes is tied to a regulator using the "cpu-supply" property. This assumption turned out to be incorrect, causing the reported error message.

This RFC patch series fixes this by adding the missing voltages.

Note that the Energy Model calculates energy efficiency by dividing the (estimated) CPU power consumption by CPU core clock frequency. When all voltages have the same value, the former is proportional to clock frequency, and energy efficiency becomes a constant. Hence all operating points are considered to have the same efficiency, and the Energy Model always picks the one with the highest clock rate (see also [1]).

Quoted 7 lines
Quoted 5 lines

Alternatively, the Energy Model could be changed to silently ignore OPP tables with missing frequencies. IMHO this is not an unusual case.

Which approach should be taken?
Thanks for your comments!

Any comments from the Energy Model and PM people?

My apologies for delay.

So you had issue with bogus Voltage values and removed them.

Quoted 8 lines

There is another way to setup EM properly, via DT:
"opp-microwatt" [1].

That micro watt value won't confuse other subsystems, like your regulator fwk. It will only be used by the EM fwk.

This would be an alternative to your voltage values. Sounds better to you?

For opp-microwatt, I do need to know the actual power consumption of the core, right?

Full system power consumption while running the in-kernel Dhrystones benchmark:

800 MHz: avg 4972,55 mW, stdef 20,474 mW
1000 MHz: avg 5025,93 mW, stdef 18,644 mW
1200 MHz: avg 5059,63 mW, stdef 15,425 mW

The system also has test points across a 0.005 Ohm sense resistor in the DVFS power supply line, but no on-board measurement sensor (like the MAX9611 on Salvator-X(S)), so I haven't measured anything there yet.

Quoted 2 lines

Do you know from /sys/kernel/debug/energy_model/ the current power values?

With this series applied:

root@ebisu:~# grep -r . /sys/kernel/debug/energy_model/
/sys/kernel/debug/energy_model/cpu0/ps:1200000/inefficient:0
/sys/kernel/debug/energy_model/cpu0/ps:1200000/performance:1024
/sys/kernel/debug/energy_model/cpu0/ps:1200000/cost:3443
/sys/kernel/debug/energy_model/cpu0/ps:1200000/power:352643
/sys/kernel/debug/energy_model/cpu0/ps:1200000/frequency:1200000
/sys/kernel/debug/energy_model/cpu0/ps:1000000/inefficient:1
/sys/kernel/debug/energy_model/cpu0/ps:1000000/performance:853
/sys/kernel/debug/energy_model/cpu0/ps:1000000/cost:3445
/sys/kernel/debug/energy_model/cpu0/ps:1000000/power:293869
/sys/kernel/debug/energy_model/cpu0/ps:1000000/frequency:1000000
/sys/kernel/debug/energy_model/cpu0/ps:800000/inefficient:1
/sys/kernel/debug/energy_model/cpu0/ps:800000/performance:682
/sys/kernel/debug/energy_model/cpu0/ps:800000/cost:3447
/sys/kernel/debug/energy_model/cpu0/ps:800000/power:235095
/sys/kernel/debug/energy_model/cpu0/ps:800000/frequency:800000
/sys/kernel/debug/energy_model/cpu0/flags:0x3
/sys/kernel/debug/energy_model/cpu0/cpus:0-1

Thanks!

Gr{oetje,eeting}s,

Geert

Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
#8 lore raw
Quoted 41 lines · Geert Uytterhoeven

Hi Lukasz,

Quoted 38 lines · Lukasz Luba
Quoted 22 lines · Geert Uytterhoeven
Quoted 20 lines · Geert Uytterhoeven

When CONFIG_ENERGY_MODEL=y, an error is printed on RZ/G2E and R-Car E3:

cpu cpu0: EM: invalid perf. state: -22

This happens because the Operating Points Parameters tables do not list voltages, as they are all identical. Previously, it was assumed they were optional, and unused, when none of the CPU nodes is tied to a regulator using the "cpu-supply" property. This assumption turned out to be incorrect, causing the reported error message.

This RFC patch series fixes this by adding the missing voltages.

Note that the Energy Model calculates energy efficiency by dividing the (estimated) CPU power consumption by CPU core clock frequency. When all voltages have the same value, the former is proportional to clock frequency, and energy efficiency becomes a constant. Hence all operating points are considered to have the same efficiency, and the Energy Model always picks the one with the highest clock rate (see also [1]).

Quoted 7 lines
Quoted 5 lines

Alternatively, the Energy Model could be changed to silently ignore OPP tables with missing frequencies. IMHO this is not an unusual case.

Which approach should be taken?
Thanks for your comments!

Any comments from the Energy Model and PM people?

My apologies for delay.

So you had issue with bogus Voltage values and removed them.

There is another way to setup EM properly, via DT:
"opp-microwatt" [1].

Quoted 8 lines
Quoted 5 lines

That micro watt value won't confuse other subsystems, like your regulator fwk. It will only be used by the EM fwk.

This would be an alternative to your voltage values. Sounds better to you?

For opp-microwatt, I do need to know the actual power consumption of the core, right?

Correct. You can try to derived that in a way you did and put below. Although, Dhrystone is a synthetic micro-benchmark with small impact to data caches, so it will not use much power.

Quoted 7 lines

Full system power consumption while running the in-kernel Dhrystones benchmark:

800 MHz: avg 4972,55 mW, stdef 20,474 mW
1000 MHz: avg 5025,93 mW, stdef 18,644 mW
1200 MHz: avg 5059,63 mW, stdef 15,425 mW

Right. From those power values can be try to derive the 'CPU only power' values - assuming only one core was running the test.

AFAIU you don't have proper DVFS due to missing voltage scaling.

Therefore...
Out of that I got these CPU power values:
800MHz -> 174mW
1000MHz -> 212mW
1200MHz -> 261mW

Quoted 21 lines

The system also has test points across a 0.005 Ohm sense resistor in the DVFS power supply line, but no on-board measurement sensor (like the MAX9611 on Salvator-X(S)), so I haven't measured anything there yet.

Quoted 2 lines

Do you know from /sys/kernel/debug/energy_model/ the current power values?

With this series applied:

root@ebisu:~# grep -r . /sys/kernel/debug/energy_model/
/sys/kernel/debug/energy_model/cpu0/ps:1200000/inefficient:0
/sys/kernel/debug/energy_model/cpu0/ps:1200000/performance:1024
/sys/kernel/debug/energy_model/cpu0/ps:1200000/cost:3443
/sys/kernel/debug/energy_model/cpu0/ps:1200000/power:352643
/sys/kernel/debug/energy_model/cpu0/ps:1200000/frequency:1200000
/sys/kernel/debug/energy_model/cpu0/ps:1000000/inefficient:1
/sys/kernel/debug/energy_model/cpu0/ps:1000000/performance:853
/sys/kernel/debug/energy_model/cpu0/ps:1000000/cost:3445
/sys/kernel/debug/energy_model/cpu0/ps:1000000/power:293869
Quoted 8 lines
/sys/kernel/debug/energy_model/cpu0/ps:1000000/frequency:1000000
/sys/kernel/debug/energy_model/cpu0/ps:800000/inefficient:1
/sys/kernel/debug/energy_model/cpu0/ps:800000/performance:682
/sys/kernel/debug/energy_model/cpu0/ps:800000/cost:3447
/sys/kernel/debug/energy_model/cpu0/ps:800000/power:235095
/sys/kernel/debug/energy_model/cpu0/ps:800000/frequency:800000
/sys/kernel/debug/energy_model/cpu0/flags:0x3
/sys/kernel/debug/energy_model/cpu0/cpus:0-1

Those power values listed above look a bit higher, but they could be more related to a benchmark which utilized caches and more parts of the CPU. I don't know if you had chance to see some of my presentations on Linux conferences, where I show how much power can vary in different scenarios at the same frequency...
TLDR; it can be even 1.8x comparing to Dhrystone.

So would say it's OK for you to put either your Dhrystone power results, or these one from EM dump (probably from some more heavy benchmark then set into DT coefficient to derive them in OPP fwk).

Your platform AFAIK is not Heterogeneous, so it won't be used in EAS w/ that EM. It will be only used for thermal governor IPA or PowerCap DTPM, which require EM to work.

Regards,
Lukasz

#9 lore raw

Hi Lukasz,

Quoted 46 lines · Lukasz Luba
Quoted 45 lines · Geert Uytterhoeven
Quoted 43 lines · Lukasz Luba
Quoted 22 lines · Geert Uytterhoeven
Quoted 20 lines · Geert Uytterhoeven

When CONFIG_ENERGY_MODEL=y, an error is printed on RZ/G2E and R-Car E3:

cpu cpu0: EM: invalid perf. state: -22

This happens because the Operating Points Parameters tables do not list voltages, as they are all identical. Previously, it was assumed they were optional, and unused, when none of the CPU nodes is tied to a regulator using the "cpu-supply" property. This assumption turned out to be incorrect, causing the reported error message.

This RFC patch series fixes this by adding the missing voltages.

Note that the Energy Model calculates energy efficiency by dividing the (estimated) CPU power consumption by CPU core clock frequency. When all voltages have the same value, the former is proportional to clock frequency, and energy efficiency becomes a constant. Hence all operating points are considered to have the same efficiency, and the Energy Model always picks the one with the highest clock rate (see also [1]).

Quoted 7 lines
Quoted 5 lines

Alternatively, the Energy Model could be changed to silently ignore OPP tables with missing frequencies. IMHO this is not an unusual case.

Which approach should be taken?
Thanks for your comments!

Any comments from the Energy Model and PM people?

My apologies for delay.

So you had issue with bogus Voltage values and removed them.

There is another way to setup EM properly, via DT:
"opp-microwatt" [1].

That micro watt value won't confuse other subsystems, like your regulator fwk. It will only be used by the EM fwk.

This would be an alternative to your voltage values. Sounds better to you?

Quoted 6 lines
Quoted 2 lines

For opp-microwatt, I do need to know the actual power consumption of the core, right?

Correct. You can try to derived that in a way you did and put below. Although, Dhrystone is a synthetic micro-benchmark with small impact to data caches, so it will not use much power.

Do you have a suggestion for a better load test? stress-ng?

Quoted 7 lines
Quoted 6 lines

Full system power consumption while running the in-kernel Dhrystones benchmark:

800 MHz: avg 4972,55 mW, stdef 20,474 mW
1000 MHz: avg 5025,93 mW, stdef 18,644 mW
1200 MHz: avg 5059,63 mW, stdef 15,425 mW

Quoted 5 lines

Right. From those power values can be try to derive the 'CPU only power' values - assuming only one core was running the test.

AFAIU you don't have proper DVFS due to missing voltage scaling.

Indeed.

Quoted 3 lines

Therefore...
Out of that I got these CPU power values:
800MHz -> 174mW

=> 217.5 µW/MHz

Quoted 1 line

1000MHz -> 212mW

=> 212 µW/MHz

Quoted 1 line

1200MHz -> 261mW

=> 217.5 µW/MHz.

So 1000 MHz seems to be the most power-efficient.

Quoted 4 lines
Quoted 4 lines

The system also has test points across a 0.005 Ohm sense resistor in the DVFS power supply line, but no on-board measurement sensor (like the MAX9611 on Salvator-X(S)), so I haven't measured anything there yet.

I'll try to do some measurements at these test points.

Quoted 24 lines
Quoted 15 lines
Quoted 2 lines

Do you know from /sys/kernel/debug/energy_model/ the current power values?

With this series applied:

root@ebisu:~# grep -r . /sys/kernel/debug/energy_model/
/sys/kernel/debug/energy_model/cpu0/ps:1200000/inefficient:0
/sys/kernel/debug/energy_model/cpu0/ps:1200000/performance:1024
/sys/kernel/debug/energy_model/cpu0/ps:1200000/cost:3443
/sys/kernel/debug/energy_model/cpu0/ps:1200000/power:352643
/sys/kernel/debug/energy_model/cpu0/ps:1200000/frequency:1200000
/sys/kernel/debug/energy_model/cpu0/ps:1000000/inefficient:1
/sys/kernel/debug/energy_model/cpu0/ps:1000000/performance:853
/sys/kernel/debug/energy_model/cpu0/ps:1000000/cost:3445
/sys/kernel/debug/energy_model/cpu0/ps:1000000/power:293869
Quoted 8 lines
/sys/kernel/debug/energy_model/cpu0/ps:1000000/frequency:1000000
/sys/kernel/debug/energy_model/cpu0/ps:800000/inefficient:1
/sys/kernel/debug/energy_model/cpu0/ps:800000/performance:682
/sys/kernel/debug/energy_model/cpu0/ps:800000/cost:3447
/sys/kernel/debug/energy_model/cpu0/ps:800000/power:235095
/sys/kernel/debug/energy_model/cpu0/ps:800000/frequency:800000
/sys/kernel/debug/energy_model/cpu0/flags:0x3
/sys/kernel/debug/energy_model/cpu0/cpus:0-1
Quoted 6 lines

Those power values listed above look a bit higher, but they could be more related to a benchmark which utilized caches and more parts of the CPU. I don't know if you had chance to see some of my presentations on Linux conferences, where I show how much power can vary in different scenarios at the same frequency...

Thanks, I will have a look...

Quoted 6 lines

TLDR; it can be even 1.8x comparing to Dhrystone.

So would say it's OK for you to put either your Dhrystone power results, or these one from EM dump (probably from some more heavy benchmark then set into DT coefficient to derive them in OPP fwk).

Given the figures above (212 µW/MHz vs. 217.5 µW/MHz) using rather coarse measurements are already close or identical, they might end up being identical, and then we're back at square zero, where EM cannot do anything?

Thanks!

Gr{oetje,eeting}s,

Geert

Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
#10 lore raw

Hi Lukasz,

Quoted 75 lines · Geert Uytterhoeven
Quoted 46 lines · Lukasz Luba
Quoted 45 lines · Geert Uytterhoeven
Quoted 43 lines · Lukasz Luba
Quoted 22 lines · Geert Uytterhoeven
Quoted 20 lines · Geert Uytterhoeven

When CONFIG_ENERGY_MODEL=y, an error is printed on RZ/G2E and R-Car E3:

cpu cpu0: EM: invalid perf. state: -22

This happens because the Operating Points Parameters tables do not list voltages, as they are all identical. Previously, it was assumed they were optional, and unused, when none of the CPU nodes is tied to a regulator using the "cpu-supply" property. This assumption turned out to be incorrect, causing the reported error message.

This RFC patch series fixes this by adding the missing voltages.

Note that the Energy Model calculates energy efficiency by dividing the (estimated) CPU power consumption by CPU core clock frequency. When all voltages have the same value, the former is proportional to clock frequency, and energy efficiency becomes a constant. Hence all operating points are considered to have the same efficiency, and the Energy Model always picks the one with the highest clock rate (see also [1]).

Quoted 7 lines
Quoted 5 lines

Alternatively, the Energy Model could be changed to silently ignore OPP tables with missing frequencies. IMHO this is not an unusual case.

Which approach should be taken?
Thanks for your comments!

Any comments from the Energy Model and PM people?

My apologies for delay.

So you had issue with bogus Voltage values and removed them.

There is another way to setup EM properly, via DT:
"opp-microwatt" [1].

That micro watt value won't confuse other subsystems, like your regulator fwk. It will only be used by the EM fwk.

This would be an alternative to your voltage values. Sounds better to you?

Quoted 6 lines
Quoted 2 lines

For opp-microwatt, I do need to know the actual power consumption of the core, right?

Correct. You can try to derived that in a way you did and put below. Although, Dhrystone is a synthetic micro-benchmark with small impact to data caches, so it will not use much power.

Do you have a suggestion for a better load test? stress-ng?

Quoted 7 lines
Quoted 6 lines

Full system power consumption while running the in-kernel Dhrystones benchmark:

800 MHz: avg 4972,55 mW, stdef 20,474 mW
1000 MHz: avg 5025,93 mW, stdef 18,644 mW
1200 MHz: avg 5059,63 mW, stdef 15,425 mW

Quoted 5 lines

Right. From those power values can be try to derive the 'CPU only power' values - assuming only one core was running the test.

AFAIU you don't have proper DVFS due to missing voltage scaling.

Indeed.

Quoted 3 lines

Therefore...
Out of that I got these CPU power values:
800MHz -> 174mW

Quoted 7 lines

=> 217.5 µW/MHz

Quoted 1 line

1000MHz -> 212mW

=> 212 µW/MHz

Quoted 1 line

1200MHz -> 261mW

BTW, how did you get from my avg mW values above to your CPU power mW values? I seem to be missing something...

Quoted 11 lines

=> 217.5 µW/MHz.

So 1000 MHz seems to be the most power-efficient.

Quoted 4 lines
Quoted 4 lines

The system also has test points across a 0.005 Ohm sense resistor in the DVFS power supply line, but no on-board measurement sensor (like the MAX9611 on Salvator-X(S)), so I haven't measured anything there yet.

I'll try to do some measurements at these test points.

So I measured the voltage across the sense resistor, and used that to calculate the actual power draw:
A. Idle (1 or 2 CPU cores online doesn't seem to matter):

      -  765 mW @  800 MHz,
      -  786 mW @ 1000 MHz,
      -  807 mW @ 1200 MHz.

B. Maximum seen during "stress-ng -c 10" with 1 CPU core online:
- 993 mW @ 800 MHz,
- 1055 mW @ 1000 MHz,
- 1096 mW @ 1200 MHz.
C. Maximum seen during "stress-ng -c 10" with 2 CPU cores online:
- 1179 mW @ 800 MHz,
- 1303 mW @ 1000 MHz,
- 1386 mW @ 1200 MHz.

As expected, the A-C increase is about the double of the A-B increase, due to the use of 2 CPU cores.

Thanks again!

Gr{oetje,eeting}s,

Geert

Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
#11 lore raw

Hi Lukasz,

Quoted 78 lines · Geert Uytterhoeven
Quoted 77 lines · Geert Uytterhoeven
Quoted 46 lines · Lukasz Luba
Quoted 45 lines · Geert Uytterhoeven
Quoted 43 lines · Lukasz Luba
Quoted 22 lines · Geert Uytterhoeven
Quoted 20 lines · Geert Uytterhoeven

When CONFIG_ENERGY_MODEL=y, an error is printed on RZ/G2E and R-Car E3:

cpu cpu0: EM: invalid perf. state: -22

This happens because the Operating Points Parameters tables do not list voltages, as they are all identical. Previously, it was assumed they were optional, and unused, when none of the CPU nodes is tied to a regulator using the "cpu-supply" property. This assumption turned out to be incorrect, causing the reported error message.

This RFC patch series fixes this by adding the missing voltages.

Note that the Energy Model calculates energy efficiency by dividing the (estimated) CPU power consumption by CPU core clock frequency. When all voltages have the same value, the former is proportional to clock frequency, and energy efficiency becomes a constant. Hence all operating points are considered to have the same efficiency, and the Energy Model always picks the one with the highest clock rate (see also [1]).

Quoted 7 lines
Quoted 5 lines

Alternatively, the Energy Model could be changed to silently ignore OPP tables with missing frequencies. IMHO this is not an unusual case.

Which approach should be taken?
Thanks for your comments!

Any comments from the Energy Model and PM people?

My apologies for delay.

So you had issue with bogus Voltage values and removed them.

There is another way to setup EM properly, via DT:
"opp-microwatt" [1].

That micro watt value won't confuse other subsystems, like your regulator fwk. It will only be used by the EM fwk.

This would be an alternative to your voltage values. Sounds better to you?

Quoted 6 lines
Quoted 2 lines

For opp-microwatt, I do need to know the actual power consumption of the core, right?

Correct. You can try to derived that in a way you did and put below. Although, Dhrystone is a synthetic micro-benchmark with small impact to data caches, so it will not use much power.

Do you have a suggestion for a better load test? stress-ng?

Quoted 7 lines
Quoted 6 lines

Full system power consumption while running the in-kernel Dhrystones benchmark:

800 MHz: avg 4972,55 mW, stdef 20,474 mW
1000 MHz: avg 5025,93 mW, stdef 18,644 mW
1200 MHz: avg 5059,63 mW, stdef 15,425 mW

Quoted 5 lines

Right. From those power values can be try to derive the 'CPU only power' values - assuming only one core was running the test.

AFAIU you don't have proper DVFS due to missing voltage scaling.

Indeed.

Quoted 3 lines

Therefore...
Out of that I got these CPU power values:
800MHz -> 174mW

=> 217.5 µW/MHz

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

1000MHz -> 212mW

=> 212 µW/MHz

Quoted 1 line

1200MHz -> 261mW

BTW, how did you get from my avg mW values above to your CPU power mW values? I seem to be missing something...

Ping...

Gr{oetje,eeting}s,

Geert

Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
#12 lore raw

Hi Lukasz,

Quoted 81 lines · Geert Uytterhoeven
Quoted 80 lines · Geert Uytterhoeven
Quoted 79 lines · Geert Uytterhoeven
Quoted 46 lines · Lukasz Luba
Quoted 45 lines · Geert Uytterhoeven
Quoted 43 lines · Lukasz Luba
Quoted 22 lines · Geert Uytterhoeven
Quoted 20 lines · Geert Uytterhoeven

When CONFIG_ENERGY_MODEL=y, an error is printed on RZ/G2E and R-Car E3:

cpu cpu0: EM: invalid perf. state: -22

This happens because the Operating Points Parameters tables do not list voltages, as they are all identical. Previously, it was assumed they were optional, and unused, when none of the CPU nodes is tied to a regulator using the "cpu-supply" property. This assumption turned out to be incorrect, causing the reported error message.

This RFC patch series fixes this by adding the missing voltages.

Note that the Energy Model calculates energy efficiency by dividing the (estimated) CPU power consumption by CPU core clock frequency. When all voltages have the same value, the former is proportional to clock frequency, and energy efficiency becomes a constant. Hence all operating points are considered to have the same efficiency, and the Energy Model always picks the one with the highest clock rate (see also [1]).

Quoted 7 lines
Quoted 5 lines

Alternatively, the Energy Model could be changed to silently ignore OPP tables with missing frequencies. IMHO this is not an unusual case.

Which approach should be taken?
Thanks for your comments!

Any comments from the Energy Model and PM people?

My apologies for delay.

So you had issue with bogus Voltage values and removed them.

There is another way to setup EM properly, via DT:
"opp-microwatt" [1].

That micro watt value won't confuse other subsystems, like your regulator fwk. It will only be used by the EM fwk.

This would be an alternative to your voltage values. Sounds better to you?

Quoted 6 lines
Quoted 2 lines

For opp-microwatt, I do need to know the actual power consumption of the core, right?

Correct. You can try to derived that in a way you did and put below. Although, Dhrystone is a synthetic micro-benchmark with small impact to data caches, so it will not use much power.

Do you have a suggestion for a better load test? stress-ng?

Quoted 7 lines
Quoted 6 lines

Full system power consumption while running the in-kernel Dhrystones benchmark:

800 MHz: avg 4972,55 mW, stdef 20,474 mW
1000 MHz: avg 5025,93 mW, stdef 18,644 mW
1200 MHz: avg 5059,63 mW, stdef 15,425 mW

Quoted 5 lines

Right. From those power values can be try to derive the 'CPU only power' values - assuming only one core was running the test.

AFAIU you don't have proper DVFS due to missing voltage scaling.

Indeed.

Quoted 3 lines

Therefore...
Out of that I got these CPU power values:
800MHz -> 174mW

=> 217.5 µW/MHz

Quoted 1 line

1000MHz -> 212mW

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

=> 212 µW/MHz

Quoted 1 line

1200MHz -> 261mW

BTW, how did you get from my avg mW values above to your CPU power mW values? I seem to be missing something...

Ping...

Gentle ping.
Thanks!

Gr{oetje,eeting}s,

Geert

Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
#13 lore raw
In reply to Geert Uytterhoeven
Quoted 30 lines · Geert Uytterhoeven

When CONFIG_ENERGY_MODEL=y, an error is printed on RZ/G2E and R-Car E3:

cpu cpu0: EM: invalid perf. state: -22

This happens because the Operating Points Parameters tables do not list voltages, as they are all identical. Previously, it was assumed they were optional, and unused, when none of the CPU nodes is tied to a regulator using the "cpu-supply" property. This assumption turned out to be incorrect, causing the reported error message.

This RFC patch series fixes this by adding the missing voltages.

Note that the Energy Model calculates energy efficiency by dividing the (estimated) CPU power consumption by CPU core clock frequency. When all voltages have the same value, the former is proportional to clock frequency, and energy efficiency becomes a constant. Hence all operating points are considered to have the same efficiency, and the Energy Model always picks the one with the highest clock rate (see also [1]).

Alternatively, the Energy Model could be changed to silently ignore OPP tables with missing frequencies. IMHO this is not an unusual case.

Which approach should be taken?
Thanks for your comments!

[1] "PM: EM: Question Potential Issue with EM and OPP Table in cpufreq

     ondemand Governor"
    https://lore.kernel.org/all/[email protected]
Quoted 7 lines

Geert Uytterhoeven (2):
arm64: dts: renesas: r8a774c0: Re-add voltages to OPP table
arm64: dts: renesas: r8a77990: Re-add voltages to OPP table

 arch/arm64/boot/dts/renesas/r8a774c0.dtsi | 3 +++
 arch/arm64/boot/dts/renesas/r8a77990.dtsi | 3 +++
 2 files changed, 6 insertions(+)

Queuing in renesas-devel for v6.15.
This can be replaced by an alternative solution later...

Gr{oetje,eeting}s,

Geert

Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds