cpufreq/thermal regression in 6.10
Subsystems (2)
Hi,
Steev reported to me off-list that the CPU frequency of the big cores on the Lenovo ThinkPad X13s sometimes appears to get stuck at a low frequency with 6.10-rc2.
I just confirmed that once the cores are fully throttled (using the stepwise thermal governor) due to the skin temperature reaching the first trip point, scaling_max_freq gets stuck at the next OPP:
cpu4/cpufreq/scaling_max_freq:940800 cpu5/cpufreq/scaling_max_freq:940800 cpu6/cpufreq/scaling_max_freq:940800 cpu7/cpufreq/scaling_max_freq:940800
when the temperature drops again.
This obviously leads to a massive performance drop and could possibly also be related to reports like this one:
https://lore.kernel.org/all/CAHk-=wjwFGQZcDinK=BkEaA8FSyVg5NaUe0BobxowxeZ5PvetA@mail.gmail.com/
I assume the regression may have been introduced by all the thermal work that went into 6.10-rc1, but I don't have time to try to track this down myself right now (and will be away from keyboard most of next week).
I've confirmed that 6.9 works as expected.
Johan
#regzbot introduced: v6.9..v6.10-rc2
Rafael J. Wysocki
maintainerHi,
Thanks for the report.
Quoted 11 lines · Johan Hovold
Hi,
Steev reported to me off-list that the CPU frequency of the big cores on the Lenovo ThinkPad X13s sometimes appears to get stuck at a low frequency with 6.10-rc2.
I just confirmed that once the cores are fully throttled (using the stepwise thermal governor) due to the skin temperature reaching the first trip point,
scaling_max_freqgets stuck at the next OPP:
Quoted 6 lines
cpu4/cpufreq/scaling_max_freq:940800 cpu5/cpufreq/scaling_max_freq:940800 cpu6/cpufreq/scaling_max_freq:940800 cpu7/cpufreq/scaling_max_freq:940800when the temperature drops again.
So apparently something fails to update its frequency QoS request.
Would it be possible to provoke this with thermal debug enabled (CONFIG_THERMAL_DEBUGFS set) and see what's there in /sys/kernel/debug/thermal/?
Quoted 10 lines
This obviously leads to a massive performance drop and could possibly also be related to reports like this one:
https://lore.kernel.org/all/CAHk-=wjwFGQZcDinK=BkEaA8FSyVg5NaUe0BobxowxeZ5PvetA@mail.gmail.com/
I assume the regression may have been introduced by all the thermal work that went into 6.10-rc1, but I don't have time to try to track this down myself right now (and will be away from keyboard most of next week).
I've confirmed that 6.9 works as expected.
Well, I'd need to ask someone else affected by this, then.
Rafael J. Wysocki
maintainerQuoted 35 lines · Rafael J. Wysocki
Hi,
Thanks for the report.
Quoted 11 lines · Johan Hovold
Hi,
Steev reported to me off-list that the CPU frequency of the big cores on the Lenovo ThinkPad X13s sometimes appears to get stuck at a low frequency with 6.10-rc2.
I just confirmed that once the cores are fully throttled (using the stepwise thermal governor) due to the skin temperature reaching the first trip point,
scaling_max_freqgets stuck at the next OPP:Quoted 6 lines
cpu4/cpufreq/scaling_max_freq:940800 cpu5/cpufreq/scaling_max_freq:940800 cpu6/cpufreq/scaling_max_freq:940800 cpu7/cpufreq/scaling_max_freq:940800when the temperature drops again.
So apparently something fails to update its frequency QoS request.
Would it be possible to provoke this with thermal debug enabled (
CONFIG_THERMAL_DEBUGFSset) and see what's there in/sys/kernel/debug/thermal/?Quoted 5 lines
This obviously leads to a massive performance drop and could possibly also be related to reports like this one:
https://lore.kernel.org/all/CAHk-=wjwFGQZcDinK=BkEaA8FSyVg5NaUe0BobxowxeZ5PvetA@mail.gmail.com/
Quoted 7 lines
Quoted 5 lines
I assume the regression may have been introduced by all the thermal work that went into 6.10-rc1, but I don't have time to try to track this down myself right now (and will be away from keyboard most of next week).
I've confirmed that 6.9 works as expected.
Well, I'd need to ask someone else affected by this, then.
If this is the step-wise governor, the problem might have been introduced by commit
042a3d80f118 thermal: core: Move passive polling management to the core
which removed passive polling count updates from that governor, so if the thermal zone in question has passive polling only and no regular polling, temperature updates may stop coming before the governor drops the cooling device states to the "no target" level.
So please test the attached partial revert of the above commit when you can.
--- drivers/thermal/gov_step_wise.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
drivers/thermal/gov_step_wise.c +10 -0
@@ -93,6 +93,16 @@ static void thermal_zone_trip_update(strif (instance->initialized && old_target == instance->target)continue ; +if (trip->type ==THERMAL_TRIP_PASSIVE ) { +/* If needed, update the status of passive polling. */ +if (old_target ==THERMAL_NO_TARGET && + instance->target !=THERMAL_NO_TARGET ) + tz->passive++; +else if (old_target !=THERMAL_NO_TARGET && + instance->target ==THERMAL_NO_TARGET ) + tz->passive--; + } + instance->initialized = true;mutex_lock (&instance->cdev->lock);
Johan Hovold
Tested-byQuoted 21 lines · Rafael J. Wysocki
Quoted 10 lines · Rafael J. Wysocki
Quoted 8 lines · Johan Hovold
Steev reported to me off-list that the CPU frequency of the big cores on the Lenovo ThinkPad X13s sometimes appears to get stuck at a low frequency with 6.10-rc2.
I just confirmed that once the cores are fully throttled (using the stepwise thermal governor) due to the skin temperature reaching the first trip point,
scaling_max_freqgets stuck at the next OPP:Quoted 6 lines
Quoted 6 lines
cpu4/cpufreq/scaling_max_freq:940800 cpu5/cpufreq/scaling_max_freq:940800 cpu6/cpufreq/scaling_max_freq:940800 cpu7/cpufreq/scaling_max_freq:940800when the temperature drops again.
If this is the step-wise governor, the problem might have been introduced by commit
Quoted 8 lines
042a3d80f118thermal: core: Move passive polling management to the corewhich removed passive polling count updates from that governor, so if the thermal zone in question has passive polling only and no regular polling, temperature updates may stop coming before the governor drops the cooling device states to the "no target" level.
So please test the attached partial revert of the above commit when you can.
Thanks for the quick fix. The partial revert seems to do the trick:
Johan
Steev Klimaszewski
Tested-byQuoted 27 lines · Johan Hovold
Quoted 26 lines · Rafael J. Wysocki
Quoted 9 lines · Rafael J. Wysocki
Quoted 8 lines · Johan Hovold
Steev reported to me off-list that the CPU frequency of the big cores on the Lenovo ThinkPad X13s sometimes appears to get stuck at a low frequency with 6.10-rc2.
I just confirmed that once the cores are fully throttled (using the stepwise thermal governor) due to the skin temperature reaching the first trip point,
scaling_max_freqgets stuck at the next OPP:Quoted 6 lines
Quoted 6 lines
cpu4/cpufreq/scaling_max_freq:940800 cpu5/cpufreq/scaling_max_freq:940800 cpu6/cpufreq/scaling_max_freq:940800 cpu7/cpufreq/scaling_max_freq:940800when the temperature drops again.
If this is the step-wise governor, the problem might have been introduced by commit
042a3d80f118thermal: core: Move passive polling management to the corewhich removed passive polling count updates from that governor, so if the thermal zone in question has passive polling only and no regular polling, temperature updates may stop coming before the governor drops the cooling device states to the "no target" level.
Quoted 6 lines
Quoted 1 line
So please test the attached partial revert of the above commit when you can.
Thanks for the quick fix. The partial revert seems to do the trick:
Tested-by Johan Hovold <[email protected]>Johan
I can also confirm that it's working here!
Hi there,
unfortunately I experienced the issue with the fix applied. I had to revert this and the original commit to get back to normal behaviour. My system (also Lenovo Thinkpad X13s) uses the schedutil governor, the behaviour is as described from Steev and Johan. The full throttling happened during a package build and left the performance cores at 940800.
Cheers
Jens
Quoted 50 lines · Rafael J. Wysocki
Quoted 30 lines · Rafael J. Wysocki
Hi,
Thanks for the report.
Quoted 10 lines · Johan Hovold
Hi,
Steev reported to me off-list that the CPU frequency of the big cores on the Lenovo ThinkPad X13s sometimes appears to get stuck at a low frequency with 6.10-rc2.
I just confirmed that once the cores are fully throttled (using the stepwise thermal governor) due to the skin temperature reaching the first trip point,
scaling_max_freqgets stuck at the next OPP:Quoted 6 lines
cpu4/cpufreq/scaling_max_freq:940800 cpu5/cpufreq/scaling_max_freq:940800 cpu6/cpufreq/scaling_max_freq:940800 cpu7/cpufreq/scaling_max_freq:940800when the temperature drops again.
So apparently something fails to update its frequency QoS request.
Would it be possible to provoke this with thermal debug enabled (
CONFIG_THERMAL_DEBUGFSset) and see what's there in/sys/kernel/debug/thermal/?Quoted 3 lines
This obviously leads to a massive performance drop and could possibly also be related to reports like this one:
Quoted 8 lines
Quoted 7 lines
https://lore.kernel.org/all/CAHk-=wjwFGQZcDinK=BkEaA8FSyVg5NaUe0BobxowxeZ5PvetA@mail.gmail.com/
I assume the regression may have been introduced by all the thermal work that went into 6.10-rc1, but I don't have time to try to track this down myself right now (and will be away from keyboard most of next week).
I've confirmed that 6.9 works as expected.
Well, I'd need to ask someone else affected by this, then.
If this is the step-wise governor, the problem might have been introduced by commit
042a3d80f118thermal: core: Move passive polling management to the corewhich removed passive polling count updates from that governor, so if the thermal zone in question has passive polling only and no regular polling, temperature updates may stop coming before the governor drops the cooling device states to the "no target" level.
So please test the attached partial revert of the above commit when you can.
Rafael J. Wysocki
maintainerHi,
Quoted 8 lines · Jens Glathe
Hi there,
unfortunately I experienced the issue with the fix applied. I had to revert this and the original commit to get back to normal behaviour. My system (also Lenovo Thinkpad X13s) uses the schedutil governor, the behaviour is as described from Steev and Johan. The full throttling happened during a package build and left the performance cores at 940800.
So can you please test the attached patch, on top of the fix?
--- drivers/thermal/gov_step_wise.c | 19 +------------------ 1 file changed, 1 insertion(+), 18 deletions(-)
drivers/thermal/gov_step_wise.c +1 -18
@@ -55,7 +55,7 @@ static unsigned long get_target_state(stif (cur_state <= instance->lower)return THERMAL_NO_TARGET ; -return clamp (cur_state -1 , instance->lower, instance->upper); +return instance->lower; }return instance->target;
@@ -93,23 +93,6 @@ static void thermal_zone_trip_update(strif (instance->initialized && old_target == instance->target)continue ; -if (trip->type ==THERMAL_TRIP_PASSIVE ) { -/* -* If the target state for this thermal instance -* changes from THERMAL_NO_TARGET to something else, -* ensure that the zone temperature will be updated -* (assuming enabled passive cooling) until it becomes -* THERMAL_NO_TARGET again, or the cooling device may -* not be reset to its initial state. -*/ -if (old_target ==THERMAL_NO_TARGET && - instance->target !=THERMAL_NO_TARGET ) - tz->passive++; -else if (old_target !=THERMAL_NO_TARGET && - instance->target ==THERMAL_NO_TARGET ) - tz->passive--; - } - instance->initialized = true;mutex_lock (&instance->cdev->lock);
Jens Glathe
Tested-byHi there,
thank you for the fast fix. Applied, built, installed. Test is successful, performance core scaling up to 2995200 comes back when skin temp drops below 55°C.
Cheers
Jens
Quoted 12 lines · Rafael J. Wysocki
Hi,
Quoted 7 lines · Jens Glathe
Hi there,
unfortunately I experienced the issue with the fix applied. I had to revert this and the original commit to get back to normal behaviour. My system (also Lenovo Thinkpad X13s) uses the schedutil governor, the behaviour is as described from Steev and Johan. The full throttling happened during a package build and left the performance cores at 940800.
So can you please test the attached patch, on top of the fix?
Rafael J. Wysocki
maintainerQuoted 8 lines · Jens Glathe
Hi there,
thank you for the fast fix. Applied, built, installed. Test is successful, performance core scaling up to 2995200 comes back when skin temp drops below 55°C.
Tested-by Jens Glathe <[email protected]>
Great, thanks for testing!
I'll submit the patch shortly and it would be good if the others affected by this issue could try it to confirm that it doesn't regress anything.
Quoted 14 lines · Rafael J. Wysocki
Quoted 12 lines
Hi,
Quoted 7 lines · Jens Glathe
Hi there,
unfortunately I experienced the issue with the fix applied. I had to revert this and the original commit to get back to normal behaviour. My system (also Lenovo Thinkpad X13s) uses the schedutil governor, the behaviour is as described from Steev and Johan. The full throttling happened during a package build and left the performance cores at 940800.
So can you please test the attached patch, on top of the fix?