cpuidle: menu: Remove incorrect unlikely() annotation
PATCH 1/1 cpuidle: menu: Remove incorrect unlikely() annotation
The unlikely() annotation on the early-return condition in menu_select() is incorrect on systems with only one idle state (e.g., ARM64 servers with a single ACPI LPI state). Branch profiling shows 100% misprediction on such systems since drv->state_count <= 1 is always true.
On platforms where only state0 is available, this path is the common case, not an unlikely edge case. Remove the misleading annotation to let the branch predictor learn the actual behavior.
--- drivers/cpuidle/governors/menu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/cpuidle/governors/menu.c +1 -1
@@ -271,7 +271,7 @@ static int menu_select(struct cpuidle_driver *drv, struct cpuidle_device *dev, data->bucket =BUCKETS -1 ; } -if (unlikely (drv->state_count <=1 || latency_req ==0 ) || +if (drv->state_count <=1 || latency_req ==0 || ((data->next_timer_ns < drv->states[1 ].target_residency_ns || latency_req < drv->states[1 ].exit_latency_ns) && !dev->states_usage[0 ].disable)) {
---
base-commit: 34aa263125b6732375abcb908d73d98169154bb5
change-id: 20260105-annotated_idle-d6b614ecd207
Best regards,
Signature · 1 line
Breno Leitao <[email protected]>
Christian Loehle
reviewerQuoted 32 lines · Breno Leitao
The
unlikely()annotation on the early-return condition inmenu_select()is incorrect on systems with only one idle state (e.g., ARM64 servers with a single ACPI LPI state). Branch profiling shows 100% misprediction on such systems sincedrv->state_count<= 1 is always true.On platforms where only state0 is available, this path is the common case, not an unlikely edge case. Remove the misleading annotation to let the branch predictor learn the actual behavior.
Signed-off-by Breno Leitao <[email protected]>--- drivers/cpuidle/governors/menu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/cpuidle/governors/menu.c +1
-2@@ -271,7 +271,7 @@ static int menu_select(struct cpuidle_driver *drv, struct cpuidle_device *dev, data->bucket =BUCKETS -1 ; } -if (unlikely (drv->state_count <=1 || latency_req ==0 ) || +if (drv->state_count <=1 || latency_req ==0 || ((data->next_timer_ns < drv->states[1 ].target_residency_ns || latency_req < drv->states[1 ].exit_latency_ns) && !dev->states_usage[0 ].disable)) { ---base-commit:
34aa263125b6732375abcb908d73d98169154bb5
change-id: 20260105-annotated_idle-d6b614ecd207
Quoted 5 lines
Best regards,
Signature · 1 line
Breno Leitao <[email protected]>
Fine with me per se, I don't think the unlikely() annotation makes a difference for the 'good case' either, but if you run into this I'd be curious if you can see a difference with menu (which should stop the tick on every idle enter regardless) and teo (which should never stop the tick on state_count == 1). Alternative you can also just change the menu branch to not stop the tick. I'd like to know if we need something more sophisticated generally here.
Quoted 33 lines · Christian Loehle
Quoted 32 lines · Breno Leitao
The
unlikely()annotation on the early-return condition inmenu_select()is incorrect on systems with only one idle state (e.g., ARM64 servers with a single ACPI LPI state). Branch profiling shows 100% misprediction on such systems sincedrv->state_count<= 1 is always true.On platforms where only state0 is available, this path is the common case, not an unlikely edge case. Remove the misleading annotation to let the branch predictor learn the actual behavior.
Signed-off-by Breno Leitao <[email protected]>--- drivers/cpuidle/governors/menu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/cpuidle/governors/menu.c +1
-2@@ -271,7 +271,7 @@ static int menu_select(struct cpuidle_driver *drv, struct cpuidle_device *dev, data->bucket =BUCKETS -1 ; } -if (unlikely (drv->state_count <=1 || latency_req ==0 ) || +if (drv->state_count <=1 || latency_req ==0 || ((data->next_timer_ns < drv->states[1 ].target_residency_ns || latency_req < drv->states[1 ].exit_latency_ns) && !dev->states_usage[0 ].disable)) { ---base-commit:
34aa263125b6732375abcb908d73d98169154bb5
change-id: 20260105-annotated_idle-d6b614ecd207
Quoted 12 lines
Quoted 5 lines
Best regards,
Signature · 1 line
Breno Leitao <[email protected]>Fine with me per se, I don't think the
unlikely()annotation makes a difference for the 'good case' either, but if you run into this I'd be curious if you can see a difference with menu (which should stop the tick on every idle enter regardless) and teo (which should never stop the tick onstate_count== 1). Alternative you can also just change the menu branch to not stop the tick. I'd like to know if we need something more sophisticated generally here.
Probably not. I am just running PROFILE_ANNOTATED_BRANCHES tests on some production arm64 host and addressing those that are making wrong assumptions.
Quoted 20 lines · Breno Leitao
The
unlikely()annotation on the early-return condition inmenu_select()is incorrect on systems with only one idle state (e.g., ARM64 servers with a single ACPI LPI state). Branch profiling shows 100% misprediction on such systems sincedrv->state_count<= 1 is always true.On platforms where only state0 is available, this path is the common case, not an unlikely edge case. Remove the misleading annotation to let the branch predictor learn the actual behavior.
Signed-off-by Breno Leitao <[email protected]>--- drivers/cpuidle/governors/menu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/cpuidle/governors/menu.c +0
-0
drivers/cpuidle/governors/menu.c static int menu_select(struct cpuidle_driver *drv, struct cpuidle_device *dev,
Quoted 10 lines
drivers/cpuidle/governors/menu.c +1
-2@@ -271,7 +271,7 @@ static int menu_select(struct cpuidle_driver *drv, struct cpuidle_device *dev, data->bucket =BUCKETS -1 ; } -if (unlikely (drv->state_count <=1 || latency_req ==0 ) || +if (drv->state_count <=1 || latency_req ==0 || ((data->next_timer_ns < drv->states[1 ].target_residency_ns || latency_req < drv->states[1 ].exit_latency_ns) && !dev->states_usage[0 ].disable)) { ---
Applied as 6.20 material, thanks!