Skip to contentSkip to conversation

x86_energy_perf_policy

DISCUSSION Doug Smythies 2 participants 3 messages Jun 16, 2020 – Aug 13, 2020

lore
Share & tools
lore

Doug Smythies

author
#0 lore raw

Hi Len,

I have been using x86_energy_perf_policy to manage EPB while I do some tests. I noticed it says this:

"HWP can be enabled using '--hwp-enable'"

So, I thought great, I can now in one boot do:

intel_pstate, no-HWP, performance/powersave.
intel_cpufreq, no-HWP, all governors

run: x86_energy_perf_policy --hwp-enable

then repeat the above, but only passive, if trying some of Rafael's RFC stuff, which is NOT part of this e-mail.

However, enabling HWP via this method leaves my system in an unusable state.

Please confirm or deny if I am supposed to be able to enable HWP via x86_energy_perf_policy?

No need to read on:

Examples (kernel 5.8=rc1):

Note: I wrote my own MSR decoder. I know I could use turbostat, and do have it running virtually 100% of the time. However, writing my own decoder forced me to learn more.

Fresh boot (no=hwp forced):

root@s18:/home/doug# /home/doug/c/msr-decoder
8.) 0x198: IA32_PERF_STATUS     : CPU 0-5 :   8 :   8 :   8 :   8 :   8 :   8 :
B.) 0x770: IA32_PM_ENABLE: 0 : HWP disable
9.) 0x199: IA32_PERF_CTL        : CPU 0-5 :   9 :   8 :   8 :   8 :  19 :  26 :
C.) 0x1B0: IA32_ENERGY_PERF_BIAS: CPU 0-5 :   6 :   6 :   6 :   6 :   6 :   6 :
1.) 0x19C: IA32_THERM_STATUS: 88480000
2.) 0x1AA: MSR_MISC_PWR_MGMT: 401CC0 EIST enabled Coordination enabled OOB Bit 8 reset OOB Bit 18 reset
3.) 0x1B1: IA32_PACKAGE_THERM_STATUS: 88470000
4.) 0x64F: MSR_CORE_PERF_LIMIT_REASONS: 0
A.) 0x1FC: MSR_POWER_CTL: 3C005D : C1E disable : RHO disable : EEO disable

Now,

# x86_energy_perf_policy --hwp-enable
root@s18:/home/doug# /home/doug/c/msr-decoder
8.) 0x198: IA32_PERF_STATUS     : CPU 0-5 :   8 :   8 :   8 :   8 :   8 :   8 :
B.) 0x770: IA32_PM_ENABLE: 1 : HWP enable
1.) 0x19C: IA32_THERM_STATUS: 88480000
2.) 0x1AA: MSR_MISC_PWR_MGMT: 401CC0 EIST enabled Coordination enabled OOB Bit 8 reset OOB Bit 18 reset
3.) 0x1B1: IA32_PACKAGE_THERM_STATUS: 88460000
4.) 0x64F: MSR_CORE_PERF_LIMIT_REASONS: 0
A.) 0x1FC: MSR_POWER_CTL: 3C005D : C1E disable : RHO disable : EEO disable
5.) 0x771: IA32_HWP_CAPABILITIES (performance): 108252E : high 46 : guaranteed 37 : efficient 8 : lowest 1
6.) 0x774: IA32_HWP_REQUEST:    CPU 0-5 :
    raw: 8000FF01 : 8000FF01 : 8000FF01 : 8000FF01 : 8000FF01 : 8000FF01 :
    min:        1 :        1 :        1 :        1 :        1 :        1 :
    max:      255 :      255 :      255 :      255 :      255 :      255 :
    des:        0 :        0 :        0 :        0 :        0 :        0 :
    epp:      128 :      128 :      128 :      128 :      128 :      128 :
    act:        0 :        0 :        0 :        0 :        0 :        0 :
7.) 0x777: IA32_HWP_STATUS: 0 : high 0 : guaranteed 0 : efficient 0 : lowest 0

Notice that EPP is 128, and I can not achieve full CPU speed for a workflow that should. Ok. So maybe I just need to toggle between performance and powersave and back. No difference. I tried many variations. There are many situations where the misbehaves.

If I boot with HWP enabled, I can observe EPP (and min and max) behave as expected and as a function of the governor. The system seems to behave.

intel_pstate / powersave:

root@s18:/home/doug# /home/doug/c/msr-decoder
8.) 0x198: IA32_PERF_STATUS     : CPU 0-5 :   8 :   8 :   8 :   8 :   8 :   8 :
B.) 0x770: IA32_PM_ENABLE: 1 : HWP enable
1.) 0x19C: IA32_THERM_STATUS: 88480800 PWL
2.) 0x1AA: MSR_MISC_PWR_MGMT: 401CC0 EIST enabled Coordination enabled OOB Bit 8 reset OOB Bit 18 reset
3.) 0x1B1: IA32_PACKAGE_THERM_STATUS: 88470800 PWL
4.) 0x64F: MSR_CORE_PERF_LIMIT_REASONS: 4000000 PL1L
A.) 0x1FC: MSR_POWER_CTL: 3C005D : C1E disable : RHO disable : EEO disable
5.) 0x771: IA32_HWP_CAPABILITIES (performance): 108252E : high 46 : guaranteed 37 : efficient 8 : lowest 1
6.) 0x774: IA32_HWP_REQUEST:    CPU 0-5 :
    raw: 80002E08 : 80002E08 : 80002E08 : 80002E08 : 80002E08 : 80002E08 :
    min:        8 :        8 :        8 :        8 :        8 :        8 :
    max:       46 :       46 :       46 :       46 :       46 :       46 :
    des:        0 :        0 :        0 :        0 :        0 :        0 :
    epp:      128 :      128 :      128 :      128 :      128 :      128 :
    act:        0 :        0 :        0 :        0 :        0 :        0 :
7.) 0x777: IA32_HWP_STATUS: 4 : high 4 : guaranteed 0 : efficient 0 : lowest 0

intel_pstate / performance:

root@s18:/home/doug# /home/doug/c/msr-decoder
8.) 0x198: IA32_PERF_STATUS     : CPU 0-5 :  46 :  46 :  46 :  46 :  46 :  46 :
B.) 0x770: IA32_PM_ENABLE: 1 : HWP enable
1.) 0x19C: IA32_THERM_STATUS: 88470800 PWL
2.) 0x1AA: MSR_MISC_PWR_MGMT: 401CC0 EIST enabled Coordination enabled OOB Bit 8 reset OOB Bit 18 reset
3.) 0x1B1: IA32_PACKAGE_THERM_STATUS: 88460800 PWL
4.) 0x64F: MSR_CORE_PERF_LIMIT_REASONS: 4000000 PL1L
A.) 0x1FC: MSR_POWER_CTL: 3C005D : C1E disable : RHO disable : EEO disable
5.) 0x771: IA32_HWP_CAPABILITIES (performance): 108252E : high 46 : guaranteed 37 : efficient 8 : lowest 1
6.) 0x774: IA32_HWP_REQUEST:    CPU 0-5 :
    raw: 00002E2E : 00002E2E : 00002E2E : 00002E2E : 00002E2E : 00002E2E :
    min:       46 :       46 :       46 :       46 :       46 :       46 :
    max:       46 :       46 :       46 :       46 :       46 :       46 :
    des:        0 :        0 :        0 :        0 :        0 :        0 :
    epp:        0 :        0 :        0 :        0 :        0 :        0 :
    act:        0 :        0 :        0 :        0 :        0 :        0 :
7.) 0x777: IA32_HWP_STATUS: 4 : high 4 : guaranteed 0 : efficient 0 : lowest 0

... Doug

author
#1 lore raw

Hi Len,

When I look at the dates for patches,
I do not think that

tools/power/x86/x86_energy_perf_policy/x86_energy_perf_policy.c

even knows about intel_cpufreq (intel_pstate in passive mode). Anyway this tool is now banned from my computers.

I'll re-phrase my previous e-mail, in a much more blunt way:

tools/power/x86/x86_energy_perf_policy/x86_energy_perf_policy.c

should be fixed or removed.

... Doug

Quoted 128 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Doug Smythies
Sent: June 16, 2020 8:15 AM
To: 'Len Brown'
Cc: 'Srinivas Pandruvada'; 'Linux PM'
Subject: x86_energy_perf_policy

Hi Len,

I have been using x86_energy_perf_policy to manage EPB while I do some tests. I noticed it says this:

"HWP can be enabled using '--hwp-enable'"

So, I thought great, I can now in one boot do:

intel_pstate, no-HWP, performance/powersave.
intel_cpufreq, no-HWP, all governors

run: x86_energy_perf_policy --hwp-enable

then repeat the above, but only passive, if trying some of Rafael's RFC stuff, which is NOT part of this e-mail.

However, enabling HWP via this method leaves my system in an unusable state.

Please confirm or deny if I am supposed to be able to enable HWP via x86_energy_perf_policy?

No need to read on:

Examples (kernel 5.8=rc1):

Note: I wrote my own MSR decoder. I know I could use turbostat, and do have it running virtually 100% of the time. However, writing my own decoder forced me to learn more.

Fresh boot (no=hwp forced):

root@s18:/home/doug# /home/doug/c/msr-decoder
8.) 0x198: IA32_PERF_STATUS     : CPU 0-5 :   8 :   8 :   8 :   8 :   8 :   8 :
B.) 0x770: IA32_PM_ENABLE: 0 : HWP disable
9.) 0x199: IA32_PERF_CTL        : CPU 0-5 :   9 :   8 :   8 :   8 :  19 :  26 :
C.) 0x1B0: IA32_ENERGY_PERF_BIAS: CPU 0-5 :   6 :   6 :   6 :   6 :   6 :   6 :
1.) 0x19C: IA32_THERM_STATUS: 88480000
2.) 0x1AA: MSR_MISC_PWR_MGMT: 401CC0 EIST enabled Coordination enabled OOB Bit 8 reset OOB Bit 18
reset
3.) 0x1B1: IA32_PACKAGE_THERM_STATUS: 88470000
4.) 0x64F: MSR_CORE_PERF_LIMIT_REASONS: 0
A.) 0x1FC: MSR_POWER_CTL: 3C005D : C1E disable : RHO disable : EEO disable

Now,

# x86_energy_perf_policy --hwp-enable
root@s18:/home/doug# /home/doug/c/msr-decoder
8.) 0x198: IA32_PERF_STATUS     : CPU 0-5 :   8 :   8 :   8 :   8 :   8 :   8 :
B.) 0x770: IA32_PM_ENABLE: 1 : HWP enable
1.) 0x19C: IA32_THERM_STATUS: 88480000
2.) 0x1AA: MSR_MISC_PWR_MGMT: 401CC0 EIST enabled Coordination enabled OOB Bit 8 reset OOB Bit 18
reset
3.) 0x1B1: IA32_PACKAGE_THERM_STATUS: 88460000
4.) 0x64F: MSR_CORE_PERF_LIMIT_REASONS: 0
A.) 0x1FC: MSR_POWER_CTL: 3C005D : C1E disable : RHO disable : EEO disable
5.) 0x771: IA32_HWP_CAPABILITIES (performance): 108252E : high 46 : guaranteed 37 : efficient 8 :
lowest 1
6.) 0x774: IA32_HWP_REQUEST:    CPU 0-5 :
    raw: 8000FF01 : 8000FF01 : 8000FF01 : 8000FF01 : 8000FF01 : 8000FF01 :
    min:        1 :        1 :        1 :        1 :        1 :        1 :
    max:      255 :      255 :      255 :      255 :      255 :      255 :
    des:        0 :        0 :        0 :        0 :        0 :        0 :
    epp:      128 :      128 :      128 :      128 :      128 :      128 :
    act:        0 :        0 :        0 :        0 :        0 :        0 :
7.) 0x777: IA32_HWP_STATUS: 0 : high 0 : guaranteed 0 : efficient 0 : lowest 0

Notice that EPP is 128, and I can not achieve full CPU speed for a workflow that should. Ok. So maybe I just need to toggle between performance and powersave and back. No difference. I tried many variations. There are many situations where the misbehaves.

If I boot with HWP enabled, I can observe EPP (and min and max) behave as expected and as a function of the governor. The system seems to behave.

intel_pstate / powersave:

root@s18:/home/doug# /home/doug/c/msr-decoder
8.) 0x198: IA32_PERF_STATUS     : CPU 0-5 :   8 :   8 :   8 :   8 :   8 :   8 :
B.) 0x770: IA32_PM_ENABLE: 1 : HWP enable
1.) 0x19C: IA32_THERM_STATUS: 88480800 PWL
2.) 0x1AA: MSR_MISC_PWR_MGMT: 401CC0 EIST enabled Coordination enabled OOB Bit 8 reset OOB Bit 18
reset
3.) 0x1B1: IA32_PACKAGE_THERM_STATUS: 88470800 PWL
4.) 0x64F: MSR_CORE_PERF_LIMIT_REASONS: 4000000 PL1L
A.) 0x1FC: MSR_POWER_CTL: 3C005D : C1E disable : RHO disable : EEO disable
5.) 0x771: IA32_HWP_CAPABILITIES (performance): 108252E : high 46 : guaranteed 37 : efficient 8 :
lowest 1
6.) 0x774: IA32_HWP_REQUEST:    CPU 0-5 :
    raw: 80002E08 : 80002E08 : 80002E08 : 80002E08 : 80002E08 : 80002E08 :
    min:        8 :        8 :        8 :        8 :        8 :        8 :
    max:       46 :       46 :       46 :       46 :       46 :       46 :
    des:        0 :        0 :        0 :        0 :        0 :        0 :
    epp:      128 :      128 :      128 :      128 :      128 :      128 :
    act:        0 :        0 :        0 :        0 :        0 :        0 :
7.) 0x777: IA32_HWP_STATUS: 4 : high 4 : guaranteed 0 : efficient 0 : lowest 0

intel_pstate / performance:

root@s18:/home/doug# /home/doug/c/msr-decoder
8.) 0x198: IA32_PERF_STATUS     : CPU 0-5 :  46 :  46 :  46 :  46 :  46 :  46 :
B.) 0x770: IA32_PM_ENABLE: 1 : HWP enable
1.) 0x19C: IA32_THERM_STATUS: 88470800 PWL
2.) 0x1AA: MSR_MISC_PWR_MGMT: 401CC0 EIST enabled Coordination enabled OOB Bit 8 reset OOB Bit 18
reset
3.) 0x1B1: IA32_PACKAGE_THERM_STATUS: 88460800 PWL
4.) 0x64F: MSR_CORE_PERF_LIMIT_REASONS: 4000000 PL1L
A.) 0x1FC: MSR_POWER_CTL: 3C005D : C1E disable : RHO disable : EEO disable
5.) 0x771: IA32_HWP_CAPABILITIES (performance): 108252E : high 46 : guaranteed 37 : efficient 8 :
lowest 1
6.) 0x774: IA32_HWP_REQUEST:    CPU 0-5 :
    raw: 00002E2E : 00002E2E : 00002E2E : 00002E2E : 00002E2E : 00002E2E :
    min:       46 :       46 :       46 :       46 :       46 :       46 :
    max:       46 :       46 :       46 :       46 :       46 :       46 :
    des:        0 :        0 :        0 :        0 :        0 :        0 :
    epp:        0 :        0 :        0 :        0 :        0 :        0 :
    act:        0 :        0 :        0 :        0 :        0 :        0 :
7.) 0x777: IA32_HWP_STATUS: 4 : high 4 : guaranteed 0 : efficient 0 : lowest 0

... Doug

#2 lore raw

Hi Doug,
the purpose of that capability was to have the ability to enable HWP on a legacy kernel that didn't know how to.

i agree, in this day, it is misleading.

-Len

Quoted 149 lines · Doug Smythies

Hi Len,

When I look at the dates for patches,
I do not think that

tools/power/x86/x86_energy_perf_policy/x86_energy_perf_policy.c

even knows about intel_cpufreq (intel_pstate in passive mode). Anyway this tool is now banned from my computers.

I'll re-phrase my previous e-mail, in a much more blunt way:

tools/power/x86/x86_energy_perf_policy/x86_energy_perf_policy.c

should be fixed or removed.

... Doug

Quoted 128 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Doug Smythies
Sent: June 16, 2020 8:15 AM
To: 'Len Brown'
Cc: 'Srinivas Pandruvada'; 'Linux PM'
Subject: x86_energy_perf_policy

Hi Len,

I have been using x86_energy_perf_policy to manage EPB while I do some tests. I noticed it says this:

"HWP can be enabled using '--hwp-enable'"

So, I thought great, I can now in one boot do:

intel_pstate, no-HWP, performance/powersave.
intel_cpufreq, no-HWP, all governors

run: x86_energy_perf_policy --hwp-enable

then repeat the above, but only passive, if trying some of Rafael's RFC stuff, which is NOT part of this e-mail.

However, enabling HWP via this method leaves my system in an unusable state.

Please confirm or deny if I am supposed to be able to enable HWP via x86_energy_perf_policy?

No need to read on:

Examples (kernel 5.8=rc1):

Note: I wrote my own MSR decoder. I know I could use turbostat, and do have it running virtually 100% of the time. However, writing my own decoder forced me to learn more.

Fresh boot (no=hwp forced):

root@s18:/home/doug# /home/doug/c/msr-decoder
8.) 0x198: IA32_PERF_STATUS     : CPU 0-5 :   8 :   8 :   8 :   8 :   8 :   8 :
B.) 0x770: IA32_PM_ENABLE: 0 : HWP disable
9.) 0x199: IA32_PERF_CTL        : CPU 0-5 :   9 :   8 :   8 :   8 :  19 :  26 :
C.) 0x1B0: IA32_ENERGY_PERF_BIAS: CPU 0-5 :   6 :   6 :   6 :   6 :   6 :   6 :
1.) 0x19C: IA32_THERM_STATUS: 88480000
2.) 0x1AA: MSR_MISC_PWR_MGMT: 401CC0 EIST enabled Coordination enabled OOB Bit 8 reset OOB Bit 18
reset
3.) 0x1B1: IA32_PACKAGE_THERM_STATUS: 88470000
4.) 0x64F: MSR_CORE_PERF_LIMIT_REASONS: 0
A.) 0x1FC: MSR_POWER_CTL: 3C005D : C1E disable : RHO disable : EEO disable

Now,

# x86_energy_perf_policy --hwp-enable
root@s18:/home/doug# /home/doug/c/msr-decoder
8.) 0x198: IA32_PERF_STATUS     : CPU 0-5 :   8 :   8 :   8 :   8 :   8 :   8 :
B.) 0x770: IA32_PM_ENABLE: 1 : HWP enable
1.) 0x19C: IA32_THERM_STATUS: 88480000
2.) 0x1AA: MSR_MISC_PWR_MGMT: 401CC0 EIST enabled Coordination enabled OOB Bit 8 reset OOB Bit 18
reset
3.) 0x1B1: IA32_PACKAGE_THERM_STATUS: 88460000
4.) 0x64F: MSR_CORE_PERF_LIMIT_REASONS: 0
A.) 0x1FC: MSR_POWER_CTL: 3C005D : C1E disable : RHO disable : EEO disable
5.) 0x771: IA32_HWP_CAPABILITIES (performance): 108252E : high 46 : guaranteed 37 : efficient 8 :
lowest 1
6.) 0x774: IA32_HWP_REQUEST:    CPU 0-5 :
    raw: 8000FF01 : 8000FF01 : 8000FF01 : 8000FF01 : 8000FF01 : 8000FF01 :
    min:        1 :        1 :        1 :        1 :        1 :        1 :
    max:      255 :      255 :      255 :      255 :      255 :      255 :
    des:        0 :        0 :        0 :        0 :        0 :        0 :
    epp:      128 :      128 :      128 :      128 :      128 :      128 :
    act:        0 :        0 :        0 :        0 :        0 :        0 :
7.) 0x777: IA32_HWP_STATUS: 0 : high 0 : guaranteed 0 : efficient 0 : lowest 0

Notice that EPP is 128, and I can not achieve full CPU speed for a workflow that should. Ok. So maybe I just need to toggle between performance and powersave and back. No difference. I tried many variations. There are many situations where the misbehaves.

If I boot with HWP enabled, I can observe EPP (and min and max) behave as expected and as a function of the governor. The system seems to behave.

intel_pstate / powersave:

root@s18:/home/doug# /home/doug/c/msr-decoder
8.) 0x198: IA32_PERF_STATUS     : CPU 0-5 :   8 :   8 :   8 :   8 :   8 :   8 :
B.) 0x770: IA32_PM_ENABLE: 1 : HWP enable
1.) 0x19C: IA32_THERM_STATUS: 88480800 PWL
2.) 0x1AA: MSR_MISC_PWR_MGMT: 401CC0 EIST enabled Coordination enabled OOB Bit 8 reset OOB Bit 18
reset
3.) 0x1B1: IA32_PACKAGE_THERM_STATUS: 88470800 PWL
4.) 0x64F: MSR_CORE_PERF_LIMIT_REASONS: 4000000 PL1L
A.) 0x1FC: MSR_POWER_CTL: 3C005D : C1E disable : RHO disable : EEO disable
5.) 0x771: IA32_HWP_CAPABILITIES (performance): 108252E : high 46 : guaranteed 37 : efficient 8 :
lowest 1
6.) 0x774: IA32_HWP_REQUEST:    CPU 0-5 :
    raw: 80002E08 : 80002E08 : 80002E08 : 80002E08 : 80002E08 : 80002E08 :
    min:        8 :        8 :        8 :        8 :        8 :        8 :
    max:       46 :       46 :       46 :       46 :       46 :       46 :
    des:        0 :        0 :        0 :        0 :        0 :        0 :
    epp:      128 :      128 :      128 :      128 :      128 :      128 :
    act:        0 :        0 :        0 :        0 :        0 :        0 :
7.) 0x777: IA32_HWP_STATUS: 4 : high 4 : guaranteed 0 : efficient 0 : lowest 0

intel_pstate / performance:

root@s18:/home/doug# /home/doug/c/msr-decoder
8.) 0x198: IA32_PERF_STATUS     : CPU 0-5 :  46 :  46 :  46 :  46 :  46 :  46 :
B.) 0x770: IA32_PM_ENABLE: 1 : HWP enable
1.) 0x19C: IA32_THERM_STATUS: 88470800 PWL
2.) 0x1AA: MSR_MISC_PWR_MGMT: 401CC0 EIST enabled Coordination enabled OOB Bit 8 reset OOB Bit 18
reset
3.) 0x1B1: IA32_PACKAGE_THERM_STATUS: 88460800 PWL
4.) 0x64F: MSR_CORE_PERF_LIMIT_REASONS: 4000000 PL1L
A.) 0x1FC: MSR_POWER_CTL: 3C005D : C1E disable : RHO disable : EEO disable
5.) 0x771: IA32_HWP_CAPABILITIES (performance): 108252E : high 46 : guaranteed 37 : efficient 8 :
lowest 1
6.) 0x774: IA32_HWP_REQUEST:    CPU 0-5 :
    raw: 00002E2E : 00002E2E : 00002E2E : 00002E2E : 00002E2E : 00002E2E :
    min:       46 :       46 :       46 :       46 :       46 :       46 :
    max:       46 :       46 :       46 :       46 :       46 :       46 :
    des:        0 :        0 :        0 :        0 :        0 :        0 :
    epp:        0 :        0 :        0 :        0 :        0 :        0 :
    act:        0 :        0 :        0 :        0 :        0 :        0 :
7.) 0x777: IA32_HWP_STATUS: 4 : high 4 : guaranteed 0 : efficient 0 : lowest 0

... Doug

Signature · 1 line
Len Brown, Intel Open Source Technology Center