3.10-rcX: cpu governor ondemand doesn't scale well after s2ram
At a stable 32bit Gentoo Linux with a i5 processor (2.6 GHZ max, 3.1 GHz boost) I've to change the governor after a s2ram all 4 cores from "ondemand" to "performance" and back to "ondemand", otherwise cpu1 till cpu3 would stay after s2ram at 2.6 GHz (whilst cpu0 is at the expected 800 MHz if no load is at the system) - FWIW I do set ignore_nice to "1".
3.9.6 is ok, the bug was introduces somewhere in 3.10-rcX.
Signature · 3 lines
MfG/Sincerely Toralf Förster pgp finger print: 7B1A 07F4 EC82 0F90 D4C2 8936 872A E508 7DB6 9DA3
Quoted 8 lines · Toralf Förster
At a stable 32bit Gentoo Linux with a i5 processor (2.6 GHZ max, 3.1 GHz boost) I've to change the governor after a s2ram all 4 cores from "ondemand" to "performance" and back to "ondemand", otherwise cpu1 till cpu3 would stay after s2ram at 2.6 GHz (whilst cpu0 is at the expected 800 MHz if no load is at the system) - FWIW I do set
ignore_niceto "1".3.9.6 is ok, the bug was introduces somewhere in 3.10-rcX.
FWIW the following commands circumvents that issue :
# for i in $(seq 1 3); do echo performance > /sys/devices/system/cpu/cpu$i/cpufreq/scaling_governor; done # for i in $(seq 1 3); do echo ondemand > /sys/devices/system/cpu/cpu$i/cpufreq/scaling_governor; done
cpu0 is unaffected
Signature · 3 lines
MfG/Sincerely Toralf Förster pgp finger print: 7B1A 07F4 EC82 0F90 D4C2 8936 872A E508 7DB6 9DA3
Quoted 7 lines · Toralf Förster
At a stable 32bit Gentoo Linux with a i5 processor (2.6 GHZ max, 3.1 GHz boost) I've to change the governor after a s2ram all 4 cores from "ondemand" to "performance" and back to "ondemand", otherwise cpu1 till cpu3 would stay after s2ram at 2.6 GHz (whilst cpu0 is at the expected 800 MHz if no load is at the system) - FWIW I do set
ignore_niceto "1".3.9.6 is ok, the bug was introduces somewhere in 3.10-rcX.
Hmm. This looks like something unrelated to the previous reports.
How do you check the frequencies?
Does that happen every time? If so, any chance to bisect driver/cpufreq changes between 3.9 and now?
Does it work without ignore_nice?
Rafael
Signature · 2 lines
I speak only for myself. Rafael J. Wysocki, Intel Open Source Technology Center.
Quoted 1 line · Rafael J. Wysocki
Hmm. This looks like something unrelated to the previous reports.
yes
Quoted 1 line
How do you check the frequencies?
much more significant is the temperature :
$> cat /sys/devices/system/cpu/cpu\?/thermal_throttle/package_power_limit_count /proc/acpi/ibm/fan /sys/class/hwmon/hwmon\?/device/temp\?_input'
- it is around 49° if just 4 BOINC childs are running at nice level -19, but it is above 65°C if the issue exists. And in hat case I just do run
$> for g in performance ondemand; do for i in 0 1 2 3; do echo $g > /sys/devices/system/cpu/cpu$i/cpufreq/scaling_governor; done; done; echo 1 > /sys/devices/system/cpu/cpufreq/ondemand/ignore_nice
and the system is back to normal.
Quoted 2 lines
Does that happen every time? If so, any chance to bisect driver/cpufreq changes between 3.9 and now?
first bisect attempts failed. I realized that the issue only occurs if the power is plugged in. If the system is running on battery the issue doesn't occurred (I checked it more than once)
Quoted 1 line
Does it work without
ignore_nice?
Seems not to be related to that.
Signature · 3 lines
MfG/Sincerely Toralf Förster pgp finger print: 7B1A 07F4 EC82 0F90 D4C2 8936 872A E508 7DB6 9DA3
Quoted 17 lines · Toralf Förster
Quoted 1 line · Rafael J. Wysocki
Hmm. This looks like something unrelated to the previous reports.
yes
Quoted 1 line
How do you check the frequencies?
much more significant is the temperature :
$> cat /sys/devices/system/cpu/cpu\?/thermal_throttle/package_power_limit_count
/proc/acpi/ibm/fan/sys/class/hwmon/hwmon\?/device/temp\?_input'- it is around 49° if just 4 BOINC childs are running at nice level -19, but it is above 65°C if the issue exists. And in hat case I just do run
$> for g in performance ondemand; do for i in 0 1 2 3; do echo $g >
/sys/devices/system/cpu/cpu$i/cpufreq/scaling_governor; done; done; echo 1 >/sys/devices/system/cpu/cpufreq/ondemand/ignore_niceand the system is back to normal.
Quoted 7 lines
Quoted 2 lines
Does that happen every time? If so, any chance to bisect driver/cpufreq changes between 3.9 and now?
first bisect attempts failed. I realized that the issue only occurs if the power is plugged in. If the system is running on battery the issue doesn't occurred (I checked it more than once)
Quoted 1 line
Does it work without
ignore_nice?Seems not to be related to that.
Well, I'm not aware of any recent changes in cpufreq that would make things work differently depending on whether or not AC power is used.
Thanks,
Rafael
Signature · 2 lines
I speak only for myself. Rafael J. Wysocki, Intel Open Source Technology Center.
Viresh Kumar
maintainerQuoted 2 lines · Rafael J. Wysocki
Well, I'm not aware of any recent changes in cpufreq that would make things work differently depending on whether or not AC power is used.
Toralf,
It would be helpful if you can do git bisect and let us know which patch created this problem. Otherwise its very difficult to trace it.
Quoted 9 lines · Viresh Kumar
Quoted 2 lines · Rafael J. Wysocki
Well, I'm not aware of any recent changes in cpufreq that would make things work differently depending on whether or not AC power is used.
Toralf,
It would be helpful if you can do git bisect and let us know which patch created this problem. Otherwise its very difficult to trace it.
Yep, but 14 commits to bisect and about 20-30 min for each - will see, when I find a time slot.
Just FWIW thie following is now a good indicator for the issue - that's the picture after wakeup, where I'd expect always 800 MHz:
$ sudo cat /sys/devices/system/cpu/cpu?/cpufreq/cpuinfo_cur_freq 800000 2601000 2601000 2601000
Signature · 3 lines
MfG/Sincerely Toralf Förster pgp finger print: 7B1A 07F4 EC82 0F90 D4C2 8936 872A E508 7DB6 9DA3
Viresh Kumar
maintainerQuoted 2 lines · Toralf Förster
Yep, but 14 commits to bisect and about 20-30 min for each - will see, when I find a time slot.
Hmm..
Quoted 8 lines
Just FWIW thie following is now a good indicator for the issue - that's the picture after wakeup, where I'd expect always 800 MHz:
$ sudo cat /sys/devices/system/cpu/cpu?/cpufreq/cpuinfo_cur_freq 800000 2601000 2601000 2601000
Can you give dumps of complete cpufreq directories for all cpus before and after s2r. That may give us some hint about the problem.
Quoted 2 lines · Viresh Kumar
Can you give dumps of complete cpufreq directories for all cpus before and after s2r. That may give us some hint about the problem.
here it is:
$ sudo su - n22 ~ # cat /sys/devices/system/cpu/cpu?/cpufreq/scaling_cur_freq 800000 800000 800000 800000 n22 ~ # cat /sys/devices/system/cpu/cpu?/cpufreq/cpuinfo_cur_freq 800000 800000 800000 800000 n22 ~ # find /sys/devices/system/cpu -type f | xargs tail -v > /tmp/before 2>/dev/null n22 ~ # Fn4 * Bringing down interface ppp0 * Running predown ... * Stopping pppd on ppp0 [ ok ] * Bringing up interface ppp0 * Starting pppd in ppp0 ... [ ok ] * received address 92.224.127.7 * WARNING: net.ppp0 has started, but is inactive n22 ~ # cat /sys/devices/system/cpu/cpu?/cpufreq/scaling_cur_freq 800000 800000 800000 800000 n22 ~ # cat /sys/devices/system/cpu/cpu?/cpufreq/cpuinfo_cur_freq 800000 2601000 2601000 2601000 n22 ~ # find /sys/devices/system/cpu -type f | xargs tail -v > /tmp/after 2>/dev/null n22 ~ # for g in performance ondemand; do for i in 1 2 3; do echo $g > /sys/devices/system/cpu/cpu$i/cpufreq/scaling_governor; done; done; echo 1 > /sys/devices/system/cpu/cpufreq/ondemand/ignore_nice n22 ~ # cat /sys/devices/system/cpu/cpu?/cpufreq/scaling_cur_freq800000 800000 800000 800000 n22 ~ # cat /sys/devices/system/cpu/cpu?/cpufreq/cpuinfo_cur_freq800000 800000 800000 800000 n22 ~ # chmod a+r /tmp/before /tmp/after
Signature · 3 lines
MfG/Sincerely Toralf Förster pgp finger print: 7B1A 07F4 EC82 0F90 D4C2 8936 872A E508 7DB6 9DA3
==> /sys/devices/system/cpu/cpu0/cpufreq/bios_limit <== 2601000
==> /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor <== ondemand
==> /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_transition_latency <== 10000
==> /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies <== 2601000 2600000 2400000 2200000 2000000 1800000 1600000 1400000 1200000 1000000 800000
==> /sys/devices/system/cpu/cpu0/cpufreq/scaling_driver <== acpi-cpufreq
==> /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_cur_freq <== 800000
==> /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors <== ondemand performance
==> /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq <== 800000
==> /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_max_freq <== 2601000
==> /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_min_freq <== 800000
==> /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq <== 2601000
==> /sys/devices/system/cpu/cpu0/cpufreq/affected_cpus <== 0
==> /sys/devices/system/cpu/cpu0/cpufreq/scaling_min_freq <== 800000
==> /sys/devices/system/cpu/cpu0/cpufreq/related_cpus <== 0
==> /sys/devices/system/cpu/cpu0/cpufreq/scaling_setspeed <== <unsupported>
==> /sys/devices/system/cpu/cpu0/cpuidle/state0/desc <== CPUIDLE CORE POLL IDLE
==> /sys/devices/system/cpu/cpu0/cpuidle/state0/name <== POLL
==> /sys/devices/system/cpu/cpu0/cpuidle/state0/time <== 19239
==> /sys/devices/system/cpu/cpu0/cpuidle/state0/disable <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state0/power <== 4294967295
==> /sys/devices/system/cpu/cpu0/cpuidle/state0/usage <== 55
==> /sys/devices/system/cpu/cpu0/cpuidle/state0/latency <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state1/desc <== MWAIT 0x00
==> /sys/devices/system/cpu/cpu0/cpuidle/state1/name <== C1-SNB
==> /sys/devices/system/cpu/cpu0/cpuidle/state1/time <== 924048
==> /sys/devices/system/cpu/cpu0/cpuidle/state1/disable <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state1/power <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state1/usage <== 7370
==> /sys/devices/system/cpu/cpu0/cpuidle/state1/latency <== 2
==> /sys/devices/system/cpu/cpu0/cpuidle/state2/desc <== MWAIT 0x01
==> /sys/devices/system/cpu/cpu0/cpuidle/state2/name <== C1E-SNB
==> /sys/devices/system/cpu/cpu0/cpuidle/state2/time <== 8659159
==> /sys/devices/system/cpu/cpu0/cpuidle/state2/disable <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state2/power <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state2/usage <== 18164
==> /sys/devices/system/cpu/cpu0/cpuidle/state2/latency <== 10
==> /sys/devices/system/cpu/cpu0/cpuidle/state3/desc <== MWAIT 0x10
==> /sys/devices/system/cpu/cpu0/cpuidle/state3/name <== C3-SNB
==> /sys/devices/system/cpu/cpu0/cpuidle/state3/time <== 3258501
==> /sys/devices/system/cpu/cpu0/cpuidle/state3/disable <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state3/power <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state3/usage <== 4979
==> /sys/devices/system/cpu/cpu0/cpuidle/state3/latency <== 80
==> /sys/devices/system/cpu/cpu0/cpuidle/state4/desc <== MWAIT 0x20
==> /sys/devices/system/cpu/cpu0/cpuidle/state4/name <== C6-SNB
==> /sys/devices/system/cpu/cpu0/cpuidle/state4/time <== 229632
==> /sys/devices/system/cpu/cpu0/cpuidle/state4/disable <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state4/power <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state4/usage <== 185
==> /sys/devices/system/cpu/cpu0/cpuidle/state4/latency <== 104
==> /sys/devices/system/cpu/cpu0/cpuidle/state5/desc <== MWAIT 0x30
==> /sys/devices/system/cpu/cpu0/cpuidle/state5/name <== C7-SNB
==> /sys/devices/system/cpu/cpu0/cpuidle/state5/time <== 55353981
==> /sys/devices/system/cpu/cpu0/cpuidle/state5/disable <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state5/power <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state5/usage <== 34407
==> /sys/devices/system/cpu/cpu0/cpuidle/state5/latency <== 109
==> /sys/devices/system/cpu/cpu0/cache/index0/size <== 32K
==> /sys/devices/system/cpu/cpu0/cache/index0/type <== Data
==> /sys/devices/system/cpu/cpu0/cache/index0/level <== 1
==> /sys/devices/system/cpu/cpu0/cache/index0/number_of_sets <== 64
==> /sys/devices/system/cpu/cpu0/cache/index0/shared_cpu_map <== 03
==> /sys/devices/system/cpu/cpu0/cache/index0/shared_cpu_list <== 0-1
==> /sys/devices/system/cpu/cpu0/cache/index0/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu0/cache/index0/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu0/cache/index0/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu0/cache/index1/size <== 32K
==> /sys/devices/system/cpu/cpu0/cache/index1/type <== Instruction
==> /sys/devices/system/cpu/cpu0/cache/index1/level <== 1
==> /sys/devices/system/cpu/cpu0/cache/index1/number_of_sets <== 64
==> /sys/devices/system/cpu/cpu0/cache/index1/shared_cpu_map <== 03
==> /sys/devices/system/cpu/cpu0/cache/index1/shared_cpu_list <== 0-1
==> /sys/devices/system/cpu/cpu0/cache/index1/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu0/cache/index1/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu0/cache/index1/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu0/cache/index2/size <== 256K
==> /sys/devices/system/cpu/cpu0/cache/index2/type <== Unified
==> /sys/devices/system/cpu/cpu0/cache/index2/level <== 2
==> /sys/devices/system/cpu/cpu0/cache/index2/number_of_sets <== 512
==> /sys/devices/system/cpu/cpu0/cache/index2/shared_cpu_map <== 03
==> /sys/devices/system/cpu/cpu0/cache/index2/shared_cpu_list <== 0-1
==> /sys/devices/system/cpu/cpu0/cache/index2/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu0/cache/index2/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu0/cache/index2/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu0/cache/index3/size <== 3072K
==> /sys/devices/system/cpu/cpu0/cache/index3/type <== Unified
==> /sys/devices/system/cpu/cpu0/cache/index3/level <== 3
==> /sys/devices/system/cpu/cpu0/cache/index3/number_of_sets <== 4096
==> /sys/devices/system/cpu/cpu0/cache/index3/shared_cpu_map <== 0f
==> /sys/devices/system/cpu/cpu0/cache/index3/shared_cpu_list <== 0-3
==> /sys/devices/system/cpu/cpu0/cache/index3/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu0/cache/index3/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu0/cache/index3/ways_of_associativity <== 12
==> /sys/devices/system/cpu/cpu0/power/control <== auto
==> /sys/devices/system/cpu/cpu0/power/async <==
disabled
==> /sys/devices/system/cpu/cpu0/power/runtime_enabled <== disabled
==> /sys/devices/system/cpu/cpu0/power/runtime_active_kids <== 0
==> /sys/devices/system/cpu/cpu0/power/runtime_active_time <== 0
==> /sys/devices/system/cpu/cpu0/power/autosuspend_delay_ms <==
==> /sys/devices/system/cpu/cpu0/power/runtime_status <== unsupported
==> /sys/devices/system/cpu/cpu0/power/runtime_usage <== 0
==> /sys/devices/system/cpu/cpu0/power/runtime_suspended_time <== 0
==> /sys/devices/system/cpu/cpu0/uevent <== MODALIAS=x86cpu:vendor:0000:family:0006:model:002A:feature:,0000,0001,0002,0003,0004,0005,0006,0007,0008,0009,000B,000C,000D,000E,000F,0010,0011,0013,0015,0016,0017,0018,0019,001A,001B,001C,001D,001F,0034,003B,003D,0066,0068,006B,006C,006D,0072,0076,0078,007C,007D,0080,0081,0082,0083,0084,0085,0086,0087,0088,0089,008D,008E,008F,0091,0093,0094,0095,0097,0098,0099,009A,009B,009C,00C0,00E0,00E1,00E3,00E4,00E5,00E6,00E7,0100,0101,0102,0103,0104
==> /sys/devices/system/cpu/cpu0/topology/core_id <== 0
==> /sys/devices/system/cpu/cpu0/topology/thread_siblings <== 03
==> /sys/devices/system/cpu/cpu0/topology/core_siblings <== 0f
==> /sys/devices/system/cpu/cpu0/topology/core_siblings_list <== 0-3
==> /sys/devices/system/cpu/cpu0/topology/thread_siblings_list <== 0-1
==> /sys/devices/system/cpu/cpu0/topology/physical_package_id <== 0
==> /sys/devices/system/cpu/cpu0/thermal_throttle/core_power_limit_count <== 0
==> /sys/devices/system/cpu/cpu0/thermal_throttle/package_throttle_count <== 0
==> /sys/devices/system/cpu/cpu0/thermal_throttle/core_throttle_count <== 0
==> /sys/devices/system/cpu/cpu0/thermal_throttle/package_power_limit_count <== 0
==> /sys/devices/system/cpu/cpu1/cpufreq/bios_limit <== 2601000
==> /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor <== ondemand
==> /sys/devices/system/cpu/cpu1/cpufreq/cpuinfo_transition_latency <== 10000
==> /sys/devices/system/cpu/cpu1/cpufreq/scaling_available_frequencies <== 2601000 2600000 2400000 2200000 2000000 1800000 1600000 1400000 1200000 1000000 800000
==> /sys/devices/system/cpu/cpu1/cpufreq/scaling_driver <== acpi-cpufreq
==> /sys/devices/system/cpu/cpu1/cpufreq/cpuinfo_cur_freq <== 2601000
==> /sys/devices/system/cpu/cpu1/cpufreq/scaling_available_governors <== ondemand performance
==> /sys/devices/system/cpu/cpu1/cpufreq/scaling_cur_freq <== 800000
==> /sys/devices/system/cpu/cpu1/cpufreq/cpuinfo_max_freq <== 2601000
==> /sys/devices/system/cpu/cpu1/cpufreq/cpuinfo_min_freq <== 800000
==> /sys/devices/system/cpu/cpu1/cpufreq/scaling_max_freq <== 2601000
==> /sys/devices/system/cpu/cpu1/cpufreq/affected_cpus <== 1
==> /sys/devices/system/cpu/cpu1/cpufreq/scaling_min_freq <== 800000
==> /sys/devices/system/cpu/cpu1/cpufreq/related_cpus <== 1
==> /sys/devices/system/cpu/cpu1/cpufreq/scaling_setspeed <== <unsupported>
==> /sys/devices/system/cpu/cpu1/cpuidle/state0/desc <== CPUIDLE CORE POLL IDLE
==> /sys/devices/system/cpu/cpu1/cpuidle/state0/name <== POLL
==> /sys/devices/system/cpu/cpu1/cpuidle/state0/time <== 61773
==> /sys/devices/system/cpu/cpu1/cpuidle/state0/disable <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state0/power <== 4294967295
==> /sys/devices/system/cpu/cpu1/cpuidle/state0/usage <== 27
==> /sys/devices/system/cpu/cpu1/cpuidle/state0/latency <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state1/desc <== MWAIT 0x00
==> /sys/devices/system/cpu/cpu1/cpuidle/state1/name <== C1-SNB
==> /sys/devices/system/cpu/cpu1/cpuidle/state1/time <== 259855
==> /sys/devices/system/cpu/cpu1/cpuidle/state1/disable <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state1/power <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state1/usage <== 4464
==> /sys/devices/system/cpu/cpu1/cpuidle/state1/latency <== 2
==> /sys/devices/system/cpu/cpu1/cpuidle/state2/desc <== MWAIT 0x01
==> /sys/devices/system/cpu/cpu1/cpuidle/state2/name <== C1E-SNB
==> /sys/devices/system/cpu/cpu1/cpuidle/state2/time <== 7498088
==> /sys/devices/system/cpu/cpu1/cpuidle/state2/disable <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state2/power <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state2/usage <== 9108
==> /sys/devices/system/cpu/cpu1/cpuidle/state2/latency <== 10
==> /sys/devices/system/cpu/cpu1/cpuidle/state3/desc <== MWAIT 0x10
==> /sys/devices/system/cpu/cpu1/cpuidle/state3/name <== C3-SNB
==> /sys/devices/system/cpu/cpu1/cpuidle/state3/time <== 3915900
==> /sys/devices/system/cpu/cpu1/cpuidle/state3/disable <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state3/power <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state3/usage <== 2583
==> /sys/devices/system/cpu/cpu1/cpuidle/state3/latency <== 80
==> /sys/devices/system/cpu/cpu1/cpuidle/state4/desc <== MWAIT 0x20
==> /sys/devices/system/cpu/cpu1/cpuidle/state4/name <== C6-SNB
==> /sys/devices/system/cpu/cpu1/cpuidle/state4/time <== 163650
==> /sys/devices/system/cpu/cpu1/cpuidle/state4/disable <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state4/power <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state4/usage <== 73
==> /sys/devices/system/cpu/cpu1/cpuidle/state4/latency <== 104
==> /sys/devices/system/cpu/cpu1/cpuidle/state5/desc <== MWAIT 0x30
==> /sys/devices/system/cpu/cpu1/cpuidle/state5/name <== C7-SNB
==> /sys/devices/system/cpu/cpu1/cpuidle/state5/time <== 56007989
==> /sys/devices/system/cpu/cpu1/cpuidle/state5/disable <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state5/power <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state5/usage <== 10807
==> /sys/devices/system/cpu/cpu1/cpuidle/state5/latency <== 109
==> /sys/devices/system/cpu/cpu1/cache/index0/size <== 32K
==> /sys/devices/system/cpu/cpu1/cache/index0/type <== Data
==> /sys/devices/system/cpu/cpu1/cache/index0/level <== 1
==> /sys/devices/system/cpu/cpu1/cache/index0/number_of_sets <== 64
==> /sys/devices/system/cpu/cpu1/cache/index0/shared_cpu_map <== 03
==> /sys/devices/system/cpu/cpu1/cache/index0/shared_cpu_list <== 0-1
==> /sys/devices/system/cpu/cpu1/cache/index0/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu1/cache/index0/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu1/cache/index0/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu1/cache/index1/size <== 32K
==> /sys/devices/system/cpu/cpu1/cache/index1/type <== Instruction
==> /sys/devices/system/cpu/cpu1/cache/index1/level <== 1
==> /sys/devices/system/cpu/cpu1/cache/index1/number_of_sets <== 64
==> /sys/devices/system/cpu/cpu1/cache/index1/shared_cpu_map <== 03
==> /sys/devices/system/cpu/cpu1/cache/index1/shared_cpu_list <== 0-1
==> /sys/devices/system/cpu/cpu1/cache/index1/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu1/cache/index1/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu1/cache/index1/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu1/cache/index2/size <== 256K
==> /sys/devices/system/cpu/cpu1/cache/index2/type <== Unified
==> /sys/devices/system/cpu/cpu1/cache/index2/level <== 2
==> /sys/devices/system/cpu/cpu1/cache/index2/number_of_sets <== 512
==> /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map <== 03
==> /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_list <== 0-1
==> /sys/devices/system/cpu/cpu1/cache/index2/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu1/cache/index2/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu1/cache/index2/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu1/cache/index3/size <== 3072K
==> /sys/devices/system/cpu/cpu1/cache/index3/type <== Unified
==> /sys/devices/system/cpu/cpu1/cache/index3/level <== 3
==> /sys/devices/system/cpu/cpu1/cache/index3/number_of_sets <== 4096
==> /sys/devices/system/cpu/cpu1/cache/index3/shared_cpu_map <== 0f
==> /sys/devices/system/cpu/cpu1/cache/index3/shared_cpu_list <== 0-3
==> /sys/devices/system/cpu/cpu1/cache/index3/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu1/cache/index3/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu1/cache/index3/ways_of_associativity <== 12
==> /sys/devices/system/cpu/cpu1/power/control <== auto
==> /sys/devices/system/cpu/cpu1/power/async <==
disabled
==> /sys/devices/system/cpu/cpu1/power/runtime_enabled <== disabled
==> /sys/devices/system/cpu/cpu1/power/runtime_active_kids <== 0
==> /sys/devices/system/cpu/cpu1/power/runtime_active_time <== 0
==> /sys/devices/system/cpu/cpu1/power/autosuspend_delay_ms <==
==> /sys/devices/system/cpu/cpu1/power/runtime_status <== unsupported
==> /sys/devices/system/cpu/cpu1/power/runtime_usage <== 0
==> /sys/devices/system/cpu/cpu1/power/runtime_suspended_time <== 0
==> /sys/devices/system/cpu/cpu1/online <==
1
==> /sys/devices/system/cpu/cpu1/uevent <== MODALIAS=x86cpu:vendor:0000:family:0006:model:002A:feature:,0000,0001,0002,0003,0004,0005,0006,0007,0008,0009,000B,000C,000D,000E,000F,0010,0011,0013,0015,0016,0017,0018,0019,001A,001B,001C,001D,001F,0034,003B,003D,0066,0068,006B,006C,006D,0072,0076,0078,007C,007D,0080,0081,0082,0083,0084,0085,0086,0087,0088,0089,008D,008E,008F,0091,0093,0094,0095,0097,0098,0099,009A,009B,009C,00C0,00E0,00E1,00E3,00E4,00E5,00E6,00E7,0100,0101,0102,0103,0104
==> /sys/devices/system/cpu/cpu1/topology/core_id <== 0
==> /sys/devices/system/cpu/cpu1/topology/thread_siblings <== 03
==> /sys/devices/system/cpu/cpu1/topology/core_siblings <== 0f
==> /sys/devices/system/cpu/cpu1/topology/core_siblings_list <== 0-3
==> /sys/devices/system/cpu/cpu1/topology/thread_siblings_list <== 0-1
==> /sys/devices/system/cpu/cpu1/topology/physical_package_id <== 0
==> /sys/devices/system/cpu/cpu1/thermal_throttle/core_power_limit_count <== 0
==> /sys/devices/system/cpu/cpu1/thermal_throttle/package_throttle_count <== 0
==> /sys/devices/system/cpu/cpu1/thermal_throttle/core_throttle_count <== 0
==> /sys/devices/system/cpu/cpu1/thermal_throttle/package_power_limit_count <== 0
==> /sys/devices/system/cpu/cpu2/cpufreq/bios_limit <== 2601000
==> /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor <== ondemand
==> /sys/devices/system/cpu/cpu2/cpufreq/cpuinfo_transition_latency <== 10000
==> /sys/devices/system/cpu/cpu2/cpufreq/scaling_available_frequencies <== 2601000 2600000 2400000 2200000 2000000 1800000 1600000 1400000 1200000 1000000 800000
==> /sys/devices/system/cpu/cpu2/cpufreq/scaling_driver <== acpi-cpufreq
==> /sys/devices/system/cpu/cpu2/cpufreq/cpuinfo_cur_freq <== 2601000
==> /sys/devices/system/cpu/cpu2/cpufreq/scaling_available_governors <== ondemand performance
==> /sys/devices/system/cpu/cpu2/cpufreq/scaling_cur_freq <== 800000
==> /sys/devices/system/cpu/cpu2/cpufreq/cpuinfo_max_freq <== 2601000
==> /sys/devices/system/cpu/cpu2/cpufreq/cpuinfo_min_freq <== 800000
==> /sys/devices/system/cpu/cpu2/cpufreq/scaling_max_freq <== 2601000
==> /sys/devices/system/cpu/cpu2/cpufreq/affected_cpus <== 2
==> /sys/devices/system/cpu/cpu2/cpufreq/scaling_min_freq <== 800000
==> /sys/devices/system/cpu/cpu2/cpufreq/related_cpus <== 2
==> /sys/devices/system/cpu/cpu2/cpufreq/scaling_setspeed <== <unsupported>
==> /sys/devices/system/cpu/cpu2/cpuidle/state0/desc <== CPUIDLE CORE POLL IDLE
==> /sys/devices/system/cpu/cpu2/cpuidle/state0/name <== POLL
==> /sys/devices/system/cpu/cpu2/cpuidle/state0/time <== 15532
==> /sys/devices/system/cpu/cpu2/cpuidle/state0/disable <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state0/power <== 4294967295
==> /sys/devices/system/cpu/cpu2/cpuidle/state0/usage <== 36
==> /sys/devices/system/cpu/cpu2/cpuidle/state0/latency <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state1/desc <== MWAIT 0x00
==> /sys/devices/system/cpu/cpu2/cpuidle/state1/name <== C1-SNB
==> /sys/devices/system/cpu/cpu2/cpuidle/state1/time <== 425542
==> /sys/devices/system/cpu/cpu2/cpuidle/state1/disable <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state1/power <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state1/usage <== 7866
==> /sys/devices/system/cpu/cpu2/cpuidle/state1/latency <== 2
==> /sys/devices/system/cpu/cpu2/cpuidle/state2/desc <== MWAIT 0x01
==> /sys/devices/system/cpu/cpu2/cpuidle/state2/name <== C1E-SNB
==> /sys/devices/system/cpu/cpu2/cpuidle/state2/time <== 8230973
==> /sys/devices/system/cpu/cpu2/cpuidle/state2/disable <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state2/power <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state2/usage <== 8798
==> /sys/devices/system/cpu/cpu2/cpuidle/state2/latency <== 10
==> /sys/devices/system/cpu/cpu2/cpuidle/state3/desc <== MWAIT 0x10
==> /sys/devices/system/cpu/cpu2/cpuidle/state3/name <== C3-SNB
==> /sys/devices/system/cpu/cpu2/cpuidle/state3/time <== 4179929
==> /sys/devices/system/cpu/cpu2/cpuidle/state3/disable <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state3/power <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state3/usage <== 2322
==> /sys/devices/system/cpu/cpu2/cpuidle/state3/latency <== 80
==> /sys/devices/system/cpu/cpu2/cpuidle/state4/desc <== MWAIT 0x20
==> /sys/devices/system/cpu/cpu2/cpuidle/state4/name <== C6-SNB
==> /sys/devices/system/cpu/cpu2/cpuidle/state4/time <== 195865
==> /sys/devices/system/cpu/cpu2/cpuidle/state4/disable <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state4/power <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state4/usage <== 68
==> /sys/devices/system/cpu/cpu2/cpuidle/state4/latency <== 104
==> /sys/devices/system/cpu/cpu2/cpuidle/state5/desc <== MWAIT 0x30
==> /sys/devices/system/cpu/cpu2/cpuidle/state5/name <== C7-SNB
==> /sys/devices/system/cpu/cpu2/cpuidle/state5/time <== 56796175
==> /sys/devices/system/cpu/cpu2/cpuidle/state5/disable <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state5/power <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state5/usage <== 10190
==> /sys/devices/system/cpu/cpu2/cpuidle/state5/latency <== 109
==> /sys/devices/system/cpu/cpu2/cache/index0/size <== 32K
==> /sys/devices/system/cpu/cpu2/cache/index0/type <== Data
==> /sys/devices/system/cpu/cpu2/cache/index0/level <== 1
==> /sys/devices/system/cpu/cpu2/cache/index0/number_of_sets <== 64
==> /sys/devices/system/cpu/cpu2/cache/index0/shared_cpu_map <== 0c
==> /sys/devices/system/cpu/cpu2/cache/index0/shared_cpu_list <== 2-3
==> /sys/devices/system/cpu/cpu2/cache/index0/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu2/cache/index0/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu2/cache/index0/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu2/cache/index1/size <== 32K
==> /sys/devices/system/cpu/cpu2/cache/index1/type <== Instruction
==> /sys/devices/system/cpu/cpu2/cache/index1/level <== 1
==> /sys/devices/system/cpu/cpu2/cache/index1/number_of_sets <== 64
==> /sys/devices/system/cpu/cpu2/cache/index1/shared_cpu_map <== 0c
==> /sys/devices/system/cpu/cpu2/cache/index1/shared_cpu_list <== 2-3
==> /sys/devices/system/cpu/cpu2/cache/index1/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu2/cache/index1/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu2/cache/index1/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu2/cache/index2/size <== 256K
==> /sys/devices/system/cpu/cpu2/cache/index2/type <== Unified
==> /sys/devices/system/cpu/cpu2/cache/index2/level <== 2
==> /sys/devices/system/cpu/cpu2/cache/index2/number_of_sets <== 512
==> /sys/devices/system/cpu/cpu2/cache/index2/shared_cpu_map <== 0c
==> /sys/devices/system/cpu/cpu2/cache/index2/shared_cpu_list <== 2-3
==> /sys/devices/system/cpu/cpu2/cache/index2/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu2/cache/index2/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu2/cache/index2/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu2/cache/index3/size <== 3072K
==> /sys/devices/system/cpu/cpu2/cache/index3/type <== Unified
==> /sys/devices/system/cpu/cpu2/cache/index3/level <== 3
==> /sys/devices/system/cpu/cpu2/cache/index3/number_of_sets <== 4096
==> /sys/devices/system/cpu/cpu2/cache/index3/shared_cpu_map <== 0f
==> /sys/devices/system/cpu/cpu2/cache/index3/shared_cpu_list <== 0-3
==> /sys/devices/system/cpu/cpu2/cache/index3/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu2/cache/index3/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu2/cache/index3/ways_of_associativity <== 12
==> /sys/devices/system/cpu/cpu2/power/control <== auto
==> /sys/devices/system/cpu/cpu2/power/async <==
disabled
==> /sys/devices/system/cpu/cpu2/power/runtime_enabled <== disabled
==> /sys/devices/system/cpu/cpu2/power/runtime_active_kids <== 0
==> /sys/devices/system/cpu/cpu2/power/runtime_active_time <== 0
==> /sys/devices/system/cpu/cpu2/power/autosuspend_delay_ms <==
==> /sys/devices/system/cpu/cpu2/power/runtime_status <== unsupported
==> /sys/devices/system/cpu/cpu2/power/runtime_usage <== 0
==> /sys/devices/system/cpu/cpu2/power/runtime_suspended_time <== 0
==> /sys/devices/system/cpu/cpu2/online <==
1
==> /sys/devices/system/cpu/cpu2/uevent <== MODALIAS=x86cpu:vendor:0000:family:0006:model:002A:feature:,0000,0001,0002,0003,0004,0005,0006,0007,0008,0009,000B,000C,000D,000E,000F,0010,0011,0013,0015,0016,0017,0018,0019,001A,001B,001C,001D,001F,0034,003B,003D,0066,0068,006B,006C,006D,0072,0076,0078,007C,007D,0080,0081,0082,0083,0084,0085,0086,0087,0088,0089,008D,008E,008F,0091,0093,0094,0095,0097,0098,0099,009A,009B,009C,00C0,00E0,00E1,00E3,00E4,00E5,00E6,00E7,0100,0101,0102,0103,0104
==> /sys/devices/system/cpu/cpu2/topology/core_id <== 1
==> /sys/devices/system/cpu/cpu2/topology/thread_siblings <== 0c
==> /sys/devices/system/cpu/cpu2/topology/core_siblings <== 0f
==> /sys/devices/system/cpu/cpu2/topology/core_siblings_list <== 0-3
==> /sys/devices/system/cpu/cpu2/topology/thread_siblings_list <== 2-3
==> /sys/devices/system/cpu/cpu2/topology/physical_package_id <== 0
==> /sys/devices/system/cpu/cpu2/thermal_throttle/core_power_limit_count <== 0
==> /sys/devices/system/cpu/cpu2/thermal_throttle/package_throttle_count <== 0
==> /sys/devices/system/cpu/cpu2/thermal_throttle/core_throttle_count <== 0
==> /sys/devices/system/cpu/cpu2/thermal_throttle/package_power_limit_count <== 0
==> /sys/devices/system/cpu/cpu3/cpufreq/bios_limit <== 2601000
==> /sys/devices/system/cpu/cpu3/cpufreq/scaling_governor <== ondemand
==> /sys/devices/system/cpu/cpu3/cpufreq/cpuinfo_transition_latency <== 10000
==> /sys/devices/system/cpu/cpu3/cpufreq/scaling_available_frequencies <== 2601000 2600000 2400000 2200000 2000000 1800000 1600000 1400000 1200000 1000000 800000
==> /sys/devices/system/cpu/cpu3/cpufreq/scaling_driver <== acpi-cpufreq
==> /sys/devices/system/cpu/cpu3/cpufreq/cpuinfo_cur_freq <== 2601000
==> /sys/devices/system/cpu/cpu3/cpufreq/scaling_available_governors <== ondemand performance
==> /sys/devices/system/cpu/cpu3/cpufreq/scaling_cur_freq <== 800000
==> /sys/devices/system/cpu/cpu3/cpufreq/cpuinfo_max_freq <== 2601000
==> /sys/devices/system/cpu/cpu3/cpufreq/cpuinfo_min_freq <== 800000
==> /sys/devices/system/cpu/cpu3/cpufreq/scaling_max_freq <== 2601000
==> /sys/devices/system/cpu/cpu3/cpufreq/affected_cpus <== 3
==> /sys/devices/system/cpu/cpu3/cpufreq/scaling_min_freq <== 800000
==> /sys/devices/system/cpu/cpu3/cpufreq/related_cpus <== 3
==> /sys/devices/system/cpu/cpu3/cpufreq/scaling_setspeed <== <unsupported>
==> /sys/devices/system/cpu/cpu3/cpuidle/state0/desc <== CPUIDLE CORE POLL IDLE
==> /sys/devices/system/cpu/cpu3/cpuidle/state0/name <== POLL
==> /sys/devices/system/cpu/cpu3/cpuidle/state0/time <== 2323
==> /sys/devices/system/cpu/cpu3/cpuidle/state0/disable <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state0/power <== 4294967295
==> /sys/devices/system/cpu/cpu3/cpuidle/state0/usage <== 28
==> /sys/devices/system/cpu/cpu3/cpuidle/state0/latency <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state1/desc <== MWAIT 0x00
==> /sys/devices/system/cpu/cpu3/cpuidle/state1/name <== C1-SNB
==> /sys/devices/system/cpu/cpu3/cpuidle/state1/time <== 456731
==> /sys/devices/system/cpu/cpu3/cpuidle/state1/disable <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state1/power <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state1/usage <== 7560
==> /sys/devices/system/cpu/cpu3/cpuidle/state1/latency <== 2
==> /sys/devices/system/cpu/cpu3/cpuidle/state2/desc <== MWAIT 0x01
==> /sys/devices/system/cpu/cpu3/cpuidle/state2/name <== C1E-SNB
==> /sys/devices/system/cpu/cpu3/cpuidle/state2/time <== 5585596
==> /sys/devices/system/cpu/cpu3/cpuidle/state2/disable <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state2/power <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state2/usage <== 6373
==> /sys/devices/system/cpu/cpu3/cpuidle/state2/latency <== 10
==> /sys/devices/system/cpu/cpu3/cpuidle/state3/desc <== MWAIT 0x10
==> /sys/devices/system/cpu/cpu3/cpuidle/state3/name <== C3-SNB
==> /sys/devices/system/cpu/cpu3/cpuidle/state3/time <== 3177689
==> /sys/devices/system/cpu/cpu3/cpuidle/state3/disable <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state3/power <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state3/usage <== 1769
==> /sys/devices/system/cpu/cpu3/cpuidle/state3/latency <== 80
==> /sys/devices/system/cpu/cpu3/cpuidle/state4/desc <== MWAIT 0x20
==> /sys/devices/system/cpu/cpu3/cpuidle/state4/name <== C6-SNB
==> /sys/devices/system/cpu/cpu3/cpuidle/state4/time <== 73846
==> /sys/devices/system/cpu/cpu3/cpuidle/state4/disable <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state4/power <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state4/usage <== 38
==> /sys/devices/system/cpu/cpu3/cpuidle/state4/latency <== 104
==> /sys/devices/system/cpu/cpu3/cpuidle/state5/desc <== MWAIT 0x30
==> /sys/devices/system/cpu/cpu3/cpuidle/state5/name <== C7-SNB
==> /sys/devices/system/cpu/cpu3/cpuidle/state5/time <== 56910450
==> /sys/devices/system/cpu/cpu3/cpuidle/state5/disable <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state5/power <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state5/usage <== 9384
==> /sys/devices/system/cpu/cpu3/cpuidle/state5/latency <== 109
==> /sys/devices/system/cpu/cpu3/cache/index0/size <== 32K
==> /sys/devices/system/cpu/cpu3/cache/index0/type <== Data
==> /sys/devices/system/cpu/cpu3/cache/index0/level <== 1
==> /sys/devices/system/cpu/cpu3/cache/index0/number_of_sets <== 64
==> /sys/devices/system/cpu/cpu3/cache/index0/shared_cpu_map <== 0c
==> /sys/devices/system/cpu/cpu3/cache/index0/shared_cpu_list <== 2-3
==> /sys/devices/system/cpu/cpu3/cache/index0/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu3/cache/index0/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu3/cache/index0/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu3/cache/index1/size <== 32K
==> /sys/devices/system/cpu/cpu3/cache/index1/type <== Instruction
==> /sys/devices/system/cpu/cpu3/cache/index1/level <== 1
==> /sys/devices/system/cpu/cpu3/cache/index1/number_of_sets <== 64
==> /sys/devices/system/cpu/cpu3/cache/index1/shared_cpu_map <== 0c
==> /sys/devices/system/cpu/cpu3/cache/index1/shared_cpu_list <== 2-3
==> /sys/devices/system/cpu/cpu3/cache/index1/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu3/cache/index1/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu3/cache/index1/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu3/cache/index2/size <== 256K
==> /sys/devices/system/cpu/cpu3/cache/index2/type <== Unified
==> /sys/devices/system/cpu/cpu3/cache/index2/level <== 2
==> /sys/devices/system/cpu/cpu3/cache/index2/number_of_sets <== 512
==> /sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_map <== 0c
==> /sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_list <== 2-3
==> /sys/devices/system/cpu/cpu3/cache/index2/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu3/cache/index2/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu3/cache/index2/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu3/cache/index3/size <== 3072K
==> /sys/devices/system/cpu/cpu3/cache/index3/type <== Unified
==> /sys/devices/system/cpu/cpu3/cache/index3/level <== 3
==> /sys/devices/system/cpu/cpu3/cache/index3/number_of_sets <== 4096
==> /sys/devices/system/cpu/cpu3/cache/index3/shared_cpu_map <== 0f
==> /sys/devices/system/cpu/cpu3/cache/index3/shared_cpu_list <== 0-3
==> /sys/devices/system/cpu/cpu3/cache/index3/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu3/cache/index3/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu3/cache/index3/ways_of_associativity <== 12
==> /sys/devices/system/cpu/cpu3/power/control <== auto
==> /sys/devices/system/cpu/cpu3/power/async <==
disabled
==> /sys/devices/system/cpu/cpu3/power/runtime_enabled <== disabled
==> /sys/devices/system/cpu/cpu3/power/runtime_active_kids <== 0
==> /sys/devices/system/cpu/cpu3/power/runtime_active_time <== 0
==> /sys/devices/system/cpu/cpu3/power/autosuspend_delay_ms <==
==> /sys/devices/system/cpu/cpu3/power/runtime_status <== unsupported
==> /sys/devices/system/cpu/cpu3/power/runtime_usage <== 0
==> /sys/devices/system/cpu/cpu3/power/runtime_suspended_time <== 0
==> /sys/devices/system/cpu/cpu3/online <==
1
==> /sys/devices/system/cpu/cpu3/uevent <== MODALIAS=x86cpu:vendor:0000:family:0006:model:002A:feature:,0000,0001,0002,0003,0004,0005,0006,0007,0008,0009,000B,000C,000D,000E,000F,0010,0011,0013,0015,0016,0017,0018,0019,001A,001B,001C,001D,001F,0034,003B,003D,0066,0068,006B,006C,006D,0072,0076,0078,007C,007D,0080,0081,0082,0083,0084,0085,0086,0087,0088,0089,008D,008E,008F,0091,0093,0094,0095,0097,0098,0099,009A,009B,009C,00C0,00E0,00E1,00E3,00E4,00E5,00E6,00E7,0100,0101,0102,0103,0104
==> /sys/devices/system/cpu/cpu3/topology/core_id <== 1
==> /sys/devices/system/cpu/cpu3/topology/thread_siblings <== 0c
==> /sys/devices/system/cpu/cpu3/topology/core_siblings <== 0f
==> /sys/devices/system/cpu/cpu3/topology/core_siblings_list <== 0-3
==> /sys/devices/system/cpu/cpu3/topology/thread_siblings_list <== 2-3
==> /sys/devices/system/cpu/cpu3/topology/physical_package_id <== 0
==> /sys/devices/system/cpu/cpu3/thermal_throttle/core_power_limit_count <== 0
==> /sys/devices/system/cpu/cpu3/thermal_throttle/package_throttle_count <== 0
==> /sys/devices/system/cpu/cpu3/thermal_throttle/core_throttle_count <== 0
==> /sys/devices/system/cpu/cpu3/thermal_throttle/package_power_limit_count <== 0
==> /sys/devices/system/cpu/cpufreq/boost <==
1
==> /sys/devices/system/cpu/cpufreq/ondemand/powersave_bias <== 0
==> /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate <== 10000
==> /sys/devices/system/cpu/cpufreq/ondemand/io_is_busy <== 1
==> /sys/devices/system/cpu/cpufreq/ondemand/up_threshold <== 95
==> /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate_min <== 10000
==> /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor <== 1
==> /sys/devices/system/cpu/cpufreq/ondemand/ignore_nice <== 1
==> /sys/devices/system/cpu/cpuidle/current_driver <== intel_idle
==> /sys/devices/system/cpu/cpuidle/current_governor_ro <== menu
==> /sys/devices/system/cpu/power/control <==
auto
==> /sys/devices/system/cpu/power/async <==
disabled
==> /sys/devices/system/cpu/power/runtime_enabled <== disabled
==> /sys/devices/system/cpu/power/runtime_active_kids <== 0
==> /sys/devices/system/cpu/power/runtime_active_time <== 0
==> /sys/devices/system/cpu/power/autosuspend_delay_ms <==
==> /sys/devices/system/cpu/power/runtime_status <== unsupported
==> /sys/devices/system/cpu/power/runtime_usage <== 0
==> /sys/devices/system/cpu/power/runtime_suspended_time <== 0
==> /sys/devices/system/cpu/modalias <== x86cpu:vendor:0000:family:0006:model:002A:feature:,0000,0001,0002,0003,0004,0005,0006,0007,0008,0009,000B,000C,000D,000E,000F,0010,0011,0013,0015,0016,0017,0018,0019,001A,001B,001C,001D,001F,0034,003B,003D,0066,0068,006B,006C,006D,0072,0076,0078,007C,007D,0080,0081,0082,0083,0084,0085,0086,0087,0088,0089,008D,008E,008F,0091,0093,0094,0095,0097,0098,0099,009A,009B,009C,00C0,00E0,00E1,00E3,00E4,00E5,00E6,00E7,0100,0101,0102,0103,0104
==> /sys/devices/system/cpu/kernel_max <==
7
==> /sys/devices/system/cpu/possible <==
0-7
==> /sys/devices/system/cpu/online <==
0-3
==> /sys/devices/system/cpu/offline <==
4-7
==> /sys/devices/system/cpu/uevent <==
==> /sys/devices/system/cpu/present <==
0-3
==> /sys/devices/system/cpu/cpu0/cpufreq/bios_limit <== 2601000
==> /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor <== ondemand
==> /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_transition_latency <== 10000
==> /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies <== 2601000 2600000 2400000 2200000 2000000 1800000 1600000 1400000 1200000 1000000 800000
==> /sys/devices/system/cpu/cpu0/cpufreq/scaling_driver <== acpi-cpufreq
==> /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_cur_freq <== 800000
==> /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors <== ondemand performance
==> /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq <== 800000
==> /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_max_freq <== 2601000
==> /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_min_freq <== 800000
==> /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq <== 2601000
==> /sys/devices/system/cpu/cpu0/cpufreq/affected_cpus <== 0
==> /sys/devices/system/cpu/cpu0/cpufreq/scaling_min_freq <== 800000
==> /sys/devices/system/cpu/cpu0/cpufreq/related_cpus <== 0
==> /sys/devices/system/cpu/cpu0/cpufreq/scaling_setspeed <== <unsupported>
==> /sys/devices/system/cpu/cpu0/cpuidle/state0/desc <== CPUIDLE CORE POLL IDLE
==> /sys/devices/system/cpu/cpu0/cpuidle/state0/name <== POLL
==> /sys/devices/system/cpu/cpu0/cpuidle/state0/time <== 19233
==> /sys/devices/system/cpu/cpu0/cpuidle/state0/disable <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state0/power <== 4294967295
==> /sys/devices/system/cpu/cpu0/cpuidle/state0/usage <== 54
==> /sys/devices/system/cpu/cpu0/cpuidle/state0/latency <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state1/desc <== MWAIT 0x00
==> /sys/devices/system/cpu/cpu0/cpuidle/state1/name <== C1-SNB
==> /sys/devices/system/cpu/cpu0/cpuidle/state1/time <== 919026
==> /sys/devices/system/cpu/cpu0/cpuidle/state1/disable <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state1/power <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state1/usage <== 7243
==> /sys/devices/system/cpu/cpu0/cpuidle/state1/latency <== 2
==> /sys/devices/system/cpu/cpu0/cpuidle/state2/desc <== MWAIT 0x01
==> /sys/devices/system/cpu/cpu0/cpuidle/state2/name <== C1E-SNB
==> /sys/devices/system/cpu/cpu0/cpuidle/state2/time <== 8630346
==> /sys/devices/system/cpu/cpu0/cpuidle/state2/disable <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state2/power <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state2/usage <== 17935
==> /sys/devices/system/cpu/cpu0/cpuidle/state2/latency <== 10
==> /sys/devices/system/cpu/cpu0/cpuidle/state3/desc <== MWAIT 0x10
==> /sys/devices/system/cpu/cpu0/cpuidle/state3/name <== C3-SNB
==> /sys/devices/system/cpu/cpu0/cpuidle/state3/time <== 3240741
==> /sys/devices/system/cpu/cpu0/cpuidle/state3/disable <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state3/power <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state3/usage <== 4955
==> /sys/devices/system/cpu/cpu0/cpuidle/state3/latency <== 80
==> /sys/devices/system/cpu/cpu0/cpuidle/state4/desc <== MWAIT 0x20
==> /sys/devices/system/cpu/cpu0/cpuidle/state4/name <== C6-SNB
==> /sys/devices/system/cpu/cpu0/cpuidle/state4/time <== 226879
==> /sys/devices/system/cpu/cpu0/cpuidle/state4/disable <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state4/power <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state4/usage <== 184
==> /sys/devices/system/cpu/cpu0/cpuidle/state4/latency <== 104
==> /sys/devices/system/cpu/cpu0/cpuidle/state5/desc <== MWAIT 0x30
==> /sys/devices/system/cpu/cpu0/cpuidle/state5/name <== C7-SNB
==> /sys/devices/system/cpu/cpu0/cpuidle/state5/time <== 52170100
==> /sys/devices/system/cpu/cpu0/cpuidle/state5/disable <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state5/power <== 0
==> /sys/devices/system/cpu/cpu0/cpuidle/state5/usage <== 33875
==> /sys/devices/system/cpu/cpu0/cpuidle/state5/latency <== 109
==> /sys/devices/system/cpu/cpu0/cache/index0/size <== 32K
==> /sys/devices/system/cpu/cpu0/cache/index0/type <== Data
==> /sys/devices/system/cpu/cpu0/cache/index0/level <== 1
==> /sys/devices/system/cpu/cpu0/cache/index0/number_of_sets <== 64
==> /sys/devices/system/cpu/cpu0/cache/index0/shared_cpu_map <== 03
==> /sys/devices/system/cpu/cpu0/cache/index0/shared_cpu_list <== 0-1
==> /sys/devices/system/cpu/cpu0/cache/index0/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu0/cache/index0/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu0/cache/index0/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu0/cache/index1/size <== 32K
==> /sys/devices/system/cpu/cpu0/cache/index1/type <== Instruction
==> /sys/devices/system/cpu/cpu0/cache/index1/level <== 1
==> /sys/devices/system/cpu/cpu0/cache/index1/number_of_sets <== 64
==> /sys/devices/system/cpu/cpu0/cache/index1/shared_cpu_map <== 03
==> /sys/devices/system/cpu/cpu0/cache/index1/shared_cpu_list <== 0-1
==> /sys/devices/system/cpu/cpu0/cache/index1/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu0/cache/index1/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu0/cache/index1/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu0/cache/index2/size <== 256K
==> /sys/devices/system/cpu/cpu0/cache/index2/type <== Unified
==> /sys/devices/system/cpu/cpu0/cache/index2/level <== 2
==> /sys/devices/system/cpu/cpu0/cache/index2/number_of_sets <== 512
==> /sys/devices/system/cpu/cpu0/cache/index2/shared_cpu_map <== 03
==> /sys/devices/system/cpu/cpu0/cache/index2/shared_cpu_list <== 0-1
==> /sys/devices/system/cpu/cpu0/cache/index2/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu0/cache/index2/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu0/cache/index2/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu0/cache/index3/size <== 3072K
==> /sys/devices/system/cpu/cpu0/cache/index3/type <== Unified
==> /sys/devices/system/cpu/cpu0/cache/index3/level <== 3
==> /sys/devices/system/cpu/cpu0/cache/index3/number_of_sets <== 4096
==> /sys/devices/system/cpu/cpu0/cache/index3/shared_cpu_map <== 0f
==> /sys/devices/system/cpu/cpu0/cache/index3/shared_cpu_list <== 0-3
==> /sys/devices/system/cpu/cpu0/cache/index3/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu0/cache/index3/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu0/cache/index3/ways_of_associativity <== 12
==> /sys/devices/system/cpu/cpu0/power/control <== auto
==> /sys/devices/system/cpu/cpu0/power/async <==
disabled
==> /sys/devices/system/cpu/cpu0/power/runtime_enabled <== disabled
==> /sys/devices/system/cpu/cpu0/power/runtime_active_kids <== 0
==> /sys/devices/system/cpu/cpu0/power/runtime_active_time <== 0
==> /sys/devices/system/cpu/cpu0/power/autosuspend_delay_ms <==
==> /sys/devices/system/cpu/cpu0/power/runtime_status <== unsupported
==> /sys/devices/system/cpu/cpu0/power/runtime_usage <== 0
==> /sys/devices/system/cpu/cpu0/power/runtime_suspended_time <== 0
==> /sys/devices/system/cpu/cpu0/uevent <== MODALIAS=x86cpu:vendor:0000:family:0006:model:002A:feature:,0000,0001,0002,0003,0004,0005,0006,0007,0008,0009,000B,000C,000D,000E,000F,0010,0011,0013,0015,0016,0017,0018,0019,001A,001B,001C,001D,001F,0034,003B,003D,0066,0068,006B,006C,006D,0072,0076,0078,007C,007D,0080,0081,0082,0083,0084,0085,0086,0087,0088,0089,008D,008E,008F,0091,0093,0094,0095,0097,0098,0099,009A,009B,009C,00C0,00E0,00E1,00E3,00E4,00E5,00E6,00E7,0100,0101,0102,0103,0104
==> /sys/devices/system/cpu/cpu0/topology/core_id <== 0
==> /sys/devices/system/cpu/cpu0/topology/thread_siblings <== 03
==> /sys/devices/system/cpu/cpu0/topology/core_siblings <== 0f
==> /sys/devices/system/cpu/cpu0/topology/core_siblings_list <== 0-3
==> /sys/devices/system/cpu/cpu0/topology/thread_siblings_list <== 0-1
==> /sys/devices/system/cpu/cpu0/topology/physical_package_id <== 0
==> /sys/devices/system/cpu/cpu0/thermal_throttle/core_power_limit_count <== 0
==> /sys/devices/system/cpu/cpu0/thermal_throttle/package_throttle_count <== 0
==> /sys/devices/system/cpu/cpu0/thermal_throttle/core_throttle_count <== 0
==> /sys/devices/system/cpu/cpu0/thermal_throttle/package_power_limit_count <== 0
==> /sys/devices/system/cpu/cpu1/cpufreq/bios_limit <== 2601000
==> /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor <== ondemand
==> /sys/devices/system/cpu/cpu1/cpufreq/cpuinfo_transition_latency <== 10000
==> /sys/devices/system/cpu/cpu1/cpufreq/scaling_available_frequencies <== 2601000 2600000 2400000 2200000 2000000 1800000 1600000 1400000 1200000 1000000 800000
==> /sys/devices/system/cpu/cpu1/cpufreq/scaling_driver <== acpi-cpufreq
==> /sys/devices/system/cpu/cpu1/cpufreq/cpuinfo_cur_freq <== 800000
==> /sys/devices/system/cpu/cpu1/cpufreq/scaling_available_governors <== ondemand performance
==> /sys/devices/system/cpu/cpu1/cpufreq/scaling_cur_freq <== 800000
==> /sys/devices/system/cpu/cpu1/cpufreq/cpuinfo_max_freq <== 2601000
==> /sys/devices/system/cpu/cpu1/cpufreq/cpuinfo_min_freq <== 800000
==> /sys/devices/system/cpu/cpu1/cpufreq/scaling_max_freq <== 2601000
==> /sys/devices/system/cpu/cpu1/cpufreq/affected_cpus <== 1
==> /sys/devices/system/cpu/cpu1/cpufreq/scaling_min_freq <== 800000
==> /sys/devices/system/cpu/cpu1/cpufreq/related_cpus <== 1
==> /sys/devices/system/cpu/cpu1/cpufreq/scaling_setspeed <== <unsupported>
==> /sys/devices/system/cpu/cpu1/cpuidle/state0/desc <== CPUIDLE CORE POLL IDLE
==> /sys/devices/system/cpu/cpu1/cpuidle/state0/name <== POLL
==> /sys/devices/system/cpu/cpu1/cpuidle/state0/time <== 61747
==> /sys/devices/system/cpu/cpu1/cpuidle/state0/disable <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state0/power <== 4294967295
==> /sys/devices/system/cpu/cpu1/cpuidle/state0/usage <== 26
==> /sys/devices/system/cpu/cpu1/cpuidle/state0/latency <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state1/desc <== MWAIT 0x00
==> /sys/devices/system/cpu/cpu1/cpuidle/state1/name <== C1-SNB
==> /sys/devices/system/cpu/cpu1/cpuidle/state1/time <== 256430
==> /sys/devices/system/cpu/cpu1/cpuidle/state1/disable <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state1/power <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state1/usage <== 4418
==> /sys/devices/system/cpu/cpu1/cpuidle/state1/latency <== 2
==> /sys/devices/system/cpu/cpu1/cpuidle/state2/desc <== MWAIT 0x01
==> /sys/devices/system/cpu/cpu1/cpuidle/state2/name <== C1E-SNB
==> /sys/devices/system/cpu/cpu1/cpuidle/state2/time <== 7489672
==> /sys/devices/system/cpu/cpu1/cpuidle/state2/disable <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state2/power <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state2/usage <== 9076
==> /sys/devices/system/cpu/cpu1/cpuidle/state2/latency <== 10
==> /sys/devices/system/cpu/cpu1/cpuidle/state3/desc <== MWAIT 0x10
==> /sys/devices/system/cpu/cpu1/cpuidle/state3/name <== C3-SNB
==> /sys/devices/system/cpu/cpu1/cpuidle/state3/time <== 3911044
==> /sys/devices/system/cpu/cpu1/cpuidle/state3/disable <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state3/power <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state3/usage <== 2572
==> /sys/devices/system/cpu/cpu1/cpuidle/state3/latency <== 80
==> /sys/devices/system/cpu/cpu1/cpuidle/state4/desc <== MWAIT 0x20
==> /sys/devices/system/cpu/cpu1/cpuidle/state4/name <== C6-SNB
==> /sys/devices/system/cpu/cpu1/cpuidle/state4/time <== 163650
==> /sys/devices/system/cpu/cpu1/cpuidle/state4/disable <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state4/power <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state4/usage <== 73
==> /sys/devices/system/cpu/cpu1/cpuidle/state4/latency <== 104
==> /sys/devices/system/cpu/cpu1/cpuidle/state5/desc <== MWAIT 0x30
==> /sys/devices/system/cpu/cpu1/cpuidle/state5/name <== C7-SNB
==> /sys/devices/system/cpu/cpu1/cpuidle/state5/time <== 52737221
==> /sys/devices/system/cpu/cpu1/cpuidle/state5/disable <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state5/power <== 0
==> /sys/devices/system/cpu/cpu1/cpuidle/state5/usage <== 10540
==> /sys/devices/system/cpu/cpu1/cpuidle/state5/latency <== 109
==> /sys/devices/system/cpu/cpu1/cache/index0/size <== 32K
==> /sys/devices/system/cpu/cpu1/cache/index0/type <== Data
==> /sys/devices/system/cpu/cpu1/cache/index0/level <== 1
==> /sys/devices/system/cpu/cpu1/cache/index0/number_of_sets <== 64
==> /sys/devices/system/cpu/cpu1/cache/index0/shared_cpu_map <== 03
==> /sys/devices/system/cpu/cpu1/cache/index0/shared_cpu_list <== 0-1
==> /sys/devices/system/cpu/cpu1/cache/index0/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu1/cache/index0/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu1/cache/index0/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu1/cache/index1/size <== 32K
==> /sys/devices/system/cpu/cpu1/cache/index1/type <== Instruction
==> /sys/devices/system/cpu/cpu1/cache/index1/level <== 1
==> /sys/devices/system/cpu/cpu1/cache/index1/number_of_sets <== 64
==> /sys/devices/system/cpu/cpu1/cache/index1/shared_cpu_map <== 03
==> /sys/devices/system/cpu/cpu1/cache/index1/shared_cpu_list <== 0-1
==> /sys/devices/system/cpu/cpu1/cache/index1/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu1/cache/index1/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu1/cache/index1/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu1/cache/index2/size <== 256K
==> /sys/devices/system/cpu/cpu1/cache/index2/type <== Unified
==> /sys/devices/system/cpu/cpu1/cache/index2/level <== 2
==> /sys/devices/system/cpu/cpu1/cache/index2/number_of_sets <== 512
==> /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_map <== 03
==> /sys/devices/system/cpu/cpu1/cache/index2/shared_cpu_list <== 0-1
==> /sys/devices/system/cpu/cpu1/cache/index2/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu1/cache/index2/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu1/cache/index2/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu1/cache/index3/size <== 3072K
==> /sys/devices/system/cpu/cpu1/cache/index3/type <== Unified
==> /sys/devices/system/cpu/cpu1/cache/index3/level <== 3
==> /sys/devices/system/cpu/cpu1/cache/index3/number_of_sets <== 4096
==> /sys/devices/system/cpu/cpu1/cache/index3/shared_cpu_map <== 0f
==> /sys/devices/system/cpu/cpu1/cache/index3/shared_cpu_list <== 0-3
==> /sys/devices/system/cpu/cpu1/cache/index3/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu1/cache/index3/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu1/cache/index3/ways_of_associativity <== 12
==> /sys/devices/system/cpu/cpu1/power/control <== auto
==> /sys/devices/system/cpu/cpu1/power/async <==
disabled
==> /sys/devices/system/cpu/cpu1/power/runtime_enabled <== disabled
==> /sys/devices/system/cpu/cpu1/power/runtime_active_kids <== 0
==> /sys/devices/system/cpu/cpu1/power/runtime_active_time <== 0
==> /sys/devices/system/cpu/cpu1/power/autosuspend_delay_ms <==
==> /sys/devices/system/cpu/cpu1/power/runtime_status <== unsupported
==> /sys/devices/system/cpu/cpu1/power/runtime_usage <== 0
==> /sys/devices/system/cpu/cpu1/power/runtime_suspended_time <== 0
==> /sys/devices/system/cpu/cpu1/online <==
1
==> /sys/devices/system/cpu/cpu1/uevent <== MODALIAS=x86cpu:vendor:0000:family:0006:model:002A:feature:,0000,0001,0002,0003,0004,0005,0006,0007,0008,0009,000B,000C,000D,000E,000F,0010,0011,0013,0015,0016,0017,0018,0019,001A,001B,001C,001D,001F,0034,003B,003D,0066,0068,006B,006C,006D,0072,0076,0078,007C,007D,0080,0081,0082,0083,0084,0085,0086,0087,0088,0089,008D,008E,008F,0091,0093,0094,0095,0097,0098,0099,009A,009B,009C,00C0,00E0,00E1,00E3,00E4,00E5,00E6,00E7,0100,0101,0102,0103,0104
==> /sys/devices/system/cpu/cpu1/topology/core_id <== 0
==> /sys/devices/system/cpu/cpu1/topology/thread_siblings <== 03
==> /sys/devices/system/cpu/cpu1/topology/core_siblings <== 0f
==> /sys/devices/system/cpu/cpu1/topology/core_siblings_list <== 0-3
==> /sys/devices/system/cpu/cpu1/topology/thread_siblings_list <== 0-1
==> /sys/devices/system/cpu/cpu1/topology/physical_package_id <== 0
==> /sys/devices/system/cpu/cpu1/thermal_throttle/core_power_limit_count <== 0
==> /sys/devices/system/cpu/cpu1/thermal_throttle/package_throttle_count <== 0
==> /sys/devices/system/cpu/cpu1/thermal_throttle/core_throttle_count <== 0
==> /sys/devices/system/cpu/cpu1/thermal_throttle/package_power_limit_count <== 0
==> /sys/devices/system/cpu/cpu2/cpufreq/bios_limit <== 2601000
==> /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor <== ondemand
==> /sys/devices/system/cpu/cpu2/cpufreq/cpuinfo_transition_latency <== 10000
==> /sys/devices/system/cpu/cpu2/cpufreq/scaling_available_frequencies <== 2601000 2600000 2400000 2200000 2000000 1800000 1600000 1400000 1200000 1000000 800000
==> /sys/devices/system/cpu/cpu2/cpufreq/scaling_driver <== acpi-cpufreq
==> /sys/devices/system/cpu/cpu2/cpufreq/cpuinfo_cur_freq <== 800000
==> /sys/devices/system/cpu/cpu2/cpufreq/scaling_available_governors <== ondemand performance
==> /sys/devices/system/cpu/cpu2/cpufreq/scaling_cur_freq <== 800000
==> /sys/devices/system/cpu/cpu2/cpufreq/cpuinfo_max_freq <== 2601000
==> /sys/devices/system/cpu/cpu2/cpufreq/cpuinfo_min_freq <== 800000
==> /sys/devices/system/cpu/cpu2/cpufreq/scaling_max_freq <== 2601000
==> /sys/devices/system/cpu/cpu2/cpufreq/affected_cpus <== 2
==> /sys/devices/system/cpu/cpu2/cpufreq/scaling_min_freq <== 800000
==> /sys/devices/system/cpu/cpu2/cpufreq/related_cpus <== 2
==> /sys/devices/system/cpu/cpu2/cpufreq/scaling_setspeed <== <unsupported>
==> /sys/devices/system/cpu/cpu2/cpuidle/state0/desc <== CPUIDLE CORE POLL IDLE
==> /sys/devices/system/cpu/cpu2/cpuidle/state0/name <== POLL
==> /sys/devices/system/cpu/cpu2/cpuidle/state0/time <== 15532
==> /sys/devices/system/cpu/cpu2/cpuidle/state0/disable <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state0/power <== 4294967295
==> /sys/devices/system/cpu/cpu2/cpuidle/state0/usage <== 36
==> /sys/devices/system/cpu/cpu2/cpuidle/state0/latency <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state1/desc <== MWAIT 0x00
==> /sys/devices/system/cpu/cpu2/cpuidle/state1/name <== C1-SNB
==> /sys/devices/system/cpu/cpu2/cpuidle/state1/time <== 424554
==> /sys/devices/system/cpu/cpu2/cpuidle/state1/disable <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state1/power <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state1/usage <== 7838
==> /sys/devices/system/cpu/cpu2/cpuidle/state1/latency <== 2
==> /sys/devices/system/cpu/cpu2/cpuidle/state2/desc <== MWAIT 0x01
==> /sys/devices/system/cpu/cpu2/cpuidle/state2/name <== C1E-SNB
==> /sys/devices/system/cpu/cpu2/cpuidle/state2/time <== 8228801
==> /sys/devices/system/cpu/cpu2/cpuidle/state2/disable <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state2/power <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state2/usage <== 8796
==> /sys/devices/system/cpu/cpu2/cpuidle/state2/latency <== 10
==> /sys/devices/system/cpu/cpu2/cpuidle/state3/desc <== MWAIT 0x10
==> /sys/devices/system/cpu/cpu2/cpuidle/state3/name <== C3-SNB
==> /sys/devices/system/cpu/cpu2/cpuidle/state3/time <== 4179929
==> /sys/devices/system/cpu/cpu2/cpuidle/state3/disable <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state3/power <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state3/usage <== 2322
==> /sys/devices/system/cpu/cpu2/cpuidle/state3/latency <== 80
==> /sys/devices/system/cpu/cpu2/cpuidle/state4/desc <== MWAIT 0x20
==> /sys/devices/system/cpu/cpu2/cpuidle/state4/name <== C6-SNB
==> /sys/devices/system/cpu/cpu2/cpuidle/state4/time <== 184890
==> /sys/devices/system/cpu/cpu2/cpuidle/state4/disable <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state4/power <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state4/usage <== 67
==> /sys/devices/system/cpu/cpu2/cpuidle/state4/latency <== 104
==> /sys/devices/system/cpu/cpu2/cpuidle/state5/desc <== MWAIT 0x30
==> /sys/devices/system/cpu/cpu2/cpuidle/state5/name <== C7-SNB
==> /sys/devices/system/cpu/cpu2/cpuidle/state5/time <== 53545208
==> /sys/devices/system/cpu/cpu2/cpuidle/state5/disable <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state5/power <== 0
==> /sys/devices/system/cpu/cpu2/cpuidle/state5/usage <== 10083
==> /sys/devices/system/cpu/cpu2/cpuidle/state5/latency <== 109
==> /sys/devices/system/cpu/cpu2/cache/index0/size <== 32K
==> /sys/devices/system/cpu/cpu2/cache/index0/type <== Data
==> /sys/devices/system/cpu/cpu2/cache/index0/level <== 1
==> /sys/devices/system/cpu/cpu2/cache/index0/number_of_sets <== 64
==> /sys/devices/system/cpu/cpu2/cache/index0/shared_cpu_map <== 0c
==> /sys/devices/system/cpu/cpu2/cache/index0/shared_cpu_list <== 2-3
==> /sys/devices/system/cpu/cpu2/cache/index0/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu2/cache/index0/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu2/cache/index0/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu2/cache/index1/size <== 32K
==> /sys/devices/system/cpu/cpu2/cache/index1/type <== Instruction
==> /sys/devices/system/cpu/cpu2/cache/index1/level <== 1
==> /sys/devices/system/cpu/cpu2/cache/index1/number_of_sets <== 64
==> /sys/devices/system/cpu/cpu2/cache/index1/shared_cpu_map <== 0c
==> /sys/devices/system/cpu/cpu2/cache/index1/shared_cpu_list <== 2-3
==> /sys/devices/system/cpu/cpu2/cache/index1/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu2/cache/index1/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu2/cache/index1/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu2/cache/index2/size <== 256K
==> /sys/devices/system/cpu/cpu2/cache/index2/type <== Unified
==> /sys/devices/system/cpu/cpu2/cache/index2/level <== 2
==> /sys/devices/system/cpu/cpu2/cache/index2/number_of_sets <== 512
==> /sys/devices/system/cpu/cpu2/cache/index2/shared_cpu_map <== 0c
==> /sys/devices/system/cpu/cpu2/cache/index2/shared_cpu_list <== 2-3
==> /sys/devices/system/cpu/cpu2/cache/index2/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu2/cache/index2/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu2/cache/index2/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu2/cache/index3/size <== 3072K
==> /sys/devices/system/cpu/cpu2/cache/index3/type <== Unified
==> /sys/devices/system/cpu/cpu2/cache/index3/level <== 3
==> /sys/devices/system/cpu/cpu2/cache/index3/number_of_sets <== 4096
==> /sys/devices/system/cpu/cpu2/cache/index3/shared_cpu_map <== 0f
==> /sys/devices/system/cpu/cpu2/cache/index3/shared_cpu_list <== 0-3
==> /sys/devices/system/cpu/cpu2/cache/index3/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu2/cache/index3/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu2/cache/index3/ways_of_associativity <== 12
==> /sys/devices/system/cpu/cpu2/power/control <== auto
==> /sys/devices/system/cpu/cpu2/power/async <==
disabled
==> /sys/devices/system/cpu/cpu2/power/runtime_enabled <== disabled
==> /sys/devices/system/cpu/cpu2/power/runtime_active_kids <== 0
==> /sys/devices/system/cpu/cpu2/power/runtime_active_time <== 0
==> /sys/devices/system/cpu/cpu2/power/autosuspend_delay_ms <==
==> /sys/devices/system/cpu/cpu2/power/runtime_status <== unsupported
==> /sys/devices/system/cpu/cpu2/power/runtime_usage <== 0
==> /sys/devices/system/cpu/cpu2/power/runtime_suspended_time <== 0
==> /sys/devices/system/cpu/cpu2/online <==
1
==> /sys/devices/system/cpu/cpu2/uevent <== MODALIAS=x86cpu:vendor:0000:family:0006:model:002A:feature:,0000,0001,0002,0003,0004,0005,0006,0007,0008,0009,000B,000C,000D,000E,000F,0010,0011,0013,0015,0016,0017,0018,0019,001A,001B,001C,001D,001F,0034,003B,003D,0066,0068,006B,006C,006D,0072,0076,0078,007C,007D,0080,0081,0082,0083,0084,0085,0086,0087,0088,0089,008D,008E,008F,0091,0093,0094,0095,0097,0098,0099,009A,009B,009C,00C0,00E0,00E1,00E3,00E4,00E5,00E6,00E7,0100,0101,0102,0103,0104
==> /sys/devices/system/cpu/cpu2/topology/core_id <== 1
==> /sys/devices/system/cpu/cpu2/topology/thread_siblings <== 0c
==> /sys/devices/system/cpu/cpu2/topology/core_siblings <== 0f
==> /sys/devices/system/cpu/cpu2/topology/core_siblings_list <== 0-3
==> /sys/devices/system/cpu/cpu2/topology/thread_siblings_list <== 2-3
==> /sys/devices/system/cpu/cpu2/topology/physical_package_id <== 0
==> /sys/devices/system/cpu/cpu2/thermal_throttle/core_power_limit_count <== 0
==> /sys/devices/system/cpu/cpu2/thermal_throttle/package_throttle_count <== 0
==> /sys/devices/system/cpu/cpu2/thermal_throttle/core_throttle_count <== 0
==> /sys/devices/system/cpu/cpu2/thermal_throttle/package_power_limit_count <== 0
==> /sys/devices/system/cpu/cpu3/cpufreq/bios_limit <== 2601000
==> /sys/devices/system/cpu/cpu3/cpufreq/scaling_governor <== ondemand
==> /sys/devices/system/cpu/cpu3/cpufreq/cpuinfo_transition_latency <== 10000
==> /sys/devices/system/cpu/cpu3/cpufreq/scaling_available_frequencies <== 2601000 2600000 2400000 2200000 2000000 1800000 1600000 1400000 1200000 1000000 800000
==> /sys/devices/system/cpu/cpu3/cpufreq/scaling_driver <== acpi-cpufreq
==> /sys/devices/system/cpu/cpu3/cpufreq/cpuinfo_cur_freq <== 800000
==> /sys/devices/system/cpu/cpu3/cpufreq/scaling_available_governors <== ondemand performance
==> /sys/devices/system/cpu/cpu3/cpufreq/scaling_cur_freq <== 800000
==> /sys/devices/system/cpu/cpu3/cpufreq/cpuinfo_max_freq <== 2601000
==> /sys/devices/system/cpu/cpu3/cpufreq/cpuinfo_min_freq <== 800000
==> /sys/devices/system/cpu/cpu3/cpufreq/scaling_max_freq <== 2601000
==> /sys/devices/system/cpu/cpu3/cpufreq/affected_cpus <== 3
==> /sys/devices/system/cpu/cpu3/cpufreq/scaling_min_freq <== 800000
==> /sys/devices/system/cpu/cpu3/cpufreq/related_cpus <== 3
==> /sys/devices/system/cpu/cpu3/cpufreq/scaling_setspeed <== <unsupported>
==> /sys/devices/system/cpu/cpu3/cpuidle/state0/desc <== CPUIDLE CORE POLL IDLE
==> /sys/devices/system/cpu/cpu3/cpuidle/state0/name <== POLL
==> /sys/devices/system/cpu/cpu3/cpuidle/state0/time <== 2323
==> /sys/devices/system/cpu/cpu3/cpuidle/state0/disable <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state0/power <== 4294967295
==> /sys/devices/system/cpu/cpu3/cpuidle/state0/usage <== 28
==> /sys/devices/system/cpu/cpu3/cpuidle/state0/latency <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state1/desc <== MWAIT 0x00
==> /sys/devices/system/cpu/cpu3/cpuidle/state1/name <== C1-SNB
==> /sys/devices/system/cpu/cpu3/cpuidle/state1/time <== 456227
==> /sys/devices/system/cpu/cpu3/cpuidle/state1/disable <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state1/power <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state1/usage <== 7556
==> /sys/devices/system/cpu/cpu3/cpuidle/state1/latency <== 2
==> /sys/devices/system/cpu/cpu3/cpuidle/state2/desc <== MWAIT 0x01
==> /sys/devices/system/cpu/cpu3/cpuidle/state2/name <== C1E-SNB
==> /sys/devices/system/cpu/cpu3/cpuidle/state2/time <== 5578828
==> /sys/devices/system/cpu/cpu3/cpuidle/state2/disable <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state2/power <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state2/usage <== 6351
==> /sys/devices/system/cpu/cpu3/cpuidle/state2/latency <== 10
==> /sys/devices/system/cpu/cpu3/cpuidle/state3/desc <== MWAIT 0x10
==> /sys/devices/system/cpu/cpu3/cpuidle/state3/name <== C3-SNB
==> /sys/devices/system/cpu/cpu3/cpuidle/state3/time <== 3177302
==> /sys/devices/system/cpu/cpu3/cpuidle/state3/disable <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state3/power <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state3/usage <== 1767
==> /sys/devices/system/cpu/cpu3/cpuidle/state3/latency <== 80
==> /sys/devices/system/cpu/cpu3/cpuidle/state4/desc <== MWAIT 0x20
==> /sys/devices/system/cpu/cpu3/cpuidle/state4/name <== C6-SNB
==> /sys/devices/system/cpu/cpu3/cpuidle/state4/time <== 73846
==> /sys/devices/system/cpu/cpu3/cpuidle/state4/disable <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state4/power <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state4/usage <== 38
==> /sys/devices/system/cpu/cpu3/cpuidle/state4/latency <== 104
==> /sys/devices/system/cpu/cpu3/cpuidle/state5/desc <== MWAIT 0x30
==> /sys/devices/system/cpu/cpu3/cpuidle/state5/name <== C7-SNB
==> /sys/devices/system/cpu/cpu3/cpuidle/state5/time <== 53452690
==> /sys/devices/system/cpu/cpu3/cpuidle/state5/disable <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state5/power <== 0
==> /sys/devices/system/cpu/cpu3/cpuidle/state5/usage <== 9184
==> /sys/devices/system/cpu/cpu3/cpuidle/state5/latency <== 109
==> /sys/devices/system/cpu/cpu3/cache/index0/size <== 32K
==> /sys/devices/system/cpu/cpu3/cache/index0/type <== Data
==> /sys/devices/system/cpu/cpu3/cache/index0/level <== 1
==> /sys/devices/system/cpu/cpu3/cache/index0/number_of_sets <== 64
==> /sys/devices/system/cpu/cpu3/cache/index0/shared_cpu_map <== 0c
==> /sys/devices/system/cpu/cpu3/cache/index0/shared_cpu_list <== 2-3
==> /sys/devices/system/cpu/cpu3/cache/index0/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu3/cache/index0/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu3/cache/index0/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu3/cache/index1/size <== 32K
==> /sys/devices/system/cpu/cpu3/cache/index1/type <== Instruction
==> /sys/devices/system/cpu/cpu3/cache/index1/level <== 1
==> /sys/devices/system/cpu/cpu3/cache/index1/number_of_sets <== 64
==> /sys/devices/system/cpu/cpu3/cache/index1/shared_cpu_map <== 0c
==> /sys/devices/system/cpu/cpu3/cache/index1/shared_cpu_list <== 2-3
==> /sys/devices/system/cpu/cpu3/cache/index1/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu3/cache/index1/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu3/cache/index1/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu3/cache/index2/size <== 256K
==> /sys/devices/system/cpu/cpu3/cache/index2/type <== Unified
==> /sys/devices/system/cpu/cpu3/cache/index2/level <== 2
==> /sys/devices/system/cpu/cpu3/cache/index2/number_of_sets <== 512
==> /sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_map <== 0c
==> /sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_list <== 2-3
==> /sys/devices/system/cpu/cpu3/cache/index2/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu3/cache/index2/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu3/cache/index2/ways_of_associativity <== 8
==> /sys/devices/system/cpu/cpu3/cache/index3/size <== 3072K
==> /sys/devices/system/cpu/cpu3/cache/index3/type <== Unified
==> /sys/devices/system/cpu/cpu3/cache/index3/level <== 3
==> /sys/devices/system/cpu/cpu3/cache/index3/number_of_sets <== 4096
==> /sys/devices/system/cpu/cpu3/cache/index3/shared_cpu_map <== 0f
==> /sys/devices/system/cpu/cpu3/cache/index3/shared_cpu_list <== 0-3
==> /sys/devices/system/cpu/cpu3/cache/index3/coherency_line_size <== 64
==> /sys/devices/system/cpu/cpu3/cache/index3/physical_line_partition <== 1
==> /sys/devices/system/cpu/cpu3/cache/index3/ways_of_associativity <== 12
==> /sys/devices/system/cpu/cpu3/power/control <== auto
==> /sys/devices/system/cpu/cpu3/power/async <==
disabled
==> /sys/devices/system/cpu/cpu3/power/runtime_enabled <== disabled
==> /sys/devices/system/cpu/cpu3/power/runtime_active_kids <== 0
==> /sys/devices/system/cpu/cpu3/power/runtime_active_time <== 0
==> /sys/devices/system/cpu/cpu3/power/autosuspend_delay_ms <==
==> /sys/devices/system/cpu/cpu3/power/runtime_status <== unsupported
==> /sys/devices/system/cpu/cpu3/power/runtime_usage <== 0
==> /sys/devices/system/cpu/cpu3/power/runtime_suspended_time <== 0
==> /sys/devices/system/cpu/cpu3/online <==
1
==> /sys/devices/system/cpu/cpu3/uevent <== MODALIAS=x86cpu:vendor:0000:family:0006:model:002A:feature:,0000,0001,0002,0003,0004,0005,0006,0007,0008,0009,000B,000C,000D,000E,000F,0010,0011,0013,0015,0016,0017,0018,0019,001A,001B,001C,001D,001F,0034,003B,003D,0066,0068,006B,006C,006D,0072,0076,0078,007C,007D,0080,0081,0082,0083,0084,0085,0086,0087,0088,0089,008D,008E,008F,0091,0093,0094,0095,0097,0098,0099,009A,009B,009C,00C0,00E0,00E1,00E3,00E4,00E5,00E6,00E7,0100,0101,0102,0103,0104
==> /sys/devices/system/cpu/cpu3/topology/core_id <== 1
==> /sys/devices/system/cpu/cpu3/topology/thread_siblings <== 0c
==> /sys/devices/system/cpu/cpu3/topology/core_siblings <== 0f
==> /sys/devices/system/cpu/cpu3/topology/core_siblings_list <== 0-3
==> /sys/devices/system/cpu/cpu3/topology/thread_siblings_list <== 2-3
==> /sys/devices/system/cpu/cpu3/topology/physical_package_id <== 0
==> /sys/devices/system/cpu/cpu3/thermal_throttle/core_power_limit_count <== 0
==> /sys/devices/system/cpu/cpu3/thermal_throttle/package_throttle_count <== 0
==> /sys/devices/system/cpu/cpu3/thermal_throttle/core_throttle_count <== 0
==> /sys/devices/system/cpu/cpu3/thermal_throttle/package_power_limit_count <== 0
==> /sys/devices/system/cpu/cpufreq/boost <==
1
==> /sys/devices/system/cpu/cpufreq/ondemand/powersave_bias <== 0
==> /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate <== 10000
==> /sys/devices/system/cpu/cpufreq/ondemand/io_is_busy <== 1
==> /sys/devices/system/cpu/cpufreq/ondemand/up_threshold <== 95
==> /sys/devices/system/cpu/cpufreq/ondemand/sampling_rate_min <== 10000
==> /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor <== 1
==> /sys/devices/system/cpu/cpufreq/ondemand/ignore_nice <== 1
==> /sys/devices/system/cpu/cpuidle/current_driver <== intel_idle
==> /sys/devices/system/cpu/cpuidle/current_governor_ro <== menu
==> /sys/devices/system/cpu/power/control <==
auto
==> /sys/devices/system/cpu/power/async <==
disabled
==> /sys/devices/system/cpu/power/runtime_enabled <== disabled
==> /sys/devices/system/cpu/power/runtime_active_kids <== 0
==> /sys/devices/system/cpu/power/runtime_active_time <== 0
==> /sys/devices/system/cpu/power/autosuspend_delay_ms <==
==> /sys/devices/system/cpu/power/runtime_status <== unsupported
==> /sys/devices/system/cpu/power/runtime_usage <== 0
==> /sys/devices/system/cpu/power/runtime_suspended_time <== 0
==> /sys/devices/system/cpu/modalias <== x86cpu:vendor:0000:family:0006:model:002A:feature:,0000,0001,0002,0003,0004,0005,0006,0007,0008,0009,000B,000C,000D,000E,000F,0010,0011,0013,0015,0016,0017,0018,0019,001A,001B,001C,001D,001F,0034,003B,003D,0066,0068,006B,006C,006D,0072,0076,0078,007C,007D,0080,0081,0082,0083,0084,0085,0086,0087,0088,0089,008D,008E,008F,0091,0093,0094,0095,0097,0098,0099,009A,009B,009C,00C0,00E0,00E1,00E3,00E4,00E5,00E6,00E7,0100,0101,0102,0103,0104
==> /sys/devices/system/cpu/kernel_max <==
7
==> /sys/devices/system/cpu/possible <==
0-7
==> /sys/devices/system/cpu/online <==
0-3
==> /sys/devices/system/cpu/offline <==
4-7
==> /sys/devices/system/cpu/uevent <==
==> /sys/devices/system/cpu/present <==
0-3
Viresh Kumar
maintainerQuoted 10 lines · Toralf Förster
n22 ~ # cat /sys/devices/system/cpu/cpu?/cpufreq/scaling_cur_freq 800000
800000
800000
800000
n22 ~ # cat /sys/devices/system/cpu/cpu?/cpufreq/cpuinfo_cur_freq 800000
2601000
2601000
2601000
Hmm.. So this is what I make out of it.scaling_cur_freq: is the freq which cpufreq core thinks is set. Which is correct. 800 MHz.
cpuinfo_cur_freq: is the actual freq in hardware, which is wrong. @Rafael: If I am not wrong we have seen this earlier too in last few months. There was some problem with bootloader or something else I can't remember.
Toralf Förster
Acked-by Viresh KumarThe latest bisect attempt gave :
commit a66b2e503fc79fff6632d02ef5a0ee47c1d2553d
Author: Srivatsa S. Bhat <[email protected]> Date: Wed May 15 21:47:17 2013 +0200
cpufreq: Preserve sysfs files across suspend/resume
The file permissions of cpufreq per-cpu sysfs files are not preserved across suspend/resume because we internally go through the CPU Hotplug path which reinitializes the file permissions on CPU online.
But the user is not supposed to know that we are using CPU hotplug internally within suspend/resume (IOW, the kernel should not silently wreck the user-set file permissions across a suspend cycle). Therefore, we need to preserve the file permissions as they are across suspend/resume.
The simplest way to achieve that is to just not touch the sysfs files at all - ie., just ignore the CPU hotplug notifications in the suspend/resume path (_FROZEN) in the cpufreq hotplug callback.
To get a more reliable bisect result I had to start BOINC before (4 childs each with nice -19 started)
Signature · 3 lines
MfG/Sincerely Toralf Förster pgp finger print: 7B1A 07F4 EC82 0F90 D4C2 8936 872A E508 7DB6 9DA3
Quoted 30 lines · Toralf Förster
The latest bisect attempt gave :
commit
a66b2e503fc79fff6632d02ef5a0ee47c1d2553dAuthor: Srivatsa S. Bhat <[email protected]> Date: Wed May 15 21:47:17 2013 +0200cpufreq: Preserve sysfs files across suspend/resume
The file permissions of cpufreq per-cpu sysfs files are not preserved across suspend/resume because we internally go through the CPU Hotplug path which reinitializes the file permissions on CPU online.
But the user is not supposed to know that we are using CPU hotplug internally within suspend/resume (IOW, the kernel should not silently wreck the user-set file permissions across a suspend cycle). Therefore, we need to preserve the file permissions as they are across suspend/resume.
The simplest way to achieve that is to just not touch the sysfs files at all - ie., just ignore the CPU hotplug notifications in the suspend/resume path (
_FROZEN) in the cpufreq hotplug callback.Reported-by Robert Jarzmik <[email protected]>Reported-by Durgadoss R <[email protected]>Signed-off-by Srivatsa S. Bhat <[email protected]>Acked-by Viresh Kumar <[email protected]>Signed-off-by Rafael J. Wysocki <[email protected]>
Quoted 2 lines
To get a more reliable bisect result I had to start BOINC before (4 childs each with nice -19 started)
Well, to be honest, I'm not really sure how the above commit can cause the problem you're seeing to happen ...
Srivatsa, Viresh, any ideas?
Rafael
Signature · 2 lines
I speak only for myself. Rafael J. Wysocki, Intel Open Source Technology Center.
Viresh Kumar
maintainerQuoted 4 lines · Rafael J. Wysocki
Well, to be honest, I'm not really sure how the above commit can cause the problem you're seeing to happen ...
Srivatsa, Viresh, any ideas?
I don't. But its very simple to get this checked isn't it? Just revert this patch on top of your kernel (leave bisect) and see if gets fixed.
Quoted 10 lines · Viresh Kumar
Quoted 4 lines · Rafael J. Wysocki
Well, to be honest, I'm not really sure how the above commit can cause the problem you're seeing to happen ...
Srivatsa, Viresh, any ideas?
I don't. But its very simple to get this checked isn't it? Just revert this patch on top of your kernel (leave bisect) and see if gets fixed.
I do (mostly) check a bisect result before I post it to a mailing list to not blame the wrong commit to be guilty (b/c - erm - I did it in the past).
Said that I finished bisect, checked out commit a66b2e5, tested it (has the issue), did "git show a66b2e5 | patch -p1 -R" and checked that result (no issue).
Signature · 3 lines
MfG/Sincerely Toralf Förster pgp finger print: 7B1A 07F4 EC82 0F90 D4C2 8936 872A E508 7DB6 9DA3
Quoted 2 lines · Viresh Kumar
Just revert this patch on top of your kernel (leave bisect) and see if gets fixed.
negative,
reverting that commit on top of current -rc7+ tree breaks s2ram completely here (just sysrq+b works when the power led is blinking)
Signature · 3 lines
MfG/Sincerely Toralf Förster pgp finger print: 7B1A 07F4 EC82 0F90 D4C2 8936 872A E508 7DB6 9DA3
Quoted 31 lines · Rafael J. Wysocki
Quoted 30 lines · Toralf Förster
The latest bisect attempt gave :
commit
a66b2e503fc79fff6632d02ef5a0ee47c1d2553dAuthor: Srivatsa S. Bhat <[email protected]> Date: Wed May 15 21:47:17 2013 +0200cpufreq: Preserve sysfs files across suspend/resume
The file permissions of cpufreq per-cpu sysfs files are not preserved across suspend/resume because we internally go through the CPU Hotplug path which reinitializes the file permissions on CPU online.
But the user is not supposed to know that we are using CPU hotplug internally within suspend/resume (IOW, the kernel should not silently wreck the user-set file permissions across a suspend cycle). Therefore, we need to preserve the file permissions as they are across suspend/resume.
The simplest way to achieve that is to just not touch the sysfs files at all - ie., just ignore the CPU hotplug notifications in the suspend/resume path (
_FROZEN) in the cpufreq hotplug callback.Reported-by Robert Jarzmik <[email protected]>Reported-by Durgadoss R <[email protected]>Signed-off-by Srivatsa S. Bhat <[email protected]>Acked-by Viresh Kumar <[email protected]>Signed-off-by Rafael J. Wysocki <[email protected]>
Quoted 8 lines
Quoted 2 lines
To get a more reliable bisect result I had to start BOINC before (4 childs each with nice -19 started)
Well, to be honest, I'm not really sure how the above commit can cause the problem you're seeing to happen ...
Srivatsa, Viresh, any ideas?
I tried to look up what problem is being reported, but apart from the hint from the subject line, I couldn't find anything more. So, guessing that there is something wrong with cpufreq after an s3 cycle, Toralf, can you please try out the below patch and see if it improves anything? (Don't revert anything, just apply the below diff on a problematic kernel and see if it solves your issue).
---
drivers/cpufreq/cpufreq_stats.c | 1 + 1 file changed, 1 insertion(+)
drivers/cpufreq/cpufreq_stats.c +1 -0
@@ -349,6 +349,7 @@ static int __cpuinit cpufreq_stat_cpu_callback(struct notifier_block *nfb,switch (action) {case CPU_ONLINE : +case CPU_ONLINE_FROZEN :cpufreq_update_policy (cpu);break ;case CPU_DOWN_PREPARE :
Quoted 4 lines · Srivatsa S. Bhat
Toralf, can you please
try out the below patch and see if it improves anything? (Don't revert anything, just apply the below diff on a problematic kernel and see if it solves your issue).
applied on top of a66b2e5 - issue went away (either fixed or hidden now)
Thx
Signature · 3 lines
MfG/Sincerely Toralf Förster pgp finger print: 7B1A 07F4 EC82 0F90 D4C2 8936 872A E508 7DB6 9DA3
PATCH 1/1 cpufreq: Fix cpufreq regression after suspend/resume
Quoted 8 lines · Toralf Förster
Quoted 4 lines · Srivatsa S. Bhat
Toralf, can you please
try out the below patch and see if it improves anything? (Don't revert anything, just apply the below diff on a problematic kernel and see if it solves your issue).applied on top of a66b2e5 - issue went away (either fixed or hidden now)
Cool! So here is the proper patch, with changelog added.
Regards,
Srivatsa S. Bhat
-----------------------------------------------------------------------------
From: Srivatsa S. Bhat <[email protected]>
Subject: [PATCH] cpufreq: Fix cpufreq regression after suspend/resume
Toralf Förster reported that the cpufreq ondemand governor behaves erratically (doesn't scale well) after a suspend/resume cycle. The problem was that the cpufreq subsystem's idea of the cpu frequencies differed from the actual frequencies set in the hardware after a suspend/resume cycle. Toralf bisected the problem to commit a66b2e5 (cpufreq: Preserve sysfs files across suspend/resume).
Among other (harmless) things, that commit skipped the call to cpufreq_update_policy() in the resume path. But cpufreq_update_policy() plays an important role during resume, because it is responsible for checking if the BIOS changed the cpu frequencies behind our back and resynchronize the cpufreq subsystem's knowledge of the cpu frequencies, and update them accordingly.
So, restore the call to cpufreq_update_policy() in the resume path to fix the cpufreq regression.
---
drivers/cpufreq/cpufreq_stats.c | 1 + 1 file changed, 1 insertion(+)
drivers/cpufreq/cpufreq_stats.c +1 -0
@@ -349,6 +349,7 @@ static int __cpuinit cpufreq_stat_cpu_callback(struct notifier_block *nfb,switch (action) {case CPU_ONLINE : +case CPU_ONLINE_FROZEN :cpufreq_update_policy (cpu);break ;case CPU_DOWN_PREPARE :
Re: [PATCH] cpufreq: Fix cpufreq regression after suspend/resume
Quoted 35 lines · Srivatsa S. Bhat
Quoted 8 lines · Toralf Förster
Quoted 4 lines · Srivatsa S. Bhat
Toralf, can you please
try out the below patch and see if it improves anything? (Don't revert anything, just apply the below diff on a problematic kernel and see if it solves your issue).applied on top of a66b2e5 - issue went away (either fixed or hidden now)
Cool! So here is the proper patch, with changelog added.
Regards,
Srivatsa S. Bhat-----------------------------------------------------------------------------
From: Srivatsa S. Bhat <[email protected]>
Subject: [PATCH] cpufreq: Fix cpufreq regression after suspend/resumeToralf Förster reported that the cpufreq ondemand governor behaves erratically (doesn't scale well) after a suspend/resume cycle. The problem was that the cpufreq subsystem's idea of the cpu frequencies differed from the actual frequencies set in the hardware after a suspend/resume cycle. Toralf bisected the problem to commit a66b2e5 (cpufreq: Preserve sysfs files across suspend/resume).
Among other (harmless) things, that commit skipped the call to
cpufreq_update_policy()in the resume path. Butcpufreq_update_policy()plays an important role during resume, because it is responsible for checking if the BIOS changed the cpu frequencies behind our back and resynchronize the cpufreq subsystem's knowledge of the cpu frequencies, and update them accordingly.
Quoted 6 lines
So, restore the call to
cpufreq_update_policy()in the resume path to fix the cpufreq regression.Reported-by Toralf Förster <[email protected]>Tested-by Toralf Förster <[email protected]>Signed-off-by Srivatsa S. Bhat <[email protected]>
Thanks Srivatsa, I'll queue it up as 3.11 (and 3.10-stable) material.
Thanks,
Rafael
Quoted 19 lines
---
drivers/cpufreq/cpufreq_stats.c | 1 + 1 file changed, 1 insertion(+)drivers/cpufreq/cpufreq_stats.c +1
-0@@ -349,6 +349,7 @@ static int __cpuinit cpufreq_stat_cpu_callback(struct notifier_block *nfb,switch (action) {case CPU_ONLINE : +case CPU_ONLINE_FROZEN :cpufreq_update_policy (cpu);break ;case CPU_DOWN_PREPARE :
Signature · 2 lines
I speak only for myself. Rafael J. Wysocki, Intel Open Source Technology Center.
Re: [PATCH] cpufreq: Fix cpufreq regression after suspend/resume
In reply to Srivatsa S. BhatI tested the patch several times on top of a66b2e5 - the origin issue is fixed but - erratically another issue now appears : all 4 cores are runs after wakeup at 2.6 GHz.
The temporary hot fix is to switch between governor performance and ondemand for all 4 cores.
Quoted 61 lines · Srivatsa S. Bhat
Quoted 8 lines · Toralf Förster
Quoted 4 lines · Srivatsa S. Bhat
Toralf, can you please
try out the below patch and see if it improves anything? (Don't revert anything, just apply the below diff on a problematic kernel and see if it solves your issue).applied on top of a66b2e5 - issue went away (either fixed or hidden now)
Cool! So here is the proper patch, with changelog added.
Regards,
Srivatsa S. Bhat-----------------------------------------------------------------------------
From: Srivatsa S. Bhat <[email protected]>
Subject: [PATCH] cpufreq: Fix cpufreq regression after suspend/resumeToralf Förster reported that the cpufreq ondemand governor behaves erratically (doesn't scale well) after a suspend/resume cycle. The problem was that the cpufreq subsystem's idea of the cpu frequencies differed from the actual frequencies set in the hardware after a suspend/resume cycle. Toralf bisected the problem to commit a66b2e5 (cpufreq: Preserve sysfs files across suspend/resume).
Among other (harmless) things, that commit skipped the call to
cpufreq_update_policy()in the resume path. Butcpufreq_update_policy()plays an important role during resume, because it is responsible for checking if the BIOS changed the cpu frequencies behind our back and resynchronize the cpufreq subsystem's knowledge of the cpu frequencies, and update them accordingly.So, restore the call to
cpufreq_update_policy()in the resume path to fix the cpufreq regression.Reported-by Toralf Förster <[email protected]>Tested-by Toralf Förster <[email protected]>Signed-off-by Srivatsa S. Bhat <[email protected]>---
drivers/cpufreq/cpufreq_stats.c | 1 + 1 file changed, 1 insertion(+)drivers/cpufreq/cpufreq_stats.c +1
-0@@ -349,6 +349,7 @@ static int __cpuinit cpufreq_stat_cpu_callback(struct notifier_block *nfb,switch (action) {case CPU_ONLINE : +case CPU_ONLINE_FROZEN :cpufreq_update_policy (cpu);break ;case CPU_DOWN_PREPARE :
Signature · 3 lines
MfG/Sincerely Toralf Förster pgp finger print: 7B1A 07F4 EC82 0F90 D4C2 8936 872A E508 7DB6 9DA3
Re: [PATCH] cpufreq: Fix cpufreq regression after suspend/resume
Hi Toralf,
Quoted 7 lines · Toralf Förster
I tested the patch several times on top of a66b2e5 - the origin issue is fixed but - erratically another issue now appears : all 4 cores are runs after wakeup at 2.6 GHz.
The temporary hot fix is to switch between governor performance and ondemand for all 4 cores.
Thanks for all your testing efforts. The commit a66b2e5 took a shortcut to achieve its goals but failed subtly in many aspects. Below is a patch (only compile-tested) which tries to achieve the goal (preserve sysfs files) in the proper way, by separating out the cpufreq-core bits from the sysfs bits. You might want to give it a try on current mainline and see if it improves anything.
Regards,
Srivatsa S. Bhat
(Applies on current mainline)
---
drivers/cpufreq/cpufreq.c | 144 ++++++++++++++++++++++++++------------------- 1 file changed, 82 insertions(+), 62 deletions(-)
drivers/cpufreq/cpufreq.c +82 -62
@@ -834,11 +834,8 @@ static int cpufreq_add_dev_interface(unsigned int cpu,struct cpufreq_policy *policy, struct device *dev) { - struct cpufreq_policy new_policy; struct freq_attr **drv_attr; -unsigned long flags;int ret =0 ; -unsigned int j;/* prepare interface data */ ret = kobject_init_and_add(&policy->kobj, &ktype_cpufreq,
@@ -870,31 +867,10 @@ static int cpufreq_add_dev_interface(unsigned int cpu, goto err_out_kobj_put; } -write_lock_irqsave (&cpufreq_driver_lock, flags); -for_each_cpu (j, policy->cpus) { -per_cpu (cpufreq_cpu_data, j) = policy; -per_cpu (cpufreq_policy_cpu, j) = policy->cpu; - } -write_unlock_irqrestore (&cpufreq_driver_lock, flags); - ret =cpufreq_add_dev_symlink (cpu, policy);if (ret) goto err_out_kobj_put; -memcpy (&new_policy, policy,sizeof (struct cpufreq_policy )); -/* assure that the starting sequence is run in __cpufreq_set_policy */ - policy->governor =NULL ; - -/* set default policy */ - ret =__cpufreq_set_policy (policy, &new_policy); - policy->user_policy.policy = policy->policy; - policy->user_policy.governor = policy->governor; - -if (ret) { -pr_debug ("setting policy failed\n" ); -if (cpufreq_driver->exit) - cpufreq_driver->exit (policy); - }return ret; err_out_kobj_put:
@@ -905,7 +881,7 @@ err_out_kobj_put:#ifdef CONFIG_HOTPLUG_CPU static int cpufreq_add_policy_cpu (unsigned int cpu,unsigned int sibling, -struct device *dev) +struct device *dev,bool init_sysfs) {struct cpufreq_policy *policy;int ret =0 , has_target = !!cpufreq_driver->target;
@@ -933,30 +909,25 @@ static int cpufreq_add_policy_cpu(unsigned int cpu, unsigned int sibling, __cpufreq_governor(policy, CPUFREQ_GOV_LIMITS); } - ret =sysfs_create_link (&dev->kobj, &policy->kobj,"cpufreq" ); -if (ret) { -cpufreq_cpu_put (policy); -return ret; +if (init_sysfs) { + ret =sysfs_create_link (&dev->kobj, &policy->kobj,"cpufreq" ); +if (ret) { +cpufreq_cpu_put (policy); +return ret; + } }return 0 ; }#endif -/** -* cpufreq_add_dev - add a CPU device -* -* Adds the cpufreq interface for a CPU device. -* -* The Oracle says: try running cpufreq registration/unregistration concurrently -* with with cpu hotplugging and all hell will break loose. Tried to clean this -* mess up, but more thorough testing is needed. - Mathieu -*/ -static int cpufreq_add_dev (struct device *dev,struct subsys_interface *sif) +static int __cpufreq_add_dev (struct device *dev,struct subsys_interface *sif, +bool init_sysfs) {unsigned int j, cpu = dev->id;int ret = -ENOMEM ;struct cpufreq_policy *policy; +struct cpufreq_policy new_policy;unsigned long flags;#ifdef CONFIG_HOTPLUG_CPU struct cpufreq_governor *gov;
@@ -984,7 +955,8 @@ static int cpufreq_add_dev(struct device *dev, struct subsys_interface *sif)struct cpufreq_policy *cp =per_cpu (cpufreq_cpu_data, sibling);if (cp &&cpumask_test_cpu (cpu, cp->related_cpus)) {read_unlock_irqrestore (&cpufreq_driver_lock, flags); -return cpufreq_add_policy_cpu (cpu, sibling, dev); +return cpufreq_add_policy_cpu (cpu, sibling, dev, + init_sysfs); } }read_unlock_irqrestore (&cpufreq_driver_lock, flags);
@@ -1049,9 +1021,33 @@ static int cpufreq_add_dev(struct device *dev, struct subsys_interface *sif) }#endif - ret =cpufreq_add_dev_interface (cpu, policy, dev); -if (ret) - goto err_out_unregister; +write_lock_irqsave (&cpufreq_driver_lock, flags); +for_each_cpu (j, policy->cpus) { +per_cpu (cpufreq_cpu_data, j) = policy; +per_cpu (cpufreq_policy_cpu, j) = policy->cpu; + } +write_unlock_irqrestore (&cpufreq_driver_lock, flags); + +if (init_sysfs) { + ret =cpufreq_add_dev_interface (cpu, policy, dev); +if (ret) + goto err_out_unregister; + } + +memcpy (&new_policy, policy,sizeof (struct cpufreq_policy )); +/* assure that the starting sequence is run in __cpufreq_set_policy */ + policy->governor =NULL ; + +/* set default policy */ + ret =__cpufreq_set_policy (policy, &new_policy); + policy->user_policy.policy = policy->policy; + policy->user_policy.governor = policy->governor; + +if (ret) { +pr_debug ("setting policy failed\n" ); +if (cpufreq_driver->exit) + cpufreq_driver->exit (policy); + }kobject_uevent (&policy->kobj,KOBJ_ADD );module_put (cpufreq_driver->owner);
@@ -1081,6 +1077,20 @@ module_out:return ret; } +/** +* cpufreq_add_dev - add a CPU device +* +* Adds the cpufreq interface for a CPU device. +* +* The Oracle says: try running cpufreq registration/unregistration concurrently +* with with cpu hotplugging and all hell will break loose. Tried to clean this +* mess up, but more thorough testing is needed. - Mathieu +*/ +static int cpufreq_add_dev (struct device *dev,struct subsys_interface *sif) +{ +return __cpufreq_add_dev (dev, sif, true); +} +static void update_policy_cpu (struct cpufreq_policy *policy,unsigned int cpu) {int j;
@@ -1106,7 +1116,7 @@ static void update_policy_cpu(struct cpufreq_policy *policy, unsigned int cpu)* This routine frees the rwsem before returning. */ static int __cpufreq_remove_dev (struct device *dev, -struct subsys_interface *sif) +struct subsys_interface *sif,bool remove_sysfs) {unsigned int cpu = dev->id, ret, cpus;unsigned long flags;
@@ -1145,9 +1155,9 @@ static int __cpufreq_remove_dev(struct device *dev,cpumask_clear_cpu (cpu, data->cpus);unlock_policy_rwsem_write (cpu); -if (cpu != data->cpu) { +if (cpu != data->cpu && remove_sysfs) {sysfs_remove_link (&dev->kobj,"cpufreq" ); - }else if (cpus >1 ) { + }else if (cpus >1 && remove_sysfs) {/* first sibling now owns the new sysfs dir */ cpu_dev =get_cpu_device (cpumask_first (data->cpus));sysfs_remove_link (&cpu_dev->kobj,"cpufreq" );
@@ -1184,26 +1194,25 @@ static int __cpufreq_remove_dev(struct device *dev,/* If cpu is last user of policy, free policy */ if (cpus ==1 ) { -lock_policy_rwsem_read (cpu); - kobj = &data->kobj; - cmp = &data->kobj_unregister; -unlock_policy_rwsem_read (cpu); -kobject_put (kobj); - -/* we need to make sure that the underlying kobj is actually -* not referenced anymore by anybody before we proceed with -* unloading. -*/ -pr_debug ("waiting for dropping of refcount\n" ); -wait_for_completion (cmp); -pr_debug ("wait complete\n" ); -if (cpufreq_driver->exit) cpufreq_driver->exit (data);free_cpumask_var (data->related_cpus);free_cpumask_var (data->cpus);kfree (data); + +if (remove_sysfs) { +lock_policy_rwsem_read (cpu); + kobj = &data->kobj; + cmp = &data->kobj_unregister; +unlock_policy_rwsem_read (cpu); +kobject_put (kobj); + +pr_debug ("waiting for dropping of refcount\n" ); +wait_for_completion (cmp); +pr_debug ("wait complete\n" ); + } + }else if (cpufreq_driver->target) {__cpufreq_governor (data,CPUFREQ_GOV_START );__cpufreq_governor (data,CPUFREQ_GOV_LIMITS );
@@ -1221,7 +1230,7 @@ static int cpufreq_remove_dev(struct device *dev, struct subsys_interface *sif)if (cpu_is_offline (cpu))return 0 ; - retval =__cpufreq_remove_dev (dev, sif); + retval =__cpufreq_remove_dev (dev, sif, true);return retval; }
@@ -1943,13 +1952,24 @@ static int __cpuinit cpufreq_cpu_callback(struct notifier_block *nfb,case CPU_ONLINE :cpufreq_add_dev (dev,NULL );break ; +case CPU_ONLINE_FROZEN : +__cpufreq_add_dev (dev,NULL , false); +break ; +case CPU_DOWN_PREPARE :case CPU_UP_CANCELED_FROZEN : -__cpufreq_remove_dev (dev,NULL ); +__cpufreq_remove_dev (dev,NULL , true); +break ; +case CPU_DOWN_PREPARE_FROZEN : +__cpufreq_remove_dev (dev,NULL , false);break ; +case CPU_DOWN_FAILED :cpufreq_add_dev (dev,NULL );break ; +case CPU_DOWN_FAILED_FROZEN : +__cpufreq_add_dev (dev,NULL , false); +break ; } }return NOTIFY_OK ;
Re: [PATCH] cpufreq: Fix cpufreq regression after suspend/resume
2013/7/11 Srivatsa S. Bhat <[email protected]>:
Quoted 234 lines
Hi Toralf,
Quoted 7 lines · Toralf Förster
I tested the patch several times on top of a66b2e5 - the origin issue is fixed but - erratically another issue now appears : all 4 cores are runs after wakeup at 2.6 GHz.
The temporary hot fix is to switch between governor performance and ondemand for all 4 cores.Thanks for all your testing efforts. The commit a66b2e5 took a shortcut to achieve its goals but failed subtly in many aspects. Below is a patch (only compile-tested) which tries to achieve the goal (preserve sysfs files) in the proper way, by separating out the cpufreq-core bits from the sysfs bits. You might want to give it a try on current mainline and see if it improves anything.
Regards,
Srivatsa S. Bhat(Applies on current mainline)
---drivers/cpufreq/cpufreq.c | 144 ++++++++++++++++++++++++++------------------- 1 file changed, 82 insertions(+), 62 deletions(-)drivers/cpufreq/cpufreq.c +56
-60@@ -834,11 +834,8 @@ static int cpufreq_add_dev_interface(unsigned int cpu,struct cpufreq_policy *policy, struct device *dev) { - struct cpufreq_policy new_policy; struct freq_attr **drv_attr; -unsigned long flags;int ret =0 ; -unsigned int j;/* prepare interface data */ ret = kobject_init_and_add(&policy->kobj, &ktype_cpufreq,@@ -870,31 +867,10 @@ static int cpufreq_add_dev_interface(unsigned int cpu, goto err_out_kobj_put; } -write_lock_irqsave (&cpufreq_driver_lock, flags); -for_each_cpu (j, policy->cpus) { -per_cpu (cpufreq_cpu_data, j) = policy; -per_cpu (cpufreq_policy_cpu, j) = policy->cpu; - } -write_unlock_irqrestore (&cpufreq_driver_lock, flags); - ret =cpufreq_add_dev_symlink (cpu, policy);if (ret) goto err_out_kobj_put; -memcpy (&new_policy, policy,sizeof (struct cpufreq_policy )); -/* assure that the starting sequence is run in __cpufreq_set_policy */ - policy->governor =NULL ; - -/* set default policy */ - ret =__cpufreq_set_policy (policy, &new_policy); - policy->user_policy.policy = policy->policy; - policy->user_policy.governor = policy->governor; - -if (ret) { -pr_debug ("setting policy failed\n" ); -if (cpufreq_driver->exit) - cpufreq_driver->exit (policy); - }return ret; err_out_kobj_put:@@ -905,7 +881,7 @@ err_out_kobj_put:#ifdef CONFIG_HOTPLUG_CPU static int cpufreq_add_policy_cpu (unsigned int cpu,unsigned int sibling, -struct device *dev) +struct device *dev,bool init_sysfs) {struct cpufreq_policy *policy;int ret =0 , has_target = !!cpufreq_driver->target;@@ -933,30 +909,25 @@ static int cpufreq_add_policy_cpu(unsigned int cpu, unsigned int sibling, __cpufreq_governor(policy, CPUFREQ_GOV_LIMITS); } - ret =sysfs_create_link (&dev->kobj, &policy->kobj,"cpufreq" ); -if (ret) { -cpufreq_cpu_put (policy); -return ret; +if (init_sysfs) { + ret =sysfs_create_link (&dev->kobj, &policy->kobj,"cpufreq" ); +if (ret) { +cpufreq_cpu_put (policy); +return ret; + } }return 0 ; }#endif -/** -* cpufreq_add_dev - add a CPU device -* -* Adds the cpufreq interface for a CPU device. -* -* The Oracle says: try running cpufreq registration/unregistration concurrently -* with with cpu hotplugging and all hell will break loose. Tried to clean this -* mess up, but more thorough testing is needed. - Mathieu -*/ -static int cpufreq_add_dev (struct device *dev,struct subsys_interface *sif) +static int __cpufreq_add_dev (struct device *dev,struct subsys_interface *sif, +bool init_sysfs) {unsigned int j, cpu = dev->id;int ret = -ENOMEM ;struct cpufreq_policy *policy; +struct cpufreq_policy new_policy;unsigned long flags;#ifdef CONFIG_HOTPLUG_CPU struct cpufreq_governor *gov;@@ -984,7 +955,8 @@ static int cpufreq_add_dev(struct device *dev, struct subsys_interface *sif)struct cpufreq_policy *cp =per_cpu (cpufreq_cpu_data, sibling);if (cp &&cpumask_test_cpu (cpu, cp->related_cpus)) {read_unlock_irqrestore (&cpufreq_driver_lock, flags); -return cpufreq_add_policy_cpu (cpu, sibling, dev); +return cpufreq_add_policy_cpu (cpu, sibling, dev, + init_sysfs); } }read_unlock_irqrestore (&cpufreq_driver_lock, flags);@@ -1049,9 +1021,33 @@ static int cpufreq_add_dev(struct device *dev, struct subsys_interface *sif) }#endif - ret =cpufreq_add_dev_interface (cpu, policy, dev); -if (ret) - goto err_out_unregister; +write_lock_irqsave (&cpufreq_driver_lock, flags); +for_each_cpu (j, policy->cpus) { +per_cpu (cpufreq_cpu_data, j) = policy; +per_cpu (cpufreq_policy_cpu, j) = policy->cpu; + } +write_unlock_irqrestore (&cpufreq_driver_lock, flags); + +if (init_sysfs) { + ret =cpufreq_add_dev_interface (cpu, policy, dev); +if (ret) + goto err_out_unregister; + } + +memcpy (&new_policy, policy,sizeof (struct cpufreq_policy )); +/* assure that the starting sequence is run in __cpufreq_set_policy */ + policy->governor =NULL ; + +/* set default policy */ + ret =__cpufreq_set_policy (policy, &new_policy); + policy->user_policy.policy = policy->policy; + policy->user_policy.governor = policy->governor; + +if (ret) { +pr_debug ("setting policy failed\n" ); +if (cpufreq_driver->exit) + cpufreq_driver->exit (policy); + }kobject_uevent (&policy->kobj,KOBJ_ADD );module_put (cpufreq_driver->owner);@@ -1081,6 +1077,20 @@ module_out:return ret; } +/** +* cpufreq_add_dev - add a CPU device +* +* Adds the cpufreq interface for a CPU device. +* +* The Oracle says: try running cpufreq registration/unregistration concurrently +* with with cpu hotplugging and all hell will break loose. Tried to clean this +* mess up, but more thorough testing is needed. - Mathieu +*/ +static int cpufreq_add_dev (struct device *dev,struct subsys_interface *sif) +{ +return __cpufreq_add_dev (dev, sif, true); +} +static void update_policy_cpu (struct cpufreq_policy *policy,unsigned int cpu) {int j;@@ -1106,7 +1116,7 @@ static void update_policy_cpu(struct cpufreq_policy *policy, unsigned int cpu)* This routine frees the rwsem before returning. */ static int __cpufreq_remove_dev (struct device *dev, -struct subsys_interface *sif) +struct subsys_interface *sif,bool remove_sysfs) {unsigned int cpu = dev->id, ret, cpus;unsigned long flags;@@ -1145,9 +1155,9 @@ static int __cpufreq_remove_dev(struct device *dev,cpumask_clear_cpu (cpu, data->cpus);unlock_policy_rwsem_write (cpu); -if (cpu != data->cpu) { +if (cpu != data->cpu && remove_sysfs) {sysfs_remove_link (&dev->kobj,"cpufreq" ); - }else if (cpus >1 ) { + }else if (cpus >1 && remove_sysfs) {/* first sibling now owns the new sysfs dir */ cpu_dev =get_cpu_device (cpumask_first (data->cpus));sysfs_remove_link (&cpu_dev->kobj,"cpufreq" );@@ -1184,26 +1194,25 @@ static int __cpufreq_remove_dev(struct device *dev,/* If cpu is last user of policy, free policy */ if (cpus ==1 ) { -lock_policy_rwsem_read (cpu); - kobj = &data->kobj; - cmp = &data->kobj_unregister; -unlock_policy_rwsem_read (cpu); -kobject_put (kobj); - -/* we need to make sure that the underlying kobj is actually -* not referenced anymore by anybody before we proceed with -* unloading. -*/ -pr_debug ("waiting for dropping of refcount\n" ); -wait_for_completion (cmp); -pr_debug ("wait complete\n" ); -if (cpufreq_driver->exit)
drivers/cpufreq/cpufreq.c static int __cpufreq_remove_dev(struct device *dev, · line 1207
Quoted 10 lines
drivers/cpufreq/cpufreq.c +5
-0cpufreq_driver->exit (data);free_cpumask_var (data->related_cpus);free_cpumask_var (data->cpus);kfree (data); + +if (remove_sysfs) { +lock_policy_rwsem_read (cpu); + kobj = &data->kobj; + cmp = &data->kobj_unregister;
This looks no right. "data" has already been freed but data-> kobj still is referenced.
Quoted 51 lines
drivers/cpufreq/cpufreq.c +21
-3+unlock_policy_rwsem_read (cpu); +kobject_put (kobj); + +pr_debug ("waiting for dropping of refcount\n" ); +wait_for_completion (cmp); +pr_debug ("wait complete\n" ); + } + }else if (cpufreq_driver->target) {__cpufreq_governor (data,CPUFREQ_GOV_START );__cpufreq_governor (data,CPUFREQ_GOV_LIMITS );@@ -1221,7 +1230,7 @@ static int cpufreq_remove_dev(struct device *dev, struct subsys_interface *sif)if (cpu_is_offline (cpu))return 0 ; - retval =__cpufreq_remove_dev (dev, sif); + retval =__cpufreq_remove_dev (dev, sif, true);return retval; }@@ -1943,13 +1952,24 @@ static int __cpuinit cpufreq_cpu_callback(struct notifier_block *nfb,case CPU_ONLINE :cpufreq_add_dev (dev,NULL );break ; +case CPU_ONLINE_FROZEN : +__cpufreq_add_dev (dev,NULL , false); +break ; +case CPU_DOWN_PREPARE :case CPU_UP_CANCELED_FROZEN : -__cpufreq_remove_dev (dev,NULL ); +__cpufreq_remove_dev (dev,NULL , true); +break ; +case CPU_DOWN_PREPARE_FROZEN : +__cpufreq_remove_dev (dev,NULL , false);break ; +case CPU_DOWN_FAILED :cpufreq_add_dev (dev,NULL );break ; +case CPU_DOWN_FAILED_FROZEN : +__cpufreq_add_dev (dev,NULL , false); +break ; } }return NOTIFY_OK ; --To unsubscribe from this list: send the line "unsubscribe cpufreq" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 2 lines
Best regards Tianyu Lan
Re: [PATCH] cpufreq: Fix cpufreq regression after suspend/resume
Quoted 29 lines · Lan Tianyu
2013/7/11 Srivatsa S. Bhat <[email protected]>:
Quoted 28 lines
Hi Toralf,
Quoted 7 lines · Toralf Förster
I tested the patch several times on top of a66b2e5 - the origin issue is fixed but - erratically another issue now appears : all 4 cores are runs after wakeup at 2.6 GHz.
The temporary hot fix is to switch between governor performance and ondemand for all 4 cores.Thanks for all your testing efforts. The commit a66b2e5 took a shortcut to achieve its goals but failed subtly in many aspects. Below is a patch (only compile-tested) which tries to achieve the goal (preserve sysfs files) in the proper way, by separating out the cpufreq-core bits from the sysfs bits. You might want to give it a try on current mainline and see if it improves anything.
Regards,
Srivatsa S. Bhat(Applies on current mainline)
---drivers/cpufreq/cpufreq.c | 144 ++++++++++++++++++++++++++------------------- 1 file changed, 82 insertions(+), 62 deletions(-)
[...]
Quoted 59 lines
Quoted 59 lines
drivers/cpufreq/cpufreq.c +17
-17+/** +* cpufreq_add_dev - add a CPU device +* +* Adds the cpufreq interface for a CPU device. +* +* The Oracle says: try running cpufreq registration/unregistration concurrently +* with with cpu hotplugging and all hell will break loose. Tried to clean this +* mess up, but more thorough testing is needed. - Mathieu +*/ +static int cpufreq_add_dev (struct device *dev,struct subsys_interface *sif) +{ +return __cpufreq_add_dev (dev, sif, true); +} +static void update_policy_cpu (struct cpufreq_policy *policy,unsigned int cpu) {int j;@@ -1106,7 +1116,7 @@ static void update_policy_cpu(struct cpufreq_policy *policy, unsigned int cpu)* This routine frees the rwsem before returning. */ static int __cpufreq_remove_dev (struct device *dev, -struct subsys_interface *sif) +struct subsys_interface *sif,bool remove_sysfs) {unsigned int cpu = dev->id, ret, cpus;unsigned long flags;@@ -1145,9 +1155,9 @@ static int __cpufreq_remove_dev(struct device *dev,cpumask_clear_cpu (cpu, data->cpus);unlock_policy_rwsem_write (cpu); -if (cpu != data->cpu) { +if (cpu != data->cpu && remove_sysfs) {sysfs_remove_link (&dev->kobj,"cpufreq" ); - }else if (cpus >1 ) { + }else if (cpus >1 && remove_sysfs) {/* first sibling now owns the new sysfs dir */ cpu_dev =get_cpu_device (cpumask_first (data->cpus));sysfs_remove_link (&cpu_dev->kobj,"cpufreq" );@@ -1184,26 +1194,25 @@ static int __cpufreq_remove_dev(struct device *dev,/* If cpu is last user of policy, free policy */ if (cpus ==1 ) { -lock_policy_rwsem_read (cpu); - kobj = &data->kobj; - cmp = &data->kobj_unregister; -unlock_policy_rwsem_read (cpu); -kobject_put (kobj); - -/* we need to make sure that the underlying kobj is actually -* not referenced anymore by anybody before we proceed with -* unloading. -*/ -pr_debug ("waiting for dropping of refcount\n" ); -wait_for_completion (cmp); -pr_debug ("wait complete\n" ); -if (cpufreq_driver->exit) cpufreq_driver->exit (data);
Quoted 12 lines
drivers/cpufreq/cpufreq.c static int __cpufreq_remove_dev(struct device *dev, · line 1207
Quoted 8 lines
drivers/cpufreq/cpufreq.c +5
-0free_cpumask_var (data->related_cpus);free_cpumask_var (data->cpus);kfree (data); + +if (remove_sysfs) { +lock_policy_rwsem_read (cpu); + kobj = &data->kobj; + cmp = &data->kobj_unregister;This looks no right. "data" has already been freed but data-> kobj still is referenced.
Oops! You are right. Hmm, this looks quite difficult to get right :( There are multiple challenges here:
1. The sysfs files must not be removed during cpu_down, and not initialized
during cpu_up. That would help us preserve the file permissions.
2. But we should ensure that we really do the cpufreq-core parts of the cpu initialization during cpu_up. If we fail to free some of the data-structures during cpu_down, the cpu_up callback will think that a full-init is not required and not do its job. That will make cpufreq behave erratically after suspend/resume and take us back to square one.
3. A full re-init in the cpu_up callback also involves memory allocations. So if we don't release the memory in the cpu_down callback, we'll end up in a memory leak.
I tried to address all these in this patch, but you found yet another serious loop-hole. I guess I'm out of ideas now... if anybody has any thoughts on how to get this right, then I'm all ears. Else, we'll just revert the original commit like Rafael suggested and leave it upto userspace to save and restore the permissions across suspend/resume if it wants ;-(
Quoted 15 lines
Quoted 15 lines
drivers/cpufreq/cpufreq.c +8
-0+unlock_policy_rwsem_read (cpu); +kobject_put (kobj); + +pr_debug ("waiting for dropping of refcount\n" ); +wait_for_completion (cmp); +pr_debug ("wait complete\n" ); + } + }else if (cpufreq_driver->target) {__cpufreq_governor (data,CPUFREQ_GOV_START );__cpufreq_governor (data,CPUFREQ_GOV_LIMITS );@@ -1221,7 +1230,7 @@ static int cpufreq_remove_dev(struct device *dev, struct subsys_interface *sif)if (cpu_is_offline (cpu))return 0 ;
Quoted 4 lines
drivers/cpufreq/cpufreq.c static int cpufreq_remove_dev(struct device *dev, struct subsys_interface *sif) · line 1235
Quoted 4 lines
drivers/cpufreq/cpufreq.c +1
-1- retval =__cpufreq_remove_dev (dev, sif); + retval =__cpufreq_remove_dev (dev, sif, true);return retval; }
Regards,
Srivatsa S. Bhat
Re: [PATCH] cpufreq: Fix cpufreq regression after suspend/resume
2013/7/11 Srivatsa S. Bhat <[email protected]>:
Quoted 4 lines · Lan Tianyu
Quoted 1 line
2013/7/11 Srivatsa S. Bhat <[email protected]>:
Oops! You are right. Hmm, this looks quite difficult to get right :( There are multiple challenges here:
If I understand correctly, the most concern is how to deal with per cpus' cpufreq_policy structure. If something wrong , please correct me.
Quoted 4 lines
1. The sysfs files must not be removed during
cpu_down, and not initializedduring
cpu_up. That would help us preserve the file permissions.
For this case, cpufreq_policy must be reserved since all related cpufreq data and kobj is also store into it. We can't release it.
Quoted 6 lines
2. But we should ensure that we really do the cpufreq-core parts of the cpu initialization during
cpu_up. If we fail to free some of the data-structures duringcpu_down, thecpu_upcallback will think that a full-init is not required and not do its job. That will make cpufreq behave erratically after suspend/resume and take us back to square one.
cpufreq_add_dev() checks whether the cpu has been attached cpufreq_cpu_data and associated kobj has been created. If yes, it would try to release it by cpufreq_cpu_put() and return directly. This seems to be conflicted with the above.
Quoted 4 lines
3. A full re-init in the
cpu_upcallback also involves memory allocations. So if we don't release the memory in thecpu_downcallback, we'll end up in a memory leak.
Even we remove the previous check, cpu scaling driver's init() also will change some data of struct cpufreq_policy and this is also what we don't like to see.
Quoted 6 lines
I tried to address all these in this patch, but you found yet another serious loop-hole. I guess I'm out of ideas now... if anybody has any thoughts on how to get this right, then I'm all ears. Else, we'll just revert the original commit like Rafael suggested and leave it upto userspace to save and restore the permissions across suspend/resume if it wants ;-(
How about implement scaling driver's suspend/resume callback()? Although this needs to be dealt with case by case. If one's callbacks hasn't been implemented, it would have to follow current rule.
Quoted 23 lines
Quoted 15 lines
Quoted 15 lines
drivers/cpufreq/cpufreq.c +8
-0+unlock_policy_rwsem_read (cpu); +kobject_put (kobj); + +pr_debug ("waiting for dropping of refcount\n" ); +wait_for_completion (cmp); +pr_debug ("wait complete\n" ); + } + }else if (cpufreq_driver->target) {__cpufreq_governor (data,CPUFREQ_GOV_START );__cpufreq_governor (data,CPUFREQ_GOV_LIMITS );@@ -1221,7 +1230,7 @@ static int cpufreq_remove_dev(struct device *dev, struct subsys_interface *sif)if (cpu_is_offline (cpu))return 0 ;Quoted 4 lines
drivers/cpufreq/cpufreq.c static int cpufreq_remove_dev(struct device *dev, struct subsys_interface *sif) · line 1235
Quoted 4 lines
drivers/cpufreq/cpufreq.c +1
-1- retval =__cpufreq_remove_dev (dev, sif); + retval =__cpufreq_remove_dev (dev, sif, true);return retval; }Regards,
Srivatsa S. Bhat
Signature · 2 lines
Best regards Tianyu Lan
Re: [PATCH] cpufreq: Fix cpufreq regression after suspend/resume
Quoted 9 lines · Lan Tianyu
2013/7/11 Srivatsa S. Bhat <[email protected]>:
Quoted 6 lines
I tried to address all these in this patch, but you found yet another serious loop-hole. I guess I'm out of ideas now... if anybody has any thoughts on how to get this right, then I'm all ears. Else, we'll just revert the original commit like Rafael suggested and leave it upto userspace to save and restore the permissions across suspend/resume if it wants ;-(
Quoted 4 lines
How about implement scaling driver's suspend/resume
callback()? Although this needs to be dealt with case by case. If one's callbacks hasn't been implemented, it would have to follow current rule.
Well, I'm now trying a slightly different approach at reorganizing the code, and so far I think I'll be able to get it right this time. Let's see how it goes.
Regards,
Srivatsa S. Bhat
Re: [PATCH] cpufreq: Fix cpufreq regression after suspend/resume
In reply to Srivatsa S. BhatQuoted 16 lines · Srivatsa S. Bhat
Oops! You are right. Hmm, this looks quite difficult to get right :( There are multiple challenges here:
1. The sysfs files must not be removed during
cpu_down, and not initializedduring
cpu_up. That would help us preserve the file permissions.
2. But we should ensure that we really do the cpufreq-core parts of the cpu initialization duringcpu_up. If we fail to free some of the data-structures duringcpu_down, thecpu_upcallback will think that a full-init is not required and not do its job. That will make cpufreq behave erratically after suspend/resume and take us back to square one.3. A full re-init in the
cpu_upcallback also involves memory allocations. So if we don't release the memory in thecpu_downcallback, we'll end up in a memory leak.
Quoted 5 lines
I tried to address all these in this patch, but you found yet another serious loop-hole. I guess I'm out of ideas now... if anybody has any thoughts on how to get this right, then I'm all ears. Else, we'll just revert the original commit like Rafael suggested and leave it upto userspace to save and restore the permissions across suspend/resume if it wants ;-(
Asking as a naive outsider who is completely unfamiliar with the code, why are any of these things at all troublesome?
Can't cpu_up tell the difference between activating a brand-new CPU and reactivating one that was present before but was temporarily disabled?
Doesn't cpu_up know which data structures get freed when an active CPU is temporarily deactivated?
Doesn't cpu_down know what memory gets allocated in cpu_up? Can't it deallocate just the right parts for the type of transition it is doing?
It sounds like you're really asking how to make sure that cpu_up and cpu_down both know what the other is doing, so that each can do the opposite of the other. That doesn't sound hard.
Alan Stern
Re: [PATCH] cpufreq: Fix cpufreq regression after suspend/resume
Quoted 34 lines · Alan Stern
Quoted 16 lines · Srivatsa S. Bhat
Oops! You are right. Hmm, this looks quite difficult to get right :( There are multiple challenges here:
1. The sysfs files must not be removed during
cpu_down, and not initializedduring
cpu_up. That would help us preserve the file permissions.
2. But we should ensure that we really do the cpufreq-core parts of the cpu initialization duringcpu_up. If we fail to free some of the data-structures duringcpu_down, thecpu_upcallback will think that a full-init is not required and not do its job. That will make cpufreq behave erratically after suspend/resume and take us back to square one.3. A full re-init in the
cpu_upcallback also involves memory allocations. So if we don't release the memory in thecpu_downcallback, we'll end up in a memory leak.Quoted 5 lines
I tried to address all these in this patch, but you found yet another serious loop-hole. I guess I'm out of ideas now... if anybody has any thoughts on how to get this right, then I'm all ears. Else, we'll just revert the original commit like Rafael suggested and leave it upto userspace to save and restore the permissions across suspend/resume if it wants ;-(
Asking as a naive outsider who is completely unfamiliar with the code, why are any of these things at all troublesome?
Can't
cpu_uptell the difference between activating a brand-new CPU and reactivating one that was present before but was temporarily disabled?Doesn't
cpu_upknow which data structures get freed when an active CPU is temporarily deactivated?
Quoted 8 lines
Doesn't
cpu_downknow what memory gets allocated incpu_up? Can't it deallocate just the right parts for the type of transition it is doing?It sounds like you're really asking how to make sure that
cpu_upandcpu_downboth know what the other is doing, so that each can do the opposite of the other. That doesn't sound hard.
It would not have been hard if the code had been already structured that way. Currently, the code is quite a bit entangled, and it doesn't distinguish the "temporarily deactivated" and the "fully torn-down" cases. To add to the mess, the code is generously sprinkled with notifiers that are invoked every now and then (which depend on previous init steps etc).
But the bottomline is that this is purely a code-reorganization issue, nothing more than that. And hopefully we'll be able to separate out the "temporarily deactivated" and the "fully down" cases reasonably well, such that we can preserve the sysfs file permissions easily across suspend/resume. That's the code reorganization I'm working on at the moment; I'll post it out as soon as I'm done.
Regards,
Srivatsa S. Bhat
Re: [PATCH] cpufreq: Fix cpufreq regression after suspend/resume
In reply to Toralf FörsterQuoted 2 lines · Toralf Förster
I tested the patch several times on top of a66b2e5 - the origin issue is fixed
Srivatsa's patch became commit f51e1eb63d ("cpufreq: Fix cpufreq regression after suspend/resume").
Quoted 2 lines
but - erratically another issue now appears : all 4 cores are runs
after wakeup at 2.6 GHz.
Well, a laptop I use seems to run into something related: the second of its two cores can get stuck at a fixed frequency after resume. Often it's its maximum frequency. That makes the CPU run hot, without actually doing much work. But it can also get stuck at its lowest frequency.
Please note that commit f51e1eb63d, which is part of v3.10.1-rc1, doesn't seem to cause this behavior. This issue can already be seen when running v3.10.0. So perhaps that commit just made this issue more noticeable to Toralf. But it's not clear to me what Toralf's origin(al?) issue actually was.
Anyhow, I suspect that the stuck frequency is a regression introduced in v3.10.0. But I have been unable to pinpoint it to a commit added in the v3.10 cycle through bisecting. I must have marked a commit good that was actually bad, just because a few suspend and resume cycles didn't trigger this issue. To be continued...
Quoted 2 lines
The temporary hot fix is to switch between governor performance and ondemand for all 4 cores.
That workaround works here too. I switch the core that is stuck at a particular frequency to some other governor and then back to ondemand.
Thanks,
Paul Bolle
Re: [PATCH] cpufreq: Fix cpufreq regression after suspend/resume
Quoted 1 line · Paul Bolle
I suspect that the stuck frequency is a regression introduced in v3.10.0.
The culprit apparently is commit a66b2e503f ("cpufreq: Preserve sysfs files across suspend/resume"). Srivatsa submitted a patch to revert that commit (see https://lkml.org/lkml/2013/7/11/661 ). That revert seems to fix this issue too.
Paul Bolle
Re: [PATCH] cpufreq: Fix cpufreq regression after suspend/resume
Quoted 9 lines · Paul Bolle
Quoted 1 line · Paul Bolle
I suspect that the stuck frequency is a regression introduced in v3.10.0.
The culprit apparently is commit
a66b2e503f("cpufreq: Preserve sysfs files across suspend/resume"). Srivatsa submitted a patch to revert that commit (see https://lkml.org/lkml/2013/7/11/661 ). That revert seems to fix this issue too.
Thanks a lot for your tests and for confirming that the complete revert fixes your cpufreq issue.
Regards,
Srivatsa S. Bhat
Quoted 10 lines · Toralf Förster
Quoted 4 lines · Srivatsa S. Bhat
Toralf, can you please
try out the below patch and see if it improves anything? (Don't revert anything, just apply the below diff on a problematic kernel and see if it solves your issue).applied on top of a66b2e5 - issue went away (either fixed or hidden now)
Thx
But if I do apply that patch on top of kernel 3.10 then it has no effect
- 3.10 shows the same issue.
Signature · 3 lines
MfG/Sincerely Toralf Förster pgp finger print: 7B1A 07F4 EC82 0F90 D4C2 8936 872A E508 7DB6 9DA3
Quoted 7 lines · Toralf Förster
Quoted 6 lines · Toralf Förster
Quoted 4 lines · Srivatsa S. Bhat
Toralf, can you please
try out the below patch and see if it improves anything? (Don't revert anything, just apply the below diff on a problematic kernel and see if it solves your issue).
Quoted 7 lines
Quoted 4 lines
applied on top of a66b2e5 - issue went away (either fixed or hidden now)
Thx
But if I do apply that patch on top of kernel 3.10 then it has no effect
- 3.10 shows the same issue.
Weird. So I think something *else* got broken by some other patch, _after_ a66b2e5. Viresh, do you have any suspect commits in mind?
Regards,
Srivatsa S. Bhat
Viresh Kumar
maintainerQuoted 10 lines · Srivatsa S. Bhat
Quoted 9 lines · Toralf Förster
Quoted 8 lines · Toralf Förster
Quoted 4 lines · Srivatsa S. Bhat
Toralf, can you please
try out the below patch and see if it improves anything? (Don't revert anything, just apply the below diff on a problematic kernel and see if it solves your issue).applied on top of a66b2e5 - issue went away (either fixed or hidden now)
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
Thx
But if I do apply that patch on top of kernel 3.10 then it has no effect
- 3.10 shows the same issue.Weird. So I think something *else* got broken by some other patch, _after_ a66b2e5. Viresh, do you have any suspect commits in mind?
If I was God I could have. But as I am a poor human being I need more details (preferably with git bisect) :)
--
Viresh
Quoted 17 lines · Viresh Kumar
Quoted 16 lines · Srivatsa S. Bhat
Quoted 14 lines · Toralf Förster
Quoted 10 lines · Toralf Förster
Quoted 4 lines · Srivatsa S. Bhat
Toralf, can you please
try out the below patch and see if it improves anything? (Don't revert anything, just apply the below diff on a problematic kernel and see if it solves your issue).applied on top of a66b2e5 - issue went away (either fixed or hidden now)
Thx
But if I do apply that patch on top of kernel 3.10 then it has no effect
- 3.10 shows the same issue.
Quoted 6 lines
Quoted 2 lines
Weird. So I think something *else* got broken by some other patch, _after_ a66b2e5. Viresh, do you have any suspect commits in mind?
If I was God I could have. But as I am a poor human being I need more details (preferably with git bisect) :)
Haha :)
Coming to git bisect, I think it needs to be done between a66b2e5 + this-patch and 3.10 + this-patch, to ensure that we look for problematic commits in-between a66b2e5 and 3.10 (to avoid the bisect result ending up at a66b2e5 again).
Regards,
Srivatsa S. Bhat
Viresh Kumar
maintainerQuoted 3 lines · Srivatsa S. Bhat
Coming to git bisect, I think it needs to be done between a66b2e5 + this-patch and 3.10 + this-patch, to ensure that we look for problematic commits in-between a66b2e5 and 3.10 (to avoid the bisect result ending up at a66b2e5 again).
Exactly!!
Quoted 7 lines · Viresh Kumar
Quoted 3 lines · Srivatsa S. Bhat
Coming to git bisect, I think it needs to be done between a66b2e5 + this-patch and 3.10 + this-patch, to ensure that we look for problematic commits in-between a66b2e5 and 3.10 (to avoid the bisect result ending up at a66b2e5 again).
Exactly!!
Problem : yesterday I wasn't even able to 100% validate the issue at commit a66b2e5 + f51e1eb just b/c there's another issue which erratically happens here: sometimes cpu0 is too (not only cpu 1-3). In that case the frequency are set to 2.6 GHz for each coret instead of 800 MHz for cpu0 and 2.601 GHz for cpu 1-3.
Furthermore I've the strong feeling, that the plugged in USB devices might have an effect. And in the mean while I would not even exclude the influence of the type of the 4 BOINC tasks (Asteorid@Home versus Einstein@Home / World Community Grid which I have to run with nice -19 in the back ground) at the bisect results.
So all my ongoing attempts to bisect weren't successful till now.
I just tested the latest git tree from today (v3.10-8615-g23e3a1d) - although f51e1eb is merged - the origin issue (cpu 1-3 set to 2.601 GHz after wakeup) is still here.
Signature · 3 lines
MfG/Sincerely Toralf Förster pgp finger print: 7B1A 07F4 EC82 0F90 D4C2 8936 872A E508 7DB6 9DA3
Viresh Kumar
maintainerQuoted 3 lines · Srivatsa S. Bhat
Coming to git bisect, I think it needs to be done between a66b2e5 + this-patch and 3.10 + this-patch, to ensure that we look for problematic commits in-between a66b2e5 and 3.10 (to avoid the bisect result ending up at a66b2e5 again).
Toralf,
Try reverting these patches in the order specified (revert both) 419e172145cf6c51d436a8bf4afcd17511f0ff79 c28375583b6471c1cb833a628ab6afb5b69079d0
They are touching ondemand governor and this may be creating some problems. But I am not sure about it, I must admit.
Viresh Kumar
maintainerQuoted 9 lines · Viresh Kumar
Quoted 3 lines · Srivatsa S. Bhat
Coming to git bisect, I think it needs to be done between a66b2e5 + this-patch and 3.10 + this-patch, to ensure that we look for problematic commits in-between a66b2e5 and 3.10 (to avoid the bisect result ending up at a66b2e5 again).
Toralf,
Try reverting these patches in the order specified (revert both)
419e172145cf6c51d436a8bf4afcd17511f0ff79
Ok, I did a mistake here (Thanks Srivatsa for telling me), this wasn't there in 3.10 but 3.11-rc1
Quoted 4 lines
c28375583b6471c1cb833a628ab6afb5b69079d0They are touching ondemand governor and this may be creating some problems. But I am not sure about it, I must admit.
So just try reverting about commit.
In case you are using acpi-cpufreq driver, please try reverting this before above one:
8673b83bf2f013379453b4779047bf3c6ae387e4
It is playing with how frequencies are reported and may be the culprit.
Quoted 5 lines · Viresh Kumar
Ok, I did a mistake here (Thanks Srivatsa for telling me), this wasn't there in 3.10 but 3.11-rc1
Quoted 2 lines
Quoted 2 lines
c28375583b6471c1cb833a628ab6afb5b69079d0
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
They are touching ondemand governor and this may be creating some problems. But I am not sure about it, I must admit.
So just try reverting about commit.
In case you are using acpi-cpufreq driver, please try reverting this before above one:
8673b83bf2f013379453b4779047bf3c6ae387e4
I reverted 8673b83b on top of 3.10.0 - no success, then I reverted c2837558 too - no success.
Signature · 3 lines
MfG/Sincerely Toralf Förster pgp finger print: 7B1A 07F4 EC82 0F90 D4C2 8936 872A E508 7DB6 9DA3
Viresh Kumar
maintainerQuoted 8 lines · Toralf Förster
Quoted 7 lines · Viresh Kumar
Ok, I did a mistake here (Thanks Srivatsa for telling me), this wasn't there in 3.10 but 3.11-rc1
Quoted 4 lines
Quoted 4 lines
c28375583b6471c1cb833a628ab6afb5b69079d0They are touching ondemand governor and this may be creating some problems. But I am not sure about it, I must admit.
Quoted 8 lines
Quoted 6 lines
So just try reverting about commit.
In case you are using acpi-cpufreq driver, please try reverting this before above one:
8673b83bf2f013379453b4779047bf3c6ae387e4I reverted
8673b83bon top of 3.10.0 - no success, then I revertedc2837558too - no success.
I assume that you have applied the patch given by Srivatsa earlier over these reverts?
In that case you can go for a revert I believe. There aren't many cpufreq patches between 3.10 and the commit where you tested successfully earlier.
But I now also suspect that this might have been caused by something outside cpufreq.
Quoted 2 lines · Viresh Kumar
I assume that you have applied the patch given by Srivatsa earlier over these reverts?
yes
Quoted 2 lines
But I now also suspect that this might have been caused by something outside cpufreq.
+1
At least the likelihood is big and the "saving" to narrow down bisecting just to the cpufreq patch is small compared to the "cost" of the risk of an unwanted side effect. OTOH not sure when I find a time frame for the bisect nightmare.
That's why I tweaked my s2ram script in this way:
+ if [[ "$1" = "mem" && "$(uname -r)" = "3.10.0" ]]; then
+ echo " tweak governor ..."
+ for g in performance ondemand; do
+ for i in 0 1 2 3; do
+ echo $g > /sys/devices/system/cpu/cpu$i/cpufreq/scaling_governor
+ done
+ done
+ echo 1 > /sys/devices/system/cpu/cpufreq/ondemand/ignore_nice
+ fi
Signature · 3 lines
MfG/Sincerely Toralf Förster pgp finger print: 7B1A 07F4 EC82 0F90 D4C2 8936 872A E508 7DB6 9DA3
Quoted 2 lines · Viresh Kumar
Can you give dumps of complete cpufreq directories for all cpus before and after s2r. That may give us some hint about the problem.
Because CPU0 is not affected I somehow arguing about nohz patches ...
Signature · 3 lines
MfG/Sincerely Toralf Förster pgp finger print: 7B1A 07F4 EC82 0F90 D4C2 8936 872A E508 7DB6 9DA3
Quoted 5 lines · Toralf Förster
Quoted 2 lines · Viresh Kumar
Can you give dumps of complete cpufreq directories for all cpus before and after s2r. That may give us some hint about the problem.
Because CPU0 is not affected I somehow arguing about nohz patches ...
Do you have CONFIG_NO_HZ_FULL set by chance?
Rafael
Quoted 12 lines · Rafael J. Wysocki
Quoted 5 lines · Toralf Förster
Quoted 2 lines · Viresh Kumar
Can you give dumps of complete cpufreq directories for all cpus before and after s2r. That may give us some hint about the problem.
Because CPU0 is not affected I somehow arguing about nohz patches ...
Do you have
CONFIG_NO_HZ_FULLset by chance?Rafael
no
$ zgrep CONFIG_NO_HZ_FULL /proc/config.gz
I do just have :
$ zgrep CONFIG_NO_HZ /proc/config.gz CONFIG_NO_HZ_COMMON=y CONFIG_NO_HZ_IDLE=y CONFIG_NO_HZ=y
Signature · 3 lines
MfG/Sincerely Toralf Förster pgp finger print: 7B1A 07F4 EC82 0F90 D4C2 8936 872A E508 7DB6 9DA3