af7f588d8f: will-it-scale.per_thread_ops -13.9% regression
kernel test robot
build botHello,
kernel test robot noticed a -13.9% regression of will-it-scale.per_thread_ops on:
commit: af7f588d8f7355bc4298dd1962d7826358fc95f0 ("sched: Introduce per-memory-map concurrency ID")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
testcase: will-it-scale
test machine: 224 threads 2 sockets (Sapphire Rapids) with 256G memory parameters:
test: context_switch1
cpufreq_governor: performance
test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two.
test-url: https://github.com/antonblanchard/will-it-scale
In addition to that, the commit also has significant impact on the following tests:
+------------------+----------------------------------------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_thread_ops -73.8% regression | | test machine | 224 threads 2 sockets (Sapphire Rapids) with 256G memory | | test parameters | cpufreq_governor=performance | | | mode=thread | | | nr_task=16 | | | test=context_switch1 | +------------------+----------------------------------------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_thread_ops -57.9% regression | | test machine | 104 threads 2 sockets (Skylake) with 192G memory | | test parameters | cpufreq_governor=performance | | | mode=thread | | | nr_task=16 | | | test=context_switch1 | +------------------+----------------------------------------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_thread_ops -85.0% regression | | test machine | 104 threads 2 sockets (Skylake) with 192G memory | | test parameters | cpufreq_governor=performance | | | mode=thread | | | nr_task=50% | | | test=context_switch1 | +------------------+----------------------------------------------------------------------------------------------------+ | testcase: change | vm-scalability: vm-scalability.throughput -9.0% regression | | test machine | 96 threads 2 sockets Intel(R) Xeon(R) Platinum 8260L CPU @ 2.40GHz (Cascade Lake) with 128G memory | | test parameters | cpufreq_governor=performance | | | runtime=300s | | | size=2T | | | test=shm-xread-seq-mt | +------------------+----------------------------------------------------------------------------------------------------+
FYI, we noticed that commit 223baf9d17f2 (sched: Fix performance regression introduced by mm_cid) fixed a sysbench regression, but will-it-scale context_switch1 benchmark still saw a regression on this fix commit.
Furthermore, we applied the code diff in below link [1] on mainline, and the will-it-scale score was restored to the original level before this patch.
[1] https://lore.kernel.org/lkml/[email protected]/
If you fix the issue, kindly add following tag
| Reported-by: kernel test robot <[email protected]>
| Link: https://lore.kernel.org/oe-lkp/[email protected]
Details are as below:
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/tbox_group/test/testcase:
gcc-11/performance/x86_64-rhel-8.3/debian-11.1-x86_64-20220510.cgz/lkp-spr-r02/context_switch1/will-it-scale
commit:
99babd04b2 ("selftests/rseq: Implement rseq numa node id field selftest") af7f588d8f ("sched: Introduce per-memory-map concurrency ID")
99babd04b2505471 af7f588d8f7355bc4298dd1962d
---------------- ---------------------------
%stddev %change %stddev
\ | \
629260 -1.2% 621753 will-it-scale.1.linear
629260 -1.2% 621753 will-it-scale.1.processes
613314 -1.5% 604163 will-it-scale.1.threads
70477120 -1.2% 69636432 will-it-scale.112.linear
5615798 ± 4% -72.0% 1571768 ± 3% will-it-scale.112.threads
24.94 ± 2% -90.0% 2.50 ± 4% will-it-scale.112.threads_idle
1.057e+08 -1.2% 1.045e+08 will-it-scale.168.linear
56978825 -20.6% 45250154 ± 2% will-it-scale.168.threads
7.49 -73.8% 1.96 ± 28% will-it-scale.168.threads_idle
1.41e+08 -1.2% 1.393e+08 will-it-scale.224.linear
95368601 +1.9% 97207305 will-it-scale.224.processes
1.082e+08 +1.4% 1.097e+08 will-it-scale.224.threads
35238560 -1.2% 34818216 will-it-scale.56.linear
8809640 ± 2% -76.2% 2092567 will-it-scale.56.threads
68.53 -3.4% 66.22 will-it-scale.56.threads_idle
328629 -13.9% 282968 will-it-scale.per_thread_ops
0.68 +3.2% 0.70 will-it-scale.scalability
4.074e+08 ± 12% -37.1% 2.564e+08 ± 8% will-it-scale.time.involuntary_context_switches
17476 +1.4% 17713 will-it-scale.time.system_time
1623 -15.7% 1369 will-it-scale.time.user_time
6.135e+09 -8.6% 5.607e+09 will-it-scale.time.voluntary_context_switches
4.584e+08 -4.2% 4.39e+08 will-it-scale.workload
4.804e+09 -13.3% 4.166e+09 cpuidle..usage
0.12 ± 4% -0.0 0.09 ± 3% mpstat.cpu.all.soft%
22357676 -62.5% 8384327 ± 4% vmstat.memory.cache
59747111 -6.3% 55960083 vmstat.system.cs
433036 ± 12% +511.7% 2648925 ± 42% numa-numastat.node0.local_node
572995 ± 11% +391.1% 2814064 ± 39% numa-numastat.node0.numa_hit
8059420 -69.0% 2496648 ± 43% numa-numastat.node1.local_node
8122292 -68.6% 2550541 ± 42% numa-numastat.node1.numa_hit
2.202e+08 ± 5% -82.0% 39691191 ± 11% turbostat.C1
0.93 ± 5% -0.8 0.09 ± 14% turbostat.C1%
1.101e+08 ± 2% -56.5% 47858373 turbostat.C1E
3.22 -0.5 2.73 turbostat.POLL%
231050 +1893.9% 4607008 ± 4% meminfo.Active
230970 +1894.6% 4606928 ± 4% meminfo.Active(anon)
219559 ± 5% -22.6% 170030 ± 15% meminfo.AnonHugePages
572890 -24.2% 434172 meminfo.AnonPages
21995823 -62.4% 8263088 ± 4% meminfo.Cached
25221015 -55.0% 11347952 ± 3% meminfo.Committed_AS
19440153 -93.9% 1192582 ± 23% meminfo.Inactive
19440011 -93.9% 1192439 ± 23% meminfo.Inactive(anon)
214830 -14.7% 183232 meminfo.KReclaimable
25424026 -56.0% 11175036 ± 3% meminfo.Memused
214830 -14.7% 183232 meminfo.SReclaimable
19098913 -71.9% 5366191 ± 6% meminfo.Shmem
25832239 -34.2% 16986009 meminfo.max_used_kB
110043 ± 15% -88.5% 12679 ± 98% sched_debug.cfs_rq:/.MIN_vruntime.avg
2804771 ± 2% -69.8% 846737 ± 85% sched_debug.cfs_rq:/.MIN_vruntime.max
504968 ± 9% -82.6% 88033 ± 76% sched_debug.cfs_rq:/.MIN_vruntime.stddev
110043 ± 15% -88.5% 12679 ± 98% sched_debug.cfs_rq:/.max_vruntime.avg
2804771 ± 2% -69.8% 846738 ± 85% sched_debug.cfs_rq:/.max_vruntime.max
504968 ± 9% -82.6% 88033 ± 76% sched_debug.cfs_rq:/.max_vruntime.stddev
854772 ± 2% +12.8% 964472 ± 2% sched_debug.cfs_rq:/.min_vruntime.stddev
73087 ± 59% +734.2% 609692 ± 44% sched_debug.cfs_rq:/.spread0.max
861849 ± 2% +12.1% 966253 ± 2% sched_debug.cfs_rq:/.spread0.stddev
322.78 ± 2% -14.0% 277.54 ± 6% sched_debug.cfs_rq:/.util_est_enqueued.avg
470921 ± 4% +11.5% 525288 ± 4% sched_debug.cpu.avg_idle.avg
10146 ± 89% +197.5% 30184 ± 27% sched_debug.cpu.avg_idle.min
166.82 ± 15% -82.9% 28.54 ± 6% sched_debug.cpu.clock.stddev
0.00 ± 15% -78.3% 0.00 ± 16% sched_debug.cpu.next_balance.stddev
29334462 -9.5% 26550823 sched_debug.cpu.nr_switches.avg
9400046 -12.1% 8262067 sched_debug.cpu.nr_switches.stddev
1494 ± 50% +1.6e+05% 2458393 ± 56% numa-meminfo.node0.Active
1459 ± 53% +1.7e+05% 2458381 ± 56% numa-meminfo.node0.Active(anon)
193090 ± 23% +315.9% 803087 ± 36% numa-meminfo.node0.Inactive
193026 ± 23% +316.0% 803069 ± 36% numa-meminfo.node0.Inactive(anon)
9933 ± 9% +50.4% 14936 ± 4% numa-meminfo.node0.PageTables
6124 ± 15% +49221.4% 3020604 ± 55% numa-meminfo.node0.Shmem
229095 +841.3% 2156586 ± 64% numa-meminfo.node1.Active
229051 +841.5% 2156518 ± 64% numa-meminfo.node1.Active(anon)
145346 ± 16% -45.5% 79245 ± 34% numa-meminfo.node1.AnonHugePages
383868 ± 12% -49.7% 192976 ± 22% numa-meminfo.node1.AnonPages
19095555 -87.0% 2485880 ± 58% numa-meminfo.node1.FilePages
19201636 -98.0% 390419 ± 19% numa-meminfo.node1.Inactive
19201557 -98.0% 390295 ± 19% numa-meminfo.node1.Inactive(anon)
102686 ± 9% -34.7% 67069 ± 15% numa-meminfo.node1.KReclaimable
2789340 ± 3% -49.4% 1410452 ± 52% numa-meminfo.node1.Mapped
1.113e+08 +15.3% 1.283e+08 numa-meminfo.node1.MemFree
20783078 -81.8% 3782627 ± 38% numa-meminfo.node1.MemUsed
16570 ± 5% -38.1% 10254 ± 7% numa-meminfo.node1.PageTables
102686 ± 9% -34.7% 67069 ± 15% numa-meminfo.node1.SReclaimable
19047079 -87.6% 2354397 ± 60% numa-meminfo.node1.Shmem
282546 ± 6% -14.1% 242658 ± 7% numa-meminfo.node1.Slab
364.56 ± 53% +1.7e+05% 615291 ± 56% numa-vmstat.node0.nr_active_anon
48203 ± 23% +317.2% 201111 ± 35% numa-vmstat.node0.nr_inactive_anon
2477 ± 10% +50.9% 3738 ± 4% numa-vmstat.node0.nr_page_table_pages
1530 ± 15% +49298.5% 756126 ± 55% numa-vmstat.node0.nr_shmem
364.56 ± 53% +1.7e+05% 615292 ± 56% numa-vmstat.node0.nr_zone_active_anon
48203 ± 23% +317.2% 201111 ± 35% numa-vmstat.node0.nr_zone_inactive_anon
573038 ± 11% +390.9% 2813153 ± 39% numa-vmstat.node0.numa_hit
433108 ± 12% +511.4% 2648015 ± 42% numa-vmstat.node0.numa_local
57253 +843.2% 540005 ± 64% numa-vmstat.node1.nr_active_anon
95911 ± 12% -49.6% 48299 ± 22% numa-vmstat.node1.nr_anon_pages
4773715 -87.0% 622469 ± 58% numa-vmstat.node1.nr_file_pages
27815434 +15.3% 32063992 numa-vmstat.node1.nr_free_pages
4800160 -98.0% 97769 ± 19% numa-vmstat.node1.nr_inactive_anon
697703 ± 3% -49.3% 353752 ± 52% numa-vmstat.node1.nr_mapped
4135 ± 5% -37.9% 2567 ± 7% numa-vmstat.node1.nr_page_table_pages
4761596 -87.6% 589599 ± 60% numa-vmstat.node1.nr_shmem
25670 ± 9% -34.7% 16769 ± 15% numa-vmstat.node1.nr_slab_reclaimable
57253 +843.2% 540006 ± 64% numa-vmstat.node1.nr_zone_active_anon
4800159 -98.0% 97769 ± 19% numa-vmstat.node1.nr_zone_inactive_anon
8122528 -68.6% 2548812 ± 42% numa-vmstat.node1.numa_hit
8059656 -69.0% 2494919 ± 43% numa-vmstat.node1.numa_local
57680 +1900.0% 1153633 ± 4% proc-vmstat.nr_active_anon
143171 -24.2% 108550 proc-vmstat.nr_anon_pages
5919284 +6.0% 6274221 proc-vmstat.nr_dirty_background_threshold
11853042 +6.0% 12563784 proc-vmstat.nr_dirty_threshold
5493638 -62.4% 2067724 ± 4% proc-vmstat.nr_file_pages
59564263 +6.0% 63118840 proc-vmstat.nr_free_pages
4854706 -93.9% 298169 ± 23% proc-vmstat.nr_inactive_anon
6640 -4.7% 6328 proc-vmstat.nr_page_table_pages
4769410 -71.8% 1343500 ± 6% proc-vmstat.nr_shmem
53693 -14.7% 45811 proc-vmstat.nr_slab_reclaimable
57680 +1900.0% 1153633 ± 4% proc-vmstat.nr_zone_active_anon
4854706 -93.9% 298169 ± 23% proc-vmstat.nr_zone_inactive_anon
683375 ± 10% -50.0% 341609 ± 4% proc-vmstat.numa_hint_faults
519396 ± 17% -60.2% 206845 ± 27% proc-vmstat.numa_hint_faults_local
8698455 -38.3% 5371241 proc-vmstat.numa_hit
938.89 ± 18% -55.8% 414.71 ± 8% proc-vmstat.numa_huge_pte_updates
8495696 -39.4% 5152209 proc-vmstat.numa_local
1313649 ± 5% -46.7% 700042 ± 3% proc-vmstat.numa_pte_updates
61945 +1240.4% 830307 ± 21% proc-vmstat.pgactivate
8928138 -37.8% 5550603 proc-vmstat.pgalloc_normal
2256846 ± 2% -11.1% 2006618 proc-vmstat.pgfault
1520379 ± 2% +268.9% 5607989 proc-vmstat.pgfree
0.38 ± 4% +115.9% 0.82 ± 11% perf-stat.i.MPKI
7.631e+10 -4.8% 7.269e+10 perf-stat.i.branch-instructions
0.67 -0.1 0.61 perf-stat.i.branch-miss-rate%
4.989e+08 -5.5% 4.715e+08 perf-stat.i.branch-misses
6418393 -8.1% 5897662 ± 5% perf-stat.i.cache-misses
50457351 ± 5% +51.6% 76473775 ± 10% perf-stat.i.cache-references
60968060 -6.7% 56856550 perf-stat.i.context-switches
2.14 ± 2% +13.1% 2.42 perf-stat.i.cpi
4.35e+11 +1.7% 4.424e+11 perf-stat.i.cpu-cycles
1058 ± 14% -21.6% 830.16 ± 4% perf-stat.i.cpu-migrations
385252 ± 4% +63.0% 627819 ± 4% perf-stat.i.cycles-between-cache-misses
0.01 ± 22% -0.0 0.00 ± 12% perf-stat.i.dTLB-load-miss-rate%
2628303 ± 7% -65.0% 920193 ± 5% perf-stat.i.dTLB-load-misses
1.104e+11 -6.0% 1.038e+11 perf-stat.i.dTLB-loads
0.00 ± 13% +0.0 0.00 ± 12% perf-stat.i.dTLB-store-miss-rate%
242451 ± 5% -19.0% 196450 ± 11% perf-stat.i.dTLB-store-misses
6.626e+10 -8.4% 6.071e+10 perf-stat.i.dTLB-stores
3.816e+11 -5.8% 3.595e+11 perf-stat.i.instructions
0.80 -6.6% 0.75 perf-stat.i.ipc
1.94 +1.7% 1.97 perf-stat.i.metric.GHz
504.83 ± 2% -4.4% 482.60 perf-stat.i.metric.K/sec
1129 -6.2% 1058 perf-stat.i.metric.M/sec
6259 ± 2% -11.2% 5560 perf-stat.i.minor-faults
58.32 ± 3% +29.9 88.22 ± 6% perf-stat.i.node-load-miss-rate%
1117525 +34.5% 1502746 ± 10% perf-stat.i.node-load-misses
458656 ± 5% -70.6% 134710 ± 30% perf-stat.i.node-loads
6260 ± 2% -11.2% 5562 perf-stat.i.page-faults
0.13 ± 5% +61.1% 0.22 ± 10% perf-stat.overall.MPKI
12.66 ± 5% -4.9 7.76 ± 14% perf-stat.overall.cache-miss-rate%
1.15 +7.5% 1.24 perf-stat.overall.cpi
67823 +10.3% 74790 ± 4% perf-stat.overall.cycles-between-cache-misses
0.00 ± 7% -0.0 0.00 ± 5% perf-stat.overall.dTLB-load-miss-rate%
0.87 -7.0% 0.81 perf-stat.overall.ipc
69.79 ± 2% +21.7 91.45 ± 3% perf-stat.overall.node-load-miss-rate%
260155 -1.1% 257287 perf-stat.overall.path-length
7.514e+10 -4.2% 7.195e+10 perf-stat.ps.branch-instructions
4.914e+08 -5.1% 4.664e+08 perf-stat.ps.branch-misses
50614911 ± 5% +52.5% 77201196 ± 10% perf-stat.ps.cache-references
60049636 -6.4% 56232111 perf-stat.ps.context-switches
4.331e+11 +1.8% 4.408e+11 perf-stat.ps.cpu-cycles
1048 ± 13% -21.2% 826.67 ± 4% perf-stat.ps.cpu-migrations
2706759 ± 7% -65.6% 930018 ± 5% perf-stat.ps.dTLB-load-misses
1.087e+11 -5.5% 1.027e+11 perf-stat.ps.dTLB-loads
242516 ± 5% -19.3% 195733 ± 11% perf-stat.ps.dTLB-store-misses
6.523e+10 -7.9% 6.005e+10 perf-stat.ps.dTLB-stores
3.757e+11 -5.3% 3.558e+11 perf-stat.ps.instructions
6291 ± 2% -11.8% 5547 perf-stat.ps.minor-faults
1103763 +36.5% 1507093 ± 10% perf-stat.ps.node-load-misses
478062 ± 5% -71.3% 137270 ± 29% perf-stat.ps.node-loads
6292 ± 2% -11.8% 5549 perf-stat.ps.page-faults
1.192e+14 -5.3% 1.129e+14 perf-stat.total.instructions
20.06 ± 2% -20.1 0.00 perf-profile.calltrace.cycles-pp.finish_task_switch.__schedule.schedule.pipe_read.vfs_read
22.16 ± 2% -17.8 4.38 ± 3% perf-profile.calltrace.cycles-pp.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event
22.02 ± 2% -17.7 4.35 ± 3% perf-profile.calltrace.cycles-pp.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow
20.05 ± 2% -16.1 3.97 ± 3% perf-profile.calltrace.cycles-pp.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward
58.25 -9.4 48.81 perf-profile.calltrace.cycles-pp.__libc_read
11.88 ± 2% -9.3 2.53 ± 3% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule
11.73 -9.2 2.48 ± 2% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
10.32 ± 2% -9.1 1.18 ± 3% perf-profile.calltrace.cycles-pp.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
11.28 -8.8 2.45 ± 6% perf-profile.calltrace.cycles-pp.__libc_write
10.60 ± 2% -8.5 2.08 ± 3% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch
10.69 -8.4 2.33 ± 6% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_write
10.59 ± 2% -8.3 2.26 ± 3% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_read.vfs_read
10.62 -8.3 2.31 ± 6% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
10.49 -8.3 2.24 ± 5% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
10.36 ± 2% -8.2 2.19 ± 3% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_read
10.61 ± 3% -8.1 2.48 ± 13% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
8.71 ± 2% -8.1 0.66 ± 54% perf-profile.calltrace.cycles-pp.unwind_next_frame.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain
56.47 -8.0 48.46 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_read
56.28 -7.8 48.43 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
9.88 -7.8 2.08 ± 3% perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
9.76 -7.7 2.02 ± 3% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
9.76 -7.7 2.10 ± 5% perf-profile.calltrace.cycles-pp.pipe_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
9.48 ± 2% -7.5 1.95 ± 3% perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule
55.87 -7.5 48.35 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
55.45 -7.2 48.28 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
8.86 ± 2% -7.0 1.84 ± 3% perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair.__schedule
8.63 ± 2% -6.7 1.90 ± 5% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.vfs_write.ksys_write.do_syscall_64
8.41 ± 2% -6.6 1.80 ± 3% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair
8.42 ± 2% -6.6 1.86 ± 5% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.vfs_write.ksys_write
54.70 -6.5 48.19 perf-profile.calltrace.cycles-pp.pipe_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
8.30 ± 2% -6.5 1.85 ± 5% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.vfs_write
8.18 ± 2% -6.5 1.73 ± 3% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity
8.10 ± 2% -6.4 1.72 ± 3% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr
8.20 ± 2% -6.4 1.83 ± 5% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
6.10 ± 2% -6.1 0.00 perf-profile.calltrace.cycles-pp.unwind_next_frame.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
7.43 ± 2% -6.0 1.47 ± 3% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime
6.92 ± 2% -5.4 1.50 ± 3% perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry
6.85 ± 2% -5.4 1.46 ± 3% perf-profile.calltrace.cycles-pp.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
6.68 ± 2% -5.3 1.43 ± 3% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common
6.70 ± 2% -5.2 1.46 ± 4% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule_idle.do_idle
6.50 ± 2% -5.1 1.39 ± 3% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule_idle
6.47 ± 2% -5.1 1.37 ± 3% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function
6.40 ± 2% -5.0 1.36 ± 3% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up
5.69 ± 2% -4.9 0.84 ± 5% perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
5.85 ± 2% -4.7 1.14 ± 3% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_wakeup_template
5.83 ± 2% -4.6 1.25 ± 4% perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule.pipe_read.vfs_read
5.66 ± 2% -4.4 1.21 ± 4% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule.pipe_read
52.13 -4.4 47.75 perf-profile.calltrace.cycles-pp.schedule.pipe_read.vfs_read.ksys_read.do_syscall_64
5.50 ± 2% -4.3 1.16 ± 3% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule
51.86 -4.1 47.71 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_read.vfs_read.ksys_read
3.00 ± 6% -2.4 0.58 ± 2% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
3.00 ± 6% -2.4 0.58 ± 2% perf-profile.calltrace.cycles-pp.mwait_idle_with_hints.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
28.01 +19.1 47.12 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
27.98 +19.1 47.11 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
27.90 +19.2 47.09 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
28.25 +19.3 47.57 perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
13.04 +30.9 43.94 perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
12.84 +31.1 43.91 perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
0.00 +41.3 41.26 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prepare_task_switch.__schedule.schedule_idle
0.00 +41.5 41.49 perf-profile.calltrace.cycles-pp._raw_spin_lock.prepare_task_switch.__schedule.schedule_idle.do_idle
0.00 +41.6 41.57 perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry
0.00 +41.8 41.80 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prepare_task_switch.__schedule.schedule
0.00 +42.0 42.01 perf-profile.calltrace.cycles-pp._raw_spin_lock.prepare_task_switch.__schedule.schedule.pipe_read
0.00 +42.1 42.08 perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule.pipe_read.vfs_read
27.66 ± 2% -21.6 6.09 ± 3% perf-profile.children.cycles-pp.perf_tp_event
26.84 ± 2% -21.0 5.84 ± 3% perf-profile.children.cycles-pp.__perf_event_overflow
26.57 ± 2% -20.8 5.79 ± 3% perf-profile.children.cycles-pp.perf_event_output_forward
21.19 ± 2% -20.4 0.79 ± 8% perf-profile.children.cycles-pp.finish_task_switch
24.08 ± 2% -19.2 4.85 ± 3% perf-profile.children.cycles-pp.perf_prepare_sample
22.32 ± 2% -17.8 4.53 ± 3% perf-profile.children.cycles-pp.perf_callchain
22.19 ± 2% -17.7 4.51 ± 3% perf-profile.children.cycles-pp.get_perf_callchain
68.01 -16.5 51.52 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
67.76 -16.3 51.48 perf-profile.children.cycles-pp.do_syscall_64
20.43 ± 2% -16.3 4.15 ± 3% perf-profile.children.cycles-pp.perf_callchain_kernel
16.08 ± 2% -12.8 3.30 ± 3% perf-profile.children.cycles-pp.unwind_next_frame
12.85 ± 2% -10.0 2.80 ± 4% perf-profile.children.cycles-pp.perf_trace_sched_switch
58.48 -9.6 48.86 perf-profile.children.cycles-pp.__libc_read
11.84 -9.3 2.50 ± 3% perf-profile.children.cycles-pp.cpuidle_idle_call
12.30 ± 2% -9.1 3.21 ± 4% perf-profile.children.cycles-pp.__libc_write
10.55 ± 2% -8.4 2.14 ± 3% perf-profile.children.cycles-pp.__unwind_start
11.30 ± 2% -8.4 2.94 ± 3% perf-profile.children.cycles-pp.ksys_write
10.60 ± 2% -8.3 2.26 ± 3% perf-profile.children.cycles-pp.dequeue_task_fair
10.42 ± 2% -8.2 2.20 ± 3% perf-profile.children.cycles-pp.dequeue_entity
10.93 ± 2% -8.1 2.84 ± 3% perf-profile.children.cycles-pp.vfs_write
9.96 -7.9 2.09 ± 3% perf-profile.children.cycles-pp.cpuidle_enter
9.92 -7.8 2.08 ± 3% perf-profile.children.cycles-pp.cpuidle_enter_state
9.80 -7.7 2.11 ± 5% perf-profile.children.cycles-pp.pipe_write
55.91 -7.5 48.37 perf-profile.children.cycles-pp.ksys_read
9.65 ± 2% -7.5 2.12 ± 3% perf-profile.children.cycles-pp.update_curr
55.47 -7.2 48.30 perf-profile.children.cycles-pp.vfs_read
8.95 ± 2% -7.0 1.98 ± 3% perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime
8.65 ± 2% -6.7 1.91 ± 5% perf-profile.children.cycles-pp.__wake_up_common_lock
54.85 -6.6 48.21 perf-profile.children.cycles-pp.pipe_read
8.43 ± 2% -6.6 1.86 ± 5% perf-profile.children.cycles-pp.__wake_up_common
8.35 ± 2% -6.5 1.85 ± 5% perf-profile.children.cycles-pp.autoremove_wake_function
8.23 ± 2% -6.4 1.84 ± 5% perf-profile.children.cycles-pp.try_to_wake_up
6.86 ± 2% -5.4 1.46 ± 3% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
5.79 ± 2% -4.9 0.87 ± 5% perf-profile.children.cycles-pp.poll_idle
52.14 -4.3 47.81 perf-profile.children.cycles-pp.schedule
3.81 ± 5% -3.7 0.15 ± 6% perf-profile.children.cycles-pp.__switch_to
4.46 ± 2% -3.5 0.91 ± 3% perf-profile.children.cycles-pp.__orc_find
3.03 ± 6% -2.4 0.58 ± 2% perf-profile.children.cycles-pp.mwait_idle_with_hints
3.03 ± 6% -2.4 0.58 ± 2% perf-profile.children.cycles-pp.intel_idle
2.85 ± 2% -2.3 0.58 ± 3% perf-profile.children.cycles-pp.stack_access_ok
2.44 ± 2% -2.0 0.48 ± 3% perf-profile.children.cycles-pp.unwind_get_return_address
2.08 ± 2% -1.7 0.41 ± 3% perf-profile.children.cycles-pp.__kernel_text_address
2.09 ± 2% -1.6 0.44 ± 3% perf-profile.children.cycles-pp.orc_find
2.00 -1.5 0.49 ± 10% perf-profile.children.cycles-pp.sched_ttwu_pending
1.87 -1.5 0.39 ± 6% perf-profile.children.cycles-pp.flush_smp_call_function_queue
1.53 ± 2% -1.2 0.30 ± 3% perf-profile.children.cycles-pp.kernel_text_address
1.37 ± 2% -1.0 0.33 ± 14% perf-profile.children.cycles-pp.menu_select
1.49 -1.0 0.46 ± 17% perf-profile.children.cycles-pp.ttwu_do_activate
1.43 -1.0 0.45 ± 17% perf-profile.children.cycles-pp.enqueue_task_fair
1.26 -0.9 0.34 ± 9% perf-profile.children.cycles-pp.update_load_avg
1.13 ± 2% -0.9 0.23 ± 4% perf-profile.children.cycles-pp.perf_callchain_user
1.24 -0.9 0.35 ± 15% perf-profile.children.cycles-pp.enqueue_entity
1.53 ± 2% -0.8 0.75 ± 4% perf-profile.children.cycles-pp.perf_output_sample
0.96 ± 3% -0.8 0.18 ± 3% perf-profile.children.cycles-pp.__perf_event_header__init_id
0.94 -0.7 0.24 ± 4% perf-profile.children.cycles-pp.sched_clock_cpu
0.79 ± 3% -0.6 0.16 ± 2% perf-profile.children.cycles-pp.__get_user_nocheck_8
0.76 ± 3% -0.6 0.13 ± 8% perf-profile.children.cycles-pp.ttwu_queue_wakelist
0.75 ± 2% -0.6 0.13 ± 8% perf-profile.children.cycles-pp.perf_output_begin_forward
0.91 ± 4% -0.6 0.30 ± 15% perf-profile.children.cycles-pp.asm_sysvec_call_function_single
0.72 -0.6 0.11 ± 4% perf-profile.children.cycles-pp.prepare_to_wait_event
1.16 ± 2% -0.6 0.58 ± 4% perf-profile.children.cycles-pp.__output_copy
0.70 -0.6 0.14 ± 2% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.74 ± 2% -0.6 0.19 ± 5% perf-profile.children.cycles-pp.native_sched_clock
0.67 ± 2% -0.5 0.14 ± 3% perf-profile.children.cycles-pp.ftrace_graph_ret_addr
0.65 -0.5 0.12 ± 6% perf-profile.children.cycles-pp.__switch_to_asm
1.01 -0.5 0.49 ± 4% perf-profile.children.cycles-pp.perf_output_copy
0.69 ± 2% -0.5 0.19 ± 22% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.59 -0.5 0.11 ± 3% perf-profile.children.cycles-pp.set_next_entity
0.55 ± 2% -0.5 0.10 ± 6% perf-profile.children.cycles-pp.mutex_lock
0.67 ± 6% -0.4 0.23 ± 12% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
0.52 -0.4 0.11 ± 4% perf-profile.children.cycles-pp.__entry_text_start
0.50 ± 2% -0.4 0.10 ± 4% perf-profile.children.cycles-pp.core_kernel_text
0.58 ± 4% -0.4 0.20 ± 17% perf-profile.children.cycles-pp.sysvec_call_function_single
0.49 ± 2% -0.4 0.13 ± 7% perf-profile.children.cycles-pp.update_rq_clock
0.42 -0.3 0.08 ± 4% perf-profile.children.cycles-pp.copy_page_from_iter
0.45 ± 5% -0.3 0.11 ± 3% perf-profile.children.cycles-pp.__fdget_pos
0.43 ± 2% -0.3 0.11 ± 4% perf-profile.children.cycles-pp.__update_load_avg_se
0.52 ± 8% -0.3 0.20 ± 13% perf-profile.children.cycles-pp.copyin
0.42 ± 5% -0.3 0.10 ± 5% perf-profile.children.cycles-pp.__fget_light
0.43 ± 5% -0.3 0.12 ± 11% perf-profile.children.cycles-pp.ktime_get
0.48 ± 4% -0.3 0.17 ± 18% perf-profile.children.cycles-pp.__sysvec_call_function_single
0.59 ± 3% -0.3 0.29 ± 6% perf-profile.children.cycles-pp.memcpy_erms
0.38 ± 2% -0.3 0.07 ± 7% perf-profile.children.cycles-pp.copy_page_to_iter
0.38 ± 5% -0.3 0.08 ± 8% perf-profile.children.cycles-pp.__fget_files
0.36 ± 2% -0.3 0.07 ± 4% perf-profile.children.cycles-pp._copy_from_iter
0.35 ± 4% -0.3 0.07 ± 7% perf-profile.children.cycles-pp.__task_pid_nr_ns
0.42 ± 3% -0.3 0.16 ± 21% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.34 ± 7% -0.3 0.07 ± 6% perf-profile.children.cycles-pp.__pthread_disable_asynccancel
0.32 -0.3 0.06 perf-profile.children.cycles-pp.kvm_guest_state
0.39 ± 2% -0.3 0.13 ± 30% perf-profile.children.cycles-pp.tick_nohz_next_event
0.31 ± 2% -0.3 0.06 ± 5% perf-profile.children.cycles-pp._copy_to_iter
0.29 ± 5% -0.2 0.05 perf-profile.children.cycles-pp.perf_trace_buf_alloc
0.31 ± 2% -0.2 0.06 ± 5% perf-profile.children.cycles-pp.os_xsave
0.29 ± 2% -0.2 0.05 ± 9% perf-profile.children.cycles-pp.get_stack_info
0.30 ± 2% -0.2 0.07 ± 7% perf-profile.children.cycles-pp.tick_nohz_idle_enter
0.29 ± 2% -0.2 0.06 perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack
0.29 ± 2% -0.2 0.06 ± 7% perf-profile.children.cycles-pp.__might_resched
0.28 ± 4% -0.2 0.06 ± 10% perf-profile.children.cycles-pp.perf_trace_buf_update
0.27 ± 3% -0.2 0.05 ± 6% perf-profile.children.cycles-pp.security_file_permission
0.26 ± 2% -0.2 0.05 ± 6% perf-profile.children.cycles-pp.__wrgsbase_inactive
0.26 ± 3% -0.2 0.05 perf-profile.children.cycles-pp.get_next_timer_interrupt
0.24 ± 4% -0.2 0.04 ± 40% perf-profile.children.cycles-pp.hrtimer_next_event_without
0.26 ± 2% -0.2 0.07 ± 10% perf-profile.children.cycles-pp.___perf_sw_event
0.35 ± 11% -0.2 0.17 ± 16% perf-profile.children.cycles-pp.copy_page_from_iter_atomic
0.79 ± 11% -0.2 0.63 ± 7% perf-profile.children.cycles-pp.generic_perform_write
0.22 -0.2 0.06 perf-profile.children.cycles-pp.read_tsc
0.26 ± 3% -0.2 0.10 ± 8% perf-profile.children.cycles-pp.select_task_rq
0.26 ± 3% -0.2 0.11 ± 29% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
0.79 ± 11% -0.2 0.64 ± 7% perf-profile.children.cycles-pp.__generic_file_write_iter
0.79 ± 11% -0.1 0.65 ± 7% perf-profile.children.cycles-pp.generic_file_write_iter
0.22 ± 3% -0.1 0.09 ± 9% perf-profile.children.cycles-pp.select_task_rq_fair
0.21 ± 3% -0.1 0.10 ± 9% perf-profile.children.cycles-pp.native_irq_return_iret
0.22 ± 6% -0.1 0.11 ± 10% perf-profile.children.cycles-pp.__irq_exit_rcu
0.14 ± 6% -0.1 0.03 ± 86% perf-profile.children.cycles-pp.pick_next_task_idle
0.16 ± 9% -0.1 0.08 ± 11% perf-profile.children.cycles-pp.__do_softirq
0.14 ± 11% -0.1 0.05 ± 43% perf-profile.children.cycles-pp.rebalance_domains
0.16 ± 2% -0.1 0.08 ± 47% perf-profile.children.cycles-pp.exit_to_user_mode_prepare
0.13 ± 5% -0.1 0.05 ± 6% perf-profile.children.cycles-pp.native_apic_msr_eoi_write
0.13 ± 3% -0.1 0.07 ± 44% perf-profile.children.cycles-pp.nohz_run_idle_balance
0.10 ± 4% -0.0 0.05 ± 9% perf-profile.children.cycles-pp.memcpy
0.11 ± 12% -0.0 0.06 ± 18% perf-profile.children.cycles-pp.clockevents_program_event
0.12 ± 6% +0.0 0.16 ± 8% perf-profile.children.cycles-pp.scheduler_tick
0.15 ± 11% +0.1 0.21 ± 6% perf-profile.children.cycles-pp.switch_mm_irqs_off
0.15 ± 6% +0.1 0.23 ± 18% perf-profile.children.cycles-pp.tick_sched_handle
0.15 ± 7% +0.1 0.23 ± 18% perf-profile.children.cycles-pp.update_process_times
0.00 +0.1 0.13 ± 24% perf-profile.children.cycles-pp.update_blocked_averages
0.24 ± 16% +0.2 0.45 ± 12% perf-profile.children.cycles-pp.start_kernel
0.24 ± 16% +0.2 0.45 ± 12% perf-profile.children.cycles-pp.arch_call_rest_init
0.24 ± 16% +0.2 0.45 ± 12% perf-profile.children.cycles-pp.rest_init
0.00 +0.9 0.85 ± 46% perf-profile.children.cycles-pp.update_sg_lb_stats
0.00 +0.9 0.91 ± 45% perf-profile.children.cycles-pp.update_sd_lb_stats
0.00 +0.9 0.93 ± 45% perf-profile.children.cycles-pp.find_busiest_group
0.00 +1.0 1.01 ± 44% perf-profile.children.cycles-pp.load_balance
0.16 ± 4% +1.0 1.20 ± 41% perf-profile.children.cycles-pp.newidle_balance
28.01 +19.1 47.12 perf-profile.children.cycles-pp.start_secondary
28.25 +19.3 47.57 perf-profile.children.cycles-pp.secondary_startup_64_no_verify
28.25 +19.3 47.57 perf-profile.children.cycles-pp.cpu_startup_entry
28.19 +19.4 47.55 perf-profile.children.cycles-pp.do_idle
65.01 +27.1 92.13 perf-profile.children.cycles-pp.__schedule
13.19 +31.2 44.38 perf-profile.children.cycles-pp.schedule_idle
0.00 +83.5 83.49 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.42 ± 2% +83.7 84.07 perf-profile.children.cycles-pp.prepare_task_switch
0.36 ± 2% +83.8 84.12 perf-profile.children.cycles-pp._raw_spin_lock
20.28 ± 2% -19.9 0.37 ± 7% perf-profile.self.cycles-pp.finish_task_switch
14.57 -14.2 0.39 ± 6% perf-profile.self.cycles-pp.__schedule
6.94 ± 2% -5.5 1.41 ± 3% perf-profile.self.cycles-pp.unwind_next_frame
5.60 ± 2% -4.8 0.79 ± 6% perf-profile.self.cycles-pp.poll_idle
3.77 ± 5% -3.6 0.14 ± 4% perf-profile.self.cycles-pp.__switch_to
4.25 ± 2% -3.4 0.87 ± 3% perf-profile.self.cycles-pp.__orc_find
3.03 ± 6% -2.4 0.58 ± 2% perf-profile.self.cycles-pp.mwait_idle_with_hints
2.31 ± 2% -1.8 0.47 ± 3% perf-profile.self.cycles-pp.stack_access_ok
1.87 ± 2% -1.5 0.40 ± 3% perf-profile.self.cycles-pp.orc_find
0.97 ± 2% -0.8 0.19 ± 3% perf-profile.self.cycles-pp.__unwind_start
0.94 ± 2% -0.7 0.19 ± 4% perf-profile.self.cycles-pp.perf_callchain_kernel
0.75 ± 2% -0.6 0.15 ± 3% perf-profile.self.cycles-pp.__get_user_nocheck_8
0.69 ± 2% -0.6 0.13 ± 8% perf-profile.self.cycles-pp.perf_output_begin_forward
0.67 -0.5 0.14 ± 3% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.64 -0.5 0.11 ± 5% perf-profile.self.cycles-pp.__switch_to_asm
0.70 -0.5 0.18 ± 5% perf-profile.self.cycles-pp.native_sched_clock
0.51 -0.4 0.07 ± 7% perf-profile.self.cycles-pp.pipe_read
0.55 ± 2% -0.4 0.11 ± 4% perf-profile.self.cycles-pp.kernel_text_address
0.65 ± 6% -0.4 0.22 ± 10% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
0.52 ± 3% -0.4 0.09 ± 3% perf-profile.self.cycles-pp.perf_prepare_sample
0.47 ± 2% -0.4 0.09 ± 5% perf-profile.self.cycles-pp.ftrace_graph_ret_addr
0.48 ± 2% -0.4 0.11 ± 6% perf-profile.self.cycles-pp.menu_select
0.47 ± 2% -0.4 0.09 ± 5% perf-profile.self.cycles-pp.unwind_get_return_address
0.45 ± 2% -0.4 0.09 ± 3% perf-profile.self.cycles-pp.__kernel_text_address
0.45 ± 3% -0.4 0.09 ± 3% perf-profile.self.cycles-pp.get_perf_callchain
0.43 ± 2% -0.3 0.09 ± 3% perf-profile.self.cycles-pp.update_load_avg
0.40 ± 3% -0.3 0.06 ± 5% perf-profile.self.cycles-pp.cpuidle_idle_call
0.43 -0.3 0.11 ± 13% perf-profile.self.cycles-pp.enqueue_entity
0.36 ± 2% -0.3 0.07 ± 4% perf-profile.self.cycles-pp.core_kernel_text
0.36 ± 2% -0.3 0.07 ± 5% perf-profile.self.cycles-pp.do_idle
0.37 ± 5% -0.3 0.08 ± 7% perf-profile.self.cycles-pp.__fget_files
0.38 ± 2% -0.3 0.10 ± 5% perf-profile.self.cycles-pp.__update_load_avg_se
0.55 ± 4% -0.3 0.26 ± 6% perf-profile.self.cycles-pp.memcpy_erms
0.33 ± 2% -0.3 0.06 ± 8% perf-profile.self.cycles-pp.__libc_read
0.33 -0.3 0.06 ± 8% perf-profile.self.cycles-pp.prepare_to_wait_event
0.33 ± 3% -0.3 0.06 ± 7% perf-profile.self.cycles-pp.__perf_event_header__init_id
0.33 ± 6% -0.3 0.07 ± 6% perf-profile.self.cycles-pp.update_curr
0.31 ± 3% -0.3 0.06 perf-profile.self.cycles-pp.__task_pid_nr_ns
0.40 ± 4% -0.2 0.15 ± 21% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.41 ± 3% -0.2 0.17 ± 25% perf-profile.self.cycles-pp.perf_tp_event
0.30 ± 2% -0.2 0.06 ± 8% perf-profile.self.cycles-pp.os_xsave
0.31 ± 7% -0.2 0.07 ± 9% perf-profile.self.cycles-pp.__pthread_disable_asynccancel
0.48 -0.2 0.24 ± 3% perf-profile.self.cycles-pp.__output_copy
0.29 ± 2% -0.2 0.06 perf-profile.self.cycles-pp.perf_callchain_user
0.30 ± 2% -0.2 0.07 ± 7% perf-profile.self.cycles-pp.__libc_write
0.28 -0.2 0.06 perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack
0.26 ± 2% -0.2 0.04 ± 40% perf-profile.self.cycles-pp.kvm_guest_state
0.26 -0.2 0.05 perf-profile.self.cycles-pp.__wrgsbase_inactive
0.26 ± 3% -0.2 0.06 perf-profile.self.cycles-pp.__might_resched
0.22 ± 2% -0.2 0.03 ± 86% perf-profile.self.cycles-pp.perf_trace_sched_switch
0.25 ± 4% -0.2 0.06 ± 7% perf-profile.self.cycles-pp.dequeue_entity
0.33 ± 3% -0.2 0.16 ± 2% perf-profile.self.cycles-pp.perf_output_sample
0.22 ± 4% -0.2 0.06 ± 8% perf-profile.self.cycles-pp.update_rq_clock
0.22 ± 10% -0.2 0.07 ± 17% perf-profile.self.cycles-pp.ktime_get
0.21 ± 2% -0.2 0.06 ± 5% perf-profile.self.cycles-pp.read_tsc
0.18 ± 2% -0.1 0.04 ± 65% perf-profile.self.cycles-pp.___perf_sw_event
0.19 ± 5% -0.1 0.06 ± 41% perf-profile.self.cycles-pp.dequeue_task_fair
0.20 ± 5% -0.1 0.08 ± 42% perf-profile.self.cycles-pp.enqueue_task_fair
0.20 -0.1 0.08 ± 4% perf-profile.self.cycles-pp.cpuidle_enter_state
0.21 ± 3% -0.1 0.10 ± 9% perf-profile.self.cycles-pp.native_irq_return_iret
0.14 ± 4% -0.1 0.06 ± 43% perf-profile.self.cycles-pp.newidle_balance
0.15 ± 4% -0.1 0.08 ± 13% perf-profile.self.cycles-pp.prepare_task_switch
0.13 ± 5% -0.1 0.05 ± 6% perf-profile.self.cycles-pp.native_apic_msr_eoi_write
0.12 ± 2% -0.1 0.06 ± 5% perf-profile.self.cycles-pp.perf_output_copy
0.12 ± 5% -0.1 0.06 ± 43% perf-profile.self.cycles-pp.nohz_run_idle_balance
0.14 ± 11% +0.1 0.21 ± 7% perf-profile.self.cycles-pp.switch_mm_irqs_off
0.33 +0.3 0.62 ± 7% perf-profile.self.cycles-pp._raw_spin_lock
0.00 +0.7 0.66 ± 45% perf-profile.self.cycles-pp.update_sg_lb_stats
0.00 +83.5 83.49 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath***************************************************************************************************
lkp-spr-r02: 224 threads 2 sockets (Sapphire Rapids) with 256G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase:
gcc-11/performance/x86_64-rhel-8.3/thread/16/debian-11.1-x86_64-20220510.cgz/lkp-spr-r02/context_switch1/will-it-scale
commit:
99babd04b2 ("selftests/rseq: Implement rseq numa node id field selftest") af7f588d8f ("sched: Introduce per-memory-map concurrency ID")
99babd04b2505471 af7f588d8f7355bc4298dd1962d
---------------- ---------------------------
%stddev %change %stddev
\ | \
9168456 -73.8% 2400232 ± 10% will-it-scale.16.threads
573027 -73.8% 150014 ± 10% will-it-scale.per_thread_ops
9168456 -73.8% 2400232 ± 10% will-it-scale.workload
2.864e+09 -73.1% 7.715e+08 ± 11% cpuidle..usage
6.10 +1.0 7.15 mpstat.cpu.all.sys%
1.13 ± 3% -0.9 0.27 ± 10% mpstat.cpu.all.usr%
22292871 ± 7% -78.9% 4696291 ± 10% vmstat.memory.cache
19.00 ± 3% -14.7% 16.20 ± 2% vmstat.procs.r
18144385 -73.8% 4753330 ± 9% vmstat.system.cs
483366 +35.8% 656610 ± 5% vmstat.system.in
1555224 ± 5% +60.7% 2499655 ± 18% turbostat.C1
87537260 ± 26% +24.1% 1.086e+08 ± 14% turbostat.C1E
47.94 ± 62% +27.4 75.30 ± 24% turbostat.C1E%
0.59 -66.0% 0.20 ± 8% turbostat.IPC
1.468e+08 +35.9% 1.995e+08 ± 5% turbostat.IRQ
2.746e+09 -76.2% 6.523e+08 ± 14% turbostat.POLL
1.67 ± 2% -1.0 0.68 ± 4% turbostat.POLL%
34.20 -9.9% 30.80 turbostat.PkgTmp
433.23 ± 2% -10.4% 388.15 turbostat.PkgWatt
33394 ± 5% +814.1% 305243 ± 32% numa-vmstat.node1.nr_active_anon
55976 ± 9% -22.7% 43246 ± 10% numa-vmstat.node1.nr_anon_pages
4483729 ± 33% -88.5% 513735 ± 65% numa-vmstat.node1.nr_file_pages
28119032 ± 5% +14.5% 32194305 numa-vmstat.node1.nr_free_pages
4336878 ± 31% -97.7% 97706 ± 85% numa-vmstat.node1.nr_inactive_anon
1758120 ± 22% -91.3% 153214 ± 43% numa-vmstat.node1.nr_mapped
4166 ± 30% -75.4% 1025 ± 21% numa-vmstat.node1.nr_page_table_pages
4314309 ± 32% -91.7% 359713 ± 46% numa-vmstat.node1.nr_shmem
33394 ± 5% +814.1% 305243 ± 32% numa-vmstat.node1.nr_zone_active_anon
4336876 ± 31% -97.7% 97706 ± 85% numa-vmstat.node1.nr_zone_inactive_anon
134596 ± 6% +807.2% 1221042 ± 32% numa-meminfo.node1.Active
134504 ± 6% +807.7% 1220946 ± 32% numa-meminfo.node1.Active(anon)
223873 ± 9% -22.7% 172976 ± 10% numa-meminfo.node1.AnonPages
17932980 ± 33% -88.5% 2055168 ± 65% numa-meminfo.node1.FilePages
17344798 ± 31% -97.7% 391247 ± 85% numa-meminfo.node1.Inactive
17344618 ± 31% -97.7% 391073 ± 85% numa-meminfo.node1.Inactive(anon)
7027715 ± 22% -91.3% 614131 ± 43% numa-meminfo.node1.Mapped
1.125e+08 ± 5% +14.5% 1.288e+08 numa-meminfo.node1.MemFree
19565762 ± 30% -83.3% 3266862 ± 41% numa-meminfo.node1.MemUsed
16664 ± 30% -75.4% 4105 ± 21% numa-meminfo.node1.PageTables
17255300 ± 32% -91.7% 1439083 ± 46% numa-meminfo.node1.Shmem
421445 ± 2% -16.6% 351355 meminfo.AnonPages
22061123 ± 7% -79.5% 4530612 ± 11% meminfo.Cached
20088958 ± 8% -87.8% 2443971 ± 21% meminfo.Committed_AS
19170771 ± 11% -97.0% 574987 ± 60% meminfo.Inactive
19170591 ± 11% -97.0% 574813 ± 60% meminfo.Inactive(anon)
213051 -18.3% 173997 meminfo.KReclaimable
7430615 ± 13% -90.5% 705190 ± 33% meminfo.Mapped
25328866 ± 7% -71.9% 7127544 ± 7% meminfo.Memused
21777 ± 8% -63.7% 7906 ± 7% meminfo.PageTables
213051 -18.3% 173997 meminfo.SReclaimable
19164273 ± 8% -91.5% 1633768 ± 30% meminfo.Shmem
26692881 -39.4% 16170877 ± 7% meminfo.max_used_kB
65901 ± 13% -98.2% 1210 ±143% sched_debug.cfs_rq:/.MIN_vruntime.avg
1327262 ± 10% -93.4% 87886 ±144% sched_debug.cfs_rq:/.MIN_vruntime.max
265536 ± 3% -96.2% 10166 ±144% sched_debug.cfs_rq:/.MIN_vruntime.stddev
65901 ± 13% -98.2% 1210 ±143% sched_debug.cfs_rq:/.max_vruntime.avg
1327262 ± 10% -93.4% 87886 ±144% sched_debug.cfs_rq:/.max_vruntime.max
265536 ± 3% -96.2% 10166 ±144% sched_debug.cfs_rq:/.max_vruntime.stddev
33780 ± 13% +39.7% 47201 ± 13% sched_debug.cfs_rq:/.min_vruntime.min
979.30 ± 2% -15.8% 824.68 ± 4% sched_debug.cfs_rq:/.runnable_avg.max
978.91 ± 2% -17.1% 811.25 ± 3% sched_debug.cfs_rq:/.util_avg.max
860.31 ± 2% -30.0% 601.91 ± 4% sched_debug.cfs_rq:/.util_est_enqueued.max
1736 ± 3% +506.1% 10522 ± 91% sched_debug.cpu.avg_idle.min
0.09 ± 4% -13.4% 0.08 ± 5% sched_debug.cpu.nr_running.avg
10630744 ± 12% -71.4% 3041094 ± 6% sched_debug.cpu.nr_switches.avg
74675547 ± 12% -71.5% 21318321 ± 6% sched_debug.cpu.nr_switches.max
26031282 ± 12% -71.4% 7440127 ± 6% sched_debug.cpu.nr_switches.stddev
105357 ± 2% -16.6% 87842 proc-vmstat.nr_anon_pages
5921091 +7.7% 6375494 proc-vmstat.nr_dirty_background_threshold
11856660 +7.7% 12766578 proc-vmstat.nr_dirty_threshold
5515701 ± 7% -79.5% 1132643 ± 11% proc-vmstat.nr_file_pages
59582345 +7.6% 64133047 proc-vmstat.nr_free_pages
4793162 ± 11% -97.0% 143617 ± 60% proc-vmstat.nr_inactive_anon
1858672 ± 13% -90.6% 175442 ± 33% proc-vmstat.nr_mapped
5443 ± 8% -63.7% 1976 ± 7% proc-vmstat.nr_page_table_pages
4791488 ± 8% -91.5% 408431 ± 30% proc-vmstat.nr_shmem
53263 -18.3% 43499 proc-vmstat.nr_slab_reclaimable
103557 -1.4% 102106 proc-vmstat.nr_slab_unreclaimable
4793162 ± 11% -97.0% 143617 ± 60% proc-vmstat.nr_zone_inactive_anon
260965 ± 3% -73.8% 68247 ± 12% proc-vmstat.numa_hint_faults
210545 ± 4% -91.7% 17501 ± 29% proc-vmstat.numa_hint_faults_local
8964679 -46.3% 4809967 ± 10% proc-vmstat.numa_hit
1384 ± 19% -89.0% 151.80 ± 23% proc-vmstat.numa_huge_pte_updates
8648086 -46.7% 4607108 ± 10% proc-vmstat.numa_local
1033314 ± 14% -80.9% 197728 ± 15% proc-vmstat.numa_pte_updates
8998915 -45.0% 4950106 ± 9% proc-vmstat.pgalloc_normal
1704072 ± 2% -23.0% 1311857 ± 3% proc-vmstat.pgfault
0.45 ± 8% +168.1% 1.19 ± 3% perf-stat.i.MPKI
2.07e+10 -65.5% 7.138e+09 ± 7% perf-stat.i.branch-instructions
1.334e+08 -66.0% 45319480 ± 8% perf-stat.i.branch-misses
8.65 ± 7% -4.0 4.68 ± 6% perf-stat.i.cache-miss-rate%
7550889 ± 5% -48.6% 3883482 ± 15% perf-stat.i.cache-misses
18312903 -73.8% 4797849 ± 10% perf-stat.i.context-switches
0.99 +203.5% 3.00 ± 7% perf-stat.i.cpi
9.998e+10 -1.4% 9.855e+10 perf-stat.i.cpu-cycles
122169 ± 13% +148.0% 302970 ± 5% perf-stat.i.cycles-between-cache-misses
0.02 ± 3% -0.0 0.00 ± 5% perf-stat.i.dTLB-load-miss-rate%
5554209 ± 3% -97.0% 168530 ± 11% perf-stat.i.dTLB-load-misses
2.912e+10 -67.7% 9.412e+09 ± 7% perf-stat.i.dTLB-loads
0.00 ± 13% -0.0 0.00 ± 8% perf-stat.i.dTLB-store-miss-rate%
316990 ± 13% -85.2% 46788 ± 18% perf-stat.i.dTLB-store-misses
1.752e+10 -71.8% 4.938e+09 ± 9% perf-stat.i.dTLB-stores
1.016e+11 -66.7% 3.386e+10 ± 7% perf-stat.i.instructions
1.02 -66.2% 0.34 ± 7% perf-stat.i.ipc
0.45 -1.4% 0.44 perf-stat.i.metric.GHz
299.03 ± 6% -30.0% 209.39 ± 10% perf-stat.i.metric.K/sec
300.64 -68.1% 95.93 ± 8% perf-stat.i.metric.M/sec
4904 -27.1% 3576 ± 4% perf-stat.i.minor-faults
52.06 ± 27% +42.6 94.61 perf-stat.i.node-load-miss-rate%
1131145 ± 23% -47.4% 595531 ± 25% perf-stat.i.node-load-misses
1002828 ± 32% -96.5% 35044 ± 23% perf-stat.i.node-loads
4904 -27.1% 3576 ± 4% perf-stat.i.page-faults
0.46 ± 8% +168.7% 1.23 ± 2% perf-stat.overall.MPKI
16.33 ± 4% -7.0 9.34 ± 6% perf-stat.overall.cache-miss-rate%
0.98 +197.4% 2.93 ± 7% perf-stat.overall.cpi
13283 ± 6% +94.7% 25864 ± 15% perf-stat.overall.cycles-between-cache-misses
0.02 ± 3% -0.0 0.00 ± 6% perf-stat.overall.dTLB-load-miss-rate%
0.00 ± 13% -0.0 0.00 ± 10% perf-stat.overall.dTLB-store-miss-rate%
1.02 -66.2% 0.34 ± 7% perf-stat.overall.ipc
53.52 ± 19% +40.4 93.95 perf-stat.overall.node-load-miss-rate%
3330788 +27.6% 4250478 ± 2% perf-stat.overall.path-length
2.063e+10 -65.5% 7.117e+09 ± 7% perf-stat.ps.branch-instructions
1.33e+08 -66.0% 45185662 ± 8% perf-stat.ps.branch-misses
7528642 ± 5% -48.3% 3888889 ± 15% perf-stat.ps.cache-misses
18252001 -73.8% 4781904 ± 10% perf-stat.ps.context-switches
9.965e+10 -1.4% 9.823e+10 perf-stat.ps.cpu-cycles
5544297 ± 3% -97.0% 168873 ± 11% perf-stat.ps.dTLB-load-misses
2.903e+10 -67.7% 9.384e+09 ± 7% perf-stat.ps.dTLB-loads
316419 ± 13% -85.2% 46843 ± 19% perf-stat.ps.dTLB-store-misses
1.746e+10 -71.8% 4.923e+09 ± 9% perf-stat.ps.dTLB-stores
1.013e+11 -66.7% 3.376e+10 ± 7% perf-stat.ps.instructions
4885 -26.8% 3577 ± 4% perf-stat.ps.minor-faults
1127461 ± 23% -47.3% 593739 ± 25% perf-stat.ps.node-load-misses
1001303 ± 32% -96.4% 36005 ± 22% perf-stat.ps.node-loads
4885 -26.8% 3577 ± 4% perf-stat.ps.page-faults
3.054e+13 -66.7% 1.018e+13 ± 7% perf-stat.total.instructions
34.69 -20.3 14.42 ± 13% perf-profile.calltrace.cycles-pp.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event
34.45 -20.1 14.34 ± 13% perf-profile.calltrace.cycles-pp.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow
31.23 ± 2% -18.2 13.07 ± 13% perf-profile.calltrace.cycles-pp.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward
20.47 ± 2% -12.3 8.15 ± 13% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule
18.88 -11.5 7.40 ± 13% perf-profile.calltrace.cycles-pp.__libc_write
17.86 -10.9 7.01 ± 13% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_write
17.75 -10.8 6.97 ± 13% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
17.53 -10.6 6.90 ± 13% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
19.27 ± 2% -10.6 8.64 ± 10% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
16.91 ± 2% -10.3 6.65 ± 13% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch
16.38 -9.9 6.47 ± 13% perf-profile.calltrace.cycles-pp.pipe_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
16.36 ± 2% -9.5 6.84 ± 13% perf-profile.calltrace.cycles-pp.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
14.54 -8.7 5.86 ± 13% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.vfs_write.ksys_write.do_syscall_64
15.76 -8.6 7.19 ± 13% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_read.vfs_read
14.21 -8.5 5.73 ± 13% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.vfs_write.ksys_write
14.11 -8.4 5.69 ± 13% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.vfs_write
15.42 -8.4 7.05 ± 13% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_read
13.97 -8.3 5.64 ± 13% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
13.81 ± 2% -8.0 5.79 ± 13% perf-profile.calltrace.cycles-pp.unwind_next_frame.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain
14.00 -7.6 6.43 ± 13% perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule
11.27 ± 2% -7.5 3.75 ± 12% perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule.pipe_read.vfs_read
10.99 ± 2% -7.3 3.64 ± 12% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule.pipe_read
12.19 -7.3 4.89 ± 13% perf-profile.calltrace.cycles-pp.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
10.72 ± 2% -7.2 3.55 ± 12% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule
11.91 -7.1 4.78 ± 13% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common
13.13 -7.0 6.08 ± 13% perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair.__schedule
12.84 -6.9 5.95 ± 13% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair
11.43 ± 2% -6.8 4.62 ± 13% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function
12.56 -6.7 5.82 ± 13% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity
11.32 ± 2% -6.7 4.58 ± 13% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up
12.46 -6.7 5.77 ± 13% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr
9.69 ± 2% -5.8 3.85 ± 14% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_wakeup_template
10.64 -5.7 4.94 ± 13% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime
10.56 ± 2% -5.6 4.95 ± 13% perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry
10.24 ± 2% -5.4 4.81 ± 13% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule_idle.do_idle
9.23 ± 2% -5.3 3.91 ± 13% perf-profile.calltrace.cycles-pp.unwind_next_frame.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
9.97 -5.3 4.68 ± 13% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule_idle
14.74 -4.4 10.31 ± 5% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
11.02 -2.5 8.53 ± 3% perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
10.67 -2.4 8.25 ± 3% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
2.87 -2.1 0.76 ± 42% perf-profile.calltrace.cycles-pp.perf_output_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch
5.01 -1.9 3.08 ± 11% perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
3.13 ± 2% -1.9 1.23 ± 15% perf-profile.calltrace.cycles-pp.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
2.76 -1.7 1.07 ± 15% perf-profile.calltrace.cycles-pp.sched_ttwu_pending.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary
2.92 ± 4% -1.5 1.44 ± 18% perf-profile.calltrace.cycles-pp.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
2.07 ± 2% -1.3 0.81 ± 15% perf-profile.calltrace.cycles-pp.ttwu_do_activate.sched_ttwu_pending.flush_smp_call_function_queue.do_idle.cpu_startup_entry
1.96 ± 2% -1.2 0.76 ± 15% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.sched_ttwu_pending.flush_smp_call_function_queue.do_idle
1.60 ± 2% -1.0 0.63 ± 15% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.sched_ttwu_pending.flush_smp_call_function_queue
1.30 -1.0 0.34 ± 82% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule_idle.do_idle.cpu_startup_entry
1.68 ± 8% -0.8 0.86 ± 29% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry
1.46 -0.8 0.64 ± 13% perf-profile.calltrace.cycles-pp.perf_output_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime
1.37 ± 16% -0.5 0.90 perf-profile.calltrace.cycles-pp.finish_task_switch.__schedule.schedule.pipe_read.vfs_read
0.44 ± 50% +0.6 1.05 ± 21% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.rest_init.arch_call_rest_init
0.43 ± 50% +0.6 1.05 ± 21% perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.rest_init
0.00 +0.7 0.67 ± 7% perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule_idle.do_idle.cpu_startup_entry
41.34 +1.9 43.24 perf-profile.calltrace.cycles-pp.__libc_read
38.47 +3.6 42.12 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_read
38.19 +3.8 42.01 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
37.53 +4.3 41.80 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
36.83 +4.7 41.56 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
35.95 +5.3 41.23 perf-profile.calltrace.cycles-pp.pipe_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
32.14 +7.7 39.84 perf-profile.calltrace.cycles-pp.schedule.pipe_read.vfs_read.ksys_read.do_syscall_64
31.80 +7.9 39.70 ± 2% perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_read.vfs_read.ksys_read
34.12 +10.2 44.37 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
34.07 +10.3 44.35 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
33.94 +10.3 44.27 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
35.09 +10.5 45.63 perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
14.13 +17.8 31.89 ± 3% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
13.80 +18.0 31.76 ± 3% perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
0.00 +24.3 24.34 ± 9% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prepare_task_switch.__schedule.schedule_idle
0.00 +25.1 25.13 ± 8% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prepare_task_switch.__schedule.schedule
0.00 +25.3 25.28 ± 8% perf-profile.calltrace.cycles-pp._raw_spin_lock.prepare_task_switch.__schedule.schedule_idle.do_idle
0.00 +25.5 25.54 ± 8% perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry
0.00 +25.9 25.88 ± 8% perf-profile.calltrace.cycles-pp._raw_spin_lock.prepare_task_switch.__schedule.schedule.pipe_read
0.00 +26.1 26.10 ± 8% perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule.pipe_read.vfs_read
46.54 -27.0 19.49 ± 13% perf-profile.children.cycles-pp.perf_tp_event
45.22 -26.2 18.97 ± 13% perf-profile.children.cycles-pp.__perf_event_overflow
44.82 -26.0 18.82 ± 13% perf-profile.children.cycles-pp.perf_event_output_forward
37.76 -22.0 15.72 ± 13% perf-profile.children.cycles-pp.perf_prepare_sample
35.11 -20.4 14.66 ± 13% perf-profile.children.cycles-pp.perf_callchain
34.91 -20.3 14.59 ± 13% perf-profile.children.cycles-pp.get_perf_callchain
32.01 -18.6 13.46 ± 13% perf-profile.children.cycles-pp.perf_callchain_kernel
25.16 ± 2% -14.5 10.65 ± 13% perf-profile.children.cycles-pp.unwind_next_frame
22.28 ± 2% -13.4 8.91 ± 13% perf-profile.children.cycles-pp.perf_trace_sched_switch
21.63 ± 2% -11.9 9.70 ± 10% perf-profile.children.cycles-pp.__libc_write
19.92 ± 2% -10.9 8.98 ± 10% perf-profile.children.cycles-pp.ksys_write
19.32 ± 2% -10.6 8.69 ± 10% perf-profile.children.cycles-pp.vfs_write
16.44 -9.9 6.50 ± 13% perf-profile.children.cycles-pp.pipe_write
16.82 -9.8 7.05 ± 13% perf-profile.children.cycles-pp.__unwind_start
14.58 -8.7 5.87 ± 13% perf-profile.children.cycles-pp.__wake_up_common_lock
15.79 -8.6 7.20 ± 13% perf-profile.children.cycles-pp.dequeue_task_fair
14.23 -8.5 5.73 ± 13% perf-profile.children.cycles-pp.__wake_up_common
15.53 -8.4 7.10 ± 13% perf-profile.children.cycles-pp.dequeue_entity
14.13 -8.4 5.70 ± 13% perf-profile.children.cycles-pp.autoremove_wake_function
14.03 -8.4 5.67 ± 13% perf-profile.children.cycles-pp.try_to_wake_up
14.25 -7.7 6.58 ± 13% perf-profile.children.cycles-pp.update_curr
58.80 -7.5 51.32 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
12.22 -7.3 4.90 ± 13% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
58.45 -7.3 51.19 perf-profile.children.cycles-pp.do_syscall_64
13.24 -7.1 6.18 ± 12% perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime
15.07 -4.6 10.47 ± 5% perf-profile.children.cycles-pp.cpuidle_idle_call
6.83 -3.9 2.91 ± 13% perf-profile.children.cycles-pp.__orc_find
5.92 -3.5 2.41 ± 13% perf-profile.children.cycles-pp.perf_output_sample
4.58 -2.7 1.86 ± 14% perf-profile.children.cycles-pp.__output_copy
4.53 -2.6 1.91 ± 13% perf-profile.children.cycles-pp.stack_access_ok
11.21 -2.6 8.65 ± 3% perf-profile.children.cycles-pp.cpuidle_enter
11.14 -2.5 8.62 ± 3% perf-profile.children.cycles-pp.cpuidle_enter_state
3.96 -2.4 1.60 ± 13% perf-profile.children.cycles-pp.perf_output_copy
5.46 -2.2 3.24 ± 10% perf-profile.children.cycles-pp.poll_idle
3.78 -2.2 1.58 ± 12% perf-profile.children.cycles-pp.unwind_get_return_address
3.24 ± 2% -2.0 1.27 ± 14% perf-profile.children.cycles-pp.flush_smp_call_function_queue
3.27 -2.0 1.31 ± 13% perf-profile.children.cycles-pp.sched_ttwu_pending
3.31 ± 2% -1.9 1.42 ± 13% perf-profile.children.cycles-pp.orc_find
3.20 -1.9 1.34 ± 13% perf-profile.children.cycles-pp.__kernel_text_address
3.04 ± 4% -1.6 1.49 ± 18% perf-profile.children.cycles-pp.menu_select
2.43 -1.5 0.98 ± 13% perf-profile.children.cycles-pp.ttwu_do_activate
2.32 -1.4 0.93 ± 13% perf-profile.children.cycles-pp.enqueue_task_fair
2.29 -1.4 0.90 ± 14% perf-profile.children.cycles-pp.memcpy_erms
2.32 -1.3 0.97 ± 13% perf-profile.children.cycles-pp.kernel_text_address
2.02 -1.2 0.81 ± 13% perf-profile.children.cycles-pp.enqueue_entity
2.04 ± 2% -1.2 0.89 ± 14% perf-profile.children.cycles-pp.update_load_avg
1.83 -1.1 0.72 ± 13% perf-profile.children.cycles-pp.perf_callchain_user
1.77 -0.9 0.83 ± 8% perf-profile.children.cycles-pp.pick_next_task_fair
1.72 -0.9 0.79 ± 8% perf-profile.children.cycles-pp.sched_clock_cpu
1.75 ± 8% -0.9 0.89 ± 28% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
1.45 -0.9 0.60 ± 13% perf-profile.children.cycles-pp.__perf_event_header__init_id
1.26 ± 2% -0.8 0.49 ± 13% perf-profile.children.cycles-pp.__get_user_nocheck_8
1.20 ± 2% -0.7 0.46 ± 13% perf-profile.children.cycles-pp.__switch_to
1.35 -0.7 0.61 ± 8% perf-profile.children.cycles-pp.native_sched_clock
1.14 -0.7 0.48 ± 10% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
1.03 -0.6 0.38 ± 12% perf-profile.children.cycles-pp.__switch_to_asm
1.46 ± 10% -0.6 0.83 ± 16% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
1.06 ± 2% -0.6 0.45 ± 12% perf-profile.children.cycles-pp.ftrace_graph_ret_addr
1.00 ± 2% -0.6 0.40 ± 13% perf-profile.children.cycles-pp.set_next_entity
0.85 ± 2% -0.6 0.25 ± 16% perf-profile.children.cycles-pp.copy_page_from_iter
0.99 ± 2% -0.6 0.39 ± 13% perf-profile.children.cycles-pp.prepare_to_wait_event
1.06 ± 7% -0.6 0.51 ± 7% perf-profile.children.cycles-pp.asm_sysvec_call_function_single
0.88 ± 2% -0.5 0.37 ± 12% perf-profile.children.cycles-pp.__entry_text_start
1.14 ± 13% -0.5 0.64 ± 36% perf-profile.children.cycles-pp.tick_nohz_next_event
0.71 ± 2% -0.5 0.22 ± 16% perf-profile.children.cycles-pp._copy_from_iter
1.22 ± 13% -0.5 0.73 ± 16% perf-profile.children.cycles-pp.copyin
0.81 ± 2% -0.5 0.33 ± 13% perf-profile.children.cycles-pp.core_kernel_text
0.78 ± 4% -0.5 0.32 ± 13% perf-profile.children.cycles-pp.mutex_lock
0.65 ± 2% -0.4 0.22 ± 12% perf-profile.children.cycles-pp.copy_page_to_iter
0.76 ± 2% -0.4 0.35 ± 10% perf-profile.children.cycles-pp.ttwu_queue_wakelist
0.73 ± 8% -0.4 0.33 ± 10% perf-profile.children.cycles-pp.__fdget_pos
0.72 -0.4 0.33 ± 8% perf-profile.children.cycles-pp.update_rq_clock
0.68 ± 8% -0.4 0.29 ± 11% perf-profile.children.cycles-pp.__fget_light
0.66 ± 5% -0.4 0.29 ± 14% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.72 -0.4 0.35 ± 13% perf-profile.children.cycles-pp.__update_load_avg_se
0.55 ± 2% -0.4 0.18 ± 14% perf-profile.children.cycles-pp._copy_to_iter
0.70 ± 6% -0.4 0.33 ± 7% perf-profile.children.cycles-pp.sysvec_call_function_single
1.78 ± 12% -0.4 1.41 ± 2% perf-profile.children.cycles-pp.finish_task_switch
0.55 -0.3 0.20 ± 14% perf-profile.children.cycles-pp.kvm_guest_state
0.62 -0.3 0.28 ± 13% perf-profile.children.cycles-pp.select_task_rq
0.60 ± 8% -0.3 0.26 ± 10% perf-profile.children.cycles-pp.__fget_files
0.56 ± 5% -0.3 0.22 ± 13% perf-profile.children.cycles-pp.__task_pid_nr_ns
0.51 ± 3% -0.3 0.20 ± 12% perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack
0.51 ± 3% -0.3 0.20 ± 13% perf-profile.children.cycles-pp.os_xsave
0.51 -0.3 0.21 ± 14% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.53 -0.3 0.23 ± 14% perf-profile.children.cycles-pp.tick_nohz_idle_enter
0.93 ± 14% -0.3 0.63 ± 14% perf-profile.children.cycles-pp.ktime_get
0.46 -0.3 0.17 ± 11% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
0.48 ± 2% -0.3 0.19 ± 13% perf-profile.children.cycles-pp.get_stack_info
0.53 ± 2% -0.3 0.24 ± 12% perf-profile.children.cycles-pp.select_task_rq_fair
0.92 ± 18% -0.3 0.64 ± 16% perf-profile.children.cycles-pp.copy_page_from_iter_atomic
0.47 ± 2% -0.3 0.19 ± 15% perf-profile.children.cycles-pp.hrtimer_next_event_without
0.54 ± 7% -0.3 0.27 ± 6% perf-profile.children.cycles-pp.__sysvec_call_function_single
0.45 -0.3 0.17 ± 11% perf-profile.children.cycles-pp.__wrgsbase_inactive
0.48 ± 2% -0.3 0.20 ± 8% perf-profile.children.cycles-pp.__might_resched
0.46 -0.3 0.19 ± 11% perf-profile.children.cycles-pp.perf_trace_buf_update
0.44 ± 4% -0.3 0.17 ± 15% perf-profile.children.cycles-pp.security_file_permission
0.35 ± 3% -0.3 0.09 ± 14% perf-profile.children.cycles-pp.__might_fault
0.42 -0.2 0.17 ± 11% perf-profile.children.cycles-pp.memcpy
0.51 ± 2% -0.2 0.26 ± 11% perf-profile.children.cycles-pp.___perf_sw_event
0.39 ± 2% -0.2 0.16 ± 11% perf-profile.children.cycles-pp.perf_trace_buf_alloc
0.35 ± 3% -0.2 0.12 ± 16% perf-profile.children.cycles-pp.touch_atime
0.36 ± 6% -0.2 0.13 ± 15% perf-profile.children.cycles-pp.get_callchain_entry
0.46 ± 2% -0.2 0.23 ± 5% perf-profile.children.cycles-pp.read_tsc
0.37 ± 2% -0.2 0.15 ± 12% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.35 ± 5% -0.2 0.13 ± 12% perf-profile.children.cycles-pp.update_cfs_group
0.34 ± 3% -0.2 0.12 ± 13% perf-profile.children.cycles-pp.copyout
0.35 ± 2% -0.2 0.14 ± 14% perf-profile.children.cycles-pp.tick_nohz_idle_exit
0.34 ± 5% -0.2 0.13 ± 17% perf-profile.children.cycles-pp.apparmor_file_permission
0.32 -0.2 0.12 ± 13% perf-profile.children.cycles-pp.syscall_return_via_sysret
0.30 ± 5% -0.2 0.10 ± 14% perf-profile.children.cycles-pp.atime_needs_update
0.31 ± 2% -0.2 0.12 ± 8% perf-profile.children.cycles-pp._raw_spin_lock_irq
0.36 ± 2% -0.2 0.17 ± 7% perf-profile.children.cycles-pp.ct_idle_exit
0.28 ± 2% -0.2 0.10 ± 14% perf-profile.children.cycles-pp.exit_to_user_mode_prepare
0.28 -0.2 0.10 ± 10% perf-profile.children.cycles-pp.fput
0.29 ± 5% -0.2 0.11 ± 12% perf-profile.children.cycles-pp.reweight_entity
0.32 ± 3% -0.2 0.14 ± 11% perf-profile.children.cycles-pp.mutex_unlock
0.28 ± 3% -0.2 0.10 ± 15% perf-profile.children.cycles-pp.__might_sleep
0.28 ± 4% -0.2 0.11 ± 15% perf-profile.children.cycles-pp.perf_misc_flags
0.26 ± 2% -0.2 0.09 ± 12% perf-profile.children.cycles-pp.perf_instruction_pointer
0.26 ± 2% -0.2 0.09 ± 9% perf-profile.children.cycles-pp.__rdgsbase_inactive
0.28 ± 2% -0.2 0.11 ± 10% perf-profile.children.cycles-pp.tracing_gen_ctx_irq_test
0.27 -0.2 0.10 ± 14% perf-profile.children.cycles-pp.finish_wait
0.27 ± 4% -0.2 0.10 ± 14% perf-profile.children.cycles-pp.current_time
0.37 ± 2% -0.2 0.21 ± 8% perf-profile.children.cycles-pp.__pthread_disable_asynccancel
0.25 ± 5% -0.2 0.10 ± 11% perf-profile.children.cycles-pp.perf_swevent_event
0.65 ± 2% -0.2 0.50 ± 8% perf-profile.children.cycles-pp.perf_output_begin_forward
0.24 ± 10% -0.1 0.09 ± 9% perf-profile.children.cycles-pp.perf_output_end
0.24 ± 3% -0.1 0.10 ± 13% perf-profile.children.cycles-pp.get_stack_info_noinstr
0.24 ± 2% -0.1 0.10 ± 14% perf-profile.children.cycles-pp.hrtimer_get_next_event
0.25 -0.1 0.11 ± 15% perf-profile.children.cycles-pp.__flush_smp_call_function_queue
0.24 ± 4% -0.1 0.10 ± 11% perf-profile.children.cycles-pp.perf_trace_run_bpf_submit
0.23 ± 6% -0.1 0.09 ± 10% perf-profile.children.cycles-pp.cpuidle_governor_latency_req
0.22 ± 6% -0.1 0.09 ± 14% perf-profile.children.cycles-pp.ttwu_do_wakeup
0.24 ± 3% -0.1 0.11 ± 8% perf-profile.children.cycles-pp.__cond_resched
0.24 ± 2% -0.1 0.11 ± 13% perf-profile.children.cycles-pp.__is_insn_slot_addr
0.23 ± 3% -0.1 0.11 ± 9% perf-profile.children.cycles-pp.ct_kernel_exit_state
0.22 ± 4% -0.1 0.10 ± 14% perf-profile.children.cycles-pp.__hrtimer_next_event_base
0.18 ± 2% -0.1 0.06 ± 10% perf-profile.children.cycles-pp.place_entity
0.23 ± 3% -0.1 0.12 ± 8% perf-profile.children.cycles-pp.ct_kernel_enter
0.59 ± 15% -0.1 0.48 ± 7% perf-profile.children.cycles-pp.shmem_get_folio_gfp
0.21 ± 6% -0.1 0.10 ± 12% perf-profile.children.cycles-pp.file_update_time
0.19 ± 7% -0.1 0.08 ± 21% perf-profile.children.cycles-pp.update_min_vruntime
0.17 ± 2% -0.1 0.06 ± 14% perf-profile.children.cycles-pp.put_prev_task_fair
0.17 ± 3% -0.1 0.06 ± 12% perf-profile.children.cycles-pp.switch_fpu_return
0.17 ± 4% -0.1 0.07 ± 15% perf-profile.children.cycles-pp.__calc_delta
0.18 ± 4% -0.1 0.07 ± 10% perf-profile.children.cycles-pp.perf_swevent_get_recursion_context
0.16 ± 2% -0.1 0.06 ± 12% perf-profile.children.cycles-pp.testcase
0.18 ± 2% -0.1 0.07 ± 18% perf-profile.children.cycles-pp.pick_next_entity
0.21 ± 3% -0.1 0.11 ± 15% perf-profile.children.cycles-pp.select_idle_sibling
0.18 ± 4% -0.1 0.08 ± 11% perf-profile.children.cycles-pp.__smp_call_single_queue
0.15 ± 2% -0.1 0.06 ± 12% perf-profile.children.cycles-pp.__list_del_entry_valid
0.15 ± 6% -0.1 0.06 ± 14% perf-profile.children.cycles-pp.aa_file_perm
0.15 ± 2% -0.1 0.06 ± 14% perf-profile.children.cycles-pp.__perf_event_account_interrupt
0.16 ± 4% -0.1 0.07 ± 13% perf-profile.children.cycles-pp.is_bpf_text_address
0.15 ± 2% -0.1 0.06 ± 18% perf-profile.children.cycles-pp.pick_next_task_idle
0.13 ± 5% -0.1 0.04 ± 50% perf-profile.children.cycles-pp.in_task_stack
0.15 ± 4% -0.1 0.06 ± 12% perf-profile.children.cycles-pp.__list_add_valid
0.13 ± 4% -0.1 0.04 ± 50% perf-profile.children.cycles-pp.syscall_enter_from_user_mode
0.16 ± 4% -0.1 0.08 ± 13% perf-profile.children.cycles-pp.__pthread_enable_asynccancel
0.16 ± 4% -0.1 0.08 ± 13% perf-profile.children.cycles-pp.nohz_run_idle_balance
0.13 ± 8% -0.1 0.05 ± 52% perf-profile.children.cycles-pp.check_preempt_curr
0.12 ± 8% -0.1 0.03 ± 82% perf-profile.children.cycles-pp.rb_next
0.12 ± 14% -0.1 0.04 ± 50% perf-profile.children.cycles-pp.cpuacct_charge
0.34 ± 4% -0.1 0.25 ± 4% perf-profile.children.cycles-pp.native_irq_return_iret
0.12 ± 6% -0.1 0.03 ± 84% perf-profile.children.cycles-pp.__update_idle_core
0.14 ± 7% -0.1 0.06 ± 12% perf-profile.children.cycles-pp.rcu_note_context_switch
0.19 ± 14% -0.1 0.11 ± 10% perf-profile.children.cycles-pp.shmem_add_to_page_cache
0.11 ± 5% -0.1 0.03 ± 82% perf-profile.children.cycles-pp.is_module_text_address
0.20 ± 15% -0.1 0.12 ± 15% perf-profile.children.cycles-pp.shmem_alloc_and_acct_folio
0.12 ± 3% -0.1 0.04 ± 50% perf-profile.children.cycles-pp.__get_task_ioprio
0.14 ± 2% -0.1 0.07 ± 7% perf-profile.children.cycles-pp.llist_add_batch
0.17 ± 12% -0.1 0.10 ± 32% perf-profile.children.cycles-pp.arch_cpu_idle_enter
0.13 ± 5% -0.1 0.06 ± 12% perf-profile.children.cycles-pp.sched_clock
0.13 ± 5% -0.1 0.06 ± 12% perf-profile.children.cycles-pp.tick_nohz_tick_stopped
0.13 ± 3% -0.1 0.05 ± 9% perf-profile.children.cycles-pp.send_call_function_single_ipi
0.12 ± 4% -0.1 0.04 ± 50% perf-profile.children.cycles-pp.is_ftrace_trampoline
0.11 ± 4% -0.1 0.04 ± 50% perf-profile.children.cycles-pp.schedule_debug
0.20 ± 16% -0.1 0.13 ± 27% perf-profile.children.cycles-pp.ordered_events__queue
0.16 ± 12% -0.1 0.09 ± 33% perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
0.13 ± 4% -0.1 0.06 ± 6% perf-profile.children.cycles-pp.rcu_all_qs
0.16 ± 16% -0.1 0.09 ± 14% perf-profile.children.cycles-pp.shmem_alloc_folio
0.20 ± 4% -0.1 0.14 ± 5% perf-profile.children.cycles-pp.native_apic_msr_eoi_write
0.14 ± 16% -0.1 0.09 ± 13% perf-profile.children.cycles-pp.vma_alloc_folio
0.09 ± 15% -0.1 0.04 ± 51% perf-profile.children.cycles-pp.get_page_from_freelist
0.10 ± 6% -0.0 0.05 ± 7% perf-profile.children.cycles-pp.__x2apic_send_IPI_dest
0.11 ± 19% -0.0 0.06 ± 12% perf-profile.children.cycles-pp.folio_add_lru
0.12 ± 16% -0.0 0.07 ± 10% perf-profile.children.cycles-pp.__folio_alloc
0.12 ± 16% -0.0 0.07 ± 10% perf-profile.children.cycles-pp.__alloc_pages
0.10 ± 16% -0.0 0.06 ± 12% perf-profile.children.cycles-pp.folio_batch_move_lru
0.13 ± 3% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.irqtime_account_irq
0.05 ± 50% +0.0 0.09 ± 17% perf-profile.children.cycles-pp.__filemap_get_folio
0.10 ± 14% +0.0 0.14 ± 17% perf-profile.children.cycles-pp.perf_session__deliver_event
0.10 ± 16% +0.0 0.14 ± 18% perf-profile.children.cycles-pp.perf_session__process_user_event
0.10 ± 16% +0.0 0.14 ± 18% perf-profile.children.cycles-pp.__ordered_events__flush
0.04 ± 50% +0.0 0.09 ± 14% perf-profile.children.cycles-pp.update_sg_lb_stats
0.06 ± 8% +0.1 0.11 ± 15% perf-profile.children.cycles-pp.update_sd_lb_stats
0.06 ± 6% +0.1 0.12 ± 13% perf-profile.children.cycles-pp.find_busiest_group
0.00 +0.1 0.07 ± 12% perf-profile.children.cycles-pp.folio_mark_accessed
0.08 ± 10% +0.1 0.15 ± 11% perf-profile.children.cycles-pp.load_balance
0.00 +0.1 0.08 ± 27% perf-profile.children.cycles-pp.__perf_mmap__read_init
0.00 +0.1 0.08 ± 20% perf-profile.children.cycles-pp.update_blocked_averages
0.00 +0.1 0.13 ± 27% perf-profile.children.cycles-pp.perf_mmap__read_init
0.00 +0.2 0.23 ± 18% perf-profile.children.cycles-pp.ring_buffer_read_head
0.00 +0.2 0.24 ± 17% perf-profile.children.cycles-pp.perf_mmap__read_head
0.22 ± 2% +0.5 0.69 ± 7% perf-profile.children.cycles-pp.switch_mm_irqs_off
41.73 +1.7 43.40 perf-profile.children.cycles-pp.__libc_read
37.62 +4.2 41.84 perf-profile.children.cycles-pp.ksys_read
36.89 +4.7 41.60 perf-profile.children.cycles-pp.vfs_read
36.13 +5.2 41.30 perf-profile.children.cycles-pp.pipe_read
32.16 +7.7 39.86 perf-profile.children.cycles-pp.schedule
34.12 +10.2 44.37 perf-profile.children.cycles-pp.start_secondary
35.09 +10.5 45.63 perf-profile.children.cycles-pp.secondary_startup_64_no_verify
35.09 +10.5 45.63 perf-profile.children.cycles-pp.cpu_startup_entry
34.97 +10.6 45.57 perf-profile.children.cycles-pp.do_idle
14.68 +18.3 32.96 ± 4% perf-profile.children.cycles-pp.schedule_idle
46.43 +26.2 72.65 ± 2% perf-profile.children.cycles-pp.__schedule
0.00 +49.6 49.57 ± 8% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.81 ± 2% +50.9 51.66 ± 8% perf-profile.children.cycles-pp.prepare_task_switch
0.58 ± 2% +50.9 51.44 ± 8% perf-profile.children.cycles-pp._raw_spin_lock
10.88 ± 2% -6.3 4.59 ± 13% perf-profile.self.cycles-pp.unwind_next_frame
6.51 -3.7 2.77 ± 13% perf-profile.self.cycles-pp.__orc_find
3.68 -2.1 1.54 ± 13% perf-profile.self.cycles-pp.stack_access_ok
2.97 ± 2% -1.7 1.27 ± 13% perf-profile.self.cycles-pp.orc_find
4.42 ± 2% -1.4 3.00 ± 10% perf-profile.self.cycles-pp.poll_idle
2.09 -1.3 0.82 ± 15% perf-profile.self.cycles-pp.memcpy_erms
1.91 -1.1 0.80 ± 14% perf-profile.self.cycles-pp.__output_copy
2.28 ± 10% -1.0 1.32 perf-profile.self.cycles-pp.__schedule
1.58 -0.9 0.64 ± 12% perf-profile.self.cycles-pp.__unwind_start
1.50 ± 2% -0.9 0.61 ± 14% perf-profile.self.cycles-pp.perf_callchain_kernel
1.19 ± 2% -0.7 0.47 ± 13% perf-profile.self.cycles-pp.__get_user_nocheck_8
1.21 -0.7 0.50 ± 13% perf-profile.self.cycles-pp.perf_output_sample
1.29 ± 2% -0.7 0.58 ± 8% perf-profile.self.cycles-pp.native_sched_clock
1.14 ± 3% -0.7 0.44 ± 13% perf-profile.self.cycles-pp.__switch_to
1.01 ± 2% -0.6 0.37 ± 12% perf-profile.self.cycles-pp.__switch_to_asm
1.08 -0.6 0.46 ± 10% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
1.43 ± 11% -0.6 0.81 ± 15% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
0.91 ± 2% -0.5 0.37 ± 14% perf-profile.self.cycles-pp.kernel_text_address
1.47 ± 15% -0.5 0.93 perf-profile.self.cycles-pp.finish_task_switch
0.96 -0.5 0.46 ± 8% perf-profile.self.cycles-pp.menu_select
0.74 ± 2% -0.5 0.28 ± 12% perf-profile.self.cycles-pp.perf_tp_event
0.73 ± 3% -0.4 0.29 ± 14% perf-profile.self.cycles-pp.get_perf_callchain
0.75 ± 3% -0.4 0.31 ± 13% perf-profile.self.cycles-pp.unwind_get_return_address
0.65 ± 2% -0.4 0.21 ± 15% perf-profile.self.cycles-pp.pipe_read
0.72 ± 2% -0.4 0.29 ± 11% perf-profile.self.cycles-pp.perf_prepare_sample
0.70 -0.4 0.27 ± 13% perf-profile.self.cycles-pp.enqueue_entity
0.74 ± 2% -0.4 0.31 ± 12% perf-profile.self.cycles-pp.ftrace_graph_ret_addr
0.71 ± 2% -0.4 0.28 ± 14% perf-profile.self.cycles-pp.update_load_avg
0.72 ± 2% -0.4 0.30 ± 12% perf-profile.self.cycles-pp.__kernel_text_address
0.61 ± 2% -0.4 0.26 ± 14% perf-profile.self.cycles-pp.do_idle
0.60 -0.3 0.26 ± 9% perf-profile.self.cycles-pp.cpuidle_idle_call
0.62 ± 5% -0.3 0.27 ± 14% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.58 ± 2% -0.3 0.24 ± 13% perf-profile.self.cycles-pp.core_kernel_text
0.58 ± 9% -0.3 0.25 ± 12% perf-profile.self.cycles-pp.__fget_files
0.63 -0.3 0.31 ± 13% perf-profile.self.cycles-pp.__update_load_avg_se
0.51 ± 3% -0.3 0.20 ± 14% perf-profile.self.cycles-pp.prepare_to_wait_event
0.50 ± 3% -0.3 0.19 ± 12% perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack
0.51 ± 2% -0.3 0.21 ± 7% perf-profile.self.cycles-pp.__libc_write
0.50 ± 3% -0.3 0.20 ± 13% perf-profile.self.cycles-pp.os_xsave
0.49 ± 2% -0.3 0.19 ± 14% perf-profile.self.cycles-pp.perf_callchain_user
0.50 -0.3 0.20 ± 15% perf-profile.self.cycles-pp.perf_output_copy
0.49 ± 5% -0.3 0.19 ± 14% perf-profile.self.cycles-pp.__task_pid_nr_ns
0.49 -0.3 0.20 ± 13% perf-profile.self.cycles-pp.__libc_read
0.49 ± 3% -0.3 0.20 ± 20% perf-profile.self.cycles-pp.update_curr
0.44 ± 2% -0.3 0.16 ± 14% perf-profile.self.cycles-pp.kvm_guest_state
0.44 -0.3 0.17 ± 10% perf-profile.self.cycles-pp.__wrgsbase_inactive
0.47 ± 2% -0.3 0.20 ± 13% perf-profile.self.cycles-pp.__perf_event_header__init_id
0.37 ± 3% -0.2 0.13 ± 13% perf-profile.self.cycles-pp.pipe_write
0.43 ± 2% -0.2 0.18 ± 8% perf-profile.self.cycles-pp.__might_resched
0.39 ± 2% -0.2 0.14 ± 14% perf-profile.self.cycles-pp.vfs_read
0.37 ± 4% -0.2 0.14 ± 13% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.44 ± 2% -0.2 0.22 ± 5% perf-profile.self.cycles-pp.read_tsc
0.41 ± 6% -0.2 0.19 ± 15% perf-profile.self.cycles-pp.dequeue_entity
0.34 ± 7% -0.2 0.12 ± 16% perf-profile.self.cycles-pp.get_callchain_entry
0.35 ± 2% -0.2 0.14 ± 14% perf-profile.self.cycles-pp.perf_trace_sched_switch
0.31 ± 5% -0.2 0.12 ± 15% perf-profile.self.cycles-pp.update_cfs_group
0.32 -0.2 0.12 ± 13% perf-profile.self.cycles-pp.syscall_return_via_sysret
0.29 ± 6% -0.2 0.09 ± 10% perf-profile.self.cycles-pp.try_to_wake_up
0.32 ± 5% -0.2 0.13 ± 12% perf-profile.self.cycles-pp.enqueue_task_fair
0.27 ± 3% -0.2 0.08 ± 14% perf-profile.self.cycles-pp.ksys_read
0.30 ± 2% -0.2 0.11 ± 11% perf-profile.self.cycles-pp.pick_next_task_fair
0.29 ± 2% -0.2 0.11 ± 8% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.31 ± 3% -0.2 0.13 ± 15% perf-profile.self.cycles-pp.perf_event_output_forward
0.31 ± 4% -0.2 0.14 ± 15% perf-profile.self.cycles-pp.mutex_lock
0.27 ± 3% -0.2 0.09 ± 15% perf-profile.self.cycles-pp.do_syscall_64
0.27 ± 4% -0.2 0.10 ± 14% perf-profile.self.cycles-pp.__perf_event_overflow
0.27 ± 5% -0.2 0.10 ± 13% perf-profile.self.cycles-pp.reweight_entity
0.29 ± 3% -0.2 0.12 ± 8% perf-profile.self.cycles-pp.update_rq_clock
0.29 ± 4% -0.2 0.13 ± 10% perf-profile.self.cycles-pp.mutex_unlock
0.25 ± 3% -0.2 0.08 ± 14% perf-profile.self.cycles-pp.__rdgsbase_inactive
0.26 ± 3% -0.2 0.10 ± 14% perf-profile.self.cycles-pp.dequeue_task_fair
0.26 ± 2% -0.2 0.10 ± 11% perf-profile.self.cycles-pp.tracing_gen_ctx_irq_test
0.36 ± 3% -0.2 0.21 ± 13% perf-profile.self.cycles-pp.___perf_sw_event
0.24 -0.2 0.09 ± 12% perf-profile.self.cycles-pp.fput
0.34 ± 2% -0.2 0.19 ± 9% perf-profile.self.cycles-pp.__pthread_disable_asynccancel
0.23 ± 3% -0.1 0.08 ± 13% perf-profile.self.cycles-pp.__might_sleep
0.25 ± 2% -0.1 0.10 ± 13% perf-profile.self.cycles-pp.set_next_entity
0.29 -0.1 0.14 ± 12% perf-profile.self.cycles-pp.sched_clock_cpu
0.23 ± 4% -0.1 0.09 ± 15% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.63 ± 2% -0.1 0.49 ± 7% perf-profile.self.cycles-pp.perf_output_begin_forward
0.22 ± 2% -0.1 0.08 ± 11% perf-profile.self.cycles-pp.newidle_balance
0.18 ± 2% -0.1 0.05 ± 52% perf-profile.self.cycles-pp._copy_from_iter
0.22 ± 11% -0.1 0.08 ± 9% perf-profile.self.cycles-pp.perf_output_end
0.28 ± 22% -0.1 0.15 ± 38% perf-profile.self.cycles-pp.tick_nohz_next_event
0.22 -0.1 0.09 ± 11% perf-profile.self.cycles-pp.schedule_idle
0.23 ± 2% -0.1 0.10 ± 7% perf-profile.self.cycles-pp.vfs_write
0.54 ± 3% -0.1 0.41 ± 4% perf-profile.self.cycles-pp.cpuidle_enter_state
0.20 ± 2% -0.1 0.08 ± 13% perf-profile.self.cycles-pp.perf_callchain
0.21 ± 6% -0.1 0.08 ± 15% perf-profile.self.cycles-pp.perf_swevent_event
0.22 ± 2% -0.1 0.09 ± 14% perf-profile.self.cycles-pp.get_stack_info
0.20 ± 3% -0.1 0.08 ± 15% perf-profile.self.cycles-pp.schedule
0.20 ± 3% -0.1 0.08 ± 9% perf-profile.self.cycles-pp.memcpy
0.21 ± 3% -0.1 0.08 ± 12% perf-profile.self.cycles-pp.perf_trace_buf_alloc
0.22 ± 2% -0.1 0.10 ± 6% perf-profile.self.cycles-pp.ct_kernel_exit_state
0.20 ± 2% -0.1 0.09 ± 8% perf-profile.self.cycles-pp.perf_trace_run_bpf_submit
0.20 ± 2% -0.1 0.08 ± 14% perf-profile.self.cycles-pp.__flush_smp_call_function_queue
0.19 ± 5% -0.1 0.07 ± 18% perf-profile.self.cycles-pp.apparmor_file_permission
0.18 ± 2% -0.1 0.07 ± 15% perf-profile.self.cycles-pp.sched_ttwu_pending
0.19 ± 8% -0.1 0.08 ± 14% perf-profile.self.cycles-pp.__entry_text_start
0.18 ± 2% -0.1 0.07 ± 16% perf-profile.self.cycles-pp.perf_trace_buf_update
0.17 ± 2% -0.1 0.07 ± 16% perf-profile.self.cycles-pp.cpu_startup_entry
0.16 ± 3% -0.1 0.06 ± 14% perf-profile.self.cycles-pp.switch_fpu_return
0.23 ± 3% -0.1 0.13 ± 15% perf-profile.self.cycles-pp.select_task_rq_fair
0.30 ± 2% -0.1 0.20 ± 17% perf-profile.self.cycles-pp.prepare_task_switch
0.18 ± 2% -0.1 0.09 ± 13% perf-profile.self.cycles-pp.__is_insn_slot_addr
0.13 ± 3% -0.1 0.03 ± 82% perf-profile.self.cycles-pp.tick_nohz_idle_exit
0.15 ± 3% -0.1 0.06 ± 12% perf-profile.self.cycles-pp.current_time
0.15 ± 3% -0.1 0.06 ± 12% perf-profile.self.cycles-pp.testcase
0.16 ± 5% -0.1 0.07 ± 15% perf-profile.self.cycles-pp.__calc_delta
0.15 ± 3% -0.1 0.06 ± 15% perf-profile.self.cycles-pp.perf_misc_flags
0.17 ± 2% -0.1 0.08 ± 9% perf-profile.self.cycles-pp.__hrtimer_next_event_base
0.17 ± 5% -0.1 0.08 ± 18% perf-profile.self.cycles-pp.ttwu_queue_wakelist
0.15 ± 3% -0.1 0.06 ± 12% perf-profile.self.cycles-pp.pick_next_entity
0.15 ± 7% -0.1 0.06 ± 14% perf-profile.self.cycles-pp.flush_smp_call_function_queue
0.13 ± 7% -0.1 0.05 ± 52% perf-profile.self.cycles-pp.aa_file_perm
0.13 ± 3% -0.1 0.04 ± 51% perf-profile.self.cycles-pp.__perf_event_account_interrupt
0.15 ± 6% -0.1 0.06 ± 21% perf-profile.self.cycles-pp.update_min_vruntime
0.15 ± 5% -0.1 0.06 ± 12% perf-profile.self.cycles-pp.perf_swevent_get_recursion_context
0.16 -0.1 0.07 ± 13% perf-profile.self.cycles-pp.perf_trace_sched_stat_runtime
0.15 ± 5% -0.1 0.07 ± 15% perf-profile.self.cycles-pp.nohz_run_idle_balance
0.34 ± 4% -0.1 0.25 ± 4% perf-profile.self.cycles-pp.native_irq_return_iret
0.14 ± 5% -0.1 0.06 ± 12% perf-profile.self.cycles-pp.get_next_timer_interrupt
0.12 ± 3% -0.1 0.04 ± 51% perf-profile.self.cycles-pp.send_call_function_single_ipi
0.11 ± 3% -0.1 0.03 ± 82% perf-profile.self.cycles-pp.tick_nohz_tick_stopped
0.14 ± 5% -0.1 0.06 ± 14% perf-profile.self.cycles-pp.llist_add_batch
0.12 ± 4% -0.1 0.04 ± 50% perf-profile.self.cycles-pp.__list_add_valid
0.12 ± 15% -0.1 0.04 ± 50% perf-profile.self.cycles-pp.cpuacct_charge
0.13 ± 3% -0.1 0.06 ± 10% perf-profile.self.cycles-pp.ct_idle_exit
0.13 ± 3% -0.1 0.05 ± 9% perf-profile.self.cycles-pp.get_stack_info_noinstr
0.14 ± 2% -0.1 0.07 ± 9% perf-profile.self.cycles-pp.__pthread_enable_asynccancel
0.13 ± 7% -0.1 0.06 ± 14% perf-profile.self.cycles-pp.rcu_note_context_switch
0.12 ± 4% -0.1 0.04 ± 51% perf-profile.self.cycles-pp.ct_kernel_enter
0.13 ± 5% -0.1 0.06 ± 6% perf-profile.self.cycles-pp.__cond_resched
0.11 ± 3% -0.1 0.04 ± 50% perf-profile.self.cycles-pp.perf_trace_sched_wakeup_template
0.19 ± 17% -0.1 0.12 ± 27% perf-profile.self.cycles-pp.queue_event
0.13 ± 15% -0.1 0.07 ± 63% perf-profile.self.cycles-pp.tsc_verify_tsc_adjust
0.20 ± 5% -0.1 0.13 ± 3% perf-profile.self.cycles-pp.native_apic_msr_eoi_write
0.10 ± 4% -0.0 0.05 ± 7% perf-profile.self.cycles-pp.__x2apic_send_IPI_dest
0.01 ±200% +0.1 0.07 ± 13% perf-profile.self.cycles-pp.update_sg_lb_stats
0.00 +0.2 0.21 ± 18% perf-profile.self.cycles-pp.ring_buffer_read_head
0.21 ± 2% +0.5 0.69 ± 6% perf-profile.self.cycles-pp.switch_mm_irqs_off
0.54 +1.3 1.84 perf-profile.self.cycles-pp._raw_spin_lock
0.00 +49.6 49.57 ± 8% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath***************************************************************************************************
lkp-skl-fpga01: 104 threads 2 sockets (Skylake) with 192G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase:
gcc-11/performance/x86_64-rhel-8.3/thread/16/debian-11.1-x86_64-20220510.cgz/lkp-skl-fpga01/context_switch1/will-it-scale
commit:
99babd04b2 ("selftests/rseq: Implement rseq numa node id field selftest") af7f588d8f ("sched: Introduce per-memory-map concurrency ID")
99babd04b2505471 af7f588d8f7355bc4298dd1962d
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
1:6 -17% :6 kmsg.EDAC_skx:Can't_find_handle_for_NVDIMM_ADR=
1:6 -17% :6 kmsg.microcode:Attempting_late_microcode_loading-it_is_dangerous_and_taints_the_kernel
1:6 -17% :6 kmsg.microcode:You_should_switch_to_early_loading,if_possible
%stddev %change %stddev
\ | \
4324658 ± 3% -57.9% 1818800 will-it-scale.16.threads
270290 ± 3% -57.9% 113674 will-it-scale.per_thread_ops
4324658 ± 3% -57.9% 1818800 will-it-scale.workload
1.329e+09 ± 3% -58.0% 5.581e+08 cpuidle..usage
12.36 ± 2% +2.9 15.22 mpstat.cpu.all.sys%
5.59 -3.0 2.57 ± 4% mpstat.cpu.all.usr%
19.17 -11.3% 17.00 vmstat.procs.r
8547612 ± 2% -58.0% 3593440 vmstat.system.cs
233179 +16.0% 270468 ± 2% vmstat.system.in
158734 ± 25% +182.8% 448837 ± 18% turbostat.C1
0.14 ± 5% -48.8% 0.07 turbostat.IPC
70953881 +16.1% 82393754 ± 2% turbostat.IRQ
1.286e+09 ± 3% -60.0% 5.137e+08 turbostat.POLL
1.67 ± 6% -0.7 1.00 ± 2% turbostat.POLL%
252.48 -8.1% 232.09 turbostat.PkgWatt
114167 ± 20% -93.5% 7417 ± 59% sched_debug.cfs_rq:/.MIN_vruntime.avg
1525472 -76.4% 359932 ± 59% sched_debug.cfs_rq:/.MIN_vruntime.max
387653 ± 11% -87.1% 49952 ± 56% sched_debug.cfs_rq:/.MIN_vruntime.stddev
0.39 ± 6% -24.3% 0.29 ± 2% sched_debug.cfs_rq:/.h_nr_running.avg
22932 ± 5% +13.0% 25920 ± 3% sched_debug.cfs_rq:/.load.avg
59500 ± 4% +55.5% 92526 ± 68% sched_debug.cfs_rq:/.load.max
114167 ± 20% -93.5% 7417 ± 59% sched_debug.cfs_rq:/.max_vruntime.avg
1525472 -76.4% 359932 ± 59% sched_debug.cfs_rq:/.max_vruntime.max
387653 ± 11% -87.1% 49952 ± 56% sched_debug.cfs_rq:/.max_vruntime.stddev
671631 ± 3% -16.0% 564476 ± 2% sched_debug.cfs_rq:/.min_vruntime.avg
1542612 -17.8% 1267561 sched_debug.cfs_rq:/.min_vruntime.max
708750 -18.9% 575136 sched_debug.cfs_rq:/.min_vruntime.stddev
0.37 ± 6% -24.7% 0.28 ± 2% sched_debug.cfs_rq:/.nr_running.avg
-848558 -19.6% -682164 sched_debug.cfs_rq:/.spread0.avg
-1486548 -18.5% -1211535 sched_debug.cfs_rq:/.spread0.min
708763 -18.9% 575149 sched_debug.cfs_rq:/.spread0.stddev
214.42 -10.2% 192.45 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.stddev
3058 ± 4% +113.4% 6526 ± 6% sched_debug.cpu.avg_idle.min
1063 ± 2% -14.8% 906.25 ± 5% sched_debug.cpu.curr->pid.avg
0.20 ± 4% -13.3% 0.17 ± 2% sched_debug.cpu.nr_running.avg
12311145 ± 3% -57.7% 5209198 sched_debug.cpu.nr_switches.avg
40179926 ± 3% -57.8% 16957686 sched_debug.cpu.nr_switches.max
18457770 ± 3% -57.7% 7805110 sched_debug.cpu.nr_switches.stddev
1.48 ± 33% +393.8% 7.31 ± 14% perf-stat.i.MPKI
9.306e+09 -51.2% 4.543e+09 ± 5% perf-stat.i.branch-instructions
1.571e+08 ± 2% -49.6% 79141069 ± 4% perf-stat.i.branch-misses
67978805 ± 32% +129.9% 1.563e+08 ± 13% perf-stat.i.cache-references
8635700 ± 3% -58.0% 3628862 perf-stat.i.context-switches
2.00 +112.1% 4.24 ± 2% perf-stat.i.cpi
152.95 ± 4% +10.5% 168.99 ± 2% perf-stat.i.cpu-migrations
9807863 ± 8% -54.4% 4473079 ± 12% perf-stat.i.dTLB-load-misses
1.313e+10 -52.5% 6.241e+09 ± 4% perf-stat.i.dTLB-loads
7.867e+09 -54.9% 3.549e+09 ± 4% perf-stat.i.dTLB-stores
9070094 ± 3% -58.2% 3793161 ± 49% perf-stat.i.iTLB-load-misses
11272936 ± 7% -50.6% 5566942 ± 8% perf-stat.i.iTLB-loads
4.592e+10 -51.8% 2.214e+10 ± 5% perf-stat.i.instructions
0.50 -51.9% 0.24 ± 4% perf-stat.i.ipc
660.44 ± 31% -85.8% 93.67 ± 4% perf-stat.i.metric.K/sec
291.58 -52.2% 139.32 ± 4% perf-stat.i.metric.M/sec
1.48 ± 32% +375.8% 7.06 ± 12% perf-stat.overall.MPKI
1.99 +108.2% 4.14 ± 4% perf-stat.overall.cpi
0.50 -51.9% 0.24 ± 4% perf-stat.overall.ipc
3197936 ± 3% +14.7% 3668175 ± 4% perf-stat.overall.path-length
9.276e+09 -51.2% 4.528e+09 ± 5% perf-stat.ps.branch-instructions
1.566e+08 ± 2% -49.6% 78882571 ± 4% perf-stat.ps.branch-misses
67767931 ± 32% +129.8% 1.557e+08 ± 13% perf-stat.ps.cache-references
8606808 ± 3% -58.0% 3616809 perf-stat.ps.context-switches
152.61 ± 4% +10.4% 168.54 ± 2% perf-stat.ps.cpu-migrations
9780257 ± 8% -54.4% 4458181 ± 12% perf-stat.ps.dTLB-load-misses
1.309e+10 -52.5% 6.221e+09 ± 4% perf-stat.ps.dTLB-loads
7.842e+09 -54.9% 3.538e+09 ± 4% perf-stat.ps.dTLB-stores
9039813 ± 3% -58.2% 3780389 ± 49% perf-stat.ps.iTLB-load-misses
11235261 ± 7% -50.6% 5548664 ± 8% perf-stat.ps.iTLB-loads
4.577e+10 -51.8% 2.207e+10 ± 5% perf-stat.ps.instructions
1.381e+13 -51.7% 6.671e+12 ± 4% perf-stat.total.instructions
20.96 ± 7% -9.2 11.78 ± 32% perf-profile.calltrace.cycles-pp.__libc_write
15.02 ± 20% -4.7 10.34 ± 8% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
5.46 ± 28% -2.9 2.58 ± 7% perf-profile.calltrace.cycles-pp.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
4.64 ± 32% -2.4 2.25 ± 11% perf-profile.calltrace.cycles-pp.sched_ttwu_pending.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary
4.55 ± 28% -2.4 2.18 ± 8% perf-profile.calltrace.cycles-pp.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
3.70 ± 37% -1.9 1.84 ± 12% perf-profile.calltrace.cycles-pp.ttwu_do_activate.sched_ttwu_pending.flush_smp_call_function_queue.do_idle.cpu_startup_entry
3.55 ± 39% -1.8 1.78 ± 12% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.sched_ttwu_pending.flush_smp_call_function_queue.do_idle
2.78 ± 33% -1.5 1.31 ± 6% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__libc_read
2.63 ± 39% -1.4 1.28 ± 7% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.__libc_write
2.60 ± 32% -1.3 1.33 ± 6% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
2.27 ± 20% -1.2 1.05 ± 4% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule_idle.do_idle.cpu_startup_entry
2.29 ± 35% -1.1 1.15 ± 4% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
2.08 ± 32% -1.1 0.94 ± 12% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry
1.74 ± 34% -0.9 0.83 ± 3% perf-profile.calltrace.cycles-pp.tick_nohz_idle_exit.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
1.25 ± 45% -0.9 0.36 ± 71% perf-profile.calltrace.cycles-pp.ktime_get.tick_nohz_idle_exit.do_idle.cpu_startup_entry.start_secondary
1.18 ± 29% -0.9 0.30 ±100% perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call.do_idle
1.07 ± 34% -0.8 0.26 ±100% perf-profile.calltrace.cycles-pp.__entry_text_start.__libc_read
1.06 ± 38% -0.8 0.27 ±100% perf-profile.calltrace.cycles-pp.__entry_text_start.__libc_write
1.03 ± 21% -0.7 0.28 ±100% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.vfs_write.ksys_write.do_syscall_64
1.38 ± 23% -0.7 0.67 ± 4% perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule_idle.do_idle
0.00 +0.8 0.83 ± 11% perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule_idle.do_idle.cpu_startup_entry
40.47 ± 4% +2.9 43.39 ± 2% perf-profile.calltrace.cycles-pp.__libc_read
34.36 ± 8% +6.8 41.18 ± 10% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
34.34 ± 8% +6.8 41.17 ± 10% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
34.08 ± 8% +7.0 41.04 ± 10% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
35.35 ± 8% +7.0 42.35 ± 10% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
32.51 ± 14% +7.7 40.21 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_read
30.86 ± 16% +8.5 39.36 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
27.80 ± 22% +10.0 37.80 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
27.05 ± 23% +10.4 37.42 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
25.85 ± 26% +11.0 36.84 perf-profile.calltrace.cycles-pp.pipe_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
21.15 ± 38% +13.5 34.64 perf-profile.calltrace.cycles-pp.schedule.pipe_read.vfs_read.ksys_read.do_syscall_64
20.97 ± 39% +13.6 34.56 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_read.vfs_read.ksys_read
9.89 ± 37% +16.4 26.25 ± 18% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
9.57 ± 39% +16.5 26.10 ± 18% perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
0.00 +18.8 18.81 ± 49% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prepare_task_switch.__schedule.schedule_idle
0.00 +19.3 19.34 ± 47% perf-profile.calltrace.cycles-pp._raw_spin_lock.prepare_task_switch.__schedule.schedule_idle.do_idle
0.29 ±100% +19.4 19.72 ± 46% perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry
0.00 +19.8 19.76 ± 45% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prepare_task_switch.__schedule.schedule
0.00 +20.3 20.31 ± 44% perf-profile.calltrace.cycles-pp._raw_spin_lock.prepare_task_switch.__schedule.schedule.pipe_read
0.29 ±100% +20.4 20.67 ± 43% perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule.pipe_read.vfs_read
22.18 ± 7% -9.1 13.08 ± 38% perf-profile.children.cycles-pp.__libc_write
15.50 ± 20% -4.9 10.61 ± 7% perf-profile.children.cycles-pp.cpuidle_idle_call
5.65 ± 29% -3.0 2.66 ± 7% perf-profile.children.cycles-pp.flush_smp_call_function_queue
5.48 ± 35% -2.8 2.67 ± 4% perf-profile.children.cycles-pp.syscall_return_via_sysret
5.08 ± 26% -2.5 2.61 ± 4% perf-profile.children.cycles-pp.sched_ttwu_pending
4.74 ± 28% -2.5 2.27 ± 8% perf-profile.children.cycles-pp.menu_select
4.94 ± 33% -2.4 2.55 ± 4% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
4.06 ± 30% -1.9 2.13 ± 4% perf-profile.children.cycles-pp.ttwu_do_activate
3.89 ± 32% -1.8 2.06 ± 5% perf-profile.children.cycles-pp.enqueue_task_fair
3.16 ± 21% -1.6 1.59 ± 8% perf-profile.children.cycles-pp.pick_next_task_fair
3.19 ± 34% -1.4 1.75 ± 6% perf-profile.children.cycles-pp.enqueue_entity
2.62 ± 18% -1.4 1.25 ± 3% perf-profile.children.cycles-pp.update_load_avg
2.41 ± 35% -1.3 1.15 ± 4% perf-profile.children.cycles-pp.__entry_text_start
2.17 ± 32% -1.2 0.98 ± 12% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
1.76 ± 42% -1.1 0.66 ± 13% perf-profile.children.cycles-pp.__switch_to_asm
1.56 ± 48% -1.1 0.51 ± 9% perf-profile.children.cycles-pp.__switch_to
1.97 ± 34% -1.0 0.95 ± 5% perf-profile.children.cycles-pp.ktime_get
1.84 ± 34% -1.0 0.88 ± 3% perf-profile.children.cycles-pp.tick_nohz_idle_exit
1.54 ± 23% -0.8 0.74 ± 2% perf-profile.children.cycles-pp.set_next_entity
0.98 ± 51% -0.7 0.26 ± 20% perf-profile.children.cycles-pp.os_xsave
1.26 ± 30% -0.7 0.55 ± 16% perf-profile.children.cycles-pp.tick_nohz_next_event
1.15 ± 33% -0.6 0.52 ± 7% perf-profile.children.cycles-pp.mutex_lock
1.40 ± 10% -0.6 0.79 ± 12% perf-profile.children.cycles-pp.sched_clock_cpu
1.17 ± 34% -0.6 0.58 ± 8% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
1.06 ± 32% -0.5 0.52 ± 2% perf-profile.children.cycles-pp.read_tsc
0.93 ± 26% -0.5 0.41 ± 5% perf-profile.children.cycles-pp.select_task_rq
1.04 ± 21% -0.5 0.52 ± 10% perf-profile.children.cycles-pp.copy_page_from_iter
1.18 ± 12% -0.5 0.66 ± 9% perf-profile.children.cycles-pp.native_sched_clock
0.92 ± 34% -0.5 0.47 ± 3% perf-profile.children.cycles-pp.__fdget_pos
0.88 ± 36% -0.4 0.45 ± 3% perf-profile.children.cycles-pp.__fget_light
0.85 ± 36% -0.4 0.42 ± 6% perf-profile.children.cycles-pp.security_file_permission
0.88 ± 34% -0.4 0.45 ± 5% perf-profile.children.cycles-pp.prepare_to_wait_event
0.78 ± 23% -0.4 0.35 ± 5% perf-profile.children.cycles-pp.__update_load_avg_se
0.75 ± 38% -0.4 0.32 ± 10% perf-profile.children.cycles-pp.copy_page_to_iter
0.76 ± 31% -0.4 0.34 ± 7% perf-profile.children.cycles-pp.select_task_rq_fair
0.65 ± 39% -0.4 0.26 ± 21% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.71 ± 42% -0.4 0.32 ± 11% perf-profile.children.cycles-pp.hrtimer_next_event_without
0.75 ± 35% -0.4 0.37 ± 4% perf-profile.children.cycles-pp.apparmor_file_permission
0.80 ± 22% -0.4 0.43 ± 12% perf-profile.children.cycles-pp._copy_from_iter
0.64 ± 34% -0.4 0.26 ± 6% perf-profile.children.cycles-pp._copy_to_iter
0.71 ± 18% -0.4 0.34 ± 4% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.67 ± 38% -0.3 0.34 ± 3% perf-profile.children.cycles-pp.__fget_files
0.75 ± 29% -0.3 0.42 ± 9% perf-profile.children.cycles-pp.update_cfs_group
0.64 ± 40% -0.3 0.32 ± 16% perf-profile.children.cycles-pp.reweight_entity
0.66 ± 22% -0.3 0.34 ± 7% perf-profile.children.cycles-pp.ttwu_queue_wakelist
0.65 ± 20% -0.3 0.36 ± 5% perf-profile.children.cycles-pp.update_rq_clock
0.44 ± 42% -0.3 0.16 ± 6% perf-profile.children.cycles-pp.__wrgsbase_inactive
0.52 ± 31% -0.3 0.25 ± 5% perf-profile.children.cycles-pp.__might_resched
0.47 ± 18% -0.3 0.20 ± 36% perf-profile.children.cycles-pp.__flush_smp_call_function_queue
0.55 ± 24% -0.3 0.29 ± 8% perf-profile.children.cycles-pp.___perf_sw_event
0.47 ± 29% -0.3 0.22 ± 5% perf-profile.children.cycles-pp.put_prev_task_fair
0.54 ± 5% -0.3 0.29 ± 20% perf-profile.children.cycles-pp.touch_atime
0.47 ± 10% -0.3 0.22 ± 12% perf-profile.children.cycles-pp.cpuidle_governor_latency_req
0.45 ± 35% -0.2 0.22 ± 7% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack
0.43 ± 23% -0.2 0.21 ± 9% perf-profile.children.cycles-pp.__might_fault
0.36 ± 37% -0.2 0.15 ± 6% perf-profile.children.cycles-pp.__might_sleep
0.37 ± 42% -0.2 0.15 ± 4% perf-profile.children.cycles-pp.mutex_unlock
0.35 ± 26% -0.2 0.15 ± 23% perf-profile.children.cycles-pp.__calc_delta
0.43 ± 6% -0.2 0.22 ± 23% perf-profile.children.cycles-pp.atime_needs_update
0.35 ± 45% -0.2 0.14 ± 6% perf-profile.children.cycles-pp.file_update_time
0.36 ± 27% -0.2 0.16 ± 8% perf-profile.children.cycles-pp.pick_next_entity
0.53 ± 16% -0.2 0.34 ± 6% perf-profile.children.cycles-pp.tick_nohz_idle_enter
0.39 ± 35% -0.2 0.19 ± 7% perf-profile.children.cycles-pp.__cond_resched
0.38 ± 37% -0.2 0.19 ± 6% perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack
0.28 ± 52% -0.2 0.09 ± 9% perf-profile.children.cycles-pp.__rdgsbase_inactive
0.32 ± 30% -0.2 0.13 ± 12% perf-profile.children.cycles-pp.copyout
0.34 ± 15% -0.2 0.16 ± 24% perf-profile.children.cycles-pp.ttwu_do_wakeup
0.34 ± 8% -0.2 0.18 ± 23% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.33 ± 33% -0.2 0.16 ± 15% perf-profile.children.cycles-pp.exit_to_user_mode_prepare
0.31 ± 38% -0.2 0.15 ± 10% perf-profile.children.cycles-pp.__hrtimer_next_event_base
0.28 ± 30% -0.2 0.12 ± 8% perf-profile.children.cycles-pp.select_idle_sibling
0.29 ± 31% -0.2 0.14 ± 5% perf-profile.children.cycles-pp.copy_user_short_string
0.30 ± 26% -0.2 0.16 ± 9% perf-profile.children.cycles-pp.pick_next_task_idle
0.27 ± 34% -0.1 0.13 ± 5% perf-profile.children.cycles-pp.fput
0.25 ± 39% -0.1 0.11 ± 5% perf-profile.children.cycles-pp.cpuacct_charge
0.28 ± 30% -0.1 0.14 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_irq
0.28 ± 9% -0.1 0.14 ± 17% perf-profile.children.cycles-pp.current_time
0.24 ± 32% -0.1 0.12 ± 6% perf-profile.children.cycles-pp.syscall_enter_from_user_mode
0.22 ± 29% -0.1 0.10 ± 8% perf-profile.children.cycles-pp.finish_wait
0.15 ± 17% -0.1 0.03 ±100% perf-profile.children.cycles-pp.llist_reverse_order
0.22 ± 27% -0.1 0.10 ± 7% perf-profile.children.cycles-pp.arch_cpu_idle_enter
0.23 ± 28% -0.1 0.12 ± 6% perf-profile.children.cycles-pp.put_prev_entity
0.21 ± 26% -0.1 0.09 ± 7% perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
0.20 ± 38% -0.1 0.09 ± 19% perf-profile.children.cycles-pp.rcu_nocb_flush_deferred_wakeup
0.25 ± 28% -0.1 0.14 ± 13% perf-profile.children.cycles-pp.__pthread_disable_asynccancel
0.18 ± 18% -0.1 0.08 ± 10% perf-profile.children.cycles-pp.update_min_vruntime
0.22 ± 13% -0.1 0.11 ± 6% perf-profile.children.cycles-pp.tick_nohz_tick_stopped
0.19 ± 36% -0.1 0.08 ± 11% perf-profile.children.cycles-pp.switch_fpu_return
0.19 ± 29% -0.1 0.09 ± 13% perf-profile.children.cycles-pp.ct_kernel_exit_state
0.16 ± 21% -0.1 0.06 ± 9% perf-profile.children.cycles-pp.__get_task_ioprio
0.20 ± 14% -0.1 0.10 ± 19% perf-profile.children.cycles-pp.check_preempt_curr
0.19 ± 31% -0.1 0.10 ± 16% perf-profile.children.cycles-pp.rcu_note_context_switch
0.19 ± 21% -0.1 0.09 ± 12% perf-profile.children.cycles-pp.anon_pipe_buf_release
0.20 ± 8% -0.1 0.10 ± 23% perf-profile.children.cycles-pp.get_cpu_device
0.16 ± 9% -0.1 0.07 ± 20% perf-profile.children.cycles-pp.rb_erase
0.14 ± 9% -0.1 0.05 ± 45% perf-profile.children.cycles-pp.clear_buddies
0.20 ± 25% -0.1 0.12 ± 10% perf-profile.children.cycles-pp.__update_idle_core
0.16 ± 25% -0.1 0.08 ± 16% perf-profile.children.cycles-pp.llist_add_batch
0.17 ± 22% -0.1 0.09 ± 7% perf-profile.children.cycles-pp.testcase
0.18 ± 34% -0.1 0.09 ± 13% perf-profile.children.cycles-pp.cpuidle_not_available
0.18 ± 9% -0.1 0.10 ± 22% perf-profile.children.cycles-pp.__list_del_entry_valid
0.19 ± 20% -0.1 0.11 ± 6% perf-profile.children.cycles-pp.__smp_call_single_queue
0.17 ± 39% -0.1 0.09 ± 17% perf-profile.children.cycles-pp.tick_check_broadcast_expired
0.13 ± 11% -0.1 0.05 ± 7% perf-profile.children.cycles-pp.put_prev_task_idle
0.18 ± 15% -0.1 0.10 ± 33% perf-profile.children.cycles-pp.nr_iowait_cpu
0.12 ± 44% -0.1 0.04 ± 73% perf-profile.children.cycles-pp.timekeeping_max_deferment
0.11 ± 9% -0.1 0.03 ±100% perf-profile.children.cycles-pp.schedule_debug
0.16 ± 29% -0.1 0.08 ± 7% perf-profile.children.cycles-pp.__pthread_enable_asynccancel
0.14 ± 14% -0.1 0.06 ± 11% perf-profile.children.cycles-pp.tick_nohz_idle_retain_tick
0.18 ± 26% -0.1 0.11 ± 6% perf-profile.children.cycles-pp.nohz_run_idle_balance
0.10 ± 23% -0.1 0.03 ±100% perf-profile.children.cycles-pp.send_call_function_single_ipi
0.14 ± 24% -0.1 0.07 perf-profile.children.cycles-pp.ct_kernel_exit
0.13 ± 23% -0.1 0.06 ± 19% perf-profile.children.cycles-pp.__list_add_valid
0.13 ± 38% -0.1 0.06 ± 14% perf-profile.children.cycles-pp.__x64_sys_write
0.08 ± 13% -0.1 0.03 ±100% perf-profile.children.cycles-pp.init_wait_entry
0.12 ± 30% -0.1 0.06 ± 6% perf-profile.children.cycles-pp.__x64_sys_read
0.08 ± 10% -0.0 0.04 ± 71% perf-profile.children.cycles-pp.child
0.10 ± 19% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.__cgroup_account_cputime
0.00 +0.1 0.07 ± 25% perf-profile.children.cycles-pp._find_first_zero_bit
0.28 ± 23% +0.6 0.85 ± 11% perf-profile.children.cycles-pp.switch_mm_irqs_off
40.66 ± 4% +2.8 43.50 ± 2% perf-profile.children.cycles-pp.__libc_read
34.36 ± 8% +6.8 41.18 ± 10% perf-profile.children.cycles-pp.start_secondary
35.35 ± 8% +7.0 42.35 ± 10% perf-profile.children.cycles-pp.secondary_startup_64_no_verify
35.35 ± 8% +7.0 42.35 ± 10% perf-profile.children.cycles-pp.cpu_startup_entry
35.24 ± 8% +7.0 42.28 ± 10% perf-profile.children.cycles-pp.do_idle
27.86 ± 22% +10.0 37.84 perf-profile.children.cycles-pp.ksys_read
27.09 ± 23% +10.4 37.45 perf-profile.children.cycles-pp.vfs_read
25.98 ± 26% +10.9 36.90 perf-profile.children.cycles-pp.pipe_read
21.16 ± 38% +13.5 34.65 perf-profile.children.cycles-pp.schedule
10.22 ± 36% +16.9 27.08 ± 18% perf-profile.children.cycles-pp.schedule_idle
31.01 ± 38% +30.5 61.54 ± 8% perf-profile.children.cycles-pp.__schedule
0.00 +38.8 38.77 ± 47% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.66 ± 29% +39.5 40.18 ± 45% perf-profile.children.cycles-pp._raw_spin_lock
1.00 ± 18% +39.6 40.61 ± 44% perf-profile.children.cycles-pp.prepare_task_switch
5.48 ± 35% -2.8 2.67 ± 4% perf-profile.self.cycles-pp.syscall_return_via_sysret
4.56 ± 33% -2.2 2.36 ± 5% perf-profile.self.cycles-pp.syscall_exit_to_user_mode
3.36 ± 34% -1.6 1.72 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
2.10 ± 36% -1.1 0.99 ± 4% perf-profile.self.cycles-pp.__entry_text_start
1.75 ± 42% -1.1 0.65 ± 12% perf-profile.self.cycles-pp.__switch_to_asm
1.94 ± 28% -1.0 0.98 ± 5% perf-profile.self.cycles-pp.menu_select
1.42 ± 47% -0.9 0.48 ± 9% perf-profile.self.cycles-pp.__switch_to
2.45 ± 29% -0.8 1.63 perf-profile.self.cycles-pp.__schedule
0.97 ± 51% -0.7 0.26 ± 20% perf-profile.self.cycles-pp.os_xsave
1.21 ± 33% -0.6 0.62 ± 4% perf-profile.self.cycles-pp.enqueue_entity
1.14 ± 33% -0.6 0.57 ± 8% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
1.11 ± 15% -0.6 0.55 ± 4% perf-profile.self.cycles-pp.update_load_avg
1.01 ± 31% -0.5 0.50 perf-profile.self.cycles-pp.read_tsc
1.15 ± 12% -0.5 0.65 ± 9% perf-profile.self.cycles-pp.native_sched_clock
0.95 ± 36% -0.5 0.45 ± 12% perf-profile.self.cycles-pp.ktime_get
0.81 ± 34% -0.4 0.36 ± 6% perf-profile.self.cycles-pp.pipe_read
0.73 ± 17% -0.4 0.29 ± 5% perf-profile.self.cycles-pp.dequeue_entity
0.84 ± 24% -0.4 0.40 ± 7% perf-profile.self.cycles-pp.pipe_write
0.80 ± 41% -0.4 0.37 ± 17% perf-profile.self.cycles-pp.do_idle
0.75 ± 22% -0.4 0.34 ± 5% perf-profile.self.cycles-pp.__update_load_avg_se
0.70 ± 27% -0.4 0.31 ± 6% perf-profile.self.cycles-pp.enqueue_task_fair
0.64 ± 13% -0.4 0.28 ± 5% perf-profile.self.cycles-pp.pick_next_task_fair
0.67 ± 34% -0.3 0.33 ± 4% perf-profile.self.cycles-pp.apparmor_file_permission
0.65 ± 18% -0.3 0.32 ± 5% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.66 ± 38% -0.3 0.33 ± 4% perf-profile.self.cycles-pp.__fget_files
0.63 ± 40% -0.3 0.31 ± 16% perf-profile.self.cycles-pp.reweight_entity
0.71 ± 27% -0.3 0.40 ± 8% perf-profile.self.cycles-pp.update_cfs_group
0.55 ± 20% -0.3 0.25 ± 6% perf-profile.self.cycles-pp.dequeue_task_fair
0.53 ± 25% -0.3 0.26 ± 2% perf-profile.self.cycles-pp.set_next_entity
0.44 ± 42% -0.3 0.16 ± 6% perf-profile.self.cycles-pp.__wrgsbase_inactive
0.52 ± 31% -0.3 0.25 ± 6% perf-profile.self.cycles-pp.__might_resched
0.45 ± 32% -0.3 0.20 ± 11% perf-profile.self.cycles-pp.vfs_write
0.48 ± 37% -0.2 0.23 ± 5% perf-profile.self.cycles-pp.prepare_to_wait_event
0.47 ± 29% -0.2 0.23 ± 6% perf-profile.self.cycles-pp.mutex_lock
0.44 ± 35% -0.2 0.21 ± 3% perf-profile.self.cycles-pp.vfs_read
0.42 ± 18% -0.2 0.18 ± 39% perf-profile.self.cycles-pp.__flush_smp_call_function_queue
0.47 ± 30% -0.2 0.23 ± 6% perf-profile.self.cycles-pp.newidle_balance
0.45 ± 29% -0.2 0.22 ± 4% perf-profile.self.cycles-pp.__libc_write
0.51 ± 9% -0.2 0.28 ± 25% perf-profile.self.cycles-pp.update_curr
0.48 ± 23% -0.2 0.25 ± 11% perf-profile.self.cycles-pp.___perf_sw_event
0.42 ± 34% -0.2 0.19 ± 5% perf-profile.self.cycles-pp.select_task_rq_fair
0.43 ± 33% -0.2 0.22 ± 9% perf-profile.self.cycles-pp.__libc_read
0.36 ± 43% -0.2 0.14 ± 6% perf-profile.self.cycles-pp.mutex_unlock
0.34 ± 24% -0.2 0.14 ± 23% perf-profile.self.cycles-pp.__calc_delta
0.28 ± 52% -0.2 0.09 ± 9% perf-profile.self.cycles-pp.__rdgsbase_inactive
0.30 ± 36% -0.2 0.12 ± 6% perf-profile.self.cycles-pp.__might_sleep
0.30 ± 30% -0.2 0.12 ± 7% perf-profile.self.cycles-pp.pick_next_entity
0.34 ± 37% -0.2 0.16 ± 8% perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack
0.28 ± 34% -0.2 0.11 ± 13% perf-profile.self.cycles-pp.flush_smp_call_function_queue
0.30 ± 7% -0.2 0.14 ± 38% perf-profile.self.cycles-pp.try_to_wake_up
0.29 ± 46% -0.2 0.14 ± 4% perf-profile.self.cycles-pp.schedule_idle
0.28 ± 37% -0.2 0.13 ± 4% perf-profile.self.cycles-pp.__cond_resched
0.28 ± 30% -0.1 0.14 ± 8% perf-profile.self.cycles-pp.copy_user_short_string
0.24 ± 32% -0.1 0.10 ± 8% perf-profile.self.cycles-pp.put_prev_task_fair
0.27 ± 7% -0.1 0.13 ± 14% perf-profile.self.cycles-pp.sched_ttwu_pending
0.29 ± 22% -0.1 0.15 ± 10% perf-profile.self.cycles-pp._copy_from_iter
0.30 ± 16% -0.1 0.16 ± 8% perf-profile.self.cycles-pp.do_syscall_64
0.22 ± 20% -0.1 0.08 ± 5% perf-profile.self.cycles-pp.copy_page_from_iter
0.28 ± 29% -0.1 0.14 ± 8% perf-profile.self.cycles-pp.tick_nohz_next_event
0.24 ± 35% -0.1 0.11 ± 8% perf-profile.self.cycles-pp.__hrtimer_next_event_base
0.24 ± 39% -0.1 0.11 ± 8% perf-profile.self.cycles-pp.cpuacct_charge
0.26 ± 30% -0.1 0.14 ± 8% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.27 ± 23% -0.1 0.15 ± 11% perf-profile.self.cycles-pp.update_rq_clock
0.23 ± 17% -0.1 0.11 ± 8% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.40 ± 12% -0.1 0.28 ± 17% perf-profile.self.cycles-pp.tick_nohz_idle_exit
0.21 ± 36% -0.1 0.10 ± 9% perf-profile.self.cycles-pp.fput
0.15 ± 18% -0.1 0.04 ± 44% perf-profile.self.cycles-pp.__get_task_ioprio
0.17 ± 35% -0.1 0.06 ± 11% perf-profile.self.cycles-pp._copy_to_iter
0.20 ± 32% -0.1 0.10 ± 3% perf-profile.self.cycles-pp.syscall_enter_from_user_mode
0.18 ± 7% -0.1 0.07 ± 5% perf-profile.self.cycles-pp.select_task_rq
0.20 ± 15% -0.1 0.09 ± 21% perf-profile.self.cycles-pp.current_time
0.18 ± 37% -0.1 0.08 ± 7% perf-profile.self.cycles-pp.switch_fpu_return
0.19 ± 35% -0.1 0.09 ± 13% perf-profile.self.cycles-pp.autoremove_wake_function
0.16 ± 23% -0.1 0.06 ± 15% perf-profile.self.cycles-pp.ttwu_do_activate
0.18 ± 39% -0.1 0.08 ± 20% perf-profile.self.cycles-pp.rcu_nocb_flush_deferred_wakeup
0.22 ± 24% -0.1 0.12 ± 9% perf-profile.self.cycles-pp.ksys_read
0.16 ± 15% -0.1 0.06 ± 9% perf-profile.self.cycles-pp.update_min_vruntime
0.16 ± 20% -0.1 0.07 ± 23% perf-profile.self.cycles-pp.check_preempt_curr
0.19 ± 29% -0.1 0.09 ± 7% perf-profile.self.cycles-pp.put_prev_entity
0.14 ± 9% -0.1 0.05 ± 46% perf-profile.self.cycles-pp.cpuidle_governor_latency_req
0.18 ± 31% -0.1 0.09 ± 13% perf-profile.self.cycles-pp.ct_kernel_exit_state
0.18 ± 21% -0.1 0.09 ± 7% perf-profile.self.cycles-pp.tick_nohz_tick_stopped
0.18 ± 23% -0.1 0.08 ± 8% perf-profile.self.cycles-pp.anon_pipe_buf_release
0.22 ± 26% -0.1 0.13 ± 12% perf-profile.self.cycles-pp.__pthread_disable_asynccancel
0.12 ± 8% -0.1 0.03 ± 70% perf-profile.self.cycles-pp.put_prev_task_idle
0.18 ± 32% -0.1 0.09 ± 14% perf-profile.self.cycles-pp.rcu_note_context_switch
0.18 ± 9% -0.1 0.09 ± 26% perf-profile.self.cycles-pp.ksys_write
0.19 ± 8% -0.1 0.10 ± 23% perf-profile.self.cycles-pp.get_cpu_device
0.19 ± 32% -0.1 0.10 ± 13% perf-profile.self.cycles-pp.__fget_light
0.17 ± 33% -0.1 0.08 ± 10% perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack
0.15 ± 31% -0.1 0.07 ± 10% perf-profile.self.cycles-pp.select_idle_sibling
0.20 ± 2% -0.1 0.12 ± 27% perf-profile.self.cycles-pp.sched_clock_cpu
0.16 ± 27% -0.1 0.08 ± 15% perf-profile.self.cycles-pp.llist_add_batch
0.17 ± 19% -0.1 0.09 ± 5% perf-profile.self.cycles-pp.tick_nohz_get_sleep_length
0.15 ± 30% -0.1 0.06 ± 21% perf-profile.self.cycles-pp.get_next_timer_interrupt
0.16 ± 37% -0.1 0.08 ± 13% perf-profile.self.cycles-pp.tick_check_broadcast_expired
0.14 ± 11% -0.1 0.06 ± 47% perf-profile.self.cycles-pp.rb_erase
0.17 ± 26% -0.1 0.09 ± 13% perf-profile.self.cycles-pp.nohz_run_idle_balance
0.17 ± 20% -0.1 0.09 ± 13% perf-profile.self.cycles-pp.__update_idle_core
0.16 ± 22% -0.1 0.08 ± 8% perf-profile.self.cycles-pp.testcase
0.17 ± 16% -0.1 0.09 ± 31% perf-profile.self.cycles-pp.nr_iowait_cpu
0.12 ± 31% -0.1 0.04 ± 45% perf-profile.self.cycles-pp.__wake_up_common
0.18 ± 10% -0.1 0.10 ± 36% perf-profile.self.cycles-pp.atime_needs_update
0.14 ± 43% -0.1 0.06 ± 13% perf-profile.self.cycles-pp.file_update_time
0.12 ± 15% -0.1 0.04 ± 70% perf-profile.self.cycles-pp.tick_nohz_idle_retain_tick
0.11 ± 9% -0.1 0.03 ±100% perf-profile.self.cycles-pp.schedule_debug
0.16 ± 16% -0.1 0.08 ± 12% perf-profile.self.cycles-pp.tsc_verify_tsc_adjust
0.16 ± 35% -0.1 0.09 ± 12% perf-profile.self.cycles-pp.cpuidle_not_available
0.14 ± 19% -0.1 0.07 ± 27% perf-profile.self.cycles-pp.ttwu_do_wakeup
0.15 ± 30% -0.1 0.08 ± 7% perf-profile.self.cycles-pp.__pthread_enable_asynccancel
0.10 ± 23% -0.1 0.03 ±100% perf-profile.self.cycles-pp.send_call_function_single_ipi
0.15 ± 8% -0.1 0.08 ± 22% perf-profile.self.cycles-pp.__list_del_entry_valid
0.14 ± 34% -0.1 0.08 ± 16% perf-profile.self.cycles-pp.cpu_startup_entry
0.12 ± 21% -0.1 0.06 ± 19% perf-profile.self.cycles-pp.__list_add_valid
0.12 ± 38% -0.1 0.06 ± 14% perf-profile.self.cycles-pp.__x64_sys_write
0.08 ± 26% -0.1 0.03 ±100% perf-profile.self.cycles-pp.rb_insert_color
0.14 ± 8% -0.1 0.08 ± 8% perf-profile.self.cycles-pp.__might_fault
0.12 ± 28% -0.1 0.06 ± 6% perf-profile.self.cycles-pp.__x64_sys_read
0.10 ± 29% -0.1 0.05 ± 7% perf-profile.self.cycles-pp.ct_kernel_exit
0.10 ± 21% -0.0 0.06 ± 8% perf-profile.self.cycles-pp.touch_atime
0.08 ± 10% -0.0 0.04 ± 71% perf-profile.self.cycles-pp.child
0.00 +0.1 0.07 ± 25% perf-profile.self.cycles-pp._find_first_zero_bit
0.28 ± 23% +0.6 0.85 ± 11% perf-profile.self.cycles-pp.switch_mm_irqs_off
0.65 ± 30% +0.8 1.41 ± 2% perf-profile.self.cycles-pp._raw_spin_lock
0.00 +38.8 38.76 ± 47% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath***************************************************************************************************
lkp-skl-fpga01: 104 threads 2 sockets (Skylake) with 192G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase:
gcc-11/performance/x86_64-rhel-8.3/thread/50%/debian-11.1-x86_64-20220510.cgz/lkp-skl-fpga01/context_switch1/will-it-scale
commit:
99babd04b2 ("selftests/rseq: Implement rseq numa node id field selftest") af7f588d8f ("sched: Introduce per-memory-map concurrency ID")
99babd04b2505471 af7f588d8f7355bc4298dd1962d
---------------- ---------------------------
%stddev %change %stddev
\ | \
8413415 ± 2% -85.0% 1259200 will-it-scale.52.threads
22.36 -83.3% 3.74 will-it-scale.52.threads_idle
161796 ± 2% -85.0% 24214 will-it-scale.per_thread_ops
8413415 ± 2% -85.0% 1259200 will-it-scale.workload
10216 ± 2% -40.9% 6040 ± 2% uptime.idle
2.233e+09 -73.2% 5.989e+08 cpuidle..time
2.46e+09 ± 2% -89.5% 2.577e+08 ± 3% cpuidle..usage
329714 ± 11% +232.0% 1094566 ± 72% numa-numastat.node0.local_node
374570 ± 9% +205.7% 1145060 ± 69% numa-numastat.node0.numa_hit
4491204 ± 2% -53.1% 2106039 ± 36% numa-numastat.node1.local_node
4540680 ± 2% -52.7% 2149877 ± 35% numa-numastat.node1.numa_hit
22.83 -18.4 4.43 ± 2% mpstat.cpu.all.idle%
1.44 +0.2 1.68 mpstat.cpu.all.irq%
0.12 ± 2% -0.0 0.10 mpstat.cpu.all.soft%
59.94 +30.2 90.11 mpstat.cpu.all.sys%
15.67 -12.0 3.68 mpstat.cpu.all.usr%
22.83 -78.1% 5.00 vmstat.cpu.id
60.33 +50.6% 90.83 vmstat.cpu.sy
15.00 -80.0% 3.00 vmstat.cpu.us
14294773 ± 2% -53.0% 6714843 ± 9% vmstat.memory.cache
59.50 -12.3% 52.17 vmstat.procs.r
16450432 ± 2% -85.4% 2405190 vmstat.system.cs
276754 +52.0% 420615 ± 2% vmstat.system.in
1431257 ± 8% +1030.1% 16174000 ± 6% turbostat.C1
0.02 ± 17% +0.3 0.29 ± 7% turbostat.C1%
0.77 ± 6% +31.1% 1.01 ± 4% turbostat.CPU%c1
53.67 ± 2% -14.6% 45.83 ± 2% turbostat.CoreTmp
0.09 ± 4% -78.2% 0.02 turbostat.IPC
84431937 +52.1% 1.284e+08 ± 2% turbostat.IRQ
6.83 ± 3% -6.8 0.00 turbostat.PKG_%
2.458e+09 ± 2% -90.2% 2.41e+08 ± 3% turbostat.POLL
6.16 -5.5 0.69 ± 5% turbostat.POLL%
53.50 ± 2% -13.1% 46.50 ± 3% turbostat.PkgTmp
365.46 -17.4% 301.75 turbostat.PkgWatt
92948 +1761.8% 1730551 ± 4% meminfo.Active
92857 +1763.6% 1730458 ± 4% meminfo.Active(anon)
475097 -26.0% 351530 meminfo.AnonPages
14131358 ± 2% -53.3% 6596409 ± 9% meminfo.Cached
12167680 ± 2% -62.9% 4516022 ± 5% meminfo.Committed_AS
11089942 ± 2% -83.8% 1794273 ± 16% meminfo.Inactive
11089749 ± 2% -83.8% 1794082 ± 16% meminfo.Inactive(anon)
146766 -11.5% 129954 meminfo.KReclaimable
5019103 ± 3% -69.4% 1533664 ± 7% meminfo.Mapped
16059430 ± 2% -48.9% 8206619 ± 7% meminfo.Memused
14911 ± 2% -47.7% 7796 ± 2% meminfo.PageTables
146766 -11.5% 129954 meminfo.SReclaimable
10707913 ± 2% -70.4% 3172409 ± 6% meminfo.Shmem
16146332 ± 2% -25.6% 12012181 ± 4% meminfo.max_used_kB
382.33 ± 13% +30777.3% 118054 ± 98% numa-vmstat.node0.nr_active_anon
44471 ± 32% +315.1% 184580 ± 82% numa-vmstat.node0.nr_inactive_anon
44053 +211.7% 137333 ± 75% numa-vmstat.node0.nr_mapped
1598 ± 2% +15438.7% 248438 ±106% numa-vmstat.node0.nr_shmem
382.33 ± 13% +30777.3% 118054 ± 98% numa-vmstat.node0.nr_zone_active_anon
44471 ± 32% +315.1% 184580 ± 82% numa-vmstat.node0.nr_zone_inactive_anon
374056 ± 9% +206.0% 1144611 ± 69% numa-vmstat.node0.numa_hit
329200 ± 11% +232.4% 1094117 ± 72% numa-vmstat.node0.numa_local
22812 +1275.5% 313777 ± 42% numa-vmstat.node1.nr_active_anon
75507 ± 19% -55.4% 33711 ± 22% numa-vmstat.node1.nr_anon_pages
2702384 ± 2% -78.4% 583457 ± 37% numa-vmstat.node1.nr_file_pages
2727949 ± 2% -90.3% 264439 ± 35% numa-vmstat.node1.nr_inactive_anon
1210795 ± 3% -79.6% 247430 ± 32% numa-vmstat.node1.nr_mapped
3083 ± 3% -61.4% 1188 ± 24% numa-vmstat.node1.nr_page_table_pages
2675300 ± 2% -79.7% 544388 ± 41% numa-vmstat.node1.nr_shmem
15383 ± 3% -31.4% 10547 ± 10% numa-vmstat.node1.nr_slab_reclaimable
22812 +1275.5% 313776 ± 42% numa-vmstat.node1.nr_zone_active_anon
2727947 ± 2% -90.3% 264439 ± 35% numa-vmstat.node1.nr_zone_inactive_anon
4540436 ± 2% -52.7% 2149540 ± 35% numa-vmstat.node1.numa_hit
4490961 ± 3% -53.1% 2105702 ± 36% numa-vmstat.node1.numa_local
1577 ± 15% +29859.5% 472661 ± 98% numa-meminfo.node0.Active
1532 ± 13% +30744.5% 472639 ± 98% numa-meminfo.node0.Active(anon)
177949 ± 32% +315.0% 738476 ± 82% numa-meminfo.node0.Inactive
177854 ± 32% +315.2% 738443 ± 82% numa-meminfo.node0.Inactive(anon)
176243 +211.6% 549130 ± 76% numa-meminfo.node0.Mapped
6399 ± 2% +15435.9% 994269 ±107% numa-meminfo.node0.Shmem
94498 ± 6% +1225.9% 1252939 ± 42% numa-meminfo.node1.Active
94453 ± 6% +1226.4% 1252867 ± 42% numa-meminfo.node1.Active(anon)
155852 ± 34% -68.9% 48404 ± 41% numa-meminfo.node1.AnonHugePages
301933 ± 19% -55.4% 134805 ± 22% numa-meminfo.node1.AnonPages
376400 ± 15% -43.1% 214145 ± 19% numa-meminfo.node1.AnonPages.max
10818908 ± 2% -78.5% 2330479 ± 37% numa-meminfo.node1.FilePages
10917966 ± 2% -90.3% 1056761 ± 35% numa-meminfo.node1.Inactive
10917868 ± 2% -90.3% 1056603 ± 35% numa-meminfo.node1.Inactive(anon)
61560 ± 3% -31.5% 42189 ± 10% numa-meminfo.node1.KReclaimable
4856202 ± 3% -79.7% 987910 ± 32% numa-meminfo.node1.Mapped
11816207 ± 2% -74.2% 3046680 ± 28% numa-meminfo.node1.MemUsed
12364 ± 2% -61.6% 4742 ± 24% numa-meminfo.node1.PageTables
61560 ± 3% -31.5% 42189 ± 10% numa-meminfo.node1.SReclaimable
10710570 ± 2% -79.7% 2174201 ± 41% numa-meminfo.node1.Shmem
147398 -13.8% 127057 ± 6% numa-meminfo.node1.Slab
23418 +1746.6% 432430 ± 4% proc-vmstat.nr_active_anon
118759 -26.0% 87887 proc-vmstat.nr_anon_pages
4488037 +4.4% 4684192 proc-vmstat.nr_dirty_background_threshold
8987049 +4.4% 9379838 proc-vmstat.nr_dirty_threshold
3533819 ± 2% -53.3% 1648861 ± 9% proc-vmstat.nr_file_pages
45164003 +4.3% 47128423 proc-vmstat.nr_free_pages
2773197 ± 2% -83.8% 448466 ± 16% proc-vmstat.nr_inactive_anon
1256312 ± 3% -69.5% 383482 ± 6% proc-vmstat.nr_mapped
3731 ± 2% -48.1% 1936 ± 3% proc-vmstat.nr_page_table_pages
2677956 ± 2% -70.4% 792860 ± 6% proc-vmstat.nr_shmem
36695 -11.5% 32488 proc-vmstat.nr_slab_reclaimable
23418 +1746.6% 432430 ± 4% proc-vmstat.nr_zone_active_anon
2773197 ± 2% -83.8% 448466 ± 16% proc-vmstat.nr_zone_inactive_anon
624422 ± 17% -60.7% 245678 ± 15% proc-vmstat.numa_hint_faults
539128 ± 21% -73.5% 142632 ± 19% proc-vmstat.numa_hint_faults_local
4916730 ± 2% -33.0% 3296362 proc-vmstat.numa_hit
2036 ± 18% -83.4% 337.33 ± 17% proc-vmstat.numa_huge_pte_updates
4822398 ± 2% -33.6% 3202031 proc-vmstat.numa_local
1802325 ± 4% -69.5% 550361 ± 9% proc-vmstat.numa_pte_updates
83499 ±162% +1435.7% 1282307 ± 12% proc-vmstat.pgactivate
5055756 ± 2% -32.8% 3395392 proc-vmstat.pgalloc_normal
1857025 ± 5% -26.5% 1365139 ± 2% proc-vmstat.pgfault
742457 ± 9% +350.4% 3344225 proc-vmstat.pgfree
39235 +8.1% 42423 ± 6% proc-vmstat.pgreuse
308870 ± 19% -99.3% 2179 ±115% sched_debug.cfs_rq:/.MIN_vruntime.avg
4280681 -94.7% 226660 ±115% sched_debug.cfs_rq:/.MIN_vruntime.max
1073798 ± 9% -97.9% 22118 ±115% sched_debug.cfs_rq:/.MIN_vruntime.stddev
0.68 ± 2% +13.8% 0.77 sched_debug.cfs_rq:/.h_nr_running.avg
0.42 ± 2% -21.5% 0.33 ± 3% sched_debug.cfs_rq:/.h_nr_running.stddev
0.67 ± 45% +183.3% 1.89 ± 8% sched_debug.cfs_rq:/.load_avg.min
308870 ± 19% -99.3% 2179 ±115% sched_debug.cfs_rq:/.max_vruntime.avg
4280681 -94.7% 226660 ±115% sched_debug.cfs_rq:/.max_vruntime.max
1073798 ± 9% -97.9% 22118 ±115% sched_debug.cfs_rq:/.max_vruntime.stddev
4201184 -19.5% 3381693 sched_debug.cfs_rq:/.min_vruntime.avg
4494117 -13.4% 3892039 ± 2% sched_debug.cfs_rq:/.min_vruntime.max
4124946 -19.2% 3331483 sched_debug.cfs_rq:/.min_vruntime.min
0.68 +12.2% 0.76 sched_debug.cfs_rq:/.nr_running.avg
0.39 ± 3% -20.5% 0.31 ± 4% sched_debug.cfs_rq:/.nr_running.stddev
451.34 -10.3% 404.65 sched_debug.cfs_rq:/.runnable_avg.avg
1319 -11.5% 1167 ± 3% sched_debug.cfs_rq:/.runnable_avg.max
154.04 -13.1% 133.83 ± 5% sched_debug.cfs_rq:/.runnable_avg.stddev
447.52 -10.2% 401.95 sched_debug.cfs_rq:/.util_avg.avg
135.46 -11.1% 120.41 ± 5% sched_debug.cfs_rq:/.util_avg.stddev
230.77 ± 3% -23.8% 175.94 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.avg
940.25 ± 3% -12.9% 818.86 ± 8% sched_debug.cfs_rq:/.util_est_enqueued.max
225.80 -10.3% 202.54 sched_debug.cfs_rq:/.util_est_enqueued.stddev
148237 ± 2% +20.6% 178708 sched_debug.cpu.avg_idle.avg
4546 ± 13% +579.3% 30886 ± 2% sched_debug.cpu.avg_idle.min
5.75 +33.2% 7.66 sched_debug.cpu.clock.stddev
2742 ± 3% -10.0% 2468 ± 3% sched_debug.cpu.curr->pid.avg
6610 ± 23% -31.1% 4557 ± 23% sched_debug.cpu.max_idle_balance_cost.stddev
0.00 ± 5% +42.1% 0.00 ± 29% sched_debug.cpu.next_balance.stddev
0.52 ± 4% -11.0% 0.47 ± 3% sched_debug.cpu.nr_running.avg
23781980 ± 2% -85.3% 3489556 sched_debug.cpu.nr_switches.avg
26139865 ± 3% -70.4% 7732937 ± 11% sched_debug.cpu.nr_switches.max
20830845 ± 5% -85.8% 2960756 ± 6% sched_debug.cpu.nr_switches.min
1902082 ± 27% -74.7% 480695 ± 16% sched_debug.cpu.nr_switches.stddev
1.76 ± 17% +206.4% 5.38 ± 3% perf-stat.i.MPKI
1.904e+10 -73.1% 5.12e+09 perf-stat.i.branch-instructions
1.64 -0.5 1.15 perf-stat.i.branch-miss-rate%
3.056e+08 ± 2% -80.9% 58311136 ± 2% perf-stat.i.branch-misses
20.15 ± 13% +6.3 26.45 ± 2% perf-stat.i.cache-miss-rate%
16636101 ± 2% -85.4% 2432393 perf-stat.i.context-switches
3.12 ± 2% +296.6% 12.37 perf-stat.i.cpi
1861 -33.6% 1236 perf-stat.i.cpu-migrations
0.08 ± 5% -0.0 0.04 ± 3% perf-stat.i.dTLB-load-miss-rate%
20981469 ± 5% -85.8% 2981601 ± 5% perf-stat.i.dTLB-load-misses
2.678e+10 -74.9% 6.71e+09 perf-stat.i.dTLB-loads
0.00 ± 18% +0.0 0.00 ± 23% perf-stat.i.dTLB-store-miss-rate%
233103 ± 18% -67.8% 74993 ± 22% perf-stat.i.dTLB-store-misses
1.604e+10 -82.2% 2.863e+09 ± 2% perf-stat.i.dTLB-stores
11175689 ± 3% -82.1% 2004383 ± 24% perf-stat.i.iTLB-load-misses
23053320 ± 6% -81.3% 4309745 ± 13% perf-stat.i.iTLB-loads
9.415e+10 -74.4% 2.406e+10 perf-stat.i.instructions
8923 ± 2% +47.9% 13200 ± 22% perf-stat.i.instructions-per-iTLB-miss
0.33 -74.4% 0.08 perf-stat.i.ipc
596.29 -76.2% 142.20 perf-stat.i.metric.M/sec
5512 ± 6% -29.0% 3912 ± 3% perf-stat.i.minor-faults
56.02 +41.6 97.64 perf-stat.i.node-load-miss-rate%
1970708 ± 4% +314.9% 8176399 ± 4% perf-stat.i.node-load-misses
2639467 ± 5% -89.9% 266188 ± 29% perf-stat.i.node-loads
85.83 +13.0 98.81 perf-stat.i.node-store-miss-rate%
1884736 ± 3% +152.2% 4752825 ± 2% perf-stat.i.node-store-misses
357304 ± 3% -74.0% 93012 ± 6% perf-stat.i.node-stores
5512 ± 6% -29.0% 3913 ± 3% perf-stat.i.page-faults
1.74 ± 16% +210.5% 5.41 ± 3% perf-stat.overall.MPKI
1.60 -0.5 1.14 perf-stat.overall.branch-miss-rate%
19.62 ± 13% +6.0 25.62 ± 2% perf-stat.overall.cache-miss-rate%
3.06 +290.5% 11.94 perf-stat.overall.cpi
0.08 ± 5% -0.0 0.04 ± 4% perf-stat.overall.dTLB-load-miss-rate%
0.00 ± 18% +0.0 0.00 ± 22% perf-stat.overall.dTLB-store-miss-rate%
8429 ± 2% +50.4% 12675 ± 22% perf-stat.overall.instructions-per-iTLB-miss
0.33 -74.4% 0.08 perf-stat.overall.ipc
42.42 ± 2% +54.4 96.80 perf-stat.overall.node-load-miss-rate%
84.10 +14.0 98.08 perf-stat.overall.node-store-miss-rate%
3374614 +71.0% 5770768 perf-stat.overall.path-length
1.898e+10 -73.1% 5.105e+09 perf-stat.ps.branch-instructions
3.046e+08 ± 2% -80.9% 58149017 ± 2% perf-stat.ps.branch-misses
16582299 ± 2% -85.4% 2424954 perf-stat.ps.context-switches
1861 -33.8% 1231 perf-stat.ps.cpu-migrations
20900401 ± 5% -85.7% 2979011 ± 5% perf-stat.ps.dTLB-load-misses
2.669e+10 -74.9% 6.691e+09 perf-stat.ps.dTLB-loads
231656 ± 18% -67.6% 74972 ± 22% perf-stat.ps.dTLB-store-misses
1.599e+10 -82.1% 2.855e+09 ± 2% perf-stat.ps.dTLB-stores
11141515 ± 3% -82.0% 2001300 ± 24% perf-stat.ps.iTLB-load-misses
22975410 ± 6% -81.3% 4295158 ± 13% perf-stat.ps.iTLB-loads
9.385e+10 -74.4% 2.4e+10 perf-stat.ps.instructions
5511 ± 6% -29.2% 3901 ± 3% perf-stat.ps.minor-faults
1964291 ± 4% +315.4% 8159316 ± 4% perf-stat.ps.node-load-misses
2667620 ± 5% -90.0% 268054 ± 29% perf-stat.ps.node-loads
1878586 ± 3% +152.2% 4737805 ± 2% perf-stat.ps.node-store-misses
354848 ± 3% -73.9% 92786 ± 6% perf-stat.ps.node-stores
5512 ± 6% -29.2% 3902 ± 3% perf-stat.ps.page-faults
2.839e+13 -74.4% 7.266e+12 perf-stat.total.instructions
26.70 -21.8 4.92 ± 2% perf-profile.calltrace.cycles-pp.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event
26.52 -21.6 4.88 ± 2% perf-profile.calltrace.cycles-pp.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow
24.81 -20.3 4.54 ± 2% perf-profile.calltrace.cycles-pp.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample.perf_event_output_forward
19.22 -15.2 4.02 ± 3% perf-profile.calltrace.cycles-pp.__libc_write
16.54 -13.1 3.48 ± 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_write
15.23 -12.4 2.79 perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule
15.46 -12.2 3.28 ± 3% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
13.72 -11.5 2.27 ± 2% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch
13.79 -10.7 3.09 ± 7% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
13.58 -10.7 2.90 ± 3% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write
11.44 ± 2% -10.3 1.17 perf-profile.calltrace.cycles-pp.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
12.38 -9.7 2.69 ± 2% perf-profile.calltrace.cycles-pp.pipe_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
12.12 -9.5 2.58 ± 2% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_read.vfs_read
9.83 ± 2% -9.2 0.60 ± 76% perf-profile.calltrace.cycles-pp.unwind_next_frame.__unwind_start.perf_callchain_kernel.get_perf_callchain.perf_callchain
11.66 -9.2 2.44 ± 2% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_read
9.47 ± 3% -9.0 0.44 ± 44% perf-profile.calltrace.cycles-pp.finish_task_switch.__schedule.schedule.pipe_read.vfs_read
10.60 -8.2 2.37 ± 3% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.vfs_write.ksys_write.do_syscall_64
10.31 -8.1 2.18 ± 2% perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule
10.31 -8.0 2.32 ± 3% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.vfs_write.ksys_write
10.23 -7.9 2.30 ± 3% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.vfs_write
10.04 -7.8 2.26 ± 3% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
9.04 -7.6 1.42 ± 2% perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule.pipe_read.vfs_read
9.54 -7.5 2.04 ± 2% perf-profile.calltrace.cycles-pp.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair.__schedule
8.81 -7.4 1.36 ± 2% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule.pipe_read
9.34 -7.4 1.98 ± 2% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity.dequeue_task_fair
7.56 -7.2 0.34 ± 70% perf-profile.calltrace.cycles-pp.unwind_next_frame.perf_callchain_kernel.get_perf_callchain.perf_callchain.perf_prepare_sample
8.54 -7.2 1.32 ± 2% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule
9.07 -7.1 1.93 ± 2% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr.dequeue_entity
9.00 -7.1 1.92 ± 2% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime.update_curr
8.84 -7.0 1.80 ± 5% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
8.29 -6.6 1.66 ± 2% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_stat_runtime
8.19 -6.5 1.72 ± 4% perf-profile.calltrace.cycles-pp.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
7.98 -6.3 1.69 ± 3% perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function.__wake_up_common
7.59 -6.0 1.62 ± 3% perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up.autoremove_wake_function
7.54 -5.9 1.60 ± 3% perf-profile.calltrace.cycles-pp.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_wakeup_template.try_to_wake_up
7.28 -5.7 1.60 perf-profile.calltrace.cycles-pp.perf_trace_sched_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry
7.08 -5.5 1.54 perf-profile.calltrace.cycles-pp.perf_tp_event.perf_trace_sched_switch.__schedule.schedule_idle.do_idle
6.89 -5.5 1.34 ± 4% perf-profile.calltrace.cycles-pp.perf_prepare_sample.perf_event_output_forward.__perf_event_overflow.perf_tp_event.perf_trace_sched_wakeup_template
6.85 -5.4 1.49 perf-profile.calltrace.cycles-pp.__perf_event_overflow.perf_tp_event.perf_trace_sched_switch.__schedule.schedule_idle
5.32 -4.0 1.27 ± 6% perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
5.26 -4.0 1.26 ± 6% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
4.42 -3.7 0.74 ± 5% perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
52.88 -3.6 49.32 perf-profile.calltrace.cycles-pp.__libc_read
2.85 ± 2% -2.3 0.57 ± 2% perf-profile.calltrace.cycles-pp.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
2.38 -2.1 0.26 ±100% perf-profile.calltrace.cycles-pp.sched_ttwu_pending.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary
0.75 ± 14% -0.5 0.26 ±100% perf-profile.calltrace.cycles-pp.generic_perform_write.__generic_file_write_iter.generic_file_write_iter.vfs_write.ksys_write
0.75 ± 14% -0.5 0.27 ±100% perf-profile.calltrace.cycles-pp.__generic_file_write_iter.generic_file_write_iter.vfs_write.ksys_write.do_syscall_64
0.75 ± 14% -0.5 0.27 ±100% perf-profile.calltrace.cycles-pp.generic_file_write_iter.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.75 ± 14% -0.4 0.37 ± 71% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write.writen
0.76 ± 14% -0.4 0.39 ± 71% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_write.writen.record__pushfn
0.76 ± 14% -0.4 0.40 ± 71% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_write.writen.record__pushfn.perf_mmap__push
0.73 ± 6% +0.7 1.44 ± 8% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.pipe_read.vfs_read
0.00 +0.8 0.76 ± 10% perf-profile.calltrace.cycles-pp.update_sg_lb_stats.update_sd_lb_stats.find_busiest_group.load_balance.newidle_balance
0.60 ± 6% +0.8 1.41 ± 8% perf-profile.calltrace.cycles-pp.newidle_balance.pick_next_task_fair.__schedule.schedule.pipe_read
0.00 +0.9 0.89 ± 9% perf-profile.calltrace.cycles-pp.update_sd_lb_stats.find_busiest_group.load_balance.newidle_balance.pick_next_task_fair
0.00 +0.9 0.93 ± 9% perf-profile.calltrace.cycles-pp.find_busiest_group.load_balance.newidle_balance.pick_next_task_fair.__schedule
0.00 +1.1 1.09 ± 10% perf-profile.calltrace.cycles-pp.load_balance.newidle_balance.pick_next_task_fair.__schedule.schedule
45.74 +2.2 47.91 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
45.24 +2.6 47.83 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_read
44.49 +3.2 47.69 perf-profile.calltrace.cycles-pp.pipe_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
41.00 +6.1 47.15 perf-profile.calltrace.cycles-pp.schedule.pipe_read.vfs_read.ksys_read.do_syscall_64
40.89 +6.2 47.13 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_read.vfs_read.ksys_read
25.48 +19.7 45.16 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
25.46 +19.7 45.16 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
25.73 +19.8 45.52 perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
25.30 +19.8 45.14 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
11.35 +31.1 42.42 perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
11.18 +31.2 42.39 perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
0.00 +39.4 39.41 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prepare_task_switch.__schedule.schedule_idle
0.00 +39.6 39.63 perf-profile.calltrace.cycles-pp._raw_spin_lock.prepare_task_switch.__schedule.schedule_idle.do_idle
0.00 +39.7 39.73 perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry
0.00 +40.2 40.19 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prepare_task_switch.__schedule.schedule
0.00 +40.4 40.41 perf-profile.calltrace.cycles-pp._raw_spin_lock.prepare_task_switch.__schedule.schedule.pipe_read
0.00 +40.5 40.50 perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule.pipe_read.vfs_read
33.56 -26.7 6.85 ± 2% perf-profile.children.cycles-pp.perf_tp_event
32.38 -25.7 6.64 ± 2% perf-profile.children.cycles-pp.__perf_event_overflow
32.13 -25.5 6.59 ± 2% perf-profile.children.cycles-pp.perf_event_output_forward
29.34 -23.8 5.53 ± 2% perf-profile.children.cycles-pp.perf_prepare_sample
27.00 -21.9 5.14 ± 2% perf-profile.children.cycles-pp.perf_callchain
26.82 -21.7 5.10 ± 2% perf-profile.children.cycles-pp.get_perf_callchain
25.26 -20.5 4.79 ± 2% perf-profile.children.cycles-pp.perf_callchain_kernel
20.14 -15.5 4.66 perf-profile.children.cycles-pp.__libc_write
18.18 -14.7 3.44 perf-profile.children.cycles-pp.unwind_next_frame
66.41 -13.7 52.69 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
16.42 -13.4 3.05 perf-profile.children.cycles-pp.perf_trace_sched_switch
64.30 -12.1 52.24 perf-profile.children.cycles-pp.do_syscall_64
14.42 -11.0 3.45 perf-profile.children.cycles-pp.ksys_write
13.86 -10.5 3.34 perf-profile.children.cycles-pp.vfs_write
12.41 -9.7 2.70 ± 3% perf-profile.children.cycles-pp.pipe_write
10.68 ± 2% -9.7 1.02 ± 4% perf-profile.children.cycles-pp.finish_task_switch
11.71 ± 2% -9.6 2.14 perf-profile.children.cycles-pp.__unwind_start
12.13 -9.6 2.58 ± 2% perf-profile.children.cycles-pp.dequeue_task_fair
11.72 -9.3 2.45 ± 2% perf-profile.children.cycles-pp.dequeue_entity
10.62 -8.2 2.38 ± 3% perf-profile.children.cycles-pp.__wake_up_common_lock
10.68 -8.2 2.48 ± 2% perf-profile.children.cycles-pp.update_curr
10.32 -8.0 2.32 ± 3% perf-profile.children.cycles-pp.__wake_up_common
10.25 -7.9 2.31 ± 3% perf-profile.children.cycles-pp.autoremove_wake_function
10.06 -7.8 2.27 ± 3% perf-profile.children.cycles-pp.try_to_wake_up
9.79 -7.5 2.29 ± 2% perf-profile.children.cycles-pp.perf_trace_sched_stat_runtime
8.99 -7.2 1.82 ± 6% perf-profile.children.cycles-pp.cpuidle_idle_call
8.19 -6.5 1.73 ± 4% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
5.37 -4.1 1.28 ± 6% perf-profile.children.cycles-pp.cpuidle_enter
5.33 -4.1 1.27 ± 6% perf-profile.children.cycles-pp.cpuidle_enter_state
4.52 -3.8 0.76 ± 4% perf-profile.children.cycles-pp.poll_idle
52.99 -3.6 49.35 perf-profile.children.cycles-pp.__libc_read
3.85 -3.1 0.75 ± 3% perf-profile.children.cycles-pp.unwind_get_return_address
3.84 -3.1 0.75 ± 3% perf-profile.children.cycles-pp.__orc_find
3.53 -2.8 0.71 ± 5% perf-profile.children.cycles-pp.syscall_exit_to_user_mode
3.18 -2.5 0.68 ± 4% perf-profile.children.cycles-pp.syscall_return_via_sysret
3.05 -2.4 0.69 ± 4% perf-profile.children.cycles-pp.sched_ttwu_pending
2.92 -2.4 0.56 ± 3% perf-profile.children.cycles-pp.__kernel_text_address
2.89 ± 3% -2.3 0.58 perf-profile.children.cycles-pp.flush_smp_call_function_queue
2.66 ± 2% -2.2 0.46 perf-profile.children.cycles-pp.stack_access_ok
2.39 -1.9 0.48 ± 2% perf-profile.children.cycles-pp.orc_find
2.20 ± 5% -1.9 0.34 ± 7% perf-profile.children.cycles-pp.menu_select
2.62 -1.8 0.78 ± 3% perf-profile.children.cycles-pp.ttwu_do_activate
2.19 -1.8 0.44 ± 4% perf-profile.children.cycles-pp.kernel_text_address
2.50 ± 2% -1.7 0.76 ± 3% perf-profile.children.cycles-pp.enqueue_task_fair
2.04 ± 2% -1.4 0.61 ± 2% perf-profile.children.cycles-pp.enqueue_entity
1.58 ± 3% -1.1 0.44 ± 3% perf-profile.children.cycles-pp.update_load_avg
1.42 -1.1 0.29 ± 4% perf-profile.children.cycles-pp.__entry_text_start
1.48 ± 2% -1.1 0.35 ± 7% perf-profile.children.cycles-pp.asm_sysvec_call_function_single
1.30 ± 3% -1.1 0.21 ± 4% perf-profile.children.cycles-pp.__perf_event_header__init_id
1.10 ± 3% -0.9 0.22 ± 2% perf-profile.children.cycles-pp.sched_clock_cpu
0.99 -0.9 0.12 ± 4% perf-profile.children.cycles-pp.perf_output_begin_forward
1.02 ± 4% -0.9 0.15 ± 9% perf-profile.children.cycles-pp.tick_nohz_idle_exit
0.93 ± 2% -0.8 0.12 ± 6% perf-profile.children.cycles-pp.ktime_get
0.93 ± 3% -0.8 0.17 ± 5% perf-profile.children.cycles-pp.set_next_entity
1.02 ± 2% -0.8 0.26 ± 8% perf-profile.children.cycles-pp.sysvec_call_function_single
0.91 ± 2% -0.7 0.18 ± 4% perf-profile.children.cycles-pp.native_sched_clock
0.82 ± 6% -0.7 0.12 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.86 -0.7 0.18 ± 3% perf-profile.children.cycles-pp.perf_callchain_user
0.80 ± 4% -0.7 0.13 ± 7% perf-profile.children.cycles-pp.mutex_lock
0.78 ± 2% -0.7 0.13 ± 10% perf-profile.children.cycles-pp.__switch_to_asm
0.79 ± 4% -0.6 0.15 ± 7% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.74 ± 3% -0.6 0.10 ± 11% perf-profile.children.cycles-pp.ttwu_queue_wakelist
0.71 ± 6% -0.6 0.11 ± 5% perf-profile.children.cycles-pp.prepare_to_wait_event
0.70 ± 3% -0.6 0.11 ± 6% perf-profile.children.cycles-pp.__switch_to
1.44 ± 4% -0.6 0.86 perf-profile.children.cycles-pp.perf_output_sample
2.24 ± 4% -0.5 1.70 ± 7% perf-profile.children.cycles-pp.pick_next_task_fair
0.72 ± 2% -0.5 0.20 ± 9% perf-profile.children.cycles-pp.__sysvec_call_function_single
0.58 ± 4% -0.5 0.08 ± 10% perf-profile.children.cycles-pp.read_tsc
0.62 ± 2% -0.5 0.14 ± 5% perf-profile.children.cycles-pp.core_kernel_text
0.59 ± 11% -0.5 0.11 ± 8% perf-profile.children.cycles-pp.copy_page_from_iter
1.17 ± 3% -0.5 0.69 perf-profile.children.cycles-pp.__output_copy
0.57 ± 4% -0.4 0.12 ± 4% perf-profile.children.cycles-pp.__fdget_pos
0.54 ± 4% -0.4 0.11 ± 4% perf-profile.children.cycles-pp.__fget_light
0.47 ± 8% -0.4 0.05 ± 8% perf-profile.children.cycles-pp.tick_nohz_idle_enter
0.94 ± 4% -0.4 0.53 perf-profile.children.cycles-pp.perf_output_copy
0.52 ± 6% -0.4 0.11 ± 10% perf-profile.children.cycles-pp.security_file_permission
0.47 ± 6% -0.4 0.08 ± 4% perf-profile.children.cycles-pp.update_cfs_group
0.47 ± 14% -0.4 0.08 ± 20% perf-profile.children.cycles-pp.copy_page_to_iter
0.45 ± 4% -0.4 0.06 ± 7% perf-profile.children.cycles-pp.pick_next_task_idle
0.52 ± 3% -0.4 0.15 ± 2% perf-profile.children.cycles-pp.select_task_rq
0.47 -0.4 0.10 ± 3% perf-profile.children.cycles-pp.__get_user_nocheck_8
0.46 ± 10% -0.4 0.09 ± 9% perf-profile.children.cycles-pp._copy_from_iter
0.44 -0.4 0.08 ± 8% perf-profile.children.cycles-pp.ftrace_graph_ret_addr
0.45 ± 4% -0.4 0.10 ± 9% perf-profile.children.cycles-pp.apparmor_file_permission
0.45 ± 3% -0.3 0.11 ± 4% perf-profile.children.cycles-pp.__update_load_avg_se
0.41 ± 12% -0.3 0.07 ± 19% perf-profile.children.cycles-pp._copy_to_iter
0.41 ± 5% -0.3 0.08 ± 11% perf-profile.children.cycles-pp.tick_nohz_next_event
0.81 ± 3% -0.3 0.48 perf-profile.children.cycles-pp.memcpy_erms
0.41 ± 2% -0.3 0.08 ± 5% perf-profile.children.cycles-pp.__fget_files
0.38 ± 3% -0.3 0.06 ± 9% perf-profile.children.cycles-pp.__update_idle_core
0.38 ± 18% -0.3 0.07 ± 18% perf-profile.children.cycles-pp.touch_atime
0.80 ± 17% -0.3 0.50 ± 16% perf-profile.children.cycles-pp.generic_perform_write
0.38 ± 2% -0.3 0.08 ± 6% perf-profile.children.cycles-pp.__task_pid_nr_ns
0.80 ± 17% -0.3 0.51 ± 16% perf-profile.children.cycles-pp.__generic_file_write_iter
0.42 -0.3 0.12 ± 3% perf-profile.children.cycles-pp.update_rq_clock
0.81 ± 17% -0.3 0.52 ± 16% perf-profile.children.cycles-pp.generic_file_write_iter
0.44 ± 9% -0.3 0.16 ± 10% perf-profile.children.cycles-pp.copyin
0.41 ± 3% -0.3 0.13 ± 5% perf-profile.children.cycles-pp.select_task_rq_fair
0.35 ± 6% -0.3 0.07 ± 6% perf-profile.children.cycles-pp.reweight_entity
0.30 ± 3% -0.3 0.02 ± 99% perf-profile.children.cycles-pp.put_prev_task_fair
0.30 ± 22% -0.3 0.03 ±100% perf-profile.children.cycles-pp.atime_needs_update
0.35 ± 7% -0.3 0.08 ± 24% perf-profile.children.cycles-pp.exit_to_user_mode_prepare
0.32 ± 5% -0.3 0.06 ± 11% perf-profile.children.cycles-pp.os_xsave
0.32 ± 24% -0.3 0.05 ± 47% perf-profile.children.cycles-pp.__flush_smp_call_function_queue
0.33 ± 3% -0.3 0.07 ± 7% perf-profile.children.cycles-pp.___perf_sw_event
0.29 ± 8% -0.3 0.02 ± 99% perf-profile.children.cycles-pp.perf_trace_buf_update
0.32 ± 7% -0.3 0.06 ± 8% perf-profile.children.cycles-pp.get_stack_info
0.40 ± 11% -0.3 0.15 ± 10% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
0.28 ± 9% -0.3 0.03 ±100% perf-profile.children.cycles-pp.hrtimer_next_event_without
0.32 ± 3% -0.2 0.06 ± 7% perf-profile.children.cycles-pp.__might_resched
0.26 ± 2% -0.2 0.02 ± 99% perf-profile.children.cycles-pp.perf_instruction_pointer
0.42 ± 3% -0.2 0.19 ± 3% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.27 ± 2% -0.2 0.05 ± 7% perf-profile.children.cycles-pp.__might_fault
0.27 ± 4% -0.2 0.06 ± 9% perf-profile.children.cycles-pp.kvm_guest_state
0.26 ± 3% -0.2 0.06 ± 8% perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack
0.25 ± 5% -0.2 0.05 ± 7% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.22 ± 7% -0.2 0.02 ± 99% perf-profile.children.cycles-pp.get_callchain_entry
0.22 ± 3% -0.2 0.02 ± 99% perf-profile.children.cycles-pp.__cond_resched
0.68 ± 3% -0.2 0.49 ± 4% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.81 ± 16% -0.2 0.63 ± 17% perf-profile.children.cycles-pp.writen
0.81 ± 16% -0.2 0.63 ± 17% perf-profile.children.cycles-pp.record__pushfn
0.60 ± 4% -0.2 0.42 ± 4% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
0.24 ± 5% -0.2 0.07 ± 7% perf-profile.children.cycles-pp.perf_trace_buf_alloc
0.30 ± 17% -0.2 0.14 ± 13% perf-profile.children.cycles-pp.copy_page_from_iter_atomic
0.19 ± 7% -0.2 0.02 ± 99% perf-profile.children.cycles-pp.file_update_time
0.34 ± 18% -0.1 0.20 ± 17% perf-profile.children.cycles-pp.shmem_write_begin
0.33 ± 18% -0.1 0.19 ± 18% perf-profile.children.cycles-pp.shmem_get_folio_gfp
0.47 ± 4% -0.1 0.33 ± 5% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
0.46 ± 4% -0.1 0.33 ± 4% perf-profile.children.cycles-pp.hrtimer_interrupt
0.14 ± 6% -0.1 0.05 perf-profile.children.cycles-pp.native_irq_return_iret
0.15 ± 5% -0.1 0.06 ± 7% perf-profile.children.cycles-pp.select_idle_sibling
0.17 ± 4% -0.1 0.08 ± 5% perf-profile.children.cycles-pp.__irq_exit_rcu
0.12 ± 12% -0.1 0.04 ± 44% perf-profile.children.cycles-pp.place_entity
0.19 ± 12% -0.1 0.12 ± 29% perf-profile.children.cycles-pp.process_simple
0.18 ± 11% -0.1 0.11 ± 27% perf-profile.children.cycles-pp.ordered_events__queue
0.12 ± 16% -0.1 0.06 ± 45% perf-profile.children.cycles-pp.shmem_add_to_page_cache
0.13 ± 20% -0.1 0.07 ± 15% perf-profile.children.cycles-pp.shmem_alloc_and_acct_folio
0.31 ± 3% -0.1 0.26 ± 5% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.09 ± 21% -0.1 0.04 ± 71% perf-profile.children.cycles-pp.vma_alloc_folio
0.10 ± 19% -0.1 0.05 ± 47% perf-profile.children.cycles-pp.shmem_alloc_folio
0.26 ± 4% -0.0 0.21 ± 5% perf-profile.children.cycles-pp.tick_sched_timer
0.10 ± 5% -0.0 0.05 ± 7% perf-profile.children.cycles-pp.__do_softirq
0.23 ± 5% -0.0 0.20 ± 4% perf-profile.children.cycles-pp.update_process_times
0.23 ± 5% -0.0 0.20 ± 5% perf-profile.children.cycles-pp.tick_sched_handle
0.00 +0.1 0.06 ± 9% perf-profile.children.cycles-pp._find_next_and_bit
0.00 +0.1 0.08 ± 25% perf-profile.children.cycles-pp.perf_session__deliver_event
0.00 +0.1 0.08 ± 5% perf-profile.children.cycles-pp.__update_blocked_fair
0.00 +0.1 0.10 ± 8% perf-profile.children.cycles-pp.idle_cpu
0.00 +0.1 0.11 ± 22% perf-profile.children.cycles-pp.raw_spin_rq_lock_nested
0.00 +0.2 0.18 ± 2% perf-profile.children.cycles-pp.update_blocked_averages
0.09 ± 14% +0.3 0.40 ± 13% perf-profile.children.cycles-pp.intel_idle_irq
0.09 ± 11% +0.3 0.40 ± 13% perf-profile.children.cycles-pp.mwait_idle_with_hints
0.00 +0.8 0.78 ± 10% perf-profile.children.cycles-pp.update_sg_lb_stats
0.61 ± 5% +0.8 1.41 ± 8% perf-profile.children.cycles-pp.newidle_balance
0.00 +0.9 0.90 ± 9% perf-profile.children.cycles-pp.update_sd_lb_stats
0.00 +0.9 0.94 ± 9% perf-profile.children.cycles-pp.find_busiest_group
0.00 +1.1 1.11 ± 10% perf-profile.children.cycles-pp.load_balance
45.78 +2.2 47.94 perf-profile.children.cycles-pp.ksys_read
45.26 +2.6 47.85 perf-profile.children.cycles-pp.vfs_read
44.56 +3.1 47.71 perf-profile.children.cycles-pp.pipe_read
41.02 +6.2 47.19 perf-profile.children.cycles-pp.schedule
25.48 +19.7 45.16 perf-profile.children.cycles-pp.start_secondary
25.73 +19.8 45.52 perf-profile.children.cycles-pp.secondary_startup_64_no_verify
25.73 +19.8 45.52 perf-profile.children.cycles-pp.cpu_startup_entry
25.66 +19.9 45.51 perf-profile.children.cycles-pp.do_idle
11.49 +31.3 42.77 perf-profile.children.cycles-pp.schedule_idle
52.29 +37.6 89.92 perf-profile.children.cycles-pp.__schedule
0.63 ± 2% +79.9 80.56 perf-profile.children.cycles-pp.prepare_task_switch
0.00 +80.0 80.01 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.30 +80.3 80.56 perf-profile.children.cycles-pp._raw_spin_lock
9.60 ± 3% -9.2 0.40 ± 3% perf-profile.self.cycles-pp.finish_task_switch
8.30 ± 2% -7.8 0.46 ± 3% perf-profile.self.cycles-pp.__schedule
9.18 ± 2% -7.5 1.72 perf-profile.self.cycles-pp.unwind_next_frame
4.01 -3.3 0.68 ± 4% perf-profile.self.cycles-pp.poll_idle
3.81 -3.1 0.75 ± 3% perf-profile.self.cycles-pp.__orc_find
3.14 ± 2% -2.5 0.62 ± 3% perf-profile.self.cycles-pp.syscall_exit_to_user_mode
3.18 -2.5 0.68 ± 4% perf-profile.self.cycles-pp.syscall_return_via_sysret
2.39 -2.0 0.41 ± 2% perf-profile.self.cycles-pp.stack_access_ok
2.24 -1.8 0.45 ± 2% perf-profile.self.cycles-pp.orc_find
2.16 -1.7 0.45 ± 3% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
1.91 -1.5 0.37 ± 3% perf-profile.self.cycles-pp.perf_callchain_kernel
1.30 -1.1 0.23 perf-profile.self.cycles-pp.__unwind_start
1.23 -1.0 0.26 ± 3% perf-profile.self.cycles-pp.__entry_text_start
1.14 -0.9 0.22 ± 3% perf-profile.self.cycles-pp.kernel_text_address
0.99 ± 5% -0.8 0.14 ± 9% perf-profile.self.cycles-pp.menu_select
0.96 ± 2% -0.8 0.11 ± 4% perf-profile.self.cycles-pp.perf_output_begin_forward
0.89 -0.7 0.18 ± 5% perf-profile.self.cycles-pp.unwind_get_return_address
0.89 ± 2% -0.7 0.18 ± 2% perf-profile.self.cycles-pp.native_sched_clock
0.81 ± 6% -0.7 0.12 ± 6% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.85 ± 5% -0.7 0.16 ± 4% perf-profile.self.cycles-pp.perf_tp_event
0.78 ± 2% -0.6 0.13 ± 10% perf-profile.self.cycles-pp.__switch_to_asm
0.73 ± 6% -0.6 0.10 ± 14% perf-profile.self.cycles-pp.cpuidle_idle_call
0.73 ± 2% -0.6 0.12 ± 4% perf-profile.self.cycles-pp.__kernel_text_address
0.72 ± 4% -0.6 0.16 ± 6% perf-profile.self.cycles-pp.enqueue_entity
0.67 ± 7% -0.6 0.11 ± 9% perf-profile.self.cycles-pp.perf_prepare_sample
0.66 ± 4% -0.6 0.10 ± 7% perf-profile.self.cycles-pp.__switch_to
0.70 ± 4% -0.5 0.18 ± 4% perf-profile.self.cycles-pp.update_load_avg
0.55 ± 3% -0.5 0.08 ± 8% perf-profile.self.cycles-pp.read_tsc
0.58 ± 6% -0.5 0.11 ± 6% perf-profile.self.cycles-pp.newidle_balance
0.54 ± 4% -0.4 0.11 ± 4% perf-profile.self.cycles-pp.pipe_write
0.52 ± 7% -0.4 0.10 ± 5% perf-profile.self.cycles-pp.pipe_read
0.45 ± 11% -0.4 0.05 ± 8% perf-profile.self.cycles-pp.__perf_event_header__init_id
0.46 -0.4 0.09 ± 5% perf-profile.self.cycles-pp.get_perf_callchain
0.45 ± 5% -0.4 0.08 perf-profile.self.cycles-pp.update_cfs_group
0.46 ± 2% -0.4 0.10 ± 3% perf-profile.self.cycles-pp.__get_user_nocheck_8
0.42 ± 7% -0.4 0.07 ± 7% perf-profile.self.cycles-pp.dequeue_entity
0.40 ± 7% -0.3 0.06 ± 8% perf-profile.self.cycles-pp.mutex_lock
0.37 ± 4% -0.3 0.02 ± 99% perf-profile.self.cycles-pp.ktime_get
0.44 ± 5% -0.3 0.09 ± 5% perf-profile.self.cycles-pp.update_curr
0.40 ± 11% -0.3 0.06 ± 7% perf-profile.self.cycles-pp.pick_next_task_fair
0.44 ± 4% -0.3 0.10 ± 6% perf-profile.self.cycles-pp.__update_load_avg_se
0.79 ± 3% -0.3 0.47 ± 2% perf-profile.self.cycles-pp.memcpy_erms
0.40 ± 3% -0.3 0.08 ± 5% perf-profile.self.cycles-pp.__fget_files
0.40 ± 3% -0.3 0.09 ± 10% perf-profile.self.cycles-pp.apparmor_file_permission
0.37 ± 4% -0.3 0.06 ± 11% perf-profile.self.cycles-pp.do_idle
0.37 ± 3% -0.3 0.08 ± 6% perf-profile.self.cycles-pp.__task_pid_nr_ns
0.43 ± 4% -0.3 0.14 ± 8% perf-profile.self.cycles-pp.enqueue_task_fair
0.39 ± 4% -0.3 0.10 perf-profile.self.cycles-pp.dequeue_task_fair
0.35 ± 2% -0.3 0.06 ± 7% perf-profile.self.cycles-pp.ftrace_graph_ret_addr
0.34 ± 6% -0.3 0.06 ± 6% perf-profile.self.cycles-pp.reweight_entity
0.34 ± 2% -0.3 0.07 ± 9% perf-profile.self.cycles-pp.perf_event_output_forward
0.32 ± 6% -0.3 0.06 ± 11% perf-profile.self.cycles-pp.os_xsave
0.32 ± 2% -0.3 0.06 perf-profile.self.cycles-pp.perf_callchain_user
0.31 ± 6% -0.3 0.05 ± 7% perf-profile.self.cycles-pp.vfs_write
0.33 ± 8% -0.3 0.07 ± 9% perf-profile.self.cycles-pp.core_kernel_text
0.31 ± 4% -0.3 0.06 ± 6% perf-profile.self.cycles-pp.set_next_entity
0.31 ± 6% -0.3 0.06 perf-profile.self.cycles-pp.prepare_to_wait_event
0.28 ± 26% -0.2 0.03 ±100% perf-profile.self.cycles-pp.__flush_smp_call_function_queue
0.31 ± 2% -0.2 0.06 ± 7% perf-profile.self.cycles-pp.__might_resched
0.38 ± 11% -0.2 0.15 ± 10% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
0.29 ± 2% -0.2 0.06 perf-profile.self.cycles-pp.___perf_sw_event
0.28 ± 6% -0.2 0.05 ± 7% perf-profile.self.cycles-pp.vfs_read
0.27 ± 2% -0.2 0.05 ± 7% perf-profile.self.cycles-pp.__libc_write
0.26 ± 6% -0.2 0.04 ± 45% perf-profile.self.cycles-pp.__libc_read
0.38 ± 2% -0.2 0.16 ± 3% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.24 ± 3% -0.2 0.04 ± 45% perf-profile.self.cycles-pp.kvm_guest_state
0.24 ± 6% -0.2 0.04 ± 44% perf-profile.self.cycles-pp.try_to_wake_up
0.22 ± 4% -0.2 0.02 ± 99% perf-profile.self.cycles-pp.do_syscall_64
0.22 ± 5% -0.2 0.03 ± 70% perf-profile.self.cycles-pp.select_task_rq_fair
0.24 ± 9% -0.2 0.06 ± 9% perf-profile.self.cycles-pp.tick_nohz_idle_exit
0.29 ± 5% -0.2 0.11 ± 7% perf-profile.self.cycles-pp.prepare_task_switch
0.27 ± 5% -0.2 0.09 ± 9% perf-profile.self.cycles-pp.perf_trace_sched_switch
0.33 ± 6% -0.1 0.20 ± 2% perf-profile.self.cycles-pp.__output_copy
0.18 ± 4% -0.1 0.05 ± 7% perf-profile.self.cycles-pp.update_rq_clock
0.14 ± 6% -0.1 0.05 perf-profile.self.cycles-pp.native_irq_return_iret
0.23 ± 9% -0.1 0.14 ± 5% perf-profile.self.cycles-pp.perf_output_sample
0.17 ± 10% -0.1 0.10 ± 27% perf-profile.self.cycles-pp.queue_event
0.00 +0.1 0.06 ± 11% perf-profile.self.cycles-pp.load_balance
0.00 +0.1 0.10 ± 8% perf-profile.self.cycles-pp.idle_cpu
0.00 +0.1 0.10 ± 9% perf-profile.self.cycles-pp.update_sd_lb_stats
0.30 +0.3 0.55 ± 2% perf-profile.self.cycles-pp._raw_spin_lock
0.09 ± 14% +0.3 0.40 ± 13% perf-profile.self.cycles-pp.mwait_idle_with_hints
0.00 +0.6 0.64 ± 10% perf-profile.self.cycles-pp.update_sg_lb_stats
0.00 +80.0 80.01 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath***************************************************************************************************
lkp-csl-2sp3: 96 threads 2 sockets Intel(R) Xeon(R) Platinum 8260L CPU @ 2.40GHz (Cascade Lake) with 128G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase:
gcc-11/performance/x86_64-rhel-8.3/debian-11.1-x86_64-20220510.cgz/300s/2T/lkp-csl-2sp3/shm-xread-seq-mt/vm-scalability
commit:
99babd04b2 ("selftests/rseq: Implement rseq numa node id field selftest") af7f588d8f ("sched: Introduce per-memory-map concurrency ID")
99babd04b2505471 af7f588d8f7355bc4298dd1962d
---------------- ---------------------------
%stddev %change %stddev
\ | \
210861 -9.0% 191845 vm-scalability.median
20242720 -9.0% 18417158 vm-scalability.throughput
55019520 -10.2% 49390360 vm-scalability.time.maximum_resident_set_size
1.9e+08 -12.8% 1.657e+08 vm-scalability.time.minor_page_faults
2259 +7.5% 2428 vm-scalability.time.percent_of_cpu_this_job_got
4300 +19.2% 5126 vm-scalability.time.system_time
2642 ± 2% -11.8% 2329 vm-scalability.time.user_time
2.705e+08 -4.5% 2.583e+08 vm-scalability.time.voluntary_context_switches
6.075e+09 -8.9% 5.535e+09 vm-scalability.workload
14.58 +3.3 17.91 mpstat.cpu.all.sys%
11802313 ± 2% -15.5% 9968423 ± 8% numa-numastat.node1.local_node
11868695 ± 2% -15.5% 10031212 ± 7% numa-numastat.node1.numa_hit
31341412 -8.8% 28589584 vmstat.memory.cache
1739822 -4.6% 1659696 vmstat.system.cs
198273 +12.4% 222847 vmstat.system.in
29152003 -9.6% 26360019 meminfo.Committed_AS
10192230 ± 8% -23.6% 7787187 ± 10% meminfo.DirectMap2M
27542178 -9.8% 24843166 meminfo.Inactive
27541034 -9.8% 24842025 meminfo.Inactive(anon)
26690232 -9.7% 24102542 meminfo.Mapped
57049 -8.9% 51976 meminfo.PageTables
27457190 -9.9% 24751049 meminfo.Shmem
910.33 +15.6% 1052 turbostat.Avg_MHz
29.46 +4.6 34.05 turbostat.Busy%
1.992e+08 ± 2% -11.3% 1.767e+08 turbostat.C1
44.78 -5.7 39.12 turbostat.C1%
56047 ± 92% +1006.4% 620087 ± 67% turbostat.C6
0.24 -20.8% 0.19 turbostat.IPC
62645660 +12.3% 70378539 turbostat.IRQ
225.94 +2.1% 230.71 turbostat.PkgWatt
39.08 +1.0% 39.47 turbostat.RAMWatt
248641 ± 19% +35.2% 336049 ± 14% numa-meminfo.node0.AnonPages.max
17517837 -19.5% 14100517 ± 16% numa-meminfo.node0.FilePages
114473 ± 2% -35.4% 73942 ± 36% numa-meminfo.node0.KReclaimable
18308067 -18.7% 14882836 ± 15% numa-meminfo.node0.MemUsed
114473 ± 2% -35.4% 73942 ± 36% numa-meminfo.node0.SReclaimable
224848 ± 2% -17.6% 185243 ± 16% numa-meminfo.node0.Slab
3738197 -66.8% 1241460 ±140% numa-meminfo.node0.Unevictable
13816698 ± 3% -13.3% 11974436 ± 9% numa-meminfo.node1.Inactive
13816181 ± 3% -13.3% 11973743 ± 9% numa-meminfo.node1.Inactive(anon)
13327964 ± 3% -12.8% 11622819 ± 8% numa-meminfo.node1.Mapped
28651 ± 2% -13.3% 24845 ± 9% numa-meminfo.node1.PageTables
13689339 ± 2% -13.2% 11880742 ± 8% numa-meminfo.node1.Shmem
26826 ±122% +9309.0% 2524087 ± 69% numa-meminfo.node1.Unevictable
4378477 -19.5% 3524382 ± 16% numa-vmstat.node0.nr_file_pages
28614 ± 2% -35.4% 18480 ± 36% numa-vmstat.node0.nr_slab_reclaimable
934549 -66.8% 310364 ±140% numa-vmstat.node0.nr_unevictable
934549 -66.8% 310364 ±140% numa-vmstat.node0.nr_zone_unevictable
3452954 ± 3% -13.3% 2992588 ± 9% numa-vmstat.node1.nr_inactive_anon
3330866 ± 3% -12.8% 2904357 ± 8% numa-vmstat.node1.nr_mapped
7166 ± 2% -13.3% 6214 ± 9% numa-vmstat.node1.nr_page_table_pages
3421209 ± 2% -13.2% 2969320 ± 8% numa-vmstat.node1.nr_shmem
6706 ±122% +9309.8% 631021 ± 69% numa-vmstat.node1.nr_unevictable
3452953 ± 3% -13.3% 2992587 ± 9% numa-vmstat.node1.nr_zone_inactive_anon
6706 ±122% +9309.8% 631021 ± 69% numa-vmstat.node1.nr_zone_unevictable
11868617 ± 2% -15.5% 10031142 ± 7% numa-vmstat.node1.numa_hit
11802234 ± 2% -15.5% 9968353 ± 8% numa-vmstat.node1.numa_local
61039 -4.3% 58390 ± 2% proc-vmstat.nr_active_anon
2454405 +2.8% 2522406 proc-vmstat.nr_dirty_background_threshold
4914813 +2.8% 5050981 proc-vmstat.nr_dirty_threshold
7805675 -8.7% 7129014 proc-vmstat.nr_file_pages
24733767 +2.8% 25414774 proc-vmstat.nr_free_pages
6885305 -9.8% 6210359 proc-vmstat.nr_inactive_anon
6672510 -9.7% 6025638 proc-vmstat.nr_mapped
14262 -8.9% 12994 proc-vmstat.nr_page_table_pages
6864084 -9.9% 6187292 proc-vmstat.nr_shmem
44375 -3.4% 42867 proc-vmstat.nr_slab_reclaimable
61039 -4.3% 58390 ± 2% proc-vmstat.nr_zone_active_anon
6885305 -9.8% 6210359 proc-vmstat.nr_zone_inactive_anon
24681264 -12.8% 21531077 proc-vmstat.numa_hit
24592528 -12.8% 21443489 proc-vmstat.numa_local
24808648 -12.6% 21680796 proc-vmstat.pgalloc_normal
1.912e+08 -12.7% 1.669e+08 proc-vmstat.pgfault
24814673 -12.6% 21686002 proc-vmstat.pgfree
6448 ± 45% +115.5% 13893 ± 22% sched_debug.cfs_rq:/.MIN_vruntime.avg
531865 ± 56% +59.3% 847386 ± 2% sched_debug.cfs_rq:/.MIN_vruntime.max
57529 ± 51% +81.1% 104202 ± 10% sched_debug.cfs_rq:/.MIN_vruntime.stddev
0.22 ± 6% +11.1% 0.25 ± 9% sched_debug.cfs_rq:/.h_nr_running.avg
10123 ± 6% +15.6% 11702 ± 7% sched_debug.cfs_rq:/.load.avg
365.17 ± 36% -41.9% 212.22 sched_debug.cfs_rq:/.load_avg.max
6448 ± 45% +115.5% 13893 ± 22% sched_debug.cfs_rq:/.max_vruntime.avg
531865 ± 56% +59.3% 847386 ± 2% sched_debug.cfs_rq:/.max_vruntime.max
57529 ± 51% +81.1% 104202 ± 10% sched_debug.cfs_rq:/.max_vruntime.stddev
782070 +13.6% 888573 sched_debug.cfs_rq:/.min_vruntime.avg
821150 +13.6% 932625 sched_debug.cfs_rq:/.min_vruntime.max
753697 +12.3% 846465 sched_debug.cfs_rq:/.min_vruntime.min
0.22 ± 5% +11.4% 0.25 ± 9% sched_debug.cfs_rq:/.nr_running.avg
-5981 -259.2% 9521 ± 80% sched_debug.cfs_rq:/.spread0.avg
1179 +11.2% 1311 ± 3% sched_debug.cpu.curr->pid.avg
0.22 +13.5% 0.25 ± 2% sched_debug.cpu.nr_running.avg
18886 ± 16% +41.6% 26735 ± 12% sched_debug.cpu.nr_switches.stddev
4.18 +8.6% 4.54 perf-stat.i.MPKI
2.29e+10 -8.2% 2.101e+10 perf-stat.i.branch-instructions
0.19 +0.0 0.22 ± 2% perf-stat.i.branch-miss-rate%
40178596 +6.8% 42925617 perf-stat.i.branch-misses
21.94 +1.8 23.76 perf-stat.i.cache-miss-rate%
66452612 +8.9% 72391872 perf-stat.i.cache-misses
1760358 -4.6% 1678776 perf-stat.i.context-switches
1.20 +24.5% 1.49 perf-stat.i.cpi
8.839e+10 +15.2% 1.018e+11 perf-stat.i.cpu-cycles
6205 +7.3% 6660 perf-stat.i.cpu-migrations
1330 +5.9% 1408 perf-stat.i.cycles-between-cache-misses
4572848 -10.1% 4112764 perf-stat.i.dTLB-load-misses
2.122e+10 -7.9% 1.954e+10 perf-stat.i.dTLB-loads
0.02 -0.0 0.02 perf-stat.i.dTLB-store-miss-rate%
702307 -13.0% 611099 perf-stat.i.dTLB-store-misses
3.917e+09 -7.5% 3.622e+09 perf-stat.i.dTLB-stores
4714448 -5.7% 4444422 perf-stat.i.iTLB-load-misses
10445194 -7.0% 9716326 perf-stat.i.iTLB-loads
7.353e+10 -7.6% 6.797e+10 perf-stat.i.instructions
0.84 -19.4% 0.67 perf-stat.i.ipc
0.92 +15.2% 1.06 perf-stat.i.metric.GHz
473.64 +4.2% 493.63 perf-stat.i.metric.K/sec
503.52 -8.0% 463.30 perf-stat.i.metric.M/sec
620975 -12.9% 541054 perf-stat.i.minor-faults
19518092 +6.5% 20793294 perf-stat.i.node-load-misses
1039427 -8.9% 946875 ± 2% perf-stat.i.node-loads
10095749 +15.1% 11618998 perf-stat.i.node-store-misses
620979 -12.9% 541058 perf-stat.i.page-faults
4.22 +7.5% 4.54 perf-stat.overall.MPKI
0.18 +0.0 0.20 perf-stat.overall.branch-miss-rate%
21.43 +2.1 23.49 perf-stat.overall.cache-miss-rate%
1.20 +24.6% 1.50 perf-stat.overall.cpi
1330 +5.8% 1406 ± 2% perf-stat.overall.cycles-between-cache-misses
0.02 -0.0 0.02 perf-stat.overall.dTLB-load-miss-rate%
0.02 -0.0 0.02 perf-stat.overall.dTLB-store-miss-rate%
15600 -2.0% 15295 perf-stat.overall.instructions-per-iTLB-miss
0.83 -19.7% 0.67 perf-stat.overall.ipc
3724 +1.6% 3783 perf-stat.overall.path-length
2.282e+10 -8.2% 2.094e+10 perf-stat.ps.branch-instructions
40052157 +6.9% 42799990 perf-stat.ps.branch-misses
66240006 +9.0% 72170753 perf-stat.ps.cache-misses
1754492 -4.6% 1673329 perf-stat.ps.context-switches
8.81e+10 +15.2% 1.015e+11 perf-stat.ps.cpu-cycles
6191 +7.3% 6646 perf-stat.ps.cpu-migrations
4557705 -10.1% 4099312 perf-stat.ps.dTLB-load-misses
2.115e+10 -7.9% 1.947e+10 perf-stat.ps.dTLB-loads
699965 -13.0% 609113 perf-stat.ps.dTLB-store-misses
3.904e+09 -7.5% 3.611e+09 perf-stat.ps.dTLB-stores
4698565 -5.7% 4429361 perf-stat.ps.iTLB-load-misses
10410202 -7.0% 9684466 perf-stat.ps.iTLB-loads
7.329e+10 -7.6% 6.775e+10 perf-stat.ps.instructions
618900 -12.9% 539304 perf-stat.ps.minor-faults
19458339 +6.6% 20734296 perf-stat.ps.node-load-misses
1036377 -8.9% 944375 ± 2% perf-stat.ps.node-loads
10063664 +15.1% 11583746 perf-stat.ps.node-store-misses
618903 -12.9% 539307 perf-stat.ps.page-faults
2.263e+13 -7.4% 2.094e+13 perf-stat.total.instructions
23.77 -3.8 19.94 ± 3% perf-profile.calltrace.cycles-pp.mwait_idle_with_hints.intel_idle_irq.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
17.03 -3.4 13.62 perf-profile.calltrace.cycles-pp.do_rw_once
27.40 -2.9 24.55 perf-profile.calltrace.cycles-pp.intel_idle_irq.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
47.87 -2.6 45.25 perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
46.90 -2.5 44.43 perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
47.06 -2.4 44.67 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
12.10 -1.7 10.37 perf-profile.calltrace.cycles-pp.finish_fault.do_read_fault.do_fault.__handle_mm_fault.handle_mm_fault
12.01 -1.7 10.30 perf-profile.calltrace.cycles-pp._raw_spin_lock.finish_fault.do_read_fault.do_fault.__handle_mm_fault
11.80 -1.6 10.15 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.finish_fault.do_read_fault.do_fault
10.69 -0.8 9.89 perf-profile.calltrace.cycles-pp.filemap_map_pages.do_read_fault.do_fault.__handle_mm_fault.handle_mm_fault
0.94 -0.6 0.35 ± 70% perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.folio_wait_bit_common.__filemap_get_folio.shmem_get_folio_gfp.shmem_fault
8.46 -0.5 7.98 perf-profile.calltrace.cycles-pp._raw_spin_lock.filemap_map_pages.do_read_fault.do_fault.__handle_mm_fault
8.37 -0.5 7.89 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.filemap_map_pages.do_read_fault.do_fault 4.22 ± 5% -0.4 3.79 ± 2% perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
1.20 -0.3 0.93 ± 3% perf-profile.calltrace.cycles-pp.folio_wake_bit.do_read_fault.do_fault.__handle_mm_fault.handle_mm_fault
1.57 -0.3 1.30 perf-profile.calltrace.cycles-pp.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
0.97 -0.2 0.77 ± 2% perf-profile.calltrace.cycles-pp.__wake_up_common.folio_wake_bit.do_read_fault.do_fault.__handle_mm_fault
0.76 -0.2 0.56 ± 2% perf-profile.calltrace.cycles-pp.try_to_wake_up.wake_page_function.__wake_up_common.folio_wake_bit.do_read_fault
1.03 -0.2 0.85 perf-profile.calltrace.cycles-pp.sched_ttwu_pending.flush_smp_call_function_queue.do_idle.cpu_startup_entry.start_secondary
0.86 -0.2 0.68 ± 2% perf-profile.calltrace.cycles-pp.wake_page_function.__wake_up_common.folio_wake_bit.do_read_fault.do_fault
0.76 ± 2% -0.2 0.59 ± 4% perf-profile.calltrace.cycles-pp.next_uptodate_page.filemap_map_pages.do_read_fault.do_fault.__handle_mm_fault
0.80 -0.1 0.69 ± 3% perf-profile.calltrace.cycles-pp.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
0.61 -0.1 0.55 perf-profile.calltrace.cycles-pp.handle_pte_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault
0.97 -0.0 0.92 perf-profile.calltrace.cycles-pp.folio_wake_bit.filemap_map_pages.do_read_fault.do_fault.__handle_mm_fault
0.58 ± 2% -0.0 0.54 perf-profile.calltrace.cycles-pp._raw_spin_lock.handle_pte_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
0.57 ± 2% -0.0 0.53 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.handle_pte_fault.__handle_mm_fault.handle_mm_fault
14.92 +0.9 15.86 perf-profile.calltrace.cycles-pp.mwait_idle_with_hints.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
14.92 +0.9 15.86 perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
30.86 +1.4 32.30 perf-profile.calltrace.cycles-pp.asm_exc_page_fault.do_access
29.74 +1.5 31.20 perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault.do_access
29.59 +1.5 31.05 perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
28.77 +1.5 30.23 perf-profile.calltrace.cycles-pp.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault
28.76 +1.5 30.23 perf-profile.calltrace.cycles-pp.do_read_fault.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
30.41 +1.5 31.93 perf-profile.calltrace.cycles-pp.do_user_addr_fault.exc_page_fault.asm_exc_page_fault.do_access
30.42 +1.5 31.94 perf-profile.calltrace.cycles-pp.exc_page_fault.asm_exc_page_fault.do_access
7.72 ± 4% +1.9 9.65 ± 6% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.mwait_idle_with_hints.intel_idle_irq.cpuidle_enter_state.cpuidle_enter
50.46 +2.1 52.56 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
50.47 +2.1 52.57 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
50.45 +2.1 52.55 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
50.93 +2.2 53.16 perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
4.41 +4.3 8.75 ± 2% perf-profile.calltrace.cycles-pp.__do_fault.do_read_fault.do_fault.__handle_mm_fault.handle_mm_fault
4.39 +4.4 8.74 ± 2% perf-profile.calltrace.cycles-pp.shmem_fault.__do_fault.do_read_fault.do_fault.__handle_mm_fault
0.00 +4.4 4.36 ± 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prepare_task_switch.__schedule.schedule_idle
4.28 +4.4 8.68 ± 2% perf-profile.calltrace.cycles-pp.shmem_get_folio_gfp.shmem_fault.__do_fault.do_read_fault.do_fault
3.86 +4.5 8.34 ± 2% perf-profile.calltrace.cycles-pp.__filemap_get_folio.shmem_get_folio_gfp.shmem_fault.__do_fault.do_read_fault
3.39 +4.6 8.01 ± 3% perf-profile.calltrace.cycles-pp.folio_wait_bit_common.__filemap_get_folio.shmem_get_folio_gfp.shmem_fault.__do_fault
0.00 +4.7 4.71 ± 5% perf-profile.calltrace.cycles-pp._raw_spin_lock.prepare_task_switch.__schedule.schedule_idle.do_idle
0.00 +4.7 4.73 ± 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.prepare_task_switch.__schedule.schedule
0.00 +4.9 4.87 ± 5% perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule_idle.do_idle.cpu_startup_entry
0.00 +5.1 5.05 ± 5% perf-profile.calltrace.cycles-pp._raw_spin_lock.prepare_task_switch.__schedule.schedule.io_schedule
0.66 +5.1 5.73 ± 4% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
0.62 +5.1 5.70 ± 4% perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
1.77 +5.1 6.90 ± 4% perf-profile.calltrace.cycles-pp.schedule.io_schedule.folio_wait_bit_common.__filemap_get_folio.shmem_get_folio_gfp
1.78 +5.1 6.91 ± 4% perf-profile.calltrace.cycles-pp.io_schedule.folio_wait_bit_common.__filemap_get_folio.shmem_get_folio_gfp.shmem_fault
1.74 +5.1 6.88 ± 4% perf-profile.calltrace.cycles-pp.__schedule.schedule.io_schedule.folio_wait_bit_common.__filemap_get_folio
0.00 +5.2 5.17 ± 5% perf-profile.calltrace.cycles-pp.prepare_task_switch.__schedule.schedule.io_schedule.folio_wait_bit_common
14.97 -3.1 11.87 perf-profile.children.cycles-pp.do_rw_once
27.65 -2.8 24.82 perf-profile.children.cycles-pp.intel_idle_irq
48.31 -2.6 45.76 perf-profile.children.cycles-pp.cpuidle_idle_call
47.33 -2.4 44.92 perf-profile.children.cycles-pp.cpuidle_enter
47.32 -2.4 44.92 perf-profile.children.cycles-pp.cpuidle_enter_state
42.69 -1.8 40.85 perf-profile.children.cycles-pp.mwait_idle_with_hints
12.10 -1.7 10.37 perf-profile.children.cycles-pp.finish_fault
10.69 -0.8 9.89 perf-profile.children.cycles-pp.filemap_map_pages
4.27 ± 4% -0.4 3.83 ± 2% perf-profile.children.cycles-pp.poll_idle
1.02 -0.4 0.58 ± 3% perf-profile.children.cycles-pp._raw_spin_lock_irq
2.18 -0.3 1.85 perf-profile.children.cycles-pp.folio_wake_bit
1.59 -0.3 1.32 perf-profile.children.cycles-pp.flush_smp_call_function_queue
39.78 -0.2 39.54 perf-profile.children.cycles-pp.do_access
1.73 -0.2 1.52 perf-profile.children.cycles-pp.__wake_up_common
1.35 ± 2% -0.2 1.14 perf-profile.children.cycles-pp.try_to_wake_up
1.53 -0.2 1.35 perf-profile.children.cycles-pp.wake_page_function
0.76 ± 2% -0.2 0.59 ± 4% perf-profile.children.cycles-pp.next_uptodate_page
1.05 -0.2 0.88 perf-profile.children.cycles-pp.sched_ttwu_pending
0.70 ± 2% -0.2 0.55 perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.82 ± 2% -0.1 0.70 ± 3% perf-profile.children.cycles-pp.menu_select
0.89 ± 5% -0.1 0.78 perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
0.56 -0.1 0.47 perf-profile.children.cycles-pp.ttwu_do_activate
0.64 ± 8% -0.1 0.55 perf-profile.children.cycles-pp.hrtimer_interrupt
0.50 -0.1 0.41 perf-profile.children.cycles-pp.enqueue_task_fair
0.65 ± 8% -0.1 0.56 perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
0.49 -0.1 0.42 perf-profile.children.cycles-pp.dequeue_task_fair
0.52 -0.1 0.45 ± 2% perf-profile.children.cycles-pp.ttwu_queue_wakelist
0.32 ± 2% -0.1 0.25 ± 4% perf-profile.children.cycles-pp.select_task_rq
0.42 ± 2% -0.1 0.35 ± 3% perf-profile.children.cycles-pp.enqueue_entity
0.44 -0.1 0.38 ± 2% perf-profile.children.cycles-pp.dequeue_entity
0.26 -0.1 0.21 ± 6% perf-profile.children.cycles-pp.select_task_rq_fair
0.27 ± 3% -0.1 0.21 ± 2% perf-profile.children.cycles-pp.__smp_call_single_queue
0.27 -0.1 0.21 ± 2% perf-profile.children.cycles-pp.llist_add_batch
0.34 -0.1 0.28 perf-profile.children.cycles-pp.__flush_smp_call_function_queue
0.32 ± 6% -0.1 0.27 ± 3% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.61 -0.1 0.55 perf-profile.children.cycles-pp.handle_pte_fault
0.21 ± 2% -0.1 0.16 ± 2% perf-profile.children.cycles-pp.folio_unlock
0.26 ± 8% -0.0 0.21 ± 2% perf-profile.children.cycles-pp.tick_sched_timer
0.31 ± 6% -0.0 0.26 ± 7% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.34 ± 8% -0.0 0.30 ± 3% perf-profile.children.cycles-pp.ktime_get
0.22 ± 2% -0.0 0.18 perf-profile.children.cycles-pp.xas_load
0.19 ± 2% -0.0 0.14 ± 6% perf-profile.children.cycles-pp.wake_affine
0.20 ± 6% -0.0 0.15 ± 6% perf-profile.children.cycles-pp.tick_sched_handle
0.33 ± 2% -0.0 0.28 perf-profile.children.cycles-pp.update_load_avg
0.28 -0.0 0.24 perf-profile.children.cycles-pp.update_rq_clock
0.18 -0.0 0.14 ± 5% perf-profile.children.cycles-pp.available_idle_cpu
0.20 ± 2% -0.0 0.17 ± 2% perf-profile.children.cycles-pp.llist_reverse_order
0.19 ± 8% -0.0 0.15 ± 6% perf-profile.children.cycles-pp.update_process_times
0.13 ± 6% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.scheduler_tick
0.12 ± 6% -0.0 0.09 ± 5% perf-profile.children.cycles-pp.xas_find
0.20 ± 2% -0.0 0.17 ± 2% perf-profile.children.cycles-pp.native_irq_return_iret
0.06 ± 7% -0.0 0.03 ± 70% perf-profile.children.cycles-pp.tick_nohz_idle_enter
0.21 ± 9% -0.0 0.18 ± 6% perf-profile.children.cycles-pp.tick_nohz_next_event
0.15 ± 3% -0.0 0.12 ± 3% perf-profile.children.cycles-pp.native_sched_clock
0.35 -0.0 0.32 ± 2% perf-profile.children.cycles-pp.pick_next_task_fair
0.17 ± 2% -0.0 0.15 ± 3% perf-profile.children.cycles-pp.__switch_to_asm
0.18 ± 2% -0.0 0.15 ± 5% perf-profile.children.cycles-pp.sched_clock_cpu
0.21 -0.0 0.18 ± 5% perf-profile.children.cycles-pp.update_curr
0.08 ± 10% -0.0 0.06 ± 8% perf-profile.children.cycles-pp.read_tsc
0.09 ± 5% -0.0 0.07 perf-profile.children.cycles-pp.tick_nohz_idle_exit
0.12 -0.0 0.10 ± 4% perf-profile.children.cycles-pp.set_next_entity
0.22 ± 2% -0.0 0.20 ± 4% perf-profile.children.cycles-pp.perf_trace_sched_wakeup_template
0.12 ± 3% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.10 ± 9% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.__list_add_valid
0.07 ± 6% -0.0 0.06 ± 8% perf-profile.children.cycles-pp.__perf_sw_event
0.07 -0.0 0.05 ± 8% perf-profile.children.cycles-pp.reweight_entity
0.15 ± 3% -0.0 0.13 perf-profile.children.cycles-pp.__switch_to
0.12 -0.0 0.10 ± 4% perf-profile.children.cycles-pp.___perf_sw_event
0.12 -0.0 0.10 ± 4% perf-profile.children.cycles-pp.sync_regs
0.07 ± 7% -0.0 0.05 perf-profile.children.cycles-pp.shmem_alloc_and_acct_folio
0.11 ± 4% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.perf_tp_event
0.10 ± 4% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.put_page
0.07 -0.0 0.06 ± 8% perf-profile.children.cycles-pp.__list_del_entry_valid
0.14 ± 5% +0.0 0.16 ± 3% perf-profile.children.cycles-pp.load_balance
0.16 ± 5% +0.0 0.18 ± 2% perf-profile.children.cycles-pp.newidle_balance
0.10 ± 4% +0.0 0.12 perf-profile.children.cycles-pp.update_sg_lb_stats
0.12 ± 8% +0.0 0.14 perf-profile.children.cycles-pp.find_busiest_group
0.21 ± 6% +0.0 0.25 ± 3% perf-profile.children.cycles-pp.up_read
0.15 ± 5% +0.1 0.21 ± 6% perf-profile.children.cycles-pp.down_read_trylock
0.55 ± 5% +0.1 0.61 ± 2% perf-profile.children.cycles-pp.finish_task_switch
0.00 +0.1 0.10 ± 12% perf-profile.children.cycles-pp._find_first_zero_bit
0.09 +0.3 0.38 ± 3% perf-profile.children.cycles-pp.switch_mm_irqs_off
4.61 ± 4% +0.9 5.48 ± 5% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
15.05 +1.0 16.04 perf-profile.children.cycles-pp.intel_idle
30.89 +1.4 32.33 perf-profile.children.cycles-pp.asm_exc_page_fault
29.75 +1.5 31.21 perf-profile.children.cycles-pp.handle_mm_fault
29.60 +1.5 31.06 perf-profile.children.cycles-pp.__handle_mm_fault
28.77 +1.5 30.23 perf-profile.children.cycles-pp.do_read_fault
28.77 +1.5 30.24 perf-profile.children.cycles-pp.do_fault
30.42 +1.5 31.94 perf-profile.children.cycles-pp.do_user_addr_fault
30.43 +1.5 31.95 perf-profile.children.cycles-pp.exc_page_fault
50.47 +2.1 52.57 perf-profile.children.cycles-pp.start_secondary
50.93 +2.2 53.16 perf-profile.children.cycles-pp.secondary_startup_64_no_verify
50.93 +2.2 53.16 perf-profile.children.cycles-pp.cpu_startup_entry
50.91 +2.2 53.14 perf-profile.children.cycles-pp.do_idle
4.41 +4.3 8.75 ± 2% perf-profile.children.cycles-pp.__do_fault
4.39 +4.4 8.74 ± 2% perf-profile.children.cycles-pp.shmem_fault
4.29 +4.4 8.69 ± 2% perf-profile.children.cycles-pp.shmem_get_folio_gfp
3.86 +4.5 8.34 ± 2% perf-profile.children.cycles-pp.__filemap_get_folio
3.40 +4.6 8.01 ± 3% perf-profile.children.cycles-pp.folio_wait_bit_common
0.67 +5.1 5.79 ± 4% perf-profile.children.cycles-pp.schedule_idle
1.78 +5.1 6.91 ± 4% perf-profile.children.cycles-pp.schedule
1.78 +5.1 6.91 ± 4% perf-profile.children.cycles-pp.io_schedule
21.60 +6.6 28.16 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
21.40 +7.5 28.92 perf-profile.children.cycles-pp._raw_spin_lock
0.11 ± 4% +10.0 10.10 ± 5% perf-profile.children.cycles-pp.prepare_task_switch
2.40 +10.3 12.66 ± 4% perf-profile.children.cycles-pp.__schedule
10.91 -2.4 8.54 perf-profile.self.cycles-pp.do_rw_once
42.19 -1.8 40.35 perf-profile.self.cycles-pp.mwait_idle_with_hints
6.82 -1.2 5.58 perf-profile.self.cycles-pp.do_access
4.18 ± 4% -0.4 3.76 ± 2% perf-profile.self.cycles-pp.poll_idle
0.75 ± 2% -0.2 0.58 ± 5% perf-profile.self.cycles-pp.next_uptodate_page
0.32 ± 4% -0.1 0.20 ± 6% perf-profile.self.cycles-pp.__filemap_get_folio
0.57 ± 3% -0.1 0.46 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.57 -0.1 0.49 perf-profile.self.cycles-pp.folio_wait_bit_common
0.29 -0.1 0.22 ± 3% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.44 -0.1 0.38 ± 2% perf-profile.self.cycles-pp.menu_select
0.09 ± 5% -0.1 0.04 ± 71% perf-profile.self.cycles-pp.shmem_fault
0.26 ± 3% -0.1 0.21 ± 2% perf-profile.self.cycles-pp.llist_add_batch
0.21 -0.0 0.16 ± 2% perf-profile.self.cycles-pp.folio_unlock
0.34 ± 2% -0.0 0.30 ± 4% perf-profile.self.cycles-pp.filemap_map_pages
0.18 ± 2% -0.0 0.14 ± 3% perf-profile.self.cycles-pp.available_idle_cpu
0.20 ± 2% -0.0 0.17 ± 2% perf-profile.self.cycles-pp.llist_reverse_order
0.20 ± 2% -0.0 0.17 ± 2% perf-profile.self.cycles-pp.native_irq_return_iret
0.17 ± 2% -0.0 0.14 ± 3% perf-profile.self.cycles-pp.__switch_to_asm
0.15 ± 5% -0.0 0.12 ± 6% perf-profile.self.cycles-pp.native_sched_clock
0.15 ± 3% -0.0 0.12 ± 3% perf-profile.self.cycles-pp.enqueue_entity
0.17 ± 2% -0.0 0.15 ± 3% perf-profile.self.cycles-pp.xas_load
0.21 -0.0 0.18 ± 2% perf-profile.self.cycles-pp.flush_smp_call_function_queue
0.20 ± 4% -0.0 0.17 ± 2% perf-profile.self.cycles-pp.__wake_up_common
0.10 ± 9% -0.0 0.08 ± 5% perf-profile.self.cycles-pp.__list_add_valid
0.08 ± 5% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.enqueue_task_fair
0.07 ± 6% -0.0 0.06 ± 8% perf-profile.self.cycles-pp.read_tsc
0.13 -0.0 0.11 ± 4% perf-profile.self.cycles-pp.__flush_smp_call_function_queue
0.10 ± 4% -0.0 0.08 ± 10% perf-profile.self.cycles-pp.cpuidle_idle_call
0.07 -0.0 0.05 ± 8% perf-profile.self.cycles-pp.reweight_entity
0.14 ± 3% -0.0 0.13 ± 3% perf-profile.self.cycles-pp.__switch_to
0.12 -0.0 0.10 ± 4% perf-profile.self.cycles-pp.sync_regs
0.12 ± 4% -0.0 0.10 ± 8% perf-profile.self.cycles-pp.update_load_avg
0.20 ± 4% -0.0 0.18 ± 2% perf-profile.self.cycles-pp.update_rq_clock
0.16 ± 3% -0.0 0.14 ± 5% perf-profile.self.cycles-pp.ttwu_queue_wakelist
0.12 ± 4% -0.0 0.10 perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.10 -0.0 0.09 ± 5% perf-profile.self.cycles-pp.___perf_sw_event
0.07 -0.0 0.06 ± 8% perf-profile.self.cycles-pp.__list_del_entry_valid
0.06 ± 7% -0.0 0.05 perf-profile.self.cycles-pp.select_task_rq_fair
0.09 ± 5% -0.0 0.07 ± 6% perf-profile.self.cycles-pp.__update_load_avg_se
0.07 -0.0 0.06 perf-profile.self.cycles-pp.sched_ttwu_pending
0.08 ± 6% +0.0 0.09 perf-profile.self.cycles-pp.update_sg_lb_stats
0.14 ± 5% +0.0 0.17 ± 5% perf-profile.self.cycles-pp.wake_page_function
0.21 ± 6% +0.0 0.25 ± 3% perf-profile.self.cycles-pp.up_read
0.21 +0.1 0.26 perf-profile.self.cycles-pp.__handle_mm_fault
0.15 ± 5% +0.1 0.21 ± 6% perf-profile.self.cycles-pp.down_read_trylock
0.42 ± 2% +0.1 0.50 ± 3% perf-profile.self.cycles-pp.__schedule
0.00 +0.1 0.09 ± 10% perf-profile.self.cycles-pp._find_first_zero_bit
0.00 +0.1 0.12 ± 8% perf-profile.self.cycles-pp.prepare_task_switch
0.09 +0.3 0.38 ± 3% perf-profile.self.cycles-pp.switch_mm_irqs_off
0.65 +0.6 1.21 ± 2% perf-profile.self.cycles-pp._raw_spin_lock
21.50 +6.6 28.08 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance.
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests
#!/bin/sh
export_top_env()
{
export suite='will-it-scale'
export testcase='will-it-scale'
export category='benchmark'
export job_origin='will-it-scale-fpu-test.yaml'
export queue='bisect'
export testbox='lkp-csl-2sp9'
export commit='af7f588d8f7355bc4298dd1962d7826358fc95f0'
export branch='linus/master'
export name='/cephfs/tmp/d20230424-8447-1jrb2t2/will-it-scale-fpu-test.yaml'
export kernel_cmdline=
export tbox_group='lkp-csl-2sp9'
export submit_id='645bc5376f560d86a15a1974'
export job_file='/lkp/jobs/scheduled/lkp-csl-2sp9/will-it-scale-performance-context_switch1-debian-11.1-x86_64-20220510.cgz-af7f588d8f7355bc4298dd1962d7826358fc95f0-20230511-34465-pn9jw1-1.yaml'
export id='8b52890475f8222c91c7963430fa959e6783865e'
export queuer_version='/zday/lkp'
export model='Cascade Lake'
export nr_node=2
export nr_cpu=88
export memory='128G'
export nr_hdd_partitions=4
export nr_ssd_partitions=1
export hdd_partitions='/dev/disk/by-id/ata-ST4000NM0035-1V4107_ZC13Q1RD-part*'
export ssd_partitions='/dev/disk/by-id/ata-INTEL_SSDSC2BB480G7_PHDV723200JX480BGN-part1'
export rootfs_partition='/dev/disk/by-id/ata-INTEL_SSDSC2BB480G7_PHDV723200JX480BGN-part2'
export brand='Intel(R) Xeon(R) Gold 6238M CPU @ 2.10GHz'
export ucode='0x5003303'
export need_kconfig_hw='{"I40E"=>"y"}
SATA_AHCI
INTEL_TH
INTEL_TH_ACPI'
export disable_latency_stats=1
export rootfs='debian-11.1-x86_64-20220510.cgz'
export kconfig='x86_64-rhel-8.3'
export enqueue_time='2023-05-11 00:24:23 +0800'
export compiler='gcc-11'
export _id='645bc54d6f560d86a15a1975'
export _rt='/result/will-it-scale/performance-context_switch1/lkp-csl-2sp9/debian-11.1-x86_64-20220510.cgz/x86_64-rhel-8.3/gcc-11/af7f588d8f7355bc4298dd1962d7826358fc95f0'
export user='lkp'
export LKP_SERVER='internal-lkp-server'
export result_root='/result/will-it-scale/performance-context_switch1/lkp-csl-2sp9/debian-11.1-x86_64-20220510.cgz/x86_64-rhel-8.3/gcc-11/af7f588d8f7355bc4298dd1962d7826358fc95f0/0'
export scheduler_version='/lkp/lkp/src'
export arch='x86_64'
export max_uptime=2100
export initrd='/osimage/debian/debian-11.1-x86_64-20220510.cgz'
export bootloader_append='root=/dev/ram0RESULT_ROOT=/result/will-it-scale/performance-context_switch1/lkp-csl-2sp9/debian-11.1-x86_64-20220510.cgz/x86_64-rhel-8.3/gcc-11/af7f588d8f7355bc4298dd1962d7826358fc95f0/0 BOOT_IMAGE=/pkg/linux/x86_64-rhel-8.3/gcc-11/af7f588d8f7355bc4298dd1962d7826358fc95f0/vmlinuz-6.2.0-rc1-00009-gaf7f588d8f73
branch=linus/master job=/lkp/jobs/scheduled/lkp-csl-2sp9/will-it-scale-performance-context_switch1-debian-11.1-x86_64-20220510.cgz-af7f588d8f7355bc4298dd1962d7826358fc95f0-20230511-34465-pn9jw1-1.yaml user=lkp
ARCH=x86_64
kconfig=x86_64-rhel-8.3
commit=af7f588d8f7355bc4298dd1962d7826358fc95f0initcall_debug
nmi_watchdog=0
max_uptime=2100
LKP_SERVER=internal-lkp-server
nokaslr
selinux=0
debug
apic=debugsysrq_always_enabled
rcupdate.rcu_cpu_stall_timeout=100
net.ifnames=0
printk.devkmsg=on
panic=-1
softlockup_panic=1
nmi_watchdog=panic
oops=panic
load_ramdisk=2
prompt_ramdisk=0
drbd.minor_count=8
systemd.log_level=errignore_loglevel
console=tty0
earlyprintk=ttyS0,115200
console=ttyS0,115200
vga=normal
rw'
export modules_initrd='/pkg/linux/x86_64-rhel-8.3/gcc-11/af7f588d8f7355bc4298dd1962d7826358fc95f0/modules.cgz'
export bm_initrd='/osimage/deps/debian-11.1-x86_64-20220510.cgz/lkp_20220513.cgz,/osimage/deps/debian-11.1-x86_64-20220510.cgz/run-ipconfig_20220515.cgz,/osimage/deps/debian-11.1-x86_64-20220510.cgz/rsync-rootfs_20220515.cgz,/osimage/deps/debian-11.1-x86_64-20220510.cgz/will-it-scale_20220823.cgz,/osimage/pkg/debian-11.1-x86_64-20220510.cgz/will-it-scale-x86_64-a34a85c-1_20220601.cgz,/osimage/deps/debian-11.1-x86_64-20220510.cgz/mpstat_20220516.cgz,/osimage/deps/debian-11.1-x86_64-20220510.cgz/turbostat_20220514.cgz,/osimage/pkg/debian-11.1-x86_64-20220510.cgz/turbostat-x86_64-210e04ff7681-1_20220518.cgz,/osimage/deps/debian-11.1-x86_64-20220510.cgz/perf_20230402.cgz,/osimage/pkg/debian-11.1-x86_64-20220510.cgz/perf-x86_64-00c7b5f4ddc5-1_20230402.cgz,/osimage/pkg/debian-11.1-x86_64-20220510.cgz/sar-x86_64-c5bb321-1_20220518.cgz,/osimage/deps/debian-11.1-x86_64-20220510.cgz/hw_20220526.cgz'
export ucode_initrd='/osimage/ucode/intel-ucode-20230406.cgz'
export lkp_initrd='/osimage/user/lkp/lkp-x86_64.cgz'
export site='lkp-wsx01'
export LKP_CGI_PORT=80
export LKP_CIFS_PORT=139
export last_kernel='6.3.0'
export queue_cmdline_keys='branch
commit
kbuild_queue_analysis'
export repeat_to=3
export kbuild_queue_analysis=1
export kernel='/pkg/linux/x86_64-rhel-8.3/gcc-11/af7f588d8f7355bc4298dd1962d7826358fc95f0/vmlinuz-6.2.0-rc1-00009-gaf7f588d8f73'
export dequeue_time='2023-05-11 00:31:59 +0800'
export job_initrd='/lkp/jobs/scheduled/lkp-csl-2sp9/will-it-scale-performance-context_switch1-debian-11.1-x86_64-20220510.cgz-af7f588d8f7355bc4298dd1962d7826358fc95f0-20230511-34465-pn9jw1-1.cgz'
[ -n "$LKP_SRC" ] ||
export LKP_SRC=/lkp/${user:-lkp}/src
}run_job()
{
echo $$ > $TMP/run-job.pid
. $LKP_SRC/lib/http.sh
. $LKP_SRC/lib/job.sh
. $LKP_SRC/lib/env.sh
export_top_env
run_setup $LKP_SRC/setup/cpufreq_governor 'performance'
run_monitor $LKP_SRC/monitors/wrapper kmsg
run_monitor $LKP_SRC/monitors/no-stdout/wrapper boot-time run_monitor $LKP_SRC/monitors/wrapper uptime
run_monitor $LKP_SRC/monitors/wrapper iostat
run_monitor $LKP_SRC/monitors/wrapper heartbeat
run_monitor $LKP_SRC/monitors/wrapper vmstat
run_monitor $LKP_SRC/monitors/wrapper numa-numastat run_monitor $LKP_SRC/monitors/wrapper numa-vmstat run_monitor $LKP_SRC/monitors/wrapper numa-meminfo run_monitor $LKP_SRC/monitors/wrapper proc-vmstat run_monitor $LKP_SRC/monitors/wrapper proc-stat
run_monitor $LKP_SRC/monitors/wrapper meminfo
run_monitor $LKP_SRC/monitors/wrapper slabinfo
run_monitor $LKP_SRC/monitors/wrapper interrupts
run_monitor $LKP_SRC/monitors/wrapper lock_stat
run_monitor lite_mode=1 $LKP_SRC/monitors/wrapper perf-sched run_monitor $LKP_SRC/monitors/wrapper softirqs
run_monitor $LKP_SRC/monitors/one-shot/wrapper bdi_dev_mapping run_monitor $LKP_SRC/monitors/wrapper diskstats
run_monitor $LKP_SRC/monitors/wrapper nfsstat
run_monitor $LKP_SRC/monitors/wrapper cpuidle
run_monitor $LKP_SRC/monitors/wrapper cpufreq-stats run_monitor $LKP_SRC/monitors/wrapper turbostat
run_monitor $LKP_SRC/monitors/wrapper sched_debug run_monitor $LKP_SRC/monitors/wrapper perf-stat
run_monitor $LKP_SRC/monitors/wrapper mpstat
run_monitor debug_mode=0 $LKP_SRC/monitors/no-stdout/wrapper perf-profile run_monitor $LKP_SRC/monitors/wrapper oom-killer
run_monitor $LKP_SRC/monitors/plain/watchdog
run_test test='context_switch1' $LKP_SRC/tests/wrapper will-it-scale
}
extract_stats()
{
export stats_part_begin=
export stats_part_end=
env test='context_switch1' $LKP_SRC/stats/wrapper will-it-scale $LKP_SRC/stats/wrapper kmsg
$LKP_SRC/stats/wrapper boot-time $LKP_SRC/stats/wrapper uptime
$LKP_SRC/stats/wrapper iostat
$LKP_SRC/stats/wrapper vmstat
$LKP_SRC/stats/wrapper numa-numastat $LKP_SRC/stats/wrapper numa-vmstat $LKP_SRC/stats/wrapper numa-meminfo $LKP_SRC/stats/wrapper proc-vmstat $LKP_SRC/stats/wrapper meminfo $LKP_SRC/stats/wrapper slabinfo $LKP_SRC/stats/wrapper interrupts $LKP_SRC/stats/wrapper lock_stat
env lite_mode=1 $LKP_SRC/stats/wrapper perf-sched $LKP_SRC/stats/wrapper softirqs $LKP_SRC/stats/wrapper diskstats $LKP_SRC/stats/wrapper nfsstat $LKP_SRC/stats/wrapper cpuidle $LKP_SRC/stats/wrapper turbostat $LKP_SRC/stats/wrapper sched_debug $LKP_SRC/stats/wrapper perf-stat $LKP_SRC/stats/wrapper mpstat
env debug_mode=0 $LKP_SRC/stats/wrapper perf-profile
$LKP_SRC/stats/wrapper time will-it-scale.time $LKP_SRC/stats/wrapper dmesg $LKP_SRC/stats/wrapper kmsg $LKP_SRC/stats/wrapper last_state $LKP_SRC/stats/wrapper stderr $LKP_SRC/stats/wrapper time
}
"$@"
---
#! /cephfs/tmp/d20230424-8447-1jrb2t2/will-it-scale-fpu-test.yaml
suite: will-it-scale
testcase: will-it-scale
category: benchmark
will-it-scale:
test: context_switch1job_origin: will-it-scale-fpu-test.yaml
queue: bisect
testbox: lkp-csl-2sp9
commit: af7f588d8f7355bc4298dd1962d7826358fc95f0
branch: linus/master
name: "/cephfs/tmp/d20230424-8447-1jrb2t2/will-it-scale-fpu-test.yaml"kernel_cmdline:tbox_group: lkp-csl-2sp9submit_id: 645bc5376f560d86a15a1974job_file: "/lkp/jobs/scheduled/lkp-csl-2sp9/will-it-scale-performance-context_switch1-debian-11.1-x86_64-20220510.cgz-af7f588d8f7355bc4298dd1962d7826358fc95f0-20230511-34465-pn9jw1-1.yaml"
id: 8b52890475f8222c91c7963430fa959e6783865equeuer_version: "/zday/lkp"
#! hosts/lkp-csl-2sp9
model: Cascade Lakenr_node: 2nr_cpu: 88
memory: 128Gnr_hdd_partitions: 4nr_ssd_partitions: 1hdd_partitions: "/dev/disk/by-id/ata-ST4000NM0035-1V4107_ZC13Q1RD-part*"ssd_partitions: "/dev/disk/by-id/ata-INTEL_SSDSC2BB480G7_PHDV723200JX480BGN-part1"rootfs_partition: "/dev/disk/by-id/ata-INTEL_SSDSC2BB480G7_PHDV723200JX480BGN-part2"
brand: Intel(R) Xeon(R) Gold 6238M CPU @ 2.10GHz
#! include/category/benchmark
kmsg:
boot-time:
uptime:
iostat:
heartbeat:
vmstat:
numa-numastat:
numa-vmstat:
numa-meminfo:
proc-vmstat:
proc-stat:
meminfo:
slabinfo:
interrupts:lock_stat:
perf-sched:
lite_mode: 1
softirqs:bdi_dev_mapping:
diskstats:
nfsstat:
cpuidle:
cpufreq-stats:
turbostat:sched_debug:
perf-stat:
mpstat:
perf-profile:
debug_mode: 0
#! include/category/ALLcpufreq_governor: performance
#! include/testbox/lkp-csl-2sp9
ucode: '0x5003303'need_kconfig_hw:
- I40E: y
- SATA_AHCI
- INTEL_TH
- INTEL_TH_ACPIdisable_latency_stats: 1
rootfs: debian-11.1-x86_64-20220510.cgz
kconfig: x86_64-rhel-8.3enqueue_time: 2023-05-11 00:24:23.655011677 +08:00
compiler: gcc-11_id: 645bc54d6f560d86a15a1975_rt: "/result/will-it-scale/performance-context_switch1/lkp-csl-2sp9/debian-11.1-x86_64-20220510.cgz/x86_64-rhel-8.3/gcc-11/af7f588d8f7355bc4298dd1962d7826358fc95f0"
#! schedule options
user: lkpLKP_SERVER: internal-lkp-serverresult_root: "/result/will-it-scale/performance-context_switch1/lkp-csl-2sp9/debian-11.1-x86_64-20220510.cgz/x86_64-rhel-8.3/gcc-11/af7f588d8f7355bc4298dd1962d7826358fc95f0/0"scheduler_version: "/lkp/lkp/src"
arch: x86_64max_uptime: 2100
initrd: "/osimage/debian/debian-11.1-x86_64-20220510.cgz"bootloader_append:
- root=/dev/ram0
- RESULT_ROOT=/result/will-it-scale/performance-context_switch1/lkp-csl-2sp9/debian-11.1-x86_64-20220510.cgz/x86_64-rhel-8.3/gcc-11/af7f588d8f7355bc4298dd1962d7826358fc95f0/0
- BOOT_IMAGE=/pkg/linux/x86_64-rhel-8.3/gcc-11/af7f588d8f7355bc4298dd1962d7826358fc95f0/vmlinuz-6.2.0-rc1-00009-gaf7f588d8f73
- branch=linus/master
- job=/lkp/jobs/scheduled/lkp-csl-2sp9/will-it-scale-performance-context_switch1-debian-11.1-x86_64-20220510.cgz-af7f588d8f7355bc4298dd1962d7826358fc95f0-20230511-34465-pn9jw1-1.yaml
- user=lkp
- ARCH=x86_64
- kconfig=x86_64-rhel-8.3
- commit=af7f588d8f7355bc4298dd1962d7826358fc95f0
- initcall_debug
- nmi_watchdog=0
- max_uptime=2100
- LKP_SERVER=internal-lkp-server
- nokaslr
- selinux=0
- debug
- apic=debug
- sysrq_always_enabled
- rcupdate.rcu_cpu_stall_timeout=100
- net.ifnames=0
- printk.devkmsg=on
- panic=-1
- softlockup_panic=1
- nmi_watchdog=panic
- oops=panic
- load_ramdisk=2
- prompt_ramdisk=0
- drbd.minor_count=8
- systemd.log_level=err
- ignore_loglevel
- console=tty0
- earlyprintk=ttyS0,115200
- console=ttyS0,115200
- vga=normal
- rw
#! runtime statusmodules_initrd: "/pkg/linux/x86_64-rhel-8.3/gcc-11/af7f588d8f7355bc4298dd1962d7826358fc95f0/modules.cgz"bm_initrd: "/osimage/deps/debian-11.1-x86_64-20220510.cgz/lkp_20220513.cgz,/osimage/deps/debian-11.1-x86_64-20220510.cgz/run-ipconfig_20220515.cgz,/osimage/deps/debian-11.1-x86_64-20220510.cgz/rsync-rootfs_20220515.cgz,/osimage/deps/debian-11.1-x86_64-20220510.cgz/will-it-scale_20220823.cgz,/osimage/pkg/debian-11.1-x86_64-20220510.cgz/will-it-scale-x86_64-a34a85c-1_20220601.cgz,/osimage/deps/debian-11.1-x86_64-20220510.cgz/mpstat_20220516.cgz,/osimage/deps/debian-11.1-x86_64-20220510.cgz/turbostat_20220514.cgz,/osimage/pkg/debian-11.1-x86_64-20220510.cgz/turbostat-x86_64-210e04ff7681-1_20220518.cgz,/osimage/deps/debian-11.1-x86_64-20220510.cgz/perf_20230402.cgz,/osimage/pkg/debian-11.1-x86_64-20220510.cgz/perf-x86_64-00c7b5f4ddc5-1_20230402.cgz,/osimage/pkg/debian-11.1-x86_64-20220510.cgz/sar-x86_64-c5bb321-1_20220518.cgz,/osimage/deps/debian-11.1-x86_64-20220510.cgz/hw_20220526.cgz"ucode_initrd: "/osimage/ucode/intel-ucode-20230406.cgz"lkp_initrd: "/osimage/user/lkp/lkp-x86_64.cgz"
site: lkp-wsx01
#! /db/releases/20230424231518/lkp-src/include/site/lkp-wsx01LKP_CGI_PORT: 80LKP_CIFS_PORT: 139
oom-killer:
watchdog:last_kernel: 6.4.0-rc1-2023-05-09-intel-next-01765-gcc072b5075bd
#! queue optionsqueue_cmdline_keys:
- branch
- commit
- kbuild_queue_analysis
#! /db/releases/20230506144350/lkp-src/include/site/lkp-wsx01repeat_to: 3
#! user overrideskbuild_queue_analysis: 1
kernel: "/pkg/linux/x86_64-rhel-8.3/gcc-11/af7f588d8f7355bc4298dd1962d7826358fc95f0/vmlinuz-6.2.0-rc1-00009-gaf7f588d8f73"dequeue_time: 2023-05-11 00:31:59.152337560 +08:00
#! /db/releases/20230510204317/lkp-src/include/site/lkp-wsx01job_state: finished
loadavg: 84.95 41.93 16.63 1/752 14738start_time: '1683736397'end_time: '1683736713'
version: "/lkp/lkp/.src-20230508-150358:ad1384b06da5:c6711f306d8a"
for cpu_dir in /sys/devices/system/cpu/cpu[0-9]*
do
online_file="$cpu_dir"/online [ -f "$online_file" ] && [ "$(cat "$online_file")" -eq 0 ] && continue file="$cpu_dir"/cpufreq/scaling_governor [ -f "$file" ] && echo "performance" > "$file"
done
"/lkp/benchmarks/python3/bin/python3" "./runtest.py" "context_switch1" "25" "both" "0" "0" "1" "22" "44" "66" "88"
Quoted 54 lines · kernel test robot
Hello,
kernel test robot noticed a -13.9% regression of will-it-scale.per_thread_ops on:
commit:
af7f588d8f7355bc4298dd1962d7826358fc95f0("sched: Introduce per-memory-map concurrency ID")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git mastertestcase: will-it-scale
test machine: 224 threads 2 sockets (Sapphire Rapids) with 256G memory parameters:test:
context_switch1
cpufreq_governor: performancetest-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two.
test-url: https://github.com/antonblanchard/will-it-scaleIn addition to that, the commit also has significant impact on the following tests:
+------------------+----------------------------------------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_thread_ops -73.8% regression | | test machine | 224 threads 2 sockets (Sapphire Rapids) with 256G memory | | test parameters | cpufreq_governor=performance | | | mode=thread | | | nr_task=16 | | | test=context_switch1 | +------------------+----------------------------------------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_thread_ops -57.9% regression | | test machine | 104 threads 2 sockets (Skylake) with 192G memory | | test parameters | cpufreq_governor=performance | | | mode=thread | | | nr_task=16 | | | test=context_switch1 | +------------------+----------------------------------------------------------------------------------------------------+ | testcase: change | will-it-scale: will-it-scale.per_thread_ops -85.0% regression | | test machine | 104 threads 2 sockets (Skylake) with 192G memory | | test parameters | cpufreq_governor=performance | | | mode=thread | | | nr_task=50% | | | test=context_switch1 | +------------------+----------------------------------------------------------------------------------------------------+ | testcase: change | vm-scalability: vm-scalability.throughput -9.0% regression | | test machine | 96 threads 2 sockets Intel(R) Xeon(R) Platinum 8260L CPU @ 2.40GHz (Cascade Lake) with 128G memory | | test parameters | cpufreq_governor=performance | | | runtime=300s | | | size=2T | | | test=shm-xread-seq-mt | +------------------+----------------------------------------------------------------------------------------------------+FYI, we noticed that commit
223baf9d17f2(sched: Fix performance regression introduced bymm_cid) fixed a sysbench regression, but will-it-scalecontext_switch1benchmark still saw a regression on this fix commit.
Quoted 6 lines
Furthermore, we applied the code diff in below link [1] on mainline, and the will-it-scale score was restored to the original level before this patch.
Right; so I'm thinking we can do that patch -- I'll try and get the whole lazy TLB thing sorted, but I'm not sure I can find the piece and quiet to think that over in a hurry :/