c46bfba133: stress-ng.netlink-proc.ops_per_sec -97.2% regression
Subsystems (2)
kernel test robot
build botHello,
we reviewed this report and Fengwei (Cced) pointed out it could be the patch breaks functionality, then causes stress-ng netlink-proc performance drops dramatically.
below details:
kernel test robot noticed a -97.2% regression of stress-ng.netlink-proc.ops_per_sec on:
commit: c46bfba1337d301661dbb23cfd905d4cb51f27ca ("connector: Fix proc_event_num_listeners count not cleared")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
testcase: stress-ng
test machine: 64 threads 2 sockets Intel(R) Xeon(R) Gold 6346 CPU @ 3.10GHz (Ice Lake) with 256G memory parameters:
nr_threads: 100% testtime: 60s sc_pid_max: 4194304 class: scheduler test: netlink-proc cpufreq_governor: performance
In addition to that, the commit also has significant impact on the following tests:
+------------------+----------------------------------------------------------------------------------------------+ | testcase: change | stress-ng: stress-ng.netlink-proc.ops_per_sec -97.4% regression | | test machine | 64 threads 2 sockets Intel(R) Xeon(R) Gold 6346 CPU @ 3.10GHz (Ice Lake) with 256G memory | | test parameters | class=os | | | cpufreq_governor=performance | | | disk=1HDD | | | fs=ext4 | | | nr_threads=10% | | | test=netlink-proc | | | testtime=60s | +------------------+----------------------------------------------------------------------------------------------+
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-lkp/[email protected]
Details are as below:
-------------------------------------------------------------------------------------------------->
The kernel config and materials to reproduce are available at:
https://download.01.org/0day-ci/archive/20240111/[email protected]
=========================================================================================
class/compiler/cpufreq_governor/kconfig/nr_threads/rootfs/sc_pid_max/tbox_group/test/testcase/testtime:
scheduler/gcc-12/performance/x86_64-rhel-8.3/100%/debian-11.1-x86_64-20220510.cgz/4194304/lkp-icl-2sp8/netlink-proc/stress-ng/60s
commit:
059d37b718 ("net: phy: linux/phy.h: fix Excess kernel-doc description warning") c46bfba133 ("connector: Fix proc_event_num_listeners count not cleared")
059d37b718d38d26 c46bfba1337d301661dbb23cfd9
---------------- ---------------------------
%stddev %change %stddev
\ | \
2359 +150.7% 5915 uptime.idle
3.537e+08 ± 3% +1019.9% 3.961e+09 cpuidle..time
2103185 ± 4% +94.6% 4092323 cpuidle..usage
53818952 ± 2% -96.4% 1950100 ± 13% numa-numastat.node0.local_node
53849056 ± 2% -96.3% 1978128 ± 13% numa-numastat.node0.numa_hit
52086289 ± 2% -95.2% 2523104 ± 7% numa-numastat.node1.local_node
52111562 ± 2% -95.1% 2561457 ± 7% numa-numastat.node1.numa_hit
7.85 ± 8% +88.4 96.28 mpstat.cpu.all.idle%
1.24 -0.6 0.62 mpstat.cpu.all.irq%
0.44 -0.4 0.04 mpstat.cpu.all.soft%
87.22 -84.6 2.58 mpstat.cpu.all.sys%
3.23 -2.8 0.46 mpstat.cpu.all.usr%
1117 ± 30% -99.8% 2.62 ± 53% perf-c2c.DRAM.local
6349 ± 23% -99.7% 19.25 ± 34% perf-c2c.DRAM.remote
5462 ± 23% -99.0% 52.50 ± 16% perf-c2c.HITM.local
4325 ± 23% -99.7% 12.12 ± 36% perf-c2c.HITM.remote
9788 ± 23% -99.3% 64.62 ± 16% perf-c2c.HITM.total
10.68 ± 6% +800.1% 96.13 vmstat.cpu.id
86.25 -96.0% 3.41 vmstat.cpu.sy
4111553 -25.2% 3074209 vmstat.memory.cache
63.63 -97.1% 1.82 ± 2% vmstat.procs.r
146012 -95.3% 6884 ± 4% vmstat.system.cs
152731 -55.9% 67335 vmstat.system.in
523391 -97.2% 14879 stress-ng.netlink-proc.ops
8722 -97.2% 247.98 stress-ng.netlink-proc.ops_per_sec
356165 -96.9% 11052 stress-ng.time.involuntary_context_switches
1473 ± 35% -97.8% 32.75 ± 30% stress-ng.time.major_page_faults
1.372e+08 ± 2% -97.1% 3998876 stress-ng.time.minor_page_faults
5697 -97.3% 155.62 stress-ng.time.percent_of_cpu_this_job_got
2906 -97.4% 75.58 stress-ng.time.system_time
630.73 -96.6% 21.40 stress-ng.time.user_time
5488052 -96.7% 181357 stress-ng.time.voluntary_context_switches
3303 -95.8% 139.50 turbostat.Avg_MHz
92.22 -88.3 3.88 turbostat.Busy%
2067574 ± 4% +95.9% 4050376 turbostat.C1
7.97 ± 3% +89.2 97.12 turbostat.C1%
7.78 ± 3% +1136.1% 96.12 turbostat.CPU%c1
67.00 ± 3% -28.0% 48.25 ± 2% turbostat.CoreTmp
0.14 ± 5% +39.8% 0.19 ± 11% turbostat.IPC
9966000 -56.5% 4335673 turbostat.IRQ
16.96 ± 16% -17.0 0.00 turbostat.PKG_%
3149 ± 7% -95.3% 146.88 ± 14% turbostat.POLL
67.00 ± 4% -28.2% 48.12 ± 2% turbostat.PkgTmp
400.65 -39.5% 242.59 turbostat.PkgWatt
111.39 -43.2% 63.24 turbostat.RAMWatt
136245 ± 5% -70.3% 40480 meminfo.Active
136149 ± 5% -70.3% 40383 meminfo.Active(anon)
3959315 -84.8% 602536 ± 5% meminfo.AnonPages
4004177 -25.6% 2978579 meminfo.Cached
7381742 -80.9% 1408484 ± 2% meminfo.Committed_AS
4972929 -86.2% 686105 ± 4% meminfo.Inactive
4972727 -86.2% 685902 ± 4% meminfo.Inactive(anon)
19377 -20.2% 15459 meminfo.KernelStack
803826 -84.9% 121559 meminfo.Mapped
9256289 -49.0% 4723489 meminfo.Memused
38513 ± 2% -70.0% 11559 ± 2% meminfo.PageTables
192436 -13.2% 167031 meminfo.SUnreclaim
1152571 -89.0% 126979 meminfo.Shmem
290081 -9.8% 261779 meminfo.Slab
9304070 -41.1% 5479814 meminfo.max_used_kB
34041 ± 5% -70.3% 10096 proc-vmstat.nr_active_anon
990329 -84.8% 150523 ± 5% proc-vmstat.nr_anon_pages
6324920 +1.8% 6438128 proc-vmstat.nr_dirty_background_threshold
12665306 +1.8% 12891997 proc-vmstat.nr_dirty_threshold
1001066 -25.6% 744650 proc-vmstat.nr_file_pages
63626644 +1.8% 64760377 proc-vmstat.nr_free_pages
1243693 -86.2% 171359 ± 4% proc-vmstat.nr_inactive_anon
19374 -20.2% 15459 proc-vmstat.nr_kernel_stack
201019 -84.9% 30403 proc-vmstat.nr_mapped
9667 -70.1% 2889 ± 2% proc-vmstat.nr_page_table_pages
288163 -89.0% 31749 proc-vmstat.nr_shmem
24411 -3.0% 23687 proc-vmstat.nr_slab_reclaimable
48131 -13.2% 41755 proc-vmstat.nr_slab_unreclaimable
34041 ± 5% -70.3% 10096 proc-vmstat.nr_zone_active_anon
1243693 -86.2% 171359 ± 4% proc-vmstat.nr_zone_inactive_anon
1.06e+08 ± 2% -95.7% 4543130 ± 4% proc-vmstat.numa_hit
1.059e+08 ± 2% -95.8% 4476750 ± 4% proc-vmstat.numa_local
2143961 ± 11% -63.5% 782424 ± 30% proc-vmstat.numa_pte_updates
1.119e+08 ± 2% -95.7% 4763634 ± 4% proc-vmstat.pgalloc_normal
1.419e+08 ± 2% -95.7% 6148270 ± 5% proc-vmstat.pgfault
1.08e+08 ± 2% -95.6% 4734648 ± 4% proc-vmstat.pgfree
13950826 ± 4% -95.7% 599842 ± 23% proc-vmstat.pgreuse
47893 ± 11% -94.3% 2749 ± 51% numa-meminfo.node0.Active
47845 ± 11% -94.4% 2700 ± 52% numa-meminfo.node0.Active(anon)
641538 ± 30% -64.5% 227998 ± 37% numa-meminfo.node0.AnonPages
3987839 ± 48% -68.7% 1248480 ± 73% numa-meminfo.node0.AnonPages.max
957322 ± 19% -71.7% 271292 ± 27% numa-meminfo.node0.Inactive
957220 ± 19% -71.7% 271188 ± 27% numa-meminfo.node0.Inactive(anon)
10789 ± 4% -20.0% 8627 ± 5% numa-meminfo.node0.KernelStack
282050 ± 10% -79.1% 58983 ± 69% numa-meminfo.node0.Mapped
2153663 ± 44% -48.0% 1120201 ± 30% numa-meminfo.node0.MemUsed
18360 ± 8% -64.5% 6516 ± 34% numa-meminfo.node0.PageTables
107442 ± 3% -13.2% 93210 ± 4% numa-meminfo.node0.SUnreclaim
366669 ± 13% -86.6% 49210 ± 69% numa-meminfo.node0.Shmem
87484 ± 7% -56.7% 37858 ± 4% numa-meminfo.node1.Active
87436 ± 7% -56.8% 37810 ± 4% numa-meminfo.node1.Active(anon)
3291904 ± 4% -88.6% 374444 ± 23% numa-meminfo.node1.AnonPages
10488521 ± 17% -71.2% 3018784 ± 21% numa-meminfo.node1.AnonPages.max
3989075 ± 3% -89.6% 414561 ± 18% numa-meminfo.node1.Inactive
3988976 ± 3% -89.6% 414461 ± 18% numa-meminfo.node1.Inactive(anon)
8589 ± 5% -20.5% 6832 ± 5% numa-meminfo.node1.KernelStack
519017 ± 4% -87.9% 62597 ± 65% numa-meminfo.node1.Mapped
7074343 ± 12% -49.1% 3603371 ± 8% numa-meminfo.node1.MemUsed
20325 ± 5% -75.3% 5019 ± 41% numa-meminfo.node1.PageTables
85317 ± 3% -13.5% 73813 ± 5% numa-meminfo.node1.SUnreclaim
784445 ± 6% -90.1% 77811 ± 43% numa-meminfo.node1.Shmem
11963 ± 11% -94.4% 675.25 ± 52% numa-vmstat.node0.nr_active_anon
160569 ± 30% -64.5% 57015 ± 37% numa-vmstat.node0.nr_anon_pages
239486 ± 19% -71.7% 67814 ± 27% numa-vmstat.node0.nr_inactive_anon
10796 ± 4% -20.1% 8630 ± 5% numa-vmstat.node0.nr_kernel_stack
70526 ± 10% -79.1% 14749 ± 69% numa-vmstat.node0.nr_mapped
4599 ± 7% -64.6% 1629 ± 34% numa-vmstat.node0.nr_page_table_pages
91675 ± 13% -86.6% 12302 ± 69% numa-vmstat.node0.nr_shmem
26866 ± 3% -13.3% 23302 ± 4% numa-vmstat.node0.nr_slab_unreclaimable
11963 ± 11% -94.4% 675.25 ± 52% numa-vmstat.node0.nr_zone_active_anon
239486 ± 19% -71.7% 67814 ± 27% numa-vmstat.node0.nr_zone_inactive_anon
53829138 ± 2% -96.3% 1978171 ± 13% numa-vmstat.node0.numa_hit
53799034 ± 2% -96.4% 1950144 ± 13% numa-vmstat.node0.numa_local
21861 ± 7% -56.8% 9452 ± 4% numa-vmstat.node1.nr_active_anon
823304 ± 4% -88.6% 93730 ± 23% numa-vmstat.node1.nr_anon_pages
997615 ± 3% -89.6% 103743 ± 18% numa-vmstat.node1.nr_inactive_anon
8591 ± 5% -20.5% 6831 ± 5% numa-vmstat.node1.nr_kernel_stack
129802 ± 4% -87.9% 15660 ± 65% numa-vmstat.node1.nr_mapped
5113 ± 4% -75.5% 1255 ± 41% numa-vmstat.node1.nr_page_table_pages
196146 ± 6% -90.1% 19462 ± 44% numa-vmstat.node1.nr_shmem
21332 ± 3% -13.5% 18452 ± 5% numa-vmstat.node1.nr_slab_unreclaimable
21861 ± 7% -56.8% 9452 ± 4% numa-vmstat.node1.nr_zone_active_anon
997613 ± 3% -89.6% 103743 ± 18% numa-vmstat.node1.nr_zone_inactive_anon
52111878 ± 2% -95.1% 2561265 ± 7% numa-vmstat.node1.numa_hit
52086603 ± 2% -95.2% 2522913 ± 7% numa-vmstat.node1.numa_local
2056485 -99.5% 10801 ±187% sched_debug.cfs_rq:/.avg_vruntime.avg
3020107 ± 11% -98.8% 35825 ±115% sched_debug.cfs_rq:/.avg_vruntime.max
1739016 ± 5% -99.6% 7012 ±255% sched_debug.cfs_rq:/.avg_vruntime.min
186303 ± 20% -97.5% 4730 ± 61% sched_debug.cfs_rq:/.avg_vruntime.stddev
0.70 ± 4% -68.8% 0.22 ± 20% sched_debug.cfs_rq:/.h_nr_running.avg
0.51 ± 9% -20.0% 0.41 ± 12% sched_debug.cfs_rq:/.h_nr_running.stddev
45330 ± 39% -99.9% 25.47 ±183% sched_debug.cfs_rq:/.left_vruntime.avg
1935616 ± 11% -99.9% 1630 ±183% sched_debug.cfs_rq:/.left_vruntime.max
285212 ± 24% -99.9% 202.20 ±183% sched_debug.cfs_rq:/.left_vruntime.stddev
37250 ± 23% -45.8% 20171 ± 11% sched_debug.cfs_rq:/.load.avg
2.69 ± 30% -100.0% 0.00 sched_debug.cfs_rq:/.load_avg.min
2056485 -99.5% 10801 ±187% sched_debug.cfs_rq:/.min_vruntime.avg
3020107 ± 11% -98.8% 35825 ±115% sched_debug.cfs_rq:/.min_vruntime.max
1739016 ± 5% -99.6% 7012 ±255% sched_debug.cfs_rq:/.min_vruntime.min
186303 ± 20% -97.5% 4730 ± 61% sched_debug.cfs_rq:/.min_vruntime.stddev
0.61 ± 3% -63.8% 0.22 ± 20% sched_debug.cfs_rq:/.nr_running.avg
45330 ± 39% -99.9% 25.47 ±183% sched_debug.cfs_rq:/.right_vruntime.avg
1935616 ± 11% -99.9% 1630 ±183% sched_debug.cfs_rq:/.right_vruntime.max
285212 ± 24% -99.9% 202.20 ±183% sched_debug.cfs_rq:/.right_vruntime.stddev
676.13 ± 3% -30.1% 472.64 ± 17% sched_debug.cfs_rq:/.runnable_avg.avg
81.94 ± 46% -100.0% 0.00 sched_debug.cfs_rq:/.runnable_avg.min
656.01 ± 3% -28.1% 471.81 ± 17% sched_debug.cfs_rq:/.util_avg.avg
62.02 ± 11% -49.4% 31.37 ± 18% sched_debug.cfs_rq:/.util_est_enqueued.avg
443687 ± 3% +48.9% 660640 ± 8% sched_debug.cpu.avg_idle.avg
65920 -40.6% 39179 ± 25% sched_debug.cpu.clock.avg
65923 -40.6% 39182 ± 25% sched_debug.cpu.clock.max
65914 -40.6% 39175 ± 25% sched_debug.cpu.clock.min
2.23 ± 4% -29.3% 1.57 ± 10% sched_debug.cpu.clock.stddev
65182 -40.3% 38924 ± 25% sched_debug.cpu.clock_task.avg
65438 -40.2% 39150 ± 25% sched_debug.cpu.clock_task.max
58224 -45.1% 31976 ± 30% sched_debug.cpu.clock_task.min
1239645 ± 3% -99.9% 777.38 ± 23% sched_debug.cpu.curr->pid.avg
1505269 -99.4% 9486 ±164% sched_debug.cpu.curr->pid.max
568784 ± 7% -99.6% 2059 ± 86% sched_debug.cpu.curr->pid.stddev
0.00 ± 28% -59.6% 0.00 ± 29% sched_debug.cpu.next_balance.stddev
0.72 ± 4% -69.2% 0.22 ± 20% sched_debug.cpu.nr_running.avg
2.31 ± 15% -51.4% 1.12 ± 29% sched_debug.cpu.nr_running.max
0.53 ± 9% -22.8% 0.41 ± 12% sched_debug.cpu.nr_running.stddev
74166 -95.4% 3426 ± 33% sched_debug.cpu.nr_switches.avg
114148 ± 13% -57.1% 48960 ± 20% sched_debug.cpu.nr_switches.max
54294 ± 10% -99.3% 385.56 ±145% sched_debug.cpu.nr_switches.min
67.56 ± 14% -69.9% 20.31 ± 20% sched_debug.cpu.nr_uninterruptible.max
-107.75 -88.3% -12.62 sched_debug.cpu.nr_uninterruptible.min
28.72 ± 26% -80.1% 5.71 ± 10% sched_debug.cpu.nr_uninterruptible.stddev
65916 -40.6% 39177 ± 25% sched_debug.cpu_clk
64687 -41.3% 37948 ± 26% sched_debug.ktime
0.01 -100.0% 0.00 sched_debug.rt_rq:.rt_nr_running.avg
0.50 -100.0% 0.00 sched_debug.rt_rq:.rt_nr_running.max
0.06 -100.0% 0.00 sched_debug.rt_rq:.rt_nr_running.stddev
2.00 ±151% -100.0% 0.00 sched_debug.rt_rq:.rt_time.avg
127.86 ±151% -100.0% 0.00 sched_debug.rt_rq:.rt_time.max
15.86 ±151% -100.0% 0.00 sched_debug.rt_rq:.rt_time.stddev
66642 -40.2% 39871 ± 24% sched_debug.sched_clk
6.02 ± 2% -83.3% 1.00 ± 5% perf-stat.i.MPKI
1.717e+10 -94.5% 9.477e+08 perf-stat.i.branch-instructions
0.84 +0.2 1.03 perf-stat.i.branch-miss-rate%
1.359e+08 -91.8% 11159391 perf-stat.i.branch-misses
32.98 -18.3 14.67 ± 2% perf-stat.i.cache-miss-rate%
5.277e+08 ± 2% -96.9% 16350382 perf-stat.i.cache-misses
1.558e+09 -96.7% 50901066 perf-stat.i.cache-references
153850 -95.5% 6923 ± 4% perf-stat.i.context-switches
2.147e+11 -96.1% 8.312e+09 perf-stat.i.cpu-cycles
61746 -97.0% 1845 perf-stat.i.cpu-migrations
508.94 ± 5% +919.9% 5190 ± 3% perf-stat.i.cycles-between-cache-misses
0.38 ±177% -0.3 0.04 ± 9% perf-stat.i.dTLB-load-miss-rate%
6.384e+11 ±264% -100.0% 851795 ± 5% perf-stat.i.dTLB-load-misses
2.188e+10 -94.8% 1.143e+09 perf-stat.i.dTLB-loads
0.14 ± 2% -0.1 0.05 ± 8% perf-stat.i.dTLB-store-miss-rate%
15023445 -95.8% 636073 ± 5% perf-stat.i.dTLB-store-misses
1.03e+10 -94.9% 5.211e+08 perf-stat.i.dTLB-stores
8.48e+10 -94.6% 4.59e+09 perf-stat.i.instructions
0.42 +40.3% 0.59 perf-stat.i.ipc
23.20 ± 37% -97.6% 0.57 ± 27% perf-stat.i.major-faults
3.32 -96.1% 0.13 perf-stat.i.metric.GHz
601.96 +18.8% 715.37 ± 6% perf-stat.i.metric.K/sec
791.29 -94.8% 41.02 perf-stat.i.metric.M/sec
2272184 ± 2% -95.7% 97915 ± 5% perf-stat.i.minor-faults
88.83 -14.3 74.57 ± 3% perf-stat.i.node-load-miss-rate%
1.349e+08 -97.2% 3766669 ± 2% perf-stat.i.node-load-misses
16021281 ± 2% -96.5% 554627 ± 7% perf-stat.i.node-loads
51.80 -20.3 31.53 ± 31% perf-stat.i.node-store-miss-rate%
74501930 -97.3% 1983732 perf-stat.i.node-store-misses
67613840 ± 2% -96.3% 2522333 ± 2% perf-stat.i.node-stores
2272208 ± 2% -95.7% 97915 ± 5% perf-stat.i.page-faults
6.23 ± 2% -42.7% 3.57 perf-stat.overall.MPKI
0.79 +0.4 1.18 perf-stat.overall.branch-miss-rate%
33.85 -1.7 32.13 perf-stat.overall.cache-miss-rate%
2.53 -28.3% 1.81 perf-stat.overall.cpi
406.10 ± 2% +25.1% 507.98 perf-stat.overall.cycles-between-cache-misses
12.56 ±261% -12.5 0.07 ± 5% perf-stat.overall.dTLB-load-miss-rate%
0.15 ± 2% -0.0 0.12 ± 4% perf-stat.overall.dTLB-store-miss-rate%
0.40 +39.6% 0.55 perf-stat.overall.ipc
52.20 -8.1 44.08 perf-stat.overall.node-store-miss-rate%
1.661e+10 -94.4% 9.347e+08 perf-stat.ps.branch-instructions
1.31e+08 -91.6% 10997863 perf-stat.ps.branch-misses
5.108e+08 -96.8% 16156545 perf-stat.ps.cache-misses
1.509e+09 -96.7% 50284109 perf-stat.ps.cache-references
148799 -95.4% 6841 ± 4% perf-stat.ps.context-switches
2.073e+11 -96.0% 8.206e+09 perf-stat.ps.cpu-cycles
59299 -96.9% 1823 perf-stat.ps.cpu-migrations
5.189e+11 ±264% -100.0% 842481 ± 5% perf-stat.ps.dTLB-load-misses
2.117e+10 -94.7% 1.128e+09 perf-stat.ps.dTLB-loads
14645075 ± 2% -95.7% 627416 ± 5% perf-stat.ps.dTLB-store-misses
9.985e+09 -94.9% 5.14e+08 perf-stat.ps.dTLB-stores
8.198e+10 -94.5% 4.528e+09 perf-stat.ps.instructions
22.96 ± 35% -97.6% 0.56 ± 27% perf-stat.ps.major-faults
2204142 ± 2% -95.6% 96510 ± 5% perf-stat.ps.minor-faults
1.301e+08 ± 2% -97.1% 3721434 ± 2% perf-stat.ps.node-load-misses
16476921 ± 2% -96.7% 551313 ± 7% perf-stat.ps.node-loads
71602024 -97.3% 1960249 perf-stat.ps.node-store-misses
65575204 -96.2% 2487796 ± 2% perf-stat.ps.node-stores
2204165 ± 2% -95.6% 96510 ± 5% perf-stat.ps.page-faults
5.108e+12 ± 2% -94.5% 2.825e+11 perf-stat.total.instructions
0.05 ± 47% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.__get_free_pages.pgd_alloc
0.04 ± 22% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.__pmd_alloc.copy_p4d_range
0.04 ± 39% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.__pud_alloc.copy_p4d_range
0.04 ± 18% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.pte_alloc_one.__pte_alloc
0.04 ± 20% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.pte_alloc_one.do_read_fault
0.03 ± 10% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.vma_alloc_folio.wp_page_copy
0.05 ± 48% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.__kmem_cache_alloc_node.__kmalloc.security_prepare_creds.prepare_creds
0.04 ± 39% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.__kmem_cache_alloc_node.__kmalloc.security_task_alloc.copy_process
0.06 ± 17% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.__kmem_cache_alloc_node.__kmalloc_node.__vmalloc_area_node.__vmalloc_node_range
0.04 ± 38% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.__kmem_cache_alloc_node.__kmalloc_node.memcg_alloc_slab_cgroups.allocate_slab
0.05 ± 3% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.__put_anon_vma.unlink_anon_vmas.free_pgtables.exit_mmap
0.04 ± 26% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.__vmalloc_area_node.__vmalloc_node_range.alloc_thread_stack_node.dup_task_struct
0.00 ± 37% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.__wait_for_common.affine_move_task.__set_cpus_allowed_ptr.__sched_setaffinity
0.07 ± 73% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.aa_sk_perm.security_socket_recvmsg.sock_recvmsg.__sys_recvfrom
0.04 ± 45% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.cgroup_css_set_fork.cgroup_can_fork.copy_process.kernel_clone
0.04 ± 8% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.copy_page_range.dup_mmap.dup_mm.constprop
0.04 ± 14% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.copy_pte_range.copy_p4d_range.copy_page_range.dup_mmap
0.04 ± 16% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.down_read.acct_collect.do_exit.do_group_exit
0.07 ± 32% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.down_read.exit_mm.do_exit.do_group_exit
0.08 ±103% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.down_read.exit_mmap.__mmput.exit_mm
0.05 ± 40% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.down_write.__put_anon_vma.unlink_anon_vmas.free_pgtables
0.04 ± 11% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.down_write.anon_vma_clone.anon_vma_fork.dup_mmap
0.04 ± 14% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.down_write.anon_vma_fork.dup_mmap.dup_mm
0.04 ± 8% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.down_write.dup_mmap.dup_mm.constprop
0.04 ± 10% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.down_write.dup_userfaultfd.dup_mmap.dup_mm
0.06 ± 3% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.down_write.free_pgtables.exit_mmap.__mmput
0.05 ± 11% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.down_write.unlink_anon_vmas.free_pgtables.exit_mmap
0.05 ± 6% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.down_write.unlink_file_vma.free_pgtables.exit_mmap
0.05 ± 28% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.dput.path_put.exit_fs.do_exit
0.05 ± 5% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.exit_mmap.__mmput.exit_mm.do_exit
0.05 ± 38% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.exit_signals.do_exit.do_group_exit.__x64_sys_exit_group
0.04 ± 43% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.kmem_cache_alloc.alloc_pid.copy_process.kernel_clone
0.04 ± 4% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.kmem_cache_alloc.anon_vma_fork.dup_mmap.dup_mm
0.04 ± 35% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.kmem_cache_alloc.copy_fs_struct.copy_process.kernel_clone
0.04 ± 43% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.kmem_cache_alloc.copy_sighand.copy_process.kernel_clone
0.05 ± 30% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.kmem_cache_alloc.copy_signal.copy_process.kernel_clone
0.02 ± 57% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.kmem_cache_alloc.dup_fd.copy_process.kernel_clone
0.04 ± 30% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.kmem_cache_alloc.dup_mm.constprop.0
0.04 ± 67% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.kmem_cache_alloc.mas_alloc_nodes.mas_expected_entries.dup_mmap
0.05 ± 38% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.kmem_cache_alloc.prepare_creds.copy_creds.copy_process
0.04 ± 10% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.kmem_cache_alloc.vm_area_dup.dup_mmap.dup_mm
0.03 ± 49% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.kmem_cache_alloc_node.dup_task_struct.copy_process.kernel_clone
0.05 ± 52% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.mmput.exit_mm.do_exit.do_group_exit
0.05 ± 16% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.mutex_lock.futex_exit_release.exit_mm_release.exit_mm
0.06 ± 64% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.mutex_lock.ldt_dup_context.dup_mmap.dup_mm
0.04 ± 23% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.mutex_lock_killable.pcpu_alloc.__percpu_counter_init_many.mm_init
0.04 ± 60% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.mutex_lock_killable.pcpu_alloc.mm_init.dup_mm
0.05 ± 18% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.put_files_struct.do_exit.do_group_exit.__x64_sys_exit_group
0.05 ± 18% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.remove_vma.exit_mmap.__mmput.exit_mm
0.03 ± 63% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.stop_one_cpu.sched_exec.bprm_execve.part
0.04 ± 50% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.switch_task_namespaces.do_exit.do_group_exit.__x64_sys_exit_group
0.05 ± 4% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.tlb_batch_pages_flush.tlb_finish_mmu.exit_mmap.__mmput
0.06 ± 33% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.unmap_page_range.unmap_vmas.exit_mmap.__mmput
0.06 ± 30% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.unmap_vmas.exit_mmap.__mmput.exit_mm
0.04 ± 21% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.uprobe_start_dup_mmap.dup_mmap.dup_mm.constprop
0.18 ± 89% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.vfree.part.0.delayed_vfree_work
0.04 ± 15% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.wp_page_copy.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
0.05 ± 7% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.zap_pmd_range.isra.0.unmap_page_range
0.05 ± 36% -95.4% 0.00 ± 19% perf-sched.sch_delay.avg.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
0.11 ± 96% -91.6% 0.01 ± 16% perf-sched.sch_delay.avg.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
0.08 ± 38% -97.2% 0.00 ± 41% perf-sched.sch_delay.avg.ms.do_nanosleep.hrtimer_nanosleep.common_nsleep.__x64_sys_clock_nanosleep
0.04 -93.9% 0.00 ± 18% perf-sched.sch_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
0.04 -88.8% 0.00 ± 10% perf-sched.sch_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
0.04 ± 2% -100.0% 0.00 perf-sched.sch_delay.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault
0.05 ± 5% -96.8% 0.00 ±100% perf-sched.sch_delay.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt
0.04 ± 60% -100.0% 0.00 perf-sched.sch_delay.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single
0.03 ± 57% -100.0% 0.00 perf-sched.sch_delay.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi
0.04 ± 3% -86.0% 0.01 ± 10% perf-sched.sch_delay.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64
0.06 ± 6% -100.0% 0.00 perf-sched.sch_delay.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode.ret_from_fork_asm
0.04 ± 4% -100.0% 0.00 perf-sched.sch_delay.avg.ms.io_schedule.folio_wait_bit_common.filemap_fault.__do_fault
0.06 ± 9% -95.2% 0.00 ± 10% perf-sched.sch_delay.avg.ms.pipe_read.vfs_read.ksys_read.do_syscall_64
0.05 ±115% -90.1% 0.00 ± 19% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.do_select.core_sys_select.kern_select
0.05 ± 31% -93.9% 0.00 perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
0.02 ± 9% -100.0% 0.00 perf-sched.sch_delay.avg.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.__put_anon_vma
0.02 ± 5% -100.0% 0.00 perf-sched.sch_delay.avg.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.anon_vma_clone
0.02 ± 8% -100.0% 0.00 perf-sched.sch_delay.avg.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.anon_vma_fork
0.04 ± 15% -100.0% 0.00 perf-sched.sch_delay.avg.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.dup_mmap
0.03 ± 5% -100.0% 0.00 perf-sched.sch_delay.avg.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.unlink_anon_vmas
0.05 ± 16% -100.0% 0.00 perf-sched.sch_delay.avg.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.unlink_file_vma
0.13 ± 11% -97.7% 0.00 perf-sched.sch_delay.avg.ms.schedule_timeout.__wait_for_common.wait_for_completion_state.kernel_clone
0.03 ± 28% -90.7% 0.00 ± 11% perf-sched.sch_delay.avg.ms.schedule_timeout.kcompactd.kthread.ret_from_fork
0.03 ± 7% -88.5% 0.00 perf-sched.sch_delay.avg.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
0.02 ± 9% -81.8% 0.00 perf-sched.sch_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork.ret_from_fork_asm
0.06 ± 70% -88.5% 0.01 ± 9% perf-sched.sch_delay.avg.ms.syslog_print.do_syslog.kmsg_read.vfs_read
0.03 ± 27% -93.4% 0.00 perf-sched.sch_delay.avg.ms.wait_for_partner.fifo_open.do_dentry_open.do_open
0.04 ± 2% -74.8% 0.01 ± 22% perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork.ret_from_fork_asm
0.24 ± 70% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.__get_free_pages.pgd_alloc
0.35 ± 40% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.__pmd_alloc.copy_p4d_range
0.23 ± 76% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.__pud_alloc.copy_p4d_range
0.34 ± 34% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.pte_alloc_one.__pte_alloc
0.40 ± 26% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.pte_alloc_one.do_read_fault
0.48 ± 18% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.vma_alloc_folio.wp_page_copy
0.26 ± 64% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.__kmem_cache_alloc_node.__kmalloc.security_prepare_creds.prepare_creds
0.19 ± 51% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.__kmem_cache_alloc_node.__kmalloc.security_task_alloc.copy_process
0.65 ± 27% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.__kmem_cache_alloc_node.__kmalloc_node.__vmalloc_area_node.__vmalloc_node_range
0.13 ± 48% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.__kmem_cache_alloc_node.__kmalloc_node.memcg_alloc_slab_cgroups.allocate_slab
3.03 ± 62% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.__put_anon_vma.unlink_anon_vmas.free_pgtables.exit_mmap
0.26 ± 49% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.__vmalloc_area_node.__vmalloc_node_range.alloc_thread_stack_node.dup_task_struct
0.28 ± 20% -98.2% 0.01 ± 22% perf-sched.sch_delay.max.ms.__cond_resched.__wait_for_common.affine_move_task.__set_cpus_allowed_ptr.__sched_setaffinity
0.15 ± 70% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.aa_sk_perm.security_socket_recvmsg.sock_recvmsg.__sys_recvfrom
0.35 ± 57% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.cgroup_css_set_fork.cgroup_can_fork.copy_process.kernel_clone
0.60 ± 21% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.copy_page_range.dup_mmap.dup_mm.constprop
2.27 ±201% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.copy_pte_range.copy_p4d_range.copy_page_range.dup_mmap
0.23 ± 41% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.down_read.acct_collect.do_exit.do_group_exit
0.21 ± 41% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.down_read.exit_mm.do_exit.do_group_exit
0.21 ±103% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.down_read.exit_mmap.__mmput.exit_mm
0.24 ± 57% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.down_write.__put_anon_vma.unlink_anon_vmas.free_pgtables
0.49 ± 17% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.down_write.anon_vma_clone.anon_vma_fork.dup_mmap
0.59 ± 28% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.down_write.anon_vma_fork.dup_mmap.dup_mm
2.52 ±181% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.down_write.dup_mmap.dup_mm.constprop
0.55 ± 21% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.down_write.dup_userfaultfd.dup_mmap.dup_mm
3.06 ± 66% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.down_write.free_pgtables.exit_mmap.__mmput
0.44 ± 21% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.down_write.unlink_anon_vmas.free_pgtables.exit_mmap
1.35 ±116% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.down_write.unlink_file_vma.free_pgtables.exit_mmap
0.25 ± 49% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.dput.path_put.exit_fs.do_exit
2.21 ± 95% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.exit_mmap.__mmput.exit_mm.do_exit
0.18 ± 49% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.exit_signals.do_exit.do_group_exit.__x64_sys_exit_group
0.36 ± 59% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.kmem_cache_alloc.alloc_pid.copy_process.kernel_clone
4.20 ± 78% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.kmem_cache_alloc.anon_vma_fork.dup_mmap.dup_mm
0.26 ± 44% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.kmem_cache_alloc.copy_fs_struct.copy_process.kernel_clone
0.15 ± 47% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.kmem_cache_alloc.copy_sighand.copy_process.kernel_clone
0.27 ± 56% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.kmem_cache_alloc.copy_signal.copy_process.kernel_clone
0.05 ± 81% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.kmem_cache_alloc.dup_fd.copy_process.kernel_clone
0.27 ± 39% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.kmem_cache_alloc.dup_mm.constprop.0
0.15 ± 82% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.kmem_cache_alloc.mas_alloc_nodes.mas_expected_entries.dup_mmap
1.38 ±165% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.kmem_cache_alloc.prepare_creds.copy_creds.copy_process
1.46 ±151% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.kmem_cache_alloc.vm_area_dup.dup_mmap.dup_mm
0.13 ± 69% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.kmem_cache_alloc_node.dup_task_struct.copy_process.kernel_clone
0.15 ± 95% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.mmput.exit_mm.do_exit.do_group_exit
0.42 ± 22% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.mutex_lock.futex_exit_release.exit_mm_release.exit_mm
0.12 ± 66% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.mutex_lock.ldt_dup_context.dup_mmap.dup_mm
0.36 ± 43% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.mutex_lock_killable.pcpu_alloc.__percpu_counter_init_many.mm_init
0.31 ± 57% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.mutex_lock_killable.pcpu_alloc.mm_init.dup_mm
0.38 ± 36% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.put_files_struct.do_exit.do_group_exit.__x64_sys_exit_group
0.42 ± 27% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.remove_vma.exit_mmap.__mmput.exit_mm
0.93 ± 79% -96.2% 0.04 ±249% perf-sched.sch_delay.max.ms.__cond_resched.smpboot_thread_fn.kthread.ret_from_fork.ret_from_fork_asm
0.19 ± 78% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.stop_one_cpu.sched_exec.bprm_execve.part
0.16 ± 73% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.switch_task_namespaces.do_exit.do_group_exit.__x64_sys_exit_group
1.22 ± 84% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.tlb_batch_pages_flush.tlb_finish_mmu.exit_mmap.__mmput
0.23 ± 51% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.unmap_page_range.unmap_vmas.exit_mmap.__mmput
0.38 ± 40% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.unmap_vmas.exit_mmap.__mmput.exit_mm
0.30 ± 25% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.uprobe_start_dup_mmap.dup_mmap.dup_mm.constprop
0.41 ±108% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.vfree.part.0.delayed_vfree_work
0.43 ± 18% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.wp_page_copy.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
1.98 ±121% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.zap_pmd_range.isra.0.unmap_page_range
0.11 ± 29% -96.1% 0.00 ± 8% perf-sched.sch_delay.max.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
0.15 ± 78% -90.8% 0.01 ± 22% perf-sched.sch_delay.max.ms.devkmsg_read.vfs_read.ksys_read.do_syscall_64
0.27 ± 34% -98.6% 0.00 ± 70% perf-sched.sch_delay.max.ms.do_nanosleep.hrtimer_nanosleep.common_nsleep.__x64_sys_clock_nanosleep
7.28 ± 5% -99.9% 0.01 ± 33% perf-sched.sch_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
21.86 ± 17% -100.0% 0.01 ± 36% perf-sched.sch_delay.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
1.03 ± 75% -100.0% 0.00 perf-sched.sch_delay.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault
0.70 ± 28% -99.8% 0.00 ±100% perf-sched.sch_delay.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt
0.17 ± 54% -100.0% 0.00 perf-sched.sch_delay.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single
0.12 ± 73% -100.0% 0.00 perf-sched.sch_delay.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi
2.45 ±109% -99.8% 0.01 ± 5% perf-sched.sch_delay.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode.do_syscall_64
0.59 ± 15% -100.0% 0.00 perf-sched.sch_delay.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode.ret_from_fork_asm
1.78 ± 95% -100.0% 0.00 perf-sched.sch_delay.max.ms.io_schedule.folio_wait_bit_common.filemap_fault.__do_fault
0.50 ± 25% -97.8% 0.01 ± 36% perf-sched.sch_delay.max.ms.pipe_read.vfs_read.ksys_read.do_syscall_64
0.31 ± 30% -98.7% 0.00 ± 8% perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.do_poll.constprop.0.do_sys_poll
0.19 ±127% -96.4% 0.01 ± 9% perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.do_select.core_sys_select.kern_select
0.22 ± 37% -97.3% 0.01 ± 16% perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
0.58 ± 10% -100.0% 0.00 perf-sched.sch_delay.max.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.__put_anon_vma
0.81 ± 46% -100.0% 0.00 perf-sched.sch_delay.max.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.anon_vma_clone
1.67 ±137% -100.0% 0.00 perf-sched.sch_delay.max.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.anon_vma_fork
0.41 ± 22% -100.0% 0.00 perf-sched.sch_delay.max.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.dup_mmap
1.58 ± 81% -100.0% 0.00 perf-sched.sch_delay.max.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.unlink_anon_vmas
0.44 ± 21% -100.0% 0.00 perf-sched.sch_delay.max.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.unlink_file_vma
0.69 ± 19% -99.2% 0.01 ± 50% perf-sched.sch_delay.max.ms.schedule_timeout.__wait_for_common.wait_for_completion_state.kernel_clone
0.17 ± 69% -97.2% 0.00 ± 16% perf-sched.sch_delay.max.ms.schedule_timeout.kcompactd.kthread.ret_from_fork
5.14 ± 46% -99.9% 0.01 ± 13% perf-sched.sch_delay.max.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
8.69 ± 31% -99.8% 0.01 ± 24% perf-sched.sch_delay.max.ms.smpboot_thread_fn.kthread.ret_from_fork.ret_from_fork_asm
0.12 ± 79% -93.3% 0.01 ± 9% perf-sched.sch_delay.max.ms.syslog_print.do_syslog.kmsg_read.vfs_read
0.41 ± 41% -99.3% 0.00 perf-sched.sch_delay.max.ms.wait_for_partner.fifo_open.do_dentry_open.do_open
6.23 ± 5% -63.3% 2.29 ± 37% perf-sched.sch_delay.max.ms.worker_thread.kthread.ret_from_fork.ret_from_fork_asm
0.04 -90.8% 0.00 ± 11% perf-sched.total_sch_delay.average.ms
21.86 ± 17% -88.1% 2.60 perf-sched.total_sch_delay.max.ms
3.06 +9335.7% 288.87 ± 22% perf-sched.total_wait_and_delay.average.ms
626656 -99.6% 2692 ± 31% perf-sched.total_wait_and_delay.count.ms
2751 ± 21% +76.7% 4862 ± 6% perf-sched.total_wait_and_delay.max.ms
3.02 +9463.7% 288.87 ± 22% perf-sched.total_wait_time.average.ms
2751 ± 21% +76.7% 4862 ± 6% perf-sched.total_wait_time.max.ms
40.01 ± 77% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.shmem_alloc_folio.shmem_alloc_and_add_folio
2.44 -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.__cond_resched.__put_anon_vma.unlink_anon_vmas.free_pgtables.exit_mmap
10.75 +107.4% 22.29 ± 16% perf-sched.wait_and_delay.avg.ms.__cond_resched.__wait_for_common.affine_move_task.__set_cpus_allowed_ptr.__sched_setaffinity
6.66 ± 2% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.__cond_resched.aa_sk_perm.security_socket_recvmsg.sock_recvmsg.__sys_recvfrom
42.16 ± 28% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.__cond_resched.generic_perform_write.shmem_file_write_iter.vfs_write.ksys_write
1.02 ± 4% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.__cond_resched.kmem_cache_alloc.anon_vma_fork.dup_mmap.dup_mm
9.70 ± 59% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.__cond_resched.mutex_lock.perf_poll.do_poll.constprop
59.21 ± 39% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.__cond_resched.shmem_inode_acct_blocks.shmem_alloc_and_add_folio.shmem_get_folio_gfp.shmem_write_begin
16.63 ±100% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.__cond_resched.vfree.part.0.delayed_vfree_work
2.04 +11100.8% 228.05 ± 3% perf-sched.wait_and_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
1.12 -83.8% 0.18 perf-sched.wait_and_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
41.82 ± 6% +1322.9% 595.14 ± 8% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.do_poll.constprop.0.do_sys_poll
3.38 +1567.7% 56.37 ± 10% perf-sched.wait_and_delay.avg.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
23.77 +2352.6% 582.95 ± 5% perf-sched.wait_and_delay.avg.ms.smpboot_thread_fn.kthread.ret_from_fork.ret_from_fork_asm
9.18 +5985.7% 558.74 perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork.ret_from_fork_asm
10.25 ± 35% -100.0% 0.00 perf-sched.wait_and_delay.count.__cond_resched.__alloc_pages.alloc_pages_mpol.shmem_alloc_folio.shmem_alloc_and_add_folio
3416 ± 3% -100.0% 0.00 perf-sched.wait_and_delay.count.__cond_resched.__put_anon_vma.unlink_anon_vmas.free_pgtables.exit_mmap
4.75 ± 25% -100.0% 0.00 perf-sched.wait_and_delay.count.__cond_resched.aa_sk_perm.security_socket_recvmsg.sock_recvmsg.__sys_recvfrom
30.12 ± 9% -100.0% 0.00 perf-sched.wait_and_delay.count.__cond_resched.generic_perform_write.shmem_file_write_iter.vfs_write.ksys_write
3292 ± 2% -100.0% 0.00 perf-sched.wait_and_delay.count.__cond_resched.kmem_cache_alloc.anon_vma_fork.dup_mmap.dup_mm
10.62 ± 24% -100.0% 0.00 perf-sched.wait_and_delay.count.__cond_resched.mutex_lock.perf_poll.do_poll.constprop
9.50 ± 30% -100.0% 0.00 perf-sched.wait_and_delay.count.__cond_resched.shmem_inode_acct_blocks.shmem_alloc_and_add_folio.shmem_get_folio_gfp.shmem_write_begin
1476 -99.9% 1.00 ±180% perf-sched.wait_and_delay.count.__cond_resched.smpboot_thread_fn.kthread.ret_from_fork.ret_from_fork_asm
4.38 ± 42% -100.0% 0.00 perf-sched.wait_and_delay.count.__cond_resched.vfree.part.0.delayed_vfree_work
262169 -100.0% 126.00 perf-sched.wait_and_delay.count.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
262761 -100.0% 124.00 perf-sched.wait_and_delay.count.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
184.38 ± 9% +24.3% 229.12 ± 3% perf-sched.wait_and_delay.count.pipe_read.vfs_read.ksys_read.do_syscall_64
326.38 ± 6% -93.6% 21.00 ± 2% perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.do_poll.constprop.0.do_sys_poll
1484 -94.2% 86.50 ± 10% perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
24201 -97.8% 543.88 ± 8% perf-sched.wait_and_delay.count.smpboot_thread_fn.kthread.ret_from_fork.ret_from_fork_asm
35356 -98.7% 469.25 perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork.ret_from_fork_asm
215.48 ± 67% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.shmem_alloc_folio.shmem_alloc_and_add_folio
10.87 ± 11% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.__cond_resched.__put_anon_vma.unlink_anon_vmas.free_pgtables.exit_mmap
7.12 ± 3% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.__cond_resched.aa_sk_perm.security_socket_recvmsg.sock_recvmsg.__sys_recvfrom
380.54 ± 14% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.__cond_resched.generic_perform_write.shmem_file_write_iter.vfs_write.ksys_write
14.75 ± 31% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.__cond_resched.kmem_cache_alloc.anon_vma_fork.dup_mmap.dup_mm
69.86 ± 78% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.__cond_resched.mutex_lock.perf_poll.do_poll.constprop
323.54 ± 37% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.__cond_resched.shmem_inode_acct_blocks.shmem_alloc_and_add_folio.shmem_get_folio_gfp.shmem_write_begin
31.91 ±101% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.__cond_resched.vfree.part.0.delayed_vfree_work
152.31 ±207% -99.6% 0.58 ± 5% perf-sched.wait_and_delay.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
10.17 ± 16% +3806.7% 397.25 ± 16% perf-sched.wait_and_delay.max.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
1.66 ± 47% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.__get_free_pages.pgd_alloc
0.96 ± 40% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.__pmd_alloc.copy_p4d_range
1.39 ± 18% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.__pud_alloc.copy_p4d_range
1.25 ± 32% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.pte_alloc_one.__pte_alloc
40.01 ± 77% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.shmem_alloc_folio.shmem_alloc_and_add_folio
0.33 ± 26% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.vma_alloc_folio.wp_page_copy
1.54 ± 41% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.__kmem_cache_alloc_node.__kmalloc.security_prepare_creds.prepare_creds
1.22 ± 24% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.__kmem_cache_alloc_node.__kmalloc.security_task_alloc.copy_process
0.75 ± 17% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.__kmem_cache_alloc_node.__kmalloc_node.__vmalloc_area_node.__vmalloc_node_range
2.39 ± 2% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.__put_anon_vma.unlink_anon_vmas.free_pgtables.exit_mmap
0.65 ± 46% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.__vmalloc_area_node.__vmalloc_node_range.alloc_thread_stack_node.dup_task_struct
10.74 +107.5% 22.29 ± 16% perf-sched.wait_time.avg.ms.__cond_resched.__wait_for_common.affine_move_task.__set_cpus_allowed_ptr.__sched_setaffinity
6.59 ± 2% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.aa_sk_perm.security_socket_recvmsg.sock_recvmsg.__sys_recvfrom
1.42 ± 42% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.cgroup_css_set_fork.cgroup_can_fork.copy_process.kernel_clone
1.18 ± 5% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.copy_page_range.dup_mmap.dup_mm.constprop
1.21 ± 8% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.copy_pte_range.copy_p4d_range.copy_page_range.dup_mmap
2.56 ± 12% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.down_read.acct_collect.do_exit.do_group_exit
3.08 ± 27% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.down_read.exit_mm.do_exit.do_group_exit
2.39 ± 55% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.down_read.exit_mmap.__mmput.exit_mm
2.88 ± 17% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.down_write.__put_anon_vma.unlink_anon_vmas.free_pgtables
1.33 ± 13% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.down_write.anon_vma_clone.anon_vma_fork.dup_mmap
1.22 ± 11% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.down_write.anon_vma_fork.dup_mmap.dup_mm
1.19 ± 4% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.down_write.dup_mmap.dup_mm.constprop
1.30 ± 7% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.down_write.dup_userfaultfd.dup_mmap.dup_mm
2.90 -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.down_write.free_pgtables.exit_mmap.__mmput
2.76 ± 3% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.down_write.unlink_anon_vmas.free_pgtables.exit_mmap
2.66 ± 3% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.down_write.unlink_file_vma.free_pgtables.exit_mmap
2.39 ± 11% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.dput.path_put.exit_fs.do_exit
2.68 ± 2% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.exit_mmap.__mmput.exit_mm.do_exit
2.43 ± 19% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.exit_signals.do_exit.do_group_exit.__x64_sys_exit_group
42.16 ± 28% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.generic_perform_write.shmem_file_write_iter.vfs_write.ksys_write
1.00 ± 22% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.kmem_cache_alloc.alloc_pid.copy_process.kernel_clone
0.98 ± 4% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.kmem_cache_alloc.anon_vma_fork.dup_mmap.dup_mm
1.68 ± 30% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.kmem_cache_alloc.copy_signal.copy_process.kernel_clone
1.22 ± 35% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.kmem_cache_alloc.dup_mm.constprop.0
1.35 ± 15% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.kmem_cache_alloc.prepare_creds.copy_creds.copy_process
1.45 ± 8% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.kmem_cache_alloc.vm_area_dup.dup_mmap.dup_mm
2.82 ± 19% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.mmput.exit_mm.do_exit.do_group_exit
2.69 ± 7% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.mutex_lock.futex_exit_release.exit_mm_release.exit_mm
2.10 ± 37% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.mutex_lock.perf_event_exit_task.do_exit.do_group_exit
9.70 ± 59% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.mutex_lock.perf_poll.do_poll.constprop
1.13 ± 15% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.mutex_lock_killable.pcpu_alloc.__percpu_counter_init_many.mm_init
1.18 ± 36% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.mutex_lock_killable.pcpu_alloc.mm_init.dup_mm
2.68 ± 9% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.put_files_struct.do_exit.do_group_exit.__x64_sys_exit_group
2.67 ± 7% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.remove_vma.exit_mmap.__mmput.exit_mm
4.48 ± 86% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.shmem_get_folio_gfp.shmem_write_begin.generic_perform_write.shmem_file_write_iter
59.21 ± 39% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.shmem_inode_acct_blocks.shmem_alloc_and_add_folio.shmem_get_folio_gfp.shmem_write_begin
2.12 ± 19% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.switch_task_namespaces.do_exit.do_group_exit.__x64_sys_exit_group
2.60 -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.tlb_batch_pages_flush.tlb_finish_mmu.exit_mmap.__mmput
2.83 ± 15% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.unmap_page_range.unmap_vmas.exit_mmap.__mmput
2.65 ± 16% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.unmap_vmas.exit_mmap.__mmput.exit_mm
1.31 ± 26% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.uprobe_start_dup_mmap.dup_mmap.dup_mm.constprop
16.45 ±101% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.vfree.part.0.delayed_vfree_work
0.65 ± 16% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.wp_page_copy.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
2.65 -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.zap_pmd_range.isra.0.unmap_page_range
1.99 +11342.4% 228.05 ± 3% perf-sched.wait_time.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
1.08 -83.6% 0.18 perf-sched.wait_time.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
0.98 ± 7% -100.0% 0.00 perf-sched.wait_time.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault
2.36 ± 45% -100.0% 0.00 perf-sched.wait_time.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt
1.09 ± 53% -100.0% 0.00 perf-sched.wait_time.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single
26.12 ±251% -100.0% 0.00 perf-sched.wait_time.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi
0.77 ± 21% -100.0% 0.00 perf-sched.wait_time.avg.ms.io_schedule.folio_wait_bit_common.filemap_fault.__do_fault
41.82 ± 6% +1323.1% 595.13 ± 8% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.do_poll.constprop.0.do_sys_poll
0.95 ± 8% -100.0% 0.00 perf-sched.wait_time.avg.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.__put_anon_vma
0.55 ± 4% -100.0% 0.00 perf-sched.wait_time.avg.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.anon_vma_clone
0.45 ± 8% -100.0% 0.00 perf-sched.wait_time.avg.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.anon_vma_fork
1.08 ± 18% -100.0% 0.00 perf-sched.wait_time.avg.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.dup_mmap
0.98 ± 3% -100.0% 0.00 perf-sched.wait_time.avg.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.unlink_anon_vmas
2.24 ± 16% -100.0% 0.00 perf-sched.wait_time.avg.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.unlink_file_vma
3.35 +1580.6% 56.36 ± 10% perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
23.75 +2354.3% 582.95 ± 5% perf-sched.wait_time.avg.ms.smpboot_thread_fn.kthread.ret_from_fork.ret_from_fork_asm
9.71 ±184% -100.0% 0.00 perf-sched.wait_time.avg.ms.wait_for_partner.fifo_open.do_dentry_open.do_open
9.15 +6009.0% 558.73 perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork.ret_from_fork_asm
6.99 ± 4% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.__get_free_pages.pgd_alloc
7.12 ± 5% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.__pmd_alloc.copy_p4d_range
7.11 ± 4% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.__pud_alloc.copy_p4d_range
8.05 ± 13% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.pte_alloc_one.__pte_alloc
215.48 ± 67% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.shmem_alloc_folio.shmem_alloc_and_add_folio
7.11 ± 3% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.vma_alloc_folio.wp_page_copy
7.78 ± 23% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.__kmem_cache_alloc_node.__kmalloc.security_prepare_creds.prepare_creds
7.07 ± 5% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.__kmem_cache_alloc_node.__kmalloc.security_task_alloc.copy_process
7.49 ± 3% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.__kmem_cache_alloc_node.__kmalloc_node.__vmalloc_area_node.__vmalloc_node_range
10.83 ± 11% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.__put_anon_vma.unlink_anon_vmas.free_pgtables.exit_mmap
6.92 ± 7% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.__vmalloc_area_node.__vmalloc_node_range.alloc_thread_stack_node.dup_task_struct
7.03 ± 3% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.aa_sk_perm.security_socket_recvmsg.sock_recvmsg.__sys_recvfrom
7.28 ± 2% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.cgroup_css_set_fork.cgroup_can_fork.copy_process.kernel_clone
9.35 ± 23% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.copy_page_range.dup_mmap.dup_mm.constprop
12.37 ± 31% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.copy_pte_range.copy_p4d_range.copy_page_range.dup_mmap
5.89 ± 2% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.down_read.acct_collect.do_exit.do_group_exit
5.89 ± 10% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.down_read.exit_mm.do_exit.do_group_exit
4.13 ± 46% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.down_read.exit_mmap.__mmput.exit_mm
5.70 ± 4% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.down_write.__put_anon_vma.unlink_anon_vmas.free_pgtables
8.83 ± 18% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.down_write.anon_vma_clone.anon_vma_fork.dup_mmap
9.49 ± 20% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.down_write.anon_vma_fork.dup_mmap.dup_mm
11.97 ± 11% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.down_write.dup_mmap.dup_mm.constprop
9.67 ± 25% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.down_write.dup_userfaultfd.dup_mmap.dup_mm
9.52 ± 15% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.down_write.free_pgtables.exit_mmap.__mmput
7.08 ± 20% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.down_write.unlink_anon_vmas.free_pgtables.exit_mmap
9.06 ± 16% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.down_write.unlink_file_vma.free_pgtables.exit_mmap
5.84 ± 13% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.dput.path_put.exit_fs.do_exit
9.00 ± 17% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.exit_mmap.__mmput.exit_mm.do_exit
5.53 ± 8% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.exit_signals.do_exit.do_group_exit.__x64_sys_exit_group
380.54 ± 14% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.generic_perform_write.shmem_file_write_iter.vfs_write.ksys_write
7.79 ± 12% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.kmem_cache_alloc.alloc_pid.copy_process.kernel_clone
12.82 ± 13% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.kmem_cache_alloc.anon_vma_fork.dup_mmap.dup_mm
7.23 ± 3% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.kmem_cache_alloc.copy_signal.copy_process.kernel_clone
7.51 ± 15% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.kmem_cache_alloc.dup_mm.constprop.0
8.70 ± 26% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.kmem_cache_alloc.prepare_creds.copy_creds.copy_process
12.10 ± 15% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.kmem_cache_alloc.vm_area_dup.dup_mmap.dup_mm
4.92 ± 14% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.mmput.exit_mm.do_exit.do_group_exit
6.97 ± 24% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.mutex_lock.futex_exit_release.exit_mm_release.exit_mm
3.76 ± 41% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.mutex_lock.perf_event_exit_task.do_exit.do_group_exit
69.86 ± 78% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.mutex_lock.perf_poll.do_poll.constprop
7.75 ± 11% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.mutex_lock_killable.pcpu_alloc.__percpu_counter_init_many.mm_init
7.30 ± 3% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.mutex_lock_killable.pcpu_alloc.mm_init.dup_mm
7.47 ± 27% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.put_files_struct.do_exit.do_group_exit.__x64_sys_exit_group
7.00 ± 15% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.remove_vma.exit_mmap.__mmput.exit_mm
10.40 ± 92% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.shmem_get_folio_gfp.shmem_write_begin.generic_perform_write.shmem_file_write_iter
323.54 ± 37% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.shmem_inode_acct_blocks.shmem_alloc_and_add_folio.shmem_get_folio_gfp.shmem_write_begin
5.53 ± 7% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.switch_task_namespaces.do_exit.do_group_exit.__x64_sys_exit_group
9.69 ± 17% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.tlb_batch_pages_flush.tlb_finish_mmu.exit_mmap.__mmput
6.15 ± 11% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.unmap_page_range.unmap_vmas.exit_mmap.__mmput
6.15 ± 5% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.unmap_vmas.exit_mmap.__mmput.exit_mm
7.41 ± 3% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.uprobe_start_dup_mmap.dup_mmap.dup_mm.constprop
31.69 ±102% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.vfree.part.0.delayed_vfree_work
7.84 ± 17% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.wp_page_copy.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
10.98 ± 9% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.zap_pmd_range.isra.0.unmap_page_range
141.35 ±226% -99.6% 0.57 ± 5% perf-sched.wait_time.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
9.99 ± 13% -100.0% 0.00 perf-sched.wait_time.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault
451.92 ± 98% -100.0% 0.00 perf-sched.wait_time.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt
5.83 ± 18% -100.0% 0.00 perf-sched.wait_time.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_call_function_single
67.34 ±242% -100.0% 0.00 perf-sched.wait_time.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_reschedule_ipi
10.42 ± 15% -100.0% 0.00 perf-sched.wait_time.max.ms.io_schedule.folio_wait_bit_common.filemap_fault.__do_fault
6.85 ± 12% -100.0% 0.00 perf-sched.wait_time.max.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.__put_anon_vma
9.06 ± 16% -100.0% 0.00 perf-sched.wait_time.max.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.anon_vma_clone
9.65 ± 22% -100.0% 0.00 perf-sched.wait_time.max.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.anon_vma_fork
7.49 ± 6% -100.0% 0.00 perf-sched.wait_time.max.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.dup_mmap
8.39 ± 14% -100.0% 0.00 perf-sched.wait_time.max.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.unlink_anon_vmas
6.08 ± 4% -100.0% 0.00 perf-sched.wait_time.max.ms.schedule_preempt_disabled.rwsem_down_write_slowpath.down_write.unlink_file_vma
8.09 ± 17% +4813.0% 397.25 ± 16% perf-sched.wait_time.max.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
250.19 ±173% -100.0% 0.00 ±174% perf-sched.wait_time.max.ms.wait_for_partner.fifo_open.do_dentry_open.do_open
46.38 -46.4 0.00 perf-profile.calltrace.cycles-pp.__libc_fork
44.83 -44.8 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_fork
44.83 -44.8 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork
44.82 -44.8 0.00 perf-profile.calltrace.cycles-pp.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork
44.82 -44.8 0.00 perf-profile.calltrace.cycles-pp.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork
44.19 -44.2 0.00 perf-profile.calltrace.cycles-pp.copy_process.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe
40.80 -40.8 0.00 perf-profile.calltrace.cycles-pp.dup_mm.copy_process.kernel_clone.__do_sys_clone.do_syscall_64
40.08 -40.1 0.00 perf-profile.calltrace.cycles-pp.dup_mmap.dup_mm.copy_process.kernel_clone.__do_sys_clone
35.47 -35.5 0.00 perf-profile.calltrace.cycles-pp.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
35.47 -35.5 0.00 perf-profile.calltrace.cycles-pp.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
35.47 -35.5 0.00 perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
35.52 -34.9 0.59 ± 39% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
35.52 -34.9 0.60 ± 39% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
32.63 -32.6 0.00 perf-profile.calltrace.cycles-pp.exit_mm.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
32.62 -32.6 0.00 perf-profile.calltrace.cycles-pp.__mmput.exit_mm.do_exit.do_group_exit.__x64_sys_exit_group
32.56 -32.6 0.00 perf-profile.calltrace.cycles-pp.exit_mmap.__mmput.exit_mm.do_exit.do_group_exit
26.62 -26.6 0.00 perf-profile.calltrace.cycles-pp.anon_vma_fork.dup_mmap.dup_mm.copy_process.kernel_clone
19.32 -19.3 0.00 perf-profile.calltrace.cycles-pp.free_pgtables.exit_mmap.__mmput.exit_mm.do_exit
16.07 -16.1 0.00 perf-profile.calltrace.cycles-pp.anon_vma_clone.anon_vma_fork.dup_mmap.dup_mm.copy_process
15.11 -15.1 0.00 perf-profile.calltrace.cycles-pp.unlink_anon_vmas.free_pgtables.exit_mmap.__mmput.exit_mm
8.92 -8.9 0.00 perf-profile.calltrace.cycles-pp.down_write.anon_vma_fork.dup_mmap.dup_mm.copy_process
8.76 -8.8 0.00 perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.down_write.anon_vma_fork.dup_mmap.dup_mm
8.64 -8.6 0.00 perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.down_write.anon_vma_fork.dup_mmap
7.09 -7.1 0.00 perf-profile.calltrace.cycles-pp.down_write.anon_vma_clone.anon_vma_fork.dup_mmap.dup_mm
6.80 ± 2% -6.8 0.00 perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.__mmput.exit_mm.do_exit
6.71 -6.7 0.00 perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.down_write.anon_vma_clone.anon_vma_fork.dup_mmap
6.58 -6.6 0.00 perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.down_write.anon_vma_clone.anon_vma_fork
6.53 -6.5 0.00 perf-profile.calltrace.cycles-pp.down_write.unlink_anon_vmas.free_pgtables.exit_mmap.__mmput
6.41 ± 2% -6.4 0.00 perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.__mmput.exit_mm
6.21 ± 2% -6.2 0.00 perf-profile.calltrace.cycles-pp.zap_pmd_range.unmap_page_range.unmap_vmas.exit_mmap.__mmput
6.15 -6.1 0.00 perf-profile.calltrace.cycles-pp.rwsem_down_write_slowpath.down_write.unlink_anon_vmas.free_pgtables.exit_mmap
6.02 -6.0 0.00 perf-profile.calltrace.cycles-pp.rwsem_optimistic_spin.rwsem_down_write_slowpath.down_write.unlink_anon_vmas.free_pgtables
5.87 ± 2% -5.9 0.00 perf-profile.calltrace.cycles-pp.zap_pte_range.zap_pmd_range.unmap_page_range.unmap_vmas.exit_mmap
5.62 ± 2% -5.6 0.00 perf-profile.calltrace.cycles-pp.asm_exc_page_fault
5.55 -5.6 0.00 perf-profile.calltrace.cycles-pp.osq_lock.rwsem_optimistic_spin.rwsem_down_write_slowpath.down_write.anon_vma_fork
5.25 ± 2% -5.3 0.00 perf-profile.calltrace.cycles-pp.exc_page_fault.asm_exc_page_fault
5.23 ± 2% -5.2 0.00 perf-profile.calltrace.cycles-pp.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
5.22 ± 2% -5.2 0.00 perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
0.00 +0.7 0.71 ± 16% perf-profile.calltrace.cycles-pp._raw_spin_trylock.rebalance_domains.__do_softirq.irq_exit_rcu.sysvec_apic_timer_interrupt
0.00 +0.7 0.72 ± 15% perf-profile.calltrace.cycles-pp.update_blocked_averages.run_rebalance_domains.__do_softirq.irq_exit_rcu.sysvec_apic_timer_interrupt
0.00 +0.8 0.77 ± 16% perf-profile.calltrace.cycles-pp.run_rebalance_domains.__do_softirq.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.00 +0.8 0.80 ± 16% perf-profile.calltrace.cycles-pp.update_sg_lb_stats.update_sd_lb_stats.find_busiest_group.load_balance.rebalance_domains
0.00 +1.0 0.99 ± 7% perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call.do_idle
0.00 +1.1 1.07 ± 14% perf-profile.calltrace.cycles-pp.update_sd_lb_stats.find_busiest_group.load_balance.rebalance_domains.__do_softirq
0.00 +1.1 1.07 ± 17% perf-profile.calltrace.cycles-pp.__intel_pmu_enable_all.perf_rotate_context.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt
0.00 +1.1 1.13 ± 14% perf-profile.calltrace.cycles-pp.find_busiest_group.load_balance.rebalance_domains.__do_softirq.irq_exit_rcu
0.00 +1.2 1.16 ± 11% perf-profile.calltrace.cycles-pp.lapic_next_deadline.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
0.00 +1.3 1.33 ± 21% perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.rest_init
0.00 +1.4 1.39 ± 21% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.rest_init.arch_call_rest_init
0.00 +1.4 1.40 ± 14% perf-profile.calltrace.cycles-pp.perf_rotate_context.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
0.00 +1.4 1.41 ± 22% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.rest_init.arch_call_rest_init.start_kernel
0.00 +1.4 1.41 ± 22% perf-profile.calltrace.cycles-pp.cpu_startup_entry.rest_init.arch_call_rest_init.start_kernel.x86_64_start_reservations
0.00 +1.4 1.41 ± 22% perf-profile.calltrace.cycles-pp.x86_64_start_kernel.secondary_startup_64_no_verify
0.00 +1.4 1.41 ± 22% perf-profile.calltrace.cycles-pp.x86_64_start_reservations.x86_64_start_kernel.secondary_startup_64_no_verify
0.00 +1.4 1.41 ± 22% perf-profile.calltrace.cycles-pp.start_kernel.x86_64_start_reservations.x86_64_start_kernel.secondary_startup_64_no_verify
0.00 +1.4 1.41 ± 22% perf-profile.calltrace.cycles-pp.arch_call_rest_init.start_kernel.x86_64_start_reservations.x86_64_start_kernel.secondary_startup_64_no_verify
0.00 +1.4 1.41 ± 22% perf-profile.calltrace.cycles-pp.rest_init.arch_call_rest_init.start_kernel.x86_64_start_reservations.x86_64_start_kernel
0.00 +1.6 1.57 ± 10% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry
0.00 +1.6 1.61 ± 10% perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.00 +1.9 1.89 ± 10% perf-profile.calltrace.cycles-pp.load_balance.rebalance_domains.__do_softirq.irq_exit_rcu.sysvec_apic_timer_interrupt
0.00 +2.2 2.17 ± 10% perf-profile.calltrace.cycles-pp.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
0.00 +2.2 2.21 ± 13% perf-profile.calltrace.cycles-pp.arch_scale_freq_tick.scheduler_tick.update_process_times.tick_sched_handle.tick_nohz_highres_handler
0.00 +2.5 2.52 ± 43% perf-profile.calltrace.cycles-pp.ktime_get_update_offsets_now.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.00 +2.8 2.78 ± 6% perf-profile.calltrace.cycles-pp.rebalance_domains.__do_softirq.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.00 +2.8 2.85 ± 76% perf-profile.calltrace.cycles-pp.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.acpi_safe_halt
0.00 +3.0 3.02 ± 6% perf-profile.calltrace.cycles-pp.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
0.00 +3.2 3.16 ± 67% perf-profile.calltrace.cycles-pp.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.acpi_safe_halt.acpi_idle_enter
0.00 +4.2 4.20 ± 5% perf-profile.calltrace.cycles-pp.__do_softirq.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.acpi_safe_halt
0.00 +5.0 4.99 ± 5% perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.acpi_safe_halt.acpi_idle_enter
0.00 +7.6 7.64 ± 7% perf-profile.calltrace.cycles-pp.__intel_pmu_enable_all.perf_adjust_freq_unthr_context.perf_event_task_tick.scheduler_tick.update_process_times
0.00 +11.5 11.48 ± 6% perf-profile.calltrace.cycles-pp.perf_adjust_freq_unthr_context.perf_event_task_tick.scheduler_tick.update_process_times.tick_sched_handle
0.00 +11.9 11.86 ± 6% perf-profile.calltrace.cycles-pp.perf_event_task_tick.scheduler_tick.update_process_times.tick_sched_handle.tick_nohz_highres_handler
0.00 +15.9 15.85 ± 5% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_nohz_highres_handler.__hrtimer_run_queues
0.00 +17.3 17.29 ± 4% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_nohz_highres_handler.__hrtimer_run_queues.hrtimer_interrupt
0.00 +17.5 17.47 ± 4% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_nohz_highres_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
0.00 +19.6 19.59 ± 3% perf-profile.calltrace.cycles-pp.tick_nohz_highres_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
0.00 +23.2 23.21 ± 3% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.00 +28.5 28.50 ± 5% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.acpi_safe_halt
0.00 +29.4 29.43 ± 4% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.acpi_safe_halt.acpi_idle_enter
0.00 +39.8 39.78 ± 7% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.acpi_safe_halt.acpi_idle_enter.cpuidle_enter_state
0.00 +48.8 48.81 ± 5% perf-profile.calltrace.cycles-pp.acpi_safe_halt.acpi_idle_enter.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
0.00 +89.3 89.33 perf-profile.calltrace.cycles-pp.acpi_idle_enter.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
0.00 +89.8 89.79 perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
0.00 +90.6 90.64 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
0.00 +94.0 93.97 perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
0.00 +94.8 94.76 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
0.00 +95.0 94.97 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
0.00 +95.0 94.97 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
0.00 +96.4 96.38 perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
0.00 +123.5 123.53 perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.acpi_safe_halt.acpi_idle_enter.cpuidle_enter_state.cpuidle_enter
86.80 -84.6 2.24 ± 10% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
86.79 -84.6 2.23 ± 10% perf-profile.children.cycles-pp.do_syscall_64
46.50 -46.4 0.12 ± 45% perf-profile.children.cycles-pp.__libc_fork
44.86 -44.8 0.09 ± 62% perf-profile.children.cycles-pp.__do_sys_clone
44.86 -44.7 0.14 ± 43% perf-profile.children.cycles-pp.kernel_clone
44.23 -44.1 0.11 ± 51% perf-profile.children.cycles-pp.copy_process
40.83 -40.8 0.06 ± 78% perf-profile.children.cycles-pp.dup_mm
40.17 -40.1 0.04 ±112% perf-profile.children.cycles-pp.dup_mmap
35.94 -35.7 0.23 ± 28% perf-profile.children.cycles-pp.__x64_sys_exit_group
35.93 -35.7 0.23 ± 28% perf-profile.children.cycles-pp.do_group_exit
35.92 -35.7 0.23 ± 28% perf-profile.children.cycles-pp.do_exit
32.67 -32.5 0.14 ± 31% perf-profile.children.cycles-pp.exit_mm
32.62 -32.4 0.19 ± 22% perf-profile.children.cycles-pp.__mmput
32.57 -32.4 0.19 ± 22% perf-profile.children.cycles-pp.exit_mmap
28.02 -28.0 0.01 ±264% perf-profile.children.cycles-pp.down_write
26.66 -26.7 0.00 perf-profile.children.cycles-pp.anon_vma_fork
24.98 -25.0 0.00 perf-profile.children.cycles-pp.rwsem_down_write_slowpath
24.41 -24.4 0.00 perf-profile.children.cycles-pp.rwsem_optimistic_spin
19.34 -19.3 0.03 ±108% perf-profile.children.cycles-pp.free_pgtables
16.10 -16.1 0.00 perf-profile.children.cycles-pp.anon_vma_clone
15.30 -15.3 0.00 perf-profile.children.cycles-pp.osq_lock
15.13 -15.1 0.00 perf-profile.children.cycles-pp.unlink_anon_vmas
10.03 ± 2% -9.4 0.59 ± 13% perf-profile.children.cycles-pp.asm_exc_page_fault
9.09 ± 2% -8.6 0.49 ± 14% perf-profile.children.cycles-pp.exc_page_fault
9.06 ± 2% -8.6 0.49 ± 13% perf-profile.children.cycles-pp.do_user_addr_fault
7.89 -7.9 0.00 perf-profile.children.cycles-pp.rwsem_spin_on_owner
8.12 ± 2% -7.7 0.43 ± 16% perf-profile.children.cycles-pp.handle_mm_fault
7.79 ± 2% -7.4 0.40 ± 15% perf-profile.children.cycles-pp.__handle_mm_fault
6.81 ± 2% -6.7 0.08 ± 51% perf-profile.children.cycles-pp.unmap_vmas
6.40 -6.4 0.00 perf-profile.children.cycles-pp.netlink_broadcast_filtered
6.42 ± 2% -6.3 0.07 ± 62% perf-profile.children.cycles-pp.unmap_page_range
6.26 -6.3 0.00 perf-profile.children.cycles-pp.do_one_broadcast
6.23 ± 2% -6.2 0.07 ± 63% perf-profile.children.cycles-pp.zap_pmd_range
6.34 ± 3% -6.1 0.24 ± 27% perf-profile.children.cycles-pp.do_read_fault
6.36 ± 3% -6.1 0.27 ± 22% perf-profile.children.cycles-pp.do_fault
6.07 ± 2% -6.0 0.07 ± 63% perf-profile.children.cycles-pp.zap_pte_range
6.07 ± 3% -5.9 0.20 ± 33% perf-profile.children.cycles-pp.filemap_map_pages
4.30 -4.2 0.07 ± 86% perf-profile.children.cycles-pp.kmem_cache_alloc
3.45 ± 2% -3.4 0.03 ±102% perf-profile.children.cycles-pp.tlb_finish_mmu
3.33 ± 4% -3.2 0.10 ± 39% perf-profile.children.cycles-pp.next_uptodate_folio
2.45 -2.4 0.04 ± 83% perf-profile.children.cycles-pp.kmem_cache_free
2.40 -2.4 0.04 ± 86% perf-profile.children.cycles-pp.memcg_slab_post_alloc_hook
1.70 -1.5 0.15 ± 45% perf-profile.children.cycles-pp.__schedule
1.32 ± 3% -1.3 0.06 ± 63% perf-profile.children.cycles-pp.alloc_pages_mpol
1.27 ± 3% -1.2 0.06 ± 79% perf-profile.children.cycles-pp.__alloc_pages
1.16 ± 4% -1.1 0.04 ± 83% perf-profile.children.cycles-pp.set_pte_range
1.03 ± 2% -0.9 0.10 ± 46% perf-profile.children.cycles-pp.schedule
1.18 ± 28% -0.9 0.30 ± 32% perf-profile.children.cycles-pp.__cmd_record
0.91 ± 2% -0.9 0.06 ± 83% perf-profile.children.cycles-pp.newidle_balance
0.77 ± 3% -0.7 0.03 ±104% perf-profile.children.cycles-pp.get_page_from_freelist
0.92 -0.5 0.37 ± 15% perf-profile.children.cycles-pp.ret_from_fork_asm
0.88 -0.5 0.37 ± 15% perf-profile.children.cycles-pp.ret_from_fork
0.54 -0.5 0.04 ±102% perf-profile.children.cycles-pp.__cond_resched
0.46 -0.4 0.06 ± 62% perf-profile.children.cycles-pp.rcu_do_batch
0.40 ± 3% -0.4 0.05 ± 41% perf-profile.children.cycles-pp.sync_regs
0.34 ± 2% -0.3 0.08 ± 61% perf-profile.children.cycles-pp.copy_mc_enhanced_fast_string
1.35 ± 2% -0.3 1.10 ± 16% perf-profile.children.cycles-pp._raw_spin_lock
0.28 -0.2 0.03 ±102% perf-profile.children.cycles-pp.update_load_avg
0.48 ± 2% -0.2 0.27 ± 24% perf-profile.children.cycles-pp.rcu_core
0.30 ± 3% -0.2 0.10 ± 52% perf-profile.children.cycles-pp.worker_thread
0.24 ± 3% -0.2 0.04 ± 78% perf-profile.children.cycles-pp.perf_iterate_sb
0.28 ± 3% -0.2 0.08 ± 44% perf-profile.children.cycles-pp.process_one_work
0.22 ± 18% -0.2 0.04 ± 79% perf-profile.children.cycles-pp.__do_sys_newstat
0.20 ± 18% -0.2 0.03 ±102% perf-profile.children.cycles-pp.vfs_statx
0.18 ± 18% -0.1 0.04 ± 83% perf-profile.children.cycles-pp.filename_lookup
0.18 ± 18% -0.1 0.04 ± 83% perf-profile.children.cycles-pp.path_lookupat
0.11 ± 4% -0.1 0.03 ± 79% perf-profile.children.cycles-pp._find_next_bit
0.00 +0.1 0.08 ± 28% perf-profile.children.cycles-pp.evsel__read_counter
0.00 +0.1 0.08 ± 34% perf-profile.children.cycles-pp.__bitmap_and
0.00 +0.1 0.08 ± 22% perf-profile.children.cycles-pp.__mmap
0.00 +0.1 0.09 ± 31% perf-profile.children.cycles-pp.__open64_nocancel
0.00 +0.1 0.10 ± 42% perf-profile.children.cycles-pp._dl_addr
0.01 ±264% +0.1 0.10 ± 34% perf-profile.children.cycles-pp.rb_insert_color
0.00 +0.1 0.10 ± 22% perf-profile.children.cycles-pp.setlocale
0.12 ± 3% +0.1 0.22 ± 17% perf-profile.children.cycles-pp.asm_sysvec_call_function_single
0.00 +0.1 0.11 ± 41% perf-profile.children.cycles-pp.do_vmi_align_munmap
0.00 +0.1 0.11 ± 30% perf-profile.children.cycles-pp.elf_load
0.00 +0.1 0.11 ± 28% perf-profile.children.cycles-pp.tick_nohz_stop_idle
0.00 +0.1 0.11 ± 21% perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
0.00 +0.1 0.12 ± 27% perf-profile.children.cycles-pp.fault_in_readable
0.00 +0.1 0.12 ± 27% perf-profile.children.cycles-pp.fault_in_iov_iter_readable
0.00 +0.1 0.12 ± 48% perf-profile.children.cycles-pp.note_gp_changes
0.00 +0.1 0.12 ± 41% perf-profile.children.cycles-pp.do_vmi_munmap
0.00 +0.1 0.12 ± 16% perf-profile.children.cycles-pp.arch_cpu_idle_enter
0.00 +0.1 0.13 ± 38% perf-profile.children.cycles-pp.run_posix_cpu_timers
0.00 +0.1 0.13 ± 33% perf-profile.children.cycles-pp.check_cpu_stall
0.00 +0.1 0.13 ± 39% perf-profile.children.cycles-pp.cpuidle_reflect
0.00 +0.1 0.14 ± 35% perf-profile.children.cycles-pp.menu_reflect
0.00 +0.1 0.14 ± 20% perf-profile.children.cycles-pp.rb_erase
0.00 +0.1 0.15 ± 15% perf-profile.children.cycles-pp.tick_program_event
0.00 +0.1 0.15 ± 33% perf-profile.children.cycles-pp.ct_kernel_exit
0.00 +0.2 0.16 ± 22% perf-profile.children.cycles-pp.hrtimer_forward
0.00 +0.2 0.17 ± 32% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.00 +0.2 0.19 ± 25% perf-profile.children.cycles-pp.ksys_mmap_pgoff
0.00 +0.2 0.20 ± 40% perf-profile.children.cycles-pp.read_counters
0.00 +0.2 0.20 ± 17% perf-profile.children.cycles-pp.path_openat
0.00 +0.2 0.20 ± 31% perf-profile.children.cycles-pp.irqtime_account_process_tick
0.00 +0.2 0.20 ± 23% perf-profile.children.cycles-pp.do_filp_open
0.13 ± 3% +0.2 0.34 ± 16% perf-profile.children.cycles-pp._raw_spin_lock_irq
0.00 +0.2 0.21 ± 23% perf-profile.children.cycles-pp.arch_cpu_idle_exit
0.00 +0.2 0.21 ± 39% perf-profile.children.cycles-pp.cmd_stat
0.00 +0.2 0.21 ± 39% perf-profile.children.cycles-pp.dispatch_events
0.00 +0.2 0.21 ± 39% perf-profile.children.cycles-pp.process_interval
0.04 ± 37% +0.2 0.26 ± 27% perf-profile.children.cycles-pp.error_entry
0.00 +0.2 0.22 ± 26% perf-profile.children.cycles-pp.irqentry_exit
0.00 +0.2 0.23 ± 18% perf-profile.children.cycles-pp.hrtimer_update_next_event
0.00 +0.2 0.23 ± 64% perf-profile.children.cycles-pp.trigger_load_balance
0.00 +0.2 0.24 ± 16% perf-profile.children.cycles-pp.__x64_sys_openat
0.00 +0.2 0.24 ± 16% perf-profile.children.cycles-pp.do_sys_openat2
0.00 +0.2 0.24 ± 26% perf-profile.children.cycles-pp.calc_global_load_tick
0.00 +0.2 0.24 ± 15% perf-profile.children.cycles-pp.__update_blocked_fair
0.00 +0.2 0.24 ± 22% perf-profile.children.cycles-pp.read
0.00 +0.2 0.24 ± 32% perf-profile.children.cycles-pp.ct_kernel_enter
0.00 +0.2 0.25 ± 20% perf-profile.children.cycles-pp.mmap_region
0.00 +0.3 0.25 ± 20% perf-profile.children.cycles-pp.do_mmap
0.00 +0.3 0.26 ± 18% perf-profile.children.cycles-pp.timerqueue_del
0.00 +0.3 0.26 ± 21% perf-profile.children.cycles-pp.vm_mmap_pgoff
0.00 +0.3 0.27 ± 13% perf-profile.children.cycles-pp.ct_nmi_enter
0.00 +0.3 0.28 ± 16% perf-profile.children.cycles-pp.vfs_read
0.00 +0.3 0.28 ± 14% perf-profile.children.cycles-pp.call_cpuidle
0.00 +0.3 0.28 ± 13% perf-profile.children.cycles-pp.idle_cpu
0.00 +0.3 0.28 ± 22% perf-profile.children.cycles-pp.hrtimer_get_next_event
0.00 +0.3 0.29 ± 35% perf-profile.children.cycles-pp.record__pushfn
0.00 +0.3 0.29 ± 35% perf-profile.children.cycles-pp.writen
0.00 +0.3 0.29 ± 35% perf-profile.children.cycles-pp.shmem_file_write_iter
0.00 +0.3 0.29 ± 35% perf-profile.children.cycles-pp.generic_perform_write
0.00 +0.3 0.29 ± 27% perf-profile.children.cycles-pp.tick_check_broadcast_expired
0.00 +0.3 0.29 ± 27% perf-profile.children.cycles-pp.load_elf_binary
0.00 +0.3 0.29 ± 34% perf-profile.children.cycles-pp.__libc_write
0.01 ±264% +0.3 0.30 ± 32% perf-profile.children.cycles-pp.cmd_record
0.00 +0.3 0.29 ± 34% perf-profile.children.cycles-pp.record__mmap_read_evlist
0.00 +0.3 0.29 ± 34% perf-profile.children.cycles-pp.perf_mmap__push
0.00 +0.3 0.29 ± 14% perf-profile.children.cycles-pp.ksys_read
0.00 +0.3 0.29 ± 33% perf-profile.children.cycles-pp.local_clock_noinstr
0.00 +0.3 0.30 ± 27% perf-profile.children.cycles-pp.exec_binprm
0.00 +0.3 0.30 ± 27% perf-profile.children.cycles-pp.search_binary_handler
0.02 ±129% +0.3 0.31 ± 22% perf-profile.children.cycles-pp._find_next_and_bit
0.00 +0.3 0.30 ± 31% perf-profile.children.cycles-pp.ct_idle_exit
0.00 +0.3 0.30 ± 37% perf-profile.children.cycles-pp.timekeeping_advance
0.00 +0.3 0.31 ± 38% perf-profile.children.cycles-pp.update_wall_time
0.00 +0.3 0.32 ± 22% perf-profile.children.cycles-pp.bprm_execve
0.00 +0.3 0.34 ± 17% perf-profile.children.cycles-pp.tick_nohz_irq_exit
0.00 +0.3 0.35 ± 16% perf-profile.children.cycles-pp.__hrtimer_next_event_base
0.00 +0.4 0.35 ± 29% perf-profile.children.cycles-pp.get_cpu_device
0.00 +0.4 0.35 ± 20% perf-profile.children.cycles-pp.x86_pmu_disable
0.00 +0.4 0.36 ± 31% perf-profile.children.cycles-pp.vfs_write
0.00 +0.4 0.36 ± 25% perf-profile.children.cycles-pp.timerqueue_add
0.00 +0.4 0.36 ± 30% perf-profile.children.cycles-pp.ksys_write
0.00 +0.4 0.37 ± 25% perf-profile.children.cycles-pp.rcu_pending
0.00 +0.4 0.37 ± 8% perf-profile.children.cycles-pp.irqentry_enter
0.50 ± 3% +0.4 0.91 ± 5% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.00 +0.4 0.42 ± 14% perf-profile.children.cycles-pp.hrtimer_next_event_without
0.00 +0.4 0.45 ± 23% perf-profile.children.cycles-pp.enqueue_hrtimer
0.00 +0.5 0.46 ± 22% perf-profile.children.cycles-pp.should_we_balance
0.00 +0.5 0.46 ± 21% perf-profile.children.cycles-pp.update_irq_load_avg
0.00 +0.5 0.46 ± 22% perf-profile.children.cycles-pp.execve
0.00 +0.5 0.46 ± 17% perf-profile.children.cycles-pp.ct_kernel_exit_state
0.47 ± 2% +0.5 0.94 ± 15% perf-profile.children.cycles-pp.update_sg_lb_stats
0.00 +0.5 0.47 ± 23% perf-profile.children.cycles-pp.__x64_sys_execve
0.00 +0.5 0.47 ± 23% perf-profile.children.cycles-pp.do_execveat_common
0.03 ±100% +0.5 0.51 ± 20% perf-profile.children.cycles-pp.__libc_start_main
0.03 ±100% +0.5 0.51 ± 20% perf-profile.children.cycles-pp.main
0.03 ±100% +0.5 0.51 ± 20% perf-profile.children.cycles-pp.run_builtin
0.00 +0.5 0.48 ± 23% perf-profile.children.cycles-pp.tick_check_oneshot_broadcast_this_cpu
0.00 +0.5 0.50 ± 15% perf-profile.children.cycles-pp.perf_pmu_nop_void
0.00 +0.5 0.51 ± 15% perf-profile.children.cycles-pp.update_rq_clock
0.00 +0.5 0.52 ± 16% perf-profile.children.cycles-pp.rcu_sched_clock_irq
0.00 +0.5 0.55 ± 19% perf-profile.children.cycles-pp.cpuidle_governor_latency_req
0.00 +0.6 0.56 ± 11% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.03 ± 77% +0.6 0.60 ± 18% perf-profile.children.cycles-pp.update_rq_clock_task
0.10 ± 4% +0.6 0.75 ± 14% perf-profile.children.cycles-pp._raw_spin_trylock
0.52 ± 2% +0.7 1.17 ± 14% perf-profile.children.cycles-pp.update_sd_lb_stats
0.09 ± 5% +0.7 0.74 ± 17% perf-profile.children.cycles-pp.update_blocked_averages
0.53 ± 2% +0.7 1.22 ± 14% perf-profile.children.cycles-pp.find_busiest_group
0.02 ±100% +0.8 0.80 ± 16% perf-profile.children.cycles-pp.run_rebalance_domains
0.00 +0.8 0.80 ± 15% perf-profile.children.cycles-pp.native_apic_msr_eoi
0.00 +1.0 1.01 ± 8% perf-profile.children.cycles-pp.tick_nohz_next_event
0.00 +1.0 1.03 ± 14% perf-profile.children.cycles-pp.irqtime_account_irq
0.00 +1.1 1.08 ± 52% perf-profile.children.cycles-pp.tick_sched_do_timer
0.00 +1.1 1.13 ± 7% perf-profile.children.cycles-pp.sched_clock
0.00 +1.2 1.16 ± 9% perf-profile.children.cycles-pp.native_sched_clock
0.84 ± 2% +1.2 2.00 ± 10% perf-profile.children.cycles-pp.load_balance
0.36 ± 2% +1.2 1.55 ± 11% perf-profile.children.cycles-pp.native_irq_return_iret
0.00 +1.2 1.22 ± 11% perf-profile.children.cycles-pp.lapic_next_deadline
0.00 +1.3 1.29 ± 9% perf-profile.children.cycles-pp.sched_clock_cpu
0.00 +1.4 1.41 ± 22% perf-profile.children.cycles-pp.x86_64_start_kernel
0.00 +1.4 1.41 ± 22% perf-profile.children.cycles-pp.x86_64_start_reservations
0.00 +1.4 1.41 ± 22% perf-profile.children.cycles-pp.start_kernel
0.00 +1.4 1.41 ± 22% perf-profile.children.cycles-pp.arch_call_rest_init
0.00 +1.4 1.41 ± 22% perf-profile.children.cycles-pp.rest_init
0.00 +1.4 1.44 ± 8% perf-profile.children.cycles-pp.read_tsc
0.00 +1.4 1.45 ± 13% perf-profile.children.cycles-pp.perf_rotate_context
0.00 +1.6 1.64 ± 10% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.00 +1.7 1.66 ± 10% perf-profile.children.cycles-pp.clockevents_program_event
0.00 +2.2 2.24 ± 9% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
0.00 +2.3 2.26 ± 14% perf-profile.children.cycles-pp.arch_scale_freq_tick
0.00 +2.6 2.55 ± 43% perf-profile.children.cycles-pp.ktime_get_update_offsets_now
0.02 ±100% +2.8 2.86 ± 6% perf-profile.children.cycles-pp.rebalance_domains
0.00 +2.9 2.93 ± 75% perf-profile.children.cycles-pp.tick_irq_enter
0.00 +3.1 3.11 ± 7% perf-profile.children.cycles-pp.menu_select
0.00 +3.2 3.22 ± 67% perf-profile.children.cycles-pp.irq_enter_rcu
0.59 +3.7 4.34 ± 6% perf-profile.children.cycles-pp.__do_softirq
0.00 +4.0 4.03 ± 59% perf-profile.children.cycles-pp.ktime_get
0.49 ± 2% +4.7 5.14 ± 5% perf-profile.children.cycles-pp.irq_exit_rcu
0.06 ± 5% +8.9 8.94 ± 6% perf-profile.children.cycles-pp.__intel_pmu_enable_all
0.20 ± 3% +11.8 12.04 ± 6% perf-profile.children.cycles-pp.perf_adjust_freq_unthr_context
0.20 ± 3% +11.9 12.11 ± 6% perf-profile.children.cycles-pp.perf_event_task_tick
0.36 +15.9 16.21 ± 5% perf-profile.children.cycles-pp.scheduler_tick
0.43 +17.2 17.64 ± 5% perf-profile.children.cycles-pp.update_process_times
0.43 +17.4 17.80 ± 5% perf-profile.children.cycles-pp.tick_sched_handle
0.45 +19.6 20.00 ± 3% perf-profile.children.cycles-pp.tick_nohz_highres_handler
0.50 +23.2 23.65 ± 3% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.54 ± 2% +28.3 28.87 ± 4% perf-profile.children.cycles-pp.hrtimer_interrupt
0.55 ± 2% +29.2 29.79 ± 4% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
1.05 +38.8 39.90 ± 7% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
1.08 +81.2 82.33 perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.18 ± 5% +89.0 89.23 perf-profile.children.cycles-pp.acpi_safe_halt
0.18 ± 6% +89.2 89.40 perf-profile.children.cycles-pp.acpi_idle_enter
0.20 ± 4% +90.5 90.70 perf-profile.children.cycles-pp.cpuidle_enter_state
0.20 ± 4% +91.0 91.15 perf-profile.children.cycles-pp.cpuidle_enter
0.34 ± 5% +94.6 94.97 perf-profile.children.cycles-pp.start_secondary
0.24 ± 5% +95.3 95.51 perf-profile.children.cycles-pp.cpuidle_idle_call
0.35 ± 5% +96.0 96.38 perf-profile.children.cycles-pp.secondary_startup_64_no_verify
0.35 ± 5% +96.0 96.38 perf-profile.children.cycles-pp.cpu_startup_entry
0.35 ± 5% +96.0 96.38 perf-profile.children.cycles-pp.do_idle
15.13 -15.1 0.00 perf-profile.self.cycles-pp.osq_lock
7.78 -7.8 0.00 perf-profile.self.cycles-pp.rwsem_spin_on_owner
3.16 ± 4% -3.1 0.09 ± 39% perf-profile.self.cycles-pp.next_uptodate_folio
1.49 ± 3% -1.4 0.04 ±104% perf-profile.self.cycles-pp.filemap_map_pages
0.40 ± 3% -0.4 0.05 ± 41% perf-profile.self.cycles-pp.sync_regs
0.34 ± 3% -0.3 0.08 ± 61% perf-profile.self.cycles-pp.copy_mc_enhanced_fast_string
0.08 ± 5% -0.0 0.03 ± 79% perf-profile.self.cycles-pp._find_next_bit
0.00 +0.1 0.08 ± 34% perf-profile.self.cycles-pp.__bitmap_and
0.00 +0.1 0.09 ± 21% perf-profile.self.cycles-pp.tsc_verify_tsc_adjust
0.00 +0.1 0.09 ± 31% perf-profile.self.cycles-pp.rb_insert_color
0.00 +0.1 0.11 ± 38% perf-profile.self.cycles-pp.enqueue_hrtimer
0.00 +0.1 0.11 ± 29% perf-profile.self.cycles-pp.rb_erase
0.00 +0.1 0.11 ± 38% perf-profile.self.cycles-pp.check_cpu_stall
0.00 +0.1 0.12 ± 42% perf-profile.self.cycles-pp.sched_clock
0.00 +0.1 0.12 ± 38% perf-profile.self.cycles-pp.__sysvec_apic_timer_interrupt
0.00 +0.1 0.12 ± 37% perf-profile.self.cycles-pp.run_posix_cpu_timers
0.00 +0.1 0.13 ± 19% perf-profile.self.cycles-pp.__update_blocked_fair
0.00 +0.1 0.13 ± 31% perf-profile.self.cycles-pp.ct_kernel_enter
0.00 +0.1 0.13 ± 36% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.00 +0.1 0.13 ± 33% perf-profile.self.cycles-pp.rcu_sched_clock_irq
0.00 +0.1 0.13 ± 32% perf-profile.self.cycles-pp.get_next_timer_interrupt
0.00 +0.1 0.13 ± 19% perf-profile.self.cycles-pp.tick_program_event
0.00 +0.1 0.13 ± 23% perf-profile.self.cycles-pp.hrtimer_forward
0.00 +0.1 0.13 ± 23% perf-profile.self.cycles-pp.tick_nohz_highres_handler
0.00 +0.1 0.14 ± 40% perf-profile.self.cycles-pp.acpi_idle_enter
0.00 +0.1 0.14 ± 33% perf-profile.self.cycles-pp.clockevents_program_event
0.00 +0.1 0.14 ± 23% perf-profile.self.cycles-pp.irq_exit_rcu
0.00 +0.1 0.14 ± 40% perf-profile.self.cycles-pp.tick_nohz_get_sleep_length
0.00 +0.2 0.16 ± 17% perf-profile.self.cycles-pp.ct_nmi_enter
0.00 +0.2 0.16 ± 32% perf-profile.self.cycles-pp.rebalance_domains
0.00 +0.2 0.16 ± 39% perf-profile.self.cycles-pp.sched_clock_cpu
0.00 +0.2 0.17 ± 16% perf-profile.self.cycles-pp.update_rq_clock_task
0.01 ±264% +0.2 0.18 ± 22% perf-profile.self.cycles-pp.update_sd_lb_stats
0.00 +0.2 0.18 ± 31% perf-profile.self.cycles-pp.update_rq_clock
0.00 +0.2 0.18 ± 44% perf-profile.self.cycles-pp.timekeeping_advance
0.00 +0.2 0.19 ± 28% perf-profile.self.cycles-pp.irqtime_account_process_tick
0.00 +0.2 0.19 ± 36% perf-profile.self.cycles-pp.__do_softirq
0.00 +0.2 0.20 ± 67% perf-profile.self.cycles-pp.trigger_load_balance
0.00 +0.2 0.22 ± 31% perf-profile.self.cycles-pp.rcu_pending
0.00 +0.2 0.22 ± 23% perf-profile.self.cycles-pp.perf_rotate_context
0.00 +0.2 0.22 ± 26% perf-profile.self.cycles-pp.hrtimer_interrupt
0.10 ± 3% +0.2 0.33 ± 16% perf-profile.self.cycles-pp._raw_spin_lock_irq
0.00 +0.2 0.24 ± 28% perf-profile.self.cycles-pp.calc_global_load_tick
0.00 +0.2 0.25 ± 30% perf-profile.self.cycles-pp.load_balance
0.00 +0.2 0.25 ± 32% perf-profile.self.cycles-pp.timerqueue_add
0.00 +0.3 0.25 ± 27% perf-profile.self.cycles-pp.error_entry
0.00 +0.3 0.26 ± 16% perf-profile.self.cycles-pp.idle_cpu
0.00 +0.3 0.26 ± 17% perf-profile.self.cycles-pp.perf_pmu_nop_void
0.00 +0.3 0.26 ± 19% perf-profile.self.cycles-pp.call_cpuidle
0.00 +0.3 0.27 ± 27% perf-profile.self.cycles-pp.__hrtimer_next_event_base
0.00 +0.3 0.28 ± 26% perf-profile.self.cycles-pp.tick_check_broadcast_expired
0.00 +0.3 0.29 ± 25% perf-profile.self.cycles-pp.x86_pmu_disable
0.38 ± 2% +0.3 0.68 ± 18% perf-profile.self.cycles-pp.update_sg_lb_stats
0.00 +0.3 0.29 ± 23% perf-profile.self.cycles-pp.irq_enter_rcu
0.00 +0.3 0.30 ± 45% perf-profile.self.cycles-pp.update_process_times
0.00 +0.3 0.30 ± 22% perf-profile.self.cycles-pp._find_next_and_bit
0.00 +0.3 0.30 ± 19% perf-profile.self.cycles-pp.scheduler_tick
0.00 +0.3 0.32 ± 18% perf-profile.self.cycles-pp.irqtime_account_irq
0.00 +0.3 0.34 ± 21% perf-profile.self.cycles-pp.tick_nohz_next_event
0.00 +0.3 0.34 ± 32% perf-profile.self.cycles-pp.get_cpu_device
0.71 +0.3 1.05 ± 16% perf-profile.self.cycles-pp._raw_spin_lock
0.00 +0.4 0.35 ± 27% perf-profile.self.cycles-pp.__hrtimer_run_queues
0.00 +0.4 0.36 ± 18% perf-profile.self.cycles-pp.tick_irq_enter
0.00 +0.4 0.37 ± 16% perf-profile.self.cycles-pp.do_idle
0.00 +0.4 0.44 ± 18% perf-profile.self.cycles-pp.ct_kernel_exit_state
0.00 +0.4 0.45 ± 22% perf-profile.self.cycles-pp.cpuidle_enter
0.00 +0.5 0.46 ± 18% perf-profile.self.cycles-pp.cpuidle_enter_state
0.00 +0.5 0.46 ± 23% perf-profile.self.cycles-pp.update_irq_load_avg
0.00 +0.5 0.47 ± 23% perf-profile.self.cycles-pp.tick_check_oneshot_broadcast_this_cpu
0.29 ± 2% +0.6 0.85 ± 8% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.00 +0.6 0.56 ± 19% perf-profile.self.cycles-pp.sysvec_apic_timer_interrupt
0.00 +0.6 0.58 ± 10% perf-profile.self.cycles-pp.cpuidle_idle_call
0.00 +0.6 0.61 ± 17% perf-profile.self.cycles-pp.asm_sysvec_apic_timer_interrupt
0.10 ± 4% +0.6 0.75 ± 13% perf-profile.self.cycles-pp._raw_spin_trylock
0.00 +0.7 0.68 ± 64% perf-profile.self.cycles-pp.tick_sched_do_timer
0.00 +0.8 0.80 ± 15% perf-profile.self.cycles-pp.native_apic_msr_eoi
0.00 +0.9 0.88 ± 16% perf-profile.self.cycles-pp.menu_select
0.00 +1.1 1.12 ± 8% perf-profile.self.cycles-pp.native_sched_clock
0.36 ± 2% +1.2 1.55 ± 11% perf-profile.self.cycles-pp.native_irq_return_iret
0.00 +1.2 1.21 ± 12% perf-profile.self.cycles-pp.lapic_next_deadline
0.00 +1.4 1.40 ± 8% perf-profile.self.cycles-pp.read_tsc
0.00 +2.2 2.25 ± 14% perf-profile.self.cycles-pp.arch_scale_freq_tick
0.00 +2.3 2.31 ± 47% perf-profile.self.cycles-pp.ktime_get_update_offsets_now
0.00 +2.8 2.82 ± 85% perf-profile.self.cycles-pp.ktime_get
0.13 +3.3 3.48 ± 9% perf-profile.self.cycles-pp.perf_adjust_freq_unthr_context
0.06 ± 5% +8.9 8.94 ± 6% perf-profile.self.cycles-pp.__intel_pmu_enable_all
0.12 ± 4% +46.6 46.72 ± 4% perf-profile.self.cycles-pp.acpi_safe_halt***************************************************************************************************
lkp-icl-2sp7: 64 threads 2 sockets Intel(R) Xeon(R) Gold 6346 CPU @ 3.10GHz (Ice Lake) with 256G memory
=========================================================================================
class/compiler/cpufreq_governor/disk/fs/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime:
os/gcc-12/performance/1HDD/ext4/x86_64-rhel-8.3/10%/debian-11.1-x86_64-20220510.cgz/lkp-icl-2sp7/netlink-proc/stress-ng/60s
commit:
059d37b718 ("net: phy: linux/phy.h: fix Excess kernel-doc description warning") c46bfba133 ("connector: Fix proc_event_num_listeners count not cleared")
059d37b718d38d26 c46bfba1337d301661dbb23cfd9
---------------- ---------------------------
%stddev %change %stddev
\ | \
89.27 +8.4% 96.77 iostat.cpu.idle
9.69 -71.3% 2.78 iostat.cpu.system
1.01 ± 3% -59.2% 0.41 ± 8% iostat.cpu.user
6179379 ± 3% -95.3% 292556 ± 21% numa-numastat.node0.local_node
6219182 ± 3% -94.5% 339100 ± 13% numa-numastat.node0.numa_hit
6224770 ± 3% -93.2% 422996 ± 13% numa-numastat.node1.local_node
6251192 ± 3% -92.9% 442678 ± 9% numa-numastat.node1.numa_hit
89.35 +8.4% 96.85 vmstat.cpu.id
6.89 -88.6% 0.78 ± 6% vmstat.procs.r
24070 -88.3% 2818 ± 8% vmstat.system.cs
69769 -5.7% 65793 vmstat.system.in
0.02 ± 5% +0.0 0.03 ± 15% mpstat.cpu.all.iowait%
0.86 +1.5 2.38 mpstat.cpu.all.irq%
0.12 -0.1 0.07 ± 3% mpstat.cpu.all.soft%
8.95 -8.6 0.37 mpstat.cpu.all.sys%
1.01 ± 2% -0.6 0.40 ± 8% mpstat.cpu.all.usr%
469.38 ± 19% -99.2% 3.62 ± 43% perf-c2c.DRAM.local
1304 ± 10% -97.9% 27.25 ± 23% perf-c2c.DRAM.remote
1738 ± 10% -96.9% 54.12 ± 15% perf-c2c.HITM.local
1133 ± 10% -98.4% 18.00 ± 20% perf-c2c.HITM.remote
2871 ± 8% -97.5% 72.12 ± 12% perf-c2c.HITM.total
43690 ± 68% -87.7% 5370 ± 19% sched_debug.cfs_rq:/.avg_vruntime.avg
26078 ± 78% -98.4% 408.16 ± 53% sched_debug.cfs_rq:/.avg_vruntime.min
43690 ± 68% -87.7% 5370 ± 19% sched_debug.cfs_rq:/.min_vruntime.avg
26078 ± 78% -98.4% 408.16 ± 53% sched_debug.cfs_rq:/.min_vruntime.min
166583 ± 94% -97.4% 4297 ± 11% sched_debug.cpu.avg_idle.min
55253 -97.4% 1413 stress-ng.netlink-proc.ops
920.84 -97.4% 23.55 stress-ng.netlink-proc.ops_per_sec
3512 -97.2% 96.62 ± 8% stress-ng.time.involuntary_context_switches
14344971 -97.2% 397390 ± 2% stress-ng.time.minor_page_faults
579.75 -97.3% 15.75 ± 2% stress-ng.time.percent_of_cpu_this_job_got
288.15 -97.1% 8.21 ± 3% stress-ng.time.system_time
72.30 ± 2% -97.6% 1.74 ± 4% stress-ng.time.user_time
634891 -97.3% 17180 stress-ng.time.voluntary_context_switches
351.88 -84.7% 53.88 ± 2% turbostat.Avg_MHz
11.43 -7.5 3.97 turbostat.Busy%
3084 -55.9% 1360 ± 2% turbostat.Bzy_MHz
11465 ± 2% -16.3% 9602 turbostat.C1
680235 -88.1% 81023 ± 9% turbostat.C1E
6.23 -5.6 0.68 ± 5% turbostat.C1E%
82.64 +13.2 95.80 turbostat.C6%
0.15 +62.5% 0.24 turbostat.IPC
58.12 -8.2% 53.38 ± 2% turbostat.PkgTmp
158.42 -21.9% 123.74 turbostat.PkgWatt
57.90 -7.6% 53.50 turbostat.RAMWatt
96795 -78.4% 20950 ± 2% meminfo.Active
89506 -84.5% 13908 ± 6% meminfo.Active(anon)
1009177 -65.4% 349530 meminfo.AnonPages
1590446 -59.2% 649479 meminfo.Committed_AS
1134660 -61.8% 433496 meminfo.Inactive
1122208 -62.5% 421239 meminfo.Inactive(anon)
16126 ± 3% -10.9% 14375 meminfo.KernelStack
165907 -36.5% 105360 meminfo.Mapped
5318112 -16.7% 4429934 meminfo.Memused
12261 ± 5% -50.3% 6095 meminfo.PageTables
205864 -57.0% 88596 meminfo.Shmem
5629293 -14.5% 4812679 ± 2% meminfo.max_used_kB
394598 ± 21% -58.2% 164970 ± 40% numa-meminfo.node0.AnonPages
2256635 ± 31% -88.7% 254633 ± 52% numa-meminfo.node0.AnonPages.max
423034 ± 19% -57.0% 181718 ± 41% numa-meminfo.node0.Inactive
415347 ± 19% -58.1% 174072 ± 40% numa-meminfo.node0.Inactive(anon)
5870 ± 7% -50.1% 2927 ± 11% numa-meminfo.node0.PageTables
91164 ± 3% -83.7% 14856 ± 22% numa-meminfo.node1.Active
87908 -86.0% 12266 ± 9% numa-meminfo.node1.Active(anon)
617272 ± 15% -70.0% 184963 ± 36% numa-meminfo.node1.AnonPages
3122142 ± 20% -86.0% 435739 ± 27% numa-meminfo.node1.AnonPages.max
714417 ± 12% -64.7% 252095 ± 29% numa-meminfo.node1.Inactive
709652 ± 12% -65.1% 247485 ± 28% numa-meminfo.node1.Inactive(anon)
7546 ± 9% -15.1% 6406 ± 3% numa-meminfo.node1.KernelStack
6510 ± 9% -51.3% 3172 ± 9% numa-meminfo.node1.PageTables
180425 ± 16% -58.7% 74525 ± 30% numa-meminfo.node1.Shmem
98799 ± 21% -58.2% 41249 ± 40% numa-vmstat.node0.nr_anon_pages
104004 ± 19% -58.2% 43523 ± 40% numa-vmstat.node0.nr_inactive_anon
1465 ± 7% -50.1% 732.05 ± 11% numa-vmstat.node0.nr_page_table_pages
104004 ± 19% -58.2% 43523 ± 40% numa-vmstat.node0.nr_zone_inactive_anon
6219196 ± 3% -94.5% 339138 ± 13% numa-vmstat.node0.numa_hit
6179393 ± 3% -95.3% 292594 ± 21% numa-vmstat.node0.numa_local
21922 -86.0% 3067 ± 9% numa-vmstat.node1.nr_active_anon
153302 ± 14% -69.8% 46269 ± 36% numa-vmstat.node1.nr_anon_pages
176413 ± 11% -64.9% 61899 ± 28% numa-vmstat.node1.nr_inactive_anon
7558 ± 9% -15.2% 6406 ± 3% numa-vmstat.node1.nr_kernel_stack
1644 ± 9% -51.8% 792.78 ± 9% numa-vmstat.node1.nr_page_table_pages
45071 ± 17% -58.7% 18632 ± 30% numa-vmstat.node1.nr_shmem
21922 -86.0% 3067 ± 9% numa-vmstat.node1.nr_zone_active_anon
176412 ± 11% -64.9% 61899 ± 28% numa-vmstat.node1.nr_zone_inactive_anon
6243377 ± 3% -92.9% 442671 ± 9% numa-vmstat.node1.numa_hit
6216956 ± 3% -93.2% 422989 ± 13% numa-vmstat.node1.numa_local
22377 -84.4% 3486 ± 6% proc-vmstat.nr_active_anon
252413 -65.4% 87402 proc-vmstat.nr_anon_pages
787977 -3.7% 758546 proc-vmstat.nr_file_pages
280677 -62.5% 105304 proc-vmstat.nr_inactive_anon
16131 ± 3% -10.9% 14374 proc-vmstat.nr_kernel_stack
41487 -36.5% 26333 proc-vmstat.nr_mapped
3081 ± 5% -50.6% 1523 proc-vmstat.nr_page_table_pages
51472 -57.0% 22151 proc-vmstat.nr_shmem
24685 -2.0% 24183 proc-vmstat.nr_slab_reclaimable
37640 -2.7% 36613 proc-vmstat.nr_slab_unreclaimable
22377 -84.4% 3486 ± 6% proc-vmstat.nr_zone_active_anon
280677 -62.5% 105304 proc-vmstat.nr_zone_inactive_anon
891483 ± 6% -98.8% 10906 ± 36% proc-vmstat.numa_hint_faults
432525 ± 9% -98.1% 8274 ± 60% proc-vmstat.numa_hint_faults_local
12470709 ± 2% -93.7% 785652 proc-vmstat.numa_hit
12404485 ± 2% -94.2% 719425 proc-vmstat.numa_local
459347 ± 9% -99.4% 2893 ± 86% proc-vmstat.numa_pages_migrated
1138450 ± 12% -96.4% 40628 ± 17% proc-vmstat.numa_pte_updates
67004 ± 7% -62.6% 25091 proc-vmstat.pgactivate
12986580 ± 2% -93.5% 838904 proc-vmstat.pgalloc_normal
17279401 -95.3% 819331 proc-vmstat.pgfault
12246108 ± 3% -93.4% 802615 proc-vmstat.pgfree
459347 ± 9% -99.4% 2893 ± 86% proc-vmstat.pgmigrate_success
1800112 ± 6% -97.2% 50374 ± 8% proc-vmstat.pgreuse
3.16 -80.6% 0.61 ± 6% perf-stat.i.MPKI
1.898e+09 -83.3% 3.168e+08 ± 4% perf-stat.i.branch-instructions
0.89 ± 2% +0.1 1.02 ± 4% perf-stat.i.branch-miss-rate%
17135799 ± 3% -65.2% 5963858 ± 8% perf-stat.i.branch-misses
34.56 -19.0 15.52 ± 3% perf-stat.i.cache-miss-rate%
29396419 -94.7% 1565243 ± 3% perf-stat.i.cache-misses
84235895 -93.5% 5503745 ± 2% perf-stat.i.cache-references
24974 -89.6% 2585 ± 9% perf-stat.i.context-switches
2.41 -5.6% 2.28 perf-stat.i.cpi
2.21e+10 -89.0% 2.437e+09 ± 3% perf-stat.i.cpu-cycles
1623 ± 2% -90.5% 153.99 ± 2% perf-stat.i.cpu-migrations
806.52 +931.9% 8322 ± 4% perf-stat.i.cycles-between-cache-misses
0.08 ± 3% -0.1 0.03 ± 7% perf-stat.i.dTLB-load-miss-rate%
1957035 ± 2% -94.5% 108272 ± 5% perf-stat.i.dTLB-load-misses
2.393e+09 -84.5% 3.718e+08 ± 3% perf-stat.i.dTLB-loads
0.13 ± 2% -0.1 0.02 ± 2% perf-stat.i.dTLB-store-miss-rate%
1601070 ± 2% -95.6% 69877 ± 2% perf-stat.i.dTLB-store-misses
1.21e+09 -87.1% 1.567e+08 ± 2% perf-stat.i.dTLB-stores
9.36e+09 -83.5% 1.546e+09 ± 4% perf-stat.i.instructions
0.43 +13.8% 0.49 perf-stat.i.ipc
0.34 -89.0% 0.04 ± 3% perf-stat.i.metric.GHz
486.12 +28.7% 625.46 ± 3% perf-stat.i.metric.K/sec
87.09 -85.5% 12.67 ± 3% perf-stat.i.metric.M/sec
272812 -96.1% 10632 ± 2% perf-stat.i.minor-faults
91.63 -12.9 78.73 ± 3% perf-stat.i.node-load-miss-rate%
15176190 -96.2% 583756 ± 9% perf-stat.i.node-load-misses
1548480 ± 5% -83.8% 251170 ± 14% perf-stat.i.node-loads
49.87 -19.8 30.07 ± 13% perf-stat.i.node-store-miss-rate%
5408301 -95.6% 239835 ± 9% perf-stat.i.node-store-misses
6125812 -93.0% 429311 ± 3% perf-stat.i.node-stores
272812 -96.1% 10632 ± 2% perf-stat.i.page-faults
3.14 -67.7% 1.01 ± 3% perf-stat.overall.MPKI
0.90 ± 2% +1.0 1.88 ± 3% perf-stat.overall.branch-miss-rate%
34.78 -6.4 28.43 ± 2% perf-stat.overall.cache-miss-rate%
2.36 -33.3% 1.58 perf-stat.overall.cpi
752.33 +106.9% 1556 ± 2% perf-stat.overall.cycles-between-cache-misses
0.08 ± 2% -0.1 0.03 ± 4% perf-stat.overall.dTLB-load-miss-rate%
0.13 ± 2% -0.1 0.04 ± 2% perf-stat.overall.dTLB-store-miss-rate%
0.42 +49.9% 0.63 perf-stat.overall.ipc
90.32 -20.4 69.93 ± 6% perf-stat.overall.node-load-miss-rate%
46.90 -11.1 35.83 ± 6% perf-stat.overall.node-store-miss-rate%
1.866e+09 -83.3% 3.123e+08 ± 4% perf-stat.ps.branch-instructions
16768519 ± 3% -64.9% 5877656 ± 8% perf-stat.ps.branch-misses
28912651 -94.7% 1544646 ± 3% perf-stat.ps.cache-misses
83136724 -93.5% 5432912 ± 2% perf-stat.ps.cache-references
24594 -89.6% 2553 ± 9% perf-stat.ps.context-switches
2.175e+10 -89.0% 2.403e+09 ± 3% perf-stat.ps.cpu-cycles
1607 ± 2% -90.5% 151.91 perf-stat.ps.cpu-migrations
1936940 ± 2% -94.5% 106844 ± 4% perf-stat.ps.dTLB-load-misses
2.353e+09 -84.4% 3.665e+08 ± 3% perf-stat.ps.dTLB-loads
1574562 ± 2% -95.6% 68974 ± 2% perf-stat.ps.dTLB-store-misses
1.19e+09 -87.0% 1.544e+08 ± 2% perf-stat.ps.dTLB-stores
9.198e+09 -83.4% 1.524e+09 ± 4% perf-stat.ps.instructions
268072 -96.1% 10500 perf-stat.ps.minor-faults
14881864 -96.1% 576947 ± 9% perf-stat.ps.node-load-misses
1595791 ± 3% -84.5% 247069 ± 14% perf-stat.ps.node-loads
5309475 -95.5% 236982 ± 9% perf-stat.ps.node-store-misses
6012142 -93.0% 423311 ± 3% perf-stat.ps.node-stores
268072 -96.1% 10501 perf-stat.ps.page-faults
5.783e+11 -83.6% 9.496e+10 ± 4% perf-stat.total.instructions
0.00 ± 25% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.pte_alloc_one.do_read_fault
0.00 ± 30% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.vma_alloc_folio.wp_page_copy
0.01 ± 36% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.__put_anon_vma.unlink_anon_vmas.free_pgtables.exit_mmap
0.00 ± 19% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.copy_page_range.dup_mmap.dup_mm.constprop
0.01 ±125% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.copy_pte_range.copy_p4d_range.copy_page_range.dup_mmap
0.01 ±106% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.down_write.dup_mmap.dup_mm.constprop
0.01 ± 12% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.down_write.free_pgtables.exit_mmap.__mmput
0.01 ± 11% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.down_write.unlink_file_vma.free_pgtables.exit_mmap
0.01 ± 95% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.exit_mmap.__mmput.exit_mm.do_exit
0.01 ± 57% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.kmem_cache_alloc.anon_vma_fork.dup_mmap.dup_mm
0.01 ±109% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.kmem_cache_alloc.vm_area_dup.dup_mmap.dup_mm
0.01 ±103% -88.4% 0.00 ±173% perf-sched.sch_delay.avg.ms.__cond_resched.stop_one_cpu.sched_exec.bprm_execve.part
0.01 ± 15% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.tlb_batch_pages_flush.tlb_finish_mmu.exit_mmap.__mmput
0.00 ± 22% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.wp_page_copy.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
0.01 ± 23% -100.0% 0.00 perf-sched.sch_delay.avg.ms.__cond_resched.zap_pmd_range.isra.0.unmap_page_range
0.01 ± 11% +48.3% 0.01 ± 15% perf-sched.sch_delay.avg.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
0.01 ± 17% +59.2% 0.01 ± 15% perf-sched.sch_delay.avg.ms.do_nanosleep.hrtimer_nanosleep.common_nsleep.__x64_sys_clock_nanosleep
0.01 ± 3% -34.2% 0.01 ± 7% perf-sched.sch_delay.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
0.01 ± 46% -92.6% 0.00 ±264% perf-sched.sch_delay.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault
0.00 ± 15% -94.4% 0.00 ±264% perf-sched.sch_delay.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt
0.01 ±121% -100.0% 0.00 perf-sched.sch_delay.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode.ret_from_fork_asm
0.00 ± 14% +170.4% 0.01 ± 10% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.do_poll.constprop.0.do_sys_poll
0.01 ± 9% +24.6% 0.01 ± 11% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait
0.01 ± 42% +117.0% 0.01 ± 15% perf-sched.sch_delay.avg.ms.schedule_timeout.__wait_for_common.__flush_work.isra.0
0.01 ± 29% -37.0% 0.00 ± 10% perf-sched.sch_delay.avg.ms.schedule_timeout.__wait_for_common.wait_for_completion_state.kernel_clone
0.01 ± 13% +60.0% 0.01 ± 23% perf-sched.sch_delay.avg.ms.schedule_timeout.io_schedule_timeout.__wait_for_common.submit_bio_wait
0.01 ± 10% +50.0% 0.01 ± 12% perf-sched.sch_delay.avg.ms.schedule_timeout.kcompactd.kthread.ret_from_fork
0.01 +92.5% 0.01 ± 7% perf-sched.sch_delay.avg.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
0.01 ± 30% +42.2% 0.01 ± 19% perf-sched.sch_delay.avg.ms.schedule_timeout.transaction_kthread.kthread.ret_from_fork
0.00 ± 22% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.pte_alloc_one.do_read_fault
0.01 ± 98% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.__alloc_pages.alloc_pages_mpol.vma_alloc_folio.wp_page_copy
0.06 ±129% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.__put_anon_vma.unlink_anon_vmas.free_pgtables.exit_mmap
0.00 ± 30% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.copy_page_range.dup_mmap.dup_mm.constprop
0.03 ±225% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.copy_pte_range.copy_p4d_range.copy_page_range.dup_mmap
0.06 ±142% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.down_write.dup_mmap.dup_mm.constprop
0.01 ± 15% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.down_write.free_pgtables.exit_mmap.__mmput
0.01 ± 26% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.down_write.unlink_file_vma.free_pgtables.exit_mmap
0.06 ±159% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.exit_mmap.__mmput.exit_mm.do_exit
0.07 ±115% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.kmem_cache_alloc.anon_vma_fork.dup_mmap.dup_mm
0.04 ±199% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.kmem_cache_alloc.vm_area_dup.dup_mmap.dup_mm
0.08 ±219% -98.6% 0.00 ±174% perf-sched.sch_delay.max.ms.__cond_resched.stop_one_cpu.sched_exec.bprm_execve.part
0.01 ± 71% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.tlb_batch_pages_flush.tlb_finish_mmu.exit_mmap.__mmput
0.01 ± 26% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.wp_page_copy.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
0.05 ±124% -100.0% 0.00 perf-sched.sch_delay.max.ms.__cond_resched.zap_pmd_range.isra.0.unmap_page_range
5.41 ± 46% -99.7% 0.02 ± 26% perf-sched.sch_delay.max.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
0.12 ±113% -99.6% 0.00 ±264% perf-sched.sch_delay.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault
0.01 ± 43% -97.5% 0.00 ±264% perf-sched.sch_delay.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_sysvec_apic_timer_interrupt
0.04 ±134% -100.0% 0.00 perf-sched.sch_delay.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.syscall_exit_to_user_mode.ret_from_fork_asm
0.02 ± 81% -86.3% 0.00 ±142% perf-sched.sch_delay.max.ms.io_schedule.folio_wait_bit_common.filemap_fault.__do_fault
0.01 ± 41% +116.3% 0.01 ± 10% perf-sched.sch_delay.max.ms.schedule_timeout.__wait_for_common.__flush_work.isra.0
0.07 ±210% -87.8% 0.01 ± 37% perf-sched.sch_delay.max.ms.schedule_timeout.__wait_for_common.wait_for_completion_state.kernel_clone
0.01 ± 30% +42.2% 0.01 ± 19% perf-sched.sch_delay.max.ms.schedule_timeout.transaction_kthread.kthread.ret_from_fork
14.38 ± 2% +1904.1% 288.29 ± 26% perf-sched.total_wait_and_delay.average.ms
63545 -95.4% 2891 ± 56% perf-sched.total_wait_and_delay.count.ms
4008 ± 9% +21.5% 4870 ± 6% perf-sched.total_wait_and_delay.max.ms
14.37 ± 3% +1904.3% 288.11 ± 26% perf-sched.total_wait_time.average.ms
4008 ± 9% +21.5% 4870 ± 6% perf-sched.total_wait_time.max.ms
10.60 ± 4% +116.8% 22.97 ± 5% perf-sched.wait_and_delay.avg.ms.__cond_resched.__wait_for_common.affine_move_task.__set_cpus_allowed_ptr.__sched_setaffinity
3.37 +7231.1% 247.13 ± 6% perf-sched.wait_and_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
75.13 ± 3% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.kjournald2.kthread.ret_from_fork.ret_from_fork_asm
366.09 -12.9% 318.85 ± 2% perf-sched.wait_and_delay.avg.ms.pipe_read.vfs_read.ksys_read.do_syscall_64
483.15 ± 9% +19.3% 576.62 ± 2% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.do_poll.constprop.0.do_sys_poll
4.10 ± 3% +1074.0% 48.14 ± 10% perf-sched.wait_and_delay.avg.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
74.48 +522.8% 463.81 ± 2% perf-sched.wait_and_delay.avg.ms.worker_thread.kthread.ret_from_fork.ret_from_fork_asm
5.88 ± 36% -89.4% 0.62 ±111% perf-sched.wait_and_delay.count.__cond_resched.smpboot_thread_fn.kthread.ret_from_fork.ret_from_fork_asm
28019 -99.6% 123.25 ± 3% perf-sched.wait_and_delay.count.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
28024 -99.6% 122.50 ± 3% perf-sched.wait_and_delay.count.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64
1.00 -100.0% 0.00 perf-sched.wait_and_delay.count.kjournald2.kthread.ret_from_fork.ret_from_fork_asm
196.75 +14.3% 224.88 ± 4% perf-sched.wait_and_delay.count.pipe_read.vfs_read.ksys_read.do_syscall_64
35.00 ± 10% -39.6% 21.12 perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.do_poll.constprop.0.do_sys_poll
1218 ± 3% -91.7% 101.50 ± 9% perf-sched.wait_and_delay.count.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
4256 -89.7% 438.38 ± 4% perf-sched.wait_and_delay.count.worker_thread.kthread.ret_from_fork.ret_from_fork_asm
1000 +381.2% 4816 ± 9% perf-sched.wait_and_delay.max.ms.__cond_resched.__wait_for_common.affine_move_task.__set_cpus_allowed_ptr.__sched_setaffinity
2670 ± 35% -81.2% 502.37 ±100% perf-sched.wait_and_delay.max.ms.__cond_resched.smpboot_thread_fn.kthread.ret_from_fork.ret_from_fork_asm
1000 +51.0% 1510 ± 33% perf-sched.wait_and_delay.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
75.13 ± 3% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.kjournald2.kthread.ret_from_fork.ret_from_fork_asm
3488 ± 8% -69.9% 1049 ± 12% perf-sched.wait_and_delay.max.ms.schedule_hrtimeout_range_clock.do_poll.constprop.0.do_sys_poll
5.05 +8450.8% 431.87 ± 12% perf-sched.wait_and_delay.max.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
1.26 ± 21% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.__put_anon_vma.unlink_anon_vmas.free_pgtables.exit_mmap
10.60 ± 4% +116.8% 22.97 ± 5% perf-sched.wait_time.avg.ms.__cond_resched.__wait_for_common.affine_move_task.__set_cpus_allowed_ptr.__sched_setaffinity
2.52 ± 15% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.down_write.free_pgtables.exit_mmap.__mmput
1.73 ± 28% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.exit_mmap.__mmput.exit_mm.do_exit
2.19 ± 20% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.tlb_batch_pages_flush.tlb_finish_mmu.exit_mmap.__mmput
1.73 ± 14% -100.0% 0.00 perf-sched.wait_time.avg.ms.__cond_resched.zap_pmd_range.isra.0.unmap_page_range
3.36 +7212.1% 246.01 ± 5% perf-sched.wait_time.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
0.18 ± 47% -100.0% 0.00 perf-sched.wait_time.avg.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault
365.45 -13.1% 317.69 ± 2% perf-sched.wait_time.avg.ms.pipe_read.vfs_read.ksys_read.do_syscall_64
483.15 ± 9% +19.3% 576.61 ± 2% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.do_poll.constprop.0.do_sys_poll
0.63 ± 2% -21.8% 0.49 ± 7% perf-sched.wait_time.avg.ms.schedule_timeout.__wait_for_common.wait_for_completion_state.kernel_clone
4.10 ± 3% +1075.3% 48.13 ± 10% perf-sched.wait_time.avg.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
0.00 ±145% -100.0% 0.00 perf-sched.wait_time.avg.ms.wait_for_partner.fifo_open.do_dentry_open.do_open
74.46 +522.9% 463.79 ± 2% perf-sched.wait_time.avg.ms.worker_thread.kthread.ret_from_fork.ret_from_fork_asm
5.28 ± 4% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.__put_anon_vma.unlink_anon_vmas.free_pgtables.exit_mmap
1000 +381.2% 4816 ± 9% perf-sched.wait_time.max.ms.__cond_resched.__wait_for_common.affine_move_task.__set_cpus_allowed_ptr.__sched_setaffinity
5.58 ± 5% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.down_write.free_pgtables.exit_mmap.__mmput
4.78 ± 18% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.exit_mmap.__mmput.exit_mm.do_exit
2670 ± 35% -81.1% 504.97 ± 98% perf-sched.wait_time.max.ms.__cond_resched.smpboot_thread_fn.kthread.ret_from_fork.ret_from_fork_asm
5.20 ± 7% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.tlb_batch_pages_flush.tlb_finish_mmu.exit_mmap.__mmput
5.77 ± 7% -100.0% 0.00 perf-sched.wait_time.max.ms.__cond_resched.zap_pmd_range.isra.0.unmap_page_range
1000 +51.0% 1510 ± 33% perf-sched.wait_time.max.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
4.58 ± 24% -100.0% 0.00 perf-sched.wait_time.max.ms.exit_to_user_mode_loop.exit_to_user_mode_prepare.irqentry_exit_to_user_mode.asm_exc_page_fault
3488 ± 8% -69.9% 1049 ± 12% perf-sched.wait_time.max.ms.schedule_hrtimeout_range_clock.do_poll.constprop.0.do_sys_poll
5.03 +8483.5% 431.86 ± 12% perf-sched.wait_time.max.ms.schedule_timeout.rcu_gp_fqs_loop.rcu_gp_kthread.kthread
0.08 ±209% -97.9% 0.00 ±189% perf-sched.wait_time.max.ms.wait_for_partner.fifo_open.do_dentry_open.do_open
41.46 -41.5 0.00 perf-profile.calltrace.cycles-pp.__libc_fork
39.06 -39.1 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__libc_fork
39.06 -39.1 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork
39.06 -39.1 0.00 perf-profile.calltrace.cycles-pp.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork
39.05 -39.1 0.00 perf-profile.calltrace.cycles-pp.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe.__libc_fork
38.26 -38.3 0.00 perf-profile.calltrace.cycles-pp.copy_process.kernel_clone.__do_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe
36.33 -36.3 0.00 perf-profile.calltrace.cycles-pp.dup_mm.copy_process.kernel_clone.__do_sys_clone.do_syscall_64
35.35 -35.3 0.00 perf-profile.calltrace.cycles-pp.dup_mmap.dup_mm.copy_process.kernel_clone.__do_sys_clone
31.23 -31.1 0.14 ±173% perf-profile.calltrace.cycles-pp.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
31.23 -31.1 0.14 ±173% perf-profile.calltrace.cycles-pp.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
31.22 -31.1 0.14 ±173% perf-profile.calltrace.cycles-pp.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64.entry_SYSCALL_64_after_hwframe
30.16 -30.2 0.00 perf-profile.calltrace.cycles-pp.exit_mm.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
30.15 -30.2 0.00 perf-profile.calltrace.cycles-pp.__mmput.exit_mm.do_exit.do_group_exit.__x64_sys_exit_group
31.28 -30.1 1.16 ± 15% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
31.28 -30.1 1.17 ± 14% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
30.08 -30.1 0.00 perf-profile.calltrace.cycles-pp.exit_mmap.__mmput.exit_mm.do_exit.do_group_exit
18.19 -18.2 0.00 perf-profile.calltrace.cycles-pp.anon_vma_fork.dup_mmap.dup_mm.copy_process.kernel_clone
15.25 -15.2 0.00 perf-profile.calltrace.cycles-pp.anon_vma_clone.anon_vma_fork.dup_mmap.dup_mm.copy_process
15.16 -15.2 0.00 perf-profile.calltrace.cycles-pp.free_pgtables.exit_mmap.__mmput.exit_mm.do_exit
10.13 -10.1 0.00 perf-profile.calltrace.cycles-pp.unlink_anon_vmas.free_pgtables.exit_mmap.__mmput.exit_mm
9.32 -9.3 0.00 perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.__mmput.exit_mm.do_exit
9.70 ± 6% -9.1 0.59 ± 40% perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
8.66 -8.7 0.00 perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.exit_mmap.__mmput.exit_mm
8.51 -8.5 0.00 perf-profile.calltrace.cycles-pp.zap_pmd_range.unmap_page_range.unmap_vmas.exit_mmap.__mmput
8.20 -8.2 0.00 perf-profile.calltrace.cycles-pp.zap_pte_range.zap_pmd_range.unmap_page_range.unmap_vmas.exit_mmap
8.12 ± 5% -7.9 0.23 ±130% perf-profile.calltrace.cycles-pp.do_read_fault.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
8.61 ± 2% -7.8 0.81 ± 13% perf-profile.calltrace.cycles-pp.asm_exc_page_fault
8.14 ± 5% -7.8 0.37 ± 80% perf-profile.calltrace.cycles-pp.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.exc_page_fault
8.19 ± 2% -7.4 0.76 ± 14% perf-profile.calltrace.cycles-pp.exc_page_fault.asm_exc_page_fault
8.18 ± 2% -7.4 0.75 ± 14% perf-profile.calltrace.cycles-pp.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
7.13 -7.1 0.00 perf-profile.calltrace.cycles-pp.anon_vma_interval_tree_insert.anon_vma_clone.anon_vma_fork.dup_mmap.dup_mm
7.26 ± 5% -7.1 0.15 ±173% perf-profile.calltrace.cycles-pp.filemap_map_pages.do_read_fault.do_fault.__handle_mm_fault.handle_mm_fault
7.64 ± 3% -6.9 0.69 ± 14% perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.exc_page_fault.asm_exc_page_fault
5.64 ± 3% -5.5 0.14 ±173% perf-profile.calltrace.cycles-pp.next_uptodate_folio.filemap_map_pages.do_read_fault.do_fault.__handle_mm_fault
0.00 +0.7 0.74 ± 9% perf-profile.calltrace.cycles-pp.update_blocked_averages.run_rebalance_domains.__do_softirq.irq_exit_rcu.sysvec_apic_timer_interrupt
0.00 +0.8 0.80 ± 8% perf-profile.calltrace.cycles-pp.run_rebalance_domains.__do_softirq.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.00 +0.9 0.88 ± 24% perf-profile.calltrace.cycles-pp.process_one_work.worker_thread.kthread.ret_from_fork.ret_from_fork_asm
0.00 +0.9 0.91 ± 24% perf-profile.calltrace.cycles-pp.worker_thread.kthread.ret_from_fork.ret_from_fork_asm
0.00 +0.9 0.93 ± 31% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.execve
0.00 +0.9 0.93 ± 31% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
0.00 +0.9 0.93 ± 31% perf-profile.calltrace.cycles-pp.execve
0.00 +0.9 0.93 ± 31% perf-profile.calltrace.cycles-pp.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
0.00 +0.9 0.93 ± 31% perf-profile.calltrace.cycles-pp.do_execveat_common.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
0.00 +0.9 0.94 ± 6% perf-profile.calltrace.cycles-pp.update_sd_lb_stats.find_busiest_group.load_balance.rebalance_domains.__do_softirq
0.00 +1.0 1.02 ± 5% perf-profile.calltrace.cycles-pp.find_busiest_group.load_balance.rebalance_domains.__do_softirq.irq_exit_rcu
0.00 +1.1 1.07 ± 28% perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call.do_idle
0.00 +1.3 1.34 ± 11% perf-profile.calltrace.cycles-pp.__intel_pmu_enable_all.perf_rotate_context.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt
0.00 +1.4 1.36 ± 15% perf-profile.calltrace.cycles-pp.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
0.00 +1.4 1.42 ± 12% perf-profile.calltrace.cycles-pp.lapic_next_deadline.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
0.00 +1.4 1.43 ± 13% perf-profile.calltrace.cycles-pp.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
0.00 +1.5 1.46 ± 15% perf-profile.calltrace.cycles-pp.ret_from_fork_asm
0.00 +1.5 1.46 ± 15% perf-profile.calltrace.cycles-pp.ret_from_fork.ret_from_fork_asm
0.00 +1.5 1.46 ± 15% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork.ret_from_fork_asm
0.00 +1.7 1.67 ± 5% perf-profile.calltrace.cycles-pp.load_balance.rebalance_domains.__do_softirq.irq_exit_rcu.sysvec_apic_timer_interrupt
0.00 +1.8 1.78 ± 17% perf-profile.calltrace.cycles-pp.tick_nohz_get_sleep_length.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry
0.00 +1.8 1.81 ± 10% perf-profile.calltrace.cycles-pp.perf_rotate_context.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
0.00 +2.0 1.97 ± 66% perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.rest_init
0.00 +2.0 2.02 ± 11% perf-profile.calltrace.cycles-pp.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.00 +2.0 2.03 ± 65% perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.rest_init.arch_call_rest_init
0.00 +2.0 2.05 ± 64% perf-profile.calltrace.cycles-pp.cpu_startup_entry.rest_init.arch_call_rest_init.start_kernel.x86_64_start_reservations
0.00 +2.0 2.05 ± 64% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.rest_init.arch_call_rest_init.start_kernel
0.00 +2.0 2.05 ± 64% perf-profile.calltrace.cycles-pp.x86_64_start_kernel.secondary_startup_64_no_verify
0.00 +2.0 2.05 ± 64% perf-profile.calltrace.cycles-pp.x86_64_start_reservations.x86_64_start_kernel.secondary_startup_64_no_verify
0.00 +2.0 2.05 ± 64% perf-profile.calltrace.cycles-pp.start_kernel.x86_64_start_reservations.x86_64_start_kernel.secondary_startup_64_no_verify
0.00 +2.0 2.05 ± 64% perf-profile.calltrace.cycles-pp.arch_call_rest_init.start_kernel.x86_64_start_reservations.x86_64_start_kernel.secondary_startup_64_no_verify
0.00 +2.0 2.05 ± 64% perf-profile.calltrace.cycles-pp.rest_init.arch_call_rest_init.start_kernel.x86_64_start_reservations.x86_64_start_kernel
0.00 +2.1 2.06 ± 16% perf-profile.calltrace.cycles-pp.ktime_get_update_offsets_now.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.00 +2.4 2.41 ± 19% perf-profile.calltrace.cycles-pp.arch_scale_freq_tick.scheduler_tick.update_process_times.tick_sched_handle.tick_nohz_highres_handler
0.00 +2.5 2.50 ± 14% perf-profile.calltrace.cycles-pp.rebalance_domains.__do_softirq.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
0.00 +2.7 2.75 ± 11% perf-profile.calltrace.cycles-pp.perf_mux_hrtimer_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
1.02 ± 3% +3.2 4.21 ± 16% perf-profile.calltrace.cycles-pp.__do_softirq.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
0.00 +3.6 3.60 ± 10% perf-profile.calltrace.cycles-pp.menu_select.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
1.11 ± 3% +3.8 4.94 ± 13% perf-profile.calltrace.cycles-pp.irq_exit_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
0.06 ±264% +9.5 9.56 ± 9% perf-profile.calltrace.cycles-pp.__intel_pmu_enable_all.perf_adjust_freq_unthr_context.perf_event_task_tick.scheduler_tick.update_process_times
0.78 ± 7% +13.4 14.22 ± 7% perf-profile.calltrace.cycles-pp.perf_adjust_freq_unthr_context.perf_event_task_tick.scheduler_tick.update_process_times.tick_sched_handle
0.76 ± 6% +14.0 14.71 ± 7% perf-profile.calltrace.cycles-pp.perf_event_task_tick.scheduler_tick.update_process_times.tick_sched_handle.tick_nohz_highres_handler
1.51 ± 4% +17.8 19.27 ± 5% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_nohz_highres_handler.__hrtimer_run_queues
1.72 ± 3% +19.2 20.95 ± 4% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_nohz_highres_handler.__hrtimer_run_queues.hrtimer_interrupt
1.73 ± 4% +19.4 21.16 ± 4% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_nohz_highres_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt
1.97 ± 5% +21.1 23.08 ± 5% perf-profile.calltrace.cycles-pp.tick_nohz_highres_handler.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt
2.27 ± 4% +25.4 27.62 ± 4% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt
2.52 ± 4% +30.5 33.06 ± 4% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state
2.53 ± 4% +30.7 33.19 ± 4% perf-profile.calltrace.cycles-pp.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
4.25 +31.1 35.39 ± 2% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
4.00 ± 5% +37.0 41.00 ± 3% perf-profile.calltrace.cycles-pp.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call
4.28 ± 4% +43.4 47.69 ± 3% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle
8.92 +76.3 85.25 perf-profile.calltrace.cycles-pp.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary
8.63 +77.0 85.64 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.cpuidle_idle_call.do_idle.cpu_startup_entry
9.27 +80.8 90.05 perf-profile.calltrace.cycles-pp.cpuidle_idle_call.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
9.80 +81.1 90.86 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
9.82 +81.3 91.08 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify
9.82 +81.3 91.08 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
9.98 +83.1 93.13 perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify
72.53 -69.0 3.52 ± 13% perf-profile.children.cycles-pp.do_syscall_64
72.54 -69.0 3.54 ± 13% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
41.61 -41.5 0.16 ± 37% perf-profile.children.cycles-pp.__libc_fork
39.06 -38.9 0.12 ± 44% perf-profile.children.cycles-pp.__do_sys_clone
39.06 -38.8 0.24 ± 27% perf-profile.children.cycles-pp.kernel_clone
38.26 -38.1 0.18 ± 23% perf-profile.children.cycles-pp.copy_process
36.33 -36.3 0.08 ± 64% perf-profile.children.cycles-pp.dup_mm
35.44 -35.4 0.08 ± 62% perf-profile.children.cycles-pp.dup_mmap
31.69 -31.2 0.49 ± 12% perf-profile.children.cycles-pp.__x64_sys_exit_group
31.69 -31.2 0.49 ± 12% perf-profile.children.cycles-pp.do_group_exit
31.67 -31.2 0.49 ± 13% perf-profile.children.cycles-pp.do_exit
30.20 -29.8 0.38 ± 12% perf-profile.children.cycles-pp.exit_mm
30.16 -29.7 0.50 ± 12% perf-profile.children.cycles-pp.__mmput
30.10 -29.6 0.50 ± 12% perf-profile.children.cycles-pp.exit_mmap
18.20 -18.2 0.00 perf-profile.children.cycles-pp.anon_vma_fork
15.26 -15.3 0.00 perf-profile.children.cycles-pp.anon_vma_clone
15.18 -15.1 0.12 ± 29% perf-profile.children.cycles-pp.free_pgtables
15.55 ± 2% -14.3 1.20 ± 15% perf-profile.children.cycles-pp.asm_exc_page_fault
14.40 ± 2% -13.3 1.12 ± 15% perf-profile.children.cycles-pp.exc_page_fault
14.36 ± 2% -13.2 1.12 ± 15% perf-profile.children.cycles-pp.do_user_addr_fault
13.17 ± 2% -12.1 1.06 ± 17% perf-profile.children.cycles-pp.handle_mm_fault
12.96 ± 2% -11.9 1.03 ± 17% perf-profile.children.cycles-pp.__handle_mm_fault
11.49 ± 3% -10.7 0.77 ± 23% perf-profile.children.cycles-pp.do_read_fault
11.50 ± 3% -10.7 0.82 ± 21% perf-profile.children.cycles-pp.do_fault
11.28 ± 3% -10.6 0.73 ± 21% perf-profile.children.cycles-pp.filemap_map_pages
10.15 -10.1 0.02 ±175% perf-profile.children.cycles-pp.unlink_anon_vmas
9.34 -9.1 0.28 ± 21% perf-profile.children.cycles-pp.unmap_vmas
8.67 -8.4 0.24 ± 23% perf-profile.children.cycles-pp.unmap_page_range
8.98 ± 3% -8.4 0.62 ± 25% perf-profile.children.cycles-pp.next_uptodate_folio
8.52 -8.3 0.23 ± 25% perf-profile.children.cycles-pp.zap_pmd_range
8.38 -8.2 0.23 ± 25% perf-profile.children.cycles-pp.zap_pte_range
7.68 ± 2% -7.6 0.05 ± 91% perf-profile.children.cycles-pp.down_write
7.16 -7.2 0.00 perf-profile.children.cycles-pp.anon_vma_interval_tree_insert
4.77 ± 2% -4.7 0.06 ± 88% perf-profile.children.cycles-pp._compound_head
4.50 -4.5 0.05 ± 80% perf-profile.children.cycles-pp.__slab_free
2.74 -2.6 0.12 ± 28% perf-profile.children.cycles-pp.kmem_cache_alloc
2.46 -2.4 0.07 ± 65% perf-profile.children.cycles-pp.tlb_finish_mmu
2.43 -2.4 0.05 ± 78% perf-profile.children.cycles-pp.tlb_batch_pages_flush
2.28 -2.2 0.05 ± 60% perf-profile.children.cycles-pp.page_remove_rmap
2.28 -2.2 0.12 ± 41% perf-profile.children.cycles-pp.kmem_cache_free
1.59 ± 2% -1.5 0.05 ± 87% perf-profile.children.cycles-pp.release_pages
1.22 ± 2% -1.2 0.04 ±105% perf-profile.children.cycles-pp.mod_objcg_state
1.05 ± 4% -1.0 0.08 ± 47% perf-profile.children.cycles-pp.alloc_pages_mpol
1.01 ± 4% -1.0 0.06 ± 67% perf-profile.children.cycles-pp.wp_page_copy
1.00 ± 5% -0.9 0.07 ± 44% perf-profile.children.cycles-pp.__alloc_pages
0.96 ± 2% -0.9 0.04 ±106% perf-profile.children.cycles-pp.wait4
0.84 ± 3% -0.8 0.04 ±104% perf-profile.children.cycles-pp.mm_init
0.81 ± 2% -0.8 0.03 ±104% perf-profile.children.cycles-pp.lock_vma_under_rcu
0.82 ± 2% -0.8 0.06 ± 69% perf-profile.children.cycles-pp.mas_wr_bnode
0.75 ± 3% -0.7 0.05 ± 84% perf-profile.children.cycles-pp.dup_task_struct
0.74 ± 2% -0.7 0.05 ± 62% perf-profile.children.cycles-pp.mas_split
0.74 ± 3% -0.7 0.06 ±122% perf-profile.children.cycles-pp.__cond_resched
0.66 ± 7% -0.6 0.03 ±101% perf-profile.children.cycles-pp.pte_alloc_one
0.59 ± 5% -0.6 0.03 ±101% perf-profile.children.cycles-pp.finish_task_switch
0.81 ± 3% -0.5 0.26 ± 22% perf-profile.children.cycles-pp.__schedule
0.61 ± 5% -0.5 0.10 ± 19% perf-profile.children.cycles-pp.select_task_rq_fair
0.58 ± 6% -0.5 0.08 ± 32% perf-profile.children.cycles-pp.find_idlest_cpu
0.48 ± 4% -0.4 0.04 ± 80% perf-profile.children.cycles-pp.alloc_thread_stack_node
0.49 ± 5% -0.4 0.07 ± 31% perf-profile.children.cycles-pp.find_idlest_group
0.48 ± 5% -0.4 0.06 ± 52% perf-profile.children.cycles-pp.update_sg_wakeup_stats
0.45 ± 7% -0.4 0.05 ± 80% perf-profile.children.cycles-pp.__pte_offset_map_lock
0.42 ± 2% -0.4 0.05 ± 60% perf-profile.children.cycles-pp.sync_regs
0.47 ± 4% -0.4 0.11 ± 42% perf-profile.children.cycles-pp.rcu_do_batch
0.57 ± 4% -0.4 0.21 ± 34% perf-profile.children.cycles-pp.rcu_core
0.46 ± 3% -0.3 0.15 ± 17% perf-profile.children.cycles-pp.schedule
0.34 ± 4% -0.3 0.04 ±107% perf-profile.children.cycles-pp.mas_wr_store_entry
0.29 ± 4% -0.2 0.05 ± 78% perf-profile.children.cycles-pp.pick_next_task_fair
0.26 ± 9% -0.2 0.06 ± 93% perf-profile.children.cycles-pp.update_load_avg
0.27 ± 5% -0.2 0.08 ± 49% perf-profile.children.cycles-pp.schedule_idle
0.41 ± 7% -0.2 0.22 ± 34% perf-profile.children.cycles-pp.copy_mc_enhanced_fast_string
0.21 ± 5% -0.2 0.04 ± 79% perf-profile.children.cycles-pp.clear_page_erms
0.20 ± 10% -0.2 0.05 ±121% perf-profile.children.cycles-pp.activate_task
0.24 ± 5% -0.1 0.09 ± 50% perf-profile.children.cycles-pp.perf_iterate_sb
0.20 ± 9% -0.1 0.05 ±108% perf-profile.children.cycles-pp.enqueue_task_fair
0.18 ± 4% -0.1 0.04 ± 80% perf-profile.children.cycles-pp.__vmalloc_node_range
0.24 ± 4% -0.1 0.11 ± 42% perf-profile.children.cycles-pp.newidle_balance
0.17 ± 10% -0.1 0.05 ±112% perf-profile.children.cycles-pp.enqueue_entity
0.13 ± 20% -0.1 0.03 ±101% perf-profile.children.cycles-pp.cmd_record
0.13 ± 20% -0.1 0.03 ±101% perf-profile.children.cycles-pp.__cmd_record
0.14 ± 5% -0.1 0.04 ± 83% perf-profile.children.cycles-pp._find_next_bit
0.13 ± 4% -0.1 0.06 ± 60% perf-profile.children.cycles-pp.__wake_up_sync_key
0.12 ± 5% -0.1 0.06 ± 60% perf-profile.children.cycles-pp.__wake_up_common
0.06 ± 12% +0.1 0.13 ± 29% perf-profile.children.cycles-pp.cpu_util
0.01 ±264% +0.1 0.09 ± 29% perf-profile.children.cycles-pp.do_anonymous_page
0.00 +0.1 0.10 ± 30% perf-profile.children.cycles-pp.path_lookupat
0.00 +0.1 0.10 ± 34% perf-profile.children.cycles-pp.strnlen_user
0.00 +0.1 0.10 ± 29% perf-profile.children.cycles-pp.filename_lookup
0.00 +0.1 0.10 ± 30% perf-profile.children.cycles-pp.evsel__read_counter
0.00 +0.1 0.11 ± 16% perf-profile.children.cycles-pp.evlist_cpu_iterator__next
0.00 +0.1 0.11 ± 21% perf-profile.children.cycles-pp.perf_event_mmap_event
0.00 +0.1 0.11 ± 21% perf-profile.children.cycles-pp.perf_event_mmap
0.00 +0.1 0.12 ± 34% perf-profile.children.cycles-pp.__x64_sys_vfork
0.00 +0.1 0.12 ± 39% perf-profile.children.cycles-pp.tick_nohz_stop_idle
0.00 +0.1 0.12 ± 36% perf-profile.children.cycles-pp.__libc_read
0.00 +0.1 0.12 ± 26% perf-profile.children.cycles-pp.pipe_read
0.00 +0.1 0.12 ± 40% perf-profile.children.cycles-pp.irqentry_exit
0.00 +0.1 0.12 ± 21% perf-profile.children.cycles-pp.__bitmap_and
0.00 +0.1 0.12 ± 63% perf-profile.children.cycles-pp.tsc_verify_tsc_adjust
0.00 +0.1 0.12 ± 47% perf-profile.children.cycles-pp.__open64_nocancel
0.00 +0.1 0.13 ± 18% perf-profile.children.cycles-pp.rb_insert_color
0.00 +0.1 0.13 ± 34% perf-profile.children.cycles-pp.readn
0.00 +0.1 0.13 ± 35% perf-profile.children.cycles-pp.copy_strings
0.00 +0.1 0.13 ± 32% perf-profile.children.cycles-pp.walk_component
0.00 +0.1 0.14 ± 37% perf-profile.children.cycles-pp.run_posix_cpu_timers
0.00 +0.1 0.14 ± 33% perf-profile.children.cycles-pp.pipe_write
0.00 +0.1 0.14 ± 39% perf-profile.children.cycles-pp.smpboot_thread_fn
0.00 +0.1 0.14 ± 35% perf-profile.children.cycles-pp.__vfork
0.00 +0.1 0.14 ± 39% perf-profile.children.cycles-pp.intel_idle_irq
0.00 +0.1 0.14 ± 26% perf-profile.children.cycles-pp.sched_clock_noinstr
0.00 +0.1 0.14 ± 38% perf-profile.children.cycles-pp.tick_check_broadcast_expired
0.00 +0.1 0.14 ± 27% perf-profile.children.cycles-pp.exec_mmap
0.00 +0.1 0.14 ± 58% perf-profile.children.cycles-pp.arch_cpu_idle_enter
0.00 +0.1 0.15 ± 29% perf-profile.children.cycles-pp.__split_vma
0.00 +0.2 0.15 ± 50% perf-profile.children.cycles-pp.timekeeping_advance
0.00 +0.2 0.15 ± 50% perf-profile.children.cycles-pp.update_wall_time
0.00 +0.2 0.15 ± 24% perf-profile.children.cycles-pp.__mmap
0.00 +0.2 0.15 ± 36% perf-profile.children.cycles-pp.ct_kernel_exit
0.04 ± 79% +0.2 0.19 ± 28% perf-profile.children.cycles-pp.irqtime_account_process_tick
0.09 ± 11% +0.2 0.24 ± 29% perf-profile.children.cycles-pp.__update_blocked_fair
0.00 +0.2 0.16 ± 33% perf-profile.children.cycles-pp.irqentry_enter
0.00 +0.2 0.16 ± 22% perf-profile.children.cycles-pp.tick_program_event
0.00 +0.2 0.16 ± 38% perf-profile.children.cycles-pp.check_cpu_stall
0.00 +0.2 0.16 ± 34% perf-profile.children.cycles-pp.rb_next
0.00 +0.2 0.16 ± 43% perf-profile.children.cycles-pp.setlocale
0.00 +0.2 0.17 ± 32% perf-profile.children.cycles-pp.write
0.00 +0.2 0.17 ± 22% perf-profile.children.cycles-pp.begin_new_exec
0.05 ± 38% +0.2 0.22 ± 37% perf-profile.children.cycles-pp.trigger_load_balance
0.00 +0.2 0.18 ± 30% perf-profile.children.cycles-pp._dl_addr
0.00 +0.2 0.18 ± 33% perf-profile.children.cycles-pp.intel_pmu_disable_all
0.00 +0.2 0.18 ± 29% perf-profile.children.cycles-pp.link_path_walk
0.00 +0.2 0.18 ± 24% perf-profile.children.cycles-pp.do_vmi_align_munmap
0.18 ± 13% +0.2 0.37 ± 18% perf-profile.children.cycles-pp.__libc_start_main
0.18 ± 13% +0.2 0.37 ± 18% perf-profile.children.cycles-pp.main
0.18 ± 13% +0.2 0.37 ± 18% perf-profile.children.cycles-pp.run_builtin
0.00 +0.2 0.19 ± 34% perf-profile.children.cycles-pp.hrtimer_forward
0.00 +0.2 0.19 ± 26% perf-profile.children.cycles-pp.do_vmi_munmap
0.00 +0.2 0.19 ± 20% perf-profile.children.cycles-pp.rb_erase
0.09 ± 11% +0.2 0.30 ± 21% perf-profile.children.cycles-pp.idle_cpu
0.00 +0.2 0.22 ± 30% perf-profile.children.cycles-pp.arch_cpu_idle_exit
0.00 +0.2 0.22 ± 23% perf-profile.children.cycles-pp.seq_read_iter
0.00 +0.2 0.24 ± 28% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.01 ±174% +0.3 0.27 ± 27% perf-profile.children.cycles-pp.ct_kernel_exit_state
0.00 +0.3 0.26 ± 29% perf-profile.children.cycles-pp.__collapse_huge_page_copy
0.01 ±173% +0.3 0.28 ± 22% perf-profile.children.cycles-pp.read_counters
0.00 +0.3 0.27 ± 33% perf-profile.children.cycles-pp._find_next_and_bit
0.03 ± 77% +0.3 0.31 ± 28% perf-profile.children.cycles-pp.ct_kernel_enter
0.00 +0.3 0.28 ± 29% perf-profile.children.cycles-pp.collapse_huge_page
0.00 +0.3 0.28 ± 43% perf-profile.children.cycles-pp.path_openat
0.00 +0.3 0.29 ± 42% perf-profile.children.cycles-pp.do_filp_open
0.00 +0.3 0.29 ± 29% perf-profile.children.cycles-pp.khugepaged_scan_mm_slot
0.00 +0.3 0.29 ± 29% perf-profile.children.cycles-pp.hpage_collapse_scan_pmd
0.00 +0.3 0.29 ± 17% perf-profile.children.cycles-pp.hrtimer_get_next_event
0.00 +0.3 0.30 ± 14% perf-profile.children.cycles-pp.call_cpuidle
0.00 +0.3 0.30 ± 29% perf-profile.children.cycles-pp.khugepaged
0.02 ±100% +0.3 0.32 ± 18% perf-profile.children.cycles-pp.cmd_stat
0.02 ±100% +0.3 0.32 ± 18% perf-profile.children.cycles-pp.dispatch_events
0.02 ±100% +0.3 0.32 ± 18% perf-profile.children.cycles-pp.process_interval
0.00 +0.3 0.30 ± 40% perf-profile.children.cycles-pp.do_sys_openat2
0.00 +0.3 0.31 ± 38% perf-profile.children.cycles-pp.__x64_sys_openat
0.04 ± 83% +0.3 0.37 ± 46% perf-profile.children.cycles-pp.tick_check_oneshot_broadcast_this_cpu
0.00 +0.3 0.32 ± 18% perf-profile.children.cycles-pp.sched_clock_idle_wakeup_event
0.08 ± 10% +0.3 0.41 ± 20% perf-profile.children.cycles-pp.rcu_pending
0.00 +0.3 0.34 ± 17% perf-profile.children.cycles-pp.ksys_mmap_pgoff
0.04 ± 58% +0.4 0.40 ± 22% perf-profile.children.cycles-pp.ct_idle_exit
0.05 ± 39% +0.4 0.41 ± 18% perf-profile.children.cycles-pp.should_we_balance
0.00 +0.4 0.36 ± 25% perf-profile.children.cycles-pp.hrtimer_update_next_event
0.00 +0.4 0.36 ± 19% perf-profile.children.cycles-pp.timerqueue_add
0.00 +0.4 0.38 ± 21% perf-profile.children.cycles-pp.mmap_region
0.58 ± 3% +0.4 0.96 ± 11% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.00 +0.4 0.40 ± 18% perf-profile.children.cycles-pp.timerqueue_del
0.00 +0.4 0.41 ± 21% perf-profile.children.cycles-pp.do_mmap
0.00 +0.4 0.42 ± 12% perf-profile.children.cycles-pp.read
0.03 ± 77% +0.4 0.46 ± 9% perf-profile.children.cycles-pp.vfs_read
0.00 +0.4 0.43 ± 23% perf-profile.children.cycles-pp.vm_mmap_pgoff
0.10 ± 6% +0.4 0.54 ± 22% perf-profile.children.cycles-pp.rcu_sched_clock_irq
0.04 ± 57% +0.5 0.49 ± 9% perf-profile.children.cycles-pp.ksys_read
0.00 +0.5 0.46 ± 17% perf-profile.children.cycles-pp.get_cpu_device
0.01 ±264% +0.5 0.47 ± 23% perf-profile.children.cycles-pp.update_rq_clock
0.00 +0.5 0.47 ± 18% perf-profile.children.cycles-pp.enqueue_hrtimer
0.22 ± 8% +0.5 0.69 ± 43% perf-profile.children.cycles-pp._raw_spin_trylock
0.00 +0.5 0.48 ± 53% perf-profile.children.cycles-pp.tick_nohz_irq_exit
0.00 +0.5 0.50 ± 24% perf-profile.children.cycles-pp.__hrtimer_next_event_base
0.03 ±100% +0.5 0.53 ± 18% perf-profile.children.cycles-pp.update_irq_load_avg
0.01 ±173% +0.5 0.52 ± 18% perf-profile.children.cycles-pp.load_elf_binary
0.00 +0.5 0.51 ± 20% perf-profile.children.cycles-pp.perf_pmu_nop_void
0.01 ±173% +0.5 0.53 ± 19% perf-profile.children.cycles-pp.exec_binprm
0.01 ±173% +0.5 0.53 ± 19% perf-profile.children.cycles-pp.search_binary_handler
0.18 ± 8% +0.5 0.70 ± 16% perf-profile.children.cycles-pp.update_rq_clock_task
0.01 ±173% +0.5 0.54 ± 15% perf-profile.children.cycles-pp.local_clock_noinstr
0.00 +0.5 0.55 ± 17% perf-profile.children.cycles-pp.x86_pmu_disable
0.00 +0.6 0.55 ± 19% perf-profile.children.cycles-pp.hrtimer_next_event_without
0.27 ± 4% +0.6 0.84 ± 11% perf-profile.children.cycles-pp.update_sg_lb_stats
0.00 +0.6 0.62 ± 13% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.01 ±264% +0.7 0.66 ± 15% perf-profile.children.cycles-pp.cpuidle_governor_latency_req
0.14 ± 8% +0.7 0.80 ± 8% perf-profile.children.cycles-pp.update_blocked_averages
0.00 +0.7 0.67 ± 27% perf-profile.children.cycles-pp.drm_atomic_helper_commit
0.00 +0.7 0.67 ± 27% perf-profile.children.cycles-pp.commit_tail
0.00 +0.7 0.67 ± 27% perf-profile.children.cycles-pp.ast_mode_config_helper_atomic_commit_tail
0.00 +0.7 0.67 ± 27% perf-profile.children.cycles-pp.drm_atomic_helper_commit_tail_rpm
0.00 +0.7 0.67 ± 27% perf-profile.children.cycles-pp.drm_atomic_helper_commit_planes
0.00 +0.7 0.67 ± 27% perf-profile.children.cycles-pp.ast_primary_plane_helper_atomic_update
0.00 +0.7 0.67 ± 27% perf-profile.children.cycles-pp.drm_fb_memcpy
0.00 +0.7 0.67 ± 27% perf-profile.children.cycles-pp.memcpy_toio
0.03 ±100% +0.7 0.70 ± 45% perf-profile.children.cycles-pp.bprm_execve
0.00 +0.7 0.67 ± 27% perf-profile.children.cycles-pp.drm_atomic_helper_dirtyfb
0.00 +0.7 0.67 ± 27% perf-profile.children.cycles-pp.drm_atomic_commit
0.14 ± 8% +0.7 0.83 ± 7% perf-profile.children.cycles-pp.run_rebalance_domains
0.19 ± 8% +0.7 0.91 ± 24% perf-profile.children.cycles-pp.worker_thread
0.15 ± 11% +0.7 0.88 ± 24% perf-profile.children.cycles-pp.process_one_work
0.00 +0.8 0.76 ± 25% perf-profile.children.cycles-pp.drm_fb_helper_damage_work
0.00 +0.8 0.76 ± 25% perf-profile.children.cycles-pp.drm_fbdev_generic_helper_fb_dirty
0.30 ± 3% +0.8 1.07 ± 6% perf-profile.children.cycles-pp.update_sd_lb_stats
0.30 ± 4% +0.8 1.15 ± 6% perf-profile.children.cycles-pp.find_busiest_group
0.06 ± 10% +0.9 0.93 ± 31% perf-profile.children.cycles-pp.execve
0.06 ± 10% +0.9 0.93 ± 31% perf-profile.children.cycles-pp.__x64_sys_execve
0.06 ± 10% +0.9 0.93 ± 31% perf-profile.children.cycles-pp.do_execveat_common
0.18 ± 41% +0.9 1.10 ± 31% perf-profile.children.cycles-pp.tick_sched_do_timer
0.07 ± 9% +1.0 1.05 ± 12% perf-profile.children.cycles-pp.irqtime_account_irq
0.06 ± 9% +1.1 1.13 ± 27% perf-profile.children.cycles-pp.tick_nohz_next_event
0.29 ± 25% +1.1 1.41 ± 14% perf-profile.children.cycles-pp.tick_irq_enter
0.29 ± 25% +1.2 1.46 ± 13% perf-profile.children.cycles-pp.irq_enter_rcu
0.09 ± 6% +1.2 1.28 ± 9% perf-profile.children.cycles-pp.sched_clock
0.26 ± 5% +1.2 1.46 ± 15% perf-profile.children.cycles-pp.kthread
0.46 ± 3% +1.4 1.83 ± 6% perf-profile.children.cycles-pp.load_balance
0.10 ± 16% +1.4 1.48 ± 11% perf-profile.children.cycles-pp.read_tsc
0.10 ± 4% +1.4 1.50 ± 6% perf-profile.children.cycles-pp.sched_clock_cpu
0.08 ± 9% +1.4 1.52 ± 10% perf-profile.children.cycles-pp.lapic_next_deadline
0.11 ± 6% +1.5 1.57 ± 10% perf-profile.children.cycles-pp.native_sched_clock
0.44 ± 3% +1.5 1.95 ± 9% perf-profile.children.cycles-pp.native_irq_return_iret
0.09 ± 11% +1.8 1.86 ± 16% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
0.10 ± 19% +1.8 1.88 ± 10% perf-profile.children.cycles-pp.perf_rotate_context
0.17 ± 18% +1.9 2.05 ± 64% perf-profile.children.cycles-pp.x86_64_start_kernel
0.17 ± 18% +1.9 2.05 ± 64% perf-profile.children.cycles-pp.x86_64_start_reservations
0.17 ± 18% +1.9 2.05 ± 64% perf-profile.children.cycles-pp.start_kernel
0.17 ± 18% +1.9 2.05 ± 64% perf-profile.children.cycles-pp.arch_call_rest_init
0.17 ± 18% +1.9 2.05 ± 64% perf-profile.children.cycles-pp.rest_init
0.11 ± 5% +2.0 2.07 ± 11% perf-profile.children.cycles-pp.clockevents_program_event
0.12 ± 23% +2.0 2.10 ± 15% perf-profile.children.cycles-pp.ktime_get_update_offsets_now
0.40 ± 4% +2.2 2.56 ± 13% perf-profile.children.cycles-pp.rebalance_domains
0.37 ± 21% +2.3 2.62 ± 23% perf-profile.children.cycles-pp.ktime_get
0.16 ± 19% +2.3 2.46 ± 19% perf-profile.children.cycles-pp.arch_scale_freq_tick
0.17 ± 11% +2.7 2.86 ± 11% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
1.17 ± 3% +3.1 4.30 ± 15% perf-profile.children.cycles-pp.__do_softirq
0.26 ± 2% +3.4 3.70 ± 9% perf-profile.children.cycles-pp.menu_select
1.27 ± 3% +3.8 5.08 ± 12% perf-profile.children.cycles-pp.irq_exit_rcu
0.55 ± 8% +10.6 11.17 ± 8% perf-profile.children.cycles-pp.__intel_pmu_enable_all
0.90 ± 6% +14.0 14.85 ± 6% perf-profile.children.cycles-pp.perf_adjust_freq_unthr_context
0.91 ± 5% +14.1 14.96 ± 7% perf-profile.children.cycles-pp.perf_event_task_tick
1.79 ± 4% +17.9 19.69 ± 5% perf-profile.children.cycles-pp.scheduler_tick
2.06 ± 3% +19.3 21.34 ± 4% perf-profile.children.cycles-pp.update_process_times
2.08 ± 3% +19.4 21.51 ± 4% perf-profile.children.cycles-pp.tick_sched_handle
2.33 ± 5% +21.1 23.39 ± 5% perf-profile.children.cycles-pp.tick_nohz_highres_handler
2.67 ± 4% +25.3 28.00 ± 4% perf-profile.children.cycles-pp.__hrtimer_run_queues
2.96 ± 4% +30.4 33.39 ± 3% perf-profile.children.cycles-pp.hrtimer_interrupt
2.97 ± 4% +30.5 33.48 ± 3% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
4.32 +31.3 35.60 ± 2% perf-profile.children.cycles-pp.intel_idle
4.60 ± 4% +36.7 41.33 ± 3% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
4.94 ± 4% +40.1 44.99 ± 3% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
9.05 +77.8 86.90 perf-profile.children.cycles-pp.cpuidle_enter_state
9.07 +78.2 87.24 perf-profile.children.cycles-pp.cpuidle_enter
9.82 +81.3 91.08 perf-profile.children.cycles-pp.start_secondary
9.43 +82.8 92.19 perf-profile.children.cycles-pp.cpuidle_idle_call
9.98 +83.1 93.12 perf-profile.children.cycles-pp.do_idle
9.98 +83.1 93.13 perf-profile.children.cycles-pp.secondary_startup_64_no_verify
9.98 +83.1 93.13 perf-profile.children.cycles-pp.cpu_startup_entry
8.81 ± 3% -8.2 0.60 ± 25% perf-profile.self.cycles-pp.next_uptodate_folio
7.11 -7.1 0.00 perf-profile.self.cycles-pp.anon_vma_interval_tree_insert
4.66 ± 2% -4.6 0.06 ± 88% perf-profile.self.cycles-pp._compound_head
4.46 -4.4 0.05 ± 80% perf-profile.self.cycles-pp.__slab_free
2.24 ± 5% -2.2 0.05 ± 91% perf-profile.self.cycles-pp.zap_pte_range
2.23 -2.2 0.04 ± 80% perf-profile.self.cycles-pp.page_remove_rmap
1.33 ± 4% -1.3 0.06 ± 71% perf-profile.self.cycles-pp.filemap_map_pages
1.33 ± 2% -1.3 0.07 ± 61% perf-profile.self.cycles-pp.kmem_cache_free
0.73 ± 3% -0.7 0.06 ± 40% perf-profile.self.cycles-pp.kmem_cache_alloc
0.44 ± 5% -0.4 0.06 ± 52% perf-profile.self.cycles-pp.update_sg_wakeup_stats
0.42 ± 2% -0.4 0.05 ± 60% perf-profile.self.cycles-pp.sync_regs
0.41 ± 6% -0.2 0.22 ± 34% perf-profile.self.cycles-pp.copy_mc_enhanced_fast_string
0.20 ± 6% -0.2 0.04 ± 79% perf-profile.self.cycles-pp.clear_page_erms
0.05 ± 43% +0.1 0.12 ± 26% perf-profile.self.cycles-pp.perf_event_task_tick
0.00 +0.1 0.07 ± 31% perf-profile.self.cycles-pp.find_busiest_group
0.00 +0.1 0.08 ± 21% perf-profile.self.cycles-pp.ct_idle_exit
0.00 +0.1 0.09 ± 50% perf-profile.self.cycles-pp.ct_kernel_exit
0.00 +0.1 0.09 ± 38% perf-profile.self.cycles-pp.strnlen_user
0.01 ±264% +0.1 0.11 ± 42% perf-profile.self.cycles-pp.cpu_util
0.00 +0.1 0.10 ± 53% perf-profile.self.cycles-pp.tsc_verify_tsc_adjust
0.00 +0.1 0.10 ± 49% perf-profile.self.cycles-pp.hrtimer_update_next_event
0.00 +0.1 0.11 ± 27% perf-profile.self.cycles-pp.rb_insert_color
0.00 +0.1 0.11 ± 39% perf-profile.self.cycles-pp.timerqueue_del
0.00 +0.1 0.12 ± 23% perf-profile.self.cycles-pp.__bitmap_and
0.00 +0.1 0.12 ± 36% perf-profile.self.cycles-pp.run_posix_cpu_timers
0.00 +0.1 0.12 ± 52% perf-profile.self.cycles-pp.hrtimer_next_event_without
0.00 +0.1 0.13 ± 42% perf-profile.self.cycles-pp.clockevents_program_event
0.00 +0.1 0.13 ± 41% perf-profile.self.cycles-pp.perf_mux_hrtimer_handler
0.00 +0.1 0.13 ± 45% perf-profile.self.cycles-pp.rcu_sched_clock_irq
0.03 ±100% +0.1 0.16 ± 40% perf-profile.self.cycles-pp.trigger_load_balance
0.00 +0.1 0.13 ± 44% perf-profile.self.cycles-pp.intel_idle_irq
0.00 +0.1 0.14 ± 30% perf-profile.self.cycles-pp.cpuidle_governor_latency_req
0.00 +0.1 0.14 ± 23% perf-profile.self.cycles-pp.update_blocked_averages
0.00 +0.1 0.14 ± 39% perf-profile.self.cycles-pp.tick_check_broadcast_expired
0.00 +0.1 0.14 ± 27% perf-profile.self.cycles-pp.tick_program_event
0.00 +0.1 0.14 ± 30% perf-profile.self.cycles-pp.tick_nohz_get_sleep_length
0.03 ± 78% +0.1 0.17 ± 26% perf-profile.self.cycles-pp.irqtime_account_process_tick
0.00 +0.1 0.14 ± 32% perf-profile.self.cycles-pp.rb_next
0.00 +0.1 0.15 ± 22% perf-profile.self.cycles-pp.ct_kernel_enter
0.00 +0.1 0.15 ± 34% perf-profile.self.cycles-pp._dl_addr
0.00 +0.2 0.15 ± 36% perf-profile.self.cycles-pp.check_cpu_stall
0.00 +0.2 0.16 ± 18% perf-profile.self.cycles-pp.rebalance_domains
0.00 +0.2 0.16 ± 38% perf-profile.self.cycles-pp.hrtimer_forward
0.00 +0.2 0.16 ± 34% perf-profile.self.cycles-pp.irq_exit_rcu
0.00 +0.2 0.16 ± 31% perf-profile.self.cycles-pp.get_next_timer_interrupt
0.00 +0.2 0.16 ± 26% perf-profile.self.cycles-pp.rb_erase
0.00 +0.2 0.16 ± 29% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.05 ± 39% +0.2 0.21 ± 18% perf-profile.self.cycles-pp.rcu_pending
0.00 +0.2 0.18 ± 54% perf-profile.self.cycles-pp.update_sd_lb_stats
0.00 +0.2 0.18 ± 33% perf-profile.self.cycles-pp.sched_clock_cpu
0.00 +0.2 0.20 ± 32% perf-profile.self.cycles-pp.__do_softirq
0.09 ± 11% +0.2 0.29 ± 21% perf-profile.self.cycles-pp.idle_cpu
0.00 +0.2 0.23 ± 30% perf-profile.self.cycles-pp.perf_pmu_nop_void
0.00 +0.2 0.24 ± 27% perf-profile.self.cycles-pp.timerqueue_add
0.01 ±174% +0.2 0.26 ± 26% perf-profile.self.cycles-pp.ct_kernel_exit_state
0.00 +0.3 0.25 ± 20% perf-profile.self.cycles-pp.hrtimer_interrupt
0.00 +0.3 0.26 ± 32% perf-profile.self.cycles-pp._find_next_and_bit
0.00 +0.3 0.28 ± 33% perf-profile.self.cycles-pp.irqtime_account_irq
0.00 +0.3 0.28 ± 19% perf-profile.self.cycles-pp.call_cpuidle
0.00 +0.3 0.29 ± 24% perf-profile.self.cycles-pp.perf_rotate_context
0.00 +0.3 0.30 ± 17% perf-profile.self.cycles-pp.sched_clock_idle_wakeup_event
0.04 ± 83% +0.3 0.35 ± 49% perf-profile.self.cycles-pp.tick_check_oneshot_broadcast_this_cpu
0.00 +0.3 0.32 ± 31% perf-profile.self.cycles-pp.cpuidle_enter
0.00 +0.3 0.33 ± 34% perf-profile.self.cycles-pp.tick_nohz_next_event
0.01 ±264% +0.3 0.34 ± 14% perf-profile.self.cycles-pp.scheduler_tick
0.22 ± 4% +0.4 0.58 ± 20% perf-profile.self.cycles-pp.update_sg_lb_stats
0.22 ± 6% +0.4 0.60 ± 27% perf-profile.self.cycles-pp._raw_spin_trylock
0.53 ± 3% +0.4 0.93 ± 10% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.00 +0.4 0.40 ± 18% perf-profile.self.cycles-pp.tick_irq_enter
0.00 +0.4 0.41 ± 25% perf-profile.self.cycles-pp.__hrtimer_next_event_base
0.01 ±174% +0.4 0.44 ± 33% perf-profile.self.cycles-pp.update_process_times
0.00 +0.4 0.44 ± 18% perf-profile.self.cycles-pp.get_cpu_device
0.00 +0.4 0.44 ± 21% perf-profile.self.cycles-pp.do_idle
0.00 +0.5 0.46 ± 16% perf-profile.self.cycles-pp.x86_pmu_disable
0.03 ±100% +0.5 0.52 ± 18% perf-profile.self.cycles-pp.update_irq_load_avg
0.00 +0.5 0.50 ± 9% perf-profile.self.cycles-pp.asm_sysvec_apic_timer_interrupt
0.01 ±264% +0.5 0.53 ± 27% perf-profile.self.cycles-pp.__hrtimer_run_queues
0.06 ± 12% +0.6 0.63 ± 20% perf-profile.self.cycles-pp.cpuidle_idle_call
0.00 +0.6 0.60 ± 28% perf-profile.self.cycles-pp.memcpy_toio
0.16 ± 45% +0.8 0.93 ± 38% perf-profile.self.cycles-pp.tick_sched_do_timer
0.12 ± 6% +1.0 1.11 ± 10% perf-profile.self.cycles-pp.menu_select
0.29 ± 27% +1.1 1.37 ± 48% perf-profile.self.cycles-pp.ktime_get
0.09 ± 15% +1.3 1.44 ± 12% perf-profile.self.cycles-pp.read_tsc
0.11 ± 5% +1.4 1.50 ± 9% perf-profile.self.cycles-pp.native_sched_clock
0.08 ± 9% +1.4 1.51 ± 10% perf-profile.self.cycles-pp.lapic_next_deadline
0.44 ± 3% +1.5 1.95 ± 9% perf-profile.self.cycles-pp.native_irq_return_iret
0.10 ± 25% +1.7 1.84 ± 17% perf-profile.self.cycles-pp.ktime_get_update_offsets_now
0.15 ± 18% +2.3 2.44 ± 19% perf-profile.self.cycles-pp.arch_scale_freq_tick
0.30 ± 6% +3.8 4.14 ± 6% perf-profile.self.cycles-pp.perf_adjust_freq_unthr_context
0.34 ± 5% +4.2 4.58 ± 6% perf-profile.self.cycles-pp.cpuidle_enter_state
0.55 ± 8% +10.6 11.17 ± 8% perf-profile.self.cycles-pp.__intel_pmu_enable_all
4.32 +31.3 35.58 ± 2% perf-profile.self.cycles-pp.intel_idleDisclaimer:
Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance.
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Quoted 8 lines · kernel test robot
Hello,
we reviewed this report and Fengwei (Cced) pointed out it could be the patch breaks functionality, then causes stress-ng netlink-proc performance drops dramatically.
Just FYI. Here is what I observed when running stress-ng.netlink-proc testing:
Whatever with/without the patch, cn_netlink_send_mult() returns -ESRCH in most case.
The following is what the cn_netlink_send_mult() returns when stress-ng.netlink-proc is running:
...
213801 213801 stress-ng-3 cn_netlink_send_mult -3 213801 213801 stress-ng-spawn cn_netlink_send_mult -3 213801 213801 stress-ng-spawn cn_netlink_send_mult -3 213801 213801 stress-ng-wait cn_netlink_send_mult -3 213802 213802 stress-ng-4 cn_netlink_send_mult -3 213802 213802 stress-ng-spawn cn_netlink_send_mult -3 213802 213802 stress-ng-spawn cn_netlink_send_mult -3 213802 213802 stress-ng-wait cn_netlink_send_mult -3 213803 213803 stress-ng-5 cn_netlink_send_mult -3 213803 213803 stress-ng-dead cn_netlink_send_mult -3 213803 213803 stress-ng-dead cn_netlink_send_mult -3 213802 213802 stress-ng-wait cn_netlink_send_mult -3 213801 213801 stress-ng-wait cn_netlink_send_mult -3 213800 213800 stress-ng-wait cn_netlink_send_mult -3 213799 213799 stress-ng-wait cn_netlink_send_mult -3 213798 213798 stress-ng-wait cn_netlink_send_mult -3 154697 154697 stress-ng cn_netlink_send_mult -3
...
Looks like it's not accurate to reset proc_event_num_listeners according to cn_netlink_send_mult() return value -3.
Regards
Yin, Fengwei
Hi Fengwei
Sorry, reply so late.
I don't think this will cause a drastic drop in stress-ng netlink-proc performance. Because after returning -ESRCH, proc_event_num_listeners is cleared and the send_msg function will not be called.
However, there is a problem with judging clearing based on whether the return value is -ESRCH. Because netlink_broadcast will return -ESRCH in this case. Can you try the following patch to solve your problem?
From 6e6c36aed156bbb185f54d0c0fef2f6683df3288 Mon Sep 17 00:00:00 2001
From: wangkeqi <[email protected]>
Date: Sat, 23 Dec 2023 13:21:17 +0800
Subject: [PATCH] connector: Fix proc_event_num_listeners count not cleared
When we register a cn_proc listening event, the proc_event_num_listener variable will be incremented by one, but if PROC_CN_MCAST_IGNORE is not called, the count will not decrease.
This will cause the proc_*_connector function to take the wrong path. It will reappear when the forkstat tool exits via ctrl + c. We solve this problem by determining whether
there are still listeners to clear proc_event_num_listener.
Signed-off-by wangkeqi <[email protected]>
--- drivers/connector/cn_proc.c | 5 ++++- drivers/connector/connector.c | 6 ++++++ include/linux/connector.h | 1 + 3 files changed, 11 insertions(+), 1 deletion(-)
drivers/connector/cn_proc.c +4 -1
@@ -108,8 +108,11 @@ static inline void send_msg(struct cn_msg *msg) filter_data[1 ] =0 ; } -cn_netlink_send_mult (msg, msg->len,0 ,CN_IDX_PROC ,GFP_NOWAIT , +if (netlink_has_listeners (get_cdev_nls (),CN_IDX_PROC )) +cn_netlink_send_mult (msg, msg->len,0 ,CN_IDX_PROC ,GFP_NOWAIT , cn_filter, (void *)filter_data); +else +atomic_set (&proc_event_num_listeners,0 );local_unlock (&local_event.lock); }
drivers/connector/connector.c +6 -0
@@ -120,6 +120,12 @@ int cn_netlink_send_mult(struct cn_msg *msg, u16 len, u32 portid, u32 __group, }EXPORT_SYMBOL_GPL (cn_netlink_send_mult); +struct sock *get_cdev_nls (void ) +{ +return cdev.nls; +} +EXPORT_SYMBOL_GPL (get_cdev_nls); +/* same as cn_netlink_send_mult except msg->len is used for len */ int cn_netlink_send (struct cn_msg *msg,u32 portid,u32 __group,gfp_t gfp_mask)
include/linux/connector.h +1 -0
@@ -126,6 +126,7 @@ int cn_netlink_send_mult(struct cn_msg *msg, u16 len, u32 portid,* If there are no listeners for given group %-ESRCH can be returned. */ int cn_netlink_send (struct cn_msg *msg,u32 portid,u32 group,gfp_t gfp_mask); +struct sock *get_cdev_nls (void ); int cn_queue_add_callback(struct cn_queue_dev *dev, const char *name, const struct cb_id *id,
--
2.27.0
在 2024/1/12 19:28，“Yin, Fengwei”<[email protected] <mailto:[email protected]>> 写入:
Quoted 8 lines · kernel test robot
Hello,
we reviewed this report and Fengwei (Cced) pointed out it could be the patch breaks functionality, then causes stress-ng netlink-proc performance drops dramatically.
Just FYI. Here is what I observed when running
stress-ng.netlink-proc testing:
Whatever with/without the patch, cn_netlink_send_mult() returns -ESRCH in most case.
The following is what the cn_netlink_send_mult() returns when stress-ng.netlink-proc is running:
...
213801 213801 stress-ng-3 cn_netlink_send_mult -3 213801 213801 stress-ng-spawn cn_netlink_send_mult -3 213801 213801 stress-ng-spawn cn_netlink_send_mult -3 213801 213801 stress-ng-wait cn_netlink_send_mult -3 213802 213802 stress-ng-4 cn_netlink_send_mult -3 213802 213802 stress-ng-spawn cn_netlink_send_mult -3 213802 213802 stress-ng-spawn cn_netlink_send_mult -3 213802 213802 stress-ng-wait cn_netlink_send_mult -3 213803 213803 stress-ng-5 cn_netlink_send_mult -3 213803 213803 stress-ng-dead cn_netlink_send_mult -3 213803 213803 stress-ng-dead cn_netlink_send_mult -3 213802 213802 stress-ng-wait cn_netlink_send_mult -3 213801 213801 stress-ng-wait cn_netlink_send_mult -3 213800 213800 stress-ng-wait cn_netlink_send_mult -3 213799 213799 stress-ng-wait cn_netlink_send_mult -3 213798 213798 stress-ng-wait cn_netlink_send_mult -3 154697 154697 stress-ng cn_netlink_send_mult -3
...
Looks like it's not accurate to reset proc_event_num_listeners according to cn_netlink_send_mult() return value -3.
Regards
Yin, Fengwei
Quoted 2 lines · 王珂琦 Keqi Wang Wang
Hi Fengwei
Sorry, reply so late.
No problem.
Quoted 2 lines
I don't think this will cause a drastic drop in stress-ng netlink-proc performance. Because after returning
-ESRCH,proc_event_num_listenersis cleared and thesend_msgfunction will not be called.
However, there is a problem with judging clearing based on whether the return value is-ESRCH. Becausenetlink_broadcastwill return-ESRCHin this case. Can you try the following patch to solve your problem?
Yes. This patch can make the regression of stress-ng.netlink-proc gone.
Regards
Yin, Fengwei
Quoted 132 lines
From
6e6c36aed156bbb185f54d0c0fef2f6683df3288Mon Sep 17 00:00:00 2001
From: wangkeqi <[email protected]>
Date: Sat, 23 Dec 2023 13:21:17 +0800
Subject: [PATCH] connector: Fixproc_event_num_listenerscount not clearedWhen we register a
cn_proclistening event, theproc_event_num_listenervariable will be incremented by one, but ifPROC_CN_MCAST_IGNOREis not called, the count will not decrease.
This will cause the proc_*_connector function to take the wrong path. It will reappear when the forkstat tool exits via ctrl + c. We solve this problem by determining whether
there are still listeners to clearproc_event_num_listener.Signed-off-by wangkeqi <[email protected]>--- drivers/connector/cn_proc.c | 5 ++++- drivers/connector/connector.c | 6 ++++++ include/linux/connector.h | 1 + 3 files changed, 11 insertions(+), 1 deletion(-)drivers/connector/cn_proc.c +4
-1@@ -108,8 +108,11 @@ static inline void send_msg(struct cn_msg *msg) filter_data[1 ] =0 ; } -cn_netlink_send_mult (msg, msg->len,0 ,CN_IDX_PROC ,GFP_NOWAIT , +if (netlink_has_listeners (get_cdev_nls (),CN_IDX_PROC )) +cn_netlink_send_mult (msg, msg->len,0 ,CN_IDX_PROC ,GFP_NOWAIT , cn_filter, (void *)filter_data); +else +atomic_set (&proc_event_num_listeners,0 );local_unlock (&local_event.lock); }drivers/connector/connector.c +6
-0@@ -120,6 +120,12 @@ int cn_netlink_send_mult(struct cn_msg *msg, u16 len, u32 portid, u32 __group, }EXPORT_SYMBOL_GPL (cn_netlink_send_mult); +struct sock *get_cdev_nls (void ) +{ +return cdev.nls; +} +EXPORT_SYMBOL_GPL (get_cdev_nls); +/* same as cn_netlink_send_mult except msg->len is used for len */ int cn_netlink_send (struct cn_msg *msg,u32 portid,u32 __group,gfp_t gfp_mask)include/linux/connector.h +1
-1@@ -126,6 +126,7 @@ int cn_netlink_send_mult(struct cn_msg *msg, u16 len, u32 portid,* If there are no listeners for given group %-ESRCH can be returned. */ int cn_netlink_send (struct cn_msg *msg,u32 portid,u32 group,gfp_t gfp_mask); +struct sock *get_cdev_nls (void ); int cn_queue_add_callback(struct cn_queue_dev *dev, const char *name, const struct cb_id *id, --2.27.0
在 2024/1/12 19:28，“Yin, Fengwei”<[email protected] <mailto:[email protected]>> 写入:
Quoted 8 lines · kernel test robot
Hello,
we reviewed this report and Fengwei (Cced) pointed out it could be the patch breaks functionality, then causes stress-ng netlink-proc performance drops dramatically.
Just FYI. Here is what I observed when running
stress-ng.netlink-proc testing:Whatever with/without the patch,
cn_netlink_send_mult()returns-ESRCHin most case.The following is what the
cn_netlink_send_mult()returns when stress-ng.netlink-proc is running:...
213801 213801 stress-ng-3cn_netlink_send_mult-3 213801 213801 stress-ng-spawncn_netlink_send_mult-3 213801 213801 stress-ng-spawncn_netlink_send_mult-3 213801 213801 stress-ng-waitcn_netlink_send_mult-3 213802 213802 stress-ng-4cn_netlink_send_mult-3 213802 213802 stress-ng-spawncn_netlink_send_mult-3 213802 213802 stress-ng-spawncn_netlink_send_mult-3 213802 213802 stress-ng-waitcn_netlink_send_mult-3 213803 213803 stress-ng-5cn_netlink_send_mult-3 213803 213803 stress-ng-deadcn_netlink_send_mult-3 213803 213803 stress-ng-deadcn_netlink_send_mult-3 213802 213802 stress-ng-waitcn_netlink_send_mult-3 213801 213801 stress-ng-waitcn_netlink_send_mult-3 213800 213800 stress-ng-waitcn_netlink_send_mult-3 213799 213799 stress-ng-waitcn_netlink_send_mult-3 213798 213798 stress-ng-waitcn_netlink_send_mult-3 154697 154697 stress-ngcn_netlink_send_mult-3
...Looks like it's not accurate to reset
proc_event_num_listenersaccording tocn_netlink_send_mult()return value -3.Regards
Yin, Fengwei