alpha/perf: Enable duration-based sampling throttling
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v2 Latest1 patch 1 replyIn review
- v11 patch 2 repliesSuperseded
Patches in v2 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/1 | alpha/perf: Account for time spent handling PMU interruptsarch/ | +33 | 1 reply |
Reviews trailers per patch
The patch has a Reviewed-by or Acked-by.
| Patch | Matt Turner |
|---|---|
| 1/1 | R-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH v2 0/1] alpha/perf: Enable duration-based sampling throttling
Hi,
Changes in v2, following Matt's review:
- Describe the unresponsive-machine/RCU-starvation symptom in the commit.
- Include proto.h for est_cycle_freq and use cycles_t without the cast.
- Precompute the cycles-to-nanoseconds multiplier/shift and mark both __ro_after_init, removing division from the interrupt path.
- Add Matt's Reviewed-by.
Both v2 mainline and ES40 integration builds passed (vmlinux, boot image and modules), with only the existing saved_config COMMON-symbol warning. Disassembly confirms no division call in either interrupt wrapper. The multiplier conversion checks cover rounding, overflow and RPCC wrap. The historical hardware results below apply to v1's division-based conversion. V2 also passed seven bounded tests on the ES40 with the alpha-devel integration: the 1024/s and 4096/s CPU0 controls, 1024/s controls on CPUs 1 and 2, and period-17 tests starting at 100000/s on each CPU. Duration feedback reduced the limits to 7168/s, 5120/s and 5120/s respectively. All 26156 samples were recorded without reported loss, and no new RCU stalls or kernel faults appeared, including a 30-second observation after the final run. This does not repeat the full v1 stress matrix below. The selected-frequency cleanup remains outside this patch.
While testing Matt's v7 ProfileMe series [1] on my EV67 AlphaServer ES40, I found that short-period sampling could make the machine unresponsive. Once serial logging was enabled, the console showed RCU grace-period kthread starvation reports. The failure also reproduced without my generic-entry follow-up patches.
Count-based throttling was working: at HZ=1024 and the default limit of 100000 samples/s, the observed cadence was approximately 98 interrupts per throttle cycle. However, Alpha never called perf_sample_event_took(), so perf could not reduce that limit based on interrupt handling cost. Setting perf_cpu_time_max_percent to 25 did not provide that feedback.
Keeping period 17 but lowering the limit to 1024 or 4096 samples/s made the tests pass while still exercising repeated stop/park/restart cycles. Adding duration feedback also made the original high-rate test pass, without changing those counter-state operations. The minimal fix measured average handler costs of 24-26 us against a 2.5 us budget and reduced the limit to 7168-8192 samples/s.
Reproducer
==========
The reproducer requires Matt's ProfileMe kernel series and matching perf tools. On the ES40 I used the following one-second workload, built natively as "busy":
#include <stdint.h>
#include <time.h>
static double now(void)
{
struct timespec t; clock_gettime(CLOCK_MONOTONIC, &t);
return t.tv_sec + t.tv_nsec * 1e-9;}
int main(void)
{
uint64_t x = 1;
double end = now() + 1; do {
for (int i = 0; i < 1000; i++)
asm volatile("mulq %0,3,%0" : "+r" (x));
} while (now() < end);
return 0;}
cc -O2 -o busy busy.c
As root, with serial capture running:
dmesg -n 7
sysctl -w kernel.perf_event_max_sample_rate=100000 sysctl -w kernel.perf_cpu_time_max_percent=25
The period-10007 control completed successfully:
taskset -c 0 perf record -B -R \
-e cpu/instructions,profileme,randomize/ -c 10007 \
-o control.data -- ./busyChanging only the period reproduced the failure:
taskset -c 0 perf record -B -R \
-e cpu/instructions,profileme,randomize/ -c 17 \
-o period17.data -- ./busyThe unpatched runs required a manual reset; a userspace timeout did not reliably recover the system. The commands above are the essential recording commands from the controlled tests, which also used a timeout supervisor and saved logs on another machine.
Validation and submission scope
===============================
The v1 minimal fix, without diagnostic instrumentation, passed three repeats of the high-rate reproducer and the two low-rate controls. Broader testing covered 21 cases on all three CPUs: randomized ProfileMe instruction sampling at periods 16, 17 and 31, ProfileMe and Aggregate cycle sampling, and concurrent ProfileMe and mixed-mode workloads. All completed, with 1352488 samples, no recorded losses and no new RCU stalls or kernel faults. This was bounded stress testing, not a long soak.
The missing feedback predates ProfileMe. This patch is therefore based on mainline and can be reviewed and applied independently of Matt's series. I cross-built vmlinux, the Alpha boot image and modules on mainline d48723711bae with the ES40 configuration and GCC 15.0.1. The only build warning was the existing saved_config COMMON-symbol warning.
The hardware results above used Matt's v7 series with the v1 fix applied; the mainline-only build has not been boot-tested. RPCC supplies a short, same-CPU interval because Alpha's tick-based sched_clock() is too coarse for this measurement. Counter programming and event handling are unchanged.
[1] https://lore.kernel.org/linux-alpha/[email protected]/
Magnus Lindholm (1):
alpha/perf: Account for time spent handling PMU interrupts
arch/alpha/kernel/perf_event.c | 35 ++++++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-)
Signature · 1 line
2.43.0