Skip to contentSkip to conversation

io_uring/futex: use GFP_KERNEL_ACCOUNT for futex data allocation

PATCH In review Jens Axboe 2 participants 2 messages Jan 25 – Aug 30

Subsystems (2)
lore
Share & tools
lore

Jens Axboe

author maintainer
#0 lore raw

PATCH 1/1 io_uring/futex: use GFP_KERNEL_ACCOUNT for futex data allocation

Be a bit nicer and ensure it plays nice with memcg accounting.

---

io_uring/futex.c +1 -1
index 11bfff5a80df..1dabcfd503b8 100644
@@ -186,7 +186,7 @@ int io_futexv_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)
 		return -EINVAL;
 
 	ifd = kzalloc(struct_size_t(struct io_futexv_data, futexv, iof->futex_nr),
-			GFP_KERNEL);
+			GFP_KERNEL_ACCOUNT);
 	if (!ifd)
 		return -ENOMEM;
 
Signature · 1 line
Jens Axboe
#1 lore raw

Hi Jens,

I tested 6e0d71c288fd against its direct parent on bare metal. In a narrow io_uring futex WAITV -> WAKE workload, the child was 8.09% slower. A separate 424-line standalone reproducer showed a 6.49% slowdown. A matched scalar WAIT -> WAKE control changed by -0.55%. All compared kernels actually ran with preempt=full.

The test system was a Core i7-12700KF with 32 GiB RAM. The workload was pinned to P-core CPU 2, with the governor and EPP set to performance, Turbo disabled, and GCC 15.2.0.

#regzbot introduced: 6e0d71c288fdcf5866f5d0c6cde850a091cc3c55 #regzbot title: io_uring futex WAITV accounted-allocation slowdown

This is a focused synthetic microbenchmark, not an application benchmark. It uses one raw-UAPI ring and eight independent wait vectors. Each vector has eight cacheline-separated private futex words. A timed cycle submits eight IORING_OP_FUTEX_WAITV requests, wakes element 3 in every vector, and validates all 16 CQEs. An untimed wake then verifies that the seven remaining waiters in each vector were removed.

I used a fresh boot for each point:

816095894c0f parent A -> 6e0d71c288fd child -> 816095894c0f parent B

Each point had 3 warm-up rounds and 15 measured rounds. Every measured round ran 512 cycles, or 4,096 WAITV/wake pairs. Results in ns/pair were:

implementation parent A child parent B child vs midpoint

  formal             1045.687  1128.704   1042.755        +8.091%
  standalone         1040.270  1110.916   1046.198        +6.488%

The formal drop-first result was +8.080%, parent drift was -0.280%, and the maximum CV was 0.151%. The standalone drop-first result was +6.511%, with +0.570% parent drift. All 90 WAITV timing rows passed the CQE, returned-value, overflow, residual-waiter, and CPU checks. Untimed child traces also hit io_futexv_prep(), io_futexv_wait(), io_futexv_complete(), and the wake path with the expected request counts.

The exact source change is only GFP_KERNEL -> GFP_KERNEL_ACCOUNT for the per-WAITV data allocation. I understand the memcg-accounting purpose and am not suggesting a revert.

As a separate current-baseline diagnostic, I compared unmodified v7.2 against a direct child changing only this allocation back to GFP_KERNEL. The no-account child was 8.12% and 9.00% faster in the formal and standalone WAITV tests, while the scalar control changed by +0.09%. These values use a different baseline and are not combined with the direct-parent results above.

Is this per-WAITV cost an expected accounting trade-off, or could the same memcg accounting be retained with lower per-request overhead?

Evidence bundle:

https://github.com/lcf0399/linux-regression-evidence/tree/25ed417f566fc83b942e1787a0b53d555ccec291/io-uring-futex-waitv-accounted-allocation

Standalone reproducer:

https://github.com/lcf0399/linux-regression-evidence/tree/25ed417f566fc83b942e1787a0b53d555ccec291/io-uring-futex-waitv-accounted-allocation/reproducer

Thanks,
Chengfeng