io_uring/futex: use GFP_KERNEL_ACCOUNT for futex data allocation
Subsystems (2)
Jens Axboe
maintainerPATCH 1/1 io_uring/futex: use GFP_KERNEL_ACCOUNT for futex data allocation
Be a bit nicer and ensure it plays nice with memcg accounting.
---
io_uring/futex.c +1 -1
@@ -186,7 +186,7 @@ int io_futexv_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe)return -EINVAL ; ifd = kzalloc(struct_size_t(struct io_futexv_data, futexv, iof->futex_nr), -GFP_KERNEL ); +GFP_KERNEL_ACCOUNT );if (!ifd)return -ENOMEM ;
Signature · 1 line
Jens Axboe
Hi Jens,
I tested 6e0d71c288fd against its direct parent on bare metal. In a narrow io_uring futex WAITV -> WAKE workload, the child was 8.09% slower. A separate 424-line standalone reproducer showed a 6.49% slowdown. A matched scalar WAIT -> WAKE control changed by -0.55%. All compared kernels actually ran with preempt=full.
The test system was a Core i7-12700KF with 32 GiB RAM. The workload was pinned to P-core CPU 2, with the governor and EPP set to performance, Turbo disabled, and GCC 15.2.0.
#regzbot introduced: 6e0d71c288fdcf5866f5d0c6cde850a091cc3c55 #regzbot title: io_uring futex WAITV accounted-allocation slowdown
This is a focused synthetic microbenchmark, not an application benchmark. It uses one raw-UAPI ring and eight independent wait vectors. Each vector has eight cacheline-separated private futex words. A timed cycle submits eight IORING_OP_FUTEX_WAITV requests, wakes element 3 in every vector, and validates all 16 CQEs. An untimed wake then verifies that the seven remaining waiters in each vector were removed.
I used a fresh boot for each point:
816095894c0f parent A -> 6e0d71c288fd child -> 816095894c0f parent B
Each point had 3 warm-up rounds and 15 measured rounds. Every measured round ran 512 cycles, or 4,096 WAITV/wake pairs. Results in ns/pair were:
implementation parent A child parent B child vs midpoint
formal 1045.687 1128.704 1042.755 +8.091% standalone 1040.270 1110.916 1046.198 +6.488%
The formal drop-first result was +8.080%, parent drift was -0.280%, and the maximum CV was 0.151%. The standalone drop-first result was +6.511%, with +0.570% parent drift. All 90 WAITV timing rows passed the CQE, returned-value, overflow, residual-waiter, and CPU checks. Untimed child traces also hit io_futexv_prep(), io_futexv_wait(), io_futexv_complete(), and the wake path with the expected request counts.
The exact source change is only GFP_KERNEL -> GFP_KERNEL_ACCOUNT for the per-WAITV data allocation. I understand the memcg-accounting purpose and am not suggesting a revert.
As a separate current-baseline diagnostic, I compared unmodified v7.2 against a direct child changing only this allocation back to GFP_KERNEL. The no-account child was 8.12% and 9.00% faster in the formal and standalone WAITV tests, while the scalar control changed by +0.09%. These values use a different baseline and are not combined with the direct-parent results above.
Is this per-WAITV cost an expected accounting trade-off, or could the same memcg accounting be retained with lower per-request overhead?
Evidence bundle:
Standalone reproducer:
Thanks,
Chengfeng