RDMA/rxe: reject non-8-byte ATOMIC_WRITE payloads
Subsystems (2)
PATCH 1/1 RDMA/rxe: reject non-8-byte ATOMIC_WRITE payloads
atomic_write_reply() at drivers/infiniband/sw/rxe/rxe_resp.c unconditionally dereferences 8 bytes at payload_addr(pkt):
value = *(u64 *)payload_addr(pkt);
check_rkey() previously accepted an ATOMIC_WRITE request with pktlen == resid == 0 because the length validation only compared pktlen against resid. A remote initiator that sets the RETH length to 0 therefore reaches atomic_write_reply() with a zero-byte logical payload, and the responder reads sizeof(u64) bytes from past the logical end of the packet into skb->head tailroom, then writes those 8 bytes into the attacker's MR via rxe_mr_do_atomic_write(). That is a remote disclosure of 4 bytes of kernel tailroom per probe (the other 4 bytes are the packet's own trailing ICRC).
IBA oA19-28 defines ATOMIC_WRITE as exactly 8 bytes. Anything else is protocol-invalid. Hoist a strict length check into check_rkey() so the responder never reaches the unchecked dereference, and keep the existing WRITE-family length logic for the normal RDMA WRITE path.
Reproduced on mainline with an unmodified rxe driver: a sustained zero-length ATOMIC_WRITE probe repeatedly leaks adjacent skb head-buffer bytes into the attacker's MR, including recognisable kernel strings and partial kernel-direct-map pointer words. With this patch applied the responder rejects the PDU and the MR stays all-zero.
---
Previously reported to security@ (2026-04-18); reposting publicly at the maintainer's request.
Per-probe evidence from a 100K-packet run on the clean unpatched tree at 9ca18fc915c6 (single attacker QP against a hairpin target QP over a veth pair; each probe one crafted zero-length ATOMIC_WRITE PDU):
transmitted packets: 100,000
observed MR writes: 48,575
non-zero leaked tails: 33,297 (68.55% of observed writes)
mostly-printable tails: 3,796 (7.81%)
fully-printable tails: 2,241 (4.61%)
unique non-zero tails: 22,220
Each probe is a fresh skb head-buffer allocation, so the 4 attacker-visible bytes after the ICRC are an independent sample of slab-adjacent memory. Content distribution across the 48,575 observed writes: 31.45% zero, 4.61% fully printable, 3.20% mostly printable, 12.06% header/sentinel- looking (08004500, 08004508, ffffffff, ...), 48.68% other binary. 80.9% of unique non-zero tails were singletons, so the leak is not dominated by one repeated value.
Representative printable fragments observed on the attacker side:
74 6f 70 2e "top."
66 72 65 65 "free"
45 78 65 63 "Exec"
2f 73 79 73 "/sys"
72 6f 6f 74 "root"
45 56 50 41 "EVPA"
43 4f 44 45 "CODE"Partial pointer-like recoveries (4-byte words ending in the kernel-direct-map prefix 0xffff....):
3,361 observations ending in ffff
1,364 unique ....ffff tails
most common:
81 88 ff ff LE 0xffff8881 1.68% of observed writes
80 88 ff ff LE 0xffff8880 0.22%The run did not recover full 64-bit kernel virtual addresses (only 4 bytes per probe are attacker-observable), but the partial pointer material is consistent with a KASLR-weakening primitive under sustained probing. With the fix applied, the same harness leaves the attacker MR all-zero.
--- drivers/infiniband/sw/rxe/rxe_resp.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
drivers/infiniband/sw/rxe/rxe_resp.c +13 -1
@@ -526,7 +526,19 @@ static enum resp_states check_rkey(struct rxe_qp *qp, } skip_check_range: -if (pkt->mask & (RXE_WRITE_MASK |RXE_ATOMIC_WRITE_MASK )) { +if (pkt->mask &RXE_ATOMIC_WRITE_MASK ) { +/* IBA oA19-28: ATOMIC_WRITE payload is exactly 8 bytes. +* Reject any other length before the responder reads +* sizeof(u64) bytes from payload_addr(pkt); a shorter +* payload would read past the logical end of the packet +* into skb->head tailroom. +*/ +if (resid !=sizeof (u64) || pktlen !=sizeof (u64) || +bth_pad (pkt)) { + state =RESPST_ERR_LENGTH ; + goto err; + } + }else if (pkt->mask &RXE_WRITE_MASK ) {if (resid > mtu) {if (pktlen != mtu ||bth_pad (pkt)) { state =RESPST_ERR_LENGTH ;
Signature · 1 line
2.53.0
Quoted 78 lines · Michael Bommarito
atomic_write_reply()atdrivers/infiniband/sw/rxe/rxe_resp.cunconditionally dereferences 8 bytes atpayload_addr(pkt):value = *(u64 *)payload_addr(pkt);
check_rkey()previously accepted anATOMIC_WRITErequest with pktlen == resid == 0 because the length validation only compared pktlen against resid. A remote initiator that sets the RETH length to 0 therefore reachesatomic_write_reply()with a zero-byte logical payload, and the responder reads sizeof(u64) bytes from past the logical end of the packet intoskb->headtailroom, then writes those 8 bytes into the attacker's MR viarxe_mr_do_atomic_write(). That is a remote disclosure of 4 bytes of kernel tailroom per probe (the other 4 bytes are the packet's own trailing ICRC).IBA oA19-28 defines
ATOMIC_WRITEas exactly 8 bytes. Anything else is protocol-invalid. Hoist a strict length check intocheck_rkey()so the responder never reaches the unchecked dereference, and keep the existing WRITE-family length logic for the normal RDMA WRITE path.Reproduced on mainline with an unmodified rxe driver: a sustained zero-length
ATOMIC_WRITEprobe repeatedly leaks adjacent skb head-buffer bytes into the attacker's MR, including recognisable kernel strings and partial kernel-direct-map pointer words. With this patch applied the responder rejects the PDU and the MR stays all-zero.Fixes 034e285f8b99 ("RDMA/rxe: Make responder support atomic write on RC service")Assisted-by Claude:claude-opus-4-7Signed-off-by Michael Bommarito <[email protected]>---
Previously reported to security@ (2026-04-18); reposting publicly at the maintainer's request.Per-probe evidence from a 100K-packet run on the clean unpatched tree at
9ca18fc915c6(single attacker QP against a hairpin target QP over a veth pair; each probe one crafted zero-lengthATOMIC_WRITEPDU):transmitted packets: 100,000
observed MR writes: 48,575
non-zero leaked tails: 33,297 (68.55% of observed writes)
mostly-printable tails: 3,796 (7.81%)
fully-printable tails: 2,241 (4.61%)
unique non-zero tails: 22,220Each probe is a fresh skb head-buffer allocation, so the 4 attacker-visible bytes after the ICRC are an independent sample of slab-adjacent memory. Content distribution across the 48,575 observed writes: 31.45% zero, 4.61% fully printable, 3.20% mostly printable, 12.06% header/sentinel- looking (08004500, 08004508, ffffffff, ...), 48.68% other binary. 80.9% of unique non-zero tails were singletons, so the leak is not dominated by one repeated value.
Representative printable fragments observed on the attacker side:
74 6f 70 2e "top." 66 72 65 65 "free" 45 78 65 63 "Exec" 2f 73 79 73 "/sys" 72 6f 6f 74 "root" 45 56 50 41 "EVPA" 43 4f 44 45 "CODE"Partial pointer-like recoveries (4-byte words ending in the kernel-direct-map prefix
0xffff....):3,361 observations ending in ffff 1,364 unique ....ffff tails most common: 81 88 ff ff LE 0xffff8881 1.68% of observed writes 80 88 ff ff LE 0xffff8880 0.22%
Quoted 5 lines
The run did not recover full 64-bit kernel virtual addresses (only 4 bytes per probe are attacker-observable), but the partial pointer material is consistent with a KASLR-weakening primitive under sustained probing. With the fix applied, the same harness leaves the attacker MR all-zero.
Thanks a lot. It would be great to have a corresponding negative test in tools/testing/selftests/rdma that sends malformed ATOMIC_WRITE requests (e.g., zero-length) and verifies that they are rejected and do not modify the target MR.
It is up to you. To this commit, I am fine with it.
Zhu Yanjun
Quoted 29 lines
--- drivers/infiniband/sw/rxe/rxe_resp.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)drivers/infiniband/sw/rxe/rxe_resp.c +13
-1@@ -526,7 +526,19 @@ static enum resp_states check_rkey(struct rxe_qp *qp, } skip_check_range: -if (pkt->mask & (RXE_WRITE_MASK |RXE_ATOMIC_WRITE_MASK )) { +if (pkt->mask &RXE_ATOMIC_WRITE_MASK ) { +/* IBA oA19-28: ATOMIC_WRITE payload is exactly 8 bytes. +* Reject any other length before the responder reads +* sizeof(u64) bytes from payload_addr(pkt); a shorter +* payload would read past the logical end of the packet +* into skb->head tailroom. +*/ +if (resid !=sizeof (u64) || pktlen !=sizeof (u64) || +bth_pad (pkt)) { + state =RESPST_ERR_LENGTH ; + goto err; + } + }else if (pkt->mask &RXE_WRITE_MASK ) {if (resid > mtu) {if (pktlen != mtu ||bth_pad (pkt)) { state =RESPST_ERR_LENGTH ;
Quoted 4 lines · Zhu Yanjun
Thanks a lot. It would be great to have a corresponding negative test in
tools/testing/selftests/rdmathat sends malformedATOMIC_WRITErequests (e.g., zero-length) and verifies that they are rejected and do not modify the target MR.
Good idea. Do you want a v2 with this fix + a separate test case in rdma or should I submit it separately?
Thanks,
Mike Bommarito
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 3 lines · Zhu Yanjun
The fix should stand on its own, and the test case can be added as a follow-up patch under
tools/testing/selftests/rdma(or rdma-core if more appropriate). This keeps each patch focused on a single logical change and makes review easier.
It seems like mainlining rdma tests might not be a good idea. I found the last time here:
Kamal Heib, 23 Oct 2019 — [PATCH for-next] selftests: rdma: Add rdma tests
I put the tests here and will PR it into linux-rdma/rdma-core tomorrow:
https://github.com/mjbommar/rdma-core/blob/4104d991a764de4aad9f645a2f0ec723f6076209/tests/test_rxe_atomic_write_oob.py
Thanks,
Mike Bommarito
Quoted 10 lines · Michael Bommarito
Quoted 3 lines · Zhu Yanjun
The fix should stand on its own, and the test case can be added as a follow-up patch under
tools/testing/selftests/rdma(or rdma-core if more appropriate). This keeps each patch focused on a single logical change and makes review easier.It seems like mainlining rdma tests might not be a good idea. I found the last time here:
Kamal Heib, 23 Oct 2019 — [PATCH for-next] selftests: rdma: Add rdma testsI put the tests here and will PR it into linux-rdma/rdma-core tomorrow:
https://github.com/mjbommar/rdma-core/blob/4104d991a764de4aad9f645a2f0ec723f6076209/tests/test_rxe_atomic_write_oob.py
Thanks a lot. I am fine with the above link.
Zhu Yanjun
Quoted 3 lines
Thanks,
Mike Bommarito
Signature · 2 lines
Best Regards, Yanjun.Zhu
Quoted 29 lines · Michael Bommarito
atomic_write_reply()atdrivers/infiniband/sw/rxe/rxe_resp.cunconditionally dereferences 8 bytes atpayload_addr(pkt):value = *(u64 *)payload_addr(pkt);
check_rkey()previously accepted anATOMIC_WRITErequest with pktlen == resid == 0 because the length validation only compared pktlen against resid. A remote initiator that sets the RETH length to 0 therefore reachesatomic_write_reply()with a zero-byte logical payload, and the responder reads sizeof(u64) bytes from past the logical end of the packet intoskb->headtailroom, then writes those 8 bytes into the attacker's MR viarxe_mr_do_atomic_write(). That is a remote disclosure of 4 bytes of kernel tailroom per probe (the other 4 bytes are the packet's own trailing ICRC).IBA oA19-28 defines
ATOMIC_WRITEas exactly 8 bytes. Anything else is protocol-invalid. Hoist a strict length check intocheck_rkey()so the responder never reaches the unchecked dereference, and keep the existing WRITE-family length logic for the normal RDMA WRITE path.Reproduced on mainline with an unmodified rxe driver: a sustained zero-length
ATOMIC_WRITEprobe repeatedly leaks adjacent skb head-buffer bytes into the attacker's MR, including recognisable kernel strings and partial kernel-direct-map pointer words. With this patch applied the responder rejects the PDU and the MR stays all-zero.
Quoted 6 lines
Fixes 034e285f8b99 ("RDMA/rxe: Make responder support atomic write on RC service")Assisted-by Claude:claude-opus-4-7Signed-off-by Michael Bommarito <[email protected]>Reviewed-by Zhu Yanjun <[email protected]>---
Applied to for-rc thanks
Jason