tcp: make TCP-AO lookups more predictable
Subsystems (4)
cover tcp: make TCP-AO lookups more predictable
This series fixes a TCP-AO key lookup precedence bug.
TCP-AO stores MKTs in an unsorted list and returns the first match. This allows newer, less-specific keys (wildcard VRF or shorter prefixes) to shadow older, more-specific keys if inserted later.
Fix this by implementing sorted insertion in tcp_ao_link_mkt() based on key specificity (VRF binding, then prefix length). This keeps the RX lookup path fast while ensuring correctness.
The second patch adds a selftest to verify this behavior.
Eric Dumazet (2):
tcp: fix TCP-AO key lookup precedence (shadowing) selftests/net: Add TCP-AO key shadowing test
net/ipv4/tcp_ao.c | 27 +++++- tools/testing/selftests/net/tcp_ao/Makefile | 1 + .../testing/selftests/net/tcp_ao/shadowing.c | 93 +++++++++++++++++++ 3 files changed, 120 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/net/tcp_ao/shadowing.c
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
PATCH 1/2 tcp: fix TCP-AO key lookup precedence (shadowing)
TCP-AO implementation stores Master Key Tuples (MKTs) in an unsorted doubly-linked list (ao_info->head) and inserts new keys at the head. When looking up a key, __tcp_ao_do_lookup() walks this list and returns the first match it finds.
Because the list is unsorted, a newer, less-specific key can shadow an older, more-specific key if it happens to be inserted later. This leads to incorrect key selection in two scenarios:
1. VRF Shadowing: A wildcard VRF key (not bound to an interface) added after a VRF-specific key will shadow the VRF-specific key for traffic arriving on that VRF.
2. Prefix Shadowing: A less-specific prefix key (e.g., /24) added after a more-specific prefix key (e.g., /32) will shadow the more-specific key during outbound connection establishment.
Unlike TCP MD5, which walks the entire list and evaluates the "best match" using better_md5_match(), TCP-AO expects the list order to determine precedence.
Fix this by implementing sorted insertion in tcp_ao_link_mkt(). Keys are inserted in descending order of specificity:
- VRF-bound keys take precedence over unbound keys.
- Longer prefix matches (LPM) take precedence over shorter ones.
This preserves the performance of the lockless RX lookup path (early return on first match) while ensuring correct precedence.
--- net/ipv4/tcp_ao.c | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-)
net/ipv4/tcp_ao.c +26 -1
@@ -341,9 +341,34 @@ static struct tcp_ao_info *tcp_ao_alloc_info(gfp_t flags)return ao; } +static bool tcp_ao_key_is_more_specific (const struct tcp_ao_key *a, +const struct tcp_ao_key *b) +{ +bool a_vrf = !!(a->keyflags &TCP_AO_KEYF_IFINDEX ); +bool b_vrf = !!(b->keyflags &TCP_AO_KEYF_IFINDEX ); + +if (a_vrf != b_vrf) +return a_vrf;/* VRF-bound is more specific */ + +return a->prefixlen > b->prefixlen;/* Longer prefix is more specific */ +} +static void tcp_ao_link_mkt (struct tcp_ao_info *ao,struct tcp_ao_key *mkt) { -hlist_add_head_rcu (&mkt->node, &ao->head); +struct tcp_ao_key *pos; +struct hlist_node *last =NULL ; + +hlist_for_each_entry (pos, &ao->head, node) { +if (tcp_ao_key_is_more_specific (mkt, pos)) { +hlist_add_before_rcu (&mkt->node, &pos->node); +return ; + } + last = &pos->node; + } +if (last) +hlist_add_behind_rcu (&mkt->node, last); +else +hlist_add_head_rcu (&mkt->node, &ao->head); } static struct tcp_ao_key *tcp_ao_copy_key(struct sock *sk,
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
PATCH 2/2 selftests/net: Add TCP-AO key shadowing test
In reply to Eric DumazetAdd a new selftest shadowing.c to tools/testing/selftests/net/tcp_ao to verify that more specific keys are correctly preferred over less specific ones (shadowing prevention), regardless of their insertion order.
The test configures a server with a specific host key, and a client with both a specific host key and a wildcard subnet key, inserted in the "wrong" order (wildcard last, which would shadow the specific one under the bug). It then verifies that the client can still successfully connect to the server, which only succeeds if the client correctly selects the more specific key for the outbound connection.
--- tools/testing/selftests/net/tcp_ao/Makefile | 1 + .../testing/selftests/net/tcp_ao/shadowing.c | 93 +++++++++++++++++++ 2 files changed, 94 insertions(+) create mode 100644 tools/testing/selftests/net/tcp_ao/shadowing.c
tools/testing/selftests/net/tcp_ao/Makefile +1 -0
@@ -2,6 +2,7 @@ TEST_BOTH_AF := bench-lookups TEST_BOTH_AF += connect TEST_BOTH_AF += connect-deny +TEST_BOTH_AF += shadowing TEST_BOTH_AF += icmps-accept icmps-discard TEST_BOTH_AF += key-management TEST_BOTH_AF += restore
tools/testing/selftests/net/tcp_ao/shadowing.c new file +93 -0
93 lines not on this page · raw message
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
Re: [PATCH net 2/2] selftests/net: Add TCP-AO key shadowing test
Hi Eric,
Thanks for adding a selftest,
Quoted 15 lines · Eric Dumazet
Add a new selftest
shadowing.ctotools/testing/selftests/net/tcp_aoto verify that more specific keys are correctly preferred over less specific ones (shadowing prevention), regardless of their insertion order.The test configures a server with a specific host key, and a client with both a specific host key and a wildcard subnet key, inserted in the "wrong" order (wildcard last, which would shadow the specific one under the bug). It then verifies that the client can still successfully connect to the server, which only succeeds if the client correctly selects the more specific key for the outbound connection.
Signed-off-by Eric Dumazet <[email protected]>Assisted-by Gemini:gemini-3.1-pro---
[..]
Quoted 14 lines
tools/testing/selftests/net/tcp_ao/shadowing.c new file +14
-0+static void *client_fn (void *arg) +{ +int sk =socket (test_family,SOCK_STREAM ,IPPROTO_TCP ); +union tcp_addr wildcard_addr = {}; + +if (sk <0 ) +test_error ("socket()" ); + +/* Client adds keys in the "wrong" order (wildcard last) to trigger shadowing. +* 1. Specific key (Key B, ID 100) +* 2. Wildcard key (Key A, ID 101) +* +* Without the fix, the wildcard key will be at the head of the list +* and will shadow the specific key during outbound lookup, causing
tools/testing/selftests/net/tcp_ao/shadowing.c line 65
Quoted 10 lines
tools/testing/selftests/net/tcp_ao/shadowing.c new file +10
-0+* the client to send a SYN with KeyID 101 (which the server doesn't have). +*/ + +/* 1. Add specific key */ +if (test_add_key (sk,"pass_specific" , this_ip_dest,-1 ,100 ,100 )) +test_error ("setsockopt(TCP_AO_ADD_KEY) specific" ); + +/* 2. Add wildcard key (any address, prefix 0) */ +if (test_add_key (sk,"pass_wildcard" , wildcard_addr,0 ,101 ,101 )) +test_error ("setsockopt(TCP_AO_ADD_KEY) wildcard" );
Two notes here:
1. The two keys do not match, so I likely misunderstood your cover letter. I thought you wanted to override more generic keys (longer prefix; non-VRF-specific), but your goal is to choose between two distinct keys, likely with different password-seeds.
2. This only changes behaviour on the initial connect() from a client, where the client did not provide any priority of which keys to use on SYN segment (by .set_current=1).
I think the patch is fine, yet it sounds like it's limited in usefulness, if I didn't miss something obvious.
Thanks,
Dmitry
Jakub Kicinski
maintainerQuoted 11 lines · On Mon, 22 Jun 2026 18:52:46 +0000 Eric Dumazet wrote:
This series fixes a TCP-AO key lookup precedence bug.
TCP-AO stores MKTs in an unsorted list and returns the first match. This allows newer, less-specific keys (wildcard VRF or shorter prefixes) to shadow older, more-specific keys if inserted later.
Fix this by implementing sorted insertion in
tcp_ao_link_mkt()based on key specificity (VRF binding, then prefix length). This keeps the RX lookup path fast while ensuring correctness.The second patch adds a selftest to verify this behavior.
Unhappiness:
Hi Eric,
Quoted 6 lines · Eric Dumazet
This series fixes a TCP-AO key lookup precedence bug.
TCP-AO stores MKTs in an unsorted list and returns the first match. This allows newer, less-specific keys (wildcard VRF or shorter prefixes) to shadow older, more-specific keys if inserted later.
Yeah, at this moment, TCP-AO doesn't allow any intersection of the keys:
If you have matching VRFs, matching keyids for matching peer/masks – then when the userspace tries to add the second key, setsockopt() is going to return -EKEYREJECTED/-EEXIST. This is quite different from TCP-MD5, where the most matching key is the one that's going to be used by the kernel.
This simplification (not allowing any key intersects) is mostly from a very permissive RFC5925, where MKT matches can be: ip-addr/mask; ip address ranges; wildcards of addresses; tcp ports. So, this part was intentionally simplified until there is a user who requires one of these things. And based on their requirements, a better data structure than a simple list could be used. Basically, the longest prefix match is like adding power-of-two ip ranges. Also, that's another reason why I wanted an extendable setsockopt(), where one can add new flags/fields to uAPI without breaking the existing users.
Anyways, if you have the requirement to have intersecting keys with bigger mask matching (imitating TCP-MD5 behaviour), we can do that, but I think that needs a new TCP_AO_KEYF_PREFIX_MATCH (or something of a kind). Then the keys with everything matching, but a prefix could be added to the socket, and the longest prefix match will be used.
I think one API decision should be documented straight away (besides the key flag) – how this flag works with multiple keys. Say there are 4 keys on a socket, all match the peer being connected:
keyA: ip 10.0.0.0 /8 (keyid = 100)
keyB: ip 10.0.0.0 /16 (keyid = 100)
keyC: ip 10.0.0.0 /8 (keyid = 101)
keyD: ip 10.0.0.0 /16 (keyid = 102)
So, keyA and keyB obviously will have to use this new TCP_AO_KEYF_PREFIX_MATCH. Should keyC or keyD be copied to the established connection socket or not?
I'd think the presence of TCP_AO_KEYF_PREFIX_MATCH flag on keyC&keyD should also affect whether they are copied or not. If the flag is not on keyC&keyD – they should be copied to the established socket (together with keyB, preserving the previous behaviour).
Otherwise, if they have the flag, what should happen?
1. keyB + keyC + keyD
2. keyB + keyD
If we go with (2), then if a user wants keyC on a socket, they could either remove TCP_AO_KEYF_PREFIX_MATCH from keyC or add keyC1 with mask /16 and the same password as keyC – slightly inconvenient, but quite flexible.
What do you think?
Quoted 19 lines
Fix this by implementing sorted insertion in
tcp_ao_link_mkt()based on key specificity (VRF binding, then prefix length). This keeps the RX lookup path fast while ensuring correctness.The second patch adds a selftest to verify this behavior.
Eric Dumazet (2):
tcp: fix TCP-AO key lookup precedence (shadowing) selftests/net: Add TCP-AO key shadowing testnet/ipv4/tcp_ao.c | 27 +++++- tools/testing/selftests/net/tcp_ao/Makefile | 1 + .../testing/selftests/net/tcp_ao/shadowing.c | 93 +++++++++++++++++++ 3 files changed, 120 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/net/tcp_ao/shadowing.c--
2.55.0.rc0.799.gd6f94ed593-goog
Signature · 1 line
Dmitry
Quoted 58 lines · Dmitry Safonov
Hi Eric,
Quoted 6 lines · Eric Dumazet
This series fixes a TCP-AO key lookup precedence bug.
TCP-AO stores MKTs in an unsorted list and returns the first match. This allows newer, less-specific keys (wildcard VRF or shorter prefixes) to shadow older, more-specific keys if inserted later.
Yeah, at this moment, TCP-AO doesn't allow any intersection of the keys:
If you have matching VRFs, matching keyids for matching peer/masks – then when the userspace tries to add the second key,setsockopt()is going to return -EKEYREJECTED/-EEXIST. This is quite different from TCP-MD5, where the most matching key is the one that's going to be used by the kernel.This simplification (not allowing any key intersects) is mostly from a very permissive RFC5925, where MKT matches can be: ip-addr/mask; ip address ranges; wildcards of addresses; tcp ports. So, this part was intentionally simplified until there is a user who requires one of these things. And based on their requirements, a better data structure than a simple list could be used. Basically, the longest prefix match is like adding power-of-two ip ranges. Also, that's another reason why I wanted an extendable
setsockopt(), where one can add new flags/fields to uAPI without breaking the existing users.Anyways, if you have the requirement to have intersecting keys with bigger mask matching (imitating TCP-MD5 behaviour), we can do that, but I think that needs a new
TCP_AO_KEYF_PREFIX_MATCH(or something of a kind). Then the keys with everything matching, but a prefix could be added to the socket, and the longest prefix match will be used.I think one API decision should be documented straight away (besides the key flag) – how this flag works with multiple keys. Say there are 4 keys on a socket, all match the peer being connected:
keyA: ip 10.0.0.0 /8 (keyid = 100)
keyB: ip 10.0.0.0 /16 (keyid = 100)
keyC: ip 10.0.0.0 /8 (keyid = 101)
keyD: ip 10.0.0.0 /16 (keyid = 102)So, keyA and keyB obviously will have to use this new
TCP_AO_KEYF_PREFIX_MATCH. Should keyC or keyD be copied to the established connection socket or not?
I'd think the presence ofTCP_AO_KEYF_PREFIX_MATCHflag on keyC&keyD should also affect whether they are copied or not. If the flag is not on keyC&keyD – they should be copied to the established socket (together with keyB, preserving the previous behaviour).Otherwise, if they have the flag, what should happen?
1. keyB + keyC + keyD
2. keyB + keyD
If we go with (2), then if a user wants keyC on a socket, they could either removeTCP_AO_KEYF_PREFIX_MATCHfrom keyC or add keyC1 with mask /16 and the same password as keyC – slightly inconvenient, but quite flexible.
Quoted 1 line
What do you think?
If intersecting keys are not yet allowed, I think we must return an error code at the insertion stage,
instead of hoping the user will do "the right thing".
Thanks.
Quoted 2 lines · Eric Dumazet
On Mon, Jun 22, 2026 at 6:13 PM Dmitry Safonov <[email protected]> wrote:
[..]
Quoted 5 lines
Quoted 1 line
What do you think?
If intersecting keys are not yet allowed, I think we must return an error code at the insertion stage,
instead of hoping the user will do "the right thing".
That is happenning already: in the new selftest you used different keyids, so that adds distinct keys and they both may be used/rotated on the connection (and have to be copied to the established socket). If you try adding two keys with different prefixes, but matching the same peer ip (same keyids; available in the same VRFs) – second setsockopt() will fail. There are tests for this in setsockopt-closed.c under duplicate_tests().
Thanks,
Dmitry