Skip to contentSkip to conversation

Add support for externally validated neighbor entries

PATCH Applied Ido Schimmel 4 participants 7 messages Jun 26, 2025 – Jul 29, 2025

Subsystems (5)
lore
Share & tools
lore

Ido Schimmel

author maintainer
#0 lore raw

v2 cover Add support for externally validated neighbor entries

Patch #1 adds a new neighbor flag ("extern_valid") that prevents the kernel from invalidating or removing a neighbor entry, while allowing the kernel to notify user space when the entry becomes reachable. See motivation and implementation details in the commit message.

Patch #2 adds a selftest.

Changes since v1 [1]:

* Patch #1: s/neigbor/neighbor/ in comment.
* Patch #1: Do not flush "extern_valid" entries upon carrier down.
* Patch #1: Reword some parts of the commit message.
* Patch #2: Fix various shellcheck warnings.
* Patch #2: Add tests for interface and carrier down.

[1] https://lore.kernel.org/netdev/[email protected]/

Ido Schimmel (2):
neighbor: Add NTF_EXT_VALIDATED flag for externally validated entries
selftests: net: Add a selftest for externally validated neighbor
entries

 Documentation/netlink/specs/rt-neigh.yaml |   1 +
 include/net/neighbour.h                   |   4 +-
 include/uapi/linux/neighbour.h            |   5 +
 net/core/neighbour.c                      |  79 ++++-
 tools/testing/selftests/net/Makefile      |   1 +
 tools/testing/selftests/net/test_neigh.sh | 366 ++++++++++++++++++++++
 6 files changed, 445 insertions(+), 11 deletions(-)
 create mode 100755 tools/testing/selftests/net/test_neigh.sh
Signature · 1 line
2.49.0
author maintainer
#1 lore raw

PATCH v2 1/2 neighbor: Add NTF_EXT_VALIDATED flag for externally validated entries

tl;dr
=====

Add a new neighbor flag ("extern_valid") that can be used to indicate to the kernel that a neighbor entry was learned and determined to be valid externally. The kernel will not try to remove or invalidate such an entry, leaving these decisions to the user space control plane. This is needed for EVPN multi-homing where a neighbor entry for a multi-homed host needs to be synced across all the VTEPs among which the host is multi-homed.

Background
==========

In a typical EVPN multi-homing setup each host is multi-homed using a set of links called ES (Ethernet Segment, i.e., LAG) to multiple leaf switches (VTEPs). VTEPs that are connected to the same ES are called ES peers.

When a neighbor entry is learned on a VTEP, it is distributed to both ES peers and remote VTEPs using EVPN MAC/IP advertisement routes. ES peers use the neighbor entry when routing traffic towards the multi-homed host and remote VTEPs use it for ARP/NS suppression.

Motivation
==========

If the ES link between a host and the VTEP on which the neighbor entry was locally learned goes down, the EVPN MAC/IP advertisement route will be withdrawn and the neighbor entries will be removed from both ES peers and remote VTEPs. Routing towards the multi-homed host and ARP/NS suppression can fail until another ES peer locally learns the neighbor entry and distributes it via an EVPN MAC/IP advertisement route.

"draft-rbickhart-evpn-ip-mac-proxy-adv-03" [1] suggests avoiding these intermittent failures by having the ES peers install the neighbor entries as before, but also injecting EVPN MAC/IP advertisement routes with a proxy indication. When the previously mentioned ES link goes down and the original EVPN MAC/IP advertisement route is withdrawn, the ES peers will not withdraw their neighbor entries, but instead start aging timers for the proxy indication.

If an ES peer locally learns the neighbor entry (i.e., it becomes "reachable"), it will restart its aging timer for the entry and emit an EVPN MAC/IP advertisement route without a proxy indication. An ES peer will stop its aging timer for the proxy indication if it observes the removal of the proxy indication from at least one of the ES peers advertising the entry.

In the event that the aging timer for the proxy indication expired, an ES peer will withdraw its EVPN MAC/IP advertisement route. If the timer expired on all ES peers and they all withdrew their proxy advertisements, the neighbor entry will be completely removed from the EVPN fabric.

Implementation
==============

In the above scheme, when the control plane (e.g., FRR) advertises a neighbor entry with a proxy indication, it expects the corresponding entry in the data plane (i.e., the kernel) to remain valid and not be removed due to garbage collection or loss of carrier. The control plane also expects the kernel to notify it if the entry was learned locally (i.e., became "reachable") so that it will remove the proxy indication from the EVPN MAC/IP advertisement route. That is why these entries cannot be programmed with dummy states such as "permanent" or "noarp".

Instead, add a new neighbor flag ("extern_valid") which indicates that the entry was learned and determined to be valid externally and should not be removed or invalidated by the kernel. The kernel can probe the entry and notify user space when it becomes "reachable" (it is initially installed as "stale"). However, if the kernel does not receive a confirmation, have it return the entry to the "stale" state instead of the "failed" state.

In other words, an entry marked with the "extern_valid" flag behaves like any other dynamically learned entry other than the fact that the kernel cannot remove or invalidate it.

One can argue that the "extern_valid" flag should not prevent garbage collection and that instead a neighbor entry should be programmed with both the "extern_valid" and "extern_learn" flags. There are two reasons for not doing that:

1. Unclear why a control plane would like to program an entry that the kernel cannot invalidate but can completely remove.

2. The "extern_learn" flag is used by FRR for neighbor entries learned on remote VTEPs (for ARP/NS suppression) whereas here we are concerned with local entries. This distinction is currently irrelevant for the kernel, but might be relevant in the future.

Given that the flag only makes sense when the neighbor has a valid state, reject attempts to add a neighbor with an invalid state and with this flag set. For example:

 # ip neigh add 192.0.2.1 nud none dev br0.10 extern_valid
 Error: Cannot create externally validated neighbor with an invalid state.
 # ip neigh add 192.0.2.1 lladdr 00:11:22:33:44:55 nud stale dev br0.10 extern_valid
 # ip neigh replace 192.0.2.1 nud failed dev br0.10 extern_valid
 Error: Cannot mark neighbor as externally validated with an invalid state.

The above means that a neighbor cannot be created with the "extern_valid" flag and flags such as "use" or "managed" as they result in a neighbor being created with an invalid state ("none") and immediately getting probed:

 # ip neigh add 192.0.2.1 lladdr 00:11:22:33:44:55 nud stale dev br0.10 extern_valid use
 Error: Cannot create externally validated neighbor with an invalid state.

However, these flags can be used together with "extern_valid" after the neighbor was created with a valid state:

 # ip neigh add 192.0.2.1 lladdr 00:11:22:33:44:55 nud stale dev br0.10 extern_valid
 # ip neigh replace 192.0.2.1 lladdr 00:11:22:33:44:55 nud stale dev br0.10 extern_valid use

One consequence of preventing the kernel from invalidating a neighbor entry is that by default it will only try to determine reachability using unicast probes. This can be changed using the "mcast_resolicit" sysctl:

 # sysctl net.ipv4.neigh.br0/10.mcast_resolicit
 0
 # tcpdump -nn -e -i br0.10 -Q out arp &
 # ip neigh replace 192.0.2.1 lladdr 00:11:22:33:44:55 nud stale dev br0.10 extern_valid use
 62:50:1d:11:93:6f > 00:11:22:33:44:55, ethertype ARP (0x0806), length 42: Request who-has 192.0.2.1 tell 192.0.2.2, length 28
 62:50:1d:11:93:6f > 00:11:22:33:44:55, ethertype ARP (0x0806), length 42: Request who-has 192.0.2.1 tell 192.0.2.2, length 28
 62:50:1d:11:93:6f > 00:11:22:33:44:55, ethertype ARP (0x0806), length 42: Request who-has 192.0.2.1 tell 192.0.2.2, length 28
 # sysctl -wq net.ipv4.neigh.br0/10.mcast_resolicit=3
 # ip neigh replace 192.0.2.1 lladdr 00:11:22:33:44:55 nud stale dev br0.10 extern_valid use
 62:50:1d:11:93:6f > 00:11:22:33:44:55, ethertype ARP (0x0806), length 42: Request who-has 192.0.2.1 tell 192.0.2.2, length 28
 62:50:1d:11:93:6f > 00:11:22:33:44:55, ethertype ARP (0x0806), length 42: Request who-has 192.0.2.1 tell 192.0.2.2, length 28
 62:50:1d:11:93:6f > 00:11:22:33:44:55, ethertype ARP (0x0806), length 42: Request who-has 192.0.2.1 tell 192.0.2.2, length 28
 62:50:1d:11:93:6f > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.0.2.1 tell 192.0.2.2, length 28
 62:50:1d:11:93:6f > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.0.2.1 tell 192.0.2.2, length 28
 62:50:1d:11:93:6f > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.0.2.1 tell 192.0.2.2, length 28

iproute2 patches can be found here [2].

[1] https://datatracker.ietf.org/doc/html/draft-rbickhart-evpn-ip-mac-proxy-adv-03
[2] https://github.com/idosch/iproute2/tree/submit/extern_valid_v1

---

Notes:
v2:

* s/neigbor/neighbor/ in comment.
* Do not flush "extern_valid" entries upon carrier down.
* Reword some parts of the commit message.

 Documentation/netlink/specs/rt-neigh.yaml |  1 +
 include/net/neighbour.h                   |  4 +-
 include/uapi/linux/neighbour.h            |  5 ++
 net/core/neighbour.c                      | 79 ++++++++++++++++++++---
 4 files changed, 78 insertions(+), 11 deletions(-)
Documentation/netlink/specs/rt-neigh.yaml +1 -0
index 25cc2d528d2f..30a9ee16f128 100644
@@ -79,6 +79,7 @@ definitions:
     entries:
       - managed
       - locked
+      - ext-validated
   -
     name: rtm-type
     type: enum
include/net/neighbour.h +3 -1
index c7ce5ec7be23..7e865b14749d 100644
@@ -261,13 +261,15 @@ static inline void *neighbour_priv(const struct neighbour *n)
 #define NEIGH_UPDATE_F_EXT_LEARNED		BIT(5)
 #define NEIGH_UPDATE_F_ISROUTER			BIT(6)
 #define NEIGH_UPDATE_F_ADMIN			BIT(7)
+#define NEIGH_UPDATE_F_EXT_VALIDATED		BIT(8)
 
 /* In-kernel representation for NDA_FLAGS_EXT flags: */
 #define NTF_OLD_MASK		0xff
 #define NTF_EXT_SHIFT		8
-#define NTF_EXT_MASK		(NTF_EXT_MANAGED)
+#define NTF_EXT_MASK		(NTF_EXT_MANAGED | NTF_EXT_EXT_VALIDATED)
 
 #define NTF_MANAGED		(NTF_EXT_MANAGED << NTF_EXT_SHIFT)
+#define NTF_EXT_VALIDATED	(NTF_EXT_EXT_VALIDATED << NTF_EXT_SHIFT)
 
 extern const struct nla_policy nda_policy[];
 
include/uapi/linux/neighbour.h +5 -0
index b851c36ad25d..c34a81245f87 100644
@@ -54,6 +54,7 @@ enum {
 /* Extended flags under NDA_FLAGS_EXT: */
 #define NTF_EXT_MANAGED		(1 << 0)
 #define NTF_EXT_LOCKED		(1 << 1)
+#define NTF_EXT_EXT_VALIDATED	(1 << 2)
 
 /*
  *	Neighbor Cache Entry States.
@@ -92,6 +93,10 @@ enum {
  * bridge in response to a host trying to communicate via a locked bridge port
  * with MAB enabled. Their purpose is to notify user space that a host requires
  * authentication.
+ *
+ * NTF_EXT_EXT_VALIDATED flagged neighbor entries were externally validated by
+ * a user space control plane. The kernel will not remove or invalidate them,
+ * but it can probe them and notify user space when they become reachable.
  */
 
 struct nda_cacheinfo {
net/core/neighbour.c +69 -10

157 lines not on this page · raw message

Signature · 1 line
2.49.0
Acked-by on v2 1/2
#2 lore raw

Re: [PATCH net-next v2 1/2] neighbor: Add NTF_EXT_VALIDATED flag for externally validated entries

On 6/26/25 9:31 AM, Ido Schimmel wrote:
[...]

Quoted 27 lines

However, these flags can be used together with "extern_valid" after the neighbor was created with a valid state:

  # ip neigh add 192.0.2.1 lladdr 00:11:22:33:44:55 nud stale dev br0.10 extern_valid
  # ip neigh replace 192.0.2.1 lladdr 00:11:22:33:44:55 nud stale dev br0.10 extern_valid use

One consequence of preventing the kernel from invalidating a neighbor entry is that by default it will only try to determine reachability using unicast probes. This can be changed using the "mcast_resolicit" sysctl:

  # sysctl net.ipv4.neigh.br0/10.mcast_resolicit
  0
  # tcpdump -nn -e -i br0.10 -Q out arp &
  # ip neigh replace 192.0.2.1 lladdr 00:11:22:33:44:55 nud stale dev br0.10 extern_valid use
  62:50:1d:11:93:6f > 00:11:22:33:44:55, ethertype ARP (0x0806), length 42: Request who-has 192.0.2.1 tell 192.0.2.2, length 28
  62:50:1d:11:93:6f > 00:11:22:33:44:55, ethertype ARP (0x0806), length 42: Request who-has 192.0.2.1 tell 192.0.2.2, length 28
  62:50:1d:11:93:6f > 00:11:22:33:44:55, ethertype ARP (0x0806), length 42: Request who-has 192.0.2.1 tell 192.0.2.2, length 28
  # sysctl -wq net.ipv4.neigh.br0/10.mcast_resolicit=3
  # ip neigh replace 192.0.2.1 lladdr 00:11:22:33:44:55 nud stale dev br0.10 extern_valid use
  62:50:1d:11:93:6f > 00:11:22:33:44:55, ethertype ARP (0x0806), length 42: Request who-has 192.0.2.1 tell 192.0.2.2, length 28
  62:50:1d:11:93:6f > 00:11:22:33:44:55, ethertype ARP (0x0806), length 42: Request who-has 192.0.2.1 tell 192.0.2.2, length 28
  62:50:1d:11:93:6f > 00:11:22:33:44:55, ethertype ARP (0x0806), length 42: Request who-has 192.0.2.1 tell 192.0.2.2, length 28
  62:50:1d:11:93:6f > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.0.2.1 tell 192.0.2.2, length 28
  62:50:1d:11:93:6f > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.0.2.1 tell 192.0.2.2, length 28
  62:50:1d:11:93:6f > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.0.2.1 tell 192.0.2.2, length 28
Quoted 6 lines

iproute2 patches can be found here [2].

[1] https://datatracker.ietf.org/doc/html/draft-rbickhart-evpn-ip-mac-proxy-adv-03
[2] https://github.com/idosch/iproute2/tree/submit/extern_valid_v1

lgtm

author maintainer
#3 lore raw

PATCH v2 2/2 selftests: net: Add a selftest for externally validated neighbor entries

In reply to Ido Schimmel

Add test cases for externally validated neighbor entries, testing both IPv4 and IPv6. Name the file "test_neigh.sh" so that it could be possibly extended in the future with more neighbor test cases.

Example output:

 # ./test_neigh.sh
 TEST: IPv4 "extern_valid" flag: Add entry                           [ OK ]
 TEST: IPv4 "extern_valid" flag: Add with an invalid state           [ OK ]
 TEST: IPv4 "extern_valid" flag: Add with "use" flag                 [ OK ]
 TEST: IPv4 "extern_valid" flag: Replace entry                       [ OK ]
 TEST: IPv4 "extern_valid" flag: Replace entry with "managed" flag   [ OK ]
 TEST: IPv4 "extern_valid" flag: Replace with an invalid state       [ OK ]
 TEST: IPv4 "extern_valid" flag: Interface down                      [ OK ]
 TEST: IPv4 "extern_valid" flag: Carrier down                        [ OK ]
 TEST: IPv4 "extern_valid" flag: Transition to "reachable" state     [ OK ]
 TEST: IPv4 "extern_valid" flag: Transition back to "stale" state    [ OK ]
 TEST: IPv4 "extern_valid" flag: Forced garbage collection           [ OK ]
 TEST: IPv4 "extern_valid" flag: Periodic garbage collection         [ OK ]
 TEST: IPv6 "extern_valid" flag: Add entry                           [ OK ]
 TEST: IPv6 "extern_valid" flag: Add with an invalid state           [ OK ]
 TEST: IPv6 "extern_valid" flag: Add with "use" flag                 [ OK ]
 TEST: IPv6 "extern_valid" flag: Replace entry                       [ OK ]
 TEST: IPv6 "extern_valid" flag: Replace entry with "managed" flag   [ OK ]
 TEST: IPv6 "extern_valid" flag: Replace with an invalid state       [ OK ]
 TEST: IPv6 "extern_valid" flag: Interface down                      [ OK ]
 TEST: IPv6 "extern_valid" flag: Carrier down                        [ OK ]
 TEST: IPv6 "extern_valid" flag: Transition to "reachable" state     [ OK ]
 TEST: IPv6 "extern_valid" flag: Transition back to "stale" state    [ OK ]
 TEST: IPv6 "extern_valid" flag: Forced garbage collection           [ OK ]
 TEST: IPv6 "extern_valid" flag: Periodic garbage collection         [ OK ]

---

Notes:
v2:

* Fix various shellcheck warnings.
* Add tests for interface and carrier down.

 tools/testing/selftests/net/Makefile      |   1 +
 tools/testing/selftests/net/test_neigh.sh | 366 ++++++++++++++++++++++
 2 files changed, 367 insertions(+)
 create mode 100755 tools/testing/selftests/net/test_neigh.sh
tools/testing/selftests/net/Makefile +1 -0
index 227f9e067d25..543776596529 100644
@@ -100,6 +100,7 @@ TEST_PROGS += test_vxlan_mdb.sh
 TEST_PROGS += test_bridge_neigh_suppress.sh
 TEST_PROGS += test_vxlan_nolocalbypass.sh
 TEST_PROGS += test_bridge_backup_port.sh
+TEST_PROGS += test_neigh.sh
 TEST_PROGS += fdb_flush.sh fdb_notify.sh
 TEST_PROGS += fq_band_pktlimit.sh
 TEST_PROGS += vlan_hw_filter.sh
tools/testing/selftests/net/test_neigh.sh new file +366 -0

366 lines not on this page · raw message

Signature · 1 line
2.49.0
author maintainer
#5 lore raw

Re: [PATCH net-next v2 2/2] selftests: net: Add a selftest for externally validated neighbor entries

Quoted 8 lines · Jakub Kicinski
Quoted 1 line · On Thu, 26 Jun 2025 10:31:11 +0300 Ido Schimmel wrote:

+TEST_PROGS += test_neigh.sh

Hi Ido!

This one is a bit flaky running on the normal/"non-debug" kernel:
https://netdev.bots.linux.dev/contest.html?executor=vmksft-net&test=test-neigh-sh
Looks like we get a flake for 1 in 10 runs :( Could you TAL?

Yes, will check.

Thanks

#6 lore raw
In reply to Ido Schimmel
This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:

Hello:

This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:

Quoted 8 lines · On Thu, 26 Jun 2025 10:31:09 +0300 you wrote:

Patch #1 adds a new neighbor flag ("extern_valid") that prevents the kernel from invalidating or removing a neighbor entry, while allowing the kernel to notify user space when the entry becomes reachable. See motivation and implementation details in the commit message.

Patch #2 adds a selftest.

[...]

Here is the summary with links:
- [net-next,v2,1/2] neighbor: Add NTF_EXT_VALIDATED flag for externally validated entries

    https://git.kernel.org/netdev/net-next/c/03dc03fa0432
  - [net-next,v2,2/2] selftests: net: Add a selftest for externally validated neighbor entries
    https://git.kernel.org/netdev/net-next/c/171f2ee31a42

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html