Skip to contentSkip to conversation

net: hsr: fix GRO/GSO super-packet handling

PATCH In review Xin Xie 4 participants 20 messages Aug 9 – Sep 28

Subsystems (5)
lore
Share & tools
lore

Xin Xie

author
#0 lore raw

v6 cover net: hsr: fix GRO/GSO super-packet handling

HSR/PRP requires per-wire-frame tags/RCTs and sequence numbers, and duplicate discard is per frame. RX GRO and TX GSO can present multiple frames as one skb and violate that assumption: a super-skb is either rejected by a constrained lower device, or forwarded without valid per-frame trailers and sequence numbers.

Patch 1 adds netif_disable_gro()/dev_disable_gro() and disables GRO and GRO_HW on lower devices at HSR/PRP enslavement time, mirroring the existing LRO treatment. Disabling is explicitly
best-effort: a later privileged override can re-enable GRO, and devices with fixed-on GRO_HW (for example a virtio-net device negotiating guest TSO without VIRTIO_NET_F_CTRL_GUEST_OFFLOADS) cannot be forced off, so enslavement succeeds whether or not the feature could be cleared.

Patch 2 shrinks hsr->seqnr_lock from whole hsr_forward_skb() calls to the sequence counter updates, so the segmentation work of patch 3 never runs under the global sequence lock. The outer lock also incidentally serialized the master/interlink tx statistics updates in hsr_forward_skb(); those are now updated without the outer lock, and consistent per-cpu/per-queue statistics for all HSR paths (including hsr_deliver_master() and multicast) are handled in a separate series.

Patch 3 unfolds GSO super-packets at the forward entry with the top-level GSO dispatch (__skb_gso_segment()), so each wire frame gets its own tag/RCT and sequence number. Admission is decided by a content-based classifier, not by the ingress port: plain-Ethernet aggregates are segmented on every ingress role (master, interlink, and LAN slaves), while aggregates whose effective protocol is ETH_P_HSR or ETH_P_PRP carry per-frame trailers that software segmentation cannot reconstruct and are dropped. Aggregates with unreadable net_iov (device-memory) fragments are also dropped: their payload is not host-readable, and software segmentation would produce segments with uninitialized payload that are silently dropped at transmit or expose uninitialized data where netmem transmit is enabled. The classifier unwraps accelerated VLAN and one in-band VLAN level, and treats NETIF_F_HW_HSR_TAG_RM lowers as plain by construction. The trailer-free premise of the plain-aggregate path is proven for in-tree software GRO: its IPv4 and IPv6 length checks reject frames with trailing bytes beyond the L3 length, so an RCT-bearing PRP frame is not merged. Device-specific fixed-on GRO_HW output is not claimed to be completely covered. Locally destined segments are delivered to the host; the rest are forwarded per frame.

Patch 4 adds a kselftest covering the series, including a LAN-slave plain-GSO regression: a plain SAN aggregate entering a PRP LAN slave must be segmented, with local delivery and per-frame arrival proven by independent counter oracles.

Patch 1 is a safe, partial, best-effort mitigation and is independently stable-selectable; patch 3, named by subject in its message, is the fallback for plain, trailer-free GSO aggregates. Patch 3 depends on patch 2, and patches 2 and 3 are selected for stable only where the sparse-bitmap duplicate discard is present, which accepts out-of-order arrival. The stable tag names that prerequisite, which exists only in 7.0 and newer.
Older branches need adapted backports.

Validation:
* the v6 kernel builds cleanly;
* LAN-slave plain-GSO regression on the v6 kernel: aggregates arrive at the PRP master and are delivered per-frame (SAN TX avg ~38 KB per frame; prp0 RX avg ~1476 B per frame);
* interlink super-packet test passes (DUT LAN legs avg ~1496 B per frame with the SAN emitting ~38 KB super-packets);
* fixed-on GRO_HW: unchanged from v5 (patch 1 control flow untouched); with a virtio guest negotiating TSO without VIRTIO_NET_F_CTRL_GUEST_OFFLOADS, HSR setup succeeds without the v4-era "failed to disable GRO!" splat (v5 evidence reused);
* the bounded server-readiness/reap waits pass on the v6 kernel;
* hsr_ping, hsr_redbox, link_faults and prp_ping all pass;
* checkpatch: 0 errors; shellcheck unchanged from v5 (0 errors);
* the series applies cleanly with plain git am on current net.

Note on the net-next conflict: this series conflicts with our own PRP RedBox support now merged in net-next
(https://lore.kernel.org/netdev/[email protected]/).
On current net it applies cleanly (re-verified by a full-series git am). For net-next the conflict is confined to two sites:

1. net/hsr/hsr_device.c, send_prp_supervision_frame(): net-next added the PRP RedBox Type-30 TLV and EOT construction where patch 2/4 shrinks the seqnr_lock critical section. Resolution:
keep the lock release immediately after the sup_sequence_nr update (as in this patch), build the whole TLV chain (LifeCheck payload, Type-30 RedBox-MAC TLV, EOT) unlocked, and drop the two stale unlocks from the padding-error and normal-exit paths.
2. tools/testing/selftests/net/hsr/Makefile: insert hsr_gro_superpacket.sh at its sorted position.

The resolution was independently proven by composition on the current net-next (4fa4977a0d90): the composed tree preserves the Type-30 RedBox-MAC TLV and EOT construction with one balanced lock region. This series' conflict-relevant hunks are unchanged from v5, which applied and validated the same resolution on an earlier net-next (2fbade662450): the composed tree builds cleanly and hsr_prp_redbox.sh passes on the composed kernel.

---

Changes in v6:

A confirmed defect is fixed: a GSO aggregate carrying unreadable net_iov (device-memory) fragments could reach the forward-entry segmentation, where the software copy path cannot read such fragments and would leave the segment payload uninitialized — silently dropped at transmit on most devices, or exposing uninitialized data where netmem transmit is enabled. v6 rejects such aggregates before segmentation (skb_frags_readable() gate) and patch 3 describes this accurately.

The selftest is hardened: the server reap now polls the wrapper's status file instead of a recycled-PID candidate, server-PID validation retries across the fork/exec window, and server readiness polls the listening socket instead of a fixed sleep; all waits stay bounded.

Further changes:
- patch 1: the comment now names fixed-on GRO_HW only (plain GRO is always changeable).
- patch 3: the prerequisite is named by subject and commit; the hw_features wording states the unconditional GSO_MASK removal (only GSO_MASK member types become fixed-off; generic-segmentation-offload stays changeable and is cleared at runtime); the classifier comment describes its own policy.
- patch 4: the message describes the new test file, the header documents the LAN-slave case, and the master-RX bound has its own named constant.
- maintainer feedback adopted: patch 2 mentions commit aae9d6b616b5 ("hsr: Implement more robust duplicate discard for HSR"), uses the commit-annotated stable tag (verified equivalent to the former 7.0.x floor: the prerequisite exists only in 7.0.y/7.1.y), carries the syzbot Reported-by/Closes pair and the two Fixes: tags, and moves all statistics work out of the series (plain increments restored; consistent per-cpu/per-queue statistics are a separate follow-up series).

The remaining review findings were assessed and left unchanged: the one-way best-effort GRO disable mirrors dev_disable_lro() and is ethtool-recoverable; the kernel-doc already names the re-enable path; out-of-order emission is tolerated by the sparse-bitmap duplicate discard on both ends under the stable floor; the remaining statistics races are accounting-precision issues covered by the follow-up series; the software-GRO RCT claim holds (IPv4/IPv6 length checks reject trailing bytes); the generic-segmentation-offload selftest oracle was verified empirically on the identical tree; test cleanup re-validates PID identity before signalling and has a netns-scoped fallback; in-place skb mutations during forwarding are pre-existing and handled by a separate series.

Previous postings (newest first):
v5: https://lore.kernel.org/netdev/[email protected]/
v4: https://lore.kernel.org/netdev/[email protected]/
v3: https://lore.kernel.org/netdev/[email protected]/
v2: https://lore.kernel.org/netdev/[email protected]/
v1: https://lore.kernel.org/netdev/[email protected]/

Xin Xie (4):
net: hsr: fix packet drops caused by GRO superpackets
net: hsr: shrink seqnr_lock to sequence counter updates
net: hsr: unfold GSO super-packets at the forward entry
selftests: net: hsr: cover GSO super-packets on PRP slave ingress

 include/linux/netdevice.h                     |   2 +
 net/core/dev.c                                |  15 +
 net/core/dev_api.c                            |  21 +
 net/hsr/hsr_device.c                          |  17 +-
 net/hsr/hsr_forward.c                         | 109 ++-
 net/hsr/hsr_slave.c                           |  16 +-
 tools/testing/selftests/net/hsr/Makefile      |   1 +
 .../selftests/net/hsr/hsr_gro_superpacket.sh  | 678 ++++++++++++++++++
 8 files changed, 835 insertions(+), 24 deletions(-)
 create mode 100755 tools/testing/selftests/net/hsr/hsr_gro_superpacket.sh
Signature · 1 line
2.43.0
author
#1 lore raw

PATCH v6 1/4 net: hsr: fix packet drops caused by GRO superpackets

HSR/PRP process each wire frame separately for tagging and duplicate discard. GRO on a lower device hides multiple frames in one skb, which cannot be forwarded with valid per-frame metadata.

Disable GRO and GRO_HW when a lower device is enslaved, matching the existing LRO handling.

This is best effort because GRO may be re-enabled and some devices cannot disable GRO_HW. The forward-entry segmentation fix handles plain, trailer-free GSO skbs that still arrive; device-specific fixed-on GRO_HW output is outside this guarantee.

Fixes f421436a591d ("net/hsr: Add support for the High-availability Seamless Redundancy protocol (HSRv0)")
---
 include/linux/netdevice.h |  2 ++
 net/core/dev.c            | 15 +++++++++++++++
 net/core/dev_api.c        | 21 +++++++++++++++++++++
 net/hsr/hsr_slave.c       |  5 +++++
 4 files changed, 43 insertions(+)
include/linux/netdevice.h +2 -0
index 9981d637f8b5..eba2c26a49ba 100644
@@ -3434,6 +3434,8 @@ void dev_close(struct net_device *dev);
 void netif_close_many(struct list_head *head, bool unlink);
 void netif_disable_lro(struct net_device *dev);
 void dev_disable_lro(struct net_device *dev);
+void netif_disable_gro(struct net_device *dev);
+void dev_disable_gro(struct net_device *dev);
 int dev_loopback_xmit(struct net *net, struct sock *sk, struct sk_buff *newskb);
 u16 dev_pick_tx_zero(struct net_device *dev, struct sk_buff *skb,
 		     struct net_device *sb_dev);
net/core/dev.c +15 -0
index 5933c5dab09e..f20d5ab0cf72 100644
@@ -1840,6 +1840,21 @@ void netif_disable_lro(struct net_device *dev)
 	}
 }
 
+void netif_disable_gro(struct net_device *dev)
+{
+	struct net_device *lower_dev;
+	struct list_head *iter;
+
+	dev->wanted_features &= ~(NETIF_F_GRO | NETIF_F_GRO_HW);
+	netdev_update_features(dev);
+
+	netdev_for_each_lower_dev(dev, lower_dev, iter) {
+		netdev_lock_ops(lower_dev);
+		netif_disable_gro(lower_dev);
+		netdev_unlock_ops(lower_dev);
+	}
+}
+
 /**
  *	dev_disable_gro_hw - disable HW Generic Receive Offload on a device
  *	@dev: device
net/core/dev_api.c +21 -0
index 437947dd08ed..3ca2515ad048 100644
@@ -269,6 +269,27 @@ void dev_disable_lro(struct net_device *dev)
 }
 EXPORT_SYMBOL(dev_disable_lro);
 
+/**
+ * dev_disable_gro() - disable Generic Receive Offload on a device
+ * @dev: device
+ *
+ * Best-effort disable of Generic Receive Offload (GRO) on a net
+ * device.  Must be called under RTNL.  This is needed if received
+ * packets may be forwarded to another interface.
+ *
+ * The disable is best-effort: a device with a fixed-on feature (for
+ * example GRO_HW on a virtio-net device negotiated without
+ * VIRTIO_NET_F_CTRL_GUEST_OFFLOADS) keeps it enabled.  Callers that
+ * need a hard guarantee must inspect the resulting feature state.
+ */
+void dev_disable_gro(struct net_device *dev)
+{
+	netdev_lock_ops(dev);
+	netif_disable_gro(dev);
+	netdev_unlock_ops(dev);
+}
+EXPORT_SYMBOL(dev_disable_gro);
+
 /**
  * dev_set_promiscuity() - update promiscuity count on a device
  * @dev: device
net/hsr/hsr_slave.c +5 -0
index 01c73b4b50dd..bb2182a169a3 100644
@@ -171,6 +171,11 @@ static int hsr_portdev_setup(struct hsr_priv *hsr, struct net_device *dev,
 		goto fail_rx_handler;
 	dev_disable_lro(dev);
 
+	/* GRO disabling is best-effort: fixed-on GRO_HW cannot be
+	 * forced off, and GRO may be re-enabled later via ethtool.
+	 */
+	dev_disable_gro(dev);
+
 	return 0;
 
 fail_rx_handler:
Signature · 1 line
2.43.0
maintainer
#2 lore raw

Re: [PATCH 1/4] net: hsr: fix packet drops caused by GRO superpackets

Quoted 11 lines · On Sun, 9 Aug 2026 14:14:51 +0200 Xin Xie wrote:

HSR/PRP process each wire frame separately for tagging and duplicate discard. GRO on a lower device hides multiple frames in one skb, which cannot be forwarded with valid per-frame metadata.

Disable GRO and GRO_HW when a lower device is enslaved, matching the existing LRO handling.

This is best effort because GRO may be re-enabled and some devices cannot disable GRO_HW. The forward-entry segmentation fix handles plain, trailer-free GSO skbs that still arrive; device-specific fixed-on GRO_HW output is outside this guarantee.

1) could you explain in more detail why GRO is breaking things? GRO shouldn't merge packets with different L2/L2.5 and packets with trailers?

2) the code seems inspired by LRO but LRO also has NETIF_F_UPPER_DISABLES. This change looks rather incomplete. Calling it "best effort" seems like an excuse.

3) Clashiko has a number of other comments:
https://netdev-ai.bots.linux.dev/sashiko/#/patchset/[email protected]

Signature · 1 line
pw-bot: cr
author
#3 lore raw

Re: [PATCH 1/4] net: hsr: fix packet drops caused by GRO superpackets

Quoted 10 lines · Jakub Kicinski

1) could you explain in more detail why GRO is breaking things? GRO shouldn't merge packets with different L2/L2.5 and packets with trailers?

2) the code seems inspired by LRO but LRO also has NETIF_F_UPPER_DISABLES. This change looks rather incomplete. Calling it "best effort" seems like an excuse.

3) Clashiko has a number of other comments:
https://netdev-ai.bots.linux.dev/sashiko/#/patchset/[email protected]

I am currently on annual leave until the middle of September. I will continue this topic when I return.

Signature · 1 line
Xin
author
#4 lore raw

Re: [PATCH 1/4] net: hsr: fix packet drops caused by GRO superpackets

In reply to Jakub Kicinski
Quoted 3 lines · Jakub Kicinski

1) could you explain in more detail why GRO is breaking things? GRO shouldn't merge packets with different L2/L2.5 and packets with trailers?

The reproducer is plain SAN traffic entering a PRP RedBox's
interlink: matching Ethernet headers, no RCT yet, so GRO can merge it. PRP then appends one RCT to the aggregate, using its length for LSDU_size and assigning one sequence number. If the aggregate LSDU exceeds 4095, the 12-bit field truncates it. Later TCP segmentation cannot generate per-segment RCTs. An aggregate left unsegmented may instead be dropped for exceeding the lower device's frame-size limit. Our wire captures showed segments without RCTs and duplicate delivery through the two LANs.Disabling interlink GRO raised single-stream TCP throughput from 27-37 Mbit/s to about 94 Mbit/s on the 100 Mbit/s rig.

Quoted 3 lines

2) the code seems inspired by LRO but LRO also has NETIF_F_UPPER_DISABLES. This change looks rather incomplete. Calling it "best effort" seems like an excuse.

NETIF_F_UPPER_DISABLES prevents a lower's ethtool request from overriding an upper's LRO-off policy. GRO is outside that mask and remains changeable; extending the global mask would affect other stacked devices too. Patch 1 sets the GRO-off default, while patch 3 segments plain, trailer-free aggregates before tagging, including when GRO is re-enabled or GSO arrives directly. GRO_HW can also be fixed on, so feature disabling alone cannot exclude all aggregates; device-specific hardware-coalesced formats remain outside v6's segmentation guarantee. I should have explained this distinction from LRO explicitly.

Quoted 2 lines

3) Clashiko has a number of other comments:
https://netdev-ai.bots.linux.dev/sashiko/#/patchset/[email protected]

I have retrieved both v6 AI reviews and will address each comment with a fix or technical explanation before posting another revision.

Signature · 1 line
Xin
#5 lore raw

Re: [PATCH 1/4] net: hsr: fix packet drops caused by GRO superpackets

Quoted 18 lines · Xin Xie
Quoted 3 lines · Jakub Kicinski

1) could you explain in more detail why GRO is breaking things? GRO shouldn't merge packets with different L2/L2.5 and packets with trailers?

The reproducer is plain SAN traffic entering a PRP RedBox's
interlink: matching Ethernet headers, no RCT yet, so GRO can merge it. PRP then appends one RCT to the aggregate, using its length for LSDU_size and assigning one sequence number. If the aggregate LSDU exceeds 4095, the 12-bit field truncates it. Later TCP segmentation cannot generate per-segment RCTs. An aggregate left unsegmented may instead be dropped for exceeding the lower device's frame-size limit. Our wire captures showed segments without RCTs and duplicate delivery through the two LANs.Disabling interlink GRO raised single-stream TCP throughput from 27-37 Mbit/s to about 94 Mbit/s on the 100 Mbit/s rig.

Quoted 3 lines

2) the code seems inspired by LRO but LRO also has NETIF_F_UPPER_DISABLES. This change looks rather incomplete. Calling it "best effort" seems like an excuse.

Quoted 10 lines

NETIF_F_UPPER_DISABLES prevents a lower's ethtool request from overriding an upper's LRO-off policy. GRO is outside that mask and remains changeable; extending the global mask would affect other stacked devices too. Patch 1 sets the GRO-off default, while patch 3 segments plain, trailer-free aggregates before tagging, including when GRO is re-enabled or GSO arrives directly. GRO_HW can also be fixed on, so feature disabling alone cannot exclude all aggregates; device-specific hardware-coalesced formats remain outside v6's segmentation guarantee. I should have explained this distinction from LRO explicitly.

Would adding a NETIF_F_HSR_SLAVE flag to slave device(s) and then filter invalid flags in netdev_fix_features() fix this? This should help against enabling and add a warning if a flag can not be removed.

A netdev_change_features() might do the job then.

Sebastian

author
#6 lore raw

Re: [PATCH 1/4] net: hsr: fix packet drops caused by GRO superpackets

Quoted 7 lines · Sebastian Andrzej Siewior

Would adding a NETIF_F_HSR_SLAVE flag to slave device(s) and then filter invalid flags in netdev_fix_features() fix this? This should help against enabling and add a warning if a flag can not be removed.

A netdev_change_features() might do the job then.

Sebastian

Yes. I'll use a port constraint in netdev_fix_features() to keep GRO, and GRO_HW (when configurable) disabled, while the device is an HSR/PRP port, incl. the interlink port. And call netdev_change_features() on attachment and detachment, preserving the GRO settings.

Signature · 1 line
Xin
author
#7 lore raw

PATCH v6 2/4 net: hsr: shrink seqnr_lock to sequence counter updates

In reply to Xin Xie

Commit 06afd2c31d33 ("hsr: Synchronize sending frames to have always incremented outgoing seq nr.") and commit 430d67bdcb04 ("net: hsr: Use the seqnr lock for frames received via interlink port.") hold seqnr_lock across the whole forwarding path. Transmitting while holding the lock can create lock-dependency issues when HSR devices are stacked with other net devices, up to a real deadlock (see Reported-by/Closes).

Since commit aae9d6b616b5 ("hsr: Implement more robust duplicate discard for HSR"), duplicate discard is order-independent (sparse bitmaps), so only the sequence counter updates need serialization. Limit seqnr_lock to the counter updates in handle_std_frame(); the master TX path (hsr_dev_xmit()) and the interlink RX path (hsr_handle_frame()) drop their outer lock, and the supervision frame builders release it right after their counter update.

Concurrent forwarding may emit frames out of allocation order, which sparse-bitmap discard tolerates. The master/interlink tx statistics are no longer serialized; consistent per-cpu/per-queue statistics for all HSR paths are handled in a separate series.

Fixes 06afd2c31d33 ("hsr: Synchronize sending frames to have always incremented outgoing seq nr.")
Fixes 430d67bdcb04 ("net: hsr: Use the seqnr lock for frames received via interlink port.")
Reported-by [email protected]
Cc <[email protected]> # aae9d6b616b5: hsr: Implement more robust duplicate discard for HSR
---
 net/hsr/hsr_device.c  | 15 ++++-----------
 net/hsr/hsr_forward.c |  3 ++-
 net/hsr/hsr_slave.c   | 11 +----------
 3 files changed, 7 insertions(+), 22 deletions(-)
net/hsr/hsr_device.c +4 -11
index 5555b71ab19b..3fd1762d8916 100644
@@ -232,9 +232,7 @@ static netdev_tx_t hsr_dev_xmit(struct sk_buff *skb, struct net_device *dev)
 		skb->dev = master->dev;
 		skb_reset_mac_header(skb);
 		skb_reset_mac_len(skb);
-		spin_lock_bh(&hsr->seqnr_lock);
 		hsr_forward_skb(skb, master);
-		spin_unlock_bh(&hsr->seqnr_lock);
 	} else {
 		dev_core_stats_tx_dropped_inc(dev);
 		dev_kfree_skb_any(skb);
@@ -335,6 +333,7 @@ static void send_hsr_supervision_frame(struct hsr_port *port,
 		hsr_stag->sequence_nr = htons(hsr->sequence_nr);
 		hsr->sequence_nr++;
 	}
+	spin_unlock_bh(&hsr->seqnr_lock);
 
 	hsr_stag->tlv.HSR_TLV_type = type;
 	/* HSRv0 has 6 unused bytes after the MAC */
@@ -356,14 +355,10 @@ static void send_hsr_supervision_frame(struct hsr_port *port,
 		ether_addr_copy(hsr_sp->macaddress_A, hsr->macaddress_redbox);
 	}
 
-	if (skb_put_padto(skb, ETH_ZLEN)) {
-		spin_unlock_bh(&hsr->seqnr_lock);
+	if (skb_put_padto(skb, ETH_ZLEN))
 		return;
-	}
 
 	hsr_forward_skb(skb, port);
-	spin_unlock_bh(&hsr->seqnr_lock);
-	return;
 }
 
 static void send_prp_supervision_frame(struct hsr_port *master,
@@ -390,6 +385,7 @@ static void send_prp_supervision_frame(struct hsr_port *master,
 	spin_lock_bh(&hsr->seqnr_lock);
 	hsr_stag->sequence_nr = htons(hsr->sup_sequence_nr);
 	hsr->sup_sequence_nr++;
+	spin_unlock_bh(&hsr->seqnr_lock);
 	hsr_stag->tlv.HSR_TLV_type = PRP_TLV_LIFE_CHECK_DD;
 	hsr_stag->tlv.HSR_TLV_length = sizeof(struct hsr_sup_payload);
 
@@ -397,13 +393,10 @@ static void send_prp_supervision_frame(struct hsr_port *master,
 	hsr_sp = skb_put(skb, sizeof(struct hsr_sup_payload));
 	ether_addr_copy(hsr_sp->macaddress_A, master->dev->dev_addr);
 
-	if (skb_put_padto(skb, ETH_ZLEN)) {
-		spin_unlock_bh(&hsr->seqnr_lock);
+	if (skb_put_padto(skb, ETH_ZLEN))
 		return;
-	}
 
 	hsr_forward_skb(skb, master);
-	spin_unlock_bh(&hsr->seqnr_lock);
 }
 
 /* Announce (supervision frame) timer function
net/hsr/hsr_forward.c +2 -1
index 0774981a65c1..8e4158a9b57c 100644
@@ -621,9 +621,10 @@ static void handle_std_frame(struct sk_buff *skb,
 	if (port->type == HSR_PT_MASTER ||
 	    port->type == HSR_PT_INTERLINK) {
 		/* Sequence nr for the master/interlink node */
-		lockdep_assert_held(&hsr->seqnr_lock);
+		spin_lock_bh(&hsr->seqnr_lock);
 		frame->sequence_nr = hsr->sequence_nr;
 		hsr->sequence_nr++;
+		spin_unlock_bh(&hsr->seqnr_lock);
 	}
 }
 
net/hsr/hsr_slave.c +1 -10
index bb2182a169a3..8b96eafe15b3 100644
@@ -73,16 +73,7 @@ static rx_handler_result_t hsr_handle_frame(struct sk_buff **pskb)
 	}
 	skb_reset_mac_len(skb);
 
-	/* Only the frames received over the interlink port will assign a
-	 * sequence number and require synchronisation vs other sender.
-	 */
-	if (port->type == HSR_PT_INTERLINK) {
-		spin_lock_bh(&hsr->seqnr_lock);
-		hsr_forward_skb(skb, port);
-		spin_unlock_bh(&hsr->seqnr_lock);
-	} else {
-		hsr_forward_skb(skb, port);
-	}
+	hsr_forward_skb(skb, port);
 
 finish_consume:
 	return RX_HANDLER_CONSUMED;
Signature · 1 line
2.43.0
Reviewed-by on v6 2/4
#8 lore raw

Re: [PATCH 2/4] net: hsr: shrink seqnr_lock to sequence counter updates

Quoted 135 lines · Xin Xie

Commit 06afd2c31d33 ("hsr: Synchronize sending frames to have always incremented outgoing seq nr.") and commit 430d67bdcb04 ("net: hsr: Use the seqnr lock for frames received via interlink port.") hold seqnr_lock across the whole forwarding path. Transmitting while holding the lock can create lock-dependency issues when HSR devices are stacked with other net devices, up to a real deadlock (see Reported-by/Closes).

Since commit aae9d6b616b5 ("hsr: Implement more robust duplicate discard for HSR"), duplicate discard is order-independent (sparse bitmaps), so only the sequence counter updates need serialization. Limit seqnr_lock to the counter updates in handle_std_frame(); the master TX path (hsr_dev_xmit()) and the interlink RX path (hsr_handle_frame()) drop their outer lock, and the supervision frame builders release it right after their counter update.

Concurrent forwarding may emit frames out of allocation order, which sparse-bitmap discard tolerates. The master/interlink tx statistics are no longer serialized; consistent per-cpu/per-queue statistics for all HSR paths are handled in a separate series.

Fixes 06afd2c31d33 ("hsr: Synchronize sending frames to have always incremented outgoing seq nr.")
Fixes 430d67bdcb04 ("net: hsr: Use the seqnr lock for frames received via interlink port.")
Reported-by [email protected]
Cc <[email protected]> # aae9d6b616b5: hsr: Implement more robust duplicate discard for HSR
---
 net/hsr/hsr_device.c  | 15 ++++-----------
 net/hsr/hsr_forward.c |  3 ++-
 net/hsr/hsr_slave.c   | 11 +----------
 3 files changed, 7 insertions(+), 22 deletions(-)
net/hsr/hsr_device.c +4 -11
index 5555b71ab19b..3fd1762d8916 100644
@@ -232,9 +232,7 @@ static netdev_tx_t hsr_dev_xmit(struct sk_buff *skb, struct net_device *dev)
 		skb->dev = master->dev;
 		skb_reset_mac_header(skb);
 		skb_reset_mac_len(skb);
-		spin_lock_bh(&hsr->seqnr_lock);
 		hsr_forward_skb(skb, master);
-		spin_unlock_bh(&hsr->seqnr_lock);
 	} else {
 		dev_core_stats_tx_dropped_inc(dev);
 		dev_kfree_skb_any(skb);
@@ -335,6 +333,7 @@ static void send_hsr_supervision_frame(struct hsr_port *port,
 		hsr_stag->sequence_nr = htons(hsr->sequence_nr);
 		hsr->sequence_nr++;
 	}
+	spin_unlock_bh(&hsr->seqnr_lock);
 
 	hsr_stag->tlv.HSR_TLV_type = type;
 	/* HSRv0 has 6 unused bytes after the MAC */
@@ -356,14 +355,10 @@ static void send_hsr_supervision_frame(struct hsr_port *port,
 		ether_addr_copy(hsr_sp->macaddress_A, hsr->macaddress_redbox);
 	}
 
-	if (skb_put_padto(skb, ETH_ZLEN)) {
-		spin_unlock_bh(&hsr->seqnr_lock);
+	if (skb_put_padto(skb, ETH_ZLEN))
 		return;
-	}
 
 	hsr_forward_skb(skb, port);
-	spin_unlock_bh(&hsr->seqnr_lock);
-	return;
 }
 
 static void send_prp_supervision_frame(struct hsr_port *master,
@@ -390,6 +385,7 @@ static void send_prp_supervision_frame(struct hsr_port *master,
 	spin_lock_bh(&hsr->seqnr_lock);
 	hsr_stag->sequence_nr = htons(hsr->sup_sequence_nr);
 	hsr->sup_sequence_nr++;
+	spin_unlock_bh(&hsr->seqnr_lock);
 	hsr_stag->tlv.HSR_TLV_type = PRP_TLV_LIFE_CHECK_DD;
 	hsr_stag->tlv.HSR_TLV_length = sizeof(struct hsr_sup_payload);
 
@@ -397,13 +393,10 @@ static void send_prp_supervision_frame(struct hsr_port *master,
 	hsr_sp = skb_put(skb, sizeof(struct hsr_sup_payload));
 	ether_addr_copy(hsr_sp->macaddress_A, master->dev->dev_addr);
 
-	if (skb_put_padto(skb, ETH_ZLEN)) {
-		spin_unlock_bh(&hsr->seqnr_lock);
+	if (skb_put_padto(skb, ETH_ZLEN))
 		return;
-	}
 
 	hsr_forward_skb(skb, master);
-	spin_unlock_bh(&hsr->seqnr_lock);
 }
 
 /* Announce (supervision frame) timer function
net/hsr/hsr_forward.c +2 -1
index 0774981a65c1..8e4158a9b57c 100644
@@ -621,9 +621,10 @@ static void handle_std_frame(struct sk_buff *skb,
 	if (port->type == HSR_PT_MASTER ||
 	    port->type == HSR_PT_INTERLINK) {
 		/* Sequence nr for the master/interlink node */
-		lockdep_assert_held(&hsr->seqnr_lock);
+		spin_lock_bh(&hsr->seqnr_lock);
 		frame->sequence_nr = hsr->sequence_nr;
 		hsr->sequence_nr++;
+		spin_unlock_bh(&hsr->seqnr_lock);
 	}
 }
 
net/hsr/hsr_slave.c +1 -10
index bb2182a169a3..8b96eafe15b3 100644
@@ -73,16 +73,7 @@ static rx_handler_result_t hsr_handle_frame(struct sk_buff **pskb)
 	}
 	skb_reset_mac_len(skb);
 
-	/* Only the frames received over the interlink port will assign a
-	 * sequence number and require synchronisation vs other sender.
-	 */
-	if (port->type == HSR_PT_INTERLINK) {
-		spin_lock_bh(&hsr->seqnr_lock);
-		hsr_forward_skb(skb, port);
-		spin_unlock_bh(&hsr->seqnr_lock);
-	} else {
-		hsr_forward_skb(skb, port);
-	}
+	hsr_forward_skb(skb, port);
 
 finish_consume:
 	return RX_HANDLER_CONSUMED;
Signature · 1 line
2.43.0

Thanks for the fix.

#9 lore raw

Re: [PATCH 2/4] net: hsr: shrink seqnr_lock to sequence counter updates

In reply to Xin Xie
Quoted 16 lines · Xin Xie

Commit 06afd2c31d33 ("hsr: Synchronize sending frames to have always incremented outgoing seq nr.") and commit 430d67bdcb04 ("net: hsr: Use the seqnr lock for frames received via interlink port.") hold seqnr_lock across the whole forwarding path. Transmitting while holding the lock can create lock-dependency issues when HSR devices are stacked with other net devices, up to a real deadlock (see Reported-by/Closes).

Since commit aae9d6b616b5 ("hsr: Implement more robust duplicate discard for HSR"), duplicate discard is order-independent (sparse bitmaps), so only the sequence counter updates need serialization. Limit seqnr_lock to the counter updates in handle_std_frame(); the master TX path (hsr_dev_xmit()) and the interlink RX path (hsr_handle_frame()) drop their outer lock, and the supervision frame builders release it right after their counter update.

Quoted 4 lines

Concurrent forwarding may emit frames out of allocation order, which sparse-bitmap discard tolerates. The master/interlink tx statistics are no longer serialized; consistent per-cpu/per-queue statistics for all HSR paths are handled in a separate series.

This fix relies on the bitmap seq nr handling while the Fixes commits below do not have it. Doesn't this revert the synchronisation change leading to the problem it tried to solve at least in the stable kernels?

Quoted 6 lines
Fixes 06afd2c31d33 ("hsr: Synchronize sending frames to have always incremented outgoing seq nr.")
Fixes 430d67bdcb04 ("net: hsr: Use the seqnr lock for frames received via interlink port.")
Reported-by [email protected]
Cc <[email protected]> # aae9d6b616b5: hsr: Implement more robust duplicate discard for HSR

Sebastian

author
#10 lore raw

Re: [PATCH 2/4] net: hsr: shrink seqnr_lock to sequence counter updates

Quoted 3 lines · Sebastian Andrzej Siewior

This fix relies on the bitmap seq nr handling while the Fixes commits below do not have it. Doesn't this revert the synchronisation change leading to the problem it tried to solve at least in the stable kernels?

You're right. Shrinking the lock can reintroduce ordering drops on pre-bitmap kernels, and the sender's bitmap does not make reordering safe for older peers.

Across ten VM runs per mode against an unmodified pre-bitmap HSRv0 peer, the narrowed-lock control delivered 19,128 of 20,000 test frames at the peer's master. The ordered-consumer prototype delivered all 20,000 exactly once.

I'll rework patch 2 around a single consumer for locally numbered traffic, including supervision. Each frame would be submitted to all its lower devices before the next frame is numbered. An idle caller would process only its own input inline; contention or re-entry would queue input to one BH worker.

No HSR lock would be held across segmentation or dev_queue_xmit(). Deferred inputs would retain their core TX recursion depth. This preserves the original per-lower submission ordering without relying on bitmap tolerance.

On PREEMPT_RT, consumer ownership would not provide priority inheritance from queued producers; progress depends on the consumer being scheduled. I will test this explicitly.

Signature · 1 line
Xin
author
#11 lore raw

PATCH v6 3/4 net: hsr: unfold GSO super-packets at the forward entry

In reply to Xin Xie

HSR/PRP require per-wire-frame tags and sequence numbers; treating a GSO skb as one frame breaks those semantics.

Classify GSO skbs at the forward entry by effective protocol rather than ingress port. Segment plain aggregates and process each segment normally, preserving local delivery and forwarding. Drop ETH_P_HSR and ETH_P_PRP aggregates (per-frame metadata cannot be rebuilt), classification failures (unreadable header, stacked or S-tag tagging), and aggregates with unreadable net_iov (device-memory) fragments:
their payload is not host-readable, and segmentation would leave the segment payload uninitialized, silently dropped at transmit or exposed where netmem transmit is enabled.

In-tree software GRO does not merge PRP RCT frames (its IPv4/IPv6 length checks reject trailing bytes); fixed-on GRO_HW output is outside this guarantee.

Also remove the GSO_MASK member types from the HSR master's
hw_features: local traffic is segmented by the core before the forward path, so the master funnel branch sees single frames, and TSO and the other GSO_MASK offloads can no longer be enabled on the master; generic-segmentation-offload stays changeable and is cleared at runtime.

This patch depends on patch 2 ("net: hsr: shrink seqnr_lock to sequence counter updates") and the sparse-bitmap duplicate discard introduced by commit aae9d6b616b5 ("hsr: Implement more robust duplicate discard for HSR"); older trees require an adapted backport.

Fixes f421436a591d ("net/hsr: Add support for the High-availability Seamless Redundancy protocol (HSRv0)")
Cc <[email protected]> # aae9d6b616b5: hsr: Implement more robust duplicate discard for HSR
---
 net/hsr/hsr_device.c  |   2 +-
 net/hsr/hsr_forward.c | 106 +++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 106 insertions(+), 2 deletions(-)
net/hsr/hsr_device.c +1 -1
index 3fd1762d8916..248cbb142e21 100644
@@ -652,7 +652,7 @@ void hsr_dev_setup(struct net_device *dev)
 	dev->needs_free_netdev = true;
 
 	dev->hw_features = NETIF_F_SG | NETIF_F_FRAGLIST | NETIF_F_HIGHDMA |
-			   NETIF_F_GSO_MASK | NETIF_F_HW_CSUM |
+			   NETIF_F_HW_CSUM |
 			   NETIF_F_HW_VLAN_CTAG_TX |
 			   NETIF_F_HW_VLAN_CTAG_FILTER;
 
net/hsr/hsr_forward.c +105 -1

121 lines not on this page · raw message

Signature · 1 line
2.43.0
#12 lore raw

Re: [PATCH 3/4] net: hsr: unfold GSO super-packets at the forward entry

Quoted 2 lines · Xin Xie

HSR/PRP require per-wire-frame tags and sequence numbers; treating a GSO skb as one frame breaks those semantics.

Wouldn't it be simpler to not allow such packets?

Sebastian

author
#13 lore raw

Re: [PATCH 3/4] net: hsr: unfold GSO super-packets at the forward entry

Quoted 1 line · Sebastian Andrzej Siewior

Wouldn't it be simpler to not allow such packets?

For master TX, yes: removing GSO features lets the core segment before ndo_start_xmit(), as patch 3 already does.

The interlink RX path is different. With GRO/GRO_HW disabled, it can still receive skbs that were already GSO at the sender. I reproduced this with ordinary TCP traffic over veth in a VM. Rejecting these skbs would drop valid traffic.

I'll retain segmentation for valid interlink GSO in the ordered consumer described in my patch 2 reply, before per-frame sequence allocation and HSR tag/PRP RCT construction. Each aggregate would finish before the next input, without holding an HSR lock across segmentation or transmission.

Signature · 1 line
Xin
#14 lore raw

Re: [PATCH 3/4] net: hsr: unfold GSO super-packets at the forward entry

Quoted 10 lines · Xin Xie
Quoted 1 line · Sebastian Andrzej Siewior

Wouldn't it be simpler to not allow such packets?

For master TX, yes: removing GSO features lets the core segment before ndo_start_xmit(), as patch 3 already does.

The interlink RX path is different. With GRO/GRO_HW disabled, it can still receive skbs that were already GSO at the sender. I reproduced this with ordinary TCP traffic over veth in a VM. Rejecting these skbs would drop valid traffic.

So you disable GRO on the interlink port and still receive GRO packets? Or is this performance related where you want to keep GRO enabled on the interlink port?

Quoted 5 lines

I'll retain segmentation for valid interlink GSO in the ordered consumer described in my patch 2 reply, before per-frame sequence allocation and HSR tag/PRP RCT construction. Each aggregate would finish before the next input, without holding an HSR lock across segmentation or transmission.

Sebastian

author
#15 lore raw

Re: [PATCH 3/4] net: hsr: unfold GSO super-packets at the forward entry

Quoted 3 lines · Sebastian Andrzej Siewior

So you disable GRO on the interlink port and still receive GRO packets? Or is this performance related where you want to keep GRO enabled on the interlink port?

I am not trying to keep GRO enabled for performance. The difference is between a physical interlink and a veth interlink.

With a physical interlink, TCP data is split into packets before it goes onto the wire via TSO/GSO. If GRO is off at the receiver, this interlink RX path needs no extra segmentation. Patch 3 is indeed unnecessary in this case.

With a veth interlink, there is no physical wire:
TCP -> prp0-peer -> prp0-int (interlink) -> PRP

A veth pair is simply an in-kernel memory pipe. When a large TCP skb arrives at prp0-peer, veth does not trigger TSO or software segmentation, it simply hands over the intact skb pointer directly to prp0-int.

Turning GRO off on prp0-int does not force prp0-peer to split it. I confirmed this with ordinary TCP traffic in a VM, with GRO off on both veth ends.

Patch 3 splits that skb before PRP assigns sequence numbers and adds RCTs. Each resulting frame needs its own sequence number and RCT. Ordinary non-GSO skbs skip the segmentation path.

In addition, if GRO disablement fails (e.g., buggy drivers or hardware-fixed GRO_HW that cannot be turned off via ethtool), Patch 3 also acts as a fallback defense for these edge cases.

--
Xin

#16 lore raw

Re: [PATCH 3/4] net: hsr: unfold GSO super-packets at the forward entry

On 2026-09-25 11:52:52 [+0200], Xin Xie wrote:
…

Quoted 7 lines

A veth pair is simply an in-kernel memory pipe. When a large TCP skb arrives at prp0-peer, veth does not trigger TSO or software segmentation, it simply hands over the intact skb pointer directly to prp0-int.

Turning GRO off on prp0-int does not force prp0-peer to split it. I confirmed this with ordinary TCP traffic in a VM, with GRO off on both veth ends.

So veth. The test suite is using veth I don't remember that it was (is) a problem there. But then there is no TCP.
Wouldn't it be also a problem if you attach veth interface to a bridge and that large TCP skb would have to leave the bridge via a physical port?
This does not sound like it is limited to hsr. I think it deserves a helper similar to skb_linearize().

Quoted 8 lines

Patch 3 splits that skb before PRP assigns sequence numbers and adds RCTs. Each resulting frame needs its own sequence number and RCT. Ordinary non-GSO skbs skip the segmentation path.

In addition, if GRO disablement fails (e.g., buggy drivers or hardware-fixed GRO_HW that cannot be turned off via ethtool), Patch 3 also acts as a fallback defense for these edge cases.

Sebastian

author
#17 lore raw

Re: [PATCH 3/4] net: hsr: unfold GSO super-packets at the forward entry

Quoted 5 lines

So veth. The test suite is using veth I don't remember that it was (is) a problem there. But then there is no TCP.
Wouldn't it be also a problem if you attach veth interface to a bridge and that large TCP skb would have to leave the bridge via a physical port?

When a bridge forwards a GSO skb from veth to a physical port, the outgoing TX path handles segmentation. validate_xmit_skb() checks whether software segmentation is needed. The networking core performs software segmentation as needed and can use hardware segmentation offload when available for that skb.

The software HSR/PRP path needs to split the GSO skb earlier. Each segment needs its own sequence number and HSR tag or PRP trailer. We add these before calling dev_queue_xmit() on the lower device. The normal TX segmentation code cannot create these per-frame HSR/PRP fields for us.

TCP is the case I reproduced, but this is not limited to TCP. UDP GSO skbs can also pass through veth without being segmented. Passing normal PROFINET RT or GOOSE frames through veth does not make them GSO skbs. Patch 3 skips segmentation for these frames. They still follow the normal HSR/PRP forwarding path.

Quoted 2 lines

This does not sound like it is limited to hsr. I think it deserves a helper similar to skb_linearize().

Patch 3 already uses the common __skb_gso_segment() helper, then passes each segment through normal HSR/PRP forwarding. Do you mean a new common wrapper around this helper?

Signature · 1 line
Xin
#18 lore raw

Re: [PATCH 3/4] net: hsr: unfold GSO super-packets at the forward entry

Quoted 6 lines · Xin Xie
Quoted 2 lines

This does not sound like it is limited to hsr. I think it deserves a helper similar to skb_linearize().

Patch 3 already uses the common __skb_gso_segment() helper, then passes each segment through normal HSR/PRP forwarding. Do you mean a new common wrapper around this helper?

I must have missed this part. It is okay then.

Sebastian

author
#19 lore raw

PATCH v6 4/4 selftests: net: hsr: cover GSO super-packets on PRP slave ingress

In reply to Xin Xie

Add a new always-run HSR/PRP kselftest (hsr_gro_superpacket.sh) covering the GRO/GRO_HW state on enslaved devices, the HSR master's GSO/TSO feature advertisement, a TSO stream through the forward path, and a PRP LAN-slave plain-GSO regression in which aggregates are segmented at the forward entry and delivered locally to the master.

The test asserts that plain aggregates are unfolded per-frame on any ingress role and that local delivery and forwarding are preserved; interface-counter deltas are the evidence. Server readiness and reap waits are bounded, and the tool dependencies (ethtool, iperf3, timeout) are enforced via check_tool().

---
 tools/testing/selftests/net/hsr/Makefile      |   1 +
 .../selftests/net/hsr/hsr_gro_superpacket.sh  | 678 ++++++++++++++++++
 2 files changed, 679 insertions(+)
 create mode 100755 tools/testing/selftests/net/hsr/hsr_gro_superpacket.sh
tools/testing/selftests/net/hsr/Makefile +1 -0
index 31fb9326cf53..0d105476e7c5 100644
@@ -3,6 +3,7 @@
 top_srcdir = ../../../../..
 
 TEST_PROGS := \
+	hsr_gro_superpacket.sh \
 	hsr_ping.sh \
 	hsr_redbox.sh \
 	link_faults.sh \
tools/testing/selftests/net/hsr/hsr_gro_superpacket.sh new file +678 -0

678 lines not on this page · raw message

Signature · 1 line
2.43.0