net: hsr: fix GRO/GSO super-packet handling
Subsystems (5)
v6 cover net: hsr: fix GRO/GSO super-packet handling
HSR/PRP requires per-wire-frame tags/RCTs and sequence numbers, and duplicate discard is per frame. RX GRO and TX GSO can present multiple frames as one skb and violate that assumption: a super-skb is either rejected by a constrained lower device, or forwarded without valid per-frame trailers and sequence numbers.
Patch 1 adds netif_disable_gro()/dev_disable_gro() and disables GRO and GRO_HW on lower devices at HSR/PRP enslavement time, mirroring the existing LRO treatment. Disabling is explicitly
best-effort: a later privileged override can re-enable GRO, and devices with fixed-on GRO_HW (for example a virtio-net device negotiating guest TSO without VIRTIO_NET_F_CTRL_GUEST_OFFLOADS) cannot be forced off, so enslavement succeeds whether or not the feature could be cleared.
Patch 2 shrinks hsr->seqnr_lock from whole hsr_forward_skb() calls to the sequence counter updates, so the segmentation work of patch 3 never runs under the global sequence lock. The outer lock also incidentally serialized the master/interlink tx statistics updates in hsr_forward_skb(); those are now updated without the outer lock, and consistent per-cpu/per-queue statistics for all HSR paths (including hsr_deliver_master() and multicast) are handled in a separate series.
Patch 3 unfolds GSO super-packets at the forward entry with the top-level GSO dispatch (__skb_gso_segment()), so each wire frame gets its own tag/RCT and sequence number. Admission is decided by a content-based classifier, not by the ingress port: plain-Ethernet aggregates are segmented on every ingress role (master, interlink, and LAN slaves), while aggregates whose effective protocol is ETH_P_HSR or ETH_P_PRP carry per-frame trailers that software segmentation cannot reconstruct and are dropped. Aggregates with unreadable net_iov (device-memory) fragments are also dropped: their payload is not host-readable, and software segmentation would produce segments with uninitialized payload that are silently dropped at transmit or expose uninitialized data where netmem transmit is enabled. The classifier unwraps accelerated VLAN and one in-band VLAN level, and treats NETIF_F_HW_HSR_TAG_RM lowers as plain by construction. The trailer-free premise of the plain-aggregate path is proven for in-tree software GRO: its IPv4 and IPv6 length checks reject frames with trailing bytes beyond the L3 length, so an RCT-bearing PRP frame is not merged. Device-specific fixed-on GRO_HW output is not claimed to be completely covered. Locally destined segments are delivered to the host; the rest are forwarded per frame.
Patch 4 adds a kselftest covering the series, including a LAN-slave plain-GSO regression: a plain SAN aggregate entering a PRP LAN slave must be segmented, with local delivery and per-frame arrival proven by independent counter oracles.
Patch 1 is a safe, partial, best-effort mitigation and is independently stable-selectable; patch 3, named by subject in its message, is the fallback for plain, trailer-free GSO aggregates. Patch 3 depends on patch 2, and patches 2 and 3 are selected for stable only where the sparse-bitmap duplicate discard is present, which accepts out-of-order arrival. The stable tag names that prerequisite, which exists only in 7.0 and newer.
Older branches need adapted backports.
Validation:
* the v6 kernel builds cleanly;
* LAN-slave plain-GSO regression on the v6 kernel: aggregates arrive at the PRP master and are delivered per-frame (SAN TX avg ~38 KB per frame; prp0 RX avg ~1476 B per frame);
* interlink super-packet test passes (DUT LAN legs avg ~1496 B per frame with the SAN emitting ~38 KB super-packets);
* fixed-on GRO_HW: unchanged from v5 (patch 1 control flow untouched); with a virtio guest negotiating TSO without VIRTIO_NET_F_CTRL_GUEST_OFFLOADS, HSR setup succeeds without the v4-era "failed to disable GRO!" splat (v5 evidence reused);
* the bounded server-readiness/reap waits pass on the v6 kernel;
* hsr_ping, hsr_redbox, link_faults and prp_ping all pass;
* checkpatch: 0 errors; shellcheck unchanged from v5 (0 errors);
* the series applies cleanly with plain git am on current net.
Note on the net-next conflict: this series conflicts with our own PRP RedBox support now merged in net-next
(https://lore.kernel.org/netdev/[email protected]/).
On current net it applies cleanly (re-verified by a full-series git am). For net-next the conflict is confined to two sites:
1. net/hsr/hsr_device.c, send_prp_supervision_frame(): net-next added the PRP RedBox Type-30 TLV and EOT construction where patch 2/4 shrinks the seqnr_lock critical section. Resolution:
keep the lock release immediately after the sup_sequence_nr update (as in this patch), build the whole TLV chain (LifeCheck payload, Type-30 RedBox-MAC TLV, EOT) unlocked, and drop the two stale unlocks from the padding-error and normal-exit paths.
2. tools/testing/selftests/net/hsr/Makefile: insert hsr_gro_superpacket.sh at its sorted position.
The resolution was independently proven by composition on the current net-next (4fa4977a0d90): the composed tree preserves the Type-30 RedBox-MAC TLV and EOT construction with one balanced lock region. This series' conflict-relevant hunks are unchanged from v5, which applied and validated the same resolution on an earlier net-next (2fbade662450): the composed tree builds cleanly and hsr_prp_redbox.sh passes on the composed kernel.
---
Changes in v6:
A confirmed defect is fixed: a GSO aggregate carrying unreadable net_iov (device-memory) fragments could reach the forward-entry segmentation, where the software copy path cannot read such fragments and would leave the segment payload uninitialized — silently dropped at transmit on most devices, or exposing uninitialized data where netmem transmit is enabled. v6 rejects such aggregates before segmentation (skb_frags_readable() gate) and patch 3 describes this accurately.
The selftest is hardened: the server reap now polls the wrapper's status file instead of a recycled-PID candidate, server-PID validation retries across the fork/exec window, and server readiness polls the listening socket instead of a fixed sleep; all waits stay bounded.
Further changes:
- patch 1: the comment now names fixed-on GRO_HW only (plain GRO is always changeable).
- patch 3: the prerequisite is named by subject and commit; the hw_features wording states the unconditional GSO_MASK removal (only GSO_MASK member types become fixed-off; generic-segmentation-offload stays changeable and is cleared at runtime); the classifier comment describes its own policy.
- patch 4: the message describes the new test file, the header documents the LAN-slave case, and the master-RX bound has its own named constant.
- maintainer feedback adopted: patch 2 mentions commit aae9d6b616b5 ("hsr: Implement more robust duplicate discard for HSR"), uses the commit-annotated stable tag (verified equivalent to the former 7.0.x floor: the prerequisite exists only in 7.0.y/7.1.y), carries the syzbot Reported-by/Closes pair and the two Fixes: tags, and moves all statistics work out of the series (plain increments restored; consistent per-cpu/per-queue statistics are a separate follow-up series).
The remaining review findings were assessed and left unchanged: the one-way best-effort GRO disable mirrors dev_disable_lro() and is ethtool-recoverable; the kernel-doc already names the re-enable path; out-of-order emission is tolerated by the sparse-bitmap duplicate discard on both ends under the stable floor; the remaining statistics races are accounting-precision issues covered by the follow-up series; the software-GRO RCT claim holds (IPv4/IPv6 length checks reject trailing bytes); the generic-segmentation-offload selftest oracle was verified empirically on the identical tree; test cleanup re-validates PID identity before signalling and has a netns-scoped fallback; in-place skb mutations during forwarding are pre-existing and handled by a separate series.
Previous postings (newest first):
v5: https://lore.kernel.org/netdev/[email protected]/
v4: https://lore.kernel.org/netdev/[email protected]/
v3: https://lore.kernel.org/netdev/[email protected]/
v2: https://lore.kernel.org/netdev/[email protected]/
v1: https://lore.kernel.org/netdev/[email protected]/
Xin Xie (4):
net: hsr: fix packet drops caused by GRO superpackets
net: hsr: shrink seqnr_lock to sequence counter updates
net: hsr: unfold GSO super-packets at the forward entry
selftests: net: hsr: cover GSO super-packets on PRP slave ingress
include/linux/netdevice.h | 2 + net/core/dev.c | 15 + net/core/dev_api.c | 21 + net/hsr/hsr_device.c | 17 +- net/hsr/hsr_forward.c | 109 ++- net/hsr/hsr_slave.c | 16 +- tools/testing/selftests/net/hsr/Makefile | 1 + .../selftests/net/hsr/hsr_gro_superpacket.sh | 678 ++++++++++++++++++ 8 files changed, 835 insertions(+), 24 deletions(-) create mode 100755 tools/testing/selftests/net/hsr/hsr_gro_superpacket.sh
Signature · 1 line
2.43.0
PATCH v6 1/4 net: hsr: fix packet drops caused by GRO superpackets
HSR/PRP process each wire frame separately for tagging and duplicate discard. GRO on a lower device hides multiple frames in one skb, which cannot be forwarded with valid per-frame metadata.
Disable GRO and GRO_HW when a lower device is enslaved, matching the existing LRO handling.
This is best effort because GRO may be re-enabled and some devices cannot disable GRO_HW. The forward-entry segmentation fix handles plain, trailer-free GSO skbs that still arrive; device-specific fixed-on GRO_HW output is outside this guarantee.
--- include/linux/netdevice.h | 2 ++ net/core/dev.c | 15 +++++++++++++++ net/core/dev_api.c | 21 +++++++++++++++++++++ net/hsr/hsr_slave.c | 5 +++++ 4 files changed, 43 insertions(+)
include/linux/netdevice.h +2 -0
@@ -3434,6 +3434,8 @@ void dev_close(struct net_device *dev);void netif_close_many (struct list_head *head,bool unlink);void netif_disable_lro (struct net_device *dev);void dev_disable_lro (struct net_device *dev); +void netif_disable_gro (struct net_device *dev); +void dev_disable_gro (struct net_device *dev);int dev_loopback_xmit (struct net *net,struct sock *sk,struct sk_buff *newskb);u16 dev_pick_tx_zero (struct net_device *dev,struct sk_buff *skb,struct net_device *sb_dev);
net/core/dev.c +15 -0
@@ -1840,6 +1840,21 @@ void netif_disable_lro(struct net_device *dev) } } +void netif_disable_gro (struct net_device *dev) +{ +struct net_device *lower_dev; +struct list_head *iter; + + dev->wanted_features &= ~(NETIF_F_GRO |NETIF_F_GRO_HW ); +netdev_update_features (dev); + +netdev_for_each_lower_dev (dev, lower_dev, iter) { +netdev_lock_ops (lower_dev); +netif_disable_gro (lower_dev); +netdev_unlock_ops (lower_dev); + } +} +/** * dev_disable_gro_hw - disable HW Generic Receive Offload on a device * @dev: device
net/core/dev_api.c +21 -0
@@ -269,6 +269,27 @@ void dev_disable_lro(struct net_device *dev) }EXPORT_SYMBOL (dev_disable_lro); +/** +* dev_disable_gro() - disable Generic Receive Offload on a device +* @dev: device +* +* Best-effort disable of Generic Receive Offload (GRO) on a net +* device. Must be called under RTNL. This is needed if received +* packets may be forwarded to another interface. +* +* The disable is best-effort: a device with a fixed-on feature (for +* example GRO_HW on a virtio-net device negotiated without +* VIRTIO_NET_F_CTRL_GUEST_OFFLOADS) keeps it enabled. Callers that +* need a hard guarantee must inspect the resulting feature state. +*/ +void dev_disable_gro (struct net_device *dev) +{ +netdev_lock_ops (dev); +netif_disable_gro (dev); +netdev_unlock_ops (dev); +} +EXPORT_SYMBOL (dev_disable_gro); +/** * dev_set_promiscuity() - update promiscuity count on a device * @dev: device
net/hsr/hsr_slave.c +5 -0
@@ -171,6 +171,11 @@ static int hsr_portdev_setup(struct hsr_priv *hsr, struct net_device *dev, goto fail_rx_handler;dev_disable_lro (dev); +/* GRO disabling is best-effort: fixed-on GRO_HW cannot be +* forced off, and GRO may be re-enabled later via ethtool. +*/ +dev_disable_gro (dev); +return 0 ; fail_rx_handler:
Signature · 1 line
2.43.0
Jakub Kicinski
maintainerRe: [PATCH 1/4] net: hsr: fix packet drops caused by GRO superpackets
Quoted 11 lines · On Sun, 9 Aug 2026 14:14:51 +0200 Xin Xie wrote:
HSR/PRP process each wire frame separately for tagging and duplicate discard. GRO on a lower device hides multiple frames in one skb, which cannot be forwarded with valid per-frame metadata.
Disable GRO and
GRO_HWwhen a lower device is enslaved, matching the existing LRO handling.This is best effort because GRO may be re-enabled and some devices cannot disable
GRO_HW. The forward-entry segmentation fix handles plain, trailer-free GSO skbs that still arrive; device-specific fixed-onGRO_HWoutput is outside this guarantee.
1) could you explain in more detail why GRO is breaking things? GRO shouldn't merge packets with different L2/L2.5 and packets with trailers?
2) the code seems inspired by LRO but LRO also has NETIF_F_UPPER_DISABLES. This change looks rather incomplete. Calling it "best effort" seems like an excuse.
3) Clashiko has a number of other comments:
https://netdev-ai.bots.linux.dev/sashiko/#/patchset/[email protected]
Signature · 1 line
pw-bot: cr
Re: [PATCH 1/4] net: hsr: fix packet drops caused by GRO superpackets
Quoted 10 lines · Jakub Kicinski
1) could you explain in more detail why GRO is breaking things? GRO shouldn't merge packets with different L2/L2.5 and packets with trailers?
2) the code seems inspired by LRO but LRO also has
NETIF_F_UPPER_DISABLES. This change looks rather incomplete. Calling it "best effort" seems like an excuse.3) Clashiko has a number of other comments:
https://netdev-ai.bots.linux.dev/sashiko/#/patchset/[email protected]
I am currently on annual leave until the middle of September. I will continue this topic when I return.
Signature · 1 line
Xin
Re: [PATCH 1/4] net: hsr: fix packet drops caused by GRO superpackets
In reply to Jakub KicinskiQuoted 3 lines · Jakub Kicinski
1) could you explain in more detail why GRO is breaking things? GRO shouldn't merge packets with different L2/L2.5 and packets with trailers?
The reproducer is plain SAN traffic entering a PRP RedBox's
interlink: matching Ethernet headers, no RCT yet, so GRO can merge it. PRP then appends one RCT to the aggregate, using its length for LSDU_size and assigning one sequence number. If the aggregate LSDU exceeds 4095, the 12-bit field truncates it. Later TCP segmentation cannot generate per-segment RCTs. An aggregate left unsegmented may instead be dropped for exceeding the lower device's frame-size limit. Our wire captures showed segments without RCTs and duplicate delivery through the two LANs.Disabling interlink GRO raised single-stream TCP throughput from 27-37 Mbit/s to about 94 Mbit/s on the 100 Mbit/s rig.
Quoted 3 lines
2) the code seems inspired by LRO but LRO also has
NETIF_F_UPPER_DISABLES. This change looks rather incomplete. Calling it "best effort" seems like an excuse.
NETIF_F_UPPER_DISABLES prevents a lower's ethtool request from overriding an upper's LRO-off policy. GRO is outside that mask and remains changeable; extending the global mask would affect other stacked devices too. Patch 1 sets the GRO-off default, while patch 3 segments plain, trailer-free aggregates before tagging, including when GRO is re-enabled or GSO arrives directly. GRO_HW can also be fixed on, so feature disabling alone cannot exclude all aggregates; device-specific hardware-coalesced formats remain outside v6's segmentation guarantee. I should have explained this distinction from LRO explicitly.
Quoted 2 lines
3) Clashiko has a number of other comments:
https://netdev-ai.bots.linux.dev/sashiko/#/patchset/[email protected]
I have retrieved both v6 AI reviews and will address each comment with a fix or technical explanation before posting another revision.
Signature · 1 line
Xin
Re: [PATCH 1/4] net: hsr: fix packet drops caused by GRO superpackets
Quoted 18 lines · Xin Xie
Quoted 3 lines · Jakub Kicinski
1) could you explain in more detail why GRO is breaking things? GRO shouldn't merge packets with different L2/L2.5 and packets with trailers?
The reproducer is plain SAN traffic entering a PRP RedBox's
interlink: matching Ethernet headers, no RCT yet, so GRO can merge it. PRP then appends one RCT to the aggregate, using its length forLSDU_sizeand assigning one sequence number. If the aggregate LSDU exceeds 4095, the 12-bit field truncates it. Later TCP segmentation cannot generate per-segment RCTs. An aggregate left unsegmented may instead be dropped for exceeding the lower device's frame-size limit. Our wire captures showed segments without RCTs and duplicate delivery through the two LANs.Disabling interlink GRO raised single-stream TCP throughput from 27-37 Mbit/s to about 94 Mbit/s on the 100 Mbit/s rig.Quoted 3 lines
2) the code seems inspired by LRO but LRO also has
NETIF_F_UPPER_DISABLES. This change looks rather incomplete. Calling it "best effort" seems like an excuse.
Quoted 10 lines
NETIF_F_UPPER_DISABLESprevents a lower's ethtool request from overriding an upper's LRO-off policy. GRO is outside that mask and remains changeable; extending the global mask would affect other stacked devices too. Patch 1 sets the GRO-off default, while patch 3 segments plain, trailer-free aggregates before tagging, including when GRO is re-enabled or GSO arrives directly.GRO_HWcan also be fixed on, so feature disabling alone cannot exclude all aggregates; device-specific hardware-coalesced formats remain outside v6's segmentation guarantee. I should have explained this distinction from LRO explicitly.
Would adding a NETIF_F_HSR_SLAVE flag to slave device(s) and then filter invalid flags in netdev_fix_features() fix this? This should help against enabling and add a warning if a flag can not be removed.
A netdev_change_features() might do the job then.
Sebastian
Re: [PATCH 1/4] net: hsr: fix packet drops caused by GRO superpackets
Quoted 7 lines · Sebastian Andrzej Siewior
Would adding a
NETIF_F_HSR_SLAVEflag to slave device(s) and then filter invalid flags innetdev_fix_features()fix this? This should help against enabling and add a warning if a flag can not be removed.A
netdev_change_features()might do the job then.Sebastian
Yes. I'll use a port constraint in netdev_fix_features() to keep GRO, and GRO_HW (when configurable) disabled, while the device is an HSR/PRP port, incl. the interlink port. And call netdev_change_features() on attachment and detachment, preserving the GRO settings.
Signature · 1 line
Xin
PATCH v6 2/4 net: hsr: shrink seqnr_lock to sequence counter updates
In reply to Xin XieCommit 06afd2c31d33 ("hsr: Synchronize sending frames to have always incremented outgoing seq nr.") and commit 430d67bdcb04 ("net: hsr: Use the seqnr lock for frames received via interlink port.") hold seqnr_lock across the whole forwarding path. Transmitting while holding the lock can create lock-dependency issues when HSR devices are stacked with other net devices, up to a real deadlock (see Reported-by/Closes).
Since commit aae9d6b616b5 ("hsr: Implement more robust duplicate discard for HSR"), duplicate discard is order-independent (sparse bitmaps), so only the sequence counter updates need serialization. Limit seqnr_lock to the counter updates in handle_std_frame(); the master TX path (hsr_dev_xmit()) and the interlink RX path (hsr_handle_frame()) drop their outer lock, and the supervision frame builders release it right after their counter update.
Concurrent forwarding may emit frames out of allocation order, which sparse-bitmap discard tolerates. The master/interlink tx statistics are no longer serialized; consistent per-cpu/per-queue statistics for all HSR paths are handled in a separate series.
--- net/hsr/hsr_device.c | 15 ++++----------- net/hsr/hsr_forward.c | 3 ++- net/hsr/hsr_slave.c | 11 +---------- 3 files changed, 7 insertions(+), 22 deletions(-)
net/hsr/hsr_device.c +4 -11
@@ -232,9 +232,7 @@ static netdev_tx_t hsr_dev_xmit(struct sk_buff *skb, struct net_device *dev) skb->dev = master->dev;skb_reset_mac_header (skb);skb_reset_mac_len (skb); -spin_lock_bh (&hsr->seqnr_lock);hsr_forward_skb (skb, master); -spin_unlock_bh (&hsr->seqnr_lock); }else {dev_core_stats_tx_dropped_inc (dev);dev_kfree_skb_any (skb);
@@ -335,6 +333,7 @@ static void send_hsr_supervision_frame(struct hsr_port *port, hsr_stag->sequence_nr =htons (hsr->sequence_nr); hsr->sequence_nr++; } +spin_unlock_bh (&hsr->seqnr_lock); hsr_stag->tlv.HSR_TLV_type = type;/* HSRv0 has 6 unused bytes after the MAC */
@@ -356,14 +355,10 @@ static void send_hsr_supervision_frame(struct hsr_port *port,ether_addr_copy (hsr_sp->macaddress_A, hsr->macaddress_redbox); } -if (skb_put_padto (skb,ETH_ZLEN )) { -spin_unlock_bh (&hsr->seqnr_lock); +if (skb_put_padto (skb,ETH_ZLEN ))return ; - }hsr_forward_skb (skb, port); -spin_unlock_bh (&hsr->seqnr_lock); -return ; } static void send_prp_supervision_frame(struct hsr_port *master,
@@ -390,6 +385,7 @@ static void send_prp_supervision_frame(struct hsr_port *master,spin_lock_bh (&hsr->seqnr_lock); hsr_stag->sequence_nr =htons (hsr->sup_sequence_nr); hsr->sup_sequence_nr++; +spin_unlock_bh (&hsr->seqnr_lock); hsr_stag->tlv.HSR_TLV_type =PRP_TLV_LIFE_CHECK_DD ; hsr_stag->tlv.HSR_TLV_length =sizeof (struct hsr_sup_payload );
@@ -397,13 +393,10 @@ static void send_prp_supervision_frame(struct hsr_port *master, hsr_sp =skb_put (skb,sizeof (struct hsr_sup_payload ));ether_addr_copy (hsr_sp->macaddress_A, master->dev->dev_addr); -if (skb_put_padto (skb,ETH_ZLEN )) { -spin_unlock_bh (&hsr->seqnr_lock); +if (skb_put_padto (skb,ETH_ZLEN ))return ; - }hsr_forward_skb (skb, master); -spin_unlock_bh (&hsr->seqnr_lock); }/* Announce (supervision frame) timer function
net/hsr/hsr_forward.c +2 -1
@@ -621,9 +621,10 @@ static void handle_std_frame(struct sk_buff *skb,if (port->type ==HSR_PT_MASTER || port->type ==HSR_PT_INTERLINK ) {/* Sequence nr for the master/interlink node */ -lockdep_assert_held (&hsr->seqnr_lock); +spin_lock_bh (&hsr->seqnr_lock); frame->sequence_nr = hsr->sequence_nr; hsr->sequence_nr++; +spin_unlock_bh (&hsr->seqnr_lock); } }
net/hsr/hsr_slave.c +1 -10
@@ -73,16 +73,7 @@ static rx_handler_result_t hsr_handle_frame(struct sk_buff **pskb) }skb_reset_mac_len (skb); -/* Only the frames received over the interlink port will assign a -* sequence number and require synchronisation vs other sender. -*/ -if (port->type ==HSR_PT_INTERLINK ) { -spin_lock_bh (&hsr->seqnr_lock); -hsr_forward_skb (skb, port); -spin_unlock_bh (&hsr->seqnr_lock); - }else { -hsr_forward_skb (skb, port); - } +hsr_forward_skb (skb, port); finish_consume:return RX_HANDLER_CONSUMED ;
Signature · 1 line
2.43.0
Re: [PATCH 2/4] net: hsr: shrink seqnr_lock to sequence counter updates
Quoted 135 lines · Xin Xie
Commit
06afd2c31d33("hsr: Synchronize sending frames to have always incremented outgoing seq nr.") and commit430d67bdcb04("net: hsr: Use the seqnr lock for frames received via interlink port.") holdseqnr_lockacross the whole forwarding path. Transmitting while holding the lock can create lock-dependency issues when HSR devices are stacked with other net devices, up to a real deadlock (see Reported-by/Closes).Since commit
aae9d6b616b5("hsr: Implement more robust duplicate discard for HSR"), duplicate discard is order-independent (sparse bitmaps), so only the sequence counter updates need serialization. Limitseqnr_lockto the counter updates inhandle_std_frame(); the master TX path (hsr_dev_xmit()) and the interlink RX path (hsr_handle_frame()) drop their outer lock, and the supervision frame builders release it right after their counter update.Concurrent forwarding may emit frames out of allocation order, which sparse-bitmap discard tolerates. The master/interlink tx statistics are no longer serialized; consistent per-cpu/per-queue statistics for all HSR paths are handled in a separate series.
Fixes 06afd2c31d33 ("hsr: Synchronize sending frames to have always incremented outgoing seq nr.")Fixes 430d67bdcb04 ("net: hsr: Use the seqnr lock for frames received via interlink port.")Reported-by [email protected]Cc <[email protected]> # aae9d6b616b5: hsr: Implement more robust duplicate discard for HSRSigned-off-by Xin Xie <[email protected]>--- net/hsr/hsr_device.c | 15 ++++----------- net/hsr/hsr_forward.c | 3 ++- net/hsr/hsr_slave.c | 11 +---------- 3 files changed, 7 insertions(+), 22 deletions(-)net/hsr/hsr_device.c +4
-11@@ -232,9 +232,7 @@ static netdev_tx_t hsr_dev_xmit(struct sk_buff *skb, struct net_device *dev) skb->dev = master->dev;skb_reset_mac_header (skb);skb_reset_mac_len (skb); -spin_lock_bh (&hsr->seqnr_lock);hsr_forward_skb (skb, master); -spin_unlock_bh (&hsr->seqnr_lock); }else {dev_core_stats_tx_dropped_inc (dev);dev_kfree_skb_any (skb);@@ -335,6 +333,7 @@ static void send_hsr_supervision_frame(struct hsr_port *port, hsr_stag->sequence_nr =htons (hsr->sequence_nr); hsr->sequence_nr++; } +spin_unlock_bh (&hsr->seqnr_lock); hsr_stag->tlv.HSR_TLV_type = type;/* HSRv0 has 6 unused bytes after the MAC */ @@ -356,14 +355,10 @@ static void send_hsr_supervision_frame(struct hsr_port *port,ether_addr_copy (hsr_sp->macaddress_A, hsr->macaddress_redbox); } -if (skb_put_padto (skb,ETH_ZLEN )) { -spin_unlock_bh (&hsr->seqnr_lock); +if (skb_put_padto (skb,ETH_ZLEN ))return ; - }hsr_forward_skb (skb, port); -spin_unlock_bh (&hsr->seqnr_lock); -return ; } static void send_prp_supervision_frame(struct hsr_port *master,@@ -390,6 +385,7 @@ static void send_prp_supervision_frame(struct hsr_port *master,spin_lock_bh (&hsr->seqnr_lock); hsr_stag->sequence_nr =htons (hsr->sup_sequence_nr); hsr->sup_sequence_nr++; +spin_unlock_bh (&hsr->seqnr_lock); hsr_stag->tlv.HSR_TLV_type =PRP_TLV_LIFE_CHECK_DD ; hsr_stag->tlv.HSR_TLV_length =sizeof (struct hsr_sup_payload );@@ -397,13 +393,10 @@ static void send_prp_supervision_frame(struct hsr_port *master, hsr_sp =skb_put (skb,sizeof (struct hsr_sup_payload ));ether_addr_copy (hsr_sp->macaddress_A, master->dev->dev_addr); -if (skb_put_padto (skb,ETH_ZLEN )) { -spin_unlock_bh (&hsr->seqnr_lock); +if (skb_put_padto (skb,ETH_ZLEN ))return ; - }hsr_forward_skb (skb, master); -spin_unlock_bh (&hsr->seqnr_lock); }/* Announce (supervision frame) timer function net/hsr/hsr_forward.c +2
-1@@ -621,9 +621,10 @@ static void handle_std_frame(struct sk_buff *skb,if (port->type ==HSR_PT_MASTER || port->type ==HSR_PT_INTERLINK ) {/* Sequence nr for the master/interlink node */ -lockdep_assert_held (&hsr->seqnr_lock); +spin_lock_bh (&hsr->seqnr_lock); frame->sequence_nr = hsr->sequence_nr; hsr->sequence_nr++; +spin_unlock_bh (&hsr->seqnr_lock); } }net/hsr/hsr_slave.c +1
-10@@ -73,16 +73,7 @@ static rx_handler_result_t hsr_handle_frame(struct sk_buff **pskb) }skb_reset_mac_len (skb); -/* Only the frames received over the interlink port will assign a -* sequence number and require synchronisation vs other sender. -*/ -if (port->type ==HSR_PT_INTERLINK ) { -spin_lock_bh (&hsr->seqnr_lock); -hsr_forward_skb (skb, port); -spin_unlock_bh (&hsr->seqnr_lock); - }else { -hsr_forward_skb (skb, port); - } +hsr_forward_skb (skb, port); finish_consume:return RX_HANDLER_CONSUMED ;Signature · 1 line
2.43.0
Thanks for the fix.
Re: [PATCH 2/4] net: hsr: shrink seqnr_lock to sequence counter updates
In reply to Xin XieQuoted 16 lines · Xin Xie
Commit
06afd2c31d33("hsr: Synchronize sending frames to have always incremented outgoing seq nr.") and commit430d67bdcb04("net: hsr: Use the seqnr lock for frames received via interlink port.") holdseqnr_lockacross the whole forwarding path. Transmitting while holding the lock can create lock-dependency issues when HSR devices are stacked with other net devices, up to a real deadlock (see Reported-by/Closes).Since commit
aae9d6b616b5("hsr: Implement more robust duplicate discard for HSR"), duplicate discard is order-independent (sparse bitmaps), so only the sequence counter updates need serialization. Limitseqnr_lockto the counter updates inhandle_std_frame(); the master TX path (hsr_dev_xmit()) and the interlink RX path (hsr_handle_frame()) drop their outer lock, and the supervision frame builders release it right after their counter update.
Quoted 4 lines
Concurrent forwarding may emit frames out of allocation order, which sparse-bitmap discard tolerates. The master/interlink tx statistics are no longer serialized; consistent per-cpu/per-queue statistics for all HSR paths are handled in a separate series.
This fix relies on the bitmap seq nr handling while the Fixes commits below do not have it. Doesn't this revert the synchronisation change leading to the problem it tried to solve at least in the stable kernels?
Quoted 6 lines
Fixes 06afd2c31d33 ("hsr: Synchronize sending frames to have always incremented outgoing seq nr.")Fixes 430d67bdcb04 ("net: hsr: Use the seqnr lock for frames received via interlink port.")Reported-by [email protected]Cc <[email protected]> # aae9d6b616b5: hsr: Implement more robust duplicate discard for HSRSigned-off-by Xin Xie <[email protected]>
Sebastian
Re: [PATCH 2/4] net: hsr: shrink seqnr_lock to sequence counter updates
Quoted 3 lines · Sebastian Andrzej Siewior
This fix relies on the bitmap seq nr handling while the Fixes commits below do not have it. Doesn't this revert the synchronisation change leading to the problem it tried to solve at least in the stable kernels?
You're right. Shrinking the lock can reintroduce ordering drops on pre-bitmap kernels, and the sender's bitmap does not make reordering safe for older peers.
Across ten VM runs per mode against an unmodified pre-bitmap HSRv0 peer, the narrowed-lock control delivered 19,128 of 20,000 test frames at the peer's master. The ordered-consumer prototype delivered all 20,000 exactly once.
I'll rework patch 2 around a single consumer for locally numbered traffic, including supervision. Each frame would be submitted to all its lower devices before the next frame is numbered. An idle caller would process only its own input inline; contention or re-entry would queue input to one BH worker.
No HSR lock would be held across segmentation or dev_queue_xmit(). Deferred inputs would retain their core TX recursion depth. This preserves the original per-lower submission ordering without relying on bitmap tolerance.
On PREEMPT_RT, consumer ownership would not provide priority inheritance from queued producers; progress depends on the consumer being scheduled. I will test this explicitly.
Signature · 1 line
Xin
PATCH v6 3/4 net: hsr: unfold GSO super-packets at the forward entry
In reply to Xin XieHSR/PRP require per-wire-frame tags and sequence numbers; treating a GSO skb as one frame breaks those semantics.
Classify GSO skbs at the forward entry by effective protocol rather than ingress port. Segment plain aggregates and process each segment normally, preserving local delivery and forwarding. Drop ETH_P_HSR and ETH_P_PRP aggregates (per-frame metadata cannot be rebuilt), classification failures (unreadable header, stacked or S-tag tagging), and aggregates with unreadable net_iov (device-memory) fragments:
their payload is not host-readable, and segmentation would leave the segment payload uninitialized, silently dropped at transmit or exposed where netmem transmit is enabled.
In-tree software GRO does not merge PRP RCT frames (its IPv4/IPv6 length checks reject trailing bytes); fixed-on GRO_HW output is outside this guarantee.
Also remove the GSO_MASK member types from the HSR master'shw_features: local traffic is segmented by the core before the forward path, so the master funnel branch sees single frames, and TSO and the other GSO_MASK offloads can no longer be enabled on the master; generic-segmentation-offload stays changeable and is cleared at runtime.
This patch depends on patch 2 ("net: hsr: shrink seqnr_lock to sequence counter updates") and the sparse-bitmap duplicate discard introduced by commit aae9d6b616b5 ("hsr: Implement more robust duplicate discard for HSR"); older trees require an adapted backport.
--- net/hsr/hsr_device.c | 2 +- net/hsr/hsr_forward.c | 106 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 106 insertions(+), 2 deletions(-)
net/hsr/hsr_device.c +1 -1
@@ -652,7 +652,7 @@ void hsr_dev_setup(struct net_device *dev) dev->needs_free_netdev = true; dev->hw_features =NETIF_F_SG |NETIF_F_FRAGLIST |NETIF_F_HIGHDMA | -NETIF_F_GSO_MASK |NETIF_F_HW_CSUM | +NETIF_F_HW_CSUM |NETIF_F_HW_VLAN_CTAG_TX |NETIF_F_HW_VLAN_CTAG_FILTER ;
net/hsr/hsr_forward.c +105 -1
121 lines not on this page · raw message
Signature · 1 line
2.43.0
Re: [PATCH 3/4] net: hsr: unfold GSO super-packets at the forward entry
Quoted 2 lines · Xin Xie
HSR/PRP require per-wire-frame tags and sequence numbers; treating a GSO skb as one frame breaks those semantics.
Wouldn't it be simpler to not allow such packets?
Sebastian
Re: [PATCH 3/4] net: hsr: unfold GSO super-packets at the forward entry
Quoted 1 line · Sebastian Andrzej Siewior
Wouldn't it be simpler to not allow such packets?
For master TX, yes: removing GSO features lets the core segment before ndo_start_xmit(), as patch 3 already does.
The interlink RX path is different. With GRO/GRO_HW disabled, it can still receive skbs that were already GSO at the sender. I reproduced this with ordinary TCP traffic over veth in a VM. Rejecting these skbs would drop valid traffic.
I'll retain segmentation for valid interlink GSO in the ordered consumer described in my patch 2 reply, before per-frame sequence allocation and HSR tag/PRP RCT construction. Each aggregate would finish before the next input, without holding an HSR lock across segmentation or transmission.
Signature · 1 line
Xin
Re: [PATCH 3/4] net: hsr: unfold GSO super-packets at the forward entry
Quoted 10 lines · Xin Xie
Quoted 1 line · Sebastian Andrzej Siewior
Wouldn't it be simpler to not allow such packets?
For master TX, yes: removing GSO features lets the core segment before
ndo_start_xmit(), as patch 3 already does.The interlink RX path is different. With GRO/GRO_HW disabled, it can still receive skbs that were already GSO at the sender. I reproduced this with ordinary TCP traffic over veth in a VM. Rejecting these skbs would drop valid traffic.
So you disable GRO on the interlink port and still receive GRO packets? Or is this performance related where you want to keep GRO enabled on the interlink port?
Quoted 5 lines
I'll retain segmentation for valid interlink GSO in the ordered consumer described in my patch 2 reply, before per-frame sequence allocation and HSR tag/PRP RCT construction. Each aggregate would finish before the next input, without holding an HSR lock across segmentation or transmission.
Sebastian
Re: [PATCH 3/4] net: hsr: unfold GSO super-packets at the forward entry
Quoted 3 lines · Sebastian Andrzej Siewior
So you disable GRO on the interlink port and still receive GRO packets? Or is this performance related where you want to keep GRO enabled on the interlink port?
I am not trying to keep GRO enabled for performance. The difference is between a physical interlink and a veth interlink.
With a physical interlink, TCP data is split into packets before it goes onto the wire via TSO/GSO. If GRO is off at the receiver, this interlink RX path needs no extra segmentation. Patch 3 is indeed unnecessary in this case.
With a veth interlink, there is no physical wire:
TCP -> prp0-peer -> prp0-int (interlink) -> PRP
A veth pair is simply an in-kernel memory pipe. When a large TCP skb arrives at prp0-peer, veth does not trigger TSO or software segmentation, it simply hands over the intact skb pointer directly to prp0-int.
Turning GRO off on prp0-int does not force prp0-peer to split it. I confirmed this with ordinary TCP traffic in a VM, with GRO off on both veth ends.
Patch 3 splits that skb before PRP assigns sequence numbers and adds RCTs. Each resulting frame needs its own sequence number and RCT. Ordinary non-GSO skbs skip the segmentation path.
In addition, if GRO disablement fails (e.g., buggy drivers or hardware-fixed GRO_HW that cannot be turned off via ethtool), Patch 3 also acts as a fallback defense for these edge cases.
--
Xin
Re: [PATCH 3/4] net: hsr: unfold GSO super-packets at the forward entry
On 2026-09-25 11:52:52 [+0200], Xin Xie wrote:
…
Quoted 7 lines
A veth pair is simply an in-kernel memory pipe. When a large TCP skb arrives at prp0-peer, veth does not trigger TSO or software segmentation, it simply hands over the intact skb pointer directly to prp0-int.
Turning GRO off on prp0-int does not force prp0-peer to split it. I confirmed this with ordinary TCP traffic in a VM, with GRO off on both veth ends.
So veth. The test suite is using veth I don't remember that it was (is) a problem there. But then there is no TCP.
Wouldn't it be also a problem if you attach veth interface to a bridge and that large TCP skb would have to leave the bridge via a physical port?
This does not sound like it is limited to hsr. I think it deserves a helper similar to skb_linearize().
Quoted 8 lines
Patch 3 splits that skb before PRP assigns sequence numbers and adds RCTs. Each resulting frame needs its own sequence number and RCT. Ordinary non-GSO skbs skip the segmentation path.
In addition, if GRO disablement fails (e.g., buggy drivers or hardware-fixed
GRO_HWthat cannot be turned off via ethtool), Patch 3 also acts as a fallback defense for these edge cases.
Sebastian
Re: [PATCH 3/4] net: hsr: unfold GSO super-packets at the forward entry
Quoted 5 lines
So veth. The test suite is using veth I don't remember that it was (is) a problem there. But then there is no TCP.
Wouldn't it be also a problem if you attach veth interface to a bridge and that large TCP skb would have to leave the bridge via a physical port?
When a bridge forwards a GSO skb from veth to a physical port, the outgoing TX path handles segmentation. validate_xmit_skb() checks whether software segmentation is needed. The networking core performs software segmentation as needed and can use hardware segmentation offload when available for that skb.
The software HSR/PRP path needs to split the GSO skb earlier. Each segment needs its own sequence number and HSR tag or PRP trailer. We add these before calling dev_queue_xmit() on the lower device. The normal TX segmentation code cannot create these per-frame HSR/PRP fields for us.
TCP is the case I reproduced, but this is not limited to TCP. UDP GSO skbs can also pass through veth without being segmented. Passing normal PROFINET RT or GOOSE frames through veth does not make them GSO skbs. Patch 3 skips segmentation for these frames. They still follow the normal HSR/PRP forwarding path.
Quoted 2 lines
This does not sound like it is limited to hsr. I think it deserves a helper similar to
skb_linearize().
Patch 3 already uses the common __skb_gso_segment() helper, then passes each segment through normal HSR/PRP forwarding. Do you mean a new common wrapper around this helper?
Signature · 1 line
Xin
Re: [PATCH 3/4] net: hsr: unfold GSO super-packets at the forward entry
Quoted 6 lines · Xin Xie
Quoted 2 lines
This does not sound like it is limited to hsr. I think it deserves a helper similar to
skb_linearize().Patch 3 already uses the common
__skb_gso_segment()helper, then passes each segment through normal HSR/PRP forwarding. Do you mean a new common wrapper around this helper?
I must have missed this part. It is okay then.
Sebastian
PATCH v6 4/4 selftests: net: hsr: cover GSO super-packets on PRP slave ingress
In reply to Xin XieAdd a new always-run HSR/PRP kselftest (hsr_gro_superpacket.sh) covering the GRO/GRO_HW state on enslaved devices, the HSR master's GSO/TSO feature advertisement, a TSO stream through the forward path, and a PRP LAN-slave plain-GSO regression in which aggregates are segmented at the forward entry and delivered locally to the master.
The test asserts that plain aggregates are unfolded per-frame on any ingress role and that local delivery and forwarding are preserved; interface-counter deltas are the evidence. Server readiness and reap waits are bounded, and the tool dependencies (ethtool, iperf3, timeout) are enforced via check_tool().
--- tools/testing/selftests/net/hsr/Makefile | 1 + .../selftests/net/hsr/hsr_gro_superpacket.sh | 678 ++++++++++++++++++ 2 files changed, 679 insertions(+) create mode 100755 tools/testing/selftests/net/hsr/hsr_gro_superpacket.sh
tools/testing/selftests/net/hsr/Makefile +1 -0
@@ -3,6 +3,7 @@ top_srcdir = ../../../../.. TEST_PROGS := \ + hsr_gro_superpacket.sh \ hsr_ping.sh \ hsr_redbox.sh \ link_faults.sh \
tools/testing/selftests/net/hsr/hsr_gro_superpacket.sh new file +678 -0
678 lines not on this page · raw message
Signature · 1 line
2.43.0