Skip to contentSkip to conversation

MACsec IEEE 802.1AE implementation

RFC PATCH Superseded Sabrina Dubroca 3 participants 9 messages Dec 28, 2015 – Jan 8, 2016

Subsystems (3)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Sabrina Dubroca

author maintainer
#0 lore raw

cover MACsec IEEE 802.1AE implementation

MACsec (IEEE 802.1AE [0]) is a protocol that provides security for wired ethernet LANs. MACsec offers two protection modes:
authentication only, or authenticated encryption.

MACsec defines "secure channels" that allow transmission from one node to one or more others. Communication on a channel is done over a succession of "secure associations", that each use a specific key. Secure associations are identified by their "association number" in the range 0..3. A secure association is retired when its 32-bit packet number would wrap, and the same association number can later be reused with a new key and packet number.

The standard mode of encryption is GCM AES with 128 bits keys, although an extension allows 256 bits keys [1] (not implemented in this submission).

When using MACsec, an extra header, called "SecTAG", is added between the ethernet header and the original payload:

 +---------------------------------+----------------+----------------+
 |        (MACsec ethertype)       |     TCI_AN     |       SL       |
 +---------------------------------+----------------+----------------+
 |                           Packet Number                           |
 +-------------------------------------------------------------------+
 |                     Secure Channel Identifier                     |
 |                            (optional)                             |
 +-------------------------------------------------------------------+

TCI_AN:
version
end_station
sci_present
scb
encrypted
changed_text
association_number (2 bits)
SL:
short_length (6 bits)
unused (2 bits)

The ethertype for the packet is set to 0x88E5, and the original ethertype becomes part of the secure payload, which may be encrypted. The ethernet header and the SecTAG are always transmitted in the clear, but are integrity-protected.

MACsec supports optional replay protection with a configurable replay window.

MACsec is designed to be used with the MKA extension to 802.1X (MACsec Key Agreement protocol) [2], which provides channel attribution and key distribution to the nodes, but can also be used with static keys getting fed manually by an administrator.

Optional (not supported yet) features:
- confidentiality offset: in encryption mode, part of the payload may be left unencrypted.
- choice of cipher suite: GCM AES with 256 bits has been standardised [1].

Implementation

A netdevice is created on top of a real device for each TX secure channel, like we do for VLANs. Multiple TX channels can be created on top of the same underlying device.

Several other approaches were considered for the RX path:

- dev_add_pack: doesn't work, because we want to filter out unprotected packets
- transparent mode: MACsec would be enabled directly on the real netdevice. For this, we cannot use a rx_handler directly because MACsec must be available for underlying devices enslaved in a bridge or in a bond, so we need a hook directly in __netif_receive_skb_core. This approach makes it harder to filter non-encrypted packets on RX without forcing the user to setup some rules, so the "transparent" mode is not so transparent after all. It also makes TX more complex than with a dedicated netdevice.

One issue with the proposed implementation is that the qdisc layer for the real device operates on already encrypted packets.

Netlink API

This is currently a mix of rtnetlink (to create the device and set up the TX channel) and genl (for RX channels, secure associations and their keys). genl provides clean demultiplexing of the {TX,RX}{SC,SA} commands.

Use cases

The normal use case is wired LANs, including veth and slave devices for bonding/teaming or bridges.

MACsec can also be used on any device that makes a full ethernet header visible, for example VXLAN.
The VXLAN+MACsec setup would be:

hypervisor | virtual machine
<real_dev>---<VXLAN>---|---<dev>---<macsec_dev>

And the packets would look like this:

| eth | IP | UDP | VXLAN | eth | MACsec | IP | ... | MACsec ICV |

One benefit on this approach to encryption in the cloud is that the payload is encrypted by the tenant, not by the tunnel provider, thus the tenant has full control over the keys.

Future plans:
- offload to hardware, on nics that support it (at least some ixgbe)
- implement optional features

The corresponding iproute code will be posted later.

[0] http://standards.ieee.org/getieee802/download/802.1AE-2006.pdf
[1] http://standards.ieee.org/getieee802/download/802.1AEbn-2011.pdf
[2] http://standards.ieee.org/getieee802/download/802.1X-2010.pdf

Sabrina Dubroca (3):
uapi: add MACsec bits
net: add MACsec netdevice priv_flags and helper
macsec: introduce IEEE 802.1AE driver

 drivers/net/Kconfig            |    7 +
 drivers/net/Makefile           |    1 +
 drivers/net/macsec.c           | 2991 ++++++++++++++++++++++++++++++++++++++++
 include/linux/netdevice.h      |    7 +
 include/uapi/linux/Kbuild      |    1 +
 include/uapi/linux/if_ether.h  |    1 +
 include/uapi/linux/if_link.h   |   21 +
 include/uapi/linux/if_macsec.h |  160 +++
 8 files changed, 3189 insertions(+)
 create mode 100644 drivers/net/macsec.c
 create mode 100644 include/uapi/linux/if_macsec.h
Signature · 1 line
2.6.4
author maintainer
#1 lore raw

RFC PATCH 1/3 uapi: add MACsec bits

---
 include/uapi/linux/Kbuild      |   1 +
 include/uapi/linux/if_ether.h  |   1 +
 include/uapi/linux/if_link.h   |  21 ++++++
 include/uapi/linux/if_macsec.h | 160 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 183 insertions(+)
 create mode 100644 include/uapi/linux/if_macsec.h
include/uapi/linux/Kbuild +1 -0
index c2e5d6cb34e3..3ee4211af77b 100644
@@ -173,6 +173,7 @@ header-y += if_hippi.h
 header-y += if_infiniband.h
 header-y += if_link.h
 header-y += if_ltalk.h
+header-y += if_macsec.h
 header-y += if_packet.h
 header-y += if_phonet.h
 header-y += if_plip.h
include/uapi/linux/if_ether.h +1 -0
index ea9221b0331a..4a93051c578c 100644
@@ -83,6 +83,7 @@
 #define ETH_P_8021AD	0x88A8          /* 802.1ad Service VLAN		*/
 #define ETH_P_802_EX1	0x88B5		/* 802.1 Local Experimental 1.  */
 #define ETH_P_TIPC	0x88CA		/* TIPC 			*/
+#define ETH_P_MACSEC	0x88E5		/* 802.1ae MACsec */
 #define ETH_P_8021AH	0x88E7          /* 802.1ah Backbone Service Tag */
 #define ETH_P_MVRP	0x88F5          /* 802.1Q MVRP                  */
 #define ETH_P_1588	0x88F7		/* IEEE 1588 Timesync */
include/uapi/linux/if_link.h +21 -0
index a30b78090594..be0e6caf0bf0 100644
@@ -401,6 +401,27 @@ enum {
 
 #define IFLA_VRF_MAX (__IFLA_VRF_MAX - 1)
 
+/* MACSEC section */
+enum {
+	IFLA_MACSEC_UNSPEC,
+	IFLA_MACSEC_SCI,
+	IFLA_MACSEC_PORT,
+	IFLA_MACSEC_ICV_LEN,
+	IFLA_MACSEC_CIPHER_SUITE,
+	IFLA_MACSEC_WINDOW,
+	IFLA_MACSEC_ENCODING_SA,
+	IFLA_MACSEC_ENCRYPT,
+	IFLA_MACSEC_PROTECT,
+	IFLA_MACSEC_INC_SCI,
+	IFLA_MACSEC_ES,
+	IFLA_MACSEC_SCB,
+	IFLA_MACSEC_REPLAY_PROTECT,
+	IFLA_MACSEC_VALIDATION,
+	__IFLA_MACSEC_MAX,
+};
+
+#define IFLA_MACSEC_MAX (__IFLA_MACSEC_MAX - 1)
+
 /* IPVLAN section */
 enum {
 	IFLA_IPVLAN_UNSPEC,
include/uapi/linux/if_macsec.h new file +160 -0

160 lines not on this page · raw message

Signature · 1 line
2.6.4
author maintainer
#2 lore raw

RFC PATCH 2/3 net: add MACsec netdevice priv_flags and helper

In reply to Sabrina Dubroca
---
 include/linux/netdevice.h | 7 +++++++
 1 file changed, 7 insertions(+)
include/linux/netdevice.h +7 -0
index c20b814e46a0..4f80f621ecf9 100644
@@ -1319,6 +1319,7 @@ enum netdev_priv_flags {
 	IFF_OPENVSWITCH			= 1<<22,
 	IFF_L3MDEV_SLAVE		= 1<<23,
 	IFF_TEAM			= 1<<24,
+	IFF_MACSEC			= 1<<25,
 };
 
 #define IFF_802_1Q_VLAN			IFF_802_1Q_VLAN
@@ -1346,6 +1347,7 @@ enum netdev_priv_flags {
 #define IFF_OPENVSWITCH			IFF_OPENVSWITCH
 #define IFF_L3MDEV_SLAVE		IFF_L3MDEV_SLAVE
 #define IFF_TEAM			IFF_TEAM
+#define IFF_MACSEC			IFF_MACSEC
 
 /**
  *	struct net_device - The DEVICE structure.
@@ -3964,6 +3966,11 @@ static inline void skb_gso_error_unwind(struct sk_buff *skb, __be16 protocol,
 	skb->mac_len = mac_len;
 }
 
+static inline bool netif_is_macsec(const struct net_device *dev)
+{
+	return dev->priv_flags & IFF_MACSEC;
+}
+
 static inline bool netif_is_macvlan(const struct net_device *dev)
 {
 	return dev->priv_flags & IFF_MACVLAN;
Signature · 1 line
2.6.4
author maintainer
#3 lore raw

RFC PATCH 3/3 macsec: introduce IEEE 802.1AE driver

In reply to Sabrina Dubroca

This is an implementation of MACsec/IEEE 802.1AE. This driver provides authentication and encryption of traffic in a LAN, typically with GCM-AES-128, and optional replay protection.

http://standards.ieee.org/getieee802/download/802.1AE-2006.pdf

---
 drivers/net/Kconfig  |    7 +
 drivers/net/Makefile |    1 +
 drivers/net/macsec.c | 2991 ++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 2999 insertions(+)
 create mode 100644 drivers/net/macsec.c
drivers/net/Kconfig +7 -0
index f184fb5bd110..2a1ba62b7da2 100644
@@ -193,6 +193,13 @@ config GENEVE
 	  To compile this driver as a module, choose M here: the module
 	  will be called geneve.
 
+config MACSEC
+	tristate "IEEE 802.1AE MAC-level encryption (MACsec)"
+	select CRYPTO_AES
+	select CRYPTO_GCM
+	---help---
+	   MACsec is an encryption standard for Ethernet.
+
 config NETCONSOLE
 	tristate "Network console logging support"
 	---help---
drivers/net/Makefile +1 -0
index 900b0c5320bb..1aa7cb845663 100644
@@ -10,6 +10,7 @@ obj-$(CONFIG_IPVLAN) += ipvlan/
 obj-$(CONFIG_DUMMY) += dummy.o
 obj-$(CONFIG_EQUALIZER) += eql.o
 obj-$(CONFIG_IFB) += ifb.o
+obj-$(CONFIG_MACSEC) += macsec.o
 obj-$(CONFIG_MACVLAN) += macvlan.o
 obj-$(CONFIG_MACVTAP) += macvtap.o
 obj-$(CONFIG_MII) += mii.o
drivers/net/macsec.c new file +2991 -0

2,980 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.6.4

Part of the diff is not shown here. View the raw message.

#4 lore raw

Re: [RFC PATCH net-next 3/3] macsec: introduce IEEE 802.1AE driver

Quoted 4 lines · Sabrina Dubroca
drivers/net/macsec.c new file +4 -0
+		if (h->short_length)
+			return len == h->short_length + 24;
+		else
+			return len >= 72;

[..]

Quoted 1 line
drivers/net/macsec.c new file +1 -0
+			return len == h->short_length + 32;

[..]

Quoted 1 line
drivers/net/macsec.c new file +1 -0
+			return len >= 80;

[..]

Quoted 3 lines
drivers/net/macsec.c new file +3 -0
+			return len == 8 + icv_len + h->short_length;
+		else
+			return len >= 8 + icv_len + 48;

[..]

drivers/net/macsec.c line 457

Quoted 4 lines
drivers/net/macsec.c new file +4 -0
+		if (h->short_length)
+			return len == 16 + icv_len + h->short_length;
+		else
+			return len >= 16 + icv_len + 48;

Could you add some defines instead of magic numbers?

drivers/net/macsec.c line 483

Quoted 3 lines
drivers/net/macsec.c new file +3 -0
+	tx_sa->next_pn++;
+	if (tx_sa->next_pn == 0) {
+		pr_notice("PN wrapped, transitionning to !oper\n");

Is that _notice intentional?
I'm only asking because it seems we printk unconditionally in response to network traffic & I don't get what operator should do in response to that message.

Quoted 7 lines
drivers/net/macsec.c new file +7 -0
+static void macsec_encrypt_done(struct crypto_async_request *base, int err)
+{
+	struct sk_buff *skb = base->data;
+	struct net_device *dev = skb->dev;
+	struct macsec_dev *macsec = macsec_priv(dev);
+	struct macsec_tx_sa *sa = macsec_skb_cb(skb)->tx_sa;
+	int len, ret;

drivers/net/macsec.c line 550

Quoted 10 lines
drivers/net/macsec.c new file +10 -0
+
+	aead_request_free(macsec_skb_cb(skb)->req);
+
+	rcu_read_lock_bh();
+	macsec_encrypt_finish(skb, dev);
+	macsec_count_tx(skb, &macsec->secy.tx_sc, macsec_skb_cb(skb)->tx_sa);
+	len = skb->len;
+	ret = dev_queue_xmit(skb);
+	count_tx(dev, ret, len);
+	rcu_read_unlock_bh();

What was the rcu_read_lock_bh protecting?

Quoted 14 lines
drivers/net/macsec.c new file +14 -0
+static void macsec_decrypt_done(struct crypto_async_request *base, int err)
+{
+	struct sk_buff *skb = base->data;
+	struct net_device *dev = skb->dev;
+	struct macsec_dev *macsec = macsec_priv(dev);
+	struct macsec_rx_sa *rx_sa = macsec_skb_cb(skb)->rx_sa;
+	int len, ret;
+
+	aead_request_free(macsec_skb_cb(skb)->req);
+
+	rcu_read_lock_bh();
+	macsec_finalize_skb(skb, macsec->secy.icv_len,
+			    macsec_extra_len(macsec_skb_cb(skb)->has_sci));
+	macsec_reset_skb(skb, macsec->secy.netdev);

drivers/net/macsec.c line 729

Quoted 10 lines
drivers/net/macsec.c new file +10 -0
+
+	macsec_rxsa_put(rx_sa);
+	len = skb->len;
+	ret = netif_rx(skb);
+	if (ret == NET_RX_SUCCESS)
+		count_rx(dev, len);
+	else
+		macsec->secy.netdev->stats.rx_dropped++;
+
+	rcu_read_unlock_bh();

Same question.

Quoted 14 lines
drivers/net/macsec.c new file +14 -0
+static void handle_not_macsec(struct sk_buff *skb)
+{
+	struct macsec_rxh_data *rxd = macsec_data_rcu(skb->dev);
+	struct macsec_dev *macsec;
+
+	/* 10.6 If the management control validateFrames is not
+	 * Strict, frames without a SecTAG are received, counted, and
+	 * delivered to the Controlled Port
+	 */
+	list_for_each_entry_rcu(macsec, &rxd->secys, secys) {
+		struct sk_buff *nskb;
+		int ret;
+		struct pcpu_secy_stats *secy_stats = this_cpu_ptr(macsec->stats);
+

drivers/net/macsec.c line 857

Quoted 10 lines
drivers/net/macsec.c new file +10 -0
+		if (macsec->secy.validate_frames == MACSEC_VALIDATE_STRICT) {
+			u64_stats_update_begin(&secy_stats->syncp);
+			secy_stats->stats.InPktsNoTag++;
+			u64_stats_update_end(&secy_stats->syncp);
+			continue;
+		}
+
+		/* deliver on this port */
+		nskb = skb_clone(skb, GFP_ATOMIC);
+		nskb->dev = macsec->secy.netdev;

nskb == NULL handling?

Quoted 8 lines
drivers/net/macsec.c new file +8 -0
+static rx_handler_result_t macsec_handle_frame(struct sk_buff **pskb)
+{
+	struct sk_buff *skb = *pskb;
+	struct net_device *dev = skb->dev;
+	struct macsec_eth_header *hdr;
+	struct macsec_secy *secy = NULL;
+	struct macsec_rx_sc *rx_sc;
+	struct macsec_rx_sa *rx_sa;

drivers/net/macsec.c line 892

Quoted 10 lines
drivers/net/macsec.c new file +10 -0
+	struct macsec_rxh_data *rxd;
+	struct macsec_dev *macsec;
+	sci_t sci;
+	u32 pn, lowest_pn;
+	bool cbit;
+	struct pcpu_rx_sc_stats *rxsc_stats;
+	struct pcpu_secy_stats *secy_stats;
+	bool pulled_sci;
+
+	rcu_read_lock_bh();

Why? Seems its because of

drivers/net/macsec.c line 897

Quoted 4 lines
drivers/net/macsec.c new file +4 -0
+	if (skb_headroom(skb) < ETH_HLEN)
+		goto drop_nosa;
+
+	rxd = macsec_data_rcu(skb->dev);

this, but rxd isn't dereferenced until a lot later in the function.

Also: macsec_data_rcu uses rcu_dereference() but this used rcu_read_lock_bh, is that structure protected by RCU or RCU-bh?

Quoted 16 lines
drivers/net/macsec.c new file +16 -0
+	pn = ntohl(hdr->packet_number);
+	if (secy->replay_protect) {
+		bool late;
+
+		spin_lock(&rx_sa->lock);
+		late = rx_sa->next_pn >= secy->replay_window &&
+		       pn < (rx_sa->next_pn - secy->replay_window);
+		spin_unlock(&rx_sa->lock);
+
+		if (late) {
+			u64_stats_update_begin(&rxsc_stats->syncp);
+			rxsc_stats->stats.InPktsLate++;
+			u64_stats_update_end(&rxsc_stats->syncp);
+			goto drop;
+		}
+	}

[..]

Quoted 4 lines
drivers/net/macsec.c new file +4 -0
+	spin_lock(&rx_sa->lock);
+	if (rx_sa->next_pn >= secy->replay_window)
+		lowest_pn = rx_sa->next_pn - secy->replay_window;
+	else

drivers/net/macsec.c line 1037

Quoted 10 lines
drivers/net/macsec.c new file +10 -0
+		lowest_pn = 0;
+
+	if (secy->replay_protect && pn < lowest_pn) {
+		spin_unlock(&rx_sa->lock);
+		pr_debug("packet_number too small: %u < %u\n", pn, lowest_pn);
+		u64_stats_update_begin(&rxsc_stats->syncp);
+		rxsc_stats->stats.InPktsLate++;
+		u64_stats_update_end(&rxsc_stats->syncp);
+		goto drop;
+	}

I don't understand why this seems to perform replay check twice?

Quoted 34 lines
drivers/net/macsec.c new file +33 -0
+	if (secy->validate_frames != MACSEC_VALIDATE_DISABLED) {
+		u64_stats_update_begin(&rxsc_stats->syncp);
+		if (hdr->tci_an & MACSEC_TCI_E)
+			rxsc_stats->stats.InOctetsDecrypted += skb->len;
+		else
+			rxsc_stats->stats.InOctetsValidated += skb->len;
+		u64_stats_update_end(&rxsc_stats->syncp);
+	}
 
+	if (!macsec_skb_cb(skb)->valid) {
+		spin_unlock(&rx_sa->lock);
+
+		/* 10.6.5 */
+		if (hdr->tci_an & MACSEC_TCI_C ||
+		    secy->validate_frames == MACSEC_VALIDATE_STRICT) {
+			u64_stats_update_begin(&rxsc_stats->syncp);
+			rxsc_stats->stats.InPktsNotValid++;
+			u64_stats_update_end(&rxsc_stats->syncp);
+			goto drop;
+		}
+
+		u64_stats_update_begin(&rxsc_stats->syncp);
+		if (secy->validate_frames == MACSEC_VALIDATE_CHECK) {
+			rxsc_stats->stats.InPktsInvalid++;
+			this_cpu_inc(rx_sa->stats->InPktsInvalid);
+		} else if (pn < lowest_pn) {
+			rxsc_stats->stats.InPktsDelayed++;
+		} else {
+			rxsc_stats->stats.InPktsUnchecked++;
+		}
+		u64_stats_update_end(&rxsc_stats->syncp);
+	} else {
+		u64_stats_update_begin(&rxsc_stats->syncp);
+		if (pn < lowest_pn) {

drivers/net/macsec.c line 1082

Quoted 10 lines
drivers/net/macsec.c new file +10 -0
+			rxsc_stats->stats.InPktsDelayed++;
+		} else {
+			rxsc_stats->stats.InPktsOK++;
+			this_cpu_inc(rx_sa->stats->InPktsOK);
+		}
+		u64_stats_update_end(&rxsc_stats->syncp);
+
+		if (pn >= rx_sa->next_pn)
+			rx_sa->next_pn = pn + 1;
+		spin_unlock(&rx_sa->lock);

Do you think its feasible to rearrange the above so that rx_sa->lock/unlock (next_pn test and increment) are grouped more closesly?

drivers/net/macsec.c line 1134

Quoted 5 lines
drivers/net/macsec.c new file +5 -0
+		/* not strict, the frame (with the SecTAG and ICV
+		 * removed) is delivered to the Controlled Port.
+		 */
+		nskb = skb_clone(skb, GFP_ATOMIC);
+		macsec_reset_skb(nskb, macsec->secy.netdev);

nskb == NULL handling?

I'll have another look at your patch set later this week.

Thanks,
Florian

author maintainer
#5 lore raw

Re: [RFC PATCH net-next 3/3] macsec: introduce IEEE 802.1AE driver

Quoted 14 lines · Florian Westphal
Quoted 4 lines · Sabrina Dubroca
drivers/net/macsec.c new file +4 -0
+		if (h->short_length)
+			return len == h->short_length + 24;
+		else
+			return len >= 72;

[..]

Quoted 1 line
drivers/net/macsec.c new file +1 -0
+			return len == h->short_length + 32;

[..]

Quoted 1 line
drivers/net/macsec.c new file +1 -0
+			return len >= 80;

[..]

Quoted 3 lines
drivers/net/macsec.c new file +3 -0
+			return len == 8 + icv_len + h->short_length;
+		else
+			return len >= 8 + icv_len + 48;

[..]

Quoted 6 lines

drivers/net/macsec.c line 457

Quoted 4 lines
drivers/net/macsec.c new file +4 -0
+		if (h->short_length)
+			return len == 16 + icv_len + h->short_length;
+		else
+			return len >= 16 + icv_len + 48;

Could you add some defines instead of magic numbers?

Sure.

Quoted 8 lines

drivers/net/macsec.c line 483

Quoted 3 lines
drivers/net/macsec.c new file +3 -0
+	tx_sa->next_pn++;
+	if (tx_sa->next_pn == 0) {
+		pr_notice("PN wrapped, transitionning to !oper\n");

Is that _notice intentional?
I'm only asking because it seems we printk unconditionally in response to network traffic & I don't get what operator should do in response to that message.

The operator should install a new tx_sa, or MKA should have already installed a new one and switched to it.
I can remove this message, or make it a pr_debug.

Quoted 11 lines
Quoted 11 lines
drivers/net/macsec.c new file +11 -0
+static void macsec_encrypt_done(struct crypto_async_request *base, int err)
+{
+	struct sk_buff *skb = base->data;
+	struct net_device *dev = skb->dev;
+	struct macsec_dev *macsec = macsec_priv(dev);
+	struct macsec_tx_sa *sa = macsec_skb_cb(skb)->tx_sa;
+	int len, ret;
+
+	aead_request_free(macsec_skb_cb(skb)->req);
+
+	rcu_read_lock_bh();
Quoted 8 lines

drivers/net/macsec.c line 550

Quoted 6 lines
drivers/net/macsec.c new file +6 -0
+	macsec_encrypt_finish(skb, dev);
+	macsec_count_tx(skb, &macsec->secy.tx_sc, macsec_skb_cb(skb)->tx_sa);
+	len = skb->len;
+	ret = dev_queue_xmit(skb);
+	count_tx(dev, ret, len);
+	rcu_read_unlock_bh();

What was the rcu_read_lock_bh protecting?

this_cpu_ptr in macsec_count_tx and count_tx. Separate get_cpu_ptr in both functions seem a bit wasteful, and dev_queue_xmit will also disable bh.

I could turn that into a preempt_disable with a comment (something like "covers multiple accesses to pcpu variables"). Or I could get rid of it, and use get/put_cpu_ptr in macsec_count_tx/count_tx. Note that macsec_count_tx/count_tx (and count_rx below) are also called from the normal packet processing path, where we already run under rcu_read_lock_bh anyway, so avoiding the overhead of an extra get_cpu_ptr seems preferable.

Quoted 18 lines
Quoted 18 lines
drivers/net/macsec.c new file +18 -0
+static void macsec_decrypt_done(struct crypto_async_request *base, int err)
+{
+	struct sk_buff *skb = base->data;
+	struct net_device *dev = skb->dev;
+	struct macsec_dev *macsec = macsec_priv(dev);
+	struct macsec_rx_sa *rx_sa = macsec_skb_cb(skb)->rx_sa;
+	int len, ret;
+
+	aead_request_free(macsec_skb_cb(skb)->req);
+
+	rcu_read_lock_bh();
+	macsec_finalize_skb(skb, macsec->secy.icv_len,
+			    macsec_extra_len(macsec_skb_cb(skb)->has_sci));
+	macsec_reset_skb(skb, macsec->secy.netdev);
+
+	macsec_rxsa_put(rx_sa);
+	len = skb->len;
+	ret = netif_rx(skb);
Quoted 8 lines

drivers/net/macsec.c line 729

Quoted 6 lines
drivers/net/macsec.c new file +6 -0
+	if (ret == NET_RX_SUCCESS)
+		count_rx(dev, len);
+	else
+		macsec->secy.netdev->stats.rx_dropped++;
+
+	rcu_read_unlock_bh();

Same question.

Yes, that's pretty much the same situation.

Quoted 18 lines
Quoted 18 lines
drivers/net/macsec.c new file +18 -0
+static void handle_not_macsec(struct sk_buff *skb)
+{
+	struct macsec_rxh_data *rxd = macsec_data_rcu(skb->dev);
+	struct macsec_dev *macsec;
+
+	/* 10.6 If the management control validateFrames is not
+	 * Strict, frames without a SecTAG are received, counted, and
+	 * delivered to the Controlled Port
+	 */
+	list_for_each_entry_rcu(macsec, &rxd->secys, secys) {
+		struct sk_buff *nskb;
+		int ret;
+		struct pcpu_secy_stats *secy_stats = this_cpu_ptr(macsec->stats);
+
+		if (macsec->secy.validate_frames == MACSEC_VALIDATE_STRICT) {
+			u64_stats_update_begin(&secy_stats->syncp);
+			secy_stats->stats.InPktsNoTag++;
+			u64_stats_update_end(&secy_stats->syncp);
Quoted 8 lines

drivers/net/macsec.c line 857

Quoted 6 lines
drivers/net/macsec.c new file +6 -0
+			continue;
+		}
+
+		/* deliver on this port */
+		nskb = skb_clone(skb, GFP_ATOMIC);
+		nskb->dev = macsec->secy.netdev;

nskb == NULL handling?

Right, I'll take care of both of these, thanks.

Quoted 21 lines
Quoted 8 lines
drivers/net/macsec.c new file +8 -0
+static rx_handler_result_t macsec_handle_frame(struct sk_buff **pskb)
+{
+	struct sk_buff *skb = *pskb;
+	struct net_device *dev = skb->dev;
+	struct macsec_eth_header *hdr;
+	struct macsec_secy *secy = NULL;
+	struct macsec_rx_sc *rx_sc;
+	struct macsec_rx_sa *rx_sa;

drivers/net/macsec.c line 892

Quoted 10 lines
drivers/net/macsec.c new file +10 -0
+	struct macsec_rxh_data *rxd;
+	struct macsec_dev *macsec;
+	sci_t sci;
+	u32 pn, lowest_pn;
+	bool cbit;
+	struct pcpu_rx_sc_stats *rxsc_stats;
+	struct pcpu_secy_stats *secy_stats;
+	bool pulled_sci;
+
+	rcu_read_lock_bh();

Why? Seems its because of

Quoted 6 lines

drivers/net/macsec.c line 897

Quoted 4 lines
drivers/net/macsec.c new file +4 -0
+	if (skb_headroom(skb) < ETH_HLEN)
+		goto drop_nosa;
+
+	rxd = macsec_data_rcu(skb->dev);

this, but rxd isn't dereferenced until a lot later in the function.

I'll move it down.

Quoted 2 lines

Also: macsec_data_rcu uses rcu_dereference() but this used rcu_read_lock_bh, is that structure protected by RCU or RCU-bh?

I'll turn that into a rcu_dereference_bh for the next submission.

Quoted 19 lines
Quoted 16 lines
drivers/net/macsec.c new file +16 -0
+	pn = ntohl(hdr->packet_number);
+	if (secy->replay_protect) {
+		bool late;
+
+		spin_lock(&rx_sa->lock);
+		late = rx_sa->next_pn >= secy->replay_window &&
+		       pn < (rx_sa->next_pn - secy->replay_window);
+		spin_unlock(&rx_sa->lock);
+
+		if (late) {
+			u64_stats_update_begin(&rxsc_stats->syncp);
+			rxsc_stats->stats.InPktsLate++;
+			u64_stats_update_end(&rxsc_stats->syncp);
+			goto drop;
+		}
+	}

[..]

Quoted 16 lines
Quoted 4 lines
drivers/net/macsec.c new file +4 -0
+	spin_lock(&rx_sa->lock);
+	if (rx_sa->next_pn >= secy->replay_window)
+		lowest_pn = rx_sa->next_pn - secy->replay_window;
+	else

drivers/net/macsec.c line 1037

Quoted 10 lines
drivers/net/macsec.c new file +10 -0
+		lowest_pn = 0;
+
+	if (secy->replay_protect && pn < lowest_pn) {
+		spin_unlock(&rx_sa->lock);
+		pr_debug("packet_number too small: %u < %u\n", pn, lowest_pn);
+		u64_stats_update_begin(&rxsc_stats->syncp);
+		rxsc_stats->stats.InPktsLate++;
+		u64_stats_update_end(&rxsc_stats->syncp);
+		goto drop;
+	}

I don't understand why this seems to perform replay check twice?

This is part of the specification (802.1AE-2006 figure 10-5). The first check is done before attempting to decrypt the packet, then once again after decrypting.
This part (and the other checks until deliver:) are missing in the macsec_decrypt_done callback, I need to add this.

Quoted 39 lines
Quoted 39 lines
drivers/net/macsec.c new file +38 -0
+	if (secy->validate_frames != MACSEC_VALIDATE_DISABLED) {
+		u64_stats_update_begin(&rxsc_stats->syncp);
+		if (hdr->tci_an & MACSEC_TCI_E)
+			rxsc_stats->stats.InOctetsDecrypted += skb->len;
+		else
+			rxsc_stats->stats.InOctetsValidated += skb->len;
+		u64_stats_update_end(&rxsc_stats->syncp);
+	}
 
+	if (!macsec_skb_cb(skb)->valid) {
+		spin_unlock(&rx_sa->lock);
+
+		/* 10.6.5 */
+		if (hdr->tci_an & MACSEC_TCI_C ||
+		    secy->validate_frames == MACSEC_VALIDATE_STRICT) {
+			u64_stats_update_begin(&rxsc_stats->syncp);
+			rxsc_stats->stats.InPktsNotValid++;
+			u64_stats_update_end(&rxsc_stats->syncp);
+			goto drop;
+		}
+
+		u64_stats_update_begin(&rxsc_stats->syncp);
+		if (secy->validate_frames == MACSEC_VALIDATE_CHECK) {
+			rxsc_stats->stats.InPktsInvalid++;
+			this_cpu_inc(rx_sa->stats->InPktsInvalid);
+		} else if (pn < lowest_pn) {
+			rxsc_stats->stats.InPktsDelayed++;
+		} else {
+			rxsc_stats->stats.InPktsUnchecked++;
+		}
+		u64_stats_update_end(&rxsc_stats->syncp);
+	} else {
+		u64_stats_update_begin(&rxsc_stats->syncp);
+		if (pn < lowest_pn) {
+			rxsc_stats->stats.InPktsDelayed++;
+		} else {
+			rxsc_stats->stats.InPktsOK++;
+			this_cpu_inc(rx_sa->stats->InPktsOK);
+		}
Quoted 8 lines

drivers/net/macsec.c line 1082

Quoted 5 lines
drivers/net/macsec.c new file +5 -0
+		u64_stats_update_end(&rxsc_stats->syncp);
+
+		if (pn >= rx_sa->next_pn)
+			rx_sa->next_pn = pn + 1;
+		spin_unlock(&rx_sa->lock);

Do you think its feasible to rearrange the above so that rx_sa->lock/unlock (next_pn test and increment) are grouped more closesly?

Not if we want to follow the order of the checks in the specification. You don't want to update the PN counter if the packet was not valid, but you need to perform replay protection before accepting the packet.

And it's a lot of code, but there isn't that much happening.

Quoted 7 lines

drivers/net/macsec.c line 1134

Quoted 5 lines
drivers/net/macsec.c new file +5 -0
+		/* not strict, the frame (with the SecTAG and ICV
+		 * removed) is delivered to the Controlled Port.
+		 */
+		nskb = skb_clone(skb, GFP_ATOMIC);
+		macsec_reset_skb(nskb, macsec->secy.netdev);

nskb == NULL handling?

Yes, as above.

Quoted 4 lines

I'll have another look at your patch set later this week.

Thanks,
Florian

Thanks.

Signature · 1 line
Sabrina
#6 lore raw

Re: [RFC PATCH net-next 3/3] macsec: introduce IEEE 802.1AE driver

Sabrina Dubroca <[email protected]> wrote:

[ Sorry for long delay ]

Quoted 10 lines
Quoted 8 lines · Florian Westphal

drivers/net/macsec.c line 483

Quoted 3 lines
drivers/net/macsec.c new file +3 -0
+	tx_sa->next_pn++;
+	if (tx_sa->next_pn == 0) {
+		pr_notice("PN wrapped, transitionning to !oper\n");

Is that _notice intentional?
I'm only asking because it seems we printk unconditionally in response to network traffic & I don't get what operator should do in response to that message.

Quoted 3 lines

The operator should install a new tx_sa, or MKA should have already installed a new one and switched to it.
I can remove this message, or make it a pr_debug.

Ok, I'll leave it up to you since I don't know what makes more sense. Basically just do whatever you think is right ;)

AFAIU this should not really happen in practice, right?

If so, pr_debug might be appropriate.

Quoted 24 lines
Quoted 11 lines
Quoted 11 lines
drivers/net/macsec.c new file +11 -0
+static void macsec_encrypt_done(struct crypto_async_request *base, int err)
+{
+	struct sk_buff *skb = base->data;
+	struct net_device *dev = skb->dev;
+	struct macsec_dev *macsec = macsec_priv(dev);
+	struct macsec_tx_sa *sa = macsec_skb_cb(skb)->tx_sa;
+	int len, ret;
+
+	aead_request_free(macsec_skb_cb(skb)->req);
+
+	rcu_read_lock_bh();
Quoted 8 lines

drivers/net/macsec.c line 550

Quoted 6 lines
drivers/net/macsec.c new file +6 -0
+	macsec_encrypt_finish(skb, dev);
+	macsec_count_tx(skb, &macsec->secy.tx_sc, macsec_skb_cb(skb)->tx_sa);
+	len = skb->len;
+	ret = dev_queue_xmit(skb);
+	count_tx(dev, ret, len);
+	rcu_read_unlock_bh();

What was the rcu_read_lock_bh protecting?

this_cpu_ptr in macsec_count_tx and count_tx. Separate get_cpu_ptr in both functions seem a bit wasteful, and dev_queue_xmit will also disable bh.

Quoted 7 lines

I could turn that into a preempt_disable with a comment (something like "covers multiple accesses to pcpu variables"). Or I could get rid of it, and use get/put_cpu_ptr in macsec_count_tx/count_tx. Note that macsec_count_tx/count_tx (and count_rx below) are also called from the normal packet processing path, where we already run under rcu_read_lock_bh anyway, so avoiding the overhead of an extra get_cpu_ptr seems preferable.

Ah, I see. In that case it seems preferrable to local_bh_dis/enable here. What do you think? (comment is still good to have wrt. pcpu and packet processing path detail, I missed the latter).

Quoted 10 lines
Quoted 9 lines

drivers/net/macsec.c line 1037

Quoted 7 lines
drivers/net/macsec.c new file +7 -0
+		spin_unlock(&rx_sa->lock);
+		pr_debug("packet_number too small: %u < %u\n", pn, lowest_pn);
+		u64_stats_update_begin(&rxsc_stats->syncp);
+		rxsc_stats->stats.InPktsLate++;
+		u64_stats_update_end(&rxsc_stats->syncp);
+		goto drop;
+	}

I don't understand why this seems to perform replay check twice?

Quoted 3 lines

This is part of the specification (802.1AE-2006 figure 10-5). The first check is done before attempting to decrypt the packet, then once again after decrypting.

I see. Could you add a short comment?
("re-check post decryption as per $ref $figure" or something like that
should suffice).

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
drivers/net/macsec.c new file +8 -0
+	if (secy->validate_frames != MACSEC_VALIDATE_DISABLED) {
+		u64_stats_update_begin(&rxsc_stats->syncp);
+		if (hdr->tci_an & MACSEC_TCI_E)
+			rxsc_stats->stats.InOctetsDecrypted += skb->len;
+		else
+			rxsc_stats->stats.InOctetsValidated += skb->len;
+		u64_stats_update_end(&rxsc_stats->syncp);
+	}

[..]

Quoted 4 lines
Quoted 2 lines

Do you think its feasible to rearrange the above so that rx_sa->lock/unlock (next_pn test and increment) are grouped more closesly?

Not if we want to follow the order of the checks in the specification.

Ok, thanks for explaining.

maintainer
#7 lore raw

Re: [RFC PATCH net-next 3/3] macsec: introduce IEEE 802.1AE driver

In reply to Sabrina Dubroca
Quoted 13 lines · Sabrina Dubroca
drivers/net/macsec.c new file +13 -0
+#define MACSEC_SCI_LEN 8
+
+/* SecTAG length = macsec_eth_header without the optional SCI */
+#define MACSEC_TAG_LEN 6
+
+struct macsec_eth_header {
+	struct ethhdr eth;
+	/* SecTAG */
+	u8  tci_an;
+#if defined(__LITTLE_ENDIAN_BITFIELD)
+	u8  short_length:6,
+		  unused:2;
+#elif defined(__BIG_ENDIAN_BITFIELD)

drivers/net/macsec.c

Quoted 10 lines
drivers/net/macsec.c new file +9 -0
+	u8        unused:2,
+	    short_length:6;
+#else
+#error	"Please fix <asm/byteorder.h>"
+#endif
+	__be32 packet_number;
+	u8 secure_channel_id[8]; /* optional */
+} __packed;
 
+#define MACSEC_NEEDED_HEADROOM sizeof(struct macsec_eth_header)

The needed_headroom field does not need to include the hard_header_len, which, for macsec devices, is set to ETH_HLEN by ether_setup().

Since on xmit path you can push up to MACSEC_TAG_LEN + MACSEC_SCI_LEN + sizeof(__be16) bytes on the skb head, possibly that should be a better MACSEC_NEEDED_HEADROOM value.

drivers/net/macsec.c line 511

Quoted 10 lines
drivers/net/macsec.c new file +10 -0
+static void macsec_count_tx(struct sk_buff *skb, struct macsec_tx_sc *tx_sc,
+			    struct macsec_tx_sa *tx_sa)
+{
+	struct pcpu_tx_sc_stats *txsc_stats = this_cpu_ptr(tx_sc->stats);
+
+	u64_stats_update_begin(&txsc_stats->syncp);
+	if (tx_sc->encrypt) {
+		txsc_stats->stats.OutOctetsEncrypted += skb->len;
+		txsc_stats->stats.OutPktsEncrypted++;
+		this_cpu_inc(tx_sa->stats->OutPktsEncrypted);

The last line is probably a duplicate

drivers/net/macsec.c line 515

Quoted 4 lines
drivers/net/macsec.c new file +4 -0
+	} else {
+		txsc_stats->stats.OutOctetsProtected += skb->len;
+		txsc_stats->stats.OutPktsProtected++;
+		this_cpu_inc(tx_sa->stats->OutPktsProtected);

Same as above.

drivers/net/macsec.c line 853

Quoted 8 lines
drivers/net/macsec.c new file +8 -0
+		struct pcpu_secy_stats *secy_stats = this_cpu_ptr(macsec->stats);
+
+		if (macsec->secy.validate_frames == MACSEC_VALIDATE_STRICT) {
+			u64_stats_update_begin(&secy_stats->syncp);
+			secy_stats->stats.InPktsNoTag++;
+			u64_stats_update_end(&secy_stats->syncp);
+			continue;
+		}

Can the 64_stats_update block be replaced by a single:
this_cpu_inc(macsec->stats->InPktsNoTag) ?

There are a few others similar blocks below.

Thanks,

Paolo

author maintainer
#8 lore raw

Re: [RFC PATCH net-next 3/3] macsec: introduce IEEE 802.1AE driver

Hi Paolo,

Quoted 23 lines · Paolo Abeni
Quoted 21 lines · Sabrina Dubroca
drivers/net/macsec.c new file +21 -0
+#define MACSEC_SCI_LEN 8
+
+/* SecTAG length = macsec_eth_header without the optional SCI */
+#define MACSEC_TAG_LEN 6
+
+struct macsec_eth_header {
+	struct ethhdr eth;
+	/* SecTAG */
+	u8  tci_an;
+#if defined(__LITTLE_ENDIAN_BITFIELD)
+	u8  short_length:6,
+		  unused:2;
+#elif defined(__BIG_ENDIAN_BITFIELD)
+	u8        unused:2,
+	    short_length:6;
+#else
+#error	"Please fix <asm/byteorder.h>"
+#endif
+	__be32 packet_number;
+	u8 secure_channel_id[8]; /* optional */
+} __packed;
Quoted 8 lines
Quoted 1 line

+#define MACSEC_NEEDED_HEADROOM sizeof(struct macsec_eth_header)

The needed_headroom field does not need to include the hard_header_len, which, for macsec devices, is set to ETH_HLEN by ether_setup().

Since on xmit path you can push up to MACSEC_TAG_LEN + MACSEC_SCI_LEN + sizeof(__be16) bytes on the skb head, possibly that should be a better MACSEC_NEEDED_HEADROOM value.

It seems you're right, I will change that.

Quoted 4 lines
Quoted 4 lines
drivers/net/macsec.c new file +4 -0
+static void macsec_count_tx(struct sk_buff *skb, struct macsec_tx_sc *tx_sc,
+			    struct macsec_tx_sa *tx_sa)
+{
+	struct pcpu_tx_sc_stats *txsc_stats = this_cpu_ptr(tx_sc->stats);
Quoted 8 lines

drivers/net/macsec.c line 511

Quoted 6 lines
drivers/net/macsec.c new file +6 -0
+
+	u64_stats_update_begin(&txsc_stats->syncp);
+	if (tx_sc->encrypt) {
+		txsc_stats->stats.OutOctetsEncrypted += skb->len;
+		txsc_stats->stats.OutPktsEncrypted++;
+		this_cpu_inc(tx_sa->stats->OutPktsEncrypted);

The last line is probably a duplicate

Actually no, we keep separate stats for the secure channel (SC) and each individual secure association (SA) within the channel.

Quoted 12 lines

drivers/net/macsec.c line 853

Quoted 8 lines
drivers/net/macsec.c new file +8 -0
+		struct pcpu_secy_stats *secy_stats = this_cpu_ptr(macsec->stats);
+
+		if (macsec->secy.validate_frames == MACSEC_VALIDATE_STRICT) {
+			u64_stats_update_begin(&secy_stats->syncp);
+			secy_stats->stats.InPktsNoTag++;
+			u64_stats_update_end(&secy_stats->syncp);
+			continue;
+		}

Can the 64_stats_update block be replaced by a single:
this_cpu_inc(macsec->stats->InPktsNoTag) ?

Quoted 1 line

There are a few others similar blocks below.

I don't think so. this_cpu_inc doesn't include the protection that u64_stats_update_begin provides.

Thanks.

Signature · 1 line
Sabrina