ipvlan: Initial check-in of the IPVLAN driver.

This driver is very similar to the macvlan driver except that it uses L3 on the frame to determine the logical interface while functioning as packet dispatcher. It inherits L2 of the master device hence the packets on wire will have the same L2 for all the packets originating from all virtual devices off of the same master device.

This driver was developed keeping the namespace use-case in mind. Hence most of the examples given here take that as the base setup where main-device belongs to the default-ns and virtual devices are assigned to the additional namespaces.

Quoted 6 lines

This driver is very similar to the macvlan driver except that it uses L3 on the frame to determine the logical interface while functioning as packet dispatcher. It inherits L2 of the master device hence the packets on wire will have the same L2 for all the packets originating from all virtual devices off of the same master device.

Why do we need this from the beginning?
IOW, what problem does this solve while macvlan doesn't?

From: Cong Wang <[email protected]>
Date: Tue, 11 Nov 2014 15:12:27 -0800

Quoted 10 lines · Mahesh Bandewar

Quoted 6 lines

This driver is very similar to the macvlan driver except that it uses L3 on the frame to determine the logical interface while functioning as packet dispatcher. It inherits L2 of the master device hence the packets on wire will have the same L2 for all the packets originating from all virtual devices off of the same master device.

Why do we need this from the beginning?
IOW, what problem does this solve while macvlan doesn't?

macvlan has several built-in limitations, which IP VLAN absolutely does not have.

Eric Dumazet spoke about this at the networking track at the kernel summit in Chicago, maybe he or another person working on this can chime in.

Quoted 6 lines · Mahesh Bandewar

This driver is very similar to the macvlan driver except that it uses L3 on the frame to determine the logical interface while functioning as packet dispatcher. It inherits L2 of the master device hence the packets on wire will have the same L2 for all the packets originating from all virtual devices off of the same master device.

Why do we need this from the beginning?
IOW, what problem does this solve while macvlan doesn't?

Quoted 8 lines · Cong Wang

Quoted 6 lines · Mahesh Bandewar

This driver is very similar to the macvlan driver except that it uses L3 on the frame to determine the logical interface while functioning as packet dispatcher. It inherits L2 of the master device hence the packets on wire will have the same L2 for all the packets originating from all virtual devices off of the same master device.

Quoted 2 lines

Why do we need this from the beginning?
IOW, what problem does this solve while macvlan doesn't?

I think it is good to reduce the number of mac addresses before a NIC switches into promisc mode.

Sounds like over-kill to have a new device just for not worrying about mac. Or you mean our neigh table doesn't scale?

Quoted 12 lines · Hannes Frederic Sowa

Quoted 10 lines · Cong Wang

Quoted 6 lines · Mahesh Bandewar

This driver is very similar to the macvlan driver except that it uses L3 on the frame to determine the logical interface while functioning as packet dispatcher. It inherits L2 of the master device hence the packets on wire will have the same L2 for all the packets originating from all virtual devices off of the same master device.

Why do we need this from the beginning?
IOW, what problem does this solve while macvlan doesn't?

Quoted 3 lines

I think it is good to reduce the number of mac addresses before a NIC switches into promisc mode.

Sounds like over-kill to have a new device just for not worrying about mac. Or you mean our neigh table doesn't scale?

drivers/net/ipvlan/ipvlan_core.c new file +29 -0

+static void *ipvlan_get_L3_hdr(struct sk_buff *skb, int *type)
+{
+	void *lyr3h = NULL;
+
+	switch (skb->protocol) {
+	case htons(ETH_P_ARP): {
+		struct arphdr *arph;
+
+		if (unlikely(!pskb_may_pull(skb, sizeof(struct arphdr))))
+			return NULL;
+
+		arph = arp_hdr(skb);
+		*type = IPVL_ARP;
+		lyr3h = arph;
+		break;
+	}
+
+	case htons(ETH_P_IP): {
+		u32 pktlen;
+		struct iphdr *ip4h;
+
+		if (unlikely(!pskb_may_pull(skb, sizeof(struct iphdr))))
+			return NULL;
+
+		ip4h = ip_hdr(skb);
+		pktlen = ntohs(ip4h->tot_len);
+		if (ip4h->ihl < 5 || ip4h->version != 4)
+			return NULL;
+		if (skb->len < pktlen || pktlen < (ip4h->ihl * 4))

drivers/net/ipvlan/ipvlan_core.c new file +23 -0

+			return NULL;
+
+		ip6h = ipv6_hdr(skb);
+		if (ip6h->version != 6)
+			return NULL;
+
+		*type = IPVL_IPV6;
+		lyr3h = ip6h;
+		/* Only Neighbour Solicitation pkts need different treatment */
+		if (ipv6_addr_any(&ip6h->saddr) &&
+		    ip6h->nexthdr == NEXTHDR_ICMP) {
+			/* Get to the ICMPv6 header */
+			*type = IPVL_ICMPV6;
+			lyr3h = ip6h + 1;
+		}
+		break;
+	}
+	default:
+		return NULL;
+	}
+
+	return lyr3h;
+}

drivers/net/ipvlan/ipvlan_core.c new file +10 -0

+static int ipvlan_process_v6_outbound(struct sk_buff *skb)
+{
+	const struct ipv6hdr *ip6h = ipv6_hdr(skb);
+	struct net_device *dev = skb->dev;
+	struct dst_entry *dst;
+	int err, ret = NET_XMIT_DROP;
+	struct flowi6 fl6 = {
+		.flowi6_iif = skb->dev->ifindex,
+		.daddr = ip6h->daddr,
+		.saddr = ip6h->saddr,

drivers/net/ipvlan/ipvlan_core.c new file +16 -0

+		goto err;
+	}
+	skb_dst_drop(skb);
+	skb_dst_set(skb, dst);
+	err = ip6_local_out(skb);
+	if (unlikely(net_xmit_eval(err)))
+		dev->stats.tx_errors++;
+	else
+		ret = NET_XMIT_SUCCESS;
+	goto out;
+err:
+	dev->stats.tx_errors++;
+	kfree_skb(skb);
+out:
+	return ret;
+}

drivers/net/ipvlan/ipvlan_core.c new file +10 -0

+static rx_handler_result_t ipvlan_handle_mode_l2(struct sk_buff **pskb,
+						 struct ipvl_port *port)
+{
+	struct sk_buff *skb = *pskb;
+	struct ethhdr *eth = eth_hdr(skb);
+	rx_handler_result_t ret = RX_HANDLER_PASS;
+	void *lyr3h;
+	int addr_type;
+
+	/* First Handle multi-cast frames */

drivers/net/ipvlan/ipvlan_core.c new file +12 -0

+
+		addr = ipvlan_addr_lookup(port, lyr3h, addr_type, true);
+		if (addr) {
+			ipvlan_dbg(4, "%s[%d]L2:Ucast Recv:[%s], PROT=[%x]\n",
+				   __func__, __LINE__, addr->master->dev->name,
+				   ntohs(skb->protocol));
+			ret = ipvlan_rcv_frame(addr, skb, false);
+		}
+	}
+
+	return ret;
+}

drivers/net/ipvlan/ipvlan_core.c new file +14 -0

+		goto out;
+
+	switch (port->mode) {
+	case IPVLAN_MODE_L2:
+		return ipvlan_handle_mode_l2(pskb, port);
+	case IPVLAN_MODE_L3:
+		return ipvlan_handle_mode_l3(pskb, port);
+	}
+
+	/* Should not reach here */
+	BUG();
+out:
+	return RX_HANDLER_PASS;
+}

drivers/net/ipvlan/ipvlan_main.c new file +10 -0

new file mode 100644
index 000000000000..e87b6eb01060
@@ -0,0 +1,828 @@
+/* Copyright (c) 2014 Mahesh Bandewar <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ *
+ */
+
+

drivers/net/ipvlan/ipvlan_main.c new file +69 -0

+		return -ENOMEM;
+
+	ipvlan_dbg(1, "%s[%d]: Adding IPv6=%x:%x:%x:%x address for %s intf\n",
+		   __func__, __LINE__, ip6_addr->s6_addr32[0],
+		   ip6_addr->s6_addr32[1], ip6_addr->s6_addr32[2],
+		   ip6_addr->s6_addr32[3], ipvlan->dev->name);
+	addr->master = ipvlan;
+	memcpy(&addr->ip6addr, ip6_addr, sizeof(struct in6_addr));
+	addr->atype = IPVL_IPV6;
+	list_add_tail_rcu(&addr->anode, &ipvlan->addrs);
+	ipvlan->ipv6cnt++;
+	ipvlan_ht_addr_add(ipvlan, addr);
+
+	return 0;
+}
+
+static void ipvlan_del_addr6(struct ipvl_dev *ipvlan, struct in6_addr *ip6_addr)
+{
+	struct ipvl_addr *addr = NULL;
+
+	if ((addr = ipvlan_ht_addr_lookup(ipvlan->port, ip6_addr, true)) ==NULL)
+		return;
+
+	ipvlan_dbg(1,
+		   "%s[%d]: Deleting IPv6=%x:%x:%x:%x address for %s intf.\n",
+		   __func__, __LINE__, ip6_addr->s6_addr32[0],
+		   ip6_addr->s6_addr32[1], ip6_addr->s6_addr32[2],
+		   ip6_addr->s6_addr32[3], ipvlan->dev->name);
+	/* Delete from the hash-table */
+	ipvlan_ht_addr_del(addr, true);
+	/* Delete from the logical's addr list */
+	list_del_rcu(&addr->anode);
+	ipvlan->ipv6cnt--;
+	WARN_ON(ipvlan->ipv6cnt < 0);
+	kfree_rcu(addr, rcu);
+
+	return;
+}
+
+static int ipvlan_addr6_event(struct notifier_block *unused,
+			      unsigned long event, void *ptr)
+{
+	struct inet6_ifaddr *if6 = (struct inet6_ifaddr *)ptr;
+	struct net_device *dev = (struct net_device *)if6->idev->dev;
+	struct ipvl_dev *ipvlan = netdev_priv(dev);
+
+	ipvlan_dbg(3, "%s[%d]: Entering...\n", __func__, __LINE__);
+	if (!ipvlan_dev_slave(dev))
+		return NOTIFY_DONE;
+
+	if (!ipvlan || !ipvlan->port)
+		return NOTIFY_DONE;
+
+	switch (event) {
+	case NETDEV_UP:
+		if (ipvlan_add_addr6(ipvlan, &if6->addr))
+			return NOTIFY_BAD;
+		break;
+
+	case NETDEV_DOWN:
+		ipvlan_del_addr6(ipvlan, &if6->addr);
+		break;
+	}
+
+	ipvlan_dbg(3, "%s[%d]: Leaving...\n", __func__, __LINE__);
+	return NOTIFY_OK;
+}
+
+static int ipvlan_add_addr4(struct ipvl_dev *ipvlan, struct in_addr *ip4_addr)

On Tue, 2014-11-11 at 14:29 -0800, Mahesh Bandewar wrote:

...

Quoted 29 lines

drivers/net/ipvlan/ipvlan_core.c new file +29 -0

+static void *ipvlan_get_L3_hdr(struct sk_buff *skb, int *type)
+{
+     void *lyr3h = NULL;
+
+     switch (skb->protocol) {
+     case htons(ETH_P_ARP): {
+             struct arphdr *arph;
+
+             if (unlikely(!pskb_may_pull(skb, sizeof(struct arphdr))))
+                     return NULL;
+
+             arph = arp_hdr(skb);
+             *type = IPVL_ARP;
+             lyr3h = arph;
+             break;
+     }
+
+     case htons(ETH_P_IP): {
+             u32 pktlen;
+             struct iphdr *ip4h;
+
+             if (unlikely(!pskb_may_pull(skb, sizeof(struct iphdr))))
+                     return NULL;
+
+             ip4h = ip_hdr(skb);
+             pktlen = ntohs(ip4h->tot_len);
+             if (ip4h->ihl < 5 || ip4h->version != 4)
+                     return NULL;
+             if (skb->len < pktlen || pktlen < (ip4h->ihl * 4))

drivers/net/ipvlan/ipvlan_core.c line 149

Quoted 10 lines

drivers/net/ipvlan/ipvlan_core.c new file +10 -0

+                     return NULL;
+
+             *type = IPVL_IPV4;
+             lyr3h = ip4h;
+             break;
+     }
+     case htons(ETH_P_IPV6): {
+             struct ipv6hdr *ip6h;
+
+             if (unlikely(!pskb_may_pull(skb, sizeof(struct iphdr))))

sizeof(struct ipv6hdr) or sizeof(*ip6h)

Quoted 23 lines

drivers/net/ipvlan/ipvlan_core.c new file +23 -0

+                     return NULL;
+
+             ip6h = ipv6_hdr(skb);
+             if (ip6h->version != 6)
+                     return NULL;
+
+             *type = IPVL_IPV6;
+             lyr3h = ip6h;
+             /* Only Neighbour Solicitation pkts need different treatment */
+             if (ipv6_addr_any(&ip6h->saddr) &&
+                 ip6h->nexthdr == NEXTHDR_ICMP) {
+                     /* Get to the ICMPv6 header */
+                     *type = IPVL_ICMPV6;
+                     lyr3h = ip6h + 1;
+             }
+             break;
+     }
+     default:
+             return NULL;
+     }
+
+     return lyr3h;
+}

...

Quoted 10 lines

drivers/net/ipvlan/ipvlan_core.c new file +10 -0

+static int ipvlan_process_v6_outbound(struct sk_buff *skb)
+{
+     const struct ipv6hdr *ip6h = ipv6_hdr(skb);
+     struct net_device *dev = skb->dev;
+     struct dst_entry *dst;
+     int err, ret = NET_XMIT_DROP;
+     struct flowi6 fl6 = {
+             .flowi6_iif = skb->dev->ifindex,
+             .daddr = ip6h->daddr,
+             .saddr = ip6h->saddr,

drivers/net/ipvlan/ipvlan_core.c line 385

Quoted 10 lines

drivers/net/ipvlan/ipvlan_core.c new file +10 -0

+             .flowi6_flags = FLOWI_FLAG_ANYSRC,
+             .flowlabel = ip6_flowinfo(ip6h),
+             .flowi6_mark = skb->mark,
+             .flowi6_proto = ip6h->nexthdr,
+     };
+
+     dst = ip6_route_output(dev_net(dev), NULL, &fl6);
+     if (IS_ERR(dst)) {
+             err = PTR_ERR(dst);
+             dst = NULL;

dst = NULL; seems not needed.

Quoted 16 lines

drivers/net/ipvlan/ipvlan_core.c new file +16 -0

+             goto err;
+     }
+     skb_dst_drop(skb);
+     skb_dst_set(skb, dst);
+     err = ip6_local_out(skb);
+     if (unlikely(net_xmit_eval(err)))
+             dev->stats.tx_errors++;
+     else
+             ret = NET_XMIT_SUCCESS;
+     goto out;
+err:
+     dev->stats.tx_errors++;
+     kfree_skb(skb);
+out:
+     return ret;
+}

...

Quoted 10 lines

drivers/net/ipvlan/ipvlan_core.c new file +10 -0

+static rx_handler_result_t ipvlan_handle_mode_l2(struct sk_buff **pskb,
+                                              struct ipvl_port *port)
+{
+     struct sk_buff *skb = *pskb;
+     struct ethhdr *eth = eth_hdr(skb);
+     rx_handler_result_t ret = RX_HANDLER_PASS;
+     void *lyr3h;
+     int addr_type;
+
+     /* First Handle multi-cast frames */

drivers/net/ipvlan/ipvlan_core.c line 598

Quoted 10 lines

drivers/net/ipvlan/ipvlan_core.c new file +10 -0

+     if (is_multicast_ether_addr(eth->h_dest)) {
+             /* Pass to virtual devs only if they haven't seen the frame. */
+             if (ipvlan_external_frame(skb, port)) {
+                     ipvlan_dbg(4, "%s[%d]L2:Mcast Recv:[%s], PROT=[%x]\n",
+                                __func__, __LINE__, port->dev->name,
+                                ntohs(skb->protocol));
+                     ipvlan_multicast_frame(port, skb, NULL, false);
+             }
+     } else if ((lyr3h = ipvlan_get_L3_hdr(skb, &addr_type)) != NULL) {
+             struct ipvl_addr *addr = NULL;

= NULL; not needed.

Quoted 12 lines

drivers/net/ipvlan/ipvlan_core.c new file +12 -0

+
+             addr = ipvlan_addr_lookup(port, lyr3h, addr_type, true);
+             if (addr) {
+                     ipvlan_dbg(4, "%s[%d]L2:Ucast Recv:[%s], PROT=[%x]\n",
+                                __func__, __LINE__, addr->master->dev->name,
+                                ntohs(skb->protocol));
+                     ret = ipvlan_rcv_frame(addr, skb, false);
+             }
+     }
+
+     return ret;
+}

drivers/net/ipvlan/ipvlan_core.c line 620

Quoted 10 lines

drivers/net/ipvlan/ipvlan_core.c new file +10 -0

+
+rx_handler_result_t ipvlan_handle_frame(struct sk_buff **pskb)
+{
+     struct sk_buff *skb = *pskb;
+     struct ipvl_port *port = ipvlan_port_get_rcu(skb->dev);
+
+     if (!port)
+             goto out;
+
+     if (unlikely(!pskb_may_pull(skb, sizeof(struct ethhdr))))

This looks strange.

Here we are sure ethernet header was already pulled by eth_type_trans()

Quoted 30 lines

drivers/net/ipvlan/ipvlan_core.c new file +14 -0

+             goto out;
+
+     switch (port->mode) {
+     case IPVLAN_MODE_L2:
+             return ipvlan_handle_mode_l2(pskb, port);
+     case IPVLAN_MODE_L3:
+             return ipvlan_handle_mode_l3(pskb, port);
+     }
+
+     /* Should not reach here */
+     BUG();
+out:
+     return RX_HANDLER_PASS;
+}

drivers/net/ipvlan/ipvlan_main.c new file +10 -0

new file mode 100644
index 000000000000..e87b6eb01060
@@ -0,0 +1,828 @@
+/* Copyright (c) 2014 Mahesh Bandewar <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ *
+ */
+
+

...

drivers/net/ipvlan/ipvlan_main.c line 632

Quoted 12 lines

drivers/net/ipvlan/ipvlan_main.c new file +12 -0

+static int ipvlan_add_addr6(struct ipvl_dev *ipvlan, struct in6_addr *ip6_addr)
+{
+     struct ipvl_addr *addr = NULL;
+
+     if (ipvlan_addr_busy(ipvlan, ip6_addr, true)) {
+             pr_warn("%s[%d]: Failed IPv6=%x:%x:%x:%x address for %s intf\n",
+                     __func__, __LINE__, ip6_addr->s6_addr32[0],
+                     ip6_addr->s6_addr32[1], ip6_addr->s6_addr32[2],
+                     ip6_addr->s6_addr32[3], ipvlan->dev->name);
+             return -EINVAL;
+     }
+     if ((addr = kzalloc(sizeof(struct ipvl_addr), GFP_ATOMIC)) == NULL)

Why is GFP_ATOMIC used here ?

Quoted 69 lines

drivers/net/ipvlan/ipvlan_main.c new file +69 -0

+             return -ENOMEM;
+
+     ipvlan_dbg(1, "%s[%d]: Adding IPv6=%x:%x:%x:%x address for %s intf\n",
+                __func__, __LINE__, ip6_addr->s6_addr32[0],
+                ip6_addr->s6_addr32[1], ip6_addr->s6_addr32[2],
+                ip6_addr->s6_addr32[3], ipvlan->dev->name);
+     addr->master = ipvlan;
+     memcpy(&addr->ip6addr, ip6_addr, sizeof(struct in6_addr));
+     addr->atype = IPVL_IPV6;
+     list_add_tail_rcu(&addr->anode, &ipvlan->addrs);
+     ipvlan->ipv6cnt++;
+     ipvlan_ht_addr_add(ipvlan, addr);
+
+     return 0;
+}
+
+static void ipvlan_del_addr6(struct ipvl_dev *ipvlan, struct in6_addr *ip6_addr)
+{
+     struct ipvl_addr *addr = NULL;
+
+     if ((addr = ipvlan_ht_addr_lookup(ipvlan->port, ip6_addr, true)) ==NULL)
+             return;
+
+     ipvlan_dbg(1,
+                "%s[%d]: Deleting IPv6=%x:%x:%x:%x address for %s intf.\n",
+                __func__, __LINE__, ip6_addr->s6_addr32[0],
+                ip6_addr->s6_addr32[1], ip6_addr->s6_addr32[2],
+                ip6_addr->s6_addr32[3], ipvlan->dev->name);
+     /* Delete from the hash-table */
+     ipvlan_ht_addr_del(addr, true);
+     /* Delete from the logical's addr list */
+     list_del_rcu(&addr->anode);
+     ipvlan->ipv6cnt--;
+     WARN_ON(ipvlan->ipv6cnt < 0);
+     kfree_rcu(addr, rcu);
+
+     return;
+}
+
+static int ipvlan_addr6_event(struct notifier_block *unused,
+                           unsigned long event, void *ptr)
+{
+     struct inet6_ifaddr *if6 = (struct inet6_ifaddr *)ptr;
+     struct net_device *dev = (struct net_device *)if6->idev->dev;
+     struct ipvl_dev *ipvlan = netdev_priv(dev);
+
+     ipvlan_dbg(3, "%s[%d]: Entering...\n", __func__, __LINE__);
+     if (!ipvlan_dev_slave(dev))
+             return NOTIFY_DONE;
+
+     if (!ipvlan || !ipvlan->port)
+             return NOTIFY_DONE;
+
+     switch (event) {
+     case NETDEV_UP:
+             if (ipvlan_add_addr6(ipvlan, &if6->addr))
+                     return NOTIFY_BAD;
+             break;
+
+     case NETDEV_DOWN:
+             ipvlan_del_addr6(ipvlan, &if6->addr);
+             break;
+     }
+
+     ipvlan_dbg(3, "%s[%d]: Leaving...\n", __func__, __LINE__);
+     return NOTIFY_OK;
+}
+
+static int ipvlan_add_addr4(struct ipvl_dev *ipvlan, struct in_addr *ip4_addr)

drivers/net/ipvlan/ipvlan_main.c line 711

Quoted 10 lines

drivers/net/ipvlan/ipvlan_main.c new file +10 -0

+{
+     struct ipvl_addr *addr = NULL;
+
+     if (ipvlan_addr_busy(ipvlan, ip4_addr, false)) {
+             pr_warn("%s[%d]: Failed to add IPv4=%x on %s intf.\n",
+                     __func__, __LINE__, ntohl(ip4_addr->s_addr),
+                        ipvlan->dev->name);
+             return -EINVAL;
+     }
+     if ((addr = kzalloc(sizeof(struct ipvl_addr), GFP_ATOMIC)) == NULL)

Same issue here ? GFP_KERNEL should be OK.

This driver is very similar to the macvlan driver except that it uses L3 on the frame to determine the logical interface while functioning as packet dispatcher. It inherits L2 of the master device hence the packets on wire will have the same L2 for all the packets originating from all virtual devices off of the same master device.

This driver was developed keeping the namespace use-case in mind. Hence most of the examples given here take that as the base setup where main-device belongs to the default-ns and virtual devices are assigned to the additional namespaces.

The device operates in two different modes and the difference in these two modes in primarily in the TX side.

(a) L2 mode : In this mode, the device behaves as a L2 device. TX processing upto L2 happens on the stack of the virtual device associated with (namespace). Packets are switched after that into the main device (default-ns) and queued for xmit.

RX processing is simple and all multicast, broadcast (if applicable), and unicast belonging to the address(es) are delivered to the virtual devices.

(b) L3 mode : In this mode, the device behaves like a L3 device. TX processing upto L3 happens on the stack of the virtual device associated with (namespace). Packets are switched to the main-device (default-ns) for the L2 processing. Hence the routing table of the default-ns will be used in this mode.

RX processins is somewhat similar to the L2 mode except that in this mode only Unicast packets are delivered to the virtual device while main-dev will handle all other packets.

The devices can be added using the "ip" command from the iproute2 package -

ip link add link <master> <virtual> type ipvlan mode [ l2 | l3 ]

Signed-off-by Mahesh Bandewar <[email protected]>

Cc Eric Dumazet <[email protected]>

Cc Maciej Żenczykowski <[email protected]>

Cc Laurent Chavey <[email protected]>

Cc Tim Hockin <[email protected]>

Cc Brandon Philips <[email protected]>

Cc Pavel Emelianov <[email protected]>

drivers/net/ipvlan/ipvlan_core.c new file +29 -0

+static void *ipvlan_get_L3_hdr(struct sk_buff *skb, int *type)
+{
+	void *lyr3h = NULL;
+
+	switch (skb->protocol) {
+	case htons(ETH_P_ARP): {
+		struct arphdr *arph;
+
+		if (unlikely(!pskb_may_pull(skb, sizeof(struct arphdr))))
+			return NULL;
+
+		arph = arp_hdr(skb);
+		*type = IPVL_ARP;
+		lyr3h = arph;
+		break;
+	}
+
+	case htons(ETH_P_IP): {
+		u32 pktlen;
+		struct iphdr *ip4h;
+
+		if (unlikely(!pskb_may_pull(skb, sizeof(struct iphdr))))
+			return NULL;
+
+		ip4h = ip_hdr(skb);
+		pktlen = ntohs(ip4h->tot_len);
+		if (ip4h->ihl < 5 || ip4h->version != 4)
+			return NULL;
+		if (skb->len < pktlen || pktlen < (ip4h->ihl * 4))

drivers/net/ipvlan/ipvlan_main.c new file +26 -0

+static int ipvlan_link_new(struct net *src_net, struct net_device *dev,
+			   struct nlattr *tb[], struct nlattr *data[])
+{
+	struct ipvl_dev *ipvlan = netdev_priv(dev);
+	struct ipvl_port *port;
+	struct net_device *phy_dev;
+	int err;
+
+	ipvlan_dbg(3, "%s[%d]: Entering...\n", __func__, __LINE__);
+	if (!tb[IFLA_LINK]) {
+		ipvlan_dbg(3, "%s[%d]: Returning -EINVAL...\n",
+			   __func__, __LINE__);
+		return -EINVAL;
+	}
+
+	phy_dev = __dev_get_by_index(src_net, nla_get_u32(tb[IFLA_LINK]));
+	if (phy_dev == NULL) {
+		ipvlan_dbg(3, "%s[%d]: Returning -ENODEV...\n",
+			   __func__, __LINE__);
+		return -ENODEV;
+	}
+
+	/* TODO will someone try creating ipvlan-dev on an ipvlan-virtual dev?*/
+	if (!ipvlan_dev_master(phy_dev)) {
+		err = ipvlan_port_create(phy_dev);
+		if (err < 0) {

drivers/net/ipvlan/ipvlan_main.c new file +28 -0

+
+	ipvlan->phy_dev = phy_dev;
+	ipvlan->dev = dev;
+	ipvlan->port = port;
+	ipvlan->sfeatures = IPVLAN_FEATURES;
+	INIT_LIST_HEAD(&ipvlan->addrs);
+	ipvlan->ipv4cnt = 0;
+	ipvlan->ipv6cnt = 0;
 
 
+static int ipvlan_device_event(struct notifier_block *unused,
+			       unsigned long event, void *ptr)
+{
+	struct net_device *dev = netdev_notifier_info_to_dev(ptr);
+	struct ipvl_dev *ipvlan, *next;
+	struct ipvl_port *port;
+	LIST_HEAD(lst_kill);
+
+	if (!ipvlan_dev_master(dev))
+		return NOTIFY_DONE;
+
+	port = ipvlan_port_get_rtnl(dev);
+
+	switch (event) {
+	case NETDEV_CHANGE:
+		list_for_each_entry(ipvlan, &port->ipvlans, pnode)
+			netif_stacked_transfer_operstate(ipvlan->phy_dev,
+							 ipvlan->dev);
+		break;
+

drivers/net/ipvlan/ipvlan_main.c new file +9 -0

+		break;
+
 
+static int ipvlan_addr4_event(struct notifier_block *unused,
+			      unsigned long event, void *ptr)
+{
+	struct in_ifaddr *if4 = (struct in_ifaddr *)ptr;
+	struct net_device *dev = (struct net_device *)if4->ifa_dev->dev;
+	struct ipvl_dev *ipvlan = netdev_priv(dev);
+	struct in_addr ip4_addr;

drivers/net/ipvlan/ipvlan_main.c new file +14 -0

+		ip4_addr.s_addr = if4->ifa_address;
+		if (ipvlan_add_addr4(ipvlan, &ip4_addr))
+			return NOTIFY_BAD;
+		break;
+
+	case NETDEV_DOWN:
+		ip4_addr.s_addr = if4->ifa_address;
+		ipvlan_del_addr4(ipvlan, &ip4_addr);
+		break;
+	}
+
+	ipvlan_dbg(3, "%s[%d]: Leaving...\n", __func__, __LINE__);
+	return NOTIFY_OK;
+}

Quoted 39 lines · Mahesh Bandewar

This driver is very similar to the macvlan driver except that it uses L3 on the frame to determine the logical interface while functioning as packet dispatcher. It inherits L2 of the master device hence the packets on wire will have the same L2 for all the packets originating from all virtual devices off of the same master device.

This driver was developed keeping the namespace use-case in mind. Hence most of the examples given here take that as the base setup where main-device belongs to the default-ns and virtual devices are assigned to the additional namespaces.

The device operates in two different modes and the difference in these two modes in primarily in the TX side.

(a) L2 mode : In this mode, the device behaves as a L2 device. TX processing upto L2 happens on the stack of the virtual device associated with (namespace). Packets are switched after that into the main device (default-ns) and queued for xmit.

RX processing is simple and all multicast, broadcast (if applicable), and unicast belonging to the address(es) are delivered to the virtual devices.

(b) L3 mode : In this mode, the device behaves like a L3 device. TX processing upto L3 happens on the stack of the virtual device associated with (namespace). Packets are switched to the main-device (default-ns) for the L2 processing. Hence the routing table of the default-ns will be used in this mode.

RX processins is somewhat similar to the L2 mode except that in this mode only Unicast packets are delivered to the virtual device while main-dev will handle all other packets.

The devices can be added using the "ip" command from the iproute2 package -

ip link add link <master> <virtual> type ipvlan mode [ l2 | l3 ]

Quoted 7 lines

Signed-off-by Mahesh Bandewar <[email protected]>

Cc Eric Dumazet <[email protected]>

Cc Maciej Żenczykowski <[email protected]>

Cc Laurent Chavey <[email protected]>

Cc Tim Hockin <[email protected]>

Cc Brandon Philips <[email protected]>

Cc Pavel Emelianov <[email protected]>

Acked-by: /me on the general idea. We use this device of type in Parallels heavily for several reasons -- not to generate too many MAC-s from one host and to "enforce" the IP address for a container. I have a comment about the latter below.

Quoted 33 lines

drivers/net/ipvlan/ipvlan_core.c new file +33 -0

+static void *ipvlan_get_L3_hdr(struct sk_buff *skb, int *type)
+{
+     void *lyr3h = NULL;
+
+     switch (skb->protocol) {
+     case htons(ETH_P_ARP): {
+             struct arphdr *arph;
+
+             if (unlikely(!pskb_may_pull(skb, sizeof(struct arphdr))))
+                     return NULL;
+
+             arph = arp_hdr(skb);
+             *type = IPVL_ARP;
+             lyr3h = arph;
+             break;
+     }
+
+     case htons(ETH_P_IP): {
+             u32 pktlen;
+             struct iphdr *ip4h;
+
+             if (unlikely(!pskb_may_pull(skb, sizeof(struct iphdr))))
+                     return NULL;
+
+             ip4h = ip_hdr(skb);
+             pktlen = ntohs(ip4h->tot_len);
+             if (ip4h->ihl < 5 || ip4h->version != 4)
+                     return NULL;
+             if (skb->len < pktlen || pktlen < (ip4h->ihl * 4))
+                     return NULL;
+
+             *type = IPVL_IPV4;
+             lyr3h = ip4h;

drivers/net/ipvlan/ipvlan_core.c line 149

Quoted 6 lines

drivers/net/ipvlan/ipvlan_core.c new file +6 -0

+             break;
+     }
+     case htons(ETH_P_IPV6): {
+             struct ipv6hdr *ip6h;
+
+             if (unlikely(!pskb_may_pull(skb, sizeof(struct iphdr))))

Misprint -- should be sizeof(struct ipv6hdr)

Quoted 31 lines

drivers/net/ipvlan/ipvlan_main.c new file +31 -0

+static int ipvlan_link_new(struct net *src_net, struct net_device *dev,
+                        struct nlattr *tb[], struct nlattr *data[])
+{
+     struct ipvl_dev *ipvlan = netdev_priv(dev);
+     struct ipvl_port *port;
+     struct net_device *phy_dev;
+     int err;
+
+     ipvlan_dbg(3, "%s[%d]: Entering...\n", __func__, __LINE__);
+     if (!tb[IFLA_LINK]) {
+             ipvlan_dbg(3, "%s[%d]: Returning -EINVAL...\n",
+                        __func__, __LINE__);
+             return -EINVAL;
+     }
+
+     phy_dev = __dev_get_by_index(src_net, nla_get_u32(tb[IFLA_LINK]));
+     if (phy_dev == NULL) {
+             ipvlan_dbg(3, "%s[%d]: Returning -ENODEV...\n",
+                        __func__, __LINE__);
+             return -ENODEV;
+     }
+
+     /* TODO will someone try creating ipvlan-dev on an ipvlan-virtual dev?*/
+     if (!ipvlan_dev_master(phy_dev)) {
+             err = ipvlan_port_create(phy_dev);
+             if (err < 0) {
+                     ipvlan_dbg(3, "%s[%d]: Returning error (%d)...\n",
+                                __func__, __LINE__, err);
+                     return err;
+             }
+     }

drivers/net/ipvlan/ipvlan_main.c line 464

Quoted 5 lines

drivers/net/ipvlan/ipvlan_main.c new file +5 -0

+
+     port = ipvlan_port_get_rtnl(phy_dev);
+     /* Get the mode if specified. */
+     if (data && data[IFLA_IPVLAN_MODE])
+             port->mode = nla_get_u16(data[IFLA_IPVLAN_MODE]);

Should the invalid value be checked here? There are places where we BUG() in mode being "unknown".

Quoted 34 lines

drivers/net/ipvlan/ipvlan_main.c new file +32 -0

+
+     ipvlan->phy_dev = phy_dev;
+     ipvlan->dev = dev;
+     ipvlan->port = port;
+     ipvlan->sfeatures = IPVLAN_FEATURES;
+     INIT_LIST_HEAD(&ipvlan->addrs);
+     ipvlan->ipv4cnt = 0;
+     ipvlan->ipv6cnt = 0;
 
 
+static int ipvlan_device_event(struct notifier_block *unused,
+                            unsigned long event, void *ptr)
+{
+     struct net_device *dev = netdev_notifier_info_to_dev(ptr);
+     struct ipvl_dev *ipvlan, *next;
+     struct ipvl_port *port;
+     LIST_HEAD(lst_kill);
+
+     if (!ipvlan_dev_master(dev))
+             return NOTIFY_DONE;
+
+     port = ipvlan_port_get_rtnl(dev);
+
+     switch (event) {
+     case NETDEV_CHANGE:
+             list_for_each_entry(ipvlan, &port->ipvlans, pnode)
+                     netif_stacked_transfer_operstate(ipvlan->phy_dev,
+                                                      ipvlan->dev);
+             break;
+
+     case NETDEV_UNREGISTER:
+             if (dev->reg_state != NETREG_UNREGISTERING)
+                     break;
+

drivers/net/ipvlan/ipvlan_main.c line 597

Quoted 6 lines

drivers/net/ipvlan/ipvlan_main.c new file +6 -0

+             list_for_each_entry_safe(ipvlan, next, &port->ipvlans,
+                                      pnode)
+                     ipvlan->dev->rtnl_link_ops->dellink(ipvlan->dev,
+                                                         &lst_kill);
+             unregister_netdevice_many(&lst_kill);
+             list_del(&lst_kill);

This list_del seems to be excessive.

Quoted 17 lines

drivers/net/ipvlan/ipvlan_main.c new file +16 -0

+             break;
+
 
+static int ipvlan_addr4_event(struct notifier_block *unused,
+                           unsigned long event, void *ptr)
+{
+     struct in_ifaddr *if4 = (struct in_ifaddr *)ptr;
+     struct net_device *dev = (struct net_device *)if4->ifa_dev->dev;
+     struct ipvl_dev *ipvlan = netdev_priv(dev);
+     struct in_addr ip4_addr;
+
+     ipvlan_dbg(3, "%s[%d]: Entering...\n", __func__, __LINE__);
+     if (!ipvlan_dev_slave(dev))
+             return NOTIFY_DONE;
+
+     if (!ipvlan || !ipvlan->port)
+             return NOTIFY_DONE;

drivers/net/ipvlan/ipvlan_main.c line 687

Quoted 3 lines

drivers/net/ipvlan/ipvlan_main.c new file +3 -0

+
+     switch (event) {
+     case NETDEV_UP:

Can it be (in the future) somehow restricted so that net-namespace wouldn't be able to assign arbitrary IP address here? One of the reasons for using such devices is to enforce the container to use the IP address given from the host.

Quoted 14 lines

drivers/net/ipvlan/ipvlan_main.c new file +14 -0

+             ip4_addr.s_addr = if4->ifa_address;
+             if (ipvlan_add_addr4(ipvlan, &ip4_addr))
+                     return NOTIFY_BAD;
+             break;
+
+     case NETDEV_DOWN:
+             ip4_addr.s_addr = if4->ifa_address;
+             ipvlan_del_addr4(ipvlan, &ip4_addr);
+             break;
+     }
+
+     ipvlan_dbg(3, "%s[%d]: Leaving...\n", __func__, __LINE__);
+     return NOTIFY_OK;
+}

Quoted 31 lines

Quoted 31 lines

drivers/net/ipvlan/ipvlan_main.c new file +31 -0

+static int ipvlan_link_new(struct net *src_net, struct net_device *dev,
+                        struct nlattr *tb[], struct nlattr *data[])
+{
+     struct ipvl_dev *ipvlan = netdev_priv(dev);
+     struct ipvl_port *port;
+     struct net_device *phy_dev;
+     int err;
+
+     ipvlan_dbg(3, "%s[%d]: Entering...\n", __func__, __LINE__);
+     if (!tb[IFLA_LINK]) {
+             ipvlan_dbg(3, "%s[%d]: Returning -EINVAL...\n",
+                        __func__, __LINE__);
+             return -EINVAL;
+     }
+
+     phy_dev = __dev_get_by_index(src_net, nla_get_u32(tb[IFLA_LINK]));
+     if (phy_dev == NULL) {
+             ipvlan_dbg(3, "%s[%d]: Returning -ENODEV...\n",
+                        __func__, __LINE__);
+             return -ENODEV;
+     }
+
+     /* TODO will someone try creating ipvlan-dev on an ipvlan-virtual dev?*/
+     if (!ipvlan_dev_master(phy_dev)) {
+             err = ipvlan_port_create(phy_dev);
+             if (err < 0) {
+                     ipvlan_dbg(3, "%s[%d]: Returning error (%d)...\n",
+                                __func__, __LINE__, err);
+                     return err;
+             }
+     }

Quoted 9 lines

drivers/net/ipvlan/ipvlan_main.c line 464

Quoted 5 lines

drivers/net/ipvlan/ipvlan_main.c new file +5 -0

+
+     port = ipvlan_port_get_rtnl(phy_dev);
+     /* Get the mode if specified. */
+     if (data && data[IFLA_IPVLAN_MODE])
+             port->mode = nla_get_u16(data[IFLA_IPVLAN_MODE]);

Should the invalid value be checked here? There are places where we BUG() in mode being "unknown".

Assuming the calls come over netlink, the ".validate" will be called before ".newlink", so that would be unnecessary, isn't it?

Quoted 17 lines

Quoted 17 lines

drivers/net/ipvlan/ipvlan_main.c new file +16 -0

+             break;
+
 
+static int ipvlan_addr4_event(struct notifier_block *unused,
+                           unsigned long event, void *ptr)
+{
+     struct in_ifaddr *if4 = (struct in_ifaddr *)ptr;
+     struct net_device *dev = (struct net_device *)if4->ifa_dev->dev;
+     struct ipvl_dev *ipvlan = netdev_priv(dev);
+     struct in_addr ip4_addr;
+
+     ipvlan_dbg(3, "%s[%d]: Entering...\n", __func__, __LINE__);
+     if (!ipvlan_dev_slave(dev))
+             return NOTIFY_DONE;
+
+     if (!ipvlan || !ipvlan->port)
+             return NOTIFY_DONE;

Quoted 9 lines

drivers/net/ipvlan/ipvlan_main.c line 687

Quoted 3 lines

drivers/net/ipvlan/ipvlan_main.c new file +3 -0

+
+     switch (event) {
+     case NETDEV_UP:

Can it be (in the future) somehow restricted so that net-namespace wouldn't be able to assign arbitrary IP address here? One of the reasons for using such devices is to enforce the container to use the IP address given from the host.

Probably this could be a config (sysfs?) entry which would lockup the config coming from ns when set. So code could look like -

          case NETDEV_UP:
                         if (!restrict_ns_config) {
                            ...
                         }
                         break;

Quoted 40 lines

Quoted 31 lines

Quoted 31 lines

drivers/net/ipvlan/ipvlan_main.c new file +31 -0

+static int ipvlan_link_new(struct net *src_net, struct net_device *dev,
+                        struct nlattr *tb[], struct nlattr *data[])
+{
+     struct ipvl_dev *ipvlan = netdev_priv(dev);
+     struct ipvl_port *port;
+     struct net_device *phy_dev;
+     int err;
+
+     ipvlan_dbg(3, "%s[%d]: Entering...\n", __func__, __LINE__);
+     if (!tb[IFLA_LINK]) {
+             ipvlan_dbg(3, "%s[%d]: Returning -EINVAL...\n",
+                        __func__, __LINE__);
+             return -EINVAL;
+     }
+
+     phy_dev = __dev_get_by_index(src_net, nla_get_u32(tb[IFLA_LINK]));
+     if (phy_dev == NULL) {
+             ipvlan_dbg(3, "%s[%d]: Returning -ENODEV...\n",
+                        __func__, __LINE__);
+             return -ENODEV;
+     }
+
+     /* TODO will someone try creating ipvlan-dev on an ipvlan-virtual dev?*/
+     if (!ipvlan_dev_master(phy_dev)) {
+             err = ipvlan_port_create(phy_dev);
+             if (err < 0) {
+                     ipvlan_dbg(3, "%s[%d]: Returning error (%d)...\n",
+                                __func__, __LINE__, err);
+                     return err;
+             }
+     }

Quoted 9 lines

drivers/net/ipvlan/ipvlan_main.c line 464

Quoted 5 lines

drivers/net/ipvlan/ipvlan_main.c new file +5 -0

+
+     port = ipvlan_port_get_rtnl(phy_dev);
+     /* Get the mode if specified. */
+     if (data && data[IFLA_IPVLAN_MODE])
+             port->mode = nla_get_u16(data[IFLA_IPVLAN_MODE]);

Should the invalid value be checked here? There are places where we BUG() in mode being "unknown".

Quoted 2 lines

Assuming the calls come over netlink, the ".validate" will be called before ".newlink", so that would be unnecessary, isn't it?

Yes, you're right. I've missed the validate callback.

Quoted 26 lines

Quoted 17 lines

Quoted 17 lines

drivers/net/ipvlan/ipvlan_main.c new file +16 -0

+             break;
+
 
+static int ipvlan_addr4_event(struct notifier_block *unused,
+                           unsigned long event, void *ptr)
+{
+     struct in_ifaddr *if4 = (struct in_ifaddr *)ptr;
+     struct net_device *dev = (struct net_device *)if4->ifa_dev->dev;
+     struct ipvl_dev *ipvlan = netdev_priv(dev);
+     struct in_addr ip4_addr;
+
+     ipvlan_dbg(3, "%s[%d]: Entering...\n", __func__, __LINE__);
+     if (!ipvlan_dev_slave(dev))
+             return NOTIFY_DONE;
+
+     if (!ipvlan || !ipvlan->port)
+             return NOTIFY_DONE;

Quoted 9 lines

drivers/net/ipvlan/ipvlan_main.c line 687

Quoted 3 lines

drivers/net/ipvlan/ipvlan_main.c new file +3 -0

+
+     switch (event) {
+     case NETDEV_UP:

Can it be (in the future) somehow restricted so that net-namespace wouldn't be able to assign arbitrary IP address here? One of the reasons for using such devices is to enforce the container to use the IP address given from the host.

Quoted 7 lines

Probably this could be a config (sysfs?) entry which would lockup the config coming from ns when set. So code could look like -

          case NETDEV_UP:
                         if (!restrict_ns_config) {
                            ...
                         }
                         break;

Maybe introduce some "lock" call for ipvlan device after which no new IP addresses can be assigned? And the configuration would look like

1. create ipvlan
2. move to proper net namespace
3. add addresses
4. lock

?

Thanks,
Pavel

Quoted 9 lines

Maybe introduce some "lock" call for ipvlan device after which no new IP addresses can be assigned? And the configuration would look like

1. create ipvlan
2. move to proper net namespace
3. add addresses
4. lock

Quoted 1 line

?

Yes. Exporting this "locked" property on the master device so that it can be controlled from masters' net-ns. Only thing we have to ensure is that both possibilities are allowed i.e. trusted ns where config do not need to be locked as well as untrusted/hostile ns where one can lock it down. However this is a future enhancement and if your implementation idea is different than this concept; we can discuss it at the time of implementation.