RFC net: reducing memory footprint of network devices

As I have mentioned many times[1], at ~43+kB per instance the use of net_devices does not scale for deployments needing 10,000+ devices. At netconf 1.2 there was a discussion about using a net_device_common for the minimal set of common attributes with other structs built on top of that one for "full" devices. It provided a means for the code to know "non-standard" net_devices. Conceptually, that approach has its merits but it is not practical given the sweeping changes required to the code base. More importantly though struct net_device is not the problem; it weighs in at less than 2kB so reorganizing the code base around a refactored net_device is not going to solve the problem. The primary issue is all of the initializations done *because* it is a struct net_device -- kobject and sysfs and the protocols (e.g., ipv4, ipv6, mpls, neighbors).

So, how do you keep the desired attributes of a net device -- network addresses, xmit function, qdisc, netfilter rules, tcpdump -- while lowering the overhead of a net_device instance and without sweeping changes across net/ and drivers/net/?

This patch set introduces the concept of labeling net_devices as "lightweight", first mentioned at netdev 1.1 [1]. Users have to opt in to lightweight devices by passing a new attribute, IFLA_LWT_NETDEV, in the new link request. This lightweight tag is meant for virtual devices such as vlan, vrf, vti, and dummy where the user expects to create a lot of them and does not want the duplication of resources. Each device type can always opt out of a lightweight label if necessary by failing device creates.

Labeling a virtual device as "lightweight" reduces the footprint for device creation from ~43kB to ~6kB. That reduction in memory is obtained by:
1. no entry in sysfs
- kobject in net_device.device is not initialized

2. no entry in procfs
- no sysctl option for these devices

3. deferred ipv4, ipv6, mpls initialization
- network layer must be enabled before an address can be assigned or mpls labels can be processed
- enables what Florian called L2 only devices [2]

Once the core premise of a lightweight device is accepted, follow on patches can reduce the overhead of network initializations. e.g.,

1. remove devconf per device (ipv4 and ipv6)
- lightweight devices use the default settings rather than replicate the same data for each device

2. reduce / remove / opt out of snmp mibs
- snmp6_alloc_dev and icmpv6msg_mib_device specifically is a heavy hitter

Patches can also be found here:
https://github.com/dsahern/linux lwt-dev-rfc

And iproute2 here:
https://github.com/dsahern/iproute2 lwt-dev

Example:
ip li add foo lwd type vrf table 123

- creates VRF device 'foo' as a lightweight netdevice.

[1] http://www.netdevconf.org/1.1/proceedings/slides/ahern-aleksandrov-prabhu-scaling-network-cumulus.pdf
[2] https://www.spinics.net/lists/netdev/msg340808.html
David Ahern (6):
net: Add accessor for kboject in a net_device
net: Add flags argument to alloc_netdev_mqs
net: Introduce IFF_LWT_NETDEV flag
net: Do not intialize kobject for lightweight netdevs
net: Delay initializations for lightweight devices
net: add uapi for creating lightweight devices

 drivers/net/ethernet/mellanox/mlx5/core/ipoib.c |  2 +-
 drivers/net/ethernet/tile/tilegx.c              |  2 +-
 drivers/net/tun.c                               |  2 +-
 drivers/net/wireless/marvell/mwifiex/cfg80211.c |  2 +-
 include/linux/netdevice.h                       | 27 ++++++++--
 include/uapi/linux/if_link.h                    |  1 +
 net/batman-adv/sysfs.c                          | 13 ++++-
 net/bridge/br_if.c                              | 12 +++--
 net/bridge/br_sysfs_br.c                        | 17 +++---
 net/bridge/br_sysfs_if.c                        |  8 ++-
 net/core/dev.c                                  | 71 ++++++++++++++++++-------
 net/core/neighbour.c                            |  3 ++
 net/core/net-sysfs.c                            | 25 ++++++---
 net/core/rtnetlink.c                            | 10 +++-
 net/ethernet/eth.c                              |  2 +-
 net/ipv4/devinet.c                              | 18 ++++++-
 net/ipv6/addrconf.c                             |  9 ++++
 net/mac80211/iface.c                            |  2 +-
 net/mpls/af_mpls.c                              |  6 +++
 net/wireless/core.c                             | 15 ++++--
 20 files changed, 190 insertions(+), 57 deletions(-)

2.11.0 (Apple Git-81)