PATCH: Network Device Naming mechanism and policy
Subsystems (2)
A message that is not in the archive (sent off-list or not yet received): EDA0A4495861324DA2618B4C45DCB3EE5894ED@blrx3m08.blr.amer.dell.com
Quoted 8 lines · Narendra
Quoted 2 lines
example udev config:
SUBSYSTEM="net",SYMLINK+="net/by-mac/$sysfs{ifindex}.$sysfs{address}"
work as well. But coupling the ifindex to the MAC address like this doesn't work. (In general, coupling any two unrelated attributes when trying to do persistent names doesn't work.)
Attaching the latest patch incorporating review comments.
By creating character devices for every network device, we can use udev to maintain alternate naming policies for devices, including additional names for the same device, without interfering with the name that the kernel assigns a device.
This is conditionalized on CONFIG_NET_CDEV. If enabled (the default), device nodes will automatically be created in /dev/netdev/ for each network device. (/dev/net/ is already populated by the tun device.)
These device nodes are not functional at the moment - open() returns -ENOSYS. Their only purpose is to provide userspace with a kernel name to ifindex mapping, in a form that udev can easily manage.
--- include/linux/netdevice.h | 4 ++++ net/Kconfig | 10 ++++++++++ net/core/Makefile | 1 + net/core/cdev.c | 42 ++++++++++++++++++++++++++++++++++++++++++ net/core/cdev.h | 13 +++++++++++++ net/core/dev.c | 10 ++++++++++ net/core/net-sysfs.c | 13 +++++++++++++ 7 files changed, 93 insertions(+), 0 deletions(-) create mode 100644 net/core/cdev.c create mode 100644 net/core/cdev.h
include/linux/netdevice.h +4 -0
@@ -44,6 +44,7 @@#include <linux/workqueue.h> #include <linux/ethtool.h> +#include <linux/cdev.h> #include <net/net_namespace.h> #include <net/dsa.h> #ifdef CONFIG_DCB
@@ -916,6 +917,9 @@ struct net_device/* max exchange id for FCoE LRO by ddp */ unsigned int fcoe_ddp_xid;#endif +#ifdef CONFIG_NET_CDEV +struct cdev cdev; +#endif };#define to_net_dev (d) container_of(d, struct net_device, dev)
net/Kconfig +10 -0
@@ -43,6 +43,16 @@ config COMPAT_NETLINK_MESSAGES Newly written code should NEVER need this option but do compat-independent messages instead! +config NET_CDEV + bool "/dev files for network devices" + default y + help + This option causes /dev entries to be created for each + network device. This allows the use of udev to create + alternate device naming policies. + + If unsure, say Y. + menu "Networking options" source "net/packet/Kconfig"
net/core/Makefile +1 -0
@@ -19,4 +19,5 @@ obj-$(CONFIG_NET_DMA) += user_dma.o obj-$(CONFIG_FIB_RULES) += fib_rules.o obj-$(CONFIG_TRACEPOINTS) += net-traces.o obj-$(CONFIG_NET_DROP_MONITOR) += drop_monitor.o +obj-$(CONFIG_NET_CDEV) += cdev.o
net/core/cdev.c new file +42 -0
42 lines not on this page · raw message
net/core/cdev.h new file +13 -0
@@ -0,0 +1,13 @@ +#include <linux/netdevice.h> + +#ifdef CONFIG_NET_CDEV +void netdev_cdev_alloc (void ); +void netdev_cdev_init (struct net_device *dev); +void netdev_cdev_del (struct net_device *dev); +void netdev_cdev_kobj_init (struct device *dev,struct net_device *net); +#else +static inline void netdev_cdev_alloc (void ) {} +static inline void netdev_cdev_init (struct net_device *dev) {} +static inline void netdev_cdev_del (struct net_device *dev) {} +static inline void netdev_cdev_kobj_init (struct device *dev,struct net_device *net) {} +#endif
net/core/dev.c +10 -0
@@ -129,6 +129,7 @@#include <trace/events/napi.h> #include "net-sysfs.h" +#include "cdev.h" /* Instead of increasing this, you should create a hash table. */ #define MAX_GRO_SKBS 8
@@ -4684,6 +4685,7 @@ static void rollback_registered(struct net_device *dev)/* Remove entries from kobject tree */ netdev_unregister_kobject (dev); +netdev_cdev_del (dev);synchronize_net ();
@@ -4835,6 +4837,8 @@ int register_netdevice(struct net_device *dev)if (dev->features &NETIF_F_SG ) dev->features |=NETIF_F_GSO ; +netdev_cdev_init (dev); +netdev_initialize_kobject (dev); ret =netdev_register_kobject (dev);if (ret)
@@ -4864,6 +4868,7 @@ out:return ret; err_uninit: +netdev_cdev_del (dev);if (dev->netdev_ops->ndo_uninit) dev->netdev_ops->ndo_uninit (dev); goto out;
@@ -5371,6 +5376,7 @@ int dev_change_net_namespace(struct net_device *dev, struct net *net, const chardev_addr_discard (dev);netdev_unregister_kobject (dev); +netdev_cdev_del (dev);/* Actually switch the network namespace */ dev_net_set (dev, net);
@@ -5387,6 +5393,8 @@ int dev_change_net_namespace(struct net_device *dev, struct net *net, const char dev->iflink = dev->ifindex; } +netdev_cdev_init (dev); +/* Fixup kobjects */ err =netdev_register_kobject (dev);WARN_ON (err);
@@ -5620,6 +5628,8 @@ static int __init net_dev_init(void)BUG_ON (!dev_boot_phase); +netdev_cdev_alloc (); +if (dev_proc_init ()) goto out;
net/core/net-sysfs.c +13 -0
@@ -19,6 +19,7 @@#include <net/wext.h> #include "net-sysfs.h" +#include "cdev.h" #ifdef CONFIG_SYSFS static const char fmt_hex[] ="%#x\n" ;
@@ -461,6 +462,14 @@ static void netdev_release(struct device *d)kfree ((char *)dev - dev->padded); } +#ifdef CONFIG_NET_CDEV +static char *netdev_devnode (struct device *d,mode_t *mode) +{ +struct net_device *dev =to_net_dev (d); +return kasprintf (GFP_KERNEL ,"netdev/%s" , dev->name); +} +#endif +static struct class net_class = { .name ="net" , .dev_release = netdev_release,
@@ -470,6 +479,9 @@ static struct class net_class = {#ifdef CONFIG_HOTPLUG .dev_uevent = netdev_uevent,#endif +#ifdef CONFIG_NET_CDEV + .devnode = netdev_devnode, +#endif };/* Delete sysfs entries but hold kobject reference until after all
@@ -496,6 +508,7 @@ int netdev_register_kobject(struct net_device *net) dev->class = &net_class; dev->platform_data = net; dev->groups = groups; +netdev_cdev_kobj_init (dev, net);dev_set_name (dev,"%s" , net->name);
Signature · 2 lines
With regards, Narendra K
Quoted 10 lines · Narendra K
Quoted 8 lines · Narendra
Quoted 2 lines
example udev config:
SUBSYSTEM="net",SYMLINK+="net/by-mac/$sysfs{ifindex}.$sysfs{address}"
work as well. But coupling the ifindex to the MAC address like this doesn't work. (In general, coupling any two unrelated attributes when trying to do persistent names doesn't work.)
Attaching the latest patch incorporating review comments.
Thank you Narendra.
Let me also note that we are prepared to have userspace consumers of this new character device node.
http://linux.dell.com/wiki/index.php/Oss/libnetdevname
notes how the kernel patch will interact with udev, describes the new library helper function in libnetdevname, and has patches for net-tools, iproute2, and ethtool to make use of the helper function.
As has been noted here, MAC addresses are not necessarily unique to an interface. As such, we are not proposing a net/by-mac/* symlink to /dev/netdev/*.
Thanks,
Matt
Signature · 3 lines
Matt Domsch Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux
Quoted 11 lines · Matt Domsch
Let me also note that we are prepared to have userspace consumers of this new character device node.
http://linux.dell.com/wiki/index.php/Oss/libnetdevname
notes how the kernel patch will interact with udev, describes the new library helper function in libnetdevname, and has patches for net-tools, iproute2, and ethtool to make use of the helper function.
As has been noted here, MAC addresses are not necessarily unique to an interface.
Only in the case of e.g. qemu (virtual hardware), I think. (Or some kinds of broken hardware. Anything not on the udev whitelist from 75-persistent-net-generator.rules.)
The combination of (MAC, ifindex) is not unique, which is what I meant earlier -- but the setup on the wiki seems to handle this properly. Assuming there was a /dev/net/by-mac/00:01:02:03:04:05 link, it should work fine...
Quoted 4 lines · Bryan Kadzban
Quoted 2 lines
As has been noted here, MAC addresses are not necessarily unique to an interface.
Only in the case of e.g. qemu (virtual hardware), I think. (Or some kinds of broken hardware.
Some Sun products have multiple interfaces sharing the same MAC address.
Signature · 2 lines
ciao, Marco
Quoted 4 lines · Matt Domsch
As has been noted here, MAC addresses are not necessarily unique to an interface. As such, we are not proposing a
net/by-mac/*symlink to/dev/netdev/*.
On the other hand, they *tend* to be unique for a wide range of systems. This makes them pretty comparable to LABELs on disks, and we have a /dev/disk/by-label
Remember that udev already supports symlink stacking, and priorities and such.
I don't think there's any danger of supporting a /dev/netdev/by-mac by default, it'll be a benefit to most and those who don't have unique MACs will just ignore it.
Scott
Signature · 2 lines
Scott James Remnant [email protected]
Quoted 9 lines · Scott James Remnant
Quoted 4 lines · Matt Domsch
As has been noted here, MAC addresses are not necessarily unique to an interface. As such, we are not proposing a
net/by-mac/*symlink to/dev/netdev/*.On the other hand, they *tend* to be unique for a wide range of systems. This makes them pretty comparable to LABELs on disks, and we have a
/dev/disk/by-label
[...]
MAC addresses are normally assigned automatically but can be overridden if necessary. In that respect they are more like UUIDs for disks.
I don't see any analogue of disk labels, though labels could conceivably be added to some NICs using VPD.
Ben.
Signature · 3 lines
Ben Hutchings, Senior Software Engineer, Solarflare Communications Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
Quoted 10 lines · Scott James Remnant
On the other hand, they *tend* to be unique for a wide range of systems. This makes them pretty comparable to LABELs on disks, and we have a
/dev/disk/by-labelRemember that udev already supports symlink stacking, and priorities and such.
I don't think there's any danger of supporting a
/dev/netdev/by-macby default, it'll be a benefit to most and those who don't have unique MACs will just ignore it.
At the moment, we do not appear to get the proper change uevents from things like 'ip link set dev <foo> address <bar>', so we can't currently maintain these symlinks.
Bill
Quoted 8 lines · Bill Nottingham
Quoted 7 lines · Scott James Remnant
On the other hand, they *tend* to be unique for a wide range of systems. This makes them pretty comparable to LABELs on disks, and we have a
/dev/disk/by-labelRemember that udev already supports symlink stacking, and priorities and such.
Quoted 7 lines
Quoted 3 lines
I don't think there's any danger of supporting a
/dev/netdev/by-macby default, it'll be a benefit to most and those who don't have unique MACs will just ignore it.At the moment, we do not appear to get the proper change uevents from things like 'ip link set dev <foo> address <bar>', so we can't currently maintain these symlinks.
And if we really want seamless support for MAC spoofing, we want ETHTOOL_GPERMADDR for all drivers too, so that if your configuration says "rename device XX:XX:XX:XX:XX:XX to YY:YY:YY:YY:YY:YY" we can actually figure stuff out after the spoof.
Dan
Quoted 13 lines · Dan Williams
Quoted 12 lines · Bill Nottingham
Quoted 10 lines · Scott James Remnant
On the other hand, they *tend* to be unique for a wide range of systems. This makes them pretty comparable to LABELs on disks, and we have a
/dev/disk/by-labelRemember that udev already supports symlink stacking, and priorities and such.
I don't think there's any danger of supporting a
/dev/netdev/by-macby default, it'll be a benefit to most and those who don't have unique MACs will just ignore it.
Quoted 8 lines
Quoted 3 lines
At the moment, we do not appear to get the proper change uevents from things like 'ip link set dev <foo> address <bar>', so we can't currently maintain these symlinks.
And if we really want seamless support for MAC spoofing, we want
ETHTOOL_GPERMADDRfor all drivers too, so that if your configuration says "rename device XX:XX:XX:XX:XX:XX to YY:YY:YY:YY:YY:YY" we can actually figure stuff out after the spoof.
ETHTOOL_GPERMADDR is handled in the ethtool core now. Are you thinking of drivers that don't have ethtool ops? Maybe it's time to add default operations.
Ben.
Signature · 3 lines
Ben Hutchings, Senior Software Engineer, Solarflare Communications Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
Quoted 10 lines · Ben Hutchings
Quoted 4 lines · Dan Williams
And if we really want seamless support for MAC spoofing, we want
ETHTOOL_GPERMADDRfor all drivers too, so that if your configuration says "rename device XX:XX:XX:XX:XX:XX to YY:YY:YY:YY:YY:YY" we can actually figure stuff out after the spoof.
ETHTOOL_GPERMADDRis handled in the ethtool core now. Are you thinking of drivers that don't have ethtool ops? Maybe it's time to add default operations.
Not quite true -- dev->perm_addr still has to be set by the driver.
John
Signature · 2 lines
John W. Linville Someday the world will need a hero, and you [email protected] might be all we have. Be ready.
Quoted 16 lines · Narendra K
Quoted 8 lines · Narendra
Quoted 2 lines
example udev config:
SUBSYSTEM="net",SYMLINK+="net/by-mac/$sysfs{ifindex}.$sysfs{address}"
work as well. But coupling the ifindex to the MAC address like this doesn't work. (In general, coupling any two unrelated attributes when trying to do persistent names doesn't work.)
Attaching the latest patch incorporating review comments.
By creating character devices for every network device, we can use udev to maintain alternate naming policies for devices, including additional names for the same device, without interfering with the name that the kernel assigns a device.
Quoted 7 lines
This is conditionalized on
CONFIG_NET_CDEV. If enabled (the default), device nodes will automatically be created in/dev/netdev/for each network device. (/dev/net/is already populated by the tun device.)These device nodes are not functional at the moment -
open()returns-ENOSYS. Their only purpose is to provide userspace with a kernel name to ifindex mapping, in a form that udev can easily manage.
How does this patch work with the network namespace functionality?
thanks,
greg k-h
Quoted 21 lines · Greg KH
Quoted 20 lines · Narendra K
Quoted 8 lines · Narendra
Quoted 2 lines
example udev config:
SUBSYSTEM="net",SYMLINK+="net/by-mac/$sysfs{ifindex}.$sysfs{address}"
work as well. But coupling the ifindex to the MAC address like this doesn't work. (In general, coupling any two unrelated attributes when trying to do persistent names doesn't work.)
Attaching the latest patch incorporating review comments.
By creating character devices for every network device, we can use udev to maintain alternate naming policies for devices, including additional names for the same device, without interfering with the name that the kernel assigns a device.
This is conditionalized on
CONFIG_NET_CDEV. If enabled (the default), device nodes will automatically be created in/dev/netdev/for each network device. (/dev/net/is already populated by the tun device.)
Quoted 5 lines
Quoted 3 lines
These device nodes are not functional at the moment -
open()returns-ENOSYS. Their only purpose is to provide userspace with a kernel name to ifindex mapping, in a form that udev can easily manage.How does this patch work with the network namespace functionality?
There is a monitonically increasing static ifindex kept in net/core/dev.c:dev_new_index(), which is shared across all namespaces. struct net_device ifindex field is assigned from this. So two devices in two different namespaces can't share an ifindex value. However, the device can be present (or not) in the per-namespace dev_name_hash and dev_index_hashes. This patch doesn't change this at all.
uevents aren't namespaced. Presumably that means /dev can't be polyinstantiated. Therefore, all devnodes in /dev/netdev/* will be visible to all processes, where 'ifconfig' and friends would only show device names in the processes namespace. This doesn't mean the app can _do_ anything (it's the same as if it tried to act on a device using an ifindex for a device not in its namespace), but yes, the fact that such a device exists will be exposed.
Signature · 3 lines
Matt Domsch Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux
Quoted 8 lines · Matt Domsch
uevents aren't namespaced. Presumably that means /dev can't be polyinstantiated. Therefore, all devnodes in
/dev/netdev/*will be visible to all processes, where 'ifconfig' and friends would only show device names in the processes namespace. This doesn't mean the app can _do_ anything (it's the same as if it tried to act on a device using an ifindex for a device not in its namespace), but yes, the fact that such a device exists will be exposed.
That's the problem that the sysfs namespace patches were trying to address.
Now I'm not saying it is a valid thing to try to work with this kind of crazy, I was just wondering how it would work out. Looks like it doesn't :)
thanks,
greg k-h
Quoted 10 lines · Narendra K
Quoted 8 lines · Narendra
Quoted 2 lines
example udev config:
SUBSYSTEM="net",SYMLINK+="net/by-mac/$sysfs{ifindex}.$sysfs{address}"
work as well. But coupling the ifindex to the MAC address like this doesn't work. (In general, coupling any two unrelated attributes when trying to do persistent names doesn't work.)
Attaching the latest patch incorporating review comments.
Same patch, rebased to linux-next.
By creating character devices for every network device, we can use udev to maintain alternate naming policies for devices, including additional names for the same device, without interfering with the name that the kernel assigns a device.
This is conditionalized on CONFIG_NET_CDEV. If enabled (the default), device nodes will automatically be created in /dev/netdev/ for each network device. (/dev/net/ is already populated by the tun device.)
These device nodes are not functional at the moment - open() returns -ENOSYS. Their only purpose is to provide userspace with a kernel name to ifindex mapping, in a form that udev can easily manage.
--- include/linux/netdevice.h | 4 ++++ net/Kconfig | 10 ++++++++++ net/core/Makefile | 1 + net/core/cdev.c | 42 ++++++++++++++++++++++++++++++++++++++++++ net/core/cdev.h | 13 +++++++++++++ net/core/dev.c | 10 ++++++++++ net/core/net-sysfs.c | 13 +++++++++++++ 7 files changed, 93 insertions(+), 0 deletions(-) create mode 100644 net/core/cdev.c create mode 100644 net/core/cdev.h
include/linux/netdevice.h +4 -0
@@ -44,6 +44,7 @@#include <linux/workqueue.h> #include <linux/ethtool.h> +#include <linux/cdev.h> #include <net/net_namespace.h> #include <net/dsa.h> #ifdef CONFIG_DCB
@@ -916,6 +917,9 @@ struct net_device/* max exchange id for FCoE LRO by ddp */ unsigned int fcoe_ddp_xid;#endif +#ifdef CONFIG_NET_CDEV +struct cdev cdev; +#endif };#define to_net_dev (d) container_of(d, struct net_device, dev)
net/Kconfig +10 -0
@@ -43,6 +43,16 @@ config COMPAT_NETLINK_MESSAGES Newly written code should NEVER need this option but do compat-independent messages instead! +config NET_CDEV + bool "/dev files for network devices" + default y + help + This option causes /dev entries to be created for each + network device. This allows the use of udev to create + alternate device naming policies. + + If unsure, say Y. + menu "Networking options" source "net/packet/Kconfig"
net/core/Makefile +1 -0
@@ -19,4 +19,5 @@ obj-$(CONFIG_NET_DMA) += user_dma.o obj-$(CONFIG_FIB_RULES) += fib_rules.o obj-$(CONFIG_TRACEPOINTS) += net-traces.o obj-$(CONFIG_NET_DROP_MONITOR) += drop_monitor.o +obj-$(CONFIG_NET_CDEV) += cdev.o
net/core/cdev.c new file +42 -0
42 lines not on this page · raw message
net/core/cdev.h new file +13 -0
@@ -0,0 +1,13 @@ +#include <linux/netdevice.h> + +#ifdef CONFIG_NET_CDEV +void netdev_cdev_alloc (void ); +void netdev_cdev_init (struct net_device *dev); +void netdev_cdev_del (struct net_device *dev); +void netdev_cdev_kobj_init (struct device *dev,struct net_device *net); +#else +static inline void netdev_cdev_alloc (void ) {} +static inline void netdev_cdev_init (struct net_device *dev) {} +static inline void netdev_cdev_del (struct net_device *dev) {} +static inline void netdev_cdev_kobj_init (struct device *dev,struct net_device *net) {} +#endif
net/core/dev.c +10 -0
@@ -129,6 +129,7 @@#include <trace/events/napi.h> #include "net-sysfs.h" +#include "cdev.h" /* Instead of increasing this, you should create a hash table. */ #define MAX_GRO_SKBS 8
@@ -4684,6 +4685,7 @@ static void rollback_registered(struct net_device *dev)/* Remove entries from kobject tree */ netdev_unregister_kobject (dev); +netdev_cdev_del (dev);synchronize_net ();
@@ -4835,6 +4837,8 @@ int register_netdevice(struct net_device *dev)if (dev->features &NETIF_F_SG ) dev->features |=NETIF_F_GSO ; +netdev_cdev_init (dev); +netdev_initialize_kobject (dev); ret =call_netdevice_notifiers (NETDEV_POST_INIT , dev);
@@ -4870,6 +4874,7 @@ out:return ret; err_uninit: +netdev_cdev_del (dev);if (dev->netdev_ops->ndo_uninit) dev->netdev_ops->ndo_uninit (dev); goto out;
@@ -5377,6 +5382,7 @@ int dev_change_net_namespace(struct net_device *dev, struct net *net, const chardev_addr_discard (dev);netdev_unregister_kobject (dev); +netdev_cdev_del (dev);/* Actually switch the network namespace */ dev_net_set (dev, net);
@@ -5393,6 +5399,8 @@ int dev_change_net_namespace(struct net_device *dev, struct net *net, const char dev->iflink = dev->ifindex; } +netdev_cdev_init (dev); +/* Fixup kobjects */ err =netdev_register_kobject (dev);WARN_ON (err);
@@ -5626,6 +5634,8 @@ static int __init net_dev_init(void)BUG_ON (!dev_boot_phase); +netdev_cdev_alloc (); +if (dev_proc_init ()) goto out;
net/core/net-sysfs.c +13 -0
@@ -19,6 +19,7 @@#include <net/wext.h> #include "net-sysfs.h" +#include "cdev.h" #ifdef CONFIG_SYSFS static const char fmt_hex[] ="%#x\n" ;
@@ -501,6 +502,14 @@ static void netdev_release(struct device *d)kfree ((char *)dev - dev->padded); } +#ifdef CONFIG_NET_CDEV +static char *netdev_devnode (struct device *d,mode_t *mode) +{ +struct net_device *dev =to_net_dev (d); +return kasprintf (GFP_KERNEL ,"netdev/%s" , dev->name); +} +#endif +static struct class net_class = { .name ="net" , .dev_release = netdev_release,
@@ -510,6 +519,9 @@ static struct class net_class = {#ifdef CONFIG_HOTPLUG .dev_uevent = netdev_uevent,#endif +#ifdef CONFIG_NET_CDEV + .devnode = netdev_devnode, +#endif };/* Delete sysfs entries but hold kobject reference until after all
@@ -536,6 +548,7 @@ int netdev_register_kobject(struct net_device *net) dev->class = &net_class; dev->platform_data = net; dev->groups = groups; +netdev_cdev_kobj_init (dev, net);dev_set_name (dev,"%s" , net->name);
Signature · 1 line
1.6.0.6
Quoted 23 lines · On Fri, 9 Oct 2009 16:09:09 -0500 Matt Domsch <[email protected]> wrote:
Quoted 10 lines · Narendra K
Quoted 8 lines · Narendra
Quoted 2 lines
example udev config:
SUBSYSTEM="net",SYMLINK+="net/by-mac/$sysfs{ifindex}.$sysfs{address}"
work as well. But coupling the ifindex to the MAC address like this doesn't work. (In general, coupling any two unrelated attributes when trying to do persistent names doesn't work.)
Attaching the latest patch incorporating review comments.
Same patch, rebased to linux-next.
By creating character devices for every network device, we can use udev to maintain alternate naming policies for devices, including additional names for the same device, without interfering with the name that the kernel assigns a device.
This is conditionalized on
CONFIG_NET_CDEV. If enabled (the default), device nodes will automatically be created in/dev/netdev/for each network device. (/dev/net/is already populated by the tun device.)
Quoted 7 lines
These device nodes are not functional at the moment -
open()returns-ENOSYS. Their only purpose is to provide userspace with a kernel name to ifindex mapping, in a form that udev can easily manage.Signed-off-by Jordan Hargrave <[email protected]>Signed-off-by Narendra K <[email protected]>Signed-off-by Matt Domsch <[email protected]>
Maybe I'm dense but can't see why having a useless /dev/net/ symlinks is a good interface choice. Perhaps you should explain the race between PCI scan and udev in more detail, and why solving it in either of those places won't work. As it stands you are proposing yet another wart to the already complex set of network interface API's which has implications for security as well as increasing the number of possible bugs.
Quoted 6 lines · Stephen Hemminger
Maybe I'm dense but can't see why having a useless
/dev/net/symlinks is a good interface choice. Perhaps you should explain the race between PCI scan and udev in more detail, and why solving it in either of those places won't work. As it stands you are proposing yet another wart to the already complex set of network interface API's which has implications for security as well as increasing the number of possible bugs.
The fundamental challenge is that system administrators, particularly those of server-class hardware with multiple network ports present (some on the motherboard, some on add-in cards), have the not-so-unreasonable expectation that there is a deterministic mapping between those ports and the name one uses to address those ports.
The fundamental roadblock to this is that enumeration != naming, except that it is for network devices, and we keep changing the enumeration order.
Today, port naming is completely nondeterministic. If you have but one NIC, there are few chances to get the name wrong (it'll be eth0). If you have >1 NIC, chances increase to get it wrong.
The complexity arises at multiple levels.
First, device driver load order. In the 2.4 kernel days, and even mostly early 2.6 kernel days, the order in which network drivers loaded played a role in determining the name of the device. Drivers loaded first would get their devices named first. If I have two types of devices, say an e100-driven NIC and a tg3-driven NIC, I could figure out that the names would be eth0á00 and eth1=tg3 by setting the load order in /etc/modules.conf (now modprobe.conf). If I wanted the other order, fine, just switch it around in modules.conf and reboot. OS installers, being the first running instance of Linux, before modprobe.conf existed to set that ordering, had to have other mechanisms to load drivers (often manually, or if programmatically such as in a kickstart or autoyast file, was still somewhat fixed).
With the advent of modaliases + udev, now modprobe.conf doesn't contain this ordering anymore, and udev loads the drivers. So while it wasn't perfect, it was better than nothing, and that's gone now.
It gets even worse as, to speed up boot time, modprobes can be run in parallel, and even within individual drivers, the NICs get initialized (and named) in parallel. Further confusing things, some devices need firmware loaded into them before getting names assigned, which is done from userspace, and they race.
Second, PCI device list order. In the 2.4 kernel days, the PCI device list was scanned "breadth-first" (for each bus; for each device; for each function; do load...). FWIW, Windows still does this. It gives BIOS, which assigns PCI bus numbers, a chance to put LOMs at a lower bus number than add-in cards. Module load order still mattered, but at least if you had say 2 e1000 ports as LOMs, and 2 e1000 ports on add-in cards, you pretty much knew the ordering would be eth0 as lowest bdf on the motherboard, eth1 as next bdf on the motherboard, and eth2 and 3 as the add-in cards in ascending slot order.
With the advent of PCI hot plug in the 2.5 kernel series, the
breadth-first ordering became depth-first. (for each bus; for each
device; if the device is a bridge, scan the busses behind it.). This caused NICs on bus 0 device 5, and bus 1 device 3, (eth0 and 1 respectively) to be enumerated differently due to the a bridge from bus 0 to bus 1 at 0:4. My crude hack of pci¿sort, with some dmi strings to match and auto-enable, at least reverted this back to the ordering the 2.4 kernel and Windows used. Now we have to keep adding systems to this DMI list (Dell has a number of systems on this list today; HP has even more). And it doesn't completely solve the problem, just masks it.
So, to address the ordering problem, I placed a constraint on our server hardware teams, forcing them to lay out their boards and assign PCIe lanes and bus numbers, such that at least the designed "first" LOM would get found first in either depth-first or breadth-first order. Our 10G and 11G servers have this restriction in place, though it wasn't easy. And it's gotten even harder, as the PCIe switches expand the number of lanes available. We no longer have the traditional tiered buses architecture, but the PCI layer for this purpose thinks we do. I need to remove this constraint on the hardware teams - it's gotten to be impossible for the chipset lanes to be laid out efficiently with this constraint.
All of the above just papered over the enumeration != naming problem.
Third, stateless computing is becoming more and more commonplace. The Field Replaceable Unit is the server itself. Got a bad server? Pull it out, move the disks to an identical unit, insert the new server, and go. Fix the bad server offline and bring it back. In this model, having MAC addresses as the mechanism that is providing the determinism (/etc/mactab or udev persistent naming rules) breaks, because the MAC addresses of the ports on the new server won't be the same as on the old server. HP even has a technology to solve _this_ problem (in their blade chassis) - Virtual Connect. The MACs get assigned by the chassis to the blades at POST, and are fixed to the slot. Slick, and Dell has an even more flexible similar feature FlexAddress. This doesn't solve the OS installer problem of "which of these NICs should I use to do an install?" but it does recognize the problem space and tries to overcome it.
Fourth, for OS installers, choosing which NIC to use at installtime, when all the NICs are plugged in, can be difficult. PXE environments, using pxelinux and its IPAPPEND 2 option, will append "BOOTIF=xx:xx:xx:xx:xx:xx" to the kernel command line, that containing the MAC address of the NIC used for PXE. Neat trick. Yes, we then had to teach the OS installers to recognize and use this. But it only works if you PXE boot, and only for that one NIC.
Fifth, network devices can have only a single name. eth0. If we look at disks, we see udev manages a tree of symlinks for /dev/disk/by-label, /dev/disk/by-path, /dev/disk/by-uuid. And as a system admin, if I wanted to also create a udev rule for /dev/disk/by-function (boot, swap, mattsstorage), it's trivial to do so. Why can't we have this flexibility for network devices too?
So, how do we get deterministic naming for all the NICs in a system? That's what I'm going for. Picture a network switch, with several blades, and several ports on each blade. The network admin addresses each port as say 1/16 (the 16th port on blade 1, clearly labeled). The parallel on servers is the chassis label printed on the outside (say, "Gb1"). But due the above, there is no guarantee, and in fact little chance, that Gb1 will be consistently named eth0 - it may vary from boot to boot. That's full of fail.
For a concrete example, the 4 bnx2 chips in my PowerEdge R610 with a current 2.6 kernel, loading only one driver, the ports get assigned names in nondeterministic order on each boot. Given that the ifcfg-eth* rules, netfilter rules, and the rest all expect deterministic naming, massive failure ensues unless some form of determinism is brought back in.
The idea to use a character device node to expose the ifindex value, and udev to manage a tree of symlinks to it, really follows the model used today for disks. It allows us to get deterministic names for devices (albeit, the names are symlinks), and multiple names for devices (through multiple symlink rules). That some people want to use the char device to call ioctl() and read/write, as is possible on the BSDs, would just be gravy IMHO.
It does require a change in behavior for a system administrator. Instead of hard-coding 'eth0' into her scripts, she uses '/dev/net/by-function/boot' or somesuch. But then that name is guaranteed to always refer to the "right" NIC. Every admin I've spoken to is willing to make this kind of change, as long as they get the consistent, deterministic naming they expect but don't have today. And it does require patching userspace apps to take both a kernel device name, or a path, and to resolve the path to device name or ifindex. We wrote libnetdevname (really, one function), and have patches for several userspace apps to use it, to prove it can be done.
One alternative would be to do something using the sysfs ifindex value already exported. e.g.
/sys/devices/pci0000:00/0000:00:05.0/0000:05:00.0/0000:06:07.0/net/eth0/ifindex
but we have never had symlinks from /dev into /sys before (doesn't mean we couldn't though). In that case, udev would grow to manage /dev/net/by-chassis-label/Embedded_NIC_1 -> /sys/devices/.../net/eth0, and libnetdevname would be used to follow the symlink in applications. This approach could solve my problem without (many or any?) kernel changes needed, but wouldn't help those who want to do ioctl/read/write to a devnode.
Given the problem, I really do need a solution. I've proposed one method, and an alternative, but I can't afford to let the problem stay unaddressed any longer, and need a clear direction to be chosen. The char device gives me what I need, and others what they want also.
Thanks for listening to the diatribe. For more examples and workarounds that we've been telling our customers for several years, check out http://linux.dell.com/papers.shtml for the Network Interface Card Naming whitepaper.
Signature · 3 lines
Matt Domsch Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux
Quoted 3 lines · Matt Domsch
The fundamental roadblock to this is that enumeration != naming, except that it is for network devices, and we keep changing the enumeration order.
No, the hardware changes the enumeration order, it places _no_ guarantees on what order stuff will be found in. So this is not the kernel changing, just to be clear.
Again, I have a machine here that likes to reorder PCI devices every 4th or so boot times, and that's fine according to the PCI spec. Yeah, it's a crappy BIOS, but the manufacturer rightly pointed out that it is not in violation of anything.
Quoted 3 lines
Today, port naming is completely nondeterministic. If you have but one NIC, there are few chances to get the name wrong (it'll be eth0). If you have >1 NIC, chances increase to get it wrong.
That is why all distros name network devices based on the only deterministic thing they have today, the MAC address. I still fail to see why you do not like this solution, it is honestly the only way to properly name network devices in a sane manner.
All distros also provide a way to easily rename the network devices, to place a specific name on a specific MAC address, so again, this should all be solved already.
No matter how badly your BIOS teams mess up the PCI enumeration order :)
thanks,
greg k-h
Greg,
Quoted 8 lines
No, the hardware changes the enumeration order, it places _no_ guarantees on what order stuff will be found in. So this is not the kernel changing, just to be clear.
Again, I have a machine here that likes to reorder PCI devices every 4th or so boot times, and that's fine according to the PCI spec. Yeah, it's a crappy BIOS, but the manufacturer rightly pointed out that it is not in violation of anything.
I think the open call should be implemented then. By the patch very little knowledge is being shared on type of network implementation it is trying to do.Also it is messing with core datastructure and procedures. This seems to be simplified by changing implementing the other operations like poll().
Quoted 4 lines
That is why all distros name network devices based on the only deterministic thing they have today, the MAC address. I still fail to see why you do not like this solution, it is honestly the only way to properly name network devices in a sane manner.
This is feature that needs to be implemented. As per the rules followed.
Quoted 6 lines
All distros also provide a way to easily rename the network devices, to place a specific name on a specific MAC address, so again, this should all be solved already.
No matter how badly your BIOS teams mess up the PCI enumeration order :)
This is an problem, But I think this can be solved by implementing some of the routines in the network device.
Quoted 12 lines · Sujit K M
Greg,
Quoted 8 lines
No, the hardware changes the enumeration order, it places _no_ guarantees on what order stuff will be found in. ?So this is not the kernel changing, just to be clear.
Again, I have a machine here that likes to reorder PCI devices every 4th or so boot times, and that's fine according to the PCI spec. ?Yeah, it's a crappy BIOS, but the manufacturer rightly pointed out that it is not in violation of anything.
Quoted 3 lines
I think the open call should be implemented then. By the patch very little knowledge is being shared on type of network implementation it is trying to do.
What would open() accomplish? What good would the file descriptor be? What could you use it for?
Quoted 2 lines
Also it is messing with core datastructure and procedures. This seems to be simplified by changing implementing the other operations like
poll().
I don't understand.
Quoted 6 lines
Quoted 4 lines
That is why all distros name network devices based on the only deterministic thing they have today, the MAC address. ?I still fail to see why you do not like this solution, it is honestly the only way to properly name network devices in a sane manner.
This is feature that needs to be implemented. As per the rules followed.
This feature is already implemented today, all distros have it.
Quoted 8 lines
Quoted 5 lines
All distros also provide a way to easily rename the network devices, to place a specific name on a specific MAC address, so again, this should all be solved already.
No matter how badly your BIOS teams mess up the PCI enumeration order :)
This is an problem, But I think this can be solved by implementing some of the routines in the network device.
I don't, see the rules that your distro ships today for persistant network devices, it's already there, no need to change the kernel at all.
thanks,
greg k-h
Quoted 13 lines · Greg KH
Quoted 12 lines · Sujit K M
Greg,
Quoted 8 lines
No, the hardware changes the enumeration order, it places _no_ guarantees on what order stuff will be found in. ?So this is not the kernel changing, just to be clear.
Again, I have a machine here that likes to reorder PCI devices every 4th or so boot times, and that's fine according to the PCI spec. ?Yeah, it's a crappy BIOS, but the manufacturer rightly pointed out that it is not in violation of anything.
Quoted 6 lines
Quoted 3 lines
I think the open call should be implemented then. By the patch very little knowledge is being shared on type of network implementation it is trying to do.
What would
open()accomplish? What good would the file descriptor be? What could you use it for?
Currently all net device ioctls are carried out through arbitrary sockets and identify the device by name (aside from one to look up the name by ifindex). Ever since it became possible to rename net devices, it has been possible for a sequence of ioctls intended for one device to race with renaming of that device. Adding open() and ioctl() to the character device (which seems reasonably easy) would provide a way to avoid this.
On the other hand, the netlink configuration APIs already use ifindex so it may be better just to say that the device ioctls are deprecated and applications should use netlink.
Quoted 5 lines
Quoted 2 lines
Also it is messing with core datastructure and procedures. This seems to be simplified by changing implementing the other operations like
poll().I don't understand.
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
That is why all distros name network devices based on the only deterministic thing they have today, the MAC address. ?I still fail to see why you do not like this solution, it is honestly the only way to properly name network devices in a sane manner.
This is feature that needs to be implemented. As per the rules followed.
This feature is already implemented today, all distros have it.
No, see below.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
All distros also provide a way to easily rename the network devices, to place a specific name on a specific MAC address, so again, this should all be solved already.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
No matter how badly your BIOS teams mess up the PCI enumeration order :)
This is an problem, But I think this can be solved by implementing some of the routines in the network device.
I don't, see the rules that your distro ships today for persistant network devices, it's already there, no need to change the kernel at all.
The udev persistent net rules work tolerably well for a single system with a stable set of net devices.
They do not solve the problem Matt's talking about, which is lack of consistency between multiple systems, because the initial enumeration order is not predictable.
They also result in name changes when a NIC (or motherboard) is swapped. For some users, that's fine; for others, it's not.
The ability to specify NICs by port name or PCI address should solve these problems.
Ben.
Signature · 3 lines
Ben Hutchings, Senior Software Engineer, Solarflare Communications Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
Quoted 3 lines · Ben Hutchings
On the other hand, the netlink configuration APIs already use ifindex so it may be better just to say that the device ioctls are deprecated and applications should use netlink.
I thought that is what was already encouraged to happen.
Quoted 10 lines
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
That is why all distros name network devices based on the only deterministic thing they have today, the MAC address. ?I still fail to see why you do not like this solution, it is honestly the only way to properly name network devices in a sane manner.
This is feature that needs to be implemented. As per the rules followed.
This feature is already implemented today, all distros have it.
No, see below.
Yes, if not, file a bug in your distro, all of the infrastructure is already in place, and the udev rules and scripts are already written.
Quoted 13 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
All distros also provide a way to easily rename the network devices, to place a specific name on a specific MAC address, so again, this should all be solved already.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
No matter how badly your BIOS teams mess up the PCI enumeration order :)
This is an problem, But I think this can be solved by implementing some of the routines in the network device.
I don't, see the rules that your distro ships today for persistant network devices, it's already there, no need to change the kernel at all.
Quoted 6 lines
The udev persistent net rules work tolerably well for a single system with a stable set of net devices.
They do not solve the problem Matt's talking about, which is lack of consistency between multiple systems, because the initial enumeration order is not predictable.
Again, you name the device as a MAC address. Or something else that the BIOS exports in a unique manner (PCI slot name, etc.). That is consistant. If not, then fix the BIOS.
Quoted 5 lines
They also result in name changes when a NIC (or motherboard) is swapped. For some users, that's fine; for others, it's not.
The ability to specify NICs by port name or PCI address should solve these problems.
That can be done today quite easily. But note that PCI addresses are not guaranteed to be stable. As lots of machines are known to have happen.
Again, none of this requires any kernel changes today at all, let alone adding dummy char devices for network devices.
thanks,
greg k-h
Quoted 8 lines · Greg KH
Quoted 3 lines · Matt Domsch
The fundamental roadblock to this is that enumeration != naming, except that it is for network devices, and we keep changing the enumeration order.
No, the hardware changes the enumeration order, it places _no_ guarantees on what order stuff will be found in. So this is not the kernel changing, just to be clear.
Over time the kernel has changed its enumeration mechanisms, and introduced parallelism into the process (which is a good thing), which, from a user perspective, makes names nondeterministic. Yes, fixing this up by hard-coding MAC addresses after install has been the traditional mechanism to address this. I think there's a better way.
Quoted 4 lines
Again, I have a machine here that likes to reorder PCI devices every 4th or so boot times, and that's fine according to the PCI spec. Yeah, it's a crappy BIOS, but the manufacturer rightly pointed out that it is not in violation of anything.
I haven't encounted this myself, but yes, it's valid but annoying.
Quoted 4 lines
Quoted 3 lines
Today, port naming is completely nondeterministic. If you have but one NIC, there are few chances to get the name wrong (it'll be eth0). If you have >1 NIC, chances increase to get it wrong.
Quoted 8 lines
That is why all distros name network devices based on the only deterministic thing they have today, the MAC address. I still fail to see why you do not like this solution, it is honestly the only way to properly name network devices in a sane manner.
All distros also provide a way to easily rename the network devices, to place a specific name on a specific MAC address, so again, this should all be solved already.
It's not the only way, it introduces state where there's a desire for a stateless solution, it's useless for getting all the names right at initial OS install time, and it restricts us to a single "name" for a given device.
We can get additional information from BIOS. SMBIOS 2.6 (types 9 and
41) has the fields to let us get a "label" for an device at a given b/d/f. On my PowerEdge R610, I see "Embedded NIC 1" .. "Embedded NIC 4" for the 4 LOMs. These labels have a clear correlation to the labels on the back of the chassis at these ports. biosdevname can parse and report this. HP made a similar vendor-specific extension to SMBIOS for their platforms, which biosdevname also parses. Even if BIOS decides they need to renumber the busses on every boot, it can keep this table correct. (insert general mistrust of BIOS authors rant; that's not the point here.)
biosdevname can be used in udev rules to create multiple names for a given device. Rules such as:
PROGRAM="/sbin/biosdevname --policy=all_names -i %k", SYMLINK+="net/by-slot-name/%c", OPTIONS+="string_escape=replace" PROGRAM="/sbin/biosdevname --policy=smbios_names -i %k", SYMLINK+="net/by-chassis-label/%c", OPTIONS+="string_escape=replace"
SMBIOS has its own problems, specifically that it's not hot-plug aware (it's a static table created during POST). And if a better way is found (perhaps through the PCI SIG or ACPI), great, biosdevname can be extended to use it. But, without at least a change in udev or the kernel, it doesn't do any good.
Quoted 2 lines
No matter how badly your BIOS teams mess up the PCI enumeration order :)
In my case, the BIOS for a given system always configures the ports the same way, and assigns b/d/f the same way. With no change in the BIOS or hardware, I still see the ports enumerated differently on each boot. :-(
Signature · 3 lines
Matt Domsch Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux
Quoted 10 lines · Matt Domsch
Quoted 8 lines · Greg KH
Quoted 3 lines · Matt Domsch
The fundamental roadblock to this is that enumeration != naming, except that it is for network devices, and we keep changing the enumeration order.
No, the hardware changes the enumeration order, it places _no_ guarantees on what order stuff will be found in. So this is not the kernel changing, just to be clear.
Quoted 6 lines
Over time the kernel has changed its enumeration mechanisms, and introduced parallelism into the process (which is a good thing), which, from a user perspective, makes names nondeterministic. Yes, fixing this up by hard-coding MAC addresses after install has been the traditional mechanism to address this. I think there's a better way.
Ok, but that way can be done in userspace, without the need for this char device, right?
Quoted 29 lines
Quoted 4 lines
Quoted 3 lines
Today, port naming is completely nondeterministic. If you have but one NIC, there are few chances to get the name wrong (it'll be eth0). If you have >1 NIC, chances increase to get it wrong.
Quoted 8 lines
That is why all distros name network devices based on the only deterministic thing they have today, the MAC address. I still fail to see why you do not like this solution, it is honestly the only way to properly name network devices in a sane manner.
All distros also provide a way to easily rename the network devices, to place a specific name on a specific MAC address, so again, this should all be solved already.
It's not the only way, it introduces state where there's a desire for a stateless solution, it's useless for getting all the names right at initial OS install time, and it restricts us to a single "name" for a given device.
We can get additional information from BIOS. SMBIOS 2.6 (types 9 and
41) has the fields to let us get a "label" for an device at a given b/d/f. On my PowerEdge R610, I see "Embedded NIC 1" .. "Embedded NIC 4" for the 4 LOMs. These labels have a clear correlation to the labels on the back of the chassis at these ports. biosdevname can parse and report this. HP made a similar vendor-specific extension to SMBIOS for their platforms, which biosdevname also parses. Even if BIOS decides they need to renumber the busses on every boot, it can keep this table correct. (insert general mistrust of BIOS authors rant; that's not the point here.)
Quoted 2 lines
biosdevname can be used in udev rules to create multiple names for a given device. Rules such as:
Yes, if you want multiple ways to name a network device, then you need the char nodes. But without that, you can just pick "always use the biosdevname" type option from your distro setup screen and go with that. Then you have everything always working properly from the very beginning.
Quoted 7 lines
Quoted 2 lines
No matter how badly your BIOS teams mess up the PCI enumeration order :)
In my case, the BIOS for a given system always configures the ports the same way, and assigns b/d/f the same way. With no change in the BIOS or hardware, I still see the ports enumerated differently on each boot. :-(
Again, that's legal from a PCI standpoint :)
So you really want this for multiple ways to name the same network device. That's a choice the network developers are going to have to make, as to if that is going to be a legal thing to have happen or not.
But this code is not a requirement to "solve" the fact that network devices can show up in different order, that problem can be solved as long as the user picks a single way to name the devices, using tools that are already present today in distros.
thanks,
greg k-h
Quoted 16 lines · Greg KH
Quoted 8 lines · Matt Domsch
Quoted 7 lines · Greg KH
Quoted 3 lines · Matt Domsch
The fundamental roadblock to this is that enumeration != naming, except that it is for network devices, and we keep changing the enumeration order.
No, the hardware changes the enumeration order, it places _no_ guarantees on what order stuff will be found in. So this is not the kernel changing, just to be clear.
Quoted 6 lines
Over time the kernel has changed its enumeration mechanisms, and introduced parallelism into the process (which is a good thing), which, from a user perspective, makes names nondeterministic. Yes, fixing this up by hard-coding MAC addresses after install has been the traditional mechanism to address this. I think there's a better way.
Quoted 2 lines
Ok, but that way can be done in userspace, without the need for this char device, right?
For the record -- when I tried to send a patch that did exactly this (provided an option to use by-path persistence for network drivers), it was rejected because "that doesn't work for USB".
True, it doesn't. But by-mac (what we have today) doesn't work for replacing motherboards in a random home system (that can't override the MAC address in the BIOS), either.
So why not provide both alternatives?
As you say below, it's up to the network devs whether this should be allowed...
Quoted 8 lines
Quoted 2 lines
biosdevname can be used in udev rules to create multiple names for a given device. Rules such as:
Yes, if you want multiple ways to name a network device, then you need the char nodes. But without that, you can just pick "always use the biosdevname" type option from your distro setup screen and go with that. Then you have everything always working properly from the very beginning.
*If* biosdevname works on your system. It doesn't on mine: this SMBIOS extension doesn't exist. :-)
Quoted 4 lines
So you really want this for multiple ways to name the same network device. That's a choice the network developers are going to have to make, as to if that is going to be a legal thing to have happen or not.
Yes. So do I, actually (for what little that's worth)...
Quoted 4 lines
But this code is not a requirement to "solve" the fact that network devices can show up in different order, that problem can be solved as long as the user picks a single way to name the devices, using tools that are already present today in distros.
This code is not a requirement, no. But -- as you say -- it does provide a halfway-decent way to assign multiple names to a NIC. And that provides admins the choice to use a couple different persistence schemes, depending on how they expect their hardware to work.
(It *may* even be possible to use some kind of layer-2 traffic to see what else is on the connected network and provide symlinks based on that. IPv6 autoconfig type of thing, maybe. That's probably a *lot* more complicated, and may be impossible, but would be even closer to what I think Dell customers are asking for based on Matt's posts.)
Quoted 19 lines · Bryan Kadzban
Quoted 15 lines · Greg KH
Quoted 7 lines · Matt Domsch
Quoted 6 lines · Greg KH
Quoted 2 lines · Matt Domsch
The fundamental roadblock to this is that enumeration !> >>>> naming, except that it is for network devices, and we keep changing the enumeration order.
No, the hardware changes the enumeration order, it places _no_ guarantees on what order stuff will be found in. So this is not the kernel changing, just to be clear.
Quoted 6 lines
Over time the kernel has changed its enumeration mechanisms, and introduced parallelism into the process (which is a good thing), which, from a user perspective, makes names nondeterministic. Yes, fixing this up by hard-coding MAC addresses after install has been the traditional mechanism to address this. I think there's a better way.
Quoted 2 lines
Ok, but that way can be done in userspace, without the need for this char device, right?
Quoted 7 lines
For the record -- when I tried to send a patch that did exactly this (provided an option to use by-path persistence for network drivers), it was rejected because "that doesn't work for USB".
True, it doesn't. But by-mac (what we have today) doesn't work for replacing motherboards in a random home system (that can't override the MAC address in the BIOS), either.
If you replace a motherboard, you honestly expect no configuration to be needed to be changed? If so, then don't use the MAC naming scheme for your systems.
Quoted 9 lines
Quoted 4 lines
But this code is not a requirement to "solve" the fact that network devices can show up in different order, that problem can be solved as long as the user picks a single way to name the devices, using tools that are already present today in distros.
This code is not a requirement, no. But -- as you say -- it does provide a halfway-decent way to assign multiple names to a NIC. And that provides admins the choice to use a couple different persistence schemes, depending on how they expect their hardware to work.
But the names need to then be resolved back to a "real" kernel name in order to do anything with that network connection, as the char devices are not real ones. So that adds an additional layer of complexity on all of the system configuration tools.
thanks,
greg k-h
Quoted 23 lines · Greg KH
Quoted 22 lines · Bryan Kadzban
Quoted 9 lines · Greg KH
Quoted 8 lines · Matt Domsch
Quoted 7 lines · Greg KH
Quoted 3 lines · Matt Domsch
The fundamental roadblock to this is that enumeration != naming, except that it is for network devices, and we keep changing the enumeration order.
No, the hardware changes the enumeration order, it places _no_ guarantees on what order stuff will be found in. So this is not the kernel changing, just to be clear.
Quoted 8 lines
Quoted 6 lines
Over time the kernel has changed its enumeration mechanisms, and introduced parallelism into the process (which is a good thing), which, from a user perspective, makes names nondeterministic. Yes, fixing this up by hard-coding MAC addresses after install has been the traditional mechanism to address this. I think there's a better way.
Ok, but that way can be done in userspace, without the need for this char device, right?
For the record -- when I tried to send a patch that did exactly this (provided an option to use by-path persistence for network drivers), it was rejected because "that doesn't work for USB".
Quoted 7 lines
Quoted 3 lines
True, it doesn't. But by-mac (what we have today) doesn't work for replacing motherboards in a random home system (that can't override the MAC address in the BIOS), either.
If you replace a motherboard, you honestly expect no configuration to be needed to be changed? If so, then don't use the MAC naming scheme for your systems.
What else is there? biosdevname doesn't work with this BIOS. It looks like at least path_id has been updated to work with NICs now, so that might work, with a bit of custom rule hacking.
Or at least, it won't work any more poorly than for disks, which seem to work pretty well... :-)
Quoted 10 lines
Quoted 9 lines
Quoted 4 lines
But this code is not a requirement to "solve" the fact that network devices can show up in different order, that problem can be solved as long as the user picks a single way to name the devices, using tools that are already present today in distros.
This code is not a requirement, no. But -- as you say -- it does provide a halfway-decent way to assign multiple names to a NIC. And that provides admins the choice to use a couple different persistence schemes, depending on how they expect their hardware to work.
Quoted 4 lines
But the names need to then be resolved back to a "real" kernel name in order to do anything with that network connection, as the char devices are not real ones. So that adds an additional layer of complexity on all of the system configuration tools.
Yes, that is true -- and no, this change isn't perfect. But it lets me have multiple "names" per interface, and have "names" that are longer than IFNAMSIZ, though, which is why I like it.
(Now, if open() would return effectively a netlink socket bound to that ifindex already, such that the program didn't need to fill in the various ifindex fields for e.g. rtnetlink... but it's probably really hard to do that, so this isn't a serious suggestion.)
Quoted 4 lines · Bryan Kadzban
(Now, if
open()would return effectively a netlink socket bound to that ifindex already, such that the program didn't need to fill in the various ifindex fields for e.g. rtnetlink... but it's probably really hard to do that, so this isn't a serious suggestion.)
Wait, scratch that. It's not "really hard", it's "almost impossible".
At open() time, you have no idea which netlink family the program wants to communicate with. bind() is also hard. (In theory, you could support bind() on this new FD -- but then why is userspace using a file in the first place, and not a socket?)
So this is even less of a serious suggestion now.
I'd still like to be able to refer to NICs by multiple names though, if we can find a way that works...
Quoted 2 lines · Greg KH
Ok, but that way can be done in userspace, without the need for this char device, right?
It might actually be nice to have a device file anyway since you can use existing udev infrastructure to adjust permissions (e.g. chown it to the netdev group) and add ACLs. This would allow running some software as an unprivileged user instead of uid 0.
David
Quoted 8 lines · David Zeuthen
Quoted 2 lines · Greg KH
Ok, but that way can be done in userspace, without the need for this char device, right?
It might actually be nice to have a device file anyway since you can use existing udev infrastructure to adjust permissions (e.g. chown it to the netdev group) and add ACLs. This would allow running some software as an unprivileged user instead of uid 0.
But as the char nodes would not actually control access to anything, how would this help? Remember, these device nodes are "dummies" with nothing behind them (open() fails).
thanks,
greg k-h
Quoted 23 lines · Greg KH
Quoted 3 lines · Matt Domsch
The fundamental roadblock to this is that enumeration != naming, except that it is for network devices, and we keep changing the enumeration order.
No, the hardware changes the enumeration order, it places _no_ guarantees on what order stuff will be found in. So this is not the kernel changing, just to be clear.
Again, I have a machine here that likes to reorder PCI devices every 4th or so boot times, and that's fine according to the PCI spec. Yeah, it's a crappy BIOS, but the manufacturer rightly pointed out that it is not in violation of anything.
Quoted 3 lines
Today, port naming is completely nondeterministic. If you have but one NIC, there are few chances to get the name wrong (it'll be eth0). If you have >1 NIC, chances increase to get it wrong.
That is why all distros name network devices based on the only deterministic thing they have today, the MAC address. I still fail to see why you do not like this solution, it is honestly the only way to properly name network devices in a sane manner.
Quoted 5 lines
All distros also provide a way to easily rename the network devices, to place a specific name on a specific MAC address, so again, this should all be solved already.
No matter how badly your BIOS teams mess up the PCI enumeration order :)
No comment on the specific implementation decision, but I am in the process of setting up a large number of test systems with identical hardware configurations, and using a master disk image to clone all the test systems. The biggest pain in this process is identiying the MAC addresses for each of the six or more network interfaces in each test system (we want eth0...ethN to always reference the same physical port on the test systems), and then having to modify the 70-persistent-net.rules udev file and the HWADDR entry for all the ifcfg-ethX files to reflect the correct MAC addresses. It would be fantastic if there were some mechanism for making this part of the process unnecessary.
-Bill
Quoted 10 lines · Bill Fink
No comment on the specific implementation decision, but I am in the process of setting up a large number of test systems with identical hardware configurations, and using a master disk image to clone all the test systems. The biggest pain in this process is identiying the MAC addresses for each of the six or more network interfaces in each test system (we want eth0...ethN to always reference the same physical port on the test systems), and then having to modify the 70-persistent-net.rules udev file and the HWADDR entry for all the ifcfg-ethX files to reflect the correct MAC addresses. It would be fantastic if there were some mechanism for making this part of the process unnecessary.
Udev creates the persistent rules only if no other rule set a name. Adding something like:
SUBSYSTEM="net", KERNEL=""eth*", NAME="eth%n"
in any earlier rules file before the udev generated one will skip all off the automatic udev rule creation.
Kay
Quoted 29 lines · Greg KH
Quoted 3 lines · Matt Domsch
The fundamental roadblock to this is that enumeration != naming, except that it is for network devices, and we keep changing the enumeration order.
No, the hardware changes the enumeration order, it places _no_ guarantees on what order stuff will be found in. So this is not the kernel changing, just to be clear.
Again, I have a machine here that likes to reorder PCI devices every 4th or so boot times, and that's fine according to the PCI spec. Yeah, it's a crappy BIOS, but the manufacturer rightly pointed out that it is not in violation of anything.
Quoted 3 lines
Today, port naming is completely nondeterministic. If you have but one NIC, there are few chances to get the name wrong (it'll be eth0). If you have >1 NIC, chances increase to get it wrong.
That is why all distros name network devices based on the only deterministic thing they have today, the MAC address. I still fail to see why you do not like this solution, it is honestly the only way to properly name network devices in a sane manner.
All distros also provide a way to easily rename the network devices, to place a specific name on a specific MAC address, so again, this should all be solved already.
No matter how badly your BIOS teams mess up the PCI enumeration order :)
Quoted 8 lines
thanks,
greg k-h
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
So when an add-in PCI NIC has a lower MAC than the motherboard NICs,
the add-in cards will come before the motherboard NICs. i don't like it.
But please whatever is done, make sure ping and tracert still works when telling it to use a ethX source interface:
eth0 = 4.3.2.8, the default gateway is thru eth1.
ping -I eth0 208.67.222.222 FAILS
ping -I 4.3.2.8 208.67.222.222 WORKS
tracert -i eth0 -I 208.67.222.222 FAILS
tracert -s 4.3.2.8 -I 208.67.222.222 WORKS
tracert -i eth0 208.67.222.222 FAILS
tracert -s 4.3.2.8 208.67.222.222 WORKS
Quoted 2 lines · Rob Townley
So when an add-in PCI NIC has a lower MAC than the motherboard NICs,
the add-in cards will come before the motherboard NICs. i don't like it.
Actually, MAC address has nothing to do with device naming/ordering at all. Often systems will have onboard NICs in ascending MAC address order, but that's not a requirement, and I've seen systems not do that. And once you get to add-in vs onboard, BIOS wouldn't be able to enforce such an ordering anyhow (in general).
But yes, you raise the point that, without using MAC-assigned names or another naming mechanism designed to cope with this, adding or removing a card can cause a difference in device enumeration, and thus name.
Signature · 3 lines
Matt Domsch Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux
Quoted 2 lines · Rob Townley
So when an add-in PCI NIC has a lower MAC than the motherboard NICs,
the add-in cards will come before the motherboard NICs. i don't like it.
Huh? Have you used the MAC persistant rules? If you add a new card, what does it pick for it?
Quoted 10 lines
But please whatever is done, make sure ping and tracert still works when telling it to use a ethX source interface:
eth0 = 4.3.2.8, the default gateway is thru eth1.
ping -I eth0 208.67.222.222 FAILS
ping -I 4.3.2.8 208.67.222.222 WORKS
tracert -i eth0 -I 208.67.222.222 FAILS
tracert -s 4.3.2.8 -I 208.67.222.222 WORKS
tracert -i eth0 208.67.222.222 FAILS
tracert -s 4.3.2.8 208.67.222.222 WORKS
Again, is what we currently have broken? I am confused as to what this is referring to.
greg k-h
Quoted 6 lines · Greg KH
Quoted 2 lines · Rob Townley
So when an add-in PCI NIC has a lower MAC than the motherboard NICs, the add-in cards will come before the motherboard NICs. i don't like it.
Huh? Have you used the MAC persistant rules? If you add a new card, what does it pick for it?
i have a hp-dl360 (two nics) with a fibre optic add in nic. On a fresh install, the add-in is eth0. i didn't like it, but ran it for years.
Quoted 12 lines
Quoted 11 lines
But please whatever is done, make sure ping and tracert still works when telling it to use a ethX source interface:
eth0 = 4.3.2.8, the default gateway is thru eth1.
ping -I eth0 208.67.222.222 FAILS
ping -I 4.3.2.8 208.67.222.222 WORKS
tracert -i eth0 -I 208.67.222.222 FAILS
tracert -s 4.3.2.8 -I 208.67.222.222 WORKS
tracert -i eth0 208.67.222.222 FAILS
tracert -s 4.3.2.8 208.67.222.222 WORKS
Quoted 2 lines
Again, is what we currently have broken? I am confused as to what this is referring to.
Yes, ping and traceroute are broken at least on Fedora, CentOS, and busybox. On a multinic, multigatewayed machine, passing ethX instead of the IP address will give the false result: "Destination Host Unreachable"
when the machine's default gateway is reached thru the other nic. In
the following example, the default gateway is thru eth1, not eth0. Pay attention to the text between the '*****'.
ping -c 1 -B -I eth0 208.67.222.222
PING 208.67.222.222 (208.67.222.222) from ***** 4.3.2.8 eth0*****:
56(84) bytes of data.
From 4.3.2.8 icmp_seq=1 Destination Host Unreachable
#ping -c 1 -B -I 4.3.2.8 208.67.222.222
PING 208.67.222.222 (208.67.222.222) from ***** 4.3.2.8 *****: 56(84) bytes of data.
64 bytes from 208.67.222.222: icmp_seq=1 ttlU timeV2 ms
Quoted 3 lines
greg k-h
Quoted 8 lines · Rob Townley
Quoted 2 lines
Again, is what we currently have broken? I am confused as to what this is referring to.
Yes, ping and traceroute are broken at least on Fedora, CentOS, and busybox. On a multinic, multigatewayed machine, passing ethX instead of the IP address will give the false result: "Destination Host Unreachable"
when the machine's default gateway is reached thru the other nic. In
the following example, the default gateway is thru eth1, not eth0.
Unrelated to this thread. We're having a hard enough time making sure this conversation accurately reflects the views and needs of everyone involved. Please let's not throw in another tangent.
Thanks,
Matt
Signature · 3 lines
Matt Domsch Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux
Quoted 4 lines · Greg KH
Quoted 3 lines
Today, port naming is completely nondeterministic. If you have but one NIC, there are few chances to get the name wrong (it'll be eth0). If you have >1 NIC, chances increase to get it wrong.
Quoted 8 lines
That is why all distros name network devices based on the only deterministic thing they have today, the MAC address. I still fail to see why you do not like this solution, it is honestly the only way to properly name network devices in a sane manner.
All distros also provide a way to easily rename the network devices, to place a specific name on a specific MAC address, so again, this should all be solved already.
No, it's not solved. Even if you have persistent names once you install, if you ever re-image, you're likely to get *different* persistent names; the first load will always be non-detmerministic.
The only way around this would be to have some sort of screen like:
Would you like your network devices to be enumerated by
[ ] MAC address
[ ] PCI device order
[ ] Driver name
[ ] Other
which is just all sorts of fail in and of itself. Especially since once you get to the point where you can coherently ask this in a native installer, the drivers have already loaded.
Bill
Quoted 18 lines · Bill Nottingham
Quoted 4 lines · Greg KH
Quoted 3 lines
Today, port naming is completely nondeterministic. If you have but one NIC, there are few chances to get the name wrong (it'll be eth0). If you have >1 NIC, chances increase to get it wrong.
Quoted 8 lines
That is why all distros name network devices based on the only deterministic thing they have today, the MAC address. I still fail to see why you do not like this solution, it is honestly the only way to properly name network devices in a sane manner.
All distros also provide a way to easily rename the network devices, to place a specific name on a specific MAC address, so again, this should all be solved already.
No, it's not solved. Even if you have persistent names once you install, if you ever re-image, you're likely to get *different* persistent names; the first load will always be non-detmerministic.
Quoted 8 lines
The only way around this would be to have some sort of screen like:
Would you like your network devices to be enumerated by
[ ] MAC address
[ ] PCI device order
[ ] Driver name
[ ] Other
[ ] PCI slot name
That's one that modern systems are now reporting, and should solve Matt's problem as well, right?
Quoted 3 lines
which is just all sorts of fail in and of itself. Especially since once you get to the point where you can coherently ask this in a native installer, the drivers have already loaded.
No, the driver load order doesn't determine this, you need the drivers loaded first before you can rename anything :)
And I don't see how Matt's proposed patch helps resolve this type of issue any better than what we currently have today, do you?
thanks,
greg k-h
Quoted 13 lines · Greg KH
Quoted 4 lines
No, it's not solved. Even if you have persistent names once you install, if you ever re-image, you're likely to get *different* persistent names; the first load will always be non-detmerministic.
Quoted 8 lines
The only way around this would be to have some sort of screen like:
Would you like your network devices to be enumerated by
[ ] MAC address
[ ] PCI device order
[ ] Driver name
[ ] Other
Quoted 4 lines
[ ] PCI slot name
That's one that modern systems are now reporting, and should solve Matt's problem as well, right?
... maybe. On my laptop, the first 'slot' enumerated appears to be the cardbus bridge, before the on-board ethernet. And on the desktop next to me, the slot driver shows nothing.
Quoted 2 lines
And I don't see how Matt's proposed patch helps resolve this type of issue any better than what we currently have today, do you?
It allows multiple addressing schemes to be active at once, which can allow the admin to choose post-install without making an active choice at installation. This is an improvement, even if it doesn't solve the world.
Bill
Quoted 14 lines · Bill Nottingham
Quoted 13 lines · Greg KH
Quoted 12 lines
No, it's not solved. Even if you have persistent names once you install, if you ever re-image, you're likely to get *different* persistent names; the first load will always be non-detmerministic.
The only way around this would be to have some sort of screen like:
Would you like your network devices to be enumerated by
[ ] MAC address
[ ] PCI device order
[ ] Driver name
[ ] Other
Quoted 8 lines
Quoted 4 lines
[ ] PCI slot name
That's one that modern systems are now reporting, and should solve Matt's problem as well, right?
... maybe. On my laptop, the first 'slot' enumerated appears to be the cardbus bridge, before the on-board ethernet. And on the desktop next to me, the slot driver shows nothing.
On servers, where this matters (multiple ethernet pci devices), this should all be present if the manufacturer wants it to be, as it is just an ACPI table entry.
Quoted 7 lines
Quoted 2 lines
And I don't see how Matt's proposed patch helps resolve this type of issue any better than what we currently have today, do you?
It allows multiple addressing schemes to be active at once, which can allow the admin to choose post-install without making an active choice at installation. This is an improvement, even if it doesn't solve the world.
But these different names can not be used by the networking stack, or in scripts, as others have pointed out. Which seems to be the big problem here.
thanks,
greg k-h
Quoted 165 lines · On Fri, 9 Oct 2009 23:40:57 -0500 Matt Domsch <[email protected]> wrote:
Quoted 6 lines · Stephen Hemminger
Maybe I'm dense but can't see why having a useless
/dev/net/symlinks is a good interface choice. Perhaps you should explain the race between PCI scan and udev in more detail, and why solving it in either of those places won't work. As it stands you are proposing yet another wart to the already complex set of network interface API's which has implications for security as well as increasing the number of possible bugs.The fundamental challenge is that system administrators, particularly those of server-class hardware with multiple network ports present (some on the motherboard, some on add-in cards), have the not-so-unreasonable expectation that there is a deterministic mapping between those ports and the name one uses to address those ports.
The fundamental roadblock to this is that enumeration != naming, except that it is for network devices, and we keep changing the enumeration order.
Today, port naming is completely nondeterministic. If you have but one NIC, there are few chances to get the name wrong (it'll be eth0). If you have >1 NIC, chances increase to get it wrong.
The complexity arises at multiple levels.
First, device driver load order. In the 2.4 kernel days, and even mostly early 2.6 kernel days, the order in which network drivers loaded played a role in determining the name of the device. Drivers loaded first would get their devices named first. If I have two types of devices, say an e100-driven NIC and a tg3-driven NIC, I could figure out that the names would be eth0á00 and eth1=tg3 by setting the load order in
/etc/modules.conf(now modprobe.conf). If I wanted the other order, fine, just switch it around in modules.conf and reboot. OS installers, being the first running instance of Linux, before modprobe.conf existed to set that ordering, had to have other mechanisms to load drivers (often manually, or if programmatically such as in a kickstart or autoyast file, was still somewhat fixed).With the advent of modaliases + udev, now modprobe.conf doesn't contain this ordering anymore, and udev loads the drivers. So while it wasn't perfect, it was better than nothing, and that's gone now.
It gets even worse as, to speed up boot time, modprobes can be run in parallel, and even within individual drivers, the NICs get initialized (and named) in parallel. Further confusing things, some devices need firmware loaded into them before getting names assigned, which is done from userspace, and they race.
Second, PCI device list order. In the 2.4 kernel days, the PCI device list was scanned "breadth-first" (for each bus; for each device; for each function; do load...). FWIW, Windows still does this. It gives BIOS, which assigns PCI bus numbers, a chance to put LOMs at a lower bus number than add-in cards. Module load order still mattered, but at least if you had say 2 e1000 ports as LOMs, and 2 e1000 ports on add-in cards, you pretty much knew the ordering would be eth0 as lowest bdf on the motherboard, eth1 as next bdf on the motherboard, and eth2 and 3 as the add-in cards in ascending slot order.
With the advent of PCI hot plug in the 2.5 kernel series, the
breadth-first ordering became depth-first. (for each bus; for each
device; if the device is a bridge, scan the busses behind it.). This caused NICs on bus 0 device 5, and bus 1 device 3, (eth0 and 1 respectively) to be enumerated differently due to the a bridge from bus 0 to bus 1 at 0:4. My crude hack of pci¿sort, with some dmi strings to match and auto-enable, at least reverted this back to the ordering the 2.4 kernel and Windows used. Now we have to keep adding systems to this DMI list (Dell has a number of systems on this list today; HP has even more). And it doesn't completely solve the problem, just masks it.So, to address the ordering problem, I placed a constraint on our server hardware teams, forcing them to lay out their boards and assign PCIe lanes and bus numbers, such that at least the designed "first" LOM would get found first in either depth-first or breadth-first order. Our 10G and 11G servers have this restriction in place, though it wasn't easy. And it's gotten even harder, as the PCIe switches expand the number of lanes available. We no longer have the traditional tiered buses architecture, but the PCI layer for this purpose thinks we do. I need to remove this constraint on the hardware teams - it's gotten to be impossible for the chipset lanes to be laid out efficiently with this constraint.
All of the above just papered over the enumeration != naming problem.
Third, stateless computing is becoming more and more commonplace. The Field Replaceable Unit is the server itself. Got a bad server? Pull it out, move the disks to an identical unit, insert the new server, and go. Fix the bad server offline and bring it back. In this model, having MAC addresses as the mechanism that is providing the determinism (
/etc/mactabor udev persistent naming rules) breaks, because the MAC addresses of the ports on the new server won't be the same as on the old server. HP even has a technology to solve _this_ problem (in their blade chassis) - Virtual Connect. The MACs get assigned by the chassis to the blades at POST, and are fixed to the slot. Slick, and Dell has an even more flexible similar feature FlexAddress. This doesn't solve the OS installer problem of "which of these NICs should I use to do an install?" but it does recognize the problem space and tries to overcome it.Fourth, for OS installers, choosing which NIC to use at installtime, when all the NICs are plugged in, can be difficult. PXE environments, using pxelinux and its IPAPPEND 2 option, will append "BOOTIF=xx:xx:xx:xx:xx:xx" to the kernel command line, that containing the MAC address of the NIC used for PXE. Neat trick. Yes, we then had to teach the OS installers to recognize and use this. But it only works if you PXE boot, and only for that one NIC.
Fifth, network devices can have only a single name. eth0. If we look at disks, we see udev manages a tree of symlinks for
/dev/disk/by-label,/dev/disk/by-path,/dev/disk/by-uuid. And as a system admin, if I wanted to also create a udev rule for/dev/disk/by-function(boot, swap, mattsstorage), it's trivial to do so. Why can't we have this flexibility for network devices too?So, how do we get deterministic naming for all the NICs in a system? That's what I'm going for. Picture a network switch, with several blades, and several ports on each blade. The network admin addresses each port as say 1/16 (the 16th port on blade 1, clearly labeled). The parallel on servers is the chassis label printed on the outside (say, "Gb1"). But due the above, there is no guarantee, and in fact little chance, that Gb1 will be consistently named eth0 - it may vary from boot to boot. That's full of fail.
For a concrete example, the 4 bnx2 chips in my PowerEdge R610 with a current 2.6 kernel, loading only one driver, the ports get assigned names in nondeterministic order on each boot. Given that the ifcfg-eth* rules, netfilter rules, and the rest all expect deterministic naming, massive failure ensues unless some form of determinism is brought back in.
The idea to use a character device node to expose the ifindex value, and udev to manage a tree of symlinks to it, really follows the model used today for disks. It allows us to get deterministic names for devices (albeit, the names are symlinks), and multiple names for devices (through multiple symlink rules). That some people want to use the char device to call
ioctl()and read/write, as is possible on the BSDs, would just be gravy IMHO.It does require a change in behavior for a system administrator. Instead of hard-coding 'eth0' into her scripts, she uses '
/dev/net/by-function/boot' or somesuch. But then that name is guaranteed to always refer to the "right" NIC. Every admin I've spoken to is willing to make this kind of change, as long as they get the consistent, deterministic naming they expect but don't have today. And it does require patching userspace apps to take both a kernel device name, or a path, and to resolve the path to device name or ifindex. We wrote libnetdevname (really, one function), and have patches for several userspace apps to use it, to prove it can be done.One alternative would be to do something using the sysfs ifindex value already exported. e.g.
/sys/devices/pci0000:00/0000:00:05.0/0000:05:00.0/0000:06:07.0/net/eth0/ifindexbut we have never had symlinks from /dev into /sys before (doesn't mean we couldn't though). In that case, udev would grow to manage
/dev/net/by-chassis-label/Embedded_NIC_1->/sys/devices/.../net/eth0, and libnetdevname would be used to follow the symlink in applications. This approach could solve my problem without (many or any?) kernel changes needed, but wouldn't help those who want to do ioctl/read/write to a devnode.Given the problem, I really do need a solution. I've proposed one method, and an alternative, but I can't afford to let the problem stay unaddressed any longer, and need a clear direction to be chosen. The char device gives me what I need, and others what they want also.
Quoted 6 lines
Thanks for listening to the diatribe. For more examples and workarounds that we've been telling our customers for several years, check out http://linux.dell.com/papers.shtml for the Network Interface Card Naming whitepaper.
Why isn't the available through sysfs enough, if not why not add the necessary attributes there.
BTW, for our distro, we are looking into device renaming based on PCI slot because that is what router OS's do. Customers expect if they replace the card in slot 0, it will come back with the same name. This is not what server customers expect.
Quoted 5 lines · Stephen Hemminger
BTW, for our distro, we are looking into device renaming based on PCI slot because that is what router OS's do. Customers expect if they replace the card in slot 0, it will come back with the same name. This is not what server customers expect.
If your bios exposes the PCI slots to userspace (through the proper ACPI namespace), doing this type of naming should be trivial with some simple udev rules, no additional kernel infrastructure is needed.
thanks,
greg k-h
Quoted 10 lines · Greg KH
Quoted 5 lines · Stephen Hemminger
BTW, for our distro, we are looking into device renaming based on PCI slot because that is what router OS's do. Customers expect if they replace the card in slot 0, it will come back with the same name. This is not what server customers expect.
If your bios exposes the PCI slots to userspace (through the proper ACPI namespace), doing this type of naming should be trivial with some simple udev rules, no additional kernel infrastructure is needed.
By and large, the people that care most about persistent network device names based on *location in the machine* are server users. This allows hotswap of cards or single-image-multiple-machine without needing configuration changes, which is nice.
Those users can reasonably be expected to choose hardware whose BIOS supports the ACPI tables that (mostly) guarantee to provide actual, stable names for their hardware. If there's even a 10% chance that on consumer-level systems the names won't be stable on a given boot (and I can't see how, without BIOS support, we can guarantee 100% stability) then it's a worthless guarantee.
If the BIOS support exists, it is trivial to use udev to create the correct naming mechanism for your machine, either using MAC address or BIOS-provided slot naming. No kernel patch is required.
If the BIOS support does not exist, you are not guaranteed a stable naming mechanism except by MAC address, because the BIOS may randomly change enumeration based on the time of day, or it may not. A 90 or 95% stability guarantee is not a guarantee at all.
Third, USB enumeration will always be unstable. Thus we have an unsolvable discrepancy in behavior between PCI and USB.
Is this correct?
Dan
Quoted 5 lines
If the BIOS support exists, it is trivial to use udev to create the correct naming mechanism for your machine, either using MAC address or BIOS-provided slot naming. No kernel patch is required.
Yes. In case, we want to rename only once. MAC address or slot names do provide persistent naming. They help in retaining whatever names are assigned during install time, which is the first instantiation of the OS. But these names may not be as expected (like first on board network interface name is expected to be "eth0" which is not always the case and might not reflect what is written on the chassis label as "Gb1" and "Gb2" etc) which would result in unattended installs break. Also image based deployments will face problems by introducing state such as MAC address.
With regards,
Narendra K
Quoted 1 line · Stephen Hemminger
Quoted 1 line · Stephen Hemminger
[...]
Quoted 3 lines
Why isn't the available through sysfs enough, if not why not add the necessary attributes there.
True. If sysfs is not sufficient, what exact naming scheme could be applied that the chardev based naming could use?
Quoted 1 line
[...]
Kurt
Quoted 10 lines · Matt Domsch
It does require a change in behavior for a system administrator. Instead of hard-coding 'eth0' into her scripts, she uses '
/dev/net/by-function/boot' or somesuch. But then that name is guaranteed to always refer to the "right" NIC. Every admin I've spoken to is willing to make this kind of change, as long as they get the consistent, deterministic naming they expect but don't have today. And it does require patching userspace apps to take both a kernel device name, or a path, and to resolve the path to device name or ifindex. We wrote libnetdevname (really, one function), and have patches for several userspace apps to use it, to prove it can be done.
For the records, before being a distribution developer I am a system administrator (who designed and manages many firewalls with multiple network interfaces) and I am still unconvinced that what you are proposing is a practical solution and that its downsides justify the significant changes both in software and in system administration practices that it requires.
The first issue which greatly concerns me is the need to modify *every* userspace application and kernel tool (what about iptables? What about the kernel logs?): from an users experience point of view it would be very annoying if different applications used different names to refer to the same network device.
I am also concerned with the practical implications of trying to use such long (and unusual) names: IFNAMSIZ is 16, so user interfaces tend to assume both short names and that they match something like /^[a-z0-9]+$/. What about e.g. distribution scripts which use the interface name as a file system path component? Do you already have a (standard) scheme to losslessly convert the names to a form without slashes?
Signature · 2 lines
ciao, Marco
Quoted 16 lines · Narendra K
Quoted 8 lines · Narendra
Quoted 2 lines
example udev config:
SUBSYSTEM="net",SYMLINK+="net/by-mac/$sysfs{ifindex}.$sysfs{address}"
work as well. But coupling the ifindex to the MAC address like this doesn't work. (In general, coupling any two unrelated attributes when trying to do persistent names doesn't work.)
Attaching the latest patch incorporating review comments.
By creating character devices for every network device, we can use udev to maintain alternate naming policies for devices, including additional names for the same device, without interfering with the name that the kernel assigns a device.
Quoted 7 lines
This is conditionalized on
CONFIG_NET_CDEV. If enabled (the default), device nodes will automatically be created in/dev/netdev/for each network device. (/dev/net/is already populated by the tun device.)These device nodes are not functional at the moment -
open()returns-ENOSYS. Their only purpose is to provide userspace with a kernel name to ifindex mapping, in a form that udev can easily manage.
If the idea is just to provide a userspace-visible mapping (and presumably take advantage of udev's infrastructure for naming) does this need kernel changes? Could this be a hierarchy under e.g. /etc/udev instead, using plain text files? It still means we need something like libnetdevname for apps to do the translation, but I'm not seeing why it matters how this map is stored. Is there some special property of the character devices (e.g. uevents) that we're not already getting with the existing interfaces?
Signature · 1 line
dann frazier
Quoted 5 lines
Quoted 4 lines
These device nodes are not functional at the moment -
open()returns-ENOSYS. Their only purpose is to provide userspace with a kernel name to ifindex mapping, in a form that udev can easily manage.
Quoted 9 lines
If the idea is just to provide a userspace-visible mapping (and presumably take advantage of udev's infrastructure for naming) does this need kernel changes? Could this be a hierarchy under e.g.
/etc/udevinstead, using plain text files? It still means we need something like libnetdevname for apps to do the translation, but I'm not seeing why it matters how this map is stored. Is there some special property of the character devices (e.g. uevents) that we're not already getting with the existing interfaces?
Yes. The char device by itself doesn't help in any way. But it provides a flexible mechanism to provide multiple names for the same device, just the way it is for disks.
With regards,
Narendra K
Quoted 15 lines · 1;2202;0cOn Tue, Oct 13, 2009 at 10:43:49PM +0530, [email protected] wrote:
Quoted 5 lines
Quoted 4 lines
These device nodes are not functional at the moment -
open()returns-ENOSYS. Their only purpose is to provide userspace with a kernel name to ifindex mapping, in a form that udev can easily manage.Quoted 9 lines
If the idea is just to provide a userspace-visible mapping (and presumably take advantage of udev's infrastructure for naming) does this need kernel changes? Could this be a hierarchy under e.g.
/etc/udevinstead, using plain text files? It still means we need something like libnetdevname for apps to do the translation, but I'm not seeing why it matters how this map is stored. Is there some special property of the character devices (e.g. uevents) that we're not already getting with the existing interfaces?
Quoted 3 lines
Yes. The char device by itself doesn't help in any way. But it provides a flexible mechanism to provide multiple names for the same device, just the way it is for disks.
Right - so any reason this couldn't be implemented completely in userspace by having udev manipulate plain text files under say /etc/udev/net/?
I do agree that it would be nice for admins/installers to tweak/use nic names in a similar way to storage names (udev rules), and it might let us take advantage of a lot of the existing udev code.
Signature · 1 line
dann frazier
Quoted 20 lines · dann frazier
Quoted 15 lines · 1;2202;0cOn Tue, Oct 13, 2009 at 10:43:49PM +0530, [email protected] wrote:
Quoted 5 lines
Quoted 4 lines
These device nodes are not functional at the moment -
open()returns-ENOSYS. Their only purpose is to provide userspace with a kernel name to ifindex mapping, in a form that udev can easily manage.Quoted 9 lines
If the idea is just to provide a userspace-visible mapping (and presumably take advantage of udev's infrastructure for naming) does this need kernel changes? Could this be a hierarchy under e.g.
/etc/udevinstead, using plain text files? It still means we need something like libnetdevname for apps to do the translation, but I'm not seeing why it matters how this map is stored. Is there some special property of the character devices (e.g. uevents) that we're not already getting with the existing interfaces?Quoted 3 lines
Yes. The char device by itself doesn't help in any way. But it provides a flexible mechanism to provide multiple names for the same device, just the way it is for disks.
Quoted 7 lines
Right - so any reason this couldn't be implemented completely in userspace by having udev manipulate plain text files under say
/etc/udev/net/?I do agree that it would be nice for admins/installers to tweak/use nic names in a similar way to storage names (udev rules), and it might let us take advantage of a lot of the existing udev code.
Is there interest in this approach?
- modify udev to manage network devices names as regular (non-device) files (stored in /etc/udev, /dev/netdev, or wherever)
- use the existing udev rules to manage symlinks to these files
- point libnetdevname at these text files for its name resolution
I've started prototyping this, and it certainly looks possible w/o any kernel changes. However, I could probably use some advice from a udev person to do a proper implementation.
Signature · 1 line
dann frazier
Quoted 6 lines · dann frazier
Quoted 4 lines
Right - so any reason this couldn't be implemented completely in userspace by having udev manipulate plain text files under say
/etc/udev/net/?
Quoted 8 lines
Quoted 2 lines
I do agree that it would be nice for admins/installers to tweak/use nic names in a similar way to storage names (udev rules),
and it might
Quoted 1 line
let us take advantage of a lot of the existing udev code.
Is there interest in this approach?
- modify udev to manage network devices names as regular (non-device) files (stored in/etc/udev,/dev/netdev, or wherever)
Yes. Would you elaborate little more on "modify udev to manage network devices as regular files". Does it mean some custom rules which will generate a regular file under, say, /dev/netdev/ or extend udev itself ? And how would the regular file look like in terms of holding ifindex of the interface, which can be passed to libnetdevname.
Quoted 6 lines
- use the existing udev rules to manage symlinks to these files
- point libnetdevname at these text files for its name resolutionI've started prototyping this, and it certainly looks possible w/o any kernel changes. However, I could probably use some advice from a udev person to do a proper implementation.
With regards,
Narendra K
Quoted 9 lines · On Fri, Oct 16, 2009 at 07:32:50PM +0530, [email protected] wrote:
Quoted 9 lines · dann frazier
Quoted 6 lines
Right - so any reason this couldn't be implemented completely in userspace by having udev manipulate plain text files under say
/etc/udev/net/?I do agree that it would be nice for admins/installers to tweak/use nic names in a similar way to storage names (udev rules),
and it might
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
let us take advantage of a lot of the existing udev code.
Is there interest in this approach?
- modify udev to manage network devices names as regular (non-device) files (stored in/etc/udev,/dev/netdev, or wherever)Yes. Would you elaborate little more on "modify udev to manage network devices as regular files".
Sure. We already get an event when netifs get added/removed - udev just doesn't create a device file for it. And since all we care about is the file's name (and the symlinks to it), there's really no point in creating a real device file anyway.
So, instead of 'mknod /dev/netdev/eth0', why not just 'touch /dev/netdev/eth0'? A file exists, so we can still maintain aliases as symlinks, we just don't need to modify the kernel.
Quoted 3 lines
Does it mean some custom rules which will
generate a regular file under, say,/dev/netdev/or extend udev itself ?
I believe we have to extend udev itself. We could probably do this completely within udev rules by running programs that do the touching and symlinking, but it would be nicer and more consistent/familiar to take advantage of the udev syntax (SYMLINK) to do this natively. Besides, udev already has the logic to know when/how to instantiate and unlink symlinks, it would suck to duplicate that.
So, udev would need to be modified to know how to go through the normal "node" creation for net devices, and to call creat() instead of mknod().
Quoted 2 lines
And how would the regular file look like in terms of holding ifindex of the interface, which can be passed to libnetdevname.
I can't think of anything we need to store in the regular file. If we have the kernel name for the device, we can look up the ifindex in /sys. Correct me if I'm wrong, but storing it ourselves seems redundant.
Quoted 11 lines
Quoted 8 lines
- use the existing udev rules to manage symlinks to these files
- point libnetdevname at these text files for its name resolutionI've started prototyping this, and it certainly looks possible w/o any kernel changes. However, I could probably use some advice from a udev person to do a proper implementation.
With regards,
Narendra K
Signature · 1 line
dann frazier
Quoted 1 line · dann frazier
On Fri, Oct 16, 2009 at 07:32:50PM +0530, [email protected] wrote:
[...]
Quoted 7 lines
Quoted 2 lines
And how would the regular file look like in terms of holding ifindex of the interface, which can be passed to libnetdevname.
I can't think of anything we need to store in the regular file. If we have the kernel name for the device, we can look up the ifindex in /sys. Correct me if I'm wrong, but storing it ourselves seems redundant.
But the name of a netdev can change whereas its ifindex never does. Identifying netdevs by name would require additional work to update the links when a netdev is renamed and would still be prone to race conditions. This is why Narendra and Matt were proposing to store the ifindex in the node all along...
Ben.
Signature · 3 lines
Ben Hutchings, Senior Software Engineer, Solarflare Communications Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
Quoted 11 lines · Ben Hutchings
Quoted 1 line · dann frazier
On Fri, Oct 16, 2009 at 07:32:50PM +0530, [email protected] wrote:
[...]
Quoted 7 lines
Quoted 2 lines
And how would the regular file look like in terms of holding ifindex of the interface, which can be passed to libnetdevname.
I can't think of anything we need to store in the regular file. If we have the kernel name for the device, we can look up the ifindex in /sys. Correct me if I'm wrong, but storing it ourselves seems redundant.
Quoted 5 lines
But the name of a netdev can change whereas its ifindex never does. Identifying netdevs by name would require additional work to update the links when a netdev is renamed and would still be prone to race conditions. This is why Narendra and Matt were proposing to store the ifindex in the node all along...
ah, yes - I see that now - the ability to rename an interface is what prevents this from working. Thanks for the explanation.
Signature · 1 line
dann frazier
Quoted 11 lines · Ben Hutchings
Quoted 1 line · dann frazier
On Fri, Oct 16, 2009 at 07:32:50PM +0530, [email protected] wrote:
[...]
Quoted 7 lines
Quoted 2 lines
And how would the regular file look like in terms of holding ifindex of the interface, which can be passed to libnetdevname.
I can't think of anything we need to store in the regular file. If we have the kernel name for the device, we can look up the ifindex in /sys. Correct me if I'm wrong, but storing it ourselves seems redundant.
Quoted 5 lines
But the name of a netdev can change whereas its ifindex never does. Identifying netdevs by name would require additional work to update the links when a netdev is renamed and would still be prone to race conditions. This is why Narendra and Matt were proposing to store the ifindex in the node all along...
Matt, Ben and I talked about a few other possibilities on IRC. The one I like the most at the moment is an idea Ben had to creat dummy files named after the ifindex. Then, use symlinks for the kernel name and the various by-$property subdirectories. This means the KOBJ events will need to expose the ifindex.
I'm a novice at net programming, but I'm told that ifindex is the information apps ultimately require here.
Signature · 1 line
dann frazier
Quoted 14 lines
Quoted 9 lines
Quoted 8 lines
Quoted 3 lines
And how would the regular file look like in terms of holding ifindex of the interface, which can be passed to libnetdevname.
I can't think of anything we need to store in the regular file. If we have the kernel name for the device, we can look up the ifindex in /sys. Correct me if I'm wrong, but storing it ourselves seems redundant.
Quoted 4 lines
But the name of a netdev can change whereas its ifindex never does. Identifying netdevs by name would require additional work to update the links when a netdev is renamed and would still be prone to race conditions. This is why Narendra and Matt were proposing to
store the
Quoted 9 lines
Quoted 1 line
ifindex in the node all along...
Matt, Ben and I talked about a few other possibilities on IRC. The one I like the most at the moment is an idea Ben had to creat dummy files named after the ifindex. Then, use symlinks for the kernel name and the various by-$property subdirectories. This means the KOBJ events will need to expose the ifindex.
I suppose the KOBJ events already expose the ifindex of a network interface. The file "/sys/class/net/ethN/uevent" contains INTERFACE=ethN and IFINDEX=n already. But it looks like udev doesn't use it in any way. For example, with the kernel patch the "/sys/class/net/ethN/uevent" contains in addition to the above details, MAJOR=M and MINOR=m which the udev knows how to make use of with a rule like
SUBSYSTEM=="net", KERNEL!="tun", NAME="netdev/%k", MODE="0600".
Quoted 2 lines
I'm a novice at net programming, but I'm told that ifindex is the information apps ultimately require here.
Yes. The minor number of the device node is retreived by libnetdevname by "stat"ing the pathname which happens to be ifindex of the device and it is mapped to corresponding kernel name by "if_indextoname" call.
With regards,
Narendra K
Quoted 21 lines · [email protected] wrote:
Quoted 12 lines
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
And how would the regular file look like in terms of holding
ifindex of the interface, which can be passed to
libnetdevname.I can't think of anything we need to store in the regular file. If we have the kernel name for the device, we can look up the ifindex in /sys. Correct me if I'm wrong, but storing it ourselves seems redundant.
Quoted 5 lines
But the name of a netdev can change whereas its ifindex never does. Identifying netdevs by name would require additional work to update the links when a netdev is renamed and would still be prone to race conditions. This is why Narendra and Matt were proposing to
Quoted 8 lines
store the
Quoted 1 line
ifindex in the node all along...
Matt, Ben and I talked about a few other possibilities on IRC. The one I like the most at the moment is an idea Ben had to creat dummy files named after the ifindex. Then, use symlinks for the kernel name and the various by-$property subdirectories. This means the KOBJ events will need to expose the ifindex.
Quoted 4 lines
I suppose the KOBJ events already expose the ifindex of a network interface. The file "
/sys/class/net/ethN/uevent" contains INTERFACE=ethN and IFINDEX=n already. But it looks like udev doesn't use it in any way.
Right; it could simply do the equivalent of:
touch /dev/netdev/$env{IFINDEX}
instead of its normal mknod(2), and then do normal SYMLINK processing. That last part is what would link /dev/netdev/by-name/$env{INTERFACE} to that device, along with /dev/netdev/by-mac/*, /dev/netdev/by-path/*, etc., etc., in as many different ways as people want to add rules.
(Or /dev/net/by-* instead of netdev; I'm mostly ambivalent about the first-level directory under /dev. Looks like libnetdevname requires /dev/netdev though.)
Quoted 5 lines
For example, with the kernel patch the "
/sys/class/net/ethN/uevent" contains in addition to the above details, MAJOR=M and MINOR=m which the udev knows how to make use of with a rule likeSUBSYSTEM=="net", KERNEL!="tun", NAME="netdev/%k", MODE="0600".
And if the only point is to get the ifindex via stat(2) on the resulting symlinks, but people don't like device files, then why not get the ifindex via readlink(2) (and a bit of string parsing, and a strtol(3) or strtoul(3) call) instead? :-)
Quoted 32 lines
Quoted 4 lines
Quoted 4 lines
Matt, Ben and I talked about a few other possibilities on IRC. The one I like the most at the moment is an idea Ben had to creat dummy files named after the ifindex. Then, use symlinks for the
kernel name
Quoted 8 lines
Quoted 3 lines
and the various by-$property subdirectories. This means the KOBJ events will need to expose the ifindex.
I suppose the KOBJ events already expose the ifindex of a network interface. The file "
/sys/class/net/ethN/uevent" contains INTERFACE=ethN and IFINDEX=n already. But it looks like udev doesn't use it in any way.Right; it could simply do the equivalent of:
touch
/dev/netdev/$env{IFINDEX}instead of its normal mknod(2), and then do normal SYMLINK processing. That last part is what would link
/dev/netdev/by-name/$env{INTERFACE}to that device, along with/dev/netdev/by-mac/*,/dev/netdev/by-path/*, etc., etc., in as many different ways as people want to add rules.(Or
/dev/net/by-*instead of netdev; I'm mostly ambivalent about the first-level directory under /dev. Looks like libnetdevname requires/dev/netdevthough.)Quoted 4 lines
For example, with the kernel patch the "
/sys/class/net/ethN/uevent" contains in addition to the above details, MAJOR=M and MINOR=m which the udev knows how to make use of with a rule like
Quoted 7 lines
Quoted 1 line
SUBSYSTEM=="net", KERNEL!="tun", NAME="netdev/%k", MODE="0600".
And if the only point is to get the ifindex via stat(2) on the resulting symlinks, but people don't like device files, then why not get the ifindex via readlink(2) (and a bit of string parsing, and a strtol(3) or
strtoul(3) call) instead? :-)
I suppose this issue can also be addressed in another way. Currently, the sysfs contains various attributes of a network interface under the directory "/sys/class/net/ethN", for example "/sys/class/net/ethN/address". This will be used by udev as below -
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:1d:09:6a:78:ec", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1".
Similarly, export an attribute named "smbios_name" to sysfs, i.e "/sys/class/net/eth0/smbios_name". "Cat /sys/class/net/eth0/smbios_name" would show "Embedded_NIC_1[23..]" and this can be used by udev in 70-persistent-net.rules as
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{smbios_name}=="Embedded_NIC_1", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0".
I suppose this would not need any changes to the udev code and existing udev infrastructure can be used as udev is capable handling ATTR{something}.
This would also ensure that whichever device is "Embedded_NIC_1" as per the BIOS, will also be "eth0" in the os.
Netdev, What are your views on this idea ?
With regards,
Narendra K
Quoted 3 lines · On Nov 04, [email protected] wrote:
SUBSYSTEM="net", ACTION="add", DRIVERS="?*", ATTR{smbios_name}="Embedded_NIC_1", ATTR{type}="1", KERNEL="eth*", NAME="eth0".
As a distribution developer I highly value solutions like this which do not require patching every application which deals with interface names and then teaching users about aliases which only work in some places and are unknown to the kernel.
Signature · 2 lines
ciao, Marco
Quoted 9 lines · Marco d'Itri
Quoted 3 lines · On Nov 04, [email protected] wrote:
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{smbios_name}=="Embedded_NIC_1", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0".
As a distribution developer I highly value solutions like this which do not require patching every application which deals with interface names and then teaching users about aliases which only work in some places and are unknown to the kernel.
Fair enough - but would you object if we changed the naming scheme from eth%d to something else?
Signature · 3 lines
Matt Domsch Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux
Quoted 6 lines · Matt Domsch
Quoted 4 lines
As a distribution developer I highly value solutions like this which do not require patching every application which deals with interface names and then teaching users about aliases which only work in some places and are unknown to the kernel.
Fair enough - but would you object if we changed the naming scheme from eth%d to something else?
I suppose that this would depend on what else. :-)
Since you want radical changes I recommend that you design the new persistent naming infrastructure in a way that will allow root to choose to use the classic naming scheme, or many users will scream a lot and at least some distributions will do it anyway.
I also expect that providing choice at the beginning of development may lead to more acceptance later if and when the new scheme will have proved itself to be superior (at least in some situations). You have tought about this for a long time and if so far you have not found a solution which is widely considered superior then I doubt that one will appear soon. Providing your favourite naming scheme as an optional add on will immediately benefit those who like it and greatly reduce opposition from those who do not.
Signature · 2 lines
ciao, Marco
Quoted 8 lines · Marco d'Itri
Quoted 6 lines · Matt Domsch
Quoted 4 lines
As a distribution developer I highly value solutions like this which do not require patching every application which deals with interface names and then teaching users about aliases which only work in some places and are unknown to the kernel.
Fair enough - but would you object if we changed the naming scheme from eth%d to something else?
Quoted 13 lines
I suppose that this would depend on what else. :-)
Since you want radical changes I recommend that you design the new persistent naming infrastructure in a way that will allow root to choose to use the classic naming scheme, or many users will scream a lot and at least some distributions will do it anyway.
I also expect that providing choice at the beginning of development may lead to more acceptance later if and when the new scheme will have proved itself to be superior (at least in some situations). You have tought about this for a long time and if so far you have not found a solution which is widely considered superior then I doubt that one will appear soon. Providing your favourite naming scheme as an optional add on will immediately benefit those who like it and greatly reduce opposition from those who do not.
This seems to me like a good installer feature - give the user an option to enter a name for an interface, with the default option to use the eth* names. To illustrate by example, I imagine an installer flow that looks like this:
[Do Hardware Discovery]
[Automatically reorder kernel names for reasonable defaults;
eth0-eth{n-1} map to n onboard nics]
Sample user interface for network configuration:
------------Choose an interface to configure --------------
| Multiple unconfigured interfaces detected. | | Select an interface to configure by: | | 1. Kernel name (eth0, eth1, etc) | | 2. Mac Address | | 3. Chassis name | | 4. PCI Slot |
-----------------------------------------------------------
----Choose an interface to configure (by chassis name)-----
| 1. LOM0 | | 2. LOM1 | | 3. Undefined | | 4. Undefined |
-----------------------------------------------------------
----------------Name interface - (chassis name LOM0)-------
| Name to use for this interface [eth0]: __mynet0_ |
-----------------------------------------------------------
-----------------------------------------------------------
| Configure interface - mynet0 | | 1. DHCP | | 2. Static | | ... |
-----------------------------------------------------------
[Generate udev rules that bind the user-selected name to
the user-selected attribute]
Quoted 10 lines
Quoted 2 lines
Quoted 2 lines
As a distribution developer I highly value solutions like
this which
Quoted 1 line
Quoted 1 line
do not require patching every application which deals with
interface
Quoted 1 line
Quoted 1 line
names and then teaching users about aliases which only
work in some
Quoted 3 lines
Quoted 1 line
places and are unknown to the kernel.
Fair enough - but would you object if we changed the naming scheme from eth%d to something else?
Quoted 15 lines
I suppose that this would depend on what else. :-) Since you want radical changes I recommend that you design the new persistent naming infrastructure in a way that will allow root to choose to use the classic naming scheme, or many users will scream a lot and at least some distributions will do it anyway. I also expect that providing choice at the beginning of development may lead to more acceptance later if and when the new scheme will have proved itself to be superior (at least in some situations).
You have tought about this for a long time and if so far you have not found a solution which is widely considered superior then I doubt that one will appear soon. Providing your favourite naming scheme as an optional add on will immediately benefit those who like it and greatly reduce opposition from those who do not.
In that way, I suppose char device node solution fits the scheme perfectly. It doesn't change or interfere with the kernel's default naming scheme (ethN) in any way. Also, the applications continue to work the way they did and in addition to supporting traditional names, they would also support pathnames. Whether all the user space applications need to be patched can be discussed and debated. But, we can patch applications like, installers and firewall code, which when don't see determinism ("eth0 mapping to integrated port 1"), fail and cause very high impact could be patched. Since users are already familiar with pathnames like /dev/disk/by-id{label, uuid}, I suppose it might not be very difficult to get used to pathnames like /dev/netdev/by-chassis-label/Embedded_NIC_1. Would that be acceptable ?
With regards,
Narendra K
Quoted 26 lines · On Mon, 9 Nov 2009 20:11:47 +0530 <[email protected]> wrote:
Quoted 10 lines
Quoted 2 lines
Quoted 2 lines
As a distribution developer I highly value solutions like
this which
Quoted 1 line
Quoted 1 line
do not require patching every application which deals with
interface
Quoted 1 line
Quoted 1 line
names and then teaching users about aliases which only
work in some
Quoted 3 lines
Quoted 1 line
places and are unknown to the kernel.
Fair enough - but would you object if we changed the naming scheme from eth%d to something else?
Quoted 15 lines
I suppose that this would depend on what else. :-) Since you want radical changes I recommend that you design the new persistent naming infrastructure in a way that will allow root to choose to use the classic naming scheme, or many users will scream a lot and at least some distributions will do it anyway. I also expect that providing choice at the beginning of development may lead to more acceptance later if and when the new scheme will have proved itself to be superior (at least in some situations).
You have tought about this for a long time and if so far you have not found a solution which is widely considered superior then I doubt that one will appear soon. Providing your favourite naming scheme as an optional add on will immediately benefit those who like it and greatly reduce opposition from those who do not.
Quoted 13 lines
In that way, I suppose char device node solution fits the scheme perfectly. It doesn't change or interfere with the kernel's default naming scheme (ethN) in any way. Also, the applications continue to work the way they did and in addition to supporting traditional names, they would also support pathnames. Whether all the user space applications need to be patched can be discussed and debated. But, we can patch applications like, installers and firewall code, which when don't see determinism ("eth0 mapping to integrated port 1"), fail and cause very high impact could be patched. Since users are already familiar with pathnames like
/dev/disk/by-id{label, uuid}, I suppose it might not be very difficult to get used to pathnames like/dev/netdev/by-chassis-label/Embedded_NIC_1. Would that be acceptable ?
IFNAMSIZ = 16 is hardwired as part of the kernel binary user space API.
Have you observed that the only developers arguing for this come from outside the normal circle of networking? It seems to be favored only by those who come to networking from a system or disk point of view.
Quoted 33 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Fair enough - but would you object if we changed the
naming scheme
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
from eth%d to something else?
I suppose that this would depend on what else. :-) Since you want radical changes I recommend that you design the new
persistent naming
Quoted 7 lines
Quoted 7 lines
infrastructure in a way that will allow root to choose to use the classic naming scheme, or many users will scream a lot and at least some distributions will do it anyway.
I also expect that providing choice at the beginning of development may lead to more acceptance later if and when the new scheme will have proved itself to be superior (at least in some situations). You have tought about this for a long time and if so faryou have not
Quoted 2 lines
Quoted 2 lines
found a solution which is widely considered superior then I doubt that one will appear soon. Providing your favourite naming
scheme as
Quoted 7 lines
Quoted 2 lines
an optional add on will immediately benefit those who like it and greatly reduce opposition from those who do not.
In that way, I suppose char device node solution fits the scheme perfectly. It doesn't change or interfere with the kernel's default naming scheme (ethN) in any way. Also, the applications continue to work the way they did and in addition to supporting
traditional names,
Quoted 2 lines
they would also support pathnames. Whether all the user space applications need to be patched can be discussed and
debated. But, we
Quoted 1 line
can patch applications like, installers and firewall code,
which when
Quoted 1 line
don't see determinism ("eth0 mapping to integrated port 1"),
fail and
Quoted 8 lines
Quoted 4 lines
cause very high impact could be patched. Since users are already familiar with pathnames like
/dev/disk/by-id{label, uuid}, I suppose it might not be very difficult to get used to pathnames like/dev/netdev/by-chassis-label/Embedded_NIC_1. Would that beacceptable ?
Quoted 1 line
IFNAMSIZ = 16 is hardwired as part of the kernel binary user space API.
This factor is taken into consideration. The user space applications take this pathname, map it to the kernel name and use the kernel name to issue ioctls (http://linux.dell.com/wiki/index.php/Oss/libnetdevname).
The pathname was suggested because it provides a way to get to the right interface when "integrated port 1" doesn't get the expected name "eth0".
With regards,
Narendra K
Quoted 9 lines · Narendra
Similarly, export an attribute named "
smbios_name" to sysfs, i.e "/sys/class/net/eth0/smbios_name". "Cat/sys/class/net/eth0/smbios_name" would show "Embedded_NIC_1[23..]" and this can be used by udev in 70-persistent-net.rules asSUBSYSTEM="net", ACTION="add", DRIVERS="?*", ATTR{smbios_name}="Embedded_NIC_1", ATTR{type}="1", KERNEL="eth*", NAME="eth0".
Quoted 6 lines
I suppose this would not need any changes to the udev code and existing udev infrastructure can be used as udev is capable handling ATTR{something}.
This would also ensure that whichever device is "
Embedded_NIC_1" as per the BIOS, will also be "eth0" in the os.
We can grab the smbios_name value using biosdevname in a PROGRAM= part of the udev rule. But it doesn't actually solve the problem. We haven't changed the network device naming scheme from "eth%d" to something else. Therefore, by having rules which simply try to re-order names within that scheme, when they're being enumerated in parallel and racing, we get collisions. Take for example, this which tries to rename the 4 onboard NICs in a particular order, in the absence of any other rules:
PROGRAM="/sbin/biosdevname --policy=smbios_names -i %k", RESULT="Embedded NIC 1", NAME="eth0" PROGRAM="/sbin/biosdevname --policy=smbios_names -i %k", RESULT="Embedded NIC 2", NAME="eth1" PROGRAM="/sbin/biosdevname --policy=smbios_names -i %k", RESULT="Embedded NIC 3", NAME="eth2" PROGRAM="/sbin/biosdevname --policy=smbios_names -i %k", RESULT="Embedded NIC 4", NAME="eth3"
I wind up with instead this in ifconfig -a:
eth0 00:1B:21:42:66:30 eth1 00:1B:21:42:66:31 eth2 00:22:19:59:8E:5A
eth2_rename 00:22:19:59:8E:56
eth3 00:22:19:59:8E:5Ceth3_rename 00:22:19:59:8E:58
When what I would have expected would have been:
eth0 00:22:19:59:8E:56
eth1 00:22:19:59:8E:58
eth2 00:22:19:59:8E:5A
eth3 00:22:19:59:8E:5C
eth4 00:1B:21:42:66:30
eth5 00:1B:21:42:66:31
I can't use eth%d as the scheme - that's the kernel's scheme. I have to switch the scheme to something else.
Signature · 3 lines
Matt Domsch Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux
PATCH 1/1 udev: create empty regular files to represent net interfaces
In reply to dann frazierHere's a proof of concept to further the discussion..
The default filename uses the format:
/dev/netdev/by-ifindex/$ifindex
This provides the infrastructure to permit udev rules to create aliases for network devices using symlinks, for example:
/dev/netdev/by-name/eth0 -> ../by-ifindex/1
/dev/netdev/by-biosname/LOM0 -> ../by-ifindex/3
A library (such as the proposed libnetdevname) could use this information to provide an alias->realname mapping for network utilities.
Tested with the following rule:
SUBSYSTEM="net", PROGRAM="/usr/local/bin/ifindex2name $attr{ifindex}", SYMLINK+="netdev/by-name/%c"
$ cat /usr/local/bin/ifindex2name #!/bin/sh
set -e
ifindex="$1"
for d in /sys/class/net/*; do
testindex="$(cat $d/ifindex)"
if [ "$ifindex" = "$testindex" ]; then
echo "$(basename $d)"
exit 0
fidone
exit 1
--- libudev/exported_symbols | 1 + libudev/libudev.c | 29 ++++++++++++++++ libudev/libudev.h | 1 + udev/udev-event.c | 82 ++++++++++++++++++++-------------------------- udev/udev-node.c | 41 ++++++++++++++++++++--- udev/udev-rules.c | 3 +- 6 files changed, 105 insertions(+), 52 deletions(-)
libudev/exported_symbols +1 -0
@@ -8,6 +8,7 @@ udev_get_userdata udev_set_userdata udev_get_sys_path udev_get_dev_path +udev_get_netdev_path udev_list_entry_get_next udev_list_entry_get_by_name udev_list_entry_get_name
libudev/libudev.c +29 -0
@@ -42,6 +42,7 @@ struct udev {void *userdata;char *sys_path;char *dev_path; +char *netdev_path;char *rules_path;struct udev_list_node properties_list;int log_priority;
@@ -125,8 +126,10 @@ struct udev *udev_new(void) udev->run =1 ; udev->dev_path =strdup ("/dev" ); udev->sys_path =strdup ("/sys" ); + udev->netdev_path =strdup ("/dev/netdev/by-ifindex" ); config_file =strdup (SYSCONFDIR "/udev/udev.conf" );if (udev->dev_path =NULL || + udev->netdev_path =NULL || udev->sys_path =NULL || config_file =NULL ) goto err;
@@ -243,6 +246,14 @@ struct udev *udev_new(void)udev_add_property (udev,"UDEV_ROOT" , udev->dev_path); } + env =getenv ("NETDEV_ROOT" ); +if (env !=NULL ) { +free (udev->netdev_path); + udev->netdev_path =strdup (env); +util_remove_trailing_chars (udev->netdev_path,'/' ); +udev_add_property (udev,"NETDEV_ROOT" , udev->netdev_path); + } + env =getenv ("UDEV_LOG" );if (env !=NULL )udev_set_log_priority (udev,util_log_priority (env));
@@ -253,6 +264,7 @@ struct udev *udev_new(void)dbg (udev,"log_priority=%d\n" , udev->log_priority);dbg (udev,"config_file='%s'\n" , config_file);dbg (udev,"dev_path='%s'\n" , udev->dev_path); +dbg (udev,"netdev_path='%s'\n" , udev->netdev_path);dbg (udev,"sys_path='%s'\n" , udev->sys_path);if (udev->rules_path !=NULL )dbg (udev,"rules_path='%s'\n" , udev->rules_path);
@@ -398,6 +410,23 @@ const char *udev_get_dev_path(struct udev *udev)return udev->dev_path; } +/** +* udev_get_netdev_path: +* @udev: udev library context +* +* Retrieve the device directory path. The default value is "/etc/udev/net", +* the actual value may be overridden in the udev configuration +* file. +* +* Returns: the device directory path +**/ +const char *udev_get_netdev_path (struct udev *udev) +{ +if (udev =NULL ) +return NULL ; +return udev->netdev_path; +} +struct udev_list_entry *udev_add_property (struct udev *udev,const char *key,const char *value) {if (value =NULL ) {
libudev/libudev.h +1 -0
@@ -77,6 +77,7 @@ struct udev_device *udev_device_get_parent_with_subsystem_devtype(struct udev_de const char *subsystem, const char *devtype);/* retrieve device properties */ const char *udev_device_get_devpath (struct udev_device *udev_device); +const char *udev_device_get_netdevpath (struct udev_device *udev_device);const char *udev_device_get_subsystem (struct udev_device *udev_device);const char *udev_device_get_devtype (struct udev_device *udev_device);const char *udev_device_get_syspath (struct udev_device *udev_device);
udev/udev-event.c +36 -46
102 lines not on this page · raw message
udev/udev-node.c +36 -5
71 lines not on this page · raw message
udev/udev-rules.c +2 -1
@@ -2435,7 +2435,8 @@ int udev_rules_apply_to_event(struct udev_rules *rules, struct udev_event *eventif (event->devlink_final)break ; -if (major (udev_device_get_devnum (event->dev)) =0 ) +if ((major (udev_device_get_devnum (event->dev)) =0 ) && + (strcmp (udev_device_get_subsystem (event->dev),"net" ) !=0 ))break ;if (cur->key.op =OP_ASSIGN_FINAL ) event->devlink_final =1 ;
Signature · 1 line
1.6.5
Re: [PATCH] udev: create empty regular files to represent net interfaces
Quoted 5 lines · dann frazier
Here's a proof of concept to further the discussion..
The default filename uses the format:
/dev/netdev/by-ifindex/$ifindex
Quoted 8 lines
This provides the infrastructure to permit udev rules to create aliases for network devices using symlinks, for example:
/dev/netdev/by-name/eth0-> ../by-ifindex/1
/dev/netdev/by-biosname/LOM0-> ../by-ifindex/3A library (such as the proposed libnetdevname) could use this information to provide an
alias->realnamemapping for network utilities.
yes, this could work, as IFINDEX is already exported in the uevents, and that's the primary value udev needs to set up the mapping.
While I like the little ifindex2name script you've got, I think udev could simply call if_indextoname() to get this, and not call an external program? I suppose it could be a really really simple external program too.
I'd be fine with this approach. It has the advantages of not requiring a kernel change at all, and not creating a whole character device which would be useless. And it doesn't preclude someone in the future from creating a char device for network devices should they so choose.
Kay, what say you as udev owner?
Thanks,
Matt
Signature · 3 lines
Matt Domsch Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux
Re: [PATCH] udev: create empty regular files to represent net interfaces
Quoted 15 lines · Matt Domsch
Quoted 5 lines · dann frazier
Here's a proof of concept to further the discussion..
The default filename uses the format:
/dev/netdev/by-ifindex/$ifindexQuoted 8 lines
This provides the infrastructure to permit udev rules to create aliases for network devices using symlinks, for example:
/dev/netdev/by-name/eth0-> ../by-ifindex/1
/dev/netdev/by-biosname/LOM0-> ../by-ifindex/3A library (such as the proposed libnetdevname) could use this information to provide an
alias->realnamemapping for network utilities.
Quoted 7 lines
yes, this could work, as IFINDEX is already exported in the uevents, and that's the primary value udev needs to set up the mapping.
While I like the little ifindex2name script you've got, I think udev could simply call
if_indextoname()to get this, and not call an external program? I suppose it could be a really really simple external program too.
What's the point of all this? Why would udev ever need to find the name of a device by the ifindex? The device name is the primary value for the kernel events udev acts on.
Quoted 7 lines
I'd be fine with this approach. It has the advantages of not requiring a kernel change at all, and not creating a whole character device which would be useless. And it doesn't preclude someone in the future from creating a char device for network devices should they so choose.
Kay, what say you as udev owner?
That all sounds very much like something which will hit us back some day. I'm not sure, if udev should publish such dead text files in /dev, it does not seem to fit the usual APIs/assumptions where /sys and /dev match, and libudev provides access to both. It all sounds more like a database for a possible netdevname library, which does not need to be public in /dev, right?
Thanks,
Kay
Re: [PATCH] udev: create empty regular files to represent net interfaces
Quoted 19 lines · Kay Sievers
Quoted 18 lines · Matt Domsch
Quoted 5 lines · dann frazier
Here's a proof of concept to further the discussion..
The default filename uses the format:
??/dev/netdev/by-ifindex/$ifindexQuoted 8 lines
This provides the infrastructure to permit udev rules to create aliases for network devices using symlinks, for example:
??
/dev/netdev/by-name/eth0-> ../by-ifindex/1 ??/dev/netdev/by-biosname/LOM0-> ../by-ifindex/3A library (such as the proposed libnetdevname) could use this information to provide an
alias->realnamemapping for network utilities.yes, this could work, as IFINDEX is already exported in the uevents, and that's the primary value udev needs to set up the mapping.
Quoted 8 lines
Quoted 4 lines
While I like the little ifindex2name script you've got, I think udev could simply call
if_indextoname()to get this, and not call an external program? ??I suppose it could be a really really simple external program too.What's the point of all this? Why would udev ever need to find the name of a device by the ifindex? The device name is the primary value for the kernel events udev acts on.
Ultimately, udev doesn't care. I just want to use udev to keep track of the pathname to device connections, like it does for all other types of devices.
Applications such as net-tools, iproute, ethtool, etc. take a kernel device name. I want to extend them to also take a path, and resolve that path to a kernel device name. libnetdevname currently is _one small function_ which does this. It need not even be in a library. But whatever the mechanism, the path names need to be anchored somewhere, so the library or all apps doing this kind of lookup know where to look.
Quoted 6 lines
That all sounds very much like something which will hit us back some day. I'm not sure, if udev should publish such dead text files in /dev, it does not seem to fit the usual APIs/assumptions where /sys and /dev match, and libudev provides access to both. It all sounds more like a database for a possible netdevname library, which does not need to be public in /dev, right?
Right, it doesn't need to be in /dev. We could have udev rules that simply call yet another program to maintain that database, in yet another way. But I really like how udev maintains the database of symlinks for other device types, using symlinks in /dev/, and which people are quite familiar with. Why can't it be extended to do likewise for network device names too?
There is a completely different approach possible here, if people don't want to use something like /dev to track device name aliases. We could put the whole name alias mechanism in the kernel, with new netlink commands to add/remove/list aliases (and now we've overloaded that term, as the old eth0:1 "alias" and dmz -> eth1 "alias" wouldn't be the same thing). But that idea hasn't met with a lot of interest either.
Signature · 3 lines
Matt Domsch Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux
Re: [PATCH] udev: create empty regular files to represent net interfaces
Quoted 1 line · Matt Domsch
On Wed, Oct 28, 2009 at 09:23:57AM +0100, Kay Sievers wrote:
[...]
Quoted 10 lines
Quoted 6 lines
That all sounds very much like something which will hit us back some day. I'm not sure, if udev should publish such dead text files in /dev, it does not seem to fit the usual APIs/assumptions where /sys and /dev match, and libudev provides access to both. It all sounds more like a database for a possible netdevname library, which does not need to be public in /dev, right?
Right, it doesn't need to be in /dev. We could have udev rules that simply call yet another program to maintain that database, in yet another way.
Or have udev maintain them in a private directory (e.g., /var/lib/udev/netalias). Personally, I like the approach of having udev manage them as files - its an abstraction our users already get, and they don't have to learn two mechanisms when aliasing disks and nics (SYMLINK ftw). Plus there's obviously a lot of code reuse to be had (most of my patch was moving code into a common section).
If we want to hide the file implementation - we could invent another udev construct that basically aliases SYMLINK (e.g. NETALIAS) that works iff the device is a netdevice. That would let us switch out implementations in the future, but would obviously be much more invasive.
Signature · 1 line
dann frazier
RE: [PATCH] udev: create empty regular files to represent net interfaces
I was thinking, if we're not planning on use the chardev/kernel route. There already exists an ifindex file in /sys/class/net/XXX/ifindex. Should udev/helper be creating links to this, or is it better to keep everything under the /dev/ tree?
Using this method would require the patch to udev to handle renaming events.
--jordan hargrave
Dell Enterprise Linux Engineering
-----Original Message-----
From: dann frazier [mailto:[email protected]]
Sent: Wed 10/28/2009 10:09
To: Domsch, Matt
Cc: Kay Sievers; [email protected]; K, Narendra; [email protected]; Hargrave, Jordan; Rose, Charles; Ben Hutchings
Subject: Re: [PATCH] udev: create empty regular files to represent net interfaces
Quoted 1 line · Matt Domsch
On Wed, Oct 28, 2009 at 09:23:57AM +0100, Kay Sievers wrote:
[...]
Quoted 10 lines
Quoted 6 lines
That all sounds very much like something which will hit us back some day. I'm not sure, if udev should publish such dead text files in /dev, it does not seem to fit the usual APIs/assumptions where /sys and /dev match, and libudev provides access to both. It all sounds more like a database for a possible netdevname library, which does not need to be public in /dev, right?
Right, it doesn't need to be in /dev. We could have udev rules that simply call yet another program to maintain that database, in yet another way.
Or have udev maintain them in a private directory (e.g., /var/lib/udev/netalias). Personally, I like the approach of having udev manage them as files - its an abstraction our users already get, and they don't have to learn two mechanisms when aliasing disks and nics (SYMLINK ftw). Plus there's obviously a lot of code reuse to be had (most of my patch was moving code into a common section).
If we want to hide the file implementation - we could invent another udev construct that basically aliases SYMLINK (e.g. NETALIAS) that works iff the device is a netdevice. That would let us switch out implementations in the future, but would obviously be much more invasive.
Signature · 1 line
dann frazier
RE: [PATCH] udev: create empty regular files to represent net interfaces
In reply to dann frazierI was thinking, if we're not planning on use the chardev/kernel route. There already exists an ifindex file in /sys/class/net/XXX/ifindex. Should udev/helper be creating links to this, or is it better to keep everything under the /dev/ tree?
Using this method would require the patch to udev to handle renaming events.
--jordan hargrave
Dell Enterprise Linux Engineering
-----Original Message-----
From: dann frazier [mailto:[email protected]]
Sent: Wed 10/28/2009 10:09
To: Domsch, Matt
Cc: Kay Sievers; [email protected]; K, Narendra; [email protected]; Hargrave, Jordan; Rose, Charles; Ben Hutchings
Subject: Re: [PATCH] udev: create empty regular files to represent net interfaces
Quoted 1 line · Matt Domsch
On Wed, Oct 28, 2009 at 09:23:57AM +0100, Kay Sievers wrote:
[...]
Quoted 10 lines
Quoted 6 lines
That all sounds very much like something which will hit us back some day. I'm not sure, if udev should publish such dead text files in /dev, it does not seem to fit the usual APIs/assumptions where /sys and /dev match, and libudev provides access to both. It all sounds more like a database for a possible netdevname library, which does not need to be public in /dev, right?
Right, it doesn't need to be in /dev. We could have udev rules that simply call yet another program to maintain that database, in yet another way.
Or have udev maintain them in a private directory (e.g., /var/lib/udev/netalias). Personally, I like the approach of having udev manage them as files - its an abstraction our users already get, and they don't have to learn two mechanisms when aliasing disks and nics (SYMLINK ftw). Plus there's obviously a lot of code reuse to be had (most of my patch was moving code into a common section).
If we want to hide the file implementation - we could invent another udev construct that basically aliases SYMLINK (e.g. NETALIAS) that works iff the device is a netdevice. That would let us switch out implementations in the future, but would obviously be much more invasive.
Signature · 1 line
dann frazier
Re: [PATCH] udev: create empty regular files to represent net interfaces
A: No.
Q: Should I include quotations after my reply?
http://daringfireball.net/2007/07/on_top
Quoted 5 lines · On Wed, Oct 28, 2009 at 11:09:36AM -0500, [email protected] wrote:
I was thinking, if we're not planning on use the chardev/kernel route. There already exists an ifindex file in
/sys/class/net/XXX/ifindex. Should udev/helper be creating links to this, or is it better to keep everything under the/dev/tree? Using this method would require the patch to udev to handle renaming events.
Please never create symlinks out of /dev/ to /sys that doesn't make sense at all and probably violates part of the LSB somewhere...
thanks,
greg k-h
Re: [PATCH] udev: create empty regular files to represent net interfaces
In reply to Kay SieversQuoted 19 lines · Kay Sievers
Quoted 18 lines · Matt Domsch
Quoted 5 lines · dann frazier
Here's a proof of concept to further the discussion..
The default filename uses the format:
/dev/netdev/by-ifindex/$ifindexQuoted 8 lines
This provides the infrastructure to permit udev rules to create aliases for network devices using symlinks, for example:
/dev/netdev/by-name/eth0-> ../by-ifindex/1
/dev/netdev/by-biosname/LOM0-> ../by-ifindex/3A library (such as the proposed libnetdevname) could use this information to provide an
alias->realnamemapping for network utilities.yes, this could work, as IFINDEX is already exported in the uevents, and that's the primary value udev needs to set up the mapping.
Quoted 8 lines
Quoted 4 lines
While I like the little ifindex2name script you've got, I think udev could simply call
if_indextoname()to get this, and not call an external program? I suppose it could be a really really simple external program too.What's the point of all this? Why would udev ever need to find the name of a device by the ifindex? The device name is the primary value for the kernel events udev acts on.
[...]
Since net devices can be renamed, unlike other devices, the ifindex is the proper stable identifier. Using the name as an identifier opens up race conditions. If there are events that don't include the ifindex, this should be fixed.
Ben.
Signature · 3 lines
Ben Hutchings, Senior Software Engineer, Solarflare Communications Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
RE: [PATCH] udev: create empty regular files to represent net interfaces
In reply to Kay SieversQuoted 8 lines
That all sounds very much like something which will hit us back some day. I'm not sure, if udev should publish such dead text files in /dev, it does not seem to fit the usual APIs/assumptions where /sys and /dev match, and libudev provides access to both. It all sounds more like a database for a possible netdevname library, which does not need to be public in /dev, right?
The char device nodes under /dev/netdev/ do seem to adhere to the assumption of what is there under /sys and /dev match.
With regards,
Narendra K
Re: [PATCH] udev: create empty regular files to represent net interfaces
In reply to Kay SieversQuoted 13 lines · Kay Sievers
Quoted 12 lines · Matt Domsch
Quoted 11 lines · dann frazier
Here's a proof of concept to further the discussion..
The default filename uses the format:
??/dev/netdev/by-ifindex/$ifindexThis provides the infrastructure to permit udev rules to create aliases for network devices using symlinks, for example:
??
/dev/netdev/by-name/eth0-> ../by-ifindex/1 ??/dev/netdev/by-biosname/LOM0-> ../by-ifindex/3
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
A library (such as the proposed libnetdevname) could use this information to provide an
alias->realnamemapping for network utilities.That all sounds very much like something which will hit us back some day. I'm not sure, if udev should publish such dead text files in /dev, it does not seem to fit the usual APIs/assumptions where /sys and /dev match, and libudev provides access to both.
While we could do this without any kernel changes at all, it does still leave unresolved the concern of the in-kernel users of netdevice-by-name - everything that uses dev_get_by_name(), and any userspace tool that doesn't get converted to use the netdevice-by-path concept.
Which brings us back to still looking for options.
Multiple names for the same device gives us a way out. Users of the ethX naming convention continue unchanged, and live with the nondeterminism; Users of other naming conventions can get names that work for them, with needed determinism.
Netdev team - are you in agreement that having multiple names to address the same netdevice is a worthwhile thing to add, to allow a variety of naming schemes to exist simultaneously? If not, this whole discussion will be moot, and my basic problem, that the ethX naming convention is nondeterministic, but we need determinism, remains unresolved.
Assuming we agree it's worthwhile, I'm open to ideas for ways to solve it. We've proposed char devices for udev to manage, and fixing up userspace programs, but that doesn't solve in-kernel users by name. Dann proposed another method (above) which Kay isn't fond of, and which has the same drawback. I need another option then.
Thanks,
Matt
Signature · 3 lines
Matt Domsch Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux
Re: [PATCH] udev: create empty regular files to represent net interfaces
Quoted 6 lines · Matt Domsch
Netdev team - are you in agreement that having multiple names to address the same netdevice is a worthwhile thing to add, to allow a variety of naming schemes to exist simultaneously? If not, this whole discussion will be moot, and my basic problem, that the ethX naming convention is nondeterministic, but we need determinism, remains unresolved.
I'm still totally confused as to why you think this. What is wrong with what we do today, which is name network devices in a deterministic manner by their MAC in userspace? That name goes into the kernel, and everyone uses the same name and is happy.
If you don't like naming by MAC, then pick some other deterministic naming scheme that works for your hardware and write udev rules for it.
You could easily name them in a way that could keep the lowest number (eth0) for the lowest PCI id if you so desired and your BIOS guaranteed it.
This way the kernel has only one name, and so does userspace, and everyone is happy.
thanks,
greg k-h
RE: [PATCH] udev: create empty regular files to represent net interfaces
Quoted 5 lines
Quoted 4 lines
Netdev team - are you in agreement that having multiple names to address the same netdevice is a worthwhile thing to add, to allow a variety of naming schemes to exist simultaneously? If not,
this whole
Quoted 8 lines
Quoted 3 lines
discussion will be moot, and my basic problem, that the ethX naming convention is nondeterministic, but we need determinism, remains unresolved.
I'm still totally confused as to why you think this. What is wrong with what we do today, which is name network devices in a deterministic manner by their MAC in userspace? That name goes into the kernel, and everyone uses the same name and is happy.
The interface name as assigned by the OS is determined by how the interface is named first during the OS installation. This name is made persistent by associating the name with it's MAC address in userspace, either by udev or ifcfg-eth files. In cases where there are one or more add-in cards along with one or more integrated cards (Lan on Motherboard), the integrated port 1, which is designated as Gb1 on the chassis may or may not get the name "eth0". And that is the customer expectation, most of the times.
Unattended installs and large scale image based installs are the most affected scenarios.
Quoted 8 lines
If you don't like naming by MAC, then pick some other deterministic naming scheme that works for your hardware and write udev rules for it.
You could easily name them in a way that could keep the lowest number (eth0) for the lowest PCI id if you so desired and your BIOS guaranteed it.
This is how the lspci tree view on a PER710 (PowerEdge R710) server with Four BCM5709 integrated NIC ports and One add-in Intel NIC port looks like. The integrated ports are always found before the add-in nic (or nics) by the BIOS consistently and BIOS guarantees it across every reboot. If the OS also found and named the network ports in the same manner, then there is no issue as integrated NIC port 1, designated Gb1 on the chassis, is always named as "eth0". But the observation is that, it is not the case always.
-[0000:00]-+-00.0 Intel Corporation 5520 I/O Hub to ESI Port
+-01.0-[0000:01]--+-00.0 Broadcom Corporation NetXtreme II
BCM5709 Gigabit Ethernet
| \-00.1 Broadcom Corporation NetXtreme II
BCM5709 Gigabit Ethernet
+-03.0-[0000:02]--+-00.0 Broadcom Corporation NetXtreme II
BCM5709 Gigabit Ethernet
| \-00.1 Broadcom Corporation NetXtreme II
BCM5709 Gigabit Ethernet
+-04.0-[0000:03]----00.0 LSI Logic / Symbios Logic MegaRAID
SAS 1078
+-05.0-[0000:04]--
+-06.0-[0000:05]--
+-07.0-[0000:06]--
+-09.0-[0000:07]----00.0 Intel Corporation 82598EB10-Gigabit AT Network Connection
In such cases, pathnames like Embedded_NIC_1 -> eth[01..], point to the right interface, and communicate a more meaningful name without any state embedded in them.
With regards,
Narendra K
Re: [PATCH] udev: create empty regular files to represent net interfaces
Quoted 9 lines · On Thu, Oct 29, 2009 at 10:14:08PM +0530, [email protected] wrote:
Quoted 9 lines
Quoted 4 lines
Netdev team - are you in agreement that having multiple names to address the same netdevice is a worthwhile thing to add, to allow a variety of naming schemes to exist simultaneously? If not,
this whole
Quoted 3 lines
discussion will be moot, and my basic problem, that the ethX naming convention is nondeterministic, but we need determinism, remains unresolved.
Quoted 7 lines
Quoted 4 lines
I'm still totally confused as to why you think this. What is wrong with what we do today, which is name network devices in a deterministic manner by their MAC in userspace? That name goes into the kernel, and everyone uses the same name and is happy.
The interface name as assigned by the OS is determined by how the interface is named first during the OS installation.
That sounds like a distro install issue to me, why not fix it there?
Quoted 5 lines
This name is made persistent by associating the name with it's MAC address in userspace, either by udev or ifcfg-eth files. In cases where there are one or more add-in cards along with one or more integrated cards (Lan on Motherboard), the integrated port 1, which is designated as Gb1 on the chassis may or may not get the name "eth0".
Exactly, who cares about "eth0" as a name?
Quoted 1 line
And that is the customer expectation, most of the times.
Then again, fix the installer to allow you to either pick the name, or specify some rule in which to use to pick the name.
Quoted 2 lines
Unattended installs and large scale image based installs are the most affected scenarios.
Then fix the installer.
Quoted 9 lines
Quoted 8 lines
If you don't like naming by MAC, then pick some other deterministic naming scheme that works for your hardware and write udev rules for it.
You could easily name them in a way that could keep the lowest number (eth0) for the lowest PCI id if you so desired and your BIOS guaranteed it.
Quoted 5 lines
This is how the lspci tree view on a PER710 (PowerEdge R710) server with Four BCM5709 integrated NIC ports and One add-in Intel NIC port looks like. The integrated ports are always found before the add-in nic (or nics) by the BIOS consistently and BIOS guarantees it across every reboot.
Great, then you are set to write a udev rule for this, right?
Quoted 4 lines
If the OS also found and named the network ports in the same manner, then there is no issue as integrated NIC port 1, designated Gb1 on the chassis, is always named as "eth0". But the observation is that, it is not the case always.
Sure, it's never guaranteed by the kernel that this will happen, especially as we speed up the boot process by doing things async.
So again, just fix your installer, or write a new udev rule for your hardware platforms, or both. But I still fail to see why multiple names for network devices _in the kernel_ is a solution for your issue.
Quoted 3 lines
In such cases, pathnames like
Embedded_NIC_1-> eth[01..], point to the right interface, and communicate a more meaningful name without any state embedded in them.
Yes, pathnames would be nice to work for network devices, but unfortunatly, that's just not how network devices work :)
thanks,
greg k-h
RE: [PATCH] udev: create empty regular files to represent netinterfaces
Quoted 10 lines
Sure, it's never guaranteed by the kernel that this will happen, especially as we speed up the boot process by doing things async.
So again, just fix your installer, or write a new udev rule for your hardware platforms, or both. But I still fail to see why multiple names for network devices
_inthekernel_is a solution for your issue.
The char device nodes solution does not propose having multiple names for the network interfaces _in the kernel_. It is suggesting that we have alternate names for kernel assigned names in user space and user space utilities refer to the interface by these alternate names. The userspace utilities would have to map the pathnames to kernel names before issuing the ioctls. That way there is determinism without embedding MAC or any other attribute. An Embedded_NIC_1 interface would always refer to the Gb1 without having to depend on any attribute.
With regards,
Narendra K
Re: [PATCH] udev: create empty regular files to represent netinterfaces
Quoted 5 lines · On Thu, Oct 29, 2009 at 10:52:52PM +0530, [email protected] wrote:
Quoted 4 lines
Sure, it's never guaranteed by the kernel that this will happen, especially as we speed up the boot process by doing things async.
Quoted 8 lines
Quoted 5 lines
So again, just fix your installer, or write a new udev rule for your hardware platforms, or both. But I still fail to see why multiple names for network devices
_inthekernel_is a solution for your issue.The char device nodes solution does not propose having multiple names for the network interfaces
_inthekernel_.
Nor does the udev-only implementation I posted which doesn't rely on new character devices.
Signature · 1 line
dann frazier
Re: [PATCH] udev: create empty regular files to represent net interfaces
In reply to Greg KHQuoted 16 lines · Greg KH
Quoted 6 lines · Matt Domsch
Netdev team - are you in agreement that having multiple names to address the same netdevice is a worthwhile thing to add, to allow a variety of naming schemes to exist simultaneously? If not, this whole discussion will be moot, and my basic problem, that the ethX naming convention is nondeterministic, but we need determinism, remains unresolved.
I'm still totally confused as to why you think this. What is wrong with what we do today, which is name network devices in a deterministic manner by their MAC in userspace? That name goes into the kernel, and everyone uses the same name and is happy.
If you don't like naming by MAC, then pick some other deterministic naming scheme that works for your hardware and write udev rules for it.
Quoted 6 lines
You could easily name them in a way that could keep the lowest number (eth0) for the lowest PCI id if you so desired and your BIOS guaranteed it.
This way the kernel has only one name, and so does userspace, and everyone is happy.
I thought there was a general trend in udev development to provide default rules that work for almost everyone, so few users/administrators need to override or add to them. Compare disks and net devices:
1. Stable kernel device id
Disks: block device number
Net devices: ifindex
2. Unique identifier (across reboot)
Disks: label or UUID (each with limitations)
Net devices: (MAC address, subtype)
3. Name assignment mechanism
Disks: kernel suggests a name; udev can assign any number Net devices: kernel assigns a single name; udev can override it
4. Default name assignment policy
Disks: names disk by device path (id), label and UUID
Net devices: assigns arbitrary stable names per (MAC address, subtype)
5. Naming by users
Disks: user can identify by any method without having to choose on a system-wide basis
Net devices: user must identify by single name; policy can be overridden on a system-wide basis
I fully understand the technical reasons for differences 3-5, but why should users have to put up with it?
Ben.
Signature · 3 lines
Ben Hutchings, Senior Software Engineer, Solarflare Communications Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
Re: [PATCH] udev: create empty regular files to represent net interfaces
Quoted 8 lines · Ben Hutchings
3. Name assignment mechanism
Disks: kernel suggests a name; udev can assign any number ???Net devices: kernel assigns a single name; udev can override it4. Default name assignment policy
Disks: names disk by device path (id), label and UUID
???Net devices: assigns arbitrary stable names per (MAC address, subtype)
Quoted 8 lines
5. Naming by users
Disks: user can identify by any method without having to choose on a system-wide basis
Net devices: user must identify by single name; policy can be overridden on a system-wide basisI fully understand the technical reasons for differences 3-5, but why should users have to put up with it?
That is because network devices are not referred to by /dev/ nodes where multiple symlinks would solve the naming problem.
thanks,
greg k-h
Re: [PATCH] udev: create empty regular files to represent net interfaces
Quoted 15 lines · Greg KH
Quoted 14 lines · Ben Hutchings
3. Name assignment mechanism
Disks: kernel suggests a name; udev can assign any number ???Net devices: kernel assigns a single name; udev can override it4. Default name assignment policy
Disks: names disk by device path (id), label and UUID
???Net devices: assigns arbitrary stable names per (MAC address, subtype)5. Naming by users
Disks: user can identify by any method without having to choose on a system-wide basis
Net devices: user must identify by single name; policy can be overridden on a system-wide basis
Quoted 5 lines
Quoted 2 lines
I fully understand the technical reasons for differences 3-5, but why should users have to put up with it?
That is because network devices are not referred to by
/dev/nodes where multiple symlinks would solve the naming problem.
Did you even read the last sentence?
Ben.
Signature · 3 lines
Ben Hutchings, Senior Software Engineer, Solarflare Communications Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
Re: [PATCH] udev: create empty regular files to represent net interfaces
Quoted 16 lines · Ben Hutchings
Quoted 15 lines · Greg KH
Quoted 14 lines · Ben Hutchings
3. Name assignment mechanism
Disks: kernel suggests a name; udev can assign any number ???Net devices: kernel assigns a single name; udev can override it4. Default name assignment policy
Disks: names disk by device path (id), label and UUID
???Net devices: assigns arbitrary stable names per (MAC address, subtype)5. Naming by users
Disks: user can identify by any method without having to choose on a system-wide basis
Net devices: user must identify by single name; policy can be overridden on a system-wide basis
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
I fully understand the technical reasons for differences 3-5, but why should users have to put up with it?
That is because network devices are not referred to by
/dev/nodes where multiple symlinks would solve the naming problem.Did you even read the last sentence?
Yes, the technical reason is the reason why users have to put up with it :)
Re: [PATCH] udev: create empty regular files to represent net interfaces
In reply to Greg KHQuoted 16 lines · Greg KH
Quoted 6 lines · Matt Domsch
Netdev team - are you in agreement that having multiple names to address the same netdevice is a worthwhile thing to add, to allow a variety of naming schemes to exist simultaneously? If not, this whole discussion will be moot, and my basic problem, that the ethX naming convention is nondeterministic, but we need determinism, remains unresolved.
I'm still totally confused as to why you think this. What is wrong with what we do today, which is name network devices in a deterministic manner by their MAC in userspace? That name goes into the kernel, and everyone uses the same name and is happy.
If you don't like naming by MAC, then pick some other deterministic naming scheme that works for your hardware and write udev rules for it.
Quoted 6 lines
You could easily name them in a way that could keep the lowest number (eth0) for the lowest PCI id if you so desired and your BIOS guaranteed it.
This way the kernel has only one name, and so does userspace, and everyone is happy.
There are two issues, which really seem distinct to me.
Users expect eth0 to map to first-onboard-nic. That's an installer issue (since the BIOS can already export this info) and I agree that if we want to "fix" that, we should fix it there.
Users also want to have a name that matches the way they think of their hardware - pci slot, bios-exposed-name, mac address, whatever. This can be done today w/ custom udev rules, and I can visualize an installer that would generate these rules for you:
Configure a NIC
\-> Choose NIC by: MAC/CHASSIS-NAME/PCI-SLOT
[ Present list of unconfigured NICs by selected property ]
\-> What name would you like to use for this interface [eth3]?
How do you want this configured (DHCP/STATIC/..)
...That would make a lot of users much happier (myself included), but it does restrict us into one view. At different times, admins think of their NICs by different properties. I may want to do IP assignment by the chassis name, but then run ethereal on a specific mac address. Or I may want to see the routes assigned to all NICs in a given PCI slot. Sure, I can lookup all of these properties and map them back to an interface name by hand, but aliasing provides a nice way to short-circuit that. And, by providing a library that translates the aliases for us, we can help ensure that all apps that want to provide aliasing can do so in a common way.
Signature · 1 line
dann frazier
Re: [PATCH] udev: create empty regular files to represent net interfaces
Quoted 3 lines · dann frazier
That would make a lot of users much happier (myself included), but it does restrict us into one view. At different times, admins think of their NICs by different properties. I may want to do IP assignment by
[Citation needed]
(An admin.)
Signature · 2 lines
ciao, Marco
Re: [PATCH] udev: create empty regular files to represent net interfaces
Quoted 6 lines · Marco d'Itri
Quoted 3 lines · dann frazier
That would make a lot of users much happier (myself included), but it does restrict us into one view. At different times, admins think of their NICs by different properties. I may want to do IP assignment by
[Citation needed]
Is "I" not clear, or do you just not believe I admin servers?
Signature · 1 line
dann frazier
Re: [PATCH] udev: create empty regular files to represent net interfaces
Quoted 8 lines · dann frazier
Quoted 6 lines · Marco d'Itri
Quoted 3 lines · dann frazier
That would make a lot of users much happier (myself included), but it does restrict us into one view. At different times, admins think of their NICs by different properties. I may want to do IP assignment by
[Citation needed]
Is "I" not clear, or do you just not believe I admin servers?
I was referring to "At different times, admins think of their NICs by different properties", which I am not sure is a valid generalization.
Signature · 2 lines
ciao, Marco
Re: [PATCH] udev: create empty regular files to represent net interfaces
Quoted 5 lines · Marco d'Itri
Quoted 3 lines · dann frazier
Quoted 2 lines · Marco d'Itri
On Oct 29, dann frazier <[email protected]> wrote:
Quoted 7 lines
Quoted 5 lines
Quoted 4 lines
Quoted 3 lines
That would make a lot of users much happier (myself included), but it does restrict us into one view. At different times, admins think of their NICs by different properties. I may want to do IP assignment by
[Citation needed]
Is "I" not clear, or do you just not believe I admin servers?
I was referring to "At different times, admins think of their NICs by different properties", which I am not sure is a valid generalization.
Fair enough, I can't speak for others.
Signature · 1 line
dann frazier
RE: [PATCH] udev: create empty regular files to represent net interfaces
In reply to dann frazierQuoted 10 lines
Quoted 3 lines
This way the kernel has only one name, and so does userspace, and everyone is happy.
There are two issues, which really seem distinct to me.
Users expect eth0 to map to first-onboard-nic. That's an installer issue (since the BIOS can already export this info) and I agree that if we want to "fix" that, we should fix it there.
I agree that installers have to be fixed in the sense that they can be told to find the right interface. But, they expect determinism and depend on "eth0 to map to first-onboard-nic". Installer is one of the applications that is affected by this and needs user intervention, if it is not told about the right interface. I discussed installer as it is so much part of a user experience.
But the real issue is "eth0 does not map to first-onboard-nic" always and applications expecting this would break in data center environments. Both the solutions proposed provide a way to overcome it without introducing state.
With regards,
Narendra K
Re: [PATCH] udev: create empty regular files to represent net interfaces
Quoted 11 lines · On Fri, Oct 30, 2009 at 08:43:44PM +0530, [email protected] wrote:
Quoted 10 lines
Quoted 3 lines
This way the kernel has only one name, and so does userspace, and everyone is happy.
There are two issues, which really seem distinct to me.
Users expect eth0 to map to first-onboard-nic. That's an installer issue (since the BIOS can already export this info) and I agree that if we want to "fix" that, we should fix it there.
Quoted 6 lines
I agree that installers have to be fixed in the sense that they can be told to find the right interface. But, they expect determinism and depend on "eth0 to map to first-onboard-nic". Installer is one of the applications that is affected by this and needs user intervention, if it is not told about the right interface. I discussed installer as it is so much part of a user experience.
Right, but couldn't the installer do the work of scanning the SMBIOS to figure out which nics are onboard, and reorder the 'eth*' names such that these are first? This state could then be written out as udev rules so that they persist across reboots.
Quoted 8 lines
But the real issue is "eth0 does not map to first-onboard-nic" always and applications expecting this would break in data center environments. Both the solutions proposed provide a way to overcome it without introducing state.
With regards,
Narendra K
Signature · 1 line
dann frazier
RE: [PATCH] udev: create empty regular files to represent netinterfaces
Quoted 16 lines
Quoted 5 lines
Quoted 5 lines
There are two issues, which really seem distinct to me.
Users expect eth0 to map to first-onboard-nic. That's an installer issue (since the BIOS can already export this info) and I
agree that
Quoted 4 lines
Quoted 2 lines
if we want to "fix" that, we should fix it there.
I agree that installers have to be fixed in the sense that
they can be
Quoted 2 lines
told to find the right interface. But, they expect determinism and depend on "eth0 to map to first-onboard-nic". Installer is
one of the
Quoted 1 line
applications that is affected by this and needs user
intervention, if
Quoted 9 lines
Quoted 1 line
it is not told about the right interface. I discussed
installer as it
Quoted 1 line
is so much part of a user experience.
Right, but couldn't the installer do the work of scanning the SMBIOS to figure out which nics are onboard, and reorder the 'eth*' names such that these are first? This state could then be written out as udev rules so that they persist across reboots.
I suppose, with udev loading modules, the rules generated at runtime could run into the problem of duplicate names, if names are reordered in the kernel namespace. (I.e the eth* namespace). Hence idea of an alternate namespace.
With regards,
Narendra K
Re: [PATCH] udev: create empty regular files to represent netinterfaces
Quoted 26 lines · On Fri, Oct 30, 2009 at 10:23:57PM +0530, [email protected] wrote:
Quoted 16 lines
Quoted 5 lines
Quoted 5 lines
There are two issues, which really seem distinct to me.
Users expect eth0 to map to first-onboard-nic. That's an installer issue (since the BIOS can already export this info) and I
agree that
Quoted 4 lines
Quoted 2 lines
if we want to "fix" that, we should fix it there.
I agree that installers have to be fixed in the sense that
they can be
Quoted 2 lines
told to find the right interface. But, they expect determinism and depend on "eth0 to map to first-onboard-nic". Installer is
one of the
Quoted 1 line
applications that is affected by this and needs user
intervention, if
Quoted 9 lines
Quoted 1 line
it is not told about the right interface. I discussed
installer as it
Quoted 1 line
is so much part of a user experience.
Right, but couldn't the installer do the work of scanning the SMBIOS to figure out which nics are onboard, and reorder the 'eth*' names such that these are first? This state could then be written out as udev rules so that they persist across reboots.
Quoted 4 lines
I suppose, with udev loading modules, the rules generated at runtime could run into the problem of duplicate names, if names are reordered in the kernel namespace. (I.e the eth* namespace). Hence idea of an alternate namespace.
I don't see a risk of duplicate names - after all drivers are loaded, the installer can take the names enumerated by the kernel, figure out what it thinks a preferrable order is (i.e. by querying SMBIOS), then change the kernel names/mac mapping appropriately. Where can a duplicate name become an issue using this method?
Signature · 1 line
dann frazier
Re: [PATCH] udev: create empty regular files to represent net interfaces
In reply to dann frazierQuoted 19 lines · dann frazier
Quoted 11 lines · On Fri, Oct 30, 2009 at 08:43:44PM +0530, [email protected] wrote:
Quoted 10 lines
Quoted 3 lines
This way the kernel has only one name, and so does userspace, and everyone is happy.
There are two issues, which really seem distinct to me.
Users expect eth0 to map to first-onboard-nic. That's an installer issue (since the BIOS can already export this info) and I agree that if we want to "fix" that, we should fix it there.
Quoted 6 lines
I agree that installers have to be fixed in the sense that they can be told to find the right interface. But, they expect determinism and depend on "eth0 to map to first-onboard-nic". Installer is one of the applications that is affected by this and needs user intervention, if it is not told about the right interface. I discussed installer as it is so much part of a user experience.
Quoted 4 lines
Right, but couldn't the installer do the work of scanning the SMBIOS to figure out which nics are onboard, and reorder the 'eth*' names such that these are first? This state could then be written out as udev rules so that they persist across reboots.
No, there is a catch-22. To be sure you know the "proper" ethN name to assign a device based on an ordering, you have to know about all the devices. When udev runs, one device at a time, it can only see the current device and all those that have come before it, but it can't know when all the drivers for all the NICs have been loaded. And if you hotplug a device in later, it should presumably just go at the end of the list, but after a reboot, it'll most likely show up somewhere in the middle of the list. SMBIOS is static from boottime, not hotplug aware. If I add a 4-port NIC in slot 3 after boot, it becomes ethN..N+3. After reboot, it may well show up at completely different places, and even the N..N+3 ordering of individiual ports on the card aren't guaranteed to be consistent.
ethN is fundamentally a nondeterministic namespace, and trying to enforce determinism on it is, from all my attempts, impossible. Hence the desire to change the namespace. But there can be many different naming policies one might want (including the nondeterministic ethN policy), and for all other types of devices this isn't a problem - we can have all the policies we want, in parallel. Only for network devices we can't.
Stephen, I hadn't seen the ifalias field you added. I can see that being helpful to a user (some tool can write a more meaningful string to it), but I can't see it being useful programmatically. It still doesn't get me to "ifconfig the NIC in slot 3 port 2" or "ifconfig the NIC I booted from".
Thanks,
Matt
Signature · 3 lines
Matt Domsch Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux
Re: [PATCH] udev: create empty regular files to represent net interfaces
Quoted 7 lines · Matt Domsch
ethN is fundamentally a nondeterministic namespace, and trying to enforce determinism on it is, from all my attempts, impossible. Hence the desire to change the namespace. But there can be many different naming policies one might want (including the nondeterministic ethN policy), and for all other types of devices this isn't a problem - we can have all the policies we want, in parallel. Only for network devices we can't.
So pick one in your installer and stick with it. Doesn't seem that complicated to me...
greg k-h
Re: [PATCH] udev: create empty regular files to represent net interfaces
In reply to Kay SieversQuoted 33 lines · Kay Sievers
Quoted 14 lines · Matt Domsch
Quoted 5 lines · dann frazier
Here's a proof of concept to further the discussion..
The default filename uses the format:
/dev/netdev/by-ifindex/$ifindexQuoted 8 lines
This provides the infrastructure to permit udev rules to create aliases for network devices using symlinks, for example:
/dev/netdev/by-name/eth0-> ../by-ifindex/1
/dev/netdev/by-biosname/LOM0-> ../by-ifindex/3A library (such as the proposed libnetdevname) could use this information to provide an
alias->realnamemapping for network utilities.Quoted 7 lines
yes, this could work, as IFINDEX is already exported in the uevents, and that's the primary value udev needs to set up the mapping.
While I like the little ifindex2name script you've got, I think udev could simply call
if_indextoname()to get this, and not call an external program? I suppose it could be a really really simple external program too.What's the point of all this? Why would udev ever need to find the name of a device by the ifindex? The device name is the primary value for the kernel events udev acts on.
Quoted 6 lines
I'd be fine with this approach. It has the advantages of not requiring a kernel change at all, and not creating a whole character device which would be useless. And it doesn't preclude someone in the future from creating a char device for network devices should they so choose.
Quoted 9 lines
Quoted 1 line
Kay, what say you as udev owner?
That all sounds very much like something which will hit us back some day. I'm not sure, if udev should publish such dead text files in /dev, it does not seem to fit the usual APIs/assumptions where /sys and /dev match, and libudev provides access to both. It all sounds more like a database for a possible netdevname library, which does not need to be public in /dev, right?
And to throw in some bit of useless information;
I've pondered the idea of persistent device names for network interfaces a while back when designing the original layout of the persistent device naming scheme.
The one reason I didn't to this was that a network interface is _not_ a file, but rather an abstract type which is known only internally in the kernel (ie the one exemption from the 'everything is a file' UNIX rule).
And as udev is primarily concerned with the _files_, using it for network interfaces would be a workaround at best.
When I were to design this, I would be implementing network interface _aliases_, so that a network interface could be accessed either by name or by alias. This mechanism can then be managed by udev, much like we (ie SUSE) is using it nowadays to assign the network interface numbers.
But the mechanism for the aliases has to live in the same instance which also handles the network interface nowadays, ie the net subsystem of the kernel. Implementing this in udev is _not_ the way to go.
Cheers,
Hannes
Signature · 4 lines
Dr. Hannes Reinecke zSeries & Storage [email protected] +49 911 74053 688 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: Markus Rex, HRB 16746 (AG Nürnberg)
Re: [PATCH] udev: create empty regular files to represent net interfaces
Quoted 1 line · Hannes Reinecke
And to throw in some bit of useless information;
Stirring the pot a bit myself with this message...
Quoted 4 lines
The one reason I didn't to this was that a network interface is _not_ a file, but rather an abstract type which is known only internally in the kernel (ie the one exemption from the 'everything is a file' UNIX rule).
Why? Why not make it a file? I've heard rumors of other Unix-like systems that do exactly that, FWIW.
(Yes, I'm joking. Well, maybe half-joking... It'd be nice, but I don't expect it to happen.)
Quoted 5 lines
When I were to design this, I would be implementing network interface _aliases_, so that a network interface could be accessed either by name or by alias. This mechanism can then be managed by udev, much like we (ie SUSE) is using it nowadays to assign the network interface numbers.
The problem with that, if I understand what you're suggesting, is the value of IFNAMSIZ, and the fact that it can't be made any bigger. All your aliases have to be IFNAMSIZ characters or less. And that's too short to be able to embed the same level of information as we get for e.g. disks. It's *barely* long enough to fit "mac-" plus 12 hex digits (for the MAC address), but is completely incapable of holding a USB bus path, for instance.
(Not that you'd want to use path persistence for USB devices. But it is possible that you'd want it for some other setup, at which point it becomes impossible to use the same rules for USB.)
Re: [PATCH] udev: create empty regular files to represent net interfaces
Quoted 30 lines · On Fri, 30 Oct 2009 09:22:36 -0700 Bryan Kadzban <[email protected]>…
Quoted 1 line · Hannes Reinecke
And to throw in some bit of useless information;
Stirring the pot a bit myself with this message...
Quoted 4 lines
The one reason I didn't to this was that a network interface is _not_ a file, but rather an abstract type which is known only internally in the kernel (ie the one exemption from the 'everything is a file' UNIX rule).
Why? Why not make it a file? I've heard rumors of other Unix-like systems that do exactly that, FWIW.
(Yes, I'm joking. Well, maybe half-joking... It'd be nice, but I don't expect it to happen.)
Quoted 5 lines
When I were to design this, I would be implementing network interface _aliases_, so that a network interface could be accessed either by name or by alias. This mechanism can then be managed by udev, much like we (ie SUSE) is using it nowadays to assign the network interface numbers.
The problem with that, if I understand what you're suggesting, is the value of IFNAMSIZ, and the fact that it can't be made any bigger. All your aliases have to be IFNAMSIZ characters or less. And that's too short to be able to embed the same level of information as we get for e.g. disks. It's *barely* long enough to fit "mac-" plus 12 hex digits (for the MAC address), but is completely incapable of holding a USB bus path, for instance.
Quoted 3 lines
(Not that you'd want to use path persistence for USB devices. But it is possible that you'd want it for some other setup, at which point it becomes impossible to use the same rules for USB.)
Not a big fan of multiple names, it is the wrong solution to the human question of "what is eth0 really?". Router o/s use description field for that kind of information.
I added ifalias to provide place to put user visible description information. It is in latest kernels via sysfs and iproute utilities. Also plan to add support for it in net-snmp.
Quoted 15 lines · On Tue, Oct 13, 2009 at 10:43:49PM +0530, [email protected] wrote:
Quoted 5 lines
Quoted 4 lines
These device nodes are not functional at the moment -
open()returns-ENOSYS. Their only purpose is to provide userspace with a kernel name to ifindex mapping, in a form that udev can easily manage.Quoted 9 lines
If the idea is just to provide a userspace-visible mapping (and presumably take advantage of udev's infrastructure for naming) does this need kernel changes? Could this be a hierarchy under e.g.
/etc/udevinstead, using plain text files? It still means we need something like libnetdevname for apps to do the translation, but I'm not seeing why it matters how this map is stored. Is there some special property of the character devices (e.g. uevents) that we're not already getting with the existing interfaces?
Quoted 3 lines
Yes. The char device by itself doesn't help in any way. But it provides a flexible mechanism to provide multiple names for the same device, just the way it is for disks.
No, it's quite different than disks in that the symlinks, _and_ the device nodes do absolutly nothing. And any reference to a name that is a symlink will not work with any existing network tool, you will have to do some kind of lookup to determine which network device you really were referring to.
These links end up being useless, and confusing, I still don't see how you can use them for anything.
thanks,
greg k-h
We have developed a mapping library that will convert the user-friendly symlink names to the kernel names necessary for socket ioctls. All network tools that normally take ethX as argument have been modified to use this mapping library. Usually it's just a one-line addition when parsing the command line arguments.
--jordan hargrave
Dell Enterprise Linux Engineering
-----Original Message-----
From: Greg KH [mailto:[email protected]]
Sent: Tue 10/13/2009 14:51
To: K, Narendra
Cc: [email protected]; [email protected]; [email protected]; Domsch, Matt; Hargrave, Jordan; Rose, Charles
Subject: Re: PATCH: Network Device Naming mechanism and policy
Quoted 15 lines · On Tue, Oct 13, 2009 at 10:43:49PM +0530, [email protected] wrote:
Quoted 5 lines
Quoted 4 lines
These device nodes are not functional at the moment -
open()returns-ENOSYS. Their only purpose is to provide userspace with a kernel name to ifindex mapping, in a form that udev can easily manage.Quoted 9 lines
If the idea is just to provide a userspace-visible mapping (and presumably take advantage of udev's infrastructure for naming) does this need kernel changes? Could this be a hierarchy under e.g.
/etc/udevinstead, using plain text files? It still means we need something like libnetdevname for apps to do the translation, but I'm not seeing why it matters how this map is stored. Is there some special property of the character devices (e.g. uevents) that we're not already getting with the existing interfaces?
Quoted 3 lines
Yes. The char device by itself doesn't help in any way. But it provides a flexible mechanism to provide multiple names for the same device, just the way it is for disks.
No, it's quite different than disks in that the symlinks, _and_ the device nodes do absolutly nothing. And any reference to a name that is a symlink will not work with any existing network tool, you will have to do some kind of lookup to determine which network device you really were referring to.
These links end up being useless, and confusing, I still don't see how you can use them for anything.
thanks,
greg k-h
A: No.
Q: Should I include quotations after my reply?
http://daringfireball.net/2007/07/on_top
Quoted 5 lines · On Tue, Oct 13, 2009 at 03:00:59PM -0500, [email protected] wrote:
We have developed a mapping library that will convert the user-friendly symlink names to the kernel names necessary for socket ioctls. All network tools that normally take ethX as argument have been modified to use this mapping library. Usually it's just a one-line addition when parsing the command line arguments.
Either I missed this in the first message in this thread, or this was never stated before, but that is nice. Where is this library, and will it be accepted by the upstream tool maintainers?
thanks,
greg k-h
Quoted 9 lines · Greg KH
Quoted 5 lines · On Tue, Oct 13, 2009 at 03:00:59PM -0500, [email protected] wrote:
We have developed a mapping library that will convert the user-friendly symlink names to the kernel names necessary for socket ioctls. All network tools that normally take ethX as argument have been modified to use this mapping library. Usually it's just a one-line addition when parsing the command line arguments.
Either I missed this in the first message in this thread, or this was never stated before, but that is nice. Where is this library,
It was not noted in the initial patch post, but I did note it immediately thereafter.
Let me also note that we are prepared to have userspace consumers of this new character device node.
http://linux.dell.com/wiki/index.php/Oss/libnetdevname
notes how the kernel patch will interact with udev, describes the new library helper function in libnetdevname, and has patches for net-tools, iproute2, and ethtool to make use of the helper function.
As has been noted here, MAC addresses are not necessarily unique to an interface. As such, we are not proposing a net/by-mac/* symlink to /dev/netdev/*.
Quoted 1 line
and will it be accepted by the upstream tool maintainers?
Unknown, we haven't proposed it to any yet as it's irrelevant until there is general acceptance of the approach (kernel or otherwise). I figured we'd start with the kernel discussion, and show how it could be used.
Signature · 3 lines
Matt Domsch Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux
Quoted 7 lines · Greg KH
A: No.
Q: Should I include quotations after my reply?http://daringfireball.net/2007/07/on_top
On Tue, Oct 13, 2009 at 03:00:59PM -0500, [email protected] wrote:
Quoted 8 lines
Quoted 5 lines
We have developed a mapping library that will convert the user-friendly symlink names to the kernel names necessary for socket ioctls. All network tools that normally take ethX as argument have been modified to use this mapping library. Usually it's just a one-line addition when parsing the command line arguments.
Either I missed this in the first message in this thread, or this was never stated before, but that is nice. Where is this library,
I read about it here:
http://linux.dell.com/wiki/index.php/Oss/libnetdevname#libnetdevname
Source appears to be here:
http://linux.dell.com/git/?p=libnetdevname.git;a=summary
Quoted 2 lines
and will
it be accepted by the upstream tool maintainers?
Signature · 1 line
dann frazier