PCI: sort device lists breadth-first
Subsystems (2)
PATCH 1/1 PCI: sort device lists breadth-first
Problem:
New Dell PowerEdge servers have 2 embedded ethernet ports, which are labeled NIC1 and NIC2 on the chassis, in the BIOS setup screens, and in the printed documentation. Assuming no other add-in ethernet ports in the system, Linux 2.4 kernels name these eth0 and eth1 respectively. Many people have come to expect this naming. Linux 2.6 kernels name these eth1 and eth0 respectively (backwards from expectations). I also have reports that various Sun and HP servers have similar behavior.
Root cause:
Linux 2.4 kernels walk the pci_devices list, which happens to be sorted in breadth-first order (or pcbios_find_device order on i386, which most often is breadth-first also). 2.6 kernels have both the pci_devices list and the pci_bus_type.klist_devices list, the latter is what is walked at driver load time to match the pci_id tables; this klist happens to be in depth-first order.
On systems where, for physical routing reasons, NIC1 appears on a lower bus number than NIC2, but NIC2's bridge is discovered first in the depth-first ordering, NIC2 will be discovered before NIC1. If the list were sorted breadth-first, NIC1 woudl be discovered before NIC2.
A PowerEdge 1955 system has the following topology which easily exhibits the difference between depth-first and breadth-first device lists.
-[0000:00]-+-00.0 Intel Corporation 5000P Chipset Memory Controller Hub
+-02.0-[0000:03-08]--+-00.0-[0000:04-07]--+-00.0-[0000:05-06]----00.0-[0000:06]----00.0 Broadcom Corporation NetXtreme II BCM5708S Gigabit Ethernet (labeled NIC2, 2.4 kernel name eth1, 2.6 kernel name eth0)
+-1c.0-[0000:01-02]----00.0-[0000:02]----00.0 Broadcom Corporation NetXtreme II BCM5708S Gigabit Ethernet (labeled NIC1, 2.4 kernel name eth0, 2.6 kernel name eth1)Other factors, such as device driver load order and the presence of PCI slots at various points in the bus hierarchy further complicate this problem; I'm not trying to solve those here, just restore the device order, and thus basic behavior, that 2.4 kernels had.
Solution:
The solution can come in multiple steps.
Suggested fix #1: kernel
Patch below sorts the two device lists into breadth-first ordering to maintain compatibility with 2.4 kernels. It also overloads the 'pci=nosort' option to disable the breadth-first sort (and on i386 it continues to disable the pcibios_find_device sort as well).
Suggested fix #2: udev rules from userland
Many people also have the expectation that embedded NICs are always discovered before add-in NICs (which this patch does not try to do). Using the PCI IRQ Routing Table provided by system BIOS, it's easy to determine which PCI devices are embedded, or if add-in, which PCI slot they're in. I'm working on a tool that would allow udev to name ethernet devices in ascending embedded, slot 1 .. slot N order, subsort by PCI bus/dev/fn breadth-first. It'll be possible to use it independent of udev as well for those distributions that don't use udev in their installers.
Suggested fix #3: system board routing rules
One can constrain the system board layout to put NIC1 ahead of NIC2 regardless of breadth-first or depth-first discovery order. This adds a significant level of complexity to board routing, and may not be possible in all instances (witness the above systems from several major manufacturers). I don't want to encourage this particular train of thought too far, at the expense of not doing #1 or #2 above.
Feedback appreciated. Patch tested on a Dell PowerEdge 1955 blade with 2.6.18-rc5.
You'll also note I took some liberty and temporarily break the klist abstraction to simplify and speed up the sort algorithm. I think that's both safe and appropriate in this instance.
Thanks,
Matt
Signature · 4 lines
Matt Domsch Software Architect Dell Linux Solutions linux.dell.com & www.dell.com/linux Linux on Dell mailing lists @ http://lists.us.dell.com
Documentation/kernel-parameters.txt +5 -1
@@ -1186,7 +1186,11 @@ running once the system is up. nomsi [MSI] If the PCI_MSI kernel config parameter is enabled, this kernel boot option can be used to disable the use of MSI interrupts system-wide. - nosort [IA-32] Don't sort PCI devices according to + nosort Don't sort PCI devices into breadth-first order. + This sorting is done to get a device + order compatible with older (<= 2.4) kernels. + and + [IA-32] Don't sort PCI devices according to order given by the PCI BIOS. This sorting is done to get a device order compatible with older kernels.
arch/i386/pci/common.c +5 -3
@@ -189,6 +189,8 @@ static int __init pcibios_init(void)pcibios_resource_survey (); +if (!(pci_probe &PCI_NO_SORT )) +pci_sort_breadthfirst ();#ifdef CONFIG_PCI_BIOS if ((pci_probe &PCI_BIOS_SORT ) && !(pci_probe &PCI_NO_SORT ))pcibios_sort ();
@@ -203,6 +205,9 @@ char * __devinit pcibios_setup(char *stif (!strcmp (str,"off" )) { pci_probe =0 ;return NULL ; + }else if (!strcmp (str,"nosort" )) { + pci_probe |=PCI_NO_SORT ; +return NULL ; }#ifdef CONFIG_PCI_BIOS else if (!strcmp(str, "bios")) {
@@ -210,9 +215,6 @@ char * __devinit pcibios_setup(char *streturn NULL ; }else if (!strcmp (str,"nobios" )) { pci_probe &= ~PCI_PROBE_BIOS ; -return NULL ; - }else if (!strcmp (str,"nosort" )) { - pci_probe |=PCI_NO_SORT ;return NULL ; }else if (!strcmp (str,"biosirq" )) { pci_probe |=PCI_BIOS_IRQ_SCAN ;
arch/i386/pci/pci.h +0 -0
drivers/pci/probe.c +92 -0
95 lines not on this page · raw message
include/linux/pci.h +1 -0
@@ -437,6 +437,7 @@ extern void pci_dev_put(struct pci_dev *extern void pci_remove_bus (struct pci_bus *b);extern void pci_remove_bus_device (struct pci_dev *dev);void pci_setup_cardbus (struct pci_bus *bus); +extern void pci_sort_breadthfirst (void );/* Generic PCI functions exported to card drivers */
Quoted 9 lines · Matt Domsch
Problem:
New Dell PowerEdge servers have 2 embedded ethernet ports, which are labeled NIC1 and NIC2 on the chassis, in the BIOS setup screens, and in the printed documentation. Assuming no other add-in ethernet ports in the system, Linux 2.4 kernels name these eth0 and eth1 respectively. Many people have come to expect this naming. Linux 2.6 kernels name these eth1 and eth0 respectively (backwards from expectations). I also have reports that various Sun and HP servers have similar behavior.
This came up years back when 2.6 was something new, and the answer then was 'bind the interface to the MAC address'.
Whilst your patch will fix the case that's currently broken (2.4->2.6), doesn't it offer equal possibility to break existing setups when people move from <=2.6.18 -> 2.6.19 ?
Dave
Quoted 11 lines · Dave Jones
Quoted 9 lines · Matt Domsch
Problem:
New Dell PowerEdge servers have 2 embedded ethernet ports, which are labeled NIC1 and NIC2 on the chassis, in the BIOS setup screens, and in the printed documentation. Assuming no other add-in ethernet ports in the system, Linux 2.4 kernels name these eth0 and eth1 respectively. Many people have come to expect this naming. Linux 2.6 kernels name these eth1 and eth0 respectively (backwards from expectations). I also have reports that various Sun and HP servers have similar behavior.
Quoted 2 lines
This came up years back when 2.6 was something new, and the answer then was 'bind the interface to the MAC address'.
Both Red Hat-based distros and openSuSE-based distros do something like this with configuration files automatically. Red Hat's anaconda/kudzu puts the HWADDR lines in the generated /etc/sysconfig/network-scripts/ifcfg-* files. openSuSE's udev rules puts lines in /etc/udev/rules.d/30-net_persistent_names.rules the first time it discovers a new interface. Both methods are intended to maintain a persistent name for each NIC, after being set up the first time. Neither deals well with replacing one NIC with another - you must edit the config files.
This works pretty well post-install. It doesn't work well at install time, all the installers use the kernel's original names, and then those names become the persistent names in the config files.
Quoted 3 lines
Whilst your patch will fix the case that's currently broken (2.4->2.6), doesn't it offer equal possibility to break existing setups when people move from <=2.6.18 -> 2.6.19 ?
If they're using config files / udev rules as suggested, it shouldn't break them.
If they're not, then yes, this could. Debian's /etc/network/interfaces file allows use of hwaddr fields, though by default it doesn't appear anything sets it up.
I'm open to suggestions on how *not* to break setups that don't use the MAC addresses.
Thanks,
Matt
Signature · 4 lines
Matt Domsch Software Architect Dell Linux Solutions linux.dell.com & www.dell.com/linux Linux on Dell mailing lists @ http://lists.us.dell.com
Quoted 11 lines · Dave Jones
Quoted 9 lines · Matt Domsch
Problem:
New Dell PowerEdge servers have 2 embedded ethernet ports, which are labeled NIC1 and NIC2 on the chassis, in the BIOS setup screens, and in the printed documentation. Assuming no other add-in ethernet ports in the system, Linux 2.4 kernels name these eth0 and eth1 respectively. Many people have come to expect this naming. Linux 2.6 kernels name these eth1 and eth0 respectively (backwards from expectations). I also have reports that various Sun and HP servers have similar behavior.
Quoted 2 lines
This came up years back when 2.6 was something new, and the answer then was 'bind the interface to the MAC address'.
It came up then? I don't recall it, if so, I hope I didn't say that...
We should fix this, it's obviously a regression. Thanks a lot Matt for tracking this down and fixing it.
Quoted 3 lines
Whilst your patch will fix the case that's currently broken (2.4->2.6), doesn't it offer equal possibility to break existing setups when people move from <=2.6.18 -> 2.6.19 ?
It might, but I'll take that heat, we do have a command line option that returns the functionality to the "broken" way.
Let's see what happens in the next -mm.
thanks,
greg k-h
Quoted 37 lines · Matt Domsch
Quoted 11 lines · Dave Jones
Quoted 9 lines · Matt Domsch
Problem:
New Dell PowerEdge servers have 2 embedded ethernet ports, which are labeled NIC1 and NIC2 on the chassis, in the BIOS setup screens, and in the printed documentation. Assuming no other add-in ethernet ports in the system, Linux 2.4 kernels name these eth0 and eth1 respectively. Many people have come to expect this naming. Linux 2.6 kernels name these eth1 and eth0 respectively (backwards from expectations). I also have reports that various Sun and HP servers have similar behavior.Quoted 2 lines
This came up years back when 2.6 was something new, and the answer then was 'bind the interface to the MAC address'.
Both Red Hat-based distros and openSuSE-based distros do something like this with configuration files automatically. Red Hat's anaconda/kudzu puts the HWADDR lines in the generated
/etc/sysconfig/network-scripts/ifcfg-*files. openSuSE's udev rules puts lines in/etc/udev/rules.d/30-net_persistent_names.rulesthe first time it discovers a new interface. Both methods are intended to maintain a persistent name for each NIC, after being set up the first time. Neither deals well with replacing one NIC with another - you must edit the config files.This works pretty well post-install. It doesn't work well at install time, all the installers use the kernel's original names, and then those names become the persistent names in the config files.
Quoted 3 lines
Whilst your patch will fix the case that's currently broken (2.4->2.6), doesn't it offer equal possibility to break existing setups when people move from <=2.6.18 -> 2.6.19 ?
If they're using config files / udev rules as suggested, it shouldn't break them.
Quoted 6 lines
If they're not, then yes, this could. Debian's
/etc/network/interfacesfile allows use of hwaddr fields, though by default it doesn't appear anything sets it up.I'm open to suggestions on how *not* to break setups that don't use the MAC addresses.
to be honest I really don't like the PCI ordering change thing for this. It's just too fragile altogether to cause a fixed ordering as you want.
Maybe the kernel's initial ordering should do a numeric sort by mac address or something.. (or userspace should)
Quoted 12 lines · On Thu, 7 Sep 2006 22:14:22 -0500 Matt Domsch <[email protected]> wrote:
arch/i386/pci/common.c +2
-0@@ -189,6 +189,8 @@ static int __init pcibios_init(void)pcibios_resource_survey (); +if (!(pci_probe &PCI_NO_SORT )) +pci_sort_breadthfirst ();
drivers/pci/probe.c EXPORT_SYMBOL(pci_scan_bridge); · line 1064
Quoted 10 lines
drivers/pci/probe.c +7
-0@@ -1055,3 +1055,95 @@ EXPORT_SYMBOL(pci_scan_bridge);EXPORT_SYMBOL (pci_scan_single_device);EXPORT_SYMBOL_GPL (pci_scan_child_bus);#endif + +static int pci_sort_bf_cmp(const struct pci_dev *a, const struct pci_dev *b) +static void pci_insertion_sort_klist(struct pci_dev *a, struct list_head *list, +static void pci_sort_breadthfirst_klist(void) +static void pci_insertion_sort_devices(struct pci_dev *a, struct list_head *list, +static void pci_sort_breadthfirst_devices(void) +void pci_sort_breadthfirst(void)
I think all these functions can+should be __init?
drivers/pci/probe.c EXPORT_SYMBOL(pci_scan_bridge); · line 1065
Quoted 1 line
drivers/pci/probe.c +1
-0+extern void pci_sort_breadthfirst (void );
In which case this needs the __init tag too (new rule, due to frv (at least)).
Quoted 28 lines · Andrew Morton
Quoted 12 lines · On Thu, 7 Sep 2006 22:14:22 -0500 Matt Domsch <[email protected]> wrote:
arch/i386/pci/common.c +2
-0@@ -189,6 +189,8 @@ static int __init pcibios_init(void)pcibios_resource_survey (); +if (!(pci_probe &PCI_NO_SORT )) +pci_sort_breadthfirst ();drivers/pci/probe.c EXPORT_SYMBOL(pci_scan_bridge); · line 1064
Quoted 10 lines
drivers/pci/probe.c +7
-0@@ -1055,3 +1055,95 @@ EXPORT_SYMBOL(pci_scan_bridge);EXPORT_SYMBOL (pci_scan_single_device);EXPORT_SYMBOL_GPL (pci_scan_child_bus);#endif + +static int pci_sort_bf_cmp(const struct pci_dev *a, const struct pci_dev *b) +static void pci_insertion_sort_klist(struct pci_dev *a, struct list_head *list, +static void pci_sort_breadthfirst_klist(void) +static void pci_insertion_sort_devices(struct pci_dev *a, struct list_head *list, +static void pci_sort_breadthfirst_devices(void) +void pci_sort_breadthfirst(void)I think all these functions can+should be
__init?
Quoted 3 lines
drivers/pci/probe.c EXPORT_SYMBOL(pci_scan_bridge); · line 1065
Quoted 1 line
drivers/pci/probe.c +1
-0+extern void pci_sort_breadthfirst (void );In which case this needs the
__inittag too (new rule, due to frv (at least)).
Will fix up immediately. I also see that it's making indirect calls through the (*cmp)() pointer, which is interesting only of there's other code that's going to be doing insertion sorting. For now, a direct call to the comparison function would be a little cleaner.
Thanks for the review!
Matt
Signature · 4 lines
Matt Domsch Software Architect Dell Linux Solutions linux.dell.com & www.dell.com/linux Linux on Dell mailing lists @ http://lists.us.dell.com
Quoted 3 lines · Andrew Morton
Quoted 1 line
+extern void
pci_sort_breadthfirst(void);In which case this needs the
__inittag too (new rule, due to frv (at least)).
Some time ago I've asked David Howells about it:
> Does this apply to function prototypes?
No, because functions can't be crammed into such a small space.
I wanted to note what Martin Mares just raised in the thread "State of the Linux PCI Subsystem for 2.6.18-rc6":
Quoted 8 lines
Changing the device order in the middle of the 2.6 cycle doesn't sound like a sane idea to me. Many people have changed their systems' configuration to adapt to the 2.6 ordering and this patch would break their setups. I have seen many such examples in my vicinity.
I believe that not breaking existing 2.6 setups is much more important than keeping compatibility with 2.4 kernels, especially when the problem is discovered after more than 2 years after release of the first 2.6.
I'm not siding with either Martin or Matt explicitly, just noting that there are good arguments for both sides.
Jeff
Hi!
Quoted 19 lines
I wanted to note what Martin Mares just raised in the thread "State of the Linux PCI Subsystem for 2.6.18-rc6":
Quoted 8 lines
Changing the device order in the middle of the 2.6 cycle doesn't sound
like a sane idea to me. Many people have changed their systems' configuration
to adapt to the 2.6 ordering and this patch would break their setups.
I have seen many such examples in my vicinity.Quoted 6 lines
I believe that not breaking existing 2.6 setups is much more important
than keeping compatibility with 2.4 kernels, especially when the problem
is discovered after more than 2 years after release of the first 2.6.
Quoted 2 lines
I'm not siding with either Martin or Matt explicitly, just noting that there are good arguments for both sides.
I agree with martin here. 'Lets break all the machines where people are currently using 2.6.x, for benefit of people currently running 2.4.x' is *very* bad idea.
Pavel
Signature · 1 line
Thanks for all the (sleeping) penguins.
Quoted 3 lines · Pavel Machek
I agree with martin here. 'Lets break all the machines where people are currently using 2.6.x, for benefit of people currently running 2.4.x' is *very* bad idea.
Small correction: Not all but "only" setups which rely on device ordering instead of persistent unique device properties are affected.
Signature · 3 lines
Stefan Richter -=====-=-==- =--= -=-=- http://arcgraph.de/sr/
Quoted 9 lines · Jeff Garzik
I wanted to note what Martin Mares just raised in the thread "State of the Linux PCI Subsystem for 2.6.18-rc6":
Quoted 6 lines
Changing the device order in the middle of the 2.6 cycle doesn't sound like a sane idea to me. Many people have changed their systems' configuration
to adapt to the 2.6 ordering and this patch would break their setups. I have seen many such examples in my vicinity.
Quoted 7 lines
Quoted 3 lines
I believe that not breaking existing 2.6 setups is much more important than keeping compatibility with 2.4 kernels, especially when the problem is discovered after more than 2 years after release of the first 2.6.
I'm not siding with either Martin or Matt explicitly, just noting that there are good arguments for both sides.
I agree, there are.
But I know we explicitly tried to keep 2.4 compability in 2.6 for this very thing. And the fact that almost no one has reported having problems with it, leads me to believe that the sorting order isn't as broken as you might think. Some machines, like the ones that Matt wrote this patch for, need this change, but for all others, it's not really needed.
Let's let this sit in -mm for a bit to see if anyone notices any problems.
thanks,
greg k-h
Quoted 40 lines · Arjan van de Ven
Quoted 39 lines · Matt Domsch
Quoted 11 lines · Dave Jones
Quoted 9 lines · Matt Domsch
Problem:
New Dell PowerEdge servers have 2 embedded ethernet ports, which are labeled NIC1 and NIC2 on the chassis, in the BIOS setup screens, and in the printed documentation. Assuming no other add-in ethernet ports in the system, Linux 2.4 kernels name these eth0 and eth1 respectively. Many people have come to expect this naming. Linux 2.6 kernels name these eth1 and eth0 respectively (backwards from expectations). I also have reports that various Sun and HP servers have similar behavior.Quoted 2 lines
This came up years back when 2.6 was something new, and the answer then was 'bind the interface to the MAC address'.
Both Red Hat-based distros and openSuSE-based distros do something like this with configuration files automatically. Red Hat's anaconda/kudzu puts the HWADDR lines in the generated
/etc/sysconfig/network-scripts/ifcfg-*files. openSuSE's udev rules puts lines in/etc/udev/rules.d/30-net_persistent_names.rulesthe first time it discovers a new interface. Both methods are intended to maintain a persistent name for each NIC, after being set up the first time. Neither deals well with replacing one NIC with another - you must edit the config files.This works pretty well post-install. It doesn't work well at install time, all the installers use the kernel's original names, and then those names become the persistent names in the config files.
Quoted 3 lines
Whilst your patch will fix the case that's currently broken (2.4->2.6), doesn't it offer equal possibility to break existing setups when people move from <=2.6.18 -> 2.6.19 ?
If they're using config files / udev rules as suggested, it shouldn't break them.
If they're not, then yes, this could. Debian's
/etc/network/interfacesfile allows use of hwaddr fields, though by default it doesn't appear anything sets it up.
Quoted 10 lines
Quoted 2 lines
I'm open to suggestions on how *not* to break setups that don't use the MAC addresses.
to be honest I really don't like the PCI ordering change thing for this. It's just too fragile altogether to cause a fixed ordering as you want.
Maybe the kernel's initial ordering should do a numeric sort by mac address or something.. (or userspace should)
That wouldn't match any existing setup, and would be subject to mid-list insertions if a NIC were added/replaced. And that is fragile.
I was looking for an easy way to do PCI slot to MAC, and from there MAC to IP, so any NIC plugged into a given slot could be called eth0 (for instance) and given the "right" IP address, but that's not easy. Can be done with some searching in /sys, but it's non-trivial.
Signature · 5 lines
Bill Davidsen <[email protected]> Obscure bug of 2004: BASH BUFFER OVERFLOW - if bash is being run by a normal user and is setuid root, with the "vi" line edit mode selected, and the character set is "big5," an off-by-one errors occurs during wildcard (glob) expansion.
Quoted 11 lines · Bill Davidsen
Quoted 3 lines · Arjan van de Ven
Maybe the kernel's initial ordering should do a numeric sort by mac address or something.. (or userspace should)
That wouldn't match any existing setup, and would be subject to mid-list insertions if a NIC were added/replaced. And that is fragile.
I was looking for an easy way to do PCI slot to MAC, and from there MAC to IP, so any NIC plugged into a given slot could be called eth0 (for instance) and given the "right" IP address, but that's not easy. Can be done with some searching in /sys, but it's non-trivial.
So, I did almost this in
userspace. http://linux.dell.com/files/name_eths/. It uses the PCI IRQ Routing table to determine if a PCI device is embedded on the motherboard, or is in an add-in slot, and if so, which slot. It orders the list of thereby possible PCI NICs with all the embeddeds first, in ascending PCI breadth-first order, then orders all the add-in NICs in PCI slot number ascending order, subsort PCI breadth-first for those nifty multiport cards. It rewrites modprobe.conf to load network drivers 'proper' order and outputs an /etc/mactab file that can be used by the second half of the script to write HWADDR lines into Red Hat-style ifcfg-eth* files, and into openSuSE udev ethernet rules file.
This works great, until you add another NIC into an add-in slot somewhere in the middle (e.g. you have 2 embedded NICs eth0 and eth1, and a NIC card in PCI slot 4 eth2, then at some later point you add a NIC card into PCI slot 2). You either have to manually configure a name for the new card, or run name_eths again and expect the NIC in PCI slot 2 to become eth2, and the one in slot4 to become eth3.
The pure C udev helper I'm working on behaves similarly, though would negate the need to edit config files, as it assigns a name "on the fly" at device discovery time.
For the relatively rare cases of adding a NIC, I'm OK with this. I don't have a better way to handle it, but am open to ideas.
We could assign names like eth-embedded-1, eth-embedded-2, eth-slot2-1, eth-slot4-1 if we wanted to change how people think of ethernet names (and this would be similar to how large network switches work: blade N, port M. We've got 15 usable chars in the name after all...
Thanks,
Matt
Signature · 4 lines
Matt Domsch Software Architect Dell Linux Solutions linux.dell.com & www.dell.com/linux Linux on Dell mailing lists @ http://lists.us.dell.com
Quoted 17 lines · Matt Domsch
Quoted 13 lines · Bill Davidsen
Quoted 5 lines · Arjan van de Ven
Maybe the kernel's initial ordering should do a numeric sort by mac address or something.. (or userspace should)
That wouldn't match any existing setup, and would be subject to mid-list insertions if a NIC were added/replaced. And that is fragile.
I was looking for an easy way to do PCI slot to MAC, and from there MAC to IP, so any NIC plugged into a given slot could be called eth0 (for instance) and given the "right" IP address, but that's not easy. Can be done with some searching in /sys, but it's non-trivial.
Quoted 14 lines
So, I did almost this in
userspace. http://linux.dell.com/files/name_eths/. It uses the PCI IRQ Routing table to determine if a PCI device is embedded on the motherboard, or is in an add-in slot, and if so, which slot. It orders the list of thereby possible PCI NICs with all the embeddeds first, in ascending PCI breadth-first order, then orders all the add-in NICs in PCI slot number ascending order, subsort PCI breadth-first for those nifty multiport cards. It rewrites modprobe.conf to load network drivers 'proper' order and outputs an/etc/mactabfile that can be used by the second half of the script to write HWADDR lines into Red Hat-style ifcfg-eth* files, and into openSuSE udev ethernet rules file.
I am not (generally) concerned with the embedded NICs, it's the slots I have had bite me. I do see what you're doing, although I didn't do it quite the same way.
Quoted 8 lines
This works great, until you add another NIC into an add-in slot somewhere in the middle (e.g. you have 2 embedded NICs eth0 and eth1, and a NIC card in PCI slot 4 eth2, then at some later point you add a NIC card into PCI slot 2). You either have to manually configure a name for the new card, or run
name_ethsagain and expect the NIC in PCI slot 2 to become eth2, and the one in slot4 to become eth3.
And this is why I want to do a slot to name translation, which may result in gaps in the names. That doesn't seem to matter. Then I can put a label over the slot, like "public" or "private" and know that adding another NIC, or replacing a failed NIC will leave my labels intact.
Quoted 8 lines
The pure C udev helper I'm working on behaves similarly, though would negate the need to edit config files, as it assigns a name "on the fly" at device discovery time.
For the relatively rare cases of adding a NIC, I'm OK with this. I don't have a better way to handle it, but am open to ideas.
You have mine. I tried putting labels on the NIC, but the machine go to live timezones away and repairs are done by whoever has the service contract.
Quoted 6 lines
We could assign names like eth-embedded-1, eth-embedded-2, eth-slot2-1, eth-slot4-1 if we wanted to change how people think of ethernet names (and this would be similar to how large network switches work: blade N, port M. We've got 15 usable chars in the name after all...
Opens the door for change, for sure. Since I have had to use multi-drop NICs and nameset like e_slot1_jack2 is an interesting concept. Or perhaps just append the jack index no matter how many connections it has. So eth2-0 would be a NIC and socket. Then when I run out of slots and have to go to a two or four post card, the primary name wouldn't change.
Your idea has just enough usefulness to be dangerous ;-) Thanks.
Signature · 3 lines
bill davidsen <[email protected]> CTO TMR Associates, Inc Doing interesting things with small computers since 1979
Quoted 6 lines · Matt Domsch
Problem:
Many people have come to expect this naming. Linux 2.6 kernels name these eth1 and eth0 respectively (backwards from expectations). I also have reports that various Sun and HP servers have similar behavior.
On RHEL3 the 32bit and 64bit versions order the NICs differently. 64bit RHEL3 orders it the same as 2.6.
I've got a lot of systems where the NIC LEDs are labelled. The labels are correct for 2.6 but not for 2.4 32 bit. I'm suspect the labels were designed for Windows originally.
My boss pointed out that this patch. It was supposed to make PCI bus ordering match 2.4.
http://kernel.org/git/?p=linux/kernel/git/torvalds/old-2.6-bkcvs.git;a=commitdiff;h=ffdd6e8f870ca6dd0d9b9169b8c2e0fdbae99549
It's still there, why did it stop working?
Couldn't we just use the labelling from the DMI data to order the NICs?
regards,
dan carpenter
Quoted 8 lines · Dan Carpenter
Quoted 6 lines · Matt Domsch
Problem:
Many people have come to expect this naming. Linux 2.6 kernels name these eth1 and eth0 respectively (backwards from expectations). I also have reports that various Sun and HP servers have similar behavior.
Quoted 6 lines
On RHEL3 the 32bit and 64bit versions order the NICs differently. 64bit RHEL3 orders it the same as 2.6.
I've got a lot of systems where the NIC LEDs are labelled. The labels are correct for 2.6 but not for 2.4 32 bit. I'm suspect the labels were designed for Windows originally.
2.4 i386 by default resorts the list by what BIOS reports the order should be. No other arches do this. So I'd expect it to be opposite of what you say.
Care to send the output of 'lspci -tv' and the first 80 or so lines of dmidecode? This is curious.
Quoted 6 lines
My boss pointed out that this patch. It was supposed to make PCI bus ordering match 2.4.
http://kernel.org/git/?p=linux/kernel/git/torvalds/old-2.6-bkcvs.git;a=commitdiff;h=ffdd6e8f870ca6dd0d9b9169b8c2e0fdbae99549
It's still there, why did it stop working?Couldn't we just use the labelling from the DMI data to order the NICs?
Unfortunately, DMI data doesn't include enough information. It says "there's a port called NIC1", but doesn't say where to find it in PCI space. :-( I'm looking at ACPI 3.0 extensions, but am not finding what I'm needing here either yet.
Signature · 4 lines
Matt Domsch Software Architect Dell Linux Solutions linux.dell.com & www.dell.com/linux Linux on Dell mailing lists @ http://lists.us.dell.com
Quoted 16 lines · Matt Domsch
Quoted 8 lines · Dan Carpenter
Quoted 6 lines · Matt Domsch
Problem:
Many people have come to expect this naming. Linux 2.6 kernels name these eth1 and eth0 respectively (backwards from expectations). I also have reports that various Sun and HP servers have similar behavior.Quoted 6 lines
On RHEL3 the 32bit and 64bit versions order the NICs differently. 64bit RHEL3 orders it the same as 2.6.
I've got a lot of systems where the NIC LEDs are labelled. The labels are correct for 2.6 but not for 2.4 32 bit. I'm suspect the labels were designed for Windows originally.
Quoted 7 lines
2.4 i386 by default resorts the list by what BIOS reports the order should be. No other arches do this. So I'd expect it to be opposite of what you say.
Care to send the output of 'lspci -tv' and the first 80 or so lines of dmidecode? This is curious.
On this system the LEDs match the BIOS and the PXE boot order and the 2.6 ordering.
-+-[06]-+-01.0-[07]--
| +-01.1 Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC
| +-02.0-[08]--
| \-02.1 Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC
\-[00]-+-00.0 nVidia Corporation CK804 Memory Controller
+-01.0 nVidia Corporation CK804 ISA Bridge
+-01.1 nVidia Corporation CK804 SMBus
+-02.0 nVidia Corporation CK804 USB Controller
+-02.1 nVidia Corporation CK804 USB Controller
+-06.0 nVidia Corporation CK804 IDE
+-07.0 nVidia Corporation CK804 Serial ATA Controller
+-08.0 nVidia Corporation CK804 Serial ATA Controller
+-09.0-[01]----07.0 ATI Technologies Inc Rage XL
+-0b.0-[02]----00.0 Broadcom Corporation NetXtreme BCM5721
Gigabit Ethernet PCI Express
+-0c.0-[03]--
+-0d.0-[04]----00.0 Broadcom Corporation NetXtreme BCM5721
Gigabit Ethernet PCI Express
+-0e.0-[05]--
+-18.0 Advanced Micro Devices [AMD] K8 [Athlon64/Opteron]
HyperTransport Technology Configuration
+-18.1 Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
+-18.2 Advanced Micro Devices [AMD] K8 [Athlon64/Opteron]DRAM Controller
+-18.3 Advanced Micro Devices [AMD] K8 [Athlon64/Opteron]
Miscellaneous Control
+-19.0 Advanced Micro Devices [AMD] K8 [Athlon64/Opteron]
HyperTransport Technology Configuration
+-19.1 Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
+-19.2 Advanced Micro Devices [AMD] K8 [Athlon64/Opteron]DRAM Controller
\-19.3 Advanced Micro Devices [AMD] K8 [Athlon64/Opteron]
Miscellaneous Control
# dmidecode 2.2 SMBIOS 2.3 present. 54 structures occupying 2065 bytes. Table at 0x000FA560. Handle 0x0000 DMI type 0, 20 bytes. BIOS Information Vendor: IR2300 Version: BIOS Version 1.50 Date: 03/17/06 13:23:14 Release Date: <BAD INDEX> Address: 0xF0000 Runtime Size: 64 kB ROM Size: 1024 kB Characteristics: ISA is supported PCI is supported PNP is supported BIOS is upgradeable BIOS shadowing is allowed ESCD support is available Boot from CD is supported Selectable boot is supported BIOS ROM is socketed EDD is supported 3.5"/2.88 MB floppy services are supported (int 13h) Print screen service is supported (int 5h) 8042 keyboard services are supported (int 9h) Serial services are supported (int 14h) CGA/mono video services are supported (int 10h) ACPI is supported USB legacy is supported LS-120 boot is supported ATAPI Zip drive boot is supported BIOS boot specification is supported Function key-initiated network boot is supported Handle 0x0001 DMI type 1, 25 bytes. System Information Manufacturer: To Be Filled By O.E.M. Product Name: IR2300 Version: To Be Filled By O.E.M. Serial Number: To Be Filled By O.E.M. UUID: 00020003-0004-0005-0006-000700080009 Wake-up Type: Power Switch Handle 0x0002 DMI type 2, 8 bytes. Base Board Information Manufacturer: InventecESC Product Name: Tuna Version: To be filled by O.E.M. Serial Number: To be filled by O.E.M. Handle 0x0003 DMI type 3, 17 bytes. Chassis Information Manufacturer: InventecESC Type: Hand Held Lock: Not Present Version: To Be Filled By O.E.M. Serial Number: To Be Filled By O.E.M. Asset Tag: To Be Filled By O.E.M. Boot-up State: Safe Power Supply State: Safe Thermal State: Safe Security Status: None OEM Information: 0x00000000
regards,
dan carpenter