igbvf: avoid name clash between PF and VF
Subsystems (2)
PATCH 1/1 igbvf: avoid name clash between PF and VF
From: Stefan Assmann <[email protected]>
It looks like the VFs get initialized before all the PFs are. Therefore the udev mapping MAC <-> ethX (for PFs) gets screwed because the VFs may grab the ethX interface names (reserved by udev) for the PFs.
Example:
igb max_vfs=0
eth0 Link encap:Ethernet HWaddr 00:13:20:F7:A5:9E eth1 Link encap:Ethernet HWaddr 00:13:20:F7:A5:9F eth2 Link encap:Ethernet HWaddr 00:13:20:F7:A5:A0 eth3 Link encap:Ethernet HWaddr 00:13:20:F7:A5:A1 igb max_vfs=1 eth0 Link encap:Ethernet HWaddr 00:13:20:F7:A5:9E eth1 Link encap:Ethernet HWaddr 0A:CF:41:69:F7:A9 eth2 Link encap:Ethernet HWaddr 3A:FE:20:4C:2A:3B eth3 Link encap:Ethernet HWaddr C6:C3:B1:56:C9:A4
eth3_rename Link encap:Ethernet HWaddr 00:13:20:F7:A5:9F
eth4 Link encap:Ethernet HWaddr 6E:8A:8A:A3:5F:69eth4_rename Link encap:Ethernet HWaddr 00:13:20:F7:A5:A0 eth5_rename Link encap:Ethernet HWaddr 00:13:20:F7:A5:A1
In the example above VF 0A:CF:41:69:F7:A9 grabs eth1 but udev has a rule that says eth1 should be assigned PF 00:13:20:F7:A5:9F (eth3_rename) and waits for the VF to disappear to rename eth3_rename to eth1. Unfortunately eth1 is not going to disappear. This is not a udev bug since udev doesn't create persistent rules for VFs as their MAC address changes every reboot.
To avoid this problem we could change the kernel name for the VFs and thus avoid confusion between VFs and PFs.
I've already discussed this with Alexander Duyck and Greg Rose, so far they have no objection. However this problem appears for all drivers that support PFs and VFs and thus the changes should be applied consistently to all of these drivers.
--- drivers/net/igbvf/netdev.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
drivers/net/igbvf/netdev.c +1 -1
@@ -2787,7 +2787,7 @@ static int __devinit igbvf_probe(struct pci_dev *pdev,netif_carrier_off (netdev);netif_stop_queue (netdev); -strcpy (netdev->name,"eth%d" ); +strcpy (netdev->name,"veth%d" ); err =register_netdev (netdev);if (err) goto err_hw_init;
Signature · 12 lines
1.6.5.2 ------------------------------------------------------------------------------ This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first _______________________________________________ E1000-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
Quoted 22 lines · Stefan Assmann
From: Stefan Assmann <[email protected]>
It looks like the VFs get initialized before all the PFs are. Therefore the udev mapping MAC <-> ethX (for PFs) gets screwed because the VFs may grab the ethX interface names (reserved by udev) for the PFs.
Example:
igb max_vfs=0eth0 Link encap:Ethernet HWaddr 00:13:20:F7:A5:9E eth1 Link encap:Ethernet HWaddr 00:13:20:F7:A5:9F eth2 Link encap:Ethernet HWaddr 00:13:20:F7:A5:A0 eth3 Link encap:Ethernet HWaddr 00:13:20:F7:A5:A1 igb max_vfs=1 eth0 Link encap:Ethernet HWaddr 00:13:20:F7:A5:9E eth1 Link encap:Ethernet HWaddr 0A:CF:41:69:F7:A9 eth2 Link encap:Ethernet HWaddr 3A:FE:20:4C:2A:3B eth3 Link encap:Ethernet HWaddr C6:C3:B1:56:C9:A4
eth3_renameLink encap:Ethernet HWaddr 00:13:20:F7:A5:9F
eth4 Link encap:Ethernet HWaddr 6E:8A:8A:A3:5F:69eth4_renameLink encap:Ethernet HWaddr 00:13:20:F7:A5:A0eth5_renameLink encap:Ethernet HWaddr 00:13:20:F7:A5:A1
Quoted 6 lines
In the example above VF 0A:CF:41:69:F7:A9 grabs eth1 but udev has a rule that says eth1 should be assigned PF 00:13:20:F7:A5:9F (
eth3_rename) and waits for the VF to disappear to renameeth3_renameto eth1. Unfortunately eth1 is not going to disappear. This is not a udev bug since udev doesn't create persistent rules for VFs as their MAC address changes every reboot.
[...]
I think it is a bug in the udev rules: udev should rename the VFs even though their names won't be persistent.
Ben.
Signature · 3 lines
Ben Hutchings, Senior Software Engineer, Solarflare Communications Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
Quoted 29 lines · Ben Hutchings
Quoted 22 lines · Stefan Assmann
From: Stefan Assmann <[email protected]>
It looks like the VFs get initialized before all the PFs are. Therefore the udev mapping MAC <-> ethX (for PFs) gets screwed because the VFs may grab the ethX interface names (reserved by udev) for the PFs.
Example:
igb max_vfs=0eth0 Link encap:Ethernet HWaddr 00:13:20:F7:A5:9E eth1 Link encap:Ethernet HWaddr 00:13:20:F7:A5:9F eth2 Link encap:Ethernet HWaddr 00:13:20:F7:A5:A0 eth3 Link encap:Ethernet HWaddr 00:13:20:F7:A5:A1 igb max_vfs=1 eth0 Link encap:Ethernet HWaddr 00:13:20:F7:A5:9E eth1 Link encap:Ethernet HWaddr 0A:CF:41:69:F7:A9 eth2 Link encap:Ethernet HWaddr 3A:FE:20:4C:2A:3B eth3 Link encap:Ethernet HWaddr C6:C3:B1:56:C9:A4
eth3_renameLink encap:Ethernet HWaddr 00:13:20:F7:A5:9F
eth4 Link encap:Ethernet HWaddr 6E:8A:8A:A3:5F:69eth4_renameLink encap:Ethernet HWaddr 00:13:20:F7:A5:A0eth5_renameLink encap:Ethernet HWaddr 00:13:20:F7:A5:A1Quoted 6 lines
In the example above VF 0A:CF:41:69:F7:A9 grabs eth1 but udev has a rule that says eth1 should be assigned PF 00:13:20:F7:A5:9F (
eth3_rename) and waits for the VF to disappear to renameeth3_renameto eth1. Unfortunately eth1 is not going to disappear. This is not a udev bug since udev doesn't create persistent rules for VFs as their MAC address changes every reboot.
Quoted 4 lines
[...]
I think it is a bug in the udev rules: udev should rename the VFs even though their names won't be persistent.
In that case let's Cc udev people.
Stefan
--
Stefan Assmann | Red Hat GmbH
Software Engineer | Otto-Hahn-Strasse 20, 85609 Dornach
| HR: Amtsgericht Muenchen HRB 153243
| GF: Brendan Lane, Charlie Peters,
sassmann at redhat.com | Michael Cunningham, Charles Cachera------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
E1000-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
Quoted 30 lines · Stefan Assmann
Quoted 29 lines · Ben Hutchings
Quoted 28 lines · Stefan Assmann
From: Stefan Assmann <[email protected]>
It looks like the VFs get initialized before all the PFs are. Therefore the udev mapping MAC <-> ethX (for PFs) gets screwed because the VFs may grab the ethX interface names (reserved by udev) for the PFs.
Example:
igb max_vfs=0
eth0 Link encap:Ethernet HWaddr 00:13:20:F7:A5:9E eth1 Link encap:Ethernet HWaddr 00:13:20:F7:A5:9F eth2 Link encap:Ethernet HWaddr 00:13:20:F7:A5:A0 eth3 Link encap:Ethernet HWaddr 00:13:20:F7:A5:A1 igb max_vfs=1
eth0 Link encap:Ethernet HWaddr 00:13:20:F7:A5:9E eth1 Link encap:Ethernet HWaddr 0A:CF:41:69:F7:A9 eth2 Link encap:Ethernet HWaddr 3A:FE:20:4C:2A:3B eth3 Link encap:Ethernet HWaddr C6:C3:B1:56:C9:A4eth3_renameLink encap:Ethernet HWaddr 00:13:20:F7:A5:9F eth4 Link encap:Ethernet HWaddr 6E:8A:8A:A3:5F:69eth4_renameLink encap:Ethernet HWaddr 00:13:20:F7:A5:A0eth5_renameLink encap:Ethernet HWaddr 00:13:20:F7:A5:A1In the example above VF 0A:CF:41:69:F7:A9 grabs eth1 but udev has a rule that says eth1 should be assigned PF 00:13:20:F7:A5:9F (
eth3_rename) and waits for the VF to disappear to renameeth3_renameto eth1. Unfortunately eth1 is not going to disappear. This is not a udev bug since udev doesn't create persistent rules for VFs as their MAC address changes every reboot.
Quoted 4 lines
Quoted 4 lines
[...]
I think it is a bug in the udev rules: udev should rename the VFs even though their names won't be persistent.
Udev writes out these configs to a rules file, and therefore can never handle random MAC addresses, as they would just accumulate in the rules file with a new entry at every bootup.
Stuff like this is just not supported at the moment with the rather simple logic it has, and there is no current plan/idea, or anybody working on changing/improving this at the moment.
Kay
Quoted 40 lines · Kay Sievers
Quoted 30 lines · Stefan Assmann
Quoted 29 lines · Ben Hutchings
Quoted 28 lines · Stefan Assmann
From: Stefan Assmann<[email protected]>
It looks like the VFs get initialized before all the PFs are. Therefore the udev mapping MAC<-> ethX (for PFs) gets screwed because the VFs may grab the ethX interface names (reserved by udev) for the PFs.
Example:
igb max_vfs=0eth0 Link encap:Ethernet HWaddr 00:13:20:F7:A5:9E eth1 Link encap:Ethernet HWaddr 00:13:20:F7:A5:9F eth2 Link encap:Ethernet HWaddr 00:13:20:F7:A5:A0 eth3 Link encap:Ethernet HWaddr 00:13:20:F7:A5:A1 igb max_vfs=1 eth0 Link encap:Ethernet HWaddr 00:13:20:F7:A5:9E eth1 Link encap:Ethernet HWaddr 0A:CF:41:69:F7:A9 eth2 Link encap:Ethernet HWaddr 3A:FE:20:4C:2A:3B eth3 Link encap:Ethernet HWaddr C6:C3:B1:56:C9:A4
eth3_renameLink encap:Ethernet HWaddr 00:13:20:F7:A5:9F
eth4 Link encap:Ethernet HWaddr 6E:8A:8A:A3:5F:69eth4_renameLink encap:Ethernet HWaddr 00:13:20:F7:A5:A0eth5_renameLink encap:Ethernet HWaddr 00:13:20:F7:A5:A1In the example above VF 0A:CF:41:69:F7:A9 grabs eth1 but udev has a rule that says eth1 should be assigned PF 00:13:20:F7:A5:9F (
eth3_rename) and waits for the VF to disappear to renameeth3_renameto eth1. Unfortunately eth1 is not going to disappear. This is not a udev bug since udev doesn't create persistent rules for VFs as their MAC address changes every reboot.Quoted 4 lines
Quoted 4 lines
[...]
I think it is a bug in the udev rules: udev should rename the VFs even though their names won't be persistent.
Udev writes out these configs to a rules file, and therefore can never handle random MAC addresses, as they would just accumulate in the rules file with a new entry at every bootup.
Quoted 5 lines
Stuff like this is just not supported at the moment with the rather simple logic it has, and there is no current plan/idea, or anybody working on changing/improving this at the moment.
Kay
Solution: move away from the "eth*" namespace and use "net*" for configured interfaces.
------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
E1000-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
| From: Stefan Assmann <[email protected]> | Date: Wednesday, June 30, 2010 01:53 am | | This is not a udev bug since udev doesn't create persistent rules for | VFs as their MAC address changes every reboot. | | To avoid this problem we could change the kernel name for the VFs and | thus avoid confusion between VFs and PFs. | | I've already discussed this with Alexander Duyck and Greg Rose, so far | they have no objection. However this problem appears for all drivers that | support PFs and VFs and thus the changes should be applied consistently | to all of these drivers.
I'm not sure that this problem affects "all drivers which support PFs and VFs."
I think that you might mean "all drivers which support PFs and VFs with non- persistent MAC addresses for the VFs." For instance, the MAC addresses associated with the new cxgb4vf VFs are persistent so, from what I understand of the scenario you outlined, I don't think that they would trigger the problem you describe. Please correct me if I've missed something. Thanks.
Casey
Quoted 14 lines · Casey Leedom
| From: Stefan Assmann <[email protected]> | Date: Wednesday, June 30, 2010 01:53 am | | This is not a udev bug since udev doesn't create persistent rules for | VFs as their MAC address changes every reboot. | | To avoid this problem we could change the kernel name for the VFs and | thus avoid confusion between VFs and PFs. | | I've already discussed this with Alexander Duyck and Greg Rose, so far | they have no objection. However this problem appears for all drivers that | support PFs and VFs and thus the changes should be applied consistently | to all of these drivers.
Quoted 8 lines
I'm not sure that this problem affects "all drivers which support PFs and VFs."
I think that you might mean "all drivers which support PFs and VFs with non- persistent MAC addresses for the VFs." For instance, the MAC addresses associated with the new cxgb4vf VFs are persistent so, from what I understand of the scenario you outlined, I don't think that they would trigger the problem you describe. Please correct me if I've missed something. Thanks.Casey
You're correct, the problem shouldn't occur with cxgb4vf and therefore this change shouldn't be necessary. However we might consider a consistent naming scheme for VFs in all drivers. But I don't have a strong opinion about this, either way would be fine by me.
Stefan
--
Stefan Assmann | Red Hat GmbH
Software Engineer | Otto-Hahn-Strasse 20, 85609 Dornach
| HR: Amtsgericht Muenchen HRB 153243
| GF: Brendan Lane, Charlie Peters,
sassmann at redhat.com | Michael Cunningham, Charles Cachera| From: Stefan Assmann <[email protected]> | Date: Wednesday, June 30, 2010 11:37 pm | | You're correct, the problem shouldn't occur with cxgb4vf and therefore | this change shouldn't be necessary. However we might consider a | consistent naming scheme for VFs in all drivers. But I don't have a | strong opinion about this, either way would be fine by me.
Sorry, I hadn't meant to imply any criticism of your naming proposal. I was
just trying to clarify when/where such a scheme might be necessary.
On the naming proposal itself, it strikes me that the most common use of PCI-E
SR-IOV Virtual Functions will be to export them to KVM Virtual Machines via PCI "Pass Through." So there shouldn't be any naming conflict there, right? Or is it the same scenario you described before: that the VF NIC device might be found before the normal "eth0", etc. withing the Virtual Machine?
In any case, I don't really have any feelings one way or the other about
interface naming. As I mentioned before, the cxgb4vf driver will end up with persistent MAC addresses so all the normal interface management stuff will work normally (naming, DHCP, etc.)
Casey
------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
E1000-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
Quoted 10 lines · Casey Leedom
| From: Stefan Assmann <[email protected]> | Date: Wednesday, June 30, 2010 11:37 pm | | You're correct, the problem shouldn't occur with cxgb4vf and therefore | this change shouldn't be necessary. However we might consider a | consistent naming scheme for VFs in all drivers. But I don't have a | strong opinion about this, either way would be fine by me.Sorry, I hadn't meant to imply any criticism of your naming proposal. I was
just trying to clarify when/where such a scheme might be necessary.
Sure, that's the reason why we're discussing this here.
Quoted 6 lines
On the naming proposal itself, it strikes me that the most common use of PCI-E
SR-IOV Virtual Functions will be to export them to KVM Virtual Machines via PCI "Pass Through." So there shouldn't be any naming conflict there, right? Or is it the same scenario you described before: that the VF NIC device might be found before the normal "eth0", etc. withing the Virtual Machine?
I haven't had a scenario were passing multiple VF NICs to the guest was necessary. In theory it might happen there as well, if you have multiple NICs (with persistent and random MACs) in the guest. But usually you just have a single VF inside the guest and then you're fine.
The scenario that I'm targeting is on the host side mostly.
Stefan
--
Stefan Assmann | Red Hat GmbH
Software Engineer | Otto-Hahn-Strasse 20, 85609 Dornach
| HR: Amtsgericht Muenchen HRB 153243
| GF: Brendan Lane, Charlie Peters,
sassmann at redhat.com | Michael Cunningham, Charles Cachera------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
E1000-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
drivers/net/igbvf/netdev.c static int __devinit igbvf_probe(struct pci_dev *pdev, · line 2793
Quoted 13 lines · Stefan Assmann
drivers/net/igbvf/netdev.c +1
-1@@ -2787,7 +2787,7 @@ static int __devinit igbvf_probe(struct pci_dev *pdev,netif_carrier_off (netdev);netif_stop_queue (netdev); -strcpy (netdev->name,"eth%d" ); +strcpy (netdev->name,"veth%d" ); err =register_netdev (netdev);if (err) goto err_hw_init;
Note that 'veth' is the name used for a virtual ethernet pair by drivers/net/veth.c. If a variant of your patch gets applied, it would probably be useful to use a different naming scheme to avoid confusion with the veth driver.
Arnd
Quoted 9 lines · Arnd Bergmann
Quoted 8 lines · Stefan Assmann
drivers/net/igbvf/netdev.c +0
-0@@ -2787,7 +2787,7 @@ static int __devinit igbvf_probe(struct pci_dev *pdev,netif_carrier_off (netdev);netif_stop_queue (netdev);
Quoted 10 lines
drivers/net/igbvf/netdev.c static int __devinit igbvf_probe(struct pci_dev *pdev, · line 2793
Quoted 5 lines
drivers/net/igbvf/netdev.c +1
-1-strcpy (netdev->name,"eth%d" ); +strcpy (netdev->name,"veth%d" ); err =register_netdev (netdev);if (err) goto err_hw_init;Note that 'veth' is the name used for a virtual ethernet pair by
drivers/net/veth.c. If a variant of your patch gets applied, it would probably be useful to use a different naming scheme to avoid confusion with the veth driver.
Good point!
Greg suggested vfeth, that should be more descriptive and unique.
Stefan
---
drivers/net/igbvf/netdev.c +1 -1
@@ -2787,7 +2787,7 @@ static int __devinit igbvf_probe(struct pci_dev *pdev,netif_carrier_off (netdev);netif_stop_queue (netdev); -strcpy (netdev->name,"eth%d" ); +strcpy (netdev->name,"vfeth%d" ); err =register_netdev (netdev);if (err) goto err_hw_init;
------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
E1000-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
Rose, Gregory V
Acked-byQuoted 48 lines
-----Original Message-----
From: Stefan Assmann [mailto:[email protected]]
Sent: Friday, July 09, 2010 2:32 AM
To: Arnd Bergmann
Cc: netdev; [email protected]; Duyck, Alexander H; Rose, Gregory V; Kirsher, Jeffrey T; Andy Gospodarek
Subject: Re: [PATCH] igbvf: avoid name clash between PF and VFQuoted 6 lines · Arnd Bergmann
drivers/net/igbvf/netdev.c static int __devinit igbvf_probe(struct pci_dev
Quoted 5 lines · Stefan Assmann
drivers/net/igbvf/netdev.c +0
-0@@ -2787,7 +2787,7 @@ static int __devinit igbvf_probe(struct pci_dev*pdev,
Quoted 9 lines
drivers/net/igbvf/netdev.c static int __devinit igbvf_probe(struct pci_dev *pdev, · line 2793
Quoted 8 lines
drivers/net/igbvf/netdev.c +1
-1netif_carrier_off (netdev);netif_stop_queue (netdev); -strcpy (netdev->name,"eth%d" ); +strcpy (netdev->name,"veth%d" ); err =register_netdev (netdev);if (err) goto err_hw_init;Quoted 4 lines
Note that 'veth' is the name used for a virtual ethernet pair by
drivers/net/veth.c. If a variant of your patch gets applied, it would probably be useful to use a different naming scheme to avoid confusion with the veth driver.Good point!
Greg suggested vfeth, that should be more descriptive and unique.Stefan
---drivers/net/igbvf/netdev.c +1
-1@@ -2787,7 +2787,7 @@ static int __devinit igbvf_probe(struct pci_dev *pdev,netif_carrier_off (netdev);netif_stop_queue (netdev); -strcpy (netdev->name,"eth%d" ); +strcpy (netdev->name,"vfeth%d" ); err =register_netdev (netdev);if (err)
Quoted 1 line
goto err_hw_init;