Add vmbus dev_num and enable netvsc async probing
Haiyang Zhang
maintainerv2 cover Add vmbus dev_num and enable netvsc async probing
Add dev_num for vmbus device based on channel offer sequence. Use this number for nic naming, and enable async probing.
Haiyang Zhang (3):
Drivers: hv: vmbus: Add a dev_num variable based on channel offer
sequence
Drivers: hv: vmbus: Add dev_num to sysfs
hv_netvsc: Name NICs based on vmbus offer sequence and use async probe
Documentation/ABI/stable/sysfs-bus-vmbus | 8 +++++++ drivers/hv/channel_mgmt.c | 38 ++++++++++++++++++++++++++++++-- drivers/hv/vmbus_drv.c | 13 +++++++++++ drivers/net/hyperv/netvsc_drv.c | 18 ++++++++++++--- include/linux/hyperv.h | 6 +++++ 5 files changed, 78 insertions(+), 5 deletions(-)
Signature · 1 line
1.8.3.1
Haiyang Zhang
maintainerPATCH v2 1/3 Drivers: hv: vmbus: Add a dev_num variable based on channel offer sequence
This number is set to the first available number, starting from zero, when a vmbus device’s primary channel is offered.
It will be used for stable naming when Async probing is used.
---
Changes
V2:
Use nest loops in hv_set_devnum, instead of goto.
drivers/hv/channel_mgmt.c | 38 ++++++++++++++++++++++++++++++++++++-- include/linux/hyperv.h | 6 ++++++ 2 files changed, 42 insertions(+), 2 deletions(-)
drivers/hv/channel_mgmt.c +36 -2
@@ -315,6 +315,8 @@ static struct vmbus_channel *alloc_channel(void)if (!channel)return NULL ; + channel->dev_num =HV_DEV_NUM_INVALID ; +spin_lock_init (&channel->lock);init_completion (&channel->rescind_event);
@@ -541,6 +543,36 @@ static void vmbus_add_channel_work(struct work_struct *work) }/* +* Get the first available device number of its type, then +* record it in the channel structure. +*/ +static void hv_set_devnum (struct vmbus_channel *newchannel) +{ +struct vmbus_channel *channel; +int i =-1 ; +bool found; + +BUG_ON (!mutex_is_locked (&vmbus_connection.channel_mutex)); + +do { + i++; + found = false; + +list_for_each_entry (channel, &vmbus_connection.chn_list, + listentry) { +if (i == channel->dev_num && +guid_equal (&channel->offermsg.offer.if_type, + &newchannel->offermsg.offer.if_type)) { + found = true; +break ; + } + } + }while (found); + + newchannel->dev_num = i; +} + +/* * vmbus_process_offer - Process the offer by creating a channel/device * associated with this offer */
@@ -573,10 +605,12 @@ static void vmbus_process_offer(struct vmbus_channel *newchannel) } } -if (fnew) +if (fnew) { +hv_set_devnum (newchannel); +list_add_tail (&newchannel->listentry, &vmbus_connection.chn_list); -else { + }else {/* * Check to see if this is a valid sub-channel. */
include/linux/hyperv.h +6 -0
@@ -718,6 +718,8 @@ struct vmbus_device {bool perf_device; }; +#define HV_DEV_NUM_INVALID (-1) +struct vmbus_channel {struct list_head listentry;
@@ -849,6 +851,10 @@ struct vmbus_channel {*/ struct vmbus_channel *primary_channel;/* +* Used for device naming based on channel offer sequence. +*/ +int dev_num; +/* * Support per-channel state for use by vmbus drivers. */ void *per_channel_state;
Signature · 1 line
1.8.3.1
RE: [PATCH V2,net-next, 1/3] Drivers: hv: vmbus: Add a dev_num variable based on channel offer sequence
From: Haiyang Zhang <[email protected]> Sent: Monday, December 30, 2019 12:14 PM
Quoted 3 lines
This number is set to the first available number, starting from zero, when a vmbus device's primary channel is offered.
Let's use "VMBus" as the capitalization in text.
Quoted 48 lines
It will be used for stable naming when Async probing is used.
Signed-off-by Haiyang Zhang <[email protected]>---
Changes
V2:
Use nest loops inhv_set_devnum, instead of goto.drivers/hv/channel_mgmt.c | 38 ++++++++++++++++++++++++++++++++++++-- include/linux/hyperv.h | 6 ++++++ 2 files changed, 42 insertions(+), 2 deletions(-)drivers/hv/channel_mgmt.c +21
-0@@ -315,6 +315,8 @@ static struct vmbus_channel *alloc_channel(void)if (!channel)return NULL ; + channel->dev_num =HV_DEV_NUM_INVALID ; +spin_lock_init (&channel->lock);init_completion (&channel->rescind_event);@@ -541,6 +543,36 @@ static void vmbus_add_channel_work(struct work_struct *work) } /* + * Get the first available device number of its type, then + * record it in the channel structure. + */ +static void hv_set_devnum(struct vmbus_channel *newchannel) +{ + struct vmbus_channel *channel; + int i = -1; + bool found; + + BUG_ON(!mutex_is_locked(&vmbus_connection.channel_mutex)); + + do { + i++; + found = false; + + list_for_each_entry(channel, &vmbus_connection.chn_list, + listentry) { + if (i == channel->dev_num && + guid_equal(&channel->offermsg.offer.if_type,
drivers/hv/channel_mgmt.c static void vmbus_add_channel_work(struct work_struct *work) · line 574
Quoted 10 lines
drivers/hv/channel_mgmt.c +10
-0+ &newchannel->offermsg.offer.if_type)) { + found = true; +break ; + } + } + }while (found); + + newchannel->dev_num = i; +} +
It took me a little while to figure out what the above algorithm is doing. Perhaps it would help to rename the "found" variable to "in_use", and add this comment before the start of the "do" loop:
Iterate through each possible device number starting at zero. If the device number is already in use for a device of this type, try the next device number
until finding one that is not in use. This approach selects the smallest
device number that is not in use, and so reuses any numbers that are freed by devices that have been removed.
Quoted 46 lines
drivers/hv/channel_mgmt.c +5
-2+/* * vmbus_process_offer - Process the offer by creating a channel/device * associated with this offer */ @@ -573,10 +605,12 @@ static void vmbus_process_offer(struct vmbus_channel *newchannel) } } -if (fnew) +if (fnew) { +hv_set_devnum (newchannel); +list_add_tail (&newchannel->listentry, &vmbus_connection.chn_list); -else { + }else {/* * Check to see if this is a valid sub-channel. drivers/hv/channel_mgmt.c +0
-0*/ include/linux/hyperv.h +6
-1@@ -718,6 +718,8 @@ struct vmbus_device {bool perf_device; }; +#define HV_DEV_NUM_INVALID (-1) +struct vmbus_channel {struct list_head listentry;@@ -849,6 +851,10 @@ struct vmbus_channel {*/ struct vmbus_channel *primary_channel;/* +* Used for device naming based on channel offer sequence. +*/ +int dev_num; +/* * Support per-channel state for use by vmbus drivers. */ void *per_channel_state; --1.8.3.1
Haiyang Zhang
maintainerRE: [PATCH V2,net-next, 1/3] Drivers: hv: vmbus: Add a dev_num variable based on channel offer sequence
Quoted 14 lines
-----Original Message-----
From: Michael Kelley <[email protected]>
Sent: Monday, December 30, 2019 8:35 PM
To: Haiyang Zhang <[email protected]>; [email protected]; linux- [email protected]; [email protected]
Cc: Haiyang Zhang <[email protected]>; KY Srinivasan <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]; vkuznets <[email protected]>; [email protected]; [email protected]
Subject: RE: [PATCH V2,net-next, 1/3] Drivers: hv: vmbus: Add adev_numvariable based on channel offer sequence
Quoted 7 lines
From: Haiyang Zhang <[email protected]> Sent: Monday, December 30, 2019 12:14 PM
Quoted 3 lines
This number is set to the first available number, starting from zero, when a vmbus device's primary channel is offered.
Let's use "VMBus" as the capitalization in text.
Sure I will.
Quoted 66 lines
Quoted 10 lines
It will be used for stable naming when Async probing is used.
Signed-off-by Haiyang Zhang <[email protected]>---
Changes
V2:
Use nest loops inhv_set_devnum, instead of goto.drivers/hv/channel_mgmt.c | 38++++++++++++++++++++++++++++++++++++--
drivers/hv/channel_mgmt.c static void vmbus_add_channel_work(struct
Quoted 17 lines
include/linux/hyperv.h | 6 ++++++ 2 files changed, 42 insertions(+), 2 deletions(-)drivers/hv/channel_mgmt.c +2
-0@@ -315,6 +315,8 @@ static struct vmbus_channel *alloc_channel(void)if (!channel)return NULL ; + channel->dev_num =HV_DEV_NUM_INVALID ; +spin_lock_init (&channel->lock);init_completion (&channel->rescind_event);@@ -541,6 +543,36 @@ static void vmbus_add_channel_work(struct
work_struct*work)Quoted 22 lines
drivers/hv/channel_mgmt.c +19
-0} /* + * Get the first available device number of its type, then + * record it in the channel structure. + */ +static void hv_set_devnum(struct vmbus_channel *newchannel) +{ + struct vmbus_channel *channel; + int i = -1; + bool found; + + BUG_ON(!mutex_is_locked(&vmbus_connection.channel_mutex)); + + do { + i++; + found = false; + + list_for_each_entry(channel, &vmbus_connection.chn_list, + listentry) { + if (i == channel->dev_num && + guid_equal(&channel->offermsg.offer.if_type,drivers/hv/channel_mgmt.c static void vmbus_add_channel_work(struct work_struct *work) · line 574
Quoted 10 lines
drivers/hv/channel_mgmt.c +10
-0+ &newchannel->offermsg.offer.if_type)) { + found = true; +break ; + } + } + }while (found); + + newchannel->dev_num = i; +} +It took me a little while to figure out what the above algorithm is doing. Perhaps it would help to rename the "found" variable to "
in_use", and add this comment before the start of the "do" loop:
Quoted 5 lines
Iterate through each possible device number starting at zero. If the device number is already in use for a device of this type, try the next device number
until finding one that is not in use. This approach selects the smallest
device number that is not in use, and so reuses any numbers that are freed by devices that have been removed.
I will rename the variable, and add the code comments. Thanks,
- Haiyang
Haiyang Zhang
maintainerPATCH v2 2/3 Drivers: hv: vmbus: Add dev_num to sysfs
In reply to Haiyang ZhangIt's a number based on the vmbus device offer sequence. Useful for device naming when using Async probing.
--- Documentation/ABI/stable/sysfs-bus-vmbus | 8 ++++++++ drivers/hv/vmbus_drv.c | 13 +++++++++++++ 2 files changed, 21 insertions(+)
Documentation/ABI/stable/sysfs-bus-vmbus +8 -0
@@ -49,6 +49,14 @@ Contact: Stephen Hemminger <[email protected]> Description: This NUMA node to which the VMBUS device is attached, or -1 if the node is unknown. +What: /sys/bus/vmbus/devices/<UUID>/dev_num +Date: Dec 2019 +KernelVersion: 5.5 +Contact: Haiyang Zhang <[email protected]> +Description: A number based on the vmbus device offer sequence. + Useful for device naming when using Async probing. +Users: Debugging tools and userspace drivers + What: /sys/bus/vmbus/devices/<UUID>/channels/<N> Date: September. 2017 KernelVersion: 4.14
drivers/hv/vmbus_drv.c +13 -0
@@ -214,6 +214,18 @@ static ssize_t numa_node_show(struct device *dev,static DEVICE_ATTR_RO (numa_node);#endif +static ssize_t dev_num_show (struct device *dev, +struct device_attribute *attr,char *buf) +{ +struct hv_device *hv_dev =device_to_hv_device (dev); + +if (!hv_dev->channel) +return -ENODEV ; + +return sprintf (buf,"%d\n" , hv_dev->channel->dev_num); +} +static DEVICE_ATTR_RO (dev_num); +static ssize_t server_monitor_pending_show (struct device *dev,struct device_attribute *dev_attr,char *buf)
@@ -598,6 +610,7 @@ static ssize_t driver_override_show(struct device *dev,#ifdef CONFIG_NUMA &dev_attr_numa_node.attr, #endif + &dev_attr_dev_num.attr, &dev_attr_server_monitor_pending.attr, &dev_attr_client_monitor_pending.attr, &dev_attr_server_monitor_latency.attr,
Signature · 1 line
1.8.3.1
RE: [PATCH V2,net-next, 2/3] Drivers: hv: vmbus: Add dev_num to sysfs
From: Haiyang Zhang <[email protected]> Sent: Monday, December 30, 2019 12:14 PM
Quoted 2 lines
It's a number based on the vmbus device offer sequence.
Let's use "VMBus" as the capitalization.
Quoted 14 lines
Useful for device naming when using Async probing.
Signed-off-by Haiyang Zhang <[email protected]>--- Documentation/ABI/stable/sysfs-bus-vmbus | 8 ++++++++ drivers/hv/vmbus_drv.c | 13 +++++++++++++ 2 files changed, 21 insertions(+)Documentation/ABI/stable/sysfs-bus-vmbus +0
-0
Documentation/ABI/stable/sysfs-bus-vmbus Contact: Stephen Hemminger <[email protected]> · line 56
Quoted 8 lines
Documentation/ABI/stable/sysfs-bus-vmbus +5
-0@@ -49,6 +49,14 @@ Contact: Stephen Hemminger <[email protected]> Description: This NUMA node to which the VMBUS device is attached, or -1 if the node is unknown. +What: /sys/bus/vmbus/devices/<UUID>/dev_num +Date: Dec 2019 +KernelVersion: 5.5 +Contact: Haiyang Zhang <[email protected]> +Description: A number based on the vmbus device offer sequence.
The capitalization of "VMBus" is already inconsistent in this file in that we have "VMBus" and "VMBUS" in the text. Let's not introduce another capitalization -- use "VMBus".
Quoted 39 lines
Documentation/ABI/stable/sysfs-bus-vmbus +3
-0+ Useful for device naming when using Async probing. +Users: Debugging tools and userspace drivers + What: /sys/bus/vmbus/devices/<UUID>/channels/<N> Date: September. 2017 KernelVersion: 4.14drivers/hv/vmbus_drv.c +13
-1@@ -214,6 +214,18 @@ static ssize_t numa_node_show(struct device *dev,static DEVICE_ATTR_RO (numa_node);#endif +static ssize_t dev_num_show (struct device *dev, +struct device_attribute *attr,char *buf) +{ +struct hv_device *hv_dev =device_to_hv_device (dev); + +if (!hv_dev->channel) +return -ENODEV ; + +return sprintf (buf,"%d\n" , hv_dev->channel->dev_num); +} +static DEVICE_ATTR_RO (dev_num); +static ssize_t server_monitor_pending_show (struct device *dev,struct device_attribute *dev_attr,char *buf)@@ -598,6 +610,7 @@ static ssize_t driver_override_show(struct device *dev,#ifdef CONFIG_NUMA &dev_attr_numa_node.attr, #endif + &dev_attr_dev_num.attr, &dev_attr_server_monitor_pending.attr, &dev_attr_client_monitor_pending.attr, &dev_attr_server_monitor_latency.attr, --1.8.3.1
Haiyang Zhang
maintainerRE: [PATCH V2,net-next, 2/3] Drivers: hv: vmbus: Add dev_num to sysfs
Quoted 13 lines
-----Original Message-----
From: Michael Kelley <[email protected]>
Sent: Monday, December 30, 2019 8:35 PM
To: Haiyang Zhang <[email protected]>; [email protected]; linux- [email protected]; [email protected]
Cc: Haiyang Zhang <[email protected]>; KY Srinivasan <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]; vkuznets <[email protected]>; [email protected]; [email protected]
Subject: RE: [PATCH V2,net-next, 2/3] Drivers: hv: vmbus: Adddev_numto sysfs
Quoted 6 lines
From: Haiyang Zhang <[email protected]> Sent: Monday, December 30, 2019 12:14 PM
Quoted 2 lines
It's a number based on the vmbus device offer sequence.
Let's use "VMBus" as the capitalization.
I will.
Quoted 19 lines
Documentation/ABI/stable/sysfs-bus-vmbus Contact: Stephen Hemminger
Quoted 15 lines
Useful for device naming when using Async probing.
Signed-off-by Haiyang Zhang <[email protected]>--- Documentation/ABI/stable/sysfs-bus-vmbus | 8 ++++++++ drivers/hv/vmbus_drv.c | 13 +++++++++++++ 2 files changed, 21 insertions(+)Documentation/ABI/stable/sysfs-bus-vmbus +0
-0@@ -49,6 +49,14 @@ Contact: Stephen HemmingerQuoted 3 lines
Documentation/ABI/stable/sysfs-bus-vmbus +0
-0Description: This NUMA node to which the VMBUS device is attached, or -1 if the node is unknown.
Quoted 9 lines
Documentation/ABI/stable/sysfs-bus-vmbus Contact: Stephen Hemminger <[email protected]> · line 56
Quoted 5 lines
Documentation/ABI/stable/sysfs-bus-vmbus +5
-0+What: /sys/bus/vmbus/devices/<UUID>/dev_num +Date: Dec 2019 +KernelVersion: 5.5 +Contact: Haiyang Zhang <[email protected]> +Description: A number based on the vmbus device offer sequence.The capitalization of "VMBus" is already inconsistent in this file in that we have "VMBus" and "VMBUS" in the text. Let's not introduce another capitalization -- use "VMBus".
I will use "VMBus".
Thanks,
- Haiyang
Haiyang Zhang
maintainerPATCH v2 3/3 hv_netvsc: Name NICs based on vmbus offer sequence and use async probe
In reply to Haiyang ZhangThe dev_num field in vmbus channel structure is assigned to the first available number when the channel is offered. So netvsc driver uses it for NIC naming based on channel offer sequence. Now re-enable the async probing mode for faster probing.
--- drivers/net/hyperv/netvsc_drv.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)
drivers/net/hyperv/netvsc_drv.c +15 -3
@@ -2267,10 +2267,14 @@ static int netvsc_probe(struct hv_device *dev,struct net_device_context *net_device_ctx;struct netvsc_device_info *device_info =NULL ;struct netvsc_device *nvdev; +char name[IFNAMSIZ ];int ret = -ENOMEM ; - net =alloc_etherdev_mq (sizeof (struct net_device_context ), -VRSS_CHANNEL_MAX ); +snprintf (name,IFNAMSIZ ,"eth%d" , dev->channel->dev_num); + net =alloc_netdev_mqs (sizeof (struct net_device_context ), name, +NET_NAME_ENUM , ether_setup, +VRSS_CHANNEL_MAX ,VRSS_CHANNEL_MAX ); +if (!net) goto no_net;
@@ -2355,6 +2359,14 @@ static int netvsc_probe(struct hv_device *dev, net->max_mtu =ETH_DATA_LEN ; ret =register_netdevice (net); + +if (ret == -EEXIST ) { +pr_info ("NIC name %s exists, request another name.\n" , + net->name); +strlcpy (net->name,"eth%d" ,IFNAMSIZ ); + ret =register_netdevice (net); + } +if (ret !=0 ) {pr_err ("Unable to register netdev.\n" ); goto register_failed;
@@ -2496,7 +2508,7 @@ static int netvsc_resume(struct hv_device *dev) .suspend = netvsc_suspend, .resume = netvsc_resume, .driver = { - .probe_type = PROBE_FORCE_SYNCHRONOUS, + .probe_type = PROBE_PREFER_ASYNCHRONOUS, }, };
Signature · 1 line
1.8.3.1
RE: [PATCH V2,net-next, 3/3] hv_netvsc: Name NICs based on vmbus offer sequence and use async probe
From: Haiyang Zhang <[email protected]> Sent: Monday, December 30, 2019 12:14 PM
Quoted 2 lines
The
dev_numfield in vmbus channel structure is assigned to the first
Let's use "VMBus" in text.
Quoted 31 lines
available number when the channel is offered. So netvsc driver uses it for NIC naming based on channel offer sequence. Now re-enable the async probing mode for faster probing.
Signed-off-by Haiyang Zhang <[email protected]>--- drivers/net/hyperv/netvsc_drv.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)drivers/net/hyperv/netvsc_drv.c +6
-2@@ -2267,10 +2267,14 @@ static int netvsc_probe(struct hv_device *dev,struct net_device_context *net_device_ctx;struct netvsc_device_info *device_info =NULL ;struct netvsc_device *nvdev; +char name[IFNAMSIZ ];int ret = -ENOMEM ; - net =alloc_etherdev_mq (sizeof (struct net_device_context ), -VRSS_CHANNEL_MAX ); +snprintf (name,IFNAMSIZ ,"eth%d" , dev->channel->dev_num); + net =alloc_netdev_mqs (sizeof (struct net_device_context ), name, +NET_NAME_ENUM , ether_setup, +VRSS_CHANNEL_MAX ,VRSS_CHANNEL_MAX ); +if (!net) goto no_net;
drivers/net/hyperv/netvsc_drv.c static int netvsc_probe(struct hv_device *dev, · line 2367
Quoted 9 lines
drivers/net/hyperv/netvsc_drv.c +6
-0@@ -2355,6 +2359,14 @@ static int netvsc_probe(struct hv_device *dev, net->max_mtu =ETH_DATA_LEN ; ret =register_netdevice (net); + +if (ret == -EEXIST ) { +pr_info ("NIC name %s exists, request another name.\n" , + net->name); +strlcpy (net->name,"eth%d" ,IFNAMSIZ ); + ret =register_netdevice (net);
The message above could be confusing. "request another name" sounds like a directive to the user/sysadmin, which I don't think it is. Perhaps better would be "requesting another name", which says the system is handling the collision automatically.
Also will this second call to register_netdevice() actually assign a numeric name? If so, it would be really nice for the message to be output *after* the second call to register_netdevice() and to show both the originally selected name that collided as well as the new name. We sometimes get into NIC naming issues with customers in Azure, and when a new name has to be selected it will help debugging if we can show both the original selection and the new selection. With both pieces of data, there's less guessing about who did what regarding NIC naming.
Finally, having to choose a different name because of a collision does *not* update the channel->dev_num value. Subsequent calls to hv_set_devnum() will detect "in use" based on the originally assigned dev_num value. I don't think that fundamentally breaks anything, but
I wondered if you had thought about that case. Maybe a comment here
to that effect would help a future reader of this code.
Michael
Quoted 16 lines
drivers/net/hyperv/netvsc_drv.c +3
-2+ } +if (ret !=0 ) {pr_err ("Unable to register netdev.\n" ); goto register_failed;@@ -2496,7 +2508,7 @@ static int netvsc_resume(struct hv_device *dev) .suspend = netvsc_suspend, .resume = netvsc_resume, .driver = { - .probe_type = PROBE_FORCE_SYNCHRONOUS, + .probe_type = PROBE_PREFER_ASYNCHRONOUS, }, }; --1.8.3.1
Haiyang Zhang
maintainerRE: [PATCH V2,net-next, 3/3] hv_netvsc: Name NICs based on vmbus offer sequence and use async probe
Quoted 14 lines
-----Original Message-----
From: Michael Kelley <[email protected]>
Sent: Monday, December 30, 2019 8:35 PM
To: Haiyang Zhang <[email protected]>; [email protected]; linux- [email protected]; [email protected]
Cc: Haiyang Zhang <[email protected]>; KY Srinivasan <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]; vkuznets <[email protected]>; [email protected]; [email protected]
Subject: RE: [PATCH V2,net-next, 3/3]hv_netvsc: Name NICs based on vmbus offer sequence and use async probe
Quoted 6 lines
From: Haiyang Zhang <[email protected]> Sent: Monday, December 30, 2019 12:14 PM
Quoted 2 lines
The
dev_numfield in vmbus channel structure is assigned to the firstLet's use "VMBus" in text.
I will.
Quoted 47 lines
Quoted 32 lines
available number when the channel is offered. So netvsc driver uses it for NIC naming based on channel offer sequence. Now re-enable the async probing mode for faster probing.
Signed-off-by Haiyang Zhang <[email protected]>--- drivers/net/hyperv/netvsc_drv.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)drivers/net/hyperv/netvsc_drv.c +6
-2@@ -2267,10 +2267,14 @@ static int netvsc_probe(struct hv_device *dev,struct net_device_context *net_device_ctx;struct netvsc_device_info *device_info =NULL ;struct netvsc_device *nvdev; +char name[IFNAMSIZ ];int ret = -ENOMEM ; - net =alloc_etherdev_mq (sizeof (struct net_device_context ), -VRSS_CHANNEL_MAX ); +snprintf (name,IFNAMSIZ ,"eth%d" , dev->channel->dev_num); + net =alloc_netdev_mqs (sizeof (struct net_device_context ), name, +NET_NAME_ENUM , ether_setup, +VRSS_CHANNEL_MAX ,VRSS_CHANNEL_MAX ); +if (!net) goto no_net;drivers/net/hyperv/netvsc_drv.c static int netvsc_probe(struct hv_device *dev, · line 2367
Quoted 9 lines
drivers/net/hyperv/netvsc_drv.c +6
-0@@ -2355,6 +2359,14 @@ static int netvsc_probe(struct hv_device *dev, net->max_mtu =ETH_DATA_LEN ; ret =register_netdevice (net); + +if (ret == -EEXIST ) { +pr_info ("NIC name %s exists, request another name.\n" , + net->name); +strlcpy (net->name,"eth%d" ,IFNAMSIZ ); + ret =register_netdevice (net);The message above could be confusing. "request another name" sounds like a directive to the user/sysadmin, which I don't think it is. Perhaps better would be "requesting another name", which says the system is handling the collision automatically.
Quoted 8 lines
Also will this second call to
register_netdevice()actually assign a numeric name? If so, it would be really nice for the message to be output *after* the second call toregister_netdevice()and to show both the originally selected name that collided as well as the new name. We sometimes get into NIC naming issues with customers in Azure, and when a new name has to be selected it will help debugging if we can show both the original selection and the new selection. With both pieces of data, there's less guessing about who did what regarding NIC naming.
Good idea! I will include both new and old names in the log messages.
Quoted 7 lines
Finally, having to choose a different name because of a collision does *not* update the
channel->dev_numvalue. Subsequent calls tohv_set_devnum()will detect "in use" based on the originally assigneddev_numvalue. I don't think that fundamentally breaks anything, but
I wondered if you had thought about that case. Maybe a comment here
to that effect would help a future reader of this code.
That's correct. And I'm aware of this situation. But, the dev_num shouldn't be changed, because we want it to keep track of the sequence of device offering.
Avoid single or multiple collisions, we should use udev or other daemons to set name of VF or other types of NICs to different formats, like "vf*", or "enP*", etc. For distros have not done so already, we need to work with the distro vendors to ensure udev or other settings are correctly included in Distros, and Azure images.
Thanks,
- Haiyang
Re: [PATCH V2,net-next, 3/3] hv_netvsc: Name NICs based on vmbus offer sequence and use async probe
In reply to Haiyang ZhangQuoted 15 lines · Haiyang Zhang
The
dev_numfield in vmbus channel structure is assigned to the first available number when the channel is offered. So netvsc driver uses it for NIC naming based on channel offer sequence. Now re-enable the async probing mode for faster probing.Signed-off-by Haiyang Zhang <[email protected]>--- drivers/net/hyperv/netvsc_drv.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)drivers/net/hyperv/netvsc_drv.c +0
-0
drivers/net/hyperv/netvsc_drv.c static int netvsc_probe(struct hv_device *dev, · line 2273
Quoted 9 lines
drivers/net/hyperv/netvsc_drv.c +2
-2@@ -2267,10 +2267,14 @@ static int netvsc_probe(struct hv_device *dev,struct net_device_context *net_device_ctx;struct netvsc_device_info *device_info =NULL ;struct netvsc_device *nvdev; +char name[IFNAMSIZ ];int ret = -ENOMEM ; - net =alloc_etherdev_mq (sizeof (struct net_device_context ), -VRSS_CHANNEL_MAX ); +snprintf (name,IFNAMSIZ ,"eth%d" , dev->channel->dev_num);
How is this supposed to work when there are other ethernet device types on the system, which may claim the same device names?
Quoted 8 lines
drivers/net/hyperv/netvsc_drv.c +4
-0+ net =alloc_netdev_mqs (sizeof (struct net_device_context ), name, +NET_NAME_ENUM , ether_setup, +VRSS_CHANNEL_MAX ,VRSS_CHANNEL_MAX ); +if (!net) goto no_net;
drivers/net/hyperv/netvsc_drv.c static int netvsc_probe(struct hv_device *dev, · line 2368
Quoted 10 lines
drivers/net/hyperv/netvsc_drv.c +7
-0@@ -2355,6 +2359,14 @@ static int netvsc_probe(struct hv_device *dev, net->max_mtu =ETH_DATA_LEN ; ret =register_netdevice (net); + +if (ret == -EEXIST ) { +pr_info ("NIC name %s exists, request another name.\n" , + net->name); +strlcpy (net->name,"eth%d" ,IFNAMSIZ ); + ret =register_netdevice (net); + }
IOW you want the device naming to be predictable, but don't guarantee this?
I think the problem this patchset is trying to solve is much better solved with a udev rule, similar to how it's done for PCI net devices. And IMO the primary channel number, being a device's "hardware" property, is more suited to be used in the device name, than this completely ephemeral device number.
Thanks,
Roman.
Quoted 16 lines
drivers/net/hyperv/netvsc_drv.c +2
-1+if (ret !=0 ) {pr_err ("Unable to register netdev.\n" ); goto register_failed;@@ -2496,7 +2508,7 @@ static int netvsc_resume(struct hv_device *dev) .suspend = netvsc_suspend, .resume = netvsc_resume, .driver = { - .probe_type = PROBE_FORCE_SYNCHRONOUS, + .probe_type = PROBE_PREFER_ASYNCHRONOUS, }, };Signature · 1 line
1.8.3.1
Haiyang Zhang
maintainerRE: [PATCH V2,net-next, 3/3] hv_netvsc: Name NICs based on vmbus offer sequence and use async probe
Quoted 63 lines
-----Original Message-----
From: Roman Kagan <[email protected]>
Sent: Tuesday, December 31, 2019 6:35 AM
To: Haiyang Zhang <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; KY Srinivasan <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]; vkuznets <[email protected]>; [email protected]; [email protected]
Subject: Re: [PATCH V2,net-next, 3/3]hv_netvsc: Name NICs based on vmbus offer sequence and use async probeQuoted 15 lines · Haiyang Zhang
The
dev_numfield in vmbus channel structure is assigned to the first available number when the channel is offered. So netvsc driver uses it for NIC naming based on channel offer sequence. Now re-enable the async probing mode for faster probing.Signed-off-by Haiyang Zhang <[email protected]>--- drivers/net/hyperv/netvsc_drv.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)(quoted diff) +0
-0
b/drivers/net/hyperv/netvsc_drv.cindex f3f9eb8..39c412f 100644drivers/net/hyperv/netvsc_drv.c static int netvsc_probe(struct hv_device *dev, · line 2273
Quoted 9 lines
drivers/net/hyperv/netvsc_drv.c +2
-2@@ -2267,10 +2267,14 @@ static int netvsc_probe(struct hv_device *dev,struct net_device_context *net_device_ctx;struct netvsc_device_info *device_info =NULL ;struct netvsc_device *nvdev; +char name[IFNAMSIZ ];int ret = -ENOMEM ; - net =alloc_etherdev_mq (sizeof (struct net_device_context ), -VRSS_CHANNEL_MAX ); +snprintf (name,IFNAMSIZ ,"eth%d" , dev->channel->dev_num);How is this supposed to work when there are other ethernet device types on the system, which may claim the same device names?
Quoted 8 lines
drivers/net/hyperv/netvsc_drv.c +4
-0+ net =alloc_netdev_mqs (sizeof (struct net_device_context ), name, +NET_NAME_ENUM , ether_setup, +VRSS_CHANNEL_MAX ,VRSS_CHANNEL_MAX ); +if (!net) goto no_net;drivers/net/hyperv/netvsc_drv.c static int netvsc_probe(struct hv_device *dev, · line 2368
Quoted 10 lines
drivers/net/hyperv/netvsc_drv.c +7
-0@@ -2355,6 +2359,14 @@ static int netvsc_probe(struct hv_device *dev, net->max_mtu =ETH_DATA_LEN ; ret =register_netdevice (net); + +if (ret == -EEXIST ) { +pr_info ("NIC name %s exists, request another name.\n" , + net->name); +strlcpy (net->name,"eth%d" ,IFNAMSIZ ); + ret =register_netdevice (net); + }IOW you want the device naming to be predictable, but don't guarantee this?
Quoted 5 lines
I think the problem this patchset is trying to solve is much better solved with a udev rule, similar to how it's done for PCI net devices.
And IMO the primary channel number, being a device's "hardware" property, is more suited to be used in the device name, than this completely ephemeral device number.
The vmbus number can be affected by other types of devices and/or subchannel offerings. They are not stable either. That's why this patch set keeps track of the offering sequence within the same device type in a new variable "dev_num".
As in my earlier email, to avoid impact by other types of NICs, we should put them into different naming formats, like "vf*", "enP*", etc. And yes, these can be done in udev.
But for netvsc (synthetic) NICs, we still want the default naming format "eth*". And the variable "dev_num" gives them the basis for stable naming with Async probing.
Thanks,
- Haiyang
Re: [PATCH V2,net-next, 3/3] hv_netvsc: Name NICs based on vmbus offer sequence and use async probe
Quoted 75 lines · On Tue, 31 Dec 2019 16:12:36 +0000 Haiyang Zhang <[email protected]> wrote:
Quoted 61 lines
-----Original Message-----
From: Roman Kagan <[email protected]>
Sent: Tuesday, December 31, 2019 6:35 AM
To: Haiyang Zhang <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; KY Srinivasan <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]; vkuznets <[email protected]>; [email protected]; [email protected]
Subject: Re: [PATCH V2,net-next, 3/3]hv_netvsc: Name NICs based on vmbus offer sequence and use async probeQuoted 15 lines · Haiyang Zhang
The
dev_numfield in vmbus channel structure is assigned to the first available number when the channel is offered. So netvsc driver uses it for NIC naming based on channel offer sequence. Now re-enable the async probing mode for faster probing.Signed-off-by Haiyang Zhang <[email protected]>--- drivers/net/hyperv/netvsc_drv.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)(quoted diff) +0
-0
b/drivers/net/hyperv/netvsc_drv.cindex f3f9eb8..39c412f 100644drivers/net/hyperv/netvsc_drv.c static int netvsc_probe(struct hv_device *dev, · line 2273
Quoted 9 lines
drivers/net/hyperv/netvsc_drv.c +2
-2@@ -2267,10 +2267,14 @@ static int netvsc_probe(struct hv_device *dev,struct net_device_context *net_device_ctx;struct netvsc_device_info *device_info =NULL ;struct netvsc_device *nvdev; +char name[IFNAMSIZ ];int ret = -ENOMEM ; - net =alloc_etherdev_mq (sizeof (struct net_device_context ), -VRSS_CHANNEL_MAX ); +snprintf (name,IFNAMSIZ ,"eth%d" , dev->channel->dev_num);How is this supposed to work when there are other ethernet device types on the system, which may claim the same device names?
Quoted 8 lines
drivers/net/hyperv/netvsc_drv.c +4
-0+ net =alloc_netdev_mqs (sizeof (struct net_device_context ), name, +NET_NAME_ENUM , ether_setup, +VRSS_CHANNEL_MAX ,VRSS_CHANNEL_MAX ); +if (!net) goto no_net;drivers/net/hyperv/netvsc_drv.c static int netvsc_probe(struct hv_device *dev, · line 2368
Quoted 10 lines
drivers/net/hyperv/netvsc_drv.c +7
-0@@ -2355,6 +2359,14 @@ static int netvsc_probe(struct hv_device *dev, net->max_mtu =ETH_DATA_LEN ; ret =register_netdevice (net); + +if (ret == -EEXIST ) { +pr_info ("NIC name %s exists, request another name.\n" , + net->name); +strlcpy (net->name,"eth%d" ,IFNAMSIZ ); + ret =register_netdevice (net); + }IOW you want the device naming to be predictable, but don't guarantee this?
Quoted 5 lines
I think the problem this patchset is trying to solve is much better solved with a udev rule, similar to how it's done for PCI net devices.
And IMO the primary channel number, being a device's "hardware" property, is more suited to be used in the device name, than this completely ephemeral device number.The vmbus number can be affected by other types of devices and/or subchannel offerings. They are not stable either. That's why this patch set keeps track of the offering sequence within the same device type in a new variable "
dev_num".As in my earlier email, to avoid impact by other types of NICs, we should put them into different naming formats, like "vf*", "enP*", etc. And yes, these can be done in udev.
Quoted 6 lines
But for netvsc (synthetic) NICs, we still want the default naming format "eth*". And the variable "
dev_num" gives them the basis for stable naming with Async probing.Thanks,
- Haiyang
The primary requirements for network naming are:
1. Network names must be repeatable on each boot. This was the original problem that PCI devices discovered back years ago when parallel probing was enabled.
2. Network names must be predictable. If new VM is created, the names should match a similar VM config.
3. Names must be persistent. If a NIC is added or deleted, the existing names must not change.
The other things which are important (and this proposal breaks):
1. Don't break it principle: an existing VM should not suddenly get interfaces renamed if kernel is upgraded. A corrallary is that a lot of current userspace code expects eth0. It doesn't look like first interface would be guaranteed to be eth0.
2. No snowflakes principle: a device driver should follow the current practice of other devices. For netvsc, this means VMBus should act like PCI as much as possible. Is there another driver doing this already?
3. Userspace policy principle: Every distribution has its own policy by now. The solution must make netvsc work reliably on Redhat (udev), Ubuntu (netplan), SuSE (Yast) doing something in the kernel violates #2.
My recommendation would be to take a multi-phase approach:
1. Expose persistent value in sysfs now.
2. Work with udev/netplan/... to use that value.
3. Make parallel VMBus probing an option. So that when distributions have picked up the udev changes they can enable parallel probe. Some will be quick to adopt and the enterprise laggards can get to it when they feel the heat.
Long term wish list (requires host side changes):
1. The interface index could be a host side property; the host networking already has the virtual device table and it is persistent.
2. The Azure NIC name should be visible as a property in guest. Then userspace could do rename based on that property. Having multiple disconnected names is leads to confusion.
Haiyang Zhang
maintainerRE: [PATCH V2,net-next, 3/3] hv_netvsc: Name NICs based on vmbus offer sequence and use async probe
Quoted 145 lines
-----Original Message-----
From: [email protected] <linux-hyperv- [email protected]> On Behalf Of Stephen Hemminger
Sent: Tuesday, December 31, 2019 12:36 PM
To: Haiyang Zhang <[email protected]>
Cc: Roman Kagan <[email protected]>; [email protected]; linux- [email protected]; [email protected]; KY Srinivasan <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]; vkuznets <[email protected]>; [email protected]
Subject: Re: [PATCH V2,net-next, 3/3]hv_netvsc: Name NICs based on vmbus offer sequence and use async probeQuoted 5 lines · On Tue, 31 Dec 2019 16:12:36 +0000 Haiyang Zhang <[email protected]> wrote:
Quoted 5 lines
-----Original Message-----
From: Roman Kagan <[email protected]>
Sent: Tuesday, December 31, 2019 6:35 AM
To: Haiyang Zhang <[email protected]>
Cc: [email protected]; [email protected];Quoted 3 lines
Quoted 3 lines
KY Srinivasan <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]; vkuznets <[email protected]>; [email protected]; linux-
Quoted 1 line
Quoted 1 line
Subject: Re: [PATCH V2,net-next, 3/3]
hv_netvsc: Name NICs based onvmbus
Quoted 24 lines
Quoted 24 lines
offer sequence and use async probe
Quoted 21 lines · Haiyang Zhang
The
dev_numfield in vmbus channel structure is assigned to the first available number when the channel is offered. So netvsc driver uses it for NIC naming based on channel offer sequence. Now re-enable the async probing mode for faster probing.Signed-off-by Haiyang Zhang <[email protected]>--- drivers/net/hyperv/netvsc_drv.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)(quoted diff) +0
-0
b/drivers/net/hyperv/netvsc_drv.cindex f3f9eb8..39c412f 100644drivers/net/hyperv/netvsc_drv.c +1
-0@@ -2267,10 +2267,14 @@ static int netvsc_probe(struct hv_device *dev,struct net_device_context *net_device_ctx;struct netvsc_device_info *device_info =NULL ;struct netvsc_device *nvdev; +char name[IFNAMSIZ ];int ret = -ENOMEM ;Quoted 5 lines
Quoted 5 lines
drivers/net/hyperv/netvsc_drv.c static int netvsc_probe(struct hv_device *dev, · line 2273
Quoted 3 lines
drivers/net/hyperv/netvsc_drv.c +1
-2- net =alloc_etherdev_mq (sizeof (struct net_device_context ), -VRSS_CHANNEL_MAX ); +snprintf (name,IFNAMSIZ ,"eth%d" , dev->channel->dev_num);How is this supposed to work when there are other ethernet device types on
the
Quoted 23 lines
Quoted 23 lines
system, which may claim the same device names?
Quoted 8 lines
drivers/net/hyperv/netvsc_drv.c +4
-0+ net =alloc_netdev_mqs (sizeof (struct net_device_context ), name, +NET_NAME_ENUM , ether_setup, +VRSS_CHANNEL_MAX ,VRSS_CHANNEL_MAX ); +if (!net) goto no_net;drivers/net/hyperv/netvsc_drv.c static int netvsc_probe(struct hv_device *dev, · line 2368
Quoted 10 lines
drivers/net/hyperv/netvsc_drv.c +7
-0@@ -2355,6 +2359,14 @@ static int netvsc_probe(struct hv_device *dev, net->max_mtu =ETH_DATA_LEN ; ret =register_netdevice (net); + +if (ret == -EEXIST ) { +pr_info ("NIC name %s exists, request another name.\n" , + net->name); +strlcpy (net->name,"eth%d" ,IFNAMSIZ ); + ret =register_netdevice (net); + }IOW you want the device naming to be predictable, but don't guarantee this?
Quoted 1 line
Quoted 1 line
I think the problem this patchset is trying to solve is much better solved with
a
Quoted 6 lines
Quoted 4 lines
udev rule, similar to how it's done for PCI net devices.
And IMO the primary channel number, being a device's "hardware" property, is more suited to be used in the device name, than this completely ephemeral device number.The vmbus number can be affected by other types of devices and/or
subchannel
Quoted 1 line
offerings. They are not stable either. That's why this patch set keeps track of
the
Quoted 3 lines
offering sequence within the same device type in a new variable "
dev_num".As in my earlier email, to avoid impact by other types of NICs, we should put
them
Quoted 1 line
into different naming formats, like "vf*", "enP*", etc. And yes, these can be
done in
Quoted 3 lines
udev.
But for netvsc (synthetic) NICs, we still want the default naming format "eth*".
And
Quoted 1 line
the variable "
dev_num" gives them the basis for stable naming with Asyncprobing.
Quoted 4 lines
Thanks,
- HaiyangThe primary requirements for network naming are:
1. Network names must be repeatable on each boot. This was the original
problem
that PCI devices discovered back years ago when parallel probing was
enabled.
2. Network names must be predictable. If new VM is created, the names
should
match a similar VM config.
3. Names must be persistent. If a NIC is added or deleted, the existing names must not change.The other things which are important (and this proposal breaks):
1. Don't break it principle: an existing VM should not suddenly get interfaces renamed if kernel is upgraded. A corrallary is that a lot of current userspace code expects eth0. It doesn't look like first interface would be guaranteed to be eth0.2. No snowflakes principle: a device driver should follow the current practice of other devices. For netvsc, this means VMBus should act like PCI as much as possible. Is there another driver doing this already?
3. Userspace policy principle: Every distribution has its own policy by now. The solution must make netvsc work reliably on Redhat (udev), Ubuntu
(netplan), SuSE (Yast)
doing something in the kernel violates #2.My recommendation would be to take a multi-phase approach:
1. Expose persistent value in sysfs now.
2. Work with udev/netplan/... to use that value.
3. Make parallel VMBus probing an option. So that when distributions have
picked up
the udev changes they can enable parallel probe. Some will be quick to
adopt
and the enterprise laggards can get to it when they feel the heat.
Quoted 6 lines
Long term wish list (requires host side changes):
1. The interface index could be a host side property; the host networking already has the virtual device table and it is persistent.
2. The Azure NIC name should be visible as a property in guest. Then userspace could do rename based on that property. Having multiple disconnected names is leads to confusion.
Thank you for the detailed recommendations!
I will do the step 1. Expose persistent value in sysfs now. And work on other changes in the future.
Thanks,
- Haiyang