net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)
Dexuan Cui
maintainerPATCH v4 1/1 net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)
Add a VF driver for Microsoft Azure Network Adapter (MANA) that will be available in the future.
--- MAINTAINERS | 4 +- drivers/net/ethernet/Kconfig | 1 + drivers/net/ethernet/Makefile | 1 + drivers/net/ethernet/microsoft/Kconfig | 29 + drivers/net/ethernet/microsoft/Makefile | 5 + drivers/net/ethernet/microsoft/mana/Makefile | 6 + drivers/net/ethernet/microsoft/mana/gdma.h | 728 +++++++ .../net/ethernet/microsoft/mana/gdma_main.c | 1525 +++++++++++++ .../net/ethernet/microsoft/mana/hw_channel.c | 854 ++++++++ .../net/ethernet/microsoft/mana/hw_channel.h | 190 ++ drivers/net/ethernet/microsoft/mana/mana.h | 549 +++++ drivers/net/ethernet/microsoft/mana/mana_en.c | 1924 +++++++++++++++++ .../ethernet/microsoft/mana/mana_ethtool.c | 252 +++ .../net/ethernet/microsoft/mana/shm_channel.c | 298 +++ .../net/ethernet/microsoft/mana/shm_channel.h | 21 + 15 files changed, 6386 insertions(+), 1 deletion(-) create mode 100644 drivers/net/ethernet/microsoft/Kconfig create mode 100644 drivers/net/ethernet/microsoft/Makefile create mode 100644 drivers/net/ethernet/microsoft/mana/Makefile create mode 100644 drivers/net/ethernet/microsoft/mana/gdma.h create mode 100644 drivers/net/ethernet/microsoft/mana/gdma_main.c create mode 100644 drivers/net/ethernet/microsoft/mana/hw_channel.c create mode 100644 drivers/net/ethernet/microsoft/mana/hw_channel.h create mode 100644 drivers/net/ethernet/microsoft/mana/mana.h create mode 100644 drivers/net/ethernet/microsoft/mana/mana_en.c create mode 100644 drivers/net/ethernet/microsoft/mana/mana_ethtool.c create mode 100644 drivers/net/ethernet/microsoft/mana/shm_channel.c create mode 100644 drivers/net/ethernet/microsoft/mana/shm_channel.h
Changes in v2:
Removed the module_param(num_queues,...). [Andrew Lunn]
Changed pr_err() to netdev_err() and dev_err(). [Andrew Lunn]
Used reverse X-mas tree style in all the functions. [Andrew Lunn]
Changed "= { 0 };" to "= {};" for struct variables. [Leon Romanovsky]
Addressed 3 build failures on i386, arc and powerpc by making the the driver dependent on X86_64 in Kconfig: so far, the driver is only validated on X86_64 (in the future, we may enable the driver for ARM64).
Also made some cosmetic changes.
Rebased the patch to the latest net-next.
BTW, the support of XDP and BQL haven't been implemented. They are on our TO-DO list.
Changes in v3:
Changed the Kconfig file:
Microsoft Azure Network Device ==> Microsoft Network Devices Drop the default m
validated ==> supported
Fixed some warnings reported by "kernel test robot" (there are
still some warnings against another driver pci-hyperv.c, which I'll fix in a separate patch for the PCI tree).
Changes in v4:
Removed all the "__packed" usages since all the structs are actually naturally aligned. We verified there is no hole in the middle of the structs or at the end of the structs.
Prefixed all the functions with "mana_".
Integrated Shachar Raindel's multi-vPort patches (i.e. create multiple net devices on the single VF PCIe device, if the PF driver reports the support). Added Shachar's Co-developed-by and Signed-off-by.
Also made some cosmetic changes.
Rebased the patch to the latest net-next.
MAINTAINERS +3 -1
@@ -8260,11 +8260,12 @@ S: Maintained T: git git://linuxtv.org/media_tree.git F: drivers/media/i2c/hi556.c -Hyper-V CORE AND DRIVERS +Hyper-V/Azure CORE AND DRIVERS M: "K. Y. Srinivasan" <[email protected]> M: Haiyang Zhang <[email protected]> M: Stephen Hemminger <[email protected]> M: Wei Liu <[email protected]> +M: Dexuan Cui <[email protected]> L: [email protected] S: Supported T: git git://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux.git
@@ -8281,6 +8282,7 @@ F: drivers/hid/hid-hyperv.c F: drivers/hv/ F: drivers/input/serio/hyperv-keyboard.c F: drivers/iommu/hyperv-iommu.c +F: drivers/net/ethernet/microsoft/ F: drivers/net/hyperv/ F: drivers/pci/controller/pci-hyperv-intf.c F: drivers/pci/controller/pci-hyperv.c
drivers/net/ethernet/Kconfig +1 -0
@@ -82,6 +82,7 @@ source "drivers/net/ethernet/huawei/Kconfig" source "drivers/net/ethernet/i825xx/Kconfig" source "drivers/net/ethernet/ibm/Kconfig" source "drivers/net/ethernet/intel/Kconfig" +source "drivers/net/ethernet/microsoft/Kconfig" source "drivers/net/ethernet/xscale/Kconfig" config JME
drivers/net/ethernet/Makefile +1 -0
@@ -45,6 +45,7 @@ obj-$(CONFIG_NET_VENDOR_HUAWEI) += huawei/ obj-$(CONFIG_NET_VENDOR_IBM) += ibm/ obj-$(CONFIG_NET_VENDOR_INTEL) += intel/ obj-$(CONFIG_NET_VENDOR_I825XX) += i825xx/ +obj-$(CONFIG_NET_VENDOR_MICROSOFT) += microsoft/ obj-$(CONFIG_NET_VENDOR_XSCALE) += xscale/ obj-$(CONFIG_JME) += jme.o obj-$(CONFIG_KORINA) += korina.o
drivers/net/ethernet/microsoft/Kconfig new file +29 -0
@@ -0,0 +1,29 @@ +# +# Microsoft Azure network device configuration +# + +config NET_VENDOR_MICROSOFT + bool "Microsoft Network Devices" + default y + help + If you have a network (Ethernet) device belonging to this class, say Y. + + Note that the answer to this question doesn't directly affect the + kernel: saying N will just cause the configurator to skip the + question about Microsoft network devices. If you say Y, you will be + asked for your specific device in the following question. + +if NET_VENDOR_MICROSOFT + +config MICROSOFT_MANA + tristate "Microsoft Azure Network Adapter (MANA) support" + depends on PCI_MSI && X86_64 + select PCI_HYPERV + help + This driver supports Microsoft Azure Network Adapter (MANA). + So far, the driver is only supported on X86_64. + + To compile this driver as a module, choose M here. + The module will be called mana. + +endif #NET_VENDOR_MICROSOFT
drivers/net/ethernet/microsoft/Makefile new file +5 -0
@@ -0,0 +1,5 @@ +# +# Makefile for the Microsoft Azure network device driver. +# + +obj-$(CONFIG_MICROSOFT_MANA) += mana/
drivers/net/ethernet/microsoft/mana/Makefile new file +6 -0
@@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause +# +# Makefile for the Microsoft Azure Network Adapter driver + +obj-$(CONFIG_MICROSOFT_MANA) += mana.o +mana-objs := gdma_main.o shm_channel.o hw_channel.o mana_en.o mana_ethtool.o
drivers/net/ethernet/microsoft/mana/gdma.h new file +728 -0
728 lines not on this page · raw message
drivers/net/ethernet/microsoft/mana/gdma_main.c new file +1525 -0
1,525 lines not on this page · raw message
drivers/net/ethernet/microsoft/mana/hw_channel.c new file +854 -0
673 lines not on this page · raw message
drivers/net/ethernet/microsoft/mana/hw_channel.h new file +190 -0
190 lines not on this page · raw message
drivers/net/ethernet/microsoft/mana/mana.h new file +549 -0
549 lines not on this page · raw message
drivers/net/ethernet/microsoft/mana/mana_en.c new file +1924 -0
1,924 lines not on this page · raw message
drivers/net/ethernet/microsoft/mana/mana_ethtool.c new file +252 -0
252 lines not on this page · raw message
drivers/net/ethernet/microsoft/mana/shm_channel.c new file +298 -0
298 lines not on this page · raw message
drivers/net/ethernet/microsoft/mana/shm_channel.h new file +21 -0
Lines of this file are not shown. View the raw message.
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.20.1
Part of the diff is not shown here. View the raw message.
Quoted 38 lines · Dexuan Cui
Add a VF driver for Microsoft Azure Network Adapter (MANA) that will be available in the future.
Co-developed-by Haiyang Zhang <[email protected]>Signed-off-by Haiyang Zhang <[email protected]>Co-developed-by Shachar Raindel <[email protected]>Signed-off-by Shachar Raindel <[email protected]>Signed-off-by Dexuan Cui <[email protected]>--- MAINTAINERS | 4 +- drivers/net/ethernet/Kconfig | 1 + drivers/net/ethernet/Makefile | 1 + drivers/net/ethernet/microsoft/Kconfig | 29 + drivers/net/ethernet/microsoft/Makefile | 5 + drivers/net/ethernet/microsoft/mana/Makefile | 6 + drivers/net/ethernet/microsoft/mana/gdma.h | 728 +++++++ .../net/ethernet/microsoft/mana/gdma_main.c | 1525 +++++++++++++ .../net/ethernet/microsoft/mana/hw_channel.c | 854 ++++++++ .../net/ethernet/microsoft/mana/hw_channel.h | 190 ++ drivers/net/ethernet/microsoft/mana/mana.h | 549 +++++ drivers/net/ethernet/microsoft/mana/mana_en.c | 1924 +++++++++++++++++ .../ethernet/microsoft/mana/mana_ethtool.c | 252 +++ .../net/ethernet/microsoft/mana/shm_channel.c | 298 +++ .../net/ethernet/microsoft/mana/shm_channel.h | 21 + 15 files changed, 6386 insertions(+), 1 deletion(-) create mode 100644 drivers/net/ethernet/microsoft/Kconfig create mode 100644 drivers/net/ethernet/microsoft/Makefile create mode 100644 drivers/net/ethernet/microsoft/mana/Makefile create mode 100644 drivers/net/ethernet/microsoft/mana/gdma.h create mode 100644 drivers/net/ethernet/microsoft/mana/gdma_main.c create mode 100644 drivers/net/ethernet/microsoft/mana/hw_channel.c create mode 100644 drivers/net/ethernet/microsoft/mana/hw_channel.h create mode 100644 drivers/net/ethernet/microsoft/mana/mana.h create mode 100644 drivers/net/ethernet/microsoft/mana/mana_en.c create mode 100644 drivers/net/ethernet/microsoft/mana/mana_ethtool.c create mode 100644 drivers/net/ethernet/microsoft/mana/shm_channel.c create mode 100644 drivers/net/ethernet/microsoft/mana/shm_channel.h
<...>
drivers/net/ethernet/microsoft/mana/mana.h line 18
Quoted 9 lines
drivers/net/ethernet/microsoft/mana/mana.h new file +9
-0+/* Microsoft Azure Network Adapter (MANA)'s definitions +* +* Structures labeled with "HW DATA" are exchanged with the hardware. All of +* them are naturally aligned and hence don't need __packed. +*/ + +#define ANA_MAJOR_VERSION 0 +#define ANA_MINOR_VERSION 1 +#define ANA_MICRO_VERSION 1
Please don't introduce drier versions.
Thanks
Dexuan Cui
maintainerQuoted 9 lines
From: Leon Romanovsky <[email protected]>
Sent: Monday, April 12, 2021 12:46 AM
To: Dexuan Cui <[email protected]>drivers/net/ethernet/microsoft/mana/mana.h line 18
Quoted 4 lines
...
drivers/net/ethernet/microsoft/mana/mana.h new file +3
-0+#define ANA_MAJOR_VERSION 0 +#define ANA_MINOR_VERSION 1 +#define ANA_MICRO_VERSION 1Please don't introduce drier versions.
This is not the usual "driver version", though it's called "drv version" :-) As you can see, the driver does not use the macro MODULE_VERSION().
Here the "drv version" actually means the version of the VF-to-PF protocol, with which the Azure Network Adapter ethernet NIC driver (i.e. the VF driver) talks to the PF driver. The protocol version determines the formats of the messages that are sent from the VF driver to the PF driver, e.g. query the MAC address, create Send/Receive queues, configure RSS, etc.
Currently the protocol versin is 0.1.1 You may ask why it's called "drv version" rather than "protocol version" -- it's because the PF driver calls it that way, so I think here the VF driver may as well use the same name. BTW, the "drv ver" info is passed to the PF driver in the below function:
static int mana_query_client_cfg(struct ana_context *ac, u32 drv_major_ver,
u32 drv_minor_ver, u32 drv_micro_ver,
u16 *max_num_vports)
{
struct gdma_context *gc = ac->gdma_dev->gdma_context;
struct ana_query_client_cfg_resp resp = {};
struct ana_query_client_cfg_req req = {};
struct device *dev = gc->dev;
int err = 0;
mana_gd_init_req_hdr(&req.hdr, ANA_QUERY_CLIENT_CONFIG,
sizeof(req), sizeof(resp));
req.drv_major_ver = drv_major_ver;
req.drv_minor_ver = drv_minor_ver;
req.drv_micro_ver = drv_micro_ver;
err = mana_send_request(ac, &req, sizeof(req), &resp, sizeof(resp));Thanks,
Dexuan
Quoted 19 lines · Dexuan Cui
Quoted 9 lines
From: Leon Romanovsky <[email protected]>
Sent: Monday, April 12, 2021 12:46 AM
To: Dexuan Cui <[email protected]>drivers/net/ethernet/microsoft/mana/mana.h line 18
Quoted 4 lines
...
drivers/net/ethernet/microsoft/mana/mana.h new file +3
-0+#define ANA_MAJOR_VERSION 0 +#define ANA_MINOR_VERSION 1 +#define ANA_MICRO_VERSION 1Please don't introduce drier versions.
This is not the usual "driver version", though it's called "drv version" :-) As you can see, the driver does not use the macro
MODULE_VERSION().Here the "drv version" actually means the version of the VF-to-PF protocol, with which the Azure Network Adapter ethernet NIC driver (i.e. the VF driver) talks to the PF driver. The protocol version determines the formats of the messages that are sent from the VF driver to the PF driver, e.g. query the MAC address, create Send/Receive queues, configure RSS, etc.
Quoted 5 lines
Currently the protocol versin is 0.1.1 You may ask why it's called "drv version" rather than "protocol version" -- it's because the PF driver calls it that way, so I think here the VF driver may as well use the same name. BTW, the "drv ver" info is passed to the PF driver in the below function:
Ohh, yes, the "driver version" is not the ideal name for that.
I already looked on it in previous patch, came to the conclusion about the protocol and forgot :(.
Quoted 22 lines
static int mana_query_client_cfg(struct
ana_context*ac, u32drv_major_ver,u32 drv_minor_ver, u32 drv_micro_ver, u16 *max_num_vports) { struct gdma_context *gc = ac->gdma_dev->gdma_context; struct ana_query_client_cfg_resp resp = {}; struct ana_query_client_cfg_req req = {}; struct device *dev = gc->dev; int err = 0; mana_gd_init_req_hdr(&req.hdr, ANA_QUERY_CLIENT_CONFIG, sizeof(req), sizeof(resp)); req.drv_major_ver = drv_major_ver; req.drv_minor_ver = drv_minor_ver; req.drv_micro_ver = drv_micro_ver; err = mana_send_request(ac, &req, sizeof(req), &resp, sizeof(resp));Thanks,
Dexuan
Quoted 10 lines
Quoted 5 lines
Currently the protocol versin is 0.1.1 You may ask why it's called "drv version" rather than "protocol version" -- it's because the PF driver calls it that way, so I think here the VF driver may as well use the same name. BTW, the "drv ver" info is passed to the PF driver in the below function:
Ohh, yes, the "driver version" is not the ideal name for that.
I already looked on it in previous patch, came to the conclusion about the protocol and forgot :(.
Which suggests it needs renaming.
Andrew
Quoted 18 lines
drivers/net/ethernet/microsoft/mana/gdma.h new file +18
-0+static inline bool is_gdma_msg (const void *req) +{ +struct gdma_req_hdr *hdr = (struct gdma_req_hdr *)req; + +if (hdr->req.hdr_type ==GDMA_STANDARD_HEADER_TYPE && + hdr->resp.hdr_type ==GDMA_STANDARD_HEADER_TYPE && + hdr->req.msg_size >=sizeof (struct gdma_req_hdr ) && + hdr->resp.msg_size >=sizeof (struct gdma_resp_hdr ) && + hdr->req.msg_type !=0 && hdr->resp.msg_type !=0 ) +return true; + +return false; +} + +static inline bool is_gdma_msg_len (const u32 req_len,const u32 resp_len, +const void *req) +{ +struct gdma_req_hdr *hdr = (struct gdma_req_hdr *)req;
drivers/net/ethernet/microsoft/mana/gdma.h line 193
Quoted 10 lines
drivers/net/ethernet/microsoft/mana/gdma.h new file +10
-0+ +if (req_len >=sizeof (struct gdma_req_hdr ) && + resp_len >=sizeof (struct gdma_resp_hdr ) && + req_len >= hdr->req.msg_size && resp_len >= hdr->resp.msg_size && +is_gdma_msg (req)) { +return true; + } + +return false; +}
You missed adding the mana_ prefix here. There might be others.
drivers/net/ethernet/microsoft/mana/gdma.h
Quoted 8 lines
drivers/net/ethernet/microsoft/mana/gdma.h new file +7
-0+#define CQE_POLLING_BUFFER 512 +struct ana_eq { +struct gdma_queue *eq; +struct gdma_comp cqe_poll[CQE_POLLING_BUFFER ]; +}; +static int ana_poll (struct napi_struct *napi,int budget) +{
You also have a few cases of ana_, not mana_. There might be others.
Andrew
drivers/net/ethernet/microsoft/mana/gdma_main.c line 481
Quoted 13 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c new file +13
-0+static void mana_gd_deregiser_irq (struct gdma_queue *queue) +{ +struct gdma_dev *gd = queue->gdma_dev; +struct gdma_irq_context *gic; +struct gdma_context *gc; +struct gdma_resource *r; +unsigned int msix_index; +unsigned long flags; + +/* At most num_online_cpus() + 1 interrupts are used. */ + msix_index = queue->eq.msix_index; +if (WARN_ON (msix_index >num_online_cpus ())) +return ;
Do you handle hot{un}plug of CPUs?
Quoted 23 lines
drivers/net/ethernet/microsoft/mana/hw_channel.c new file +23
-0+static void mana_hwc_init_event_handler (void *ctx,struct gdma_queue *q_self, +struct gdma_event *event) +{ +struct hw_channel_context *hwc = ctx; +struct gdma_dev *gd = hwc->gdma_dev; +union hwc_init_type_data type_data; +union hwc_init_eq_id_db eq_db; +u32 type, val; + +switch (event->type) { +case GDMA_EQE_HWC_INIT_EQ_ID_DB : + eq_db.as_uint32 = event->details[0 ]; + hwc->cq->gdma_eq->id = eq_db.eq_id; + gd->doorbell = eq_db.doorbell; +break ; + +case GDMA_EQE_HWC_INIT_DATA : + + type_data.as_uint32 = event->details[0 ]; + +case GDMA_EQE_HWC_INIT_DONE : +complete (&hwc->hwc_init_eqe_comp); +break ;
...
drivers/net/ethernet/microsoft/mana/gdma_main.c line 335
Quoted 4 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c new file +4
-0+default : +WARN_ON (1 ); +break ; + }
Are these events from the firmware? If you have newer firmware with an older driver, are you going to spam the kernel log with WARN_ON dumps?
drivers/net/ethernet/microsoft/mana/mana_en.c line 835
Quoted 12 lines
drivers/net/ethernet/microsoft/mana/mana_en.c new file +12
-0+static int mana_move_wq_tail (struct gdma_queue *wq,u32 num_units) +{ +u32 used_space_old; +u32 used_space_new; + + used_space_old = wq->head - wq->tail; + used_space_new = wq->head - (wq->tail + num_units); + +if (used_space_new > used_space_old) { +WARN_ON (1 ); +return -ERANGE ; + }
You could replace the 1 by the condition. There are a couple of these.
Andrew
Haiyang Zhang
maintainerQuoted 47 lines
-----Original Message-----
From: Andrew Lunn <[email protected]>
Sent: Monday, April 12, 2021 8:16 AM
To: Dexuan Cui <[email protected]>
Cc: [email protected]; [email protected]; KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]; Wei Liu <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; Shachar Raindel <[email protected]>; [email protected]; linux- [email protected]
Subject: Re: [PATCH v4 net-next] net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)Quoted 11 lines
drivers/net/ethernet/microsoft/mana/gdma.h new file +11
-0+static inline bool is_gdma_msg (const void *req) { +struct gdma_req_hdr *hdr = (struct gdma_req_hdr *)req; + +if (hdr->req.hdr_type ==GDMA_STANDARD_HEADER_TYPE && + hdr->resp.hdr_type ==GDMA_STANDARD_HEADER_TYPE && + hdr->req.msg_size >=sizeof (struct gdma_req_hdr ) && + hdr->resp.msg_size >=sizeof (struct gdma_resp_hdr ) && + hdr->req.msg_type !=0 && hdr->resp.msg_type !=0 ) +return true; + +return false;drivers/net/ethernet/microsoft/mana/gdma.h
Quoted 10 lines
drivers/net/ethernet/microsoft/mana/gdma.h new file +10
-0+} + +static inline bool is_gdma_msg_len (const u32 req_len,const u32 resp_len, +const void *req) +{ +struct gdma_req_hdr *hdr = (struct gdma_req_hdr *)req; + +if (req_len >=sizeof (struct gdma_req_hdr ) && + resp_len >=sizeof (struct gdma_resp_hdr ) && + req_len >= hdr->req.msg_size && resp_len >= hdr->resp.msg_size&&
drivers/net/ethernet/microsoft/mana/gdma.h line 193
Quoted 6 lines
drivers/net/ethernet/microsoft/mana/gdma.h new file +6
-0+ is_gdma_msg(req)) { +return true; + } + +return false; +}You missed adding the
mana_prefix here. There might be others.
Quoted 8 lines
drivers/net/ethernet/microsoft/mana/gdma.h
Quoted 6 lines
drivers/net/ethernet/microsoft/mana/gdma.h new file +5
-0+#define CQE_POLLING_BUFFER 512 +struct ana_eq { +struct gdma_queue *eq; +struct gdma_comp cqe_poll[CQE_POLLING_BUFFER ]; }; +static int ana_poll (struct napi_struct *napi,int budget) {You also have a few cases of
ana_, notmana_. There might be others.
We will rename the remaining ana_ to mana_.
Also the driver version to protocol version in your previous comments.
Thanks,
- Haiyang
Haiyang Zhang
maintainerQuoted 16 lines
-----Original Message-----
From: Andrew Lunn <[email protected]>
Sent: Monday, April 12, 2021 8:32 AM
To: Dexuan Cui <[email protected]>
Cc: [email protected]; [email protected]; KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]; Wei Liu <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; Shachar Raindel <[email protected]>; [email protected]; linux- [email protected]
Subject: Re: [PATCH v4 net-next] net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)
Quoted 14 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c line 481
Quoted 12 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c new file +12
-0+static void mana_gd_deregiser_irq (struct gdma_queue *queue) { +struct gdma_dev *gd = queue->gdma_dev; +struct gdma_irq_context *gic; +struct gdma_context *gc; +struct gdma_resource *r; +unsigned int msix_index; +unsigned long flags; + +/* At most num_online_cpus() + 1 interrupts are used. */ + msix_index = queue->eq.msix_index; +if (WARN_ON (msix_index >num_online_cpus ())) +return ;Do you handle hot{un}plug of CPUs?
We don't have hot{un}plug of CPU feature yet.
Quoted 28 lines
Quoted 2 lines
+static void mana_hwc_init_event_handler(void *ctx, struct
gdma_queue
*q_self,Quoted 22 lines
drivers/net/ethernet/microsoft/mana/hw_channel.c new file +22
-0+ struct gdma_event *event) +{ +struct hw_channel_context *hwc = ctx; +struct gdma_dev *gd = hwc->gdma_dev; +union hwc_init_type_data type_data; +union hwc_init_eq_id_db eq_db; +u32 type, val; + +switch (event->type) { +case GDMA_EQE_HWC_INIT_EQ_ID_DB : + eq_db.as_uint32 = event->details[0 ]; + hwc->cq->gdma_eq->id = eq_db.eq_id; + gd->doorbell = eq_db.doorbell; +break ; + +case GDMA_EQE_HWC_INIT_DATA : + + type_data.as_uint32 = event->details[0 ]; + +case GDMA_EQE_HWC_INIT_DONE : +complete (&hwc->hwc_init_eqe_comp); +break ;...
Quoted 7 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c line 335
Quoted 4 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c new file +4
-0+default : +WARN_ON (1 ); +break ; + }Are these events from the firmware? If you have newer firmware with an older driver, are you going to spam the kernel log with
WARN_ONdumps?
For protocol version mismatch, the host and guest will either negotiate the highest common version, or fail to probe. So this kind of warnings are not expected.
Quoted 14 lines
drivers/net/ethernet/microsoft/mana/mana_en.c line 835
Quoted 12 lines
drivers/net/ethernet/microsoft/mana/mana_en.c new file +11
-0+static int mana_move_wq_tail (struct gdma_queue *wq,u32 num_units) { +u32 used_space_old; +u32 used_space_new; + + used_space_old = wq->head - wq->tail; + used_space_new = wq->head - (wq->tail + num_units); + +if (used_space_new > used_space_old) { +WARN_ON (1 ); +return -ERANGE ; + }You could replace the 1 by the condition. There are a couple of these.
Will do.
Thanks,
- Haiyang
Jakub Kicinski
maintainerdrivers/net/ethernet/microsoft/mana/mana_en.c line 1612
Quoted 2 lines · On Sun, 11 Apr 2021 19:34:55 -0700 Dexuan Cui wrote:
drivers/net/ethernet/microsoft/mana/mana_en.c new file +2
-0+for (i =0 ; i <ANA_INDIRECT_TABLE_SIZE ; i++) + apc->indir_table[i] = i % apc->num_queues;
ethtool_rxfh_indir_default()
drivers/net/ethernet/microsoft/mana/mana_en.c line 1630
Quoted 2 lines
drivers/net/ethernet/microsoft/mana/mana_en.c new file +2
-0+ err =mana_cfg_vport_steering (apc, rx, true, update_hash, update_tab); +return err;
return mana_...
please fix everywhere.
drivers/net/ethernet/microsoft/mana/mana_en.c line 1678
Quoted 9 lines
drivers/net/ethernet/microsoft/mana/mana_en.c new file +9
-0+netif_set_real_num_tx_queues (ndev, apc->num_queues); + + err =mana_add_rx_queues (apc, ndev); +if (err) + goto destroy_vport; + + apc->rss_state = apc->num_queues >1 ?TRI_STATE_TRUE :TRI_STATE_FALSE ; + +netif_set_real_num_rx_queues (ndev, apc->num_queues);
netif_set_real_num_.. can fail.
Quoted 12 lines
drivers/net/ethernet/microsoft/mana/mana_en.c new file +12
-0+rtnl_lock (); + +netdev_lockdep_set_classes (ndev); + + ndev->hw_features =NETIF_F_SG |NETIF_F_IP_CSUM |NETIF_F_IPV6_CSUM ; + ndev->hw_features |=NETIF_F_RXCSUM ; + ndev->hw_features |=NETIF_F_TSO |NETIF_F_TSO6 ; + ndev->hw_features |=NETIF_F_RXHASH ; + ndev->features = ndev->hw_features; + ndev->vlan_features =0 ; + + err =register_netdevice (ndev);
drivers/net/ethernet/microsoft/mana/mana_en.c line 1839
Quoted 10 lines
drivers/net/ethernet/microsoft/mana/mana_en.c new file +10
-0+if (err) { +netdev_err (ndev,"Unable to register netdev.\n" ); + goto destroy_vport; + } + +rtnl_unlock (); + +return 0 ; +destroy_vport: +rtnl_unlock ();
Why do you take rtnl_lock() explicitly around this code?
drivers/net/ethernet/microsoft/mana/mana_ethtool.c line 212
Quoted 13 lines
drivers/net/ethernet/microsoft/mana/mana_ethtool.c new file +13
-0+static int mana_set_channels (struct net_device *ndev, +struct ethtool_channels *channels) +{ +struct ana_port_context *apc =netdev_priv (ndev); +unsigned int new_count; +unsigned int old_count; +int err, err2; + + new_count = channels->combined_count; + old_count = apc->num_queues; + +if (new_count <1 || new_count > apc->max_queues || + channels->rx_count || channels->tx_count || channels->other_count)
All these checks should be done by the core already.
drivers/net/ethernet/microsoft/mana/mana_ethtool.c line 216
Quoted 4 lines
drivers/net/ethernet/microsoft/mana/mana_ethtool.c new file +4
-0+return -EINVAL ; + +if (new_count == old_count) +return 0 ;
And so is this one.
Quoted 5 lines
drivers/net/ethernet/microsoft/mana/mana_ethtool.c new file +5
-0+ err =mana_detach (ndev); +if (err) { +netdev_err (ndev,"mana_detach failed: %d\n" , err); +return err; + }
Dexuan Cui
maintainerQuoted 4 lines
From: Haiyang Zhang <[email protected]>
Sent: Monday, April 12, 2021 7:40 AMQuoted 2 lines
From: Andrew Lunn <[email protected]>
Sent: Monday, April 12, 2021 8:32 AM
Quoted 8 lines
Quoted 7 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c line 481
Quoted 5 lines
...
drivers/net/ethernet/microsoft/mana/gdma_main.c new file +4
-0+/* At most num_online_cpus() + 1 interrupts are used. */ + msix_index = queue->eq.msix_index; +if (WARN_ON (msix_index >num_online_cpus ())) +return ;Do you handle hot{un}plug of CPUs?
We don't have hot{un}plug of CPU feature yet.
Hyper-V doesn't support vCPU hot plug/unplug for VMs running on it, but potentially the VM is able to offline and online its virtual CPUs, though this is not a typical usage at all in production system (to offline a vCPU, the VM also needs to unbind all the para-virtualized devices' vmbus channels from that vCPU, which is kind of undoable in a VM that has less than about 32 vCPUs, because typically all the vCPUs are bound to one of the vmbus channels of the NetVSC device(s) and StorVSC device(s), and can't be (easily) unbound.
So I think the driver does need to handle cpu online/offlining properly, but I think we can do that in some future patch, because IMO that would need more careful consideration. IMO here the WARN_ON() is good as it at least lets us know something unexpected (if any) happens.
Quoted 34 lines
Quoted 27 lines
Quoted 1 line
+static void mana_hwc_init_event_handler(void *ctx, struct
gdma_queue
*q_self,Quoted 22 lines
drivers/net/ethernet/microsoft/mana/hw_channel.c new file +22
-0+ struct gdma_event *event) +{ +struct hw_channel_context *hwc = ctx; +struct gdma_dev *gd = hwc->gdma_dev; +union hwc_init_type_data type_data; +union hwc_init_eq_id_db eq_db; +u32 type, val; + +switch (event->type) { +case GDMA_EQE_HWC_INIT_EQ_ID_DB : + eq_db.as_uint32 = event->details[0 ]; + hwc->cq->gdma_eq->id = eq_db.eq_id; + gd->doorbell = eq_db.doorbell; +break ; + +case GDMA_EQE_HWC_INIT_DATA : + + type_data.as_uint32 = event->details[0 ]; + +case GDMA_EQE_HWC_INIT_DONE : +complete (&hwc->hwc_init_eqe_comp); +break ;...
Quoted 7 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c line 335
Quoted 4 lines
drivers/net/ethernet/microsoft/mana/gdma_main.c new file +4
-0+default : +WARN_ON (1 ); +break ; + }Are these events from the firmware? If you have newer firmware with an older driver, are you going to spam the kernel log with
WARN_ONdumps?
Quoted 3 lines
For protocol version mismatch, the host and guest will either negotiate the highest common version, or fail to probe. So this kind of warnings are not expected.
I agree, but I think we'd better remove the WARN_ON(1), which was mainly for debugging purposem, and was added just in case.
Thanks,
Dexuan
Dexuan Cui
maintainerQuoted 8 lines
From: Jakub Kicinski <[email protected]>
Sent: Monday, April 12, 2021 11:21 AM
...drivers/net/ethernet/microsoft/mana/mana_en.c line 1612
Quoted 2 lines · On Sun, 11 Apr 2021 19:34:55 -0700 Dexuan Cui wrote:
drivers/net/ethernet/microsoft/mana/mana_en.c new file +2
-0+for (i =0 ; i <ANA_INDIRECT_TABLE_SIZE ; i++) + apc->indir_table[i] = i % apc->num_queues;
ethtool_rxfh_indir_default()
Will use ethtool_rxfh_indir_default().
Quoted 6 lines
drivers/net/ethernet/microsoft/mana/mana_en.c line 1630
Quoted 2 lines
drivers/net/ethernet/microsoft/mana/mana_en.c new file +2
-0+ err =mana_cfg_vport_steering (apc, rx, true, update_hash, update_tab); +return err;return
mana_...please fix everywhere.
Will fix this one, and will review if there is any similar issue.
Quoted 8 lines
drivers/net/ethernet/microsoft/mana/mana_en.c
Quoted 7 lines
drivers/net/ethernet/microsoft/mana/mana_en.c new file +7
-0+netif_set_real_num_tx_queues (ndev, apc->num_queues); + + err =mana_add_rx_queues (apc, ndev); +if (err) + goto destroy_vport; + + apc->rss_state = apc->num_queues > 1 ? TRI_STATE_TRUE :
TRI_STATE_FALSE;
Quoted 4 lines
drivers/net/ethernet/microsoft/mana/mana_en.c line 1678
Quoted 2 lines
drivers/net/ethernet/microsoft/mana/mana_en.c new file +2
-0+ +netif_set_real_num_rx_queues (ndev, apc->num_queues);
netif_set_real_num_.. can fail.
Will fix the error handling.
Quoted 17 lines
drivers/net/ethernet/microsoft/mana/mana_en.c
Quoted 5 lines
drivers/net/ethernet/microsoft/mana/mana_en.c new file +5
-0+rtnl_lock (); + +netdev_lockdep_set_classes (ndev); + + ndev->hw_features = NETIF_F_SG | NETIF_F_IP_CSUM |
NETIF_F_IPV6_CSUM;Quoted 11 lines
drivers/net/ethernet/microsoft/mana/mana_en.c new file +11
-0+ ndev->hw_features |=NETIF_F_RXCSUM ; + ndev->hw_features |=NETIF_F_TSO |NETIF_F_TSO6 ; + ndev->hw_features |=NETIF_F_RXHASH ; + ndev->features = ndev->hw_features; + ndev->vlan_features =0 ; + + err =register_netdevice (ndev); +if (err) { +netdev_err (ndev,"Unable to register netdev.\n" ); + goto destroy_vport; + }
Quoted 8 lines
drivers/net/ethernet/microsoft/mana/mana_en.c line 1839
Quoted 6 lines
drivers/net/ethernet/microsoft/mana/mana_en.c new file +6
-0+ +rtnl_unlock (); + +return 0 ; +destroy_vport: +rtnl_unlock ();Why do you take
rtnl_lock()explicitly around this code?
It looks like there is no good reason, and I guess we just copied the code from netvsc_probe(), where the RTNL lock is indeed explicitly needed.
Will change to directly use register_netdev(), which gets and release the RTNL lock automatically.
Quoted 15 lines
drivers/net/ethernet/microsoft/mana/mana_ethtool.c
Quoted 13 lines
drivers/net/ethernet/microsoft/mana/mana_ethtool.c new file +13
-0+static int mana_set_channels (struct net_device *ndev, +struct ethtool_channels *channels) +{ +struct ana_port_context *apc =netdev_priv (ndev); +unsigned int new_count; +unsigned int old_count; +int err, err2; + + new_count = channels->combined_count; + old_count = apc->num_queues; + + if (new_count < 1 || new_count > apc->max_queues || + channels->rx_count || channels->tx_count ||
channels->other_count)
Quoted 8 lines
All these checks should be done by the core already.
drivers/net/ethernet/microsoft/mana/mana_ethtool.c line 216
Quoted 4 lines
drivers/net/ethernet/microsoft/mana/mana_ethtool.c new file +4
-0+return -EINVAL ; + +if (new_count == old_count) +return 0 ;And so is this one.
Will change the code to avoid unnecessary checking.
Thanks,
Dexuan