Skip to contentSkip to conversation

net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)

PATCH In review Dexuan Cui 2 participants 6 messages Apr 16, 2021 – Apr 22, 2021

Subsystems (4)
lore
Share & tools
lore

Dexuan Cui

author maintainer
#0 lore raw

v8 cover net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)

The patchset adds the VF driver for Microsoft Azure Network Adapter (MANA), and also changes the hv_netvsc driver's netvsc/VF binding logic to check both the MAC address and the serial number (this is required by the MANA VF driver).

v7 contains both the netvsc change and the VF driver. This version (v8) posts them in 2 separate patches, as suggested by Stephen Hemminger.

Please refer to "[PATCH v8 net-next 2/2]" for the history of v1~v7.

Thanks,
Dexuan

Dexuan Cui (2):
hv_netvsc: Make netvsc/VF binding check both MAC and serial number
net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)

 MAINTAINERS                                   |    4 +-
 drivers/net/ethernet/Kconfig                  |    1 +
 drivers/net/ethernet/Makefile                 |    1 +
 drivers/net/ethernet/microsoft/Kconfig        |   29 +
 drivers/net/ethernet/microsoft/Makefile       |    5 +
 drivers/net/ethernet/microsoft/mana/Makefile  |    6 +
 drivers/net/ethernet/microsoft/mana/gdma.h    |  673 ++++++
 .../net/ethernet/microsoft/mana/gdma_main.c   | 1415 ++++++++++++
 .../net/ethernet/microsoft/mana/hw_channel.c  |  843 ++++++++
 .../net/ethernet/microsoft/mana/hw_channel.h  |  190 ++
 drivers/net/ethernet/microsoft/mana/mana.h    |  533 +++++
 drivers/net/ethernet/microsoft/mana/mana_en.c | 1895 +++++++++++++++++
 .../ethernet/microsoft/mana/mana_ethtool.c    |  250 +++
 .../net/ethernet/microsoft/mana/shm_channel.c |  291 +++
 .../net/ethernet/microsoft/mana/shm_channel.h |   21 +
 drivers/net/hyperv/netvsc_drv.c               |   14 +-
 16 files changed, 6168 insertions(+), 3 deletions(-)
 create mode 100644 drivers/net/ethernet/microsoft/Kconfig
 create mode 100644 drivers/net/ethernet/microsoft/Makefile
 create mode 100644 drivers/net/ethernet/microsoft/mana/Makefile
 create mode 100644 drivers/net/ethernet/microsoft/mana/gdma.h
 create mode 100644 drivers/net/ethernet/microsoft/mana/gdma_main.c
 create mode 100644 drivers/net/ethernet/microsoft/mana/hw_channel.c
 create mode 100644 drivers/net/ethernet/microsoft/mana/hw_channel.h
 create mode 100644 drivers/net/ethernet/microsoft/mana/mana.h
 create mode 100644 drivers/net/ethernet/microsoft/mana/mana_en.c
 create mode 100644 drivers/net/ethernet/microsoft/mana/mana_ethtool.c
 create mode 100644 drivers/net/ethernet/microsoft/mana/shm_channel.c
 create mode 100644 drivers/net/ethernet/microsoft/mana/shm_channel.h
Signature · 1 line
2.25.1
author maintainer
#1 lore raw

PATCH v8 1/2 hv_netvsc: Make netvsc/VF binding check both MAC and serial number

Currently the netvsc/VF binding logic only checks the PCI serial number.

The upcoming Microsoft Azure Network Adapter (MANA) supports multiple net_device interfaces (each such interface is called a "vPort", and has its unique MAC address) which are backed by the same VF PCI device, so the binding logic should check both the MAC address and the PCI serial number.

The change should not break any other existing VF drivers, because Hyper-V NIC SR-IOV implementation requires the netvsc network interface and the VF network interface have the same MAC address.

---
 drivers/net/hyperv/netvsc_drv.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
drivers/net/hyperv/netvsc_drv.c +12 -2
index 7349a70af083..f682a5572d84 100644
@@ -2297,6 +2297,7 @@ static struct net_device *get_netvsc_byslot(const struct net_device *vf_netdev)
 {
 	struct device *parent = vf_netdev->dev.parent;
 	struct net_device_context *ndev_ctx;
+	struct net_device *ndev;
 	struct pci_dev *pdev;
 	u32 serial;
 
@@ -2319,8 +2320,17 @@ static struct net_device *get_netvsc_byslot(const struct net_device *vf_netdev)
 		if (!ndev_ctx->vf_alloc)
 			continue;
 
-		if (ndev_ctx->vf_serial == serial)
-			return hv_get_drvdata(ndev_ctx->device_ctx);
+		if (ndev_ctx->vf_serial != serial)
+			continue;
+
+		ndev = hv_get_drvdata(ndev_ctx->device_ctx);
+		if (ndev->addr_len != vf_netdev->addr_len ||
+		    memcmp(ndev->perm_addr, vf_netdev->perm_addr,
+			   ndev->addr_len) != 0)
+			continue;
+
+		return ndev;
+
 	}
 
 	netdev_notice(vf_netdev,
Signature · 1 line
2.25.1
Acked-by on v8 1/2
#2 lore raw

Re: [PATCH v8 net-next 1/2] hv_netvsc: Make netvsc/VF binding check both MAC and serial number

Quoted 12 lines · On Fri, 16 Apr 2021 13:11:58 -0700 Dexuan Cui <[email protected]> wrote:

Currently the netvsc/VF binding logic only checks the PCI serial number.

The upcoming Microsoft Azure Network Adapter (MANA) supports multiple net_device interfaces (each such interface is called a "vPort", and has its unique MAC address) which are backed by the same VF PCI device, so the binding logic should check both the MAC address and the PCI serial number.

The change should not break any other existing VF drivers, because Hyper-V NIC SR-IOV implementation requires the netvsc network interface and the VF network interface have the same MAC address.

Quoted 5 lines
author maintainer
#3 lore raw

RE: [PATCH v8 net-next 1/2] hv_netvsc: Make netvsc/VF binding check both MAC and serial number

Quoted 19 lines

From: Stephen Hemminger <[email protected]>
Sent: Monday, April 19, 2021 8:54 AM
To: Dexuan Cui <[email protected]>
...

Quoted 12 lines · On Fri, 16 Apr 2021 13:11:58 -0700 Dexuan Cui <[email protected]> wrote:

Currently the netvsc/VF binding logic only checks the PCI serial number.

The upcoming Microsoft Azure Network Adapter (MANA) supports multiple net_device interfaces (each such interface is called a "vPort", and has its unique MAC address) which are backed by the same VF PCI device, so the binding logic should check both the MAC address and the PCI serial number.

The change should not break any other existing VF drivers, because Hyper-V NIC SR-IOV implementation requires the netvsc network interface and the VF network interface have the same MAC address.

Quoted 7 lines
Quoted 5 lines

Hi David, Jakub,
The "2/2" patch has been in the net-next tree since Monday, but this "1/2" patch is not in -- can you please pick up this patch as well? This patch is needed by the "2/2" patch.

Thanks,
-- Dexuan

author maintainer
#4 lore raw

PATCH v8 2/2 net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)

In reply to Dexuan Cui

Add a VF driver for Microsoft Azure Network Adapter (MANA) that will be available in the future.

---
 MAINTAINERS                                   |    4 +-
 drivers/net/ethernet/Kconfig                  |    1 +
 drivers/net/ethernet/Makefile                 |    1 +
 drivers/net/ethernet/microsoft/Kconfig        |   29 +
 drivers/net/ethernet/microsoft/Makefile       |    5 +
 drivers/net/ethernet/microsoft/mana/Makefile  |    6 +
 drivers/net/ethernet/microsoft/mana/gdma.h    |  673 ++++++
 .../net/ethernet/microsoft/mana/gdma_main.c   | 1415 ++++++++++++
 .../net/ethernet/microsoft/mana/hw_channel.c  |  843 ++++++++
 .../net/ethernet/microsoft/mana/hw_channel.h  |  190 ++
 drivers/net/ethernet/microsoft/mana/mana.h    |  533 +++++
 drivers/net/ethernet/microsoft/mana/mana_en.c | 1895 +++++++++++++++++
 .../ethernet/microsoft/mana/mana_ethtool.c    |  250 +++
 .../net/ethernet/microsoft/mana/shm_channel.c |  291 +++
 .../net/ethernet/microsoft/mana/shm_channel.h |   21 +
 15 files changed, 6156 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/ethernet/microsoft/Kconfig
 create mode 100644 drivers/net/ethernet/microsoft/Makefile
 create mode 100644 drivers/net/ethernet/microsoft/mana/Makefile
 create mode 100644 drivers/net/ethernet/microsoft/mana/gdma.h
 create mode 100644 drivers/net/ethernet/microsoft/mana/gdma_main.c
 create mode 100644 drivers/net/ethernet/microsoft/mana/hw_channel.c
 create mode 100644 drivers/net/ethernet/microsoft/mana/hw_channel.h
 create mode 100644 drivers/net/ethernet/microsoft/mana/mana.h
 create mode 100644 drivers/net/ethernet/microsoft/mana/mana_en.c
 create mode 100644 drivers/net/ethernet/microsoft/mana/mana_ethtool.c
 create mode 100644 drivers/net/ethernet/microsoft/mana/shm_channel.c
 create mode 100644 drivers/net/ethernet/microsoft/mana/shm_channel.h

Changes in v2:

Removed the module_param(num_queues,...). [Andrew Lunn]
Changed pr_err() to netdev_err() and dev_err(). [Andrew Lunn]
Used reverse X-mas tree style in all the functions. [Andrew Lunn]

Changed "= { 0 };" to "= {};" for struct variables. [Leon Romanovsky]

Addressed 3 build failures on i386, arc and powerpc by making the the driver dependent on X86_64 in Kconfig: so far, the driver is only validated on X86_64 (in the future, we may enable the driver for ARM64).

Also made some cosmetic changes.

Rebased the patch to the latest net-next.

BTW, the support of XDP and BQL haven't been implemented. They are on our TO-DO list.

Changes in v3:
Changed the Kconfig file:
Microsoft Azure Network Device ==> Microsoft Network Devices Drop the default m
validated ==> supported

Fixed some warnings reported by "kernel test robot" (there are
still some warnings against another driver pci-hyperv.c, which I'll fix in a separate patch for the PCI tree).

Changes in v4:
Removed all the "__packed" usages since all the structs are actually naturally aligned. We verified there is no hole n the middle of the structs or at the end of the structs.

Prefixed all the functions with "mana_".

Integrated Shachar Raindel's multi-vPort patches (i.e. create multiple net devices on the single VF PCIe device, if the PF driver reports the support). Added Shachar's Co-developed-by and Signed-off-by.

Also made some cosmetic changes.
Rebased the patch to the latest net-next.

Changes in v5:
Made a small change to drivers/net/hyperv/netvsc_drv.c's get_netvsc_byslot(): now with the multi-vPorts-per-PCI-dev, the netvsc driver needs to check both the serial number and the MAC address to find the matching vPort net device. Note: this change doesn't break any existing VF drivers.

Prefixed all the functions with "mana_". (Previously some were missed). Now there is no "ana" or "ANA" in the code.

Changed the "drv version" to "proto version". We don't have driver version. The correct name in mana_query_device_cfg() should be "proto version".
Fixed the defines and the struct field names accordingly.

Used ethtool_rxfh_indir_default() rather than direct "i % apc->num_queues;"

Handled the error in case netif_set_real_num_tx_queues() fails.

Used register_netdev() since we don't really need to explicitly get and release the RTNL lock.

Changed some WARN_ON()'s to WARN_ON_ONCE()'s. Removed a few unnecessary WARN_ON()'s.
Used "WARN_ON(xxx);" rather than "if (xxx) WARN_ON(1);"

Used "return foo();" rather than "err = foo(); return err;" in several places.

Removed unnecessary checking in mana_set_channels().

Also made some small cosmetic changes.
Rebased the patch to the latest net-next.

Changes in v6:
Used mana_do_attach()/mana_detach() in mana_open()/mana_close(). This way the related resources (e.g. SQ, RQ, etc.) are not allocated until the net interface goes UP.

Fixed IRQ allocation for the multi-vPorts case. Currently each vPort has its own event queues (EQs), and in future we'll share EQs among all the vPorts (a vPort means a net_device).

Added smp_rmb() in mana_poll_tx_cq(): ensure checking txq_stopped before checking apc->port_is_up.

Used PAGE_ALIGNED(), PAGE_ALIGN(), PHYS_PFN() where appropriate.

Removed some code that isn't really used at present, e.g. the below flags are not really used at present:
GDMA_WR_SGL_DIRECT
GDMA_WR_CONSUME_CREDIT
GDMA_WR_FENCE
GDMA_WR_CHECK_SN
so removed the code using the flags, mainly in mana_gd_write_sgl() and mana_gd_write_client_oob().

Removed unused event types in mana_gd_process_eqe(), and also simplified the function a little bit.

Rebased the patch to the latest net-next.

Changes in v7:
Removed enum mana_attach_caller {MANA_OPEN, MANA_ATTACH,MANA_PROBE}. Split mana_do_attach() and mana_detech() into different functions:
mana_init_port(), mana_alloc_queues(), mana_dealloc_queues, so now mana_open(), mana_close(), mana_probe() and mana_probe_port(),etc. are more readable.

Fixed a race condition in mana_hwc_send_request(): moved mana_hwc_put_msg_index() from mana_hwc_rx_event_handler() to the end of mana_hwc_send_request(), otherwise the msg index is released too early and a new request's response may overwrite the previous req's response buffer which may not have been used yet.

Dropped the __GFP_ZERO flag when calling dma_alloc_coherent().

Removed some unnecessary empty lines.

Fixed the 'queue' memory leak issue in a few places if mana_gd_alloc_memory() fails.

Used ether_addr_copy() rather than memcpy().

Used netdev_rss_key_fill() rather than get_random_bytes().

Removed some dead code in mana_send_request().

Rebased the patch to the latest net-next.

Changes in v8:
v7 contains both the netvsc change and the VF driver. This version (v8) posts them in 2 separate patches.

MAINTAINERS +3 -1
index 795b9941c151..3b75bc7c51f8 100644
@@ -8260,11 +8260,12 @@ S:	Maintained
 T:	git git://linuxtv.org/media_tree.git
 F:	drivers/media/i2c/hi556.c
 
-Hyper-V CORE AND DRIVERS
+Hyper-V/Azure CORE AND DRIVERS
 M:	"K. Y. Srinivasan" <[email protected]>
 M:	Haiyang Zhang <[email protected]>
 M:	Stephen Hemminger <[email protected]>
 M:	Wei Liu <[email protected]>
+M:	Dexuan Cui <[email protected]>
 L:	[email protected]
 S:	Supported
 T:	git git://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux.git
@@ -8281,6 +8282,7 @@ F:	drivers/hid/hid-hyperv.c
 F:	drivers/hv/
 F:	drivers/input/serio/hyperv-keyboard.c
 F:	drivers/iommu/hyperv-iommu.c
+F:	drivers/net/ethernet/microsoft/
 F:	drivers/net/hyperv/
 F:	drivers/pci/controller/pci-hyperv-intf.c
 F:	drivers/pci/controller/pci-hyperv.c
drivers/net/ethernet/Kconfig +1 -0
index 4b85f2b74872..d46460c5b44d 100644
@@ -82,6 +82,7 @@ source "drivers/net/ethernet/huawei/Kconfig"
 source "drivers/net/ethernet/i825xx/Kconfig"
 source "drivers/net/ethernet/ibm/Kconfig"
 source "drivers/net/ethernet/intel/Kconfig"
+source "drivers/net/ethernet/microsoft/Kconfig"
 source "drivers/net/ethernet/xscale/Kconfig"
 
 config JME
drivers/net/ethernet/Makefile +1 -0
index 9394493e8187..cb3f9084a21b 100644
@@ -45,6 +45,7 @@ obj-$(CONFIG_NET_VENDOR_HUAWEI) += huawei/
 obj-$(CONFIG_NET_VENDOR_IBM) += ibm/
 obj-$(CONFIG_NET_VENDOR_INTEL) += intel/
 obj-$(CONFIG_NET_VENDOR_I825XX) += i825xx/
+obj-$(CONFIG_NET_VENDOR_MICROSOFT) += microsoft/
 obj-$(CONFIG_NET_VENDOR_XSCALE) += xscale/
 obj-$(CONFIG_JME) += jme.o
 obj-$(CONFIG_KORINA) += korina.o
drivers/net/ethernet/microsoft/Kconfig new file +29 -0
new file mode 100644
index 000000000000..e1ac0a5d808d
@@ -0,0 +1,29 @@
+#
+# Microsoft Azure network device configuration
+#
+
+config NET_VENDOR_MICROSOFT
+	bool "Microsoft Network Devices"
+	default y
+	help
+	  If you have a network (Ethernet) device belonging to this class, say Y.
+
+	  Note that the answer to this question doesn't directly affect the
+	  kernel: saying N will just cause the configurator to skip the
+	  question about Microsoft network devices. If you say Y, you will be
+	  asked for your specific device in the following question.
+
+if NET_VENDOR_MICROSOFT
+
+config MICROSOFT_MANA
+	tristate "Microsoft Azure Network Adapter (MANA) support"
+	depends on PCI_MSI && X86_64
+	select PCI_HYPERV
+	help
+	  This driver supports Microsoft Azure Network Adapter (MANA).
+	  So far, the driver is only supported on X86_64.
+
+	  To compile this driver as a module, choose M here.
+	  The module will be called mana.
+
+endif #NET_VENDOR_MICROSOFT
drivers/net/ethernet/microsoft/Makefile new file +5 -0
new file mode 100644
index 000000000000..d2ddc218135f
@@ -0,0 +1,5 @@
+#
+# Makefile for the Microsoft Azure network device driver.
+#
+
+obj-$(CONFIG_MICROSOFT_MANA) += mana/
drivers/net/ethernet/microsoft/mana/Makefile new file +6 -0
new file mode 100644
index 000000000000..0edd5bb685f3
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
+#
+# Makefile for the Microsoft Azure Network Adapter driver
+
+obj-$(CONFIG_MICROSOFT_MANA) += mana.o
+mana-objs := gdma_main.o shm_channel.o hw_channel.o mana_en.o mana_ethtool.o
drivers/net/ethernet/microsoft/mana/gdma.h new file +673 -0

673 lines not on this page · raw message

drivers/net/ethernet/microsoft/mana/gdma_main.c new file +1415 -0

1,415 lines not on this page · raw message

drivers/net/ethernet/microsoft/mana/hw_channel.c new file +843 -0

838 lines not on this page · raw message

drivers/net/ethernet/microsoft/mana/hw_channel.h new file +190 -0

190 lines not on this page · raw message

drivers/net/ethernet/microsoft/mana/mana.h new file +533 -0

533 lines not on this page · raw message

drivers/net/ethernet/microsoft/mana/mana_en.c new file +1895 -0

1,895 lines not on this page · raw message

drivers/net/ethernet/microsoft/mana/mana_ethtool.c new file +250 -0

250 lines not on this page · raw message

drivers/net/ethernet/microsoft/mana/shm_channel.c new file +291 -0

291 lines not on this page · raw message

drivers/net/ethernet/microsoft/mana/shm_channel.h new file +21 -0
new file mode 100644
index 000000000000..5199b41497ff

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.25.1

Part of the diff is not shown here. View the raw message.

Reviewed-by on v8 2/2
#5 lore raw
Quoted 8 lines · On Fri, 16 Apr 2021 13:11:59 -0700 Dexuan Cui <[email protected]> wrote:

Add a VF driver for Microsoft Azure Network Adapter (MANA) that will be available in the future.