net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)
Dexuan Cui
maintainerPATCH v3 1/1 net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)
Add a VF driver for Microsoft Azure Network Adapter (MANA) that will be available in the future.
--- MAINTAINERS | 4 +- drivers/net/ethernet/Kconfig | 1 + drivers/net/ethernet/Makefile | 1 + drivers/net/ethernet/microsoft/Kconfig | 29 + drivers/net/ethernet/microsoft/Makefile | 5 + drivers/net/ethernet/microsoft/mana/Makefile | 6 + drivers/net/ethernet/microsoft/mana/gdma.h | 728 +++++++ .../net/ethernet/microsoft/mana/gdma_main.c | 1515 ++++++++++++++ .../net/ethernet/microsoft/mana/hw_channel.c | 859 ++++++++ .../net/ethernet/microsoft/mana/hw_channel.h | 186 ++ drivers/net/ethernet/microsoft/mana/mana.h | 531 +++++ drivers/net/ethernet/microsoft/mana/mana_en.c | 1827 +++++++++++++++++ .../ethernet/microsoft/mana/mana_ethtool.c | 278 +++ .../net/ethernet/microsoft/mana/shm_channel.c | 292 +++ .../net/ethernet/microsoft/mana/shm_channel.h | 21 + 15 files changed, 6282 insertions(+), 1 deletion(-) create mode 100644 drivers/net/ethernet/microsoft/Kconfig create mode 100644 drivers/net/ethernet/microsoft/Makefile create mode 100644 drivers/net/ethernet/microsoft/mana/Makefile create mode 100644 drivers/net/ethernet/microsoft/mana/gdma.h create mode 100644 drivers/net/ethernet/microsoft/mana/gdma_main.c create mode 100644 drivers/net/ethernet/microsoft/mana/hw_channel.c create mode 100644 drivers/net/ethernet/microsoft/mana/hw_channel.h create mode 100644 drivers/net/ethernet/microsoft/mana/mana.h create mode 100644 drivers/net/ethernet/microsoft/mana/mana_en.c create mode 100644 drivers/net/ethernet/microsoft/mana/mana_ethtool.c create mode 100644 drivers/net/ethernet/microsoft/mana/shm_channel.c create mode 100644 drivers/net/ethernet/microsoft/mana/shm_channel.h
Changes in v2:
Removed the module_param(num_queues,...). [Andrew Lunn]
Changed pr_err() to netdev_err() and dev_err(). [Andrew Lunn]
Used reverse X-mas tree style in all the functions. [Andrew Lunn]
Changed "= { 0 };" to "= {};" for struct variables. [Leon Romanovsky]
Addressed 3 build failures on i386, arc and powerpc by making the the driver dependent on X86_64 in Kconfig: so far, the driver is only validated on X86_64 (in the future, we may enable the driver for ARM64).
Also made some cosmetic changes.
Rebased the patch to the latest net-next.
BTW, the support of XDP and BQL haven't been implemented. They are on our TO-DO list.
Changes in v3:
Changed the Kconfig file:
Microsoft Azure Network Device ==> Microsoft Network Devices Drop the default m
validated ==> supported
Fixed some warnings reported by "kernel test robot" (there are
still some warnings against another driver pci-hyperv.c, which I'll fix in a separate patch for the PCI tree).
MAINTAINERS +3 -1
@@ -8261,11 +8261,12 @@ S: Maintained T: git git://linuxtv.org/media_tree.git F: drivers/media/i2c/hi556.c -Hyper-V CORE AND DRIVERS +Hyper-V/Azure CORE AND DRIVERS M: "K. Y. Srinivasan" <[email protected]> M: Haiyang Zhang <[email protected]> M: Stephen Hemminger <[email protected]> M: Wei Liu <[email protected]> +M: Dexuan Cui <[email protected]> L: [email protected] S: Supported T: git git://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux.git
@@ -8282,6 +8283,7 @@ F: drivers/hid/hid-hyperv.c F: drivers/hv/ F: drivers/input/serio/hyperv-keyboard.c F: drivers/iommu/hyperv-iommu.c +F: drivers/net/ethernet/microsoft/ F: drivers/net/hyperv/ F: drivers/pci/controller/pci-hyperv-intf.c F: drivers/pci/controller/pci-hyperv.c
drivers/net/ethernet/Kconfig +1 -0
@@ -82,6 +82,7 @@ source "drivers/net/ethernet/huawei/Kconfig" source "drivers/net/ethernet/i825xx/Kconfig" source "drivers/net/ethernet/ibm/Kconfig" source "drivers/net/ethernet/intel/Kconfig" +source "drivers/net/ethernet/microsoft/Kconfig" source "drivers/net/ethernet/xscale/Kconfig" config JME
drivers/net/ethernet/Makefile +1 -0
@@ -45,6 +45,7 @@ obj-$(CONFIG_NET_VENDOR_HUAWEI) += huawei/ obj-$(CONFIG_NET_VENDOR_IBM) += ibm/ obj-$(CONFIG_NET_VENDOR_INTEL) += intel/ obj-$(CONFIG_NET_VENDOR_I825XX) += i825xx/ +obj-$(CONFIG_NET_VENDOR_MICROSOFT) += microsoft/ obj-$(CONFIG_NET_VENDOR_XSCALE) += xscale/ obj-$(CONFIG_JME) += jme.o obj-$(CONFIG_KORINA) += korina.o
drivers/net/ethernet/microsoft/Kconfig new file +29 -0
@@ -0,0 +1,29 @@ +# +# Microsoft Azure network device configuration +# + +config NET_VENDOR_MICROSOFT + bool "Microsoft Network Devices" + default y + help + If you have a network (Ethernet) device belonging to this class, say Y. + + Note that the answer to this question doesn't directly affect the + kernel: saying N will just cause the configurator to skip the + question about Microsoft network devices. If you say Y, you will be + asked for your specific device in the following question. + +if NET_VENDOR_MICROSOFT + +config MICROSOFT_MANA + tristate "Microsoft Azure Network Adapter (MANA) support" + depends on PCI_MSI && X86_64 + select PCI_HYPERV + help + This driver supports Microsoft Azure Network Adapter (MANA). + So far, the driver is only supported on X86_64. + + To compile this driver as a module, choose M here. + The module will be called mana. + +endif #NET_VENDOR_MICROSOFT
drivers/net/ethernet/microsoft/Makefile new file +5 -0
@@ -0,0 +1,5 @@ +# +# Makefile for the Microsoft Azure network device driver. +# + +obj-$(CONFIG_MICROSOFT_MANA) += mana/
drivers/net/ethernet/microsoft/mana/Makefile new file +6 -0
@@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause +# +# Makefile for the Microsoft Azure Network Adapter driver + +obj-$(CONFIG_MICROSOFT_MANA) += mana.o +mana-objs := gdma_main.o shm_channel.o hw_channel.o mana_en.o mana_ethtool.o
drivers/net/ethernet/microsoft/mana/gdma.h new file +728 -0
728 lines not on this page · raw message
drivers/net/ethernet/microsoft/mana/gdma_main.c new file +1515 -0
1,515 lines not on this page · raw message
drivers/net/ethernet/microsoft/mana/hw_channel.c new file +859 -0
683 lines not on this page · raw message
drivers/net/ethernet/microsoft/mana/hw_channel.h new file +186 -0
186 lines not on this page · raw message
drivers/net/ethernet/microsoft/mana/mana.h new file +531 -0
531 lines not on this page · raw message
drivers/net/ethernet/microsoft/mana/mana_en.c new file +1827 -0
1,827 lines not on this page · raw message
drivers/net/ethernet/microsoft/mana/mana_ethtool.c new file +278 -0
278 lines not on this page · raw message
drivers/net/ethernet/microsoft/mana/shm_channel.c new file +292 -0
292 lines not on this page · raw message
drivers/net/ethernet/microsoft/mana/shm_channel.h new file +21 -0
Lines of this file are not shown. View the raw message.
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.20.1
Part of the diff is not shown here. View the raw message.
Quoted 27 lines · On Thu, 8 Apr 2021 15:58:40 -0700 Dexuan Cui <[email protected]> wrote:
Add a VF driver for Microsoft Azure Network Adapter (MANA) that will be available in the future.
Co-developed-by Haiyang Zhang <[email protected]>Signed-off-by Haiyang Zhang <[email protected]>Signed-off-by Dexuan Cui <[email protected]>--- MAINTAINERS | 4 +- drivers/net/ethernet/Kconfig | 1 + drivers/net/ethernet/Makefile | 1 + drivers/net/ethernet/microsoft/Kconfig | 29 + drivers/net/ethernet/microsoft/Makefile | 5 + drivers/net/ethernet/microsoft/mana/Makefile | 6 + drivers/net/ethernet/microsoft/mana/gdma.h | 728 +++++++ .../net/ethernet/microsoft/mana/gdma_main.c | 1515 ++++++++++++++ .../net/ethernet/microsoft/mana/hw_channel.c | 859 ++++++++ .../net/ethernet/microsoft/mana/hw_channel.h | 186 ++ drivers/net/ethernet/microsoft/mana/mana.h | 531 +++++ drivers/net/ethernet/microsoft/mana/mana_en.c | 1827 +++++++++++++++++ .../ethernet/microsoft/mana/mana_ethtool.c | 278 +++ .../net/ethernet/microsoft/mana/shm_channel.c | 292 +++ .../net/ethernet/microsoft/mana/shm_channel.h | 21 + 15 files changed, 6282 insertions(+), 1 deletion(-) create mode 100644 drivers/net/ethernet/microsoft/Kconfig create mode 100644 drivers/net/ethernet/microsoft/Makefile create mode 100644 drivers/net/ethernet/microsoft/mana/Makefile create mode 100644 drivers/net/ethernet/microsoft/mana/gdma.h
Quoted 8 lines
create mode 100644 drivers/net/ethernet/microsoft/mana/gdma_main.c create mode 100644 drivers/net/ethernet/microsoft/mana/hw_channel.c create mode 100644 drivers/net/ethernet/microsoft/mana/hw_channel.h create mode 100644 drivers/net/ethernet/microsoft/mana/mana.h create mode 100644 drivers/net/ethernet/microsoft/mana/mana_en.c create mode 100644 drivers/net/ethernet/microsoft/mana/mana_ethtool.c create mode 100644 drivers/net/ethernet/microsoft/mana/shm_channel.c create mode 100644 drivers/net/ethernet/microsoft/mana/shm_channel.h
Linux kernel doesn't do namespaces in the code, so every new driver needs to worry about global symbols clashing
Please make sure there are not name conflicts with other drivers around variable or functions name "gdma_". Noticed there is one driver in staging using similar names (drivers/staging/ralink-gdma/ralink-gdma.c). Granted that is in the staging doghouse so probably not important but might show up as a name conflict with something like the randconfig testing.
Quoted 2 lines
Linux kernel doesn't do namespaces in the code, so every new driver needs to worry about global symbols clashing
This driver is called mana, yet the code uses ana. It would be good to resolve this inconsistency as well. Ideally, you want to prefix everything with ana_ or mana_, depending on what you choose, so we have a clean namespace.
Andrew
Dexuan Cui
maintainerQuoted 7 lines
From: Andrew Lunn <[email protected]>
Sent: Thursday, April 8, 2021 5:30 PM
To: Stephen Hemminger <[email protected]>
...Quoted 2 lines
Linux kernel doesn't do namespaces in the code, so every new driver needs to worry about global symbols clashing
Quoted 6 lines
This driver is called mana, yet the code uses ana. It would be good to resolve this inconsistency as well. Ideally, you want to prefix everything with
ana_ormana_, depending on what you choose, so we have a clean namespace.Andrew
Thanks for the suggestion! Let me think about this and work out a solution.
David Miller
maintainerFrom: Dexuan Cui <[email protected]>
Date: Thu, 8 Apr 2021 15:58:40 -0700
Quoted 8 lines
drivers/net/ethernet/microsoft/mana/gdma.h new file +8
-0+struct gdma_msg_hdr { +u32 hdr_type; +u32 msg_type; +u16 msg_version; +u16 hwc_msg_id; +u32 msg_size; +} __packed; +
drivers/net/ethernet/microsoft/mana/gdma.h line 119
Quoted 10 lines
drivers/net/ethernet/microsoft/mana/gdma.h new file +10
-0+struct gdma_dev_id { +union { +struct { +u16 type; +u16 instance; + }; + +u32 as_uint32; + }; +} __packed;
Please don't use __packed unless absolutely necessary. It generates suboptimal code (byte at a time accesses etc.) and for many of these you don't even need it.
Thank you.
Dexuan Cui
maintainerQuoted 17 lines
From: David Miller <[email protected]>
Sent: Thursday, April 8, 2021 4:46 PM
...Quoted 14 lines
drivers/net/ethernet/microsoft/mana/gdma.h new file +14
-0+struct gdma_msg_hdr { +u32 hdr_type; +u32 msg_type; +u16 msg_version; +u16 hwc_msg_id; +u32 msg_size; +} __packed; + +struct gdma_dev_id { +union { +struct { +u16 type; +u16 instance; + };
Quoted 8 lines
drivers/net/ethernet/microsoft/mana/gdma.h line 119
Quoted 4 lines
drivers/net/ethernet/microsoft/mana/gdma.h new file +4
-0+ +u32 as_uint32; + }; +} __packed;Please don't use
__packedunless absolutely necessary. It generates suboptimal code (byte at a time
accesses etc.) and for many of these you don't even need it.
In the driver code, all the structs/unions marked by __packed are used to talk with the hardware, so I think __packed is necessary here?
Do you think if it's better if we remove all the __packed, and add static_assert(sizeof(struct XXX) == YYY) instead? e.g.
@@ -105,7 +105,8 @@ struct gdma_msg_hdr {
u16 msg_version;
u16 hwc_msg_id;
u32 msg_size;-} __packed;
+};
+static_assert(sizeof(struct gdma_msg_hdr) == 16);
struct gdma_dev_id {
union {Now I'm trying to figure out how other NIC drivers define structs/unions.
Thanks,
Dexuan
David Miller
maintainerFrom: Dexuan Cui <[email protected]>
Date: Fri, 9 Apr 2021 00:24:51 +0000
Quoted 26 lines
Quoted 17 lines
From: David Miller <[email protected]>
Sent: Thursday, April 8, 2021 4:46 PM
...Quoted 14 lines
drivers/net/ethernet/microsoft/mana/gdma.h new file +14
-0+struct gdma_msg_hdr { +u32 hdr_type; +u32 msg_type; +u16 msg_version; +u16 hwc_msg_id; +u32 msg_size; +} __packed; + +struct gdma_dev_id { +union { +struct { +u16 type; +u16 instance; + };Quoted 8 lines
drivers/net/ethernet/microsoft/mana/gdma.h line 119
Quoted 4 lines
drivers/net/ethernet/microsoft/mana/gdma.h new file +4
-0+ +u32 as_uint32; + }; +} __packed;Please don't use
__packedunless absolutely necessary. It generates suboptimal code (byte at a time
accesses etc.) and for many of these you don't even need it.
Quoted 2 lines
In the driver code, all the structs/unions marked by
__packedare used to talk with the hardware, so I think__packedis necessary here?
It actually isan't in many cases, check with and without the __packed directive and see if anything chasnges.
Review struct gdma_msg_hdr { · line 109
Quoted 10 lines
Do you think if it's better if we remove all the
__packed, and addstatic_assert(sizeof(struct XXX) == YYY)instead? e.g.(quoted diff) +2
-1@@ -105,7 +105,8 @@ struct gdma_msg_hdr { u16 msg_version; u16 hwc_msg_id; u32 msg_size; -} __packed; +}; +static_assert(sizeof(struct gdma_msg_hdr) == 16);
This won't make sure the structure member offsets are what you expect.
I think you'll have to go through the structures one-by-one by hand to figure out which ones really require the __packed attribute and which do not.
Thank you.
Dexuan Cui
maintainerQuoted 8 lines
From: David Miller <[email protected]>
Sent: Thursday, April 8, 2021 5:41 PMQuoted 3 lines
...
In the driver code, all the structs/unions marked by__packedare used to talk with the hardware, so I think__packedis necessary here?It actually isan't in many cases, check with and without the
__packeddirective and see if anything chasnges.
Will do.
Quoted 13 lines
Review struct gdma_msg_hdr { · line 109
Quoted 10 lines
Do you think if it's better if we remove all the
__packed, and addstatic_assert(sizeof(struct XXX) == YYY)instead? e.g.(quoted diff) +2
-1@@ -105,7 +105,8 @@ struct gdma_msg_hdr { u16 msg_version; u16 hwc_msg_id; u32 msg_size; -} __packed; +}; +static_assert(sizeof(struct gdma_msg_hdr) == 16);This won't make sure the structure member offsets are what you expect.
Quoted 2 lines
I think you'll have to go through the structures one-by-one by hand to figure out which ones really require the
__packedattribute and which do not.
Got it. Let me see if I can remove all the __packed.
Haiyang Zhang
maintainerQuoted 64 lines
-----Original Message-----
From: David Miller <[email protected]>
Sent: Thursday, April 8, 2021 8:41 PM
To: Dexuan Cui <[email protected]>
Cc: [email protected]; KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; Stephen Hemminger <[email protected]>; [email protected]; Wei Liu <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux- [email protected]; [email protected]
Subject: Re: [PATCH v3 net-next] net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)From: Dexuan Cui <[email protected]>
Date: Fri, 9 Apr 2021 00:24:51 +0000Quoted 26 lines
Quoted 17 lines
From: David Miller <[email protected]>
Sent: Thursday, April 8, 2021 4:46 PM
...Quoted 14 lines
drivers/net/ethernet/microsoft/mana/gdma.h new file +14
-0+struct gdma_msg_hdr { +u32 hdr_type; +u32 msg_type; +u16 msg_version; +u16 hwc_msg_id; +u32 msg_size; +} __packed; + +struct gdma_dev_id { +union { +struct { +u16 type; +u16 instance; + };Quoted 8 lines
drivers/net/ethernet/microsoft/mana/gdma.h line 119
Quoted 4 lines
drivers/net/ethernet/microsoft/mana/gdma.h new file +4
-0+ +u32 as_uint32; + }; +} __packed;Please don't use
__packedunless absolutely necessary. It generates suboptimal code (byte at a time
accesses etc.) and for many of these you don't even need it.Quoted 2 lines
In the driver code, all the structs/unions marked by
__packedare used to talk with the hardware, so I think__packedis necessary here?It actually isan't in many cases, check with and without the
__packeddirective
and see if anything chasnges.Review struct gdma_msg_hdr { · line 109
Quoted 10 lines
Do you think if it's better if we remove all the
__packed, and addstatic_assert(sizeof(struct XXX) == YYY)instead? e.g.(quoted diff) +2
-1@@ -105,7 +105,8 @@ struct gdma_msg_hdr { u16 msg_version; u16 hwc_msg_id; u32 msg_size; -} __packed; +}; +static_assert(sizeof(struct gdma_msg_hdr) == 16);This won't make sure the structure member offsets are what you expect.
Quoted 2 lines
I think you'll have to go through the structures one-by-one by hand to figure out which ones really require the
__packedattribute and which do not.
For the structs containing variables with the same sizes, or already size aligned variables, we knew the __packed has no effect. And for these structs, it doesn't cause performance impact either, correct?
But in the future, if different sized variables are added, the __packed may become necessary again. To prevent anyone accidently forget to add __packed when adding new variables to these structs, can we keep the __packed for all messages going through the "wire"?
Thanks,
- Haiyang
For the structs containing variables with the same sizes, or already size aligned
Quoted 7 lines
variables, we knew the
__packedhas no effect. And for these structs, it doesn't cause performance impact either, correct?But in the future, if different sized variables are added, the
__packedmay become necessary again. To prevent anyone accidently forget to add__packedwhen adding new variables to these structs, can we keep the__packedfor all messages going through the "wire"?
It should not be a problem because anybody adding new variables should know packed is not liked in the kernel and will take care.
If you want to be paranoid add a BUILD_BUG_ON(size(struct foo) != 42);
Andrew
Quoted 5 lines · Dexuan Cui
Changes in v3:
Changed theKconfigfile:
Microsoft Azure Network Device ==> Microsoft Network Devices Drop the default m
validated ==> supported
Hi,
I am satisfied with the Kconfig changes.
thanks.
Signature · 1 line
~Randy