Skip to contentSkip to conversation

VNIC: Adding support for Cavium ThunderX network controller

PATCH Superseded Robert Richter 4 participants 8 messages Oct 30, 2014 – Nov 3, 2014

Subsystems (5)
Networking Network drivers PCI arm64 +1 more subsystems

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 VNIC: Adding support for Cavium ThunderX network controller

Dave,

the patch below is the initial version of our network controller driver for Cavium's ThunderX boards.

The patch is also available here:

git://git.kernel.org/pub/scm/linux/kernel/git/rric/linux.git tags/cavium-for-netdev-v3.19

Please consider it for inclusion into your tree.

Thanks,

-Robert

Quoted 1 line

From e5db4968386e2ab3fc054be4fa4389a1a7125588 Mon Sep 17 00:00:00 2001

From: Sunil Goutham <[email protected]>
Date: Fri, 8 Aug 2014 16:26:12 +0200
Subject: [PATCH] VNIC: Adding support for Cavium ThunderX network controller

This patch adds support for the Cavium ThunderX network controller. The driver is on the pci bus and thus requires the Thunder PCIe host controller driver to be enabled.

---
 MAINTAINERS                                        |    7 +
 arch/arm64/Kconfig                                 |    1 +
 drivers/net/ethernet/Kconfig                       |    1 +
 drivers/net/ethernet/Makefile                      |    1 +
 drivers/net/ethernet/cavium/Kconfig                |   40 +
 drivers/net/ethernet/cavium/Makefile               |    5 +
 drivers/net/ethernet/cavium/thunder/Makefile       |   13 +
 drivers/net/ethernet/cavium/thunder/nic.h          |  434 +++++++
 drivers/net/ethernet/cavium/thunder/nic_main.c     |  807 ++++++++++++
 drivers/net/ethernet/cavium/thunder/nic_reg.h      |  214 ++++
 .../net/ethernet/cavium/thunder/nicvf_ethtool.c    |  478 +++++++
 drivers/net/ethernet/cavium/thunder/nicvf_main.c   | 1326 ++++++++++++++++++++
 drivers/net/ethernet/cavium/thunder/nicvf_queues.c | 1302 +++++++++++++++++++
 drivers/net/ethernet/cavium/thunder/nicvf_queues.h |  355 ++++++
 drivers/net/ethernet/cavium/thunder/q_struct.h     |  690 ++++++++++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.c  |  386 ++++++
 drivers/net/ethernet/cavium/thunder/thunder_bgx.h  |   78 ++
 include/linux/pci_ids.h                            |    2 +
 18 files changed, 6140 insertions(+)
 create mode 100644 drivers/net/ethernet/cavium/Kconfig
 create mode 100644 drivers/net/ethernet/cavium/Makefile
 create mode 100644 drivers/net/ethernet/cavium/thunder/Makefile
 create mode 100644 drivers/net/ethernet/cavium/thunder/nic.h
 create mode 100644 drivers/net/ethernet/cavium/thunder/nic_main.c
 create mode 100644 drivers/net/ethernet/cavium/thunder/nic_reg.h
 create mode 100644 drivers/net/ethernet/cavium/thunder/nicvf_ethtool.c
 create mode 100644 drivers/net/ethernet/cavium/thunder/nicvf_main.c
 create mode 100644 drivers/net/ethernet/cavium/thunder/nicvf_queues.c
 create mode 100644 drivers/net/ethernet/cavium/thunder/nicvf_queues.h
 create mode 100644 drivers/net/ethernet/cavium/thunder/q_struct.h
 create mode 100644 drivers/net/ethernet/cavium/thunder/thunder_bgx.c
 create mode 100644 drivers/net/ethernet/cavium/thunder/thunder_bgx.h
MAINTAINERS +7 -0
index a20df9bf8ab0..9a36569d75df 100644
@@ -882,6 +882,13 @@ M:	Krzysztof Halasa <[email protected]>
 S:	Maintained
 F:	arch/arm/mach-cns3xxx/
 
+ARM/CAVIUM THUNDER ARCHITECTURE
+M:	Sunil Goutham <[email protected]>
+M:	Robert Richter <[email protected]>
+L:	[email protected] (moderated for non-subscribers)
+S:	Supported
+F:	drivers/net/ethernet/cavium/
+
 ARM/CIRRUS LOGIC CLPS711X ARM ARCHITECTURE
 M:	Alexander Shiyan <[email protected]>
 L:	[email protected] (moderated for non-subscribers)
arch/arm64/Kconfig +1 -0
index ac9afde76dea..bc14b1a78ae2 100644
@@ -145,6 +145,7 @@ config ARCH_THUNDER
 	bool "Cavium Inc. Thunder SoC Family"
 	help
 	  This enables support for Cavium's Thunder Family of SoCs.
+	select NET_VENDOR_CAVIUM
 
 config ARCH_VEXPRESS
 	bool "ARMv8 software model (Versatile Express)"
drivers/net/ethernet/Kconfig +1 -0
index 1ed1fbba5d58..7c872ff82aed 100644
@@ -34,6 +34,7 @@ source "drivers/net/ethernet/adi/Kconfig"
 source "drivers/net/ethernet/broadcom/Kconfig"
 source "drivers/net/ethernet/brocade/Kconfig"
 source "drivers/net/ethernet/calxeda/Kconfig"
+source "drivers/net/ethernet/cavium/Kconfig"
 source "drivers/net/ethernet/chelsio/Kconfig"
 source "drivers/net/ethernet/cirrus/Kconfig"
 source "drivers/net/ethernet/cisco/Kconfig"
drivers/net/ethernet/Makefile +1 -0
index 6e0b629e9859..7a9f7d7b55bf 100644
@@ -20,6 +20,7 @@ obj-$(CONFIG_NET_BFIN) += adi/
 obj-$(CONFIG_NET_VENDOR_BROADCOM) += broadcom/
 obj-$(CONFIG_NET_VENDOR_BROCADE) += brocade/
 obj-$(CONFIG_NET_CALXEDA_XGMAC) += calxeda/
+obj-$(CONFIG_NET_VENDOR_CAVIUM) += cavium/
 obj-$(CONFIG_NET_VENDOR_CHELSIO) += chelsio/
 obj-$(CONFIG_NET_VENDOR_CIRRUS) += cirrus/
 obj-$(CONFIG_NET_VENDOR_CISCO) += cisco/
drivers/net/ethernet/cavium/Kconfig new file +40 -0
new file mode 100644
index 000000000000..6365fb4242be
@@ -0,0 +1,40 @@
+#
+# Cavium ethernet device configuration
+#
+
+config NET_VENDOR_CAVIUM
+	tristate "Cavium ethernet drivers"
+	depends on PCI
+	---help---
+	  Enable support for the Cavium ThunderX Network Interface
+	  Controller (NIC). The NIC provides the controller and DMA
+	  engines to move network traffic to/from the memory. The NIC
+	  works closely with TNS, BGX and SerDes to implement the
+	  functions replacing and virtualizing those of a typical
+	  standalone PCIe NIC chip.
+
+	  If you have a Cavium Thunder board, say Y.
+
+if NET_VENDOR_CAVIUM
+
+config THUNDER_NIC_PF
+	tristate "Thunder Physical function driver"
+	default NET_VENDOR_CAVIUM
+	select THUNDER_NIC_BGX
+	---help---
+	  This driver supports Thunder's NIC physical function.
+
+config THUNDER_NIC_VF
+	tristate "Thunder Virtual function driver"
+	default NET_VENDOR_CAVIUM
+	---help---
+	  This driver supports Thunder's NIC virtual function
+
+config	THUNDER_NIC_BGX
+	tristate "Thunder MAC interface driver (BGX)"
+	default NET_VENDOR_CAVIUM
+	---help---
+	  This driver supports programming and controlling of MAC
+	  interface from NIC physical function driver.
+
+endif # NET_VENDOR_CAVIUM
drivers/net/ethernet/cavium/Makefile new file +5 -0
new file mode 100644
index 000000000000..7aac4780d050
@@ -0,0 +1,5 @@
+#
+# Makefile for the Cavium ethernet device drivers.
+#
+
+obj-$(CONFIG_NET_VENDOR_CAVIUM) += thunder/
drivers/net/ethernet/cavium/thunder/Makefile new file +13 -0
new file mode 100644
index 000000000000..8ee6e043f8ff
@@ -0,0 +1,13 @@
+#
+# Makefile for Cavium's Thunder ethernet device
+#
+
+
+# Don't change the order, NICPF driver is dependent on BGX driver init
+obj-$(CONFIG_THUNDER_NIC_BGX) += thunder_bgx.o
+obj-$(CONFIG_THUNDER_NIC_PF) += nicpf.o
+obj-$(CONFIG_THUNDER_NIC_VF) += nicvf.o
+
+nicpf-y := nic_main.o
+nicvf-y := nicvf_main.o nicvf_queues.o
+nicvf-y += nicvf_ethtool.o
drivers/net/ethernet/cavium/thunder/nic.h new file +434 -0

434 lines not on this page · raw message

drivers/net/ethernet/cavium/thunder/nic_main.c new file +807 -0

807 lines not on this page · raw message

drivers/net/ethernet/cavium/thunder/nic_reg.h new file +214 -0

214 lines not on this page · raw message

drivers/net/ethernet/cavium/thunder/nicvf_ethtool.c new file +478 -0

478 lines not on this page · raw message

drivers/net/ethernet/cavium/thunder/nicvf_main.c new file +1326 -0

975 lines not on this page · raw message

drivers/net/ethernet/cavium/thunder/nicvf_queues.c new file +1302 -0

1,302 lines not on this page · raw message

drivers/net/ethernet/cavium/thunder/nicvf_queues.h new file +355 -0

355 lines not on this page · raw message

drivers/net/ethernet/cavium/thunder/q_struct.h new file +690 -0

690 lines not on this page · raw message

drivers/net/ethernet/cavium/thunder/thunder_bgx.c new file +386 -0

386 lines not on this page · raw message

drivers/net/ethernet/cavium/thunder/thunder_bgx.h new file +78 -0

78 lines not on this page · raw message

include/linux/pci_ids.h +2 -0
index 1fa99a301817..80bd3336691e 100644

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.1.1

Part of the diff is not shown here. View the raw message.

#1 lore raw

include/linux/pci_ids.h line 2327

Quoted 9 lines · On Thu, 30 Oct 2014 17:54:34 +0100 Robert Richter <[email protected]> wrote:
include/linux/pci_ids.h +1 -0
index 1fa99a301817..80bd3336691e 100644
@@ -2324,6 +2324,8 @@
 #define PCI_DEVICE_ID_ALTIMA_AC9100	0x03ea
 #define PCI_DEVICE_ID_ALTIMA_AC1003	0x03eb
 
+#define PCI_VENDOR_ID_CAVIUM		0x177d

I don't think PCI folks want this updated with every id anymore.

#2 lore raw
Quoted 11 lines · Stephen Hemminger
Quoted 8 lines · On Thu, 30 Oct 2014 17:54:34 +0100 Robert Richter <[email protected]> wrote:
include/linux/pci_ids.h +0 -0
index 1fa99a301817..80bd3336691e 100644
@@ -2324,6 +2324,8 @@
 #define PCI_DEVICE_ID_ALTIMA_AC9100	0x03ea
 #define PCI_DEVICE_ID_ALTIMA_AC1003	0x03eb
 
Quoted 3 lines

include/linux/pci_ids.h line 2327

Quoted 1 line
include/linux/pci_ids.h +1 -0
+#define PCI_VENDOR_ID_CAVIUM		0x177d

I don't think PCI folks want this updated with every id anymore.

This is just the vendor id, the device id is part of the driver.

Since there will be multiple drivers I put the vendor id here.

-Robert

#3 lore raw
In reply to Robert Richter

drivers/net/ethernet/cavium/thunder/nicvf_main.c line 46

Quoted 6 lines · On Thu, 30 Oct 2014 17:54:34 +0100 Robert Richter <[email protected]> wrote:
drivers/net/ethernet/cavium/thunder/nicvf_main.c new file +6 -0
+#ifdef	VNIC_RSS_SUPPORT
+static int rss_config = RSS_IP_HASH_ENA | RSS_TCP_HASH_ENA | RSS_UDP_HASH_ENA;
+module_param(rss_config, int, S_IRUGO);
+MODULE_PARM_DESC(rss_config,
+		 "RSS hash config [bits 8:0] (Bit0:L2 extended, 1:IP, 2:TCP, 3:TCP SYN, 4:UDP, 5:L4 extended, 6:ROCE 7:L3 bi-directional, 8:L4 bi-directional)");
+#endif

This should managed be via ethtool ETHTOOL_GRXFH rather than a module parameter.

#4 lore raw
Quoted 11 lines · Stephen Hemminger

drivers/net/ethernet/cavium/thunder/nicvf_main.c line 46

Quoted 6 lines · On Thu, 30 Oct 2014 17:54:34 +0100 Robert Richter <[email protected]> wrote:
drivers/net/ethernet/cavium/thunder/nicvf_main.c new file +6 -0
+#ifdef       VNIC_RSS_SUPPORT
+static int rss_config = RSS_IP_HASH_ENA | RSS_TCP_HASH_ENA | RSS_UDP_HASH_ENA;
+module_param(rss_config, int, S_IRUGO);
+MODULE_PARM_DESC(rss_config,
+              "RSS hash config [bits 8:0] (Bit0:L2 extended, 1:IP, 2:TCP, 3:TCP SYN, 4:UDP, 5:L4 extended, 6:ROCE 7:L3 bi-directional, 8:L4 bi-directional)");
+#endif

This should managed be via ethtool ETHTOOL_GRXFH rather than a module parameter.

Thanks, i will add setting hash options via ETHTOOL_SRXFH as well. The idea here is to have a choice of hash while module load (through module params) and if it needs to be changed runtime then via Ethtool.

Sunil.

#5 lore raw
Quoted 13 lines · On Fri, 31 Oct 2014 22:44:11 +0530 Sunil Kovvuri <[email protected]>…
Quoted 11 lines · Stephen Hemminger

drivers/net/ethernet/cavium/thunder/nicvf_main.c line 46

Quoted 6 lines · On Thu, 30 Oct 2014 17:54:34 +0100 Robert Richter <[email protected]> wrote:
drivers/net/ethernet/cavium/thunder/nicvf_main.c new file +6 -0
+#ifdef       VNIC_RSS_SUPPORT
+static int rss_config = RSS_IP_HASH_ENA | RSS_TCP_HASH_ENA | RSS_UDP_HASH_ENA;
+module_param(rss_config, int, S_IRUGO);
+MODULE_PARM_DESC(rss_config,
+              "RSS hash config [bits 8:0] (Bit0:L2 extended, 1:IP, 2:TCP, 3:TCP SYN, 4:UDP, 5:L4 extended, 6:ROCE 7:L3 bi-directional, 8:L4 bi-directional)");
+#endif

This should managed be via ethtool ETHTOOL_GRXFH rather than a module parameter.

Quoted 6 lines

Thanks, i will add setting hash options via ETHTOOL_SRXFH as well. The idea here is to have a choice of hash while module load (through module params) and if it needs to be changed runtime then via Ethtool.

Sunil.

Network developers do not like vendor unique module parameters. Anything device specific doesn't work in a generic distro environment.

#6 lore raw
Quoted 21 lines · Stephen Hemminger
Quoted 18 lines · On Fri, 31 Oct 2014 22:44:11 +0530 Sunil Kovvuri <[email protected]>…
Quoted 11 lines · Stephen Hemminger

drivers/net/ethernet/cavium/thunder/nicvf_main.c line 46

Quoted 6 lines · On Thu, 30 Oct 2014 17:54:34 +0100 Robert Richter <[email protected]> wrote:
drivers/net/ethernet/cavium/thunder/nicvf_main.c new file +6 -0
+#ifdef       VNIC_RSS_SUPPORT
+static int rss_config = RSS_IP_HASH_ENA | RSS_TCP_HASH_ENA | RSS_UDP_HASH_ENA;
+module_param(rss_config, int, S_IRUGO);
+MODULE_PARM_DESC(rss_config,
+              "RSS hash config [bits 8:0] (Bit0:L2 extended, 1:IP, 2:TCP, 3:TCP SYN, 4:UDP, 5:L4 extended, 6:ROCE 7:L3 bi-directional, 8:L4 bi-directional)");
+#endif

This should managed be via ethtool ETHTOOL_GRXFH rather than a module parameter.

Thanks, i will add setting hash options via ETHTOOL_SRXFH as well. The idea here is to have a choice of hash while module load (through module params) and if it needs to be changed runtime then via Ethtool.

Quoted 4 lines
Quoted 1 line

Sunil.

Network developers do not like vendor unique module parameters. Anything device specific doesn't work in a generic distro environment.

Do you accept unique module parameters in parallel to ethtool support or should this be removed?

-Robert

#7 lore raw
Quoted 22 lines · On Mon, 3 Nov 2014 19:33:45 +0100 Robert Richter…
Quoted 21 lines · Stephen Hemminger
Quoted 18 lines · On Fri, 31 Oct 2014 22:44:11 +0530 Sunil Kovvuri <[email protected]>…
Quoted 11 lines · Stephen Hemminger

drivers/net/ethernet/cavium/thunder/nicvf_main.c line 46

Quoted 6 lines · On Thu, 30 Oct 2014 17:54:34 +0100 Robert Richter <[email protected]> wrote:
drivers/net/ethernet/cavium/thunder/nicvf_main.c new file +6 -0
+#ifdef       VNIC_RSS_SUPPORT
+static int rss_config = RSS_IP_HASH_ENA | RSS_TCP_HASH_ENA | RSS_UDP_HASH_ENA;
+module_param(rss_config, int, S_IRUGO);
+MODULE_PARM_DESC(rss_config,
+              "RSS hash config [bits 8:0] (Bit0:L2 extended, 1:IP, 2:TCP, 3:TCP SYN, 4:UDP, 5:L4 extended, 6:ROCE 7:L3 bi-directional, 8:L4 bi-directional)");
+#endif

This should managed be via ethtool ETHTOOL_GRXFH rather than a module parameter.

Thanks, i will add setting hash options via ETHTOOL_SRXFH as well. The idea here is to have a choice of hash while module load (through module params) and if it needs to be changed runtime then via Ethtool.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Sunil.

Network developers do not like vendor unique module parameters. Anything device specific doesn't work in a generic distro environment.

Do you accept unique module parameters in parallel to ethtool support or should this be removed?

If there is ethtool support the module parameters are not needed. Unneeded code is to be avoided.