Skip to contentSkip to conversation

Add support for the Turris Mox router

PATCH Superseded Marek Behún 4 participants 18 messages Aug 8, 2018 – Aug 21, 2018

Subsystems (10)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover Add support for the Turris Mox router

This adds preliminary support for the Turris Mox SOHO router developed by CZ.NIC to mainline kernel.

The first patch fixes the corresponding pinctrl driver.

The second patch adds a new type of bus called "moxtet" to the kernel, via which the different Mox modules can be found and configured.

The third patch adds support for GPIOs found on the module with SFP cage. These GPIOs are configured via the "moxtet" bus.

The fourth and fifth patches add support for watchdog found on Armada 37xx SOCs.

The sixth patch fixes a weird bug I keep encountering in the mvneta driver on Armada 3720 SOCs. This may be deleted or changed in the future version of this patchset depending on what the networking guys will think about this.

The seventh patch adds Turris Mox device tree.

Marek

#1 lore raw

PATCH 1/7 From: Ken Ma <[email protected]>

pinctrl: armada-37xx: Correct mpp definitions

This patch corrects below mpp definitions:
- The sdio_sb group is composed of 6 pins and not 5;
- The rgmii group contains pins mpp2[17:6] and not mpp2[19:6];
- Pin of group "pmic0" is mpp1[6] but not mpp1[16];
- Pin of group "pmic1" is mpp1[7] but not mpp1[17];
- A new group "smi" is added in A0 with 2 pins - mpp2[19:18], its bitmask is bit4;
- Group "pcie1" has 3 pins in A0 - mpp2[5:3], its bit mask is bit5 | bit9 | bit10 but not bit4;
- Group "ptp" has 3 pins in A0 as Z1, but its bitmask is changed to bit11 | bit12 | bit13.

drivers/pinctrl/mvebu/pinctrl-armada-37xx.c +5 -4
index 53cf800688e9..95abadee8542 100644
@@ -157,8 +157,8 @@ static struct armada_37xx_pin_group armada_37xx_nb_groups[] = {
 	PIN_GRP_GPIO("pwm1", 12, 1, BIT(4), "pwm"),
 	PIN_GRP_GPIO("pwm2", 13, 1, BIT(5), "pwm"),
 	PIN_GRP_GPIO("pwm3", 14, 1, BIT(6), "pwm"),
-	PIN_GRP_GPIO("pmic1", 17, 1, BIT(7), "pmic"),
-	PIN_GRP_GPIO("pmic0", 16, 1, BIT(8), "pmic"),
+	PIN_GRP_GPIO("pmic1", 7, 1, BIT(7), "pmic"),
+	PIN_GRP_GPIO("pmic0", 6, 1, BIT(8), "pmic"),
 	PIN_GRP_GPIO("i2c2", 2, 2, BIT(9), "i2c"),
 	PIN_GRP_GPIO("i2c1", 0, 2, BIT(10), "i2c"),
 	PIN_GRP_GPIO("spi_cs1", 17, 1, BIT(12), "spi"),
@@ -182,8 +182,9 @@ static struct armada_37xx_pin_group armada_37xx_sb_groups[] = {
 	PIN_GRP_GPIO("usb2_drvvbus1", 1, 1, BIT(1), "drvbus"),
 	PIN_GRP_GPIO("sdio_sb", 24, 6, BIT(2), "sdio"),
 	PIN_GRP_GPIO("rgmii", 6, 12, BIT(3), "mii"),
-	PIN_GRP_GPIO("pcie1", 3, 2, BIT(4), "pcie"),
-	PIN_GRP_GPIO("ptp", 20, 3, BIT(5), "ptp"),
+	PIN_GRP_GPIO("smi", 18, 2, BIT(4), "smi"),
+	PIN_GRP_GPIO("pcie1", 3, 3, BIT(5) | BIT(9) | BIT(10), "pcie"),
+	PIN_GRP_GPIO("ptp", 20, 3, BIT(11) | BIT(12) | BIT(13), "ptp"),
 	PIN_GRP("ptp_clk", 21, 1, BIT(6), "ptp", "mii"),
 	PIN_GRP("ptp_trig", 22, 1, BIT(7), "ptp", "mii"),
 	PIN_GRP_GPIO_3("mii_col", 23, 1, BIT(8) | BIT(14), 0, BIT(8), BIT(14),
Signature · 1 line
2.16.4
maintainer
#2 lore raw

PATCH 1/7 From: Ken Ma <[email protected]>

Quoted 1 line · Marek Beh?n

pinctrl: armada-37xx: Correct mpp definitions

Hi Marek

The subject line is messed up.

Quoted 13 lines

This patch corrects below mpp definitions:
- The sdio_sb group is composed of 6 pins and not 5;
- The rgmii group contains pins mpp2[17:6] and not mpp2[19:6];
- Pin of group "pmic0" is mpp1[6] but not mpp1[16];
- Pin of group "pmic1" is mpp1[7] but not mpp1[17];
- A new group "smi" is added in A0 with 2 pins - mpp2[19:18], its bitmask is bit4;
- Group "pcie1" has 3 pins in A0 - mpp2[5:3], its bit mask is bit5 | bit9 | bit10 but not bit4;
- Group "ptp" has 3 pins in A0 as Z1, but its bitmask is changed to bit11 | bit12 | bit13.

Quoted 1 line

This probably should should have the original Marvell SOB as well.

Andrew

#3 lore raw

PATCH 2/7 drivers: mfd: Add support for Moxtet bus

In reply to Marek Behún

On the Turris Mox router there can be connected different modules to the main CPU board, currently a module with a SFP cage, a module with MiniPCIe connector, a 4-port switch module and a 8-port switch module, for example:
[CPU]-[PCIe]-[8-port switch]-[8-port switch]-[SFP]

Each of this modules has an input and output shift register, and these are connected via SPI to CPU board.

Via this SPI connection we are able to discover which modules are connected and we can also read/write some configuration to the modules. Fromi/to each module 8 bits can be read (of which lower 4 bits identify the module) and written.

For example from the module with a SFP cage we can read the LOS, TX-FAULT and MOD-DEF0 signals, while we can write TX-DISABLE and RATE-SELECT signals.

Other modules may support something else.

This driver creates a new bus type, called "moxtet". For each Mox module it finds via SPI, it creates a new device on the moxtet bus so that drivers can be written for them, for example a gpio driver for the module with a SFP cage.

The topology of how Mox modules are connected can then be read by listing /sys/bus/moxtet/devices.

 create mode 100644 Documentation/devicetree/bindings/mfd/moxtet.txt
 create mode 100644 drivers/mfd/moxtet.c
 create mode 100644 include/linux/mfd/moxtet.h
Documentation/devicetree/bindings/mfd/moxtet.txt new file +36 -0
new file mode 100644
index 000000000000..02b96fbd5ddd
@@ -0,0 +1,36 @@
+Turris Mox module configuration bus (over SPI)
+
+Required properties:
+ - compatible		: Should be "cznic,moxtet".
+ - #address-cells	: Has to be 1
+ - #size-cells		: Has to be 0
+For other required and optional properties of SPI slave
+nodes please refer to ../spi/spi-bus.txt.
+
+Required properties of subnodes:
+ - reg			: Should be position on the Moxtet bus
+ - moxtet,id		: Should be ID of the Moxtet device connected
+
+The driver finds the devices connected to the bus by itself, but it may be
+needed to reference some of them from other parts of the device tree. In that
+case the devices can be defined as subnodes of the moxtet node.
+
+Example:
+
+	moxtet at 1 {
+		#address-cells = <1>;
+		#size-cells = <0>;
+		compatible = "cznic,moxtet";
+		reg = <1>;
+		spi-max-frequency = <1000000>;
+		spi-cpol;
+		spi-cpha;
+
+		moxtet_sfp: moxtet-sfp at 0 {
+			compatible = "cznic,moxtet-sfp";
+			gpio-controller;
+			#gpio-cells;
+			reg = <0>;
+			moxtet,id = <1>;
+		}
+	};
MAINTAINERS +7 -0
index 7cebd5bba8a8..27ca12e8309a 100644
@@ -1378,6 +1378,13 @@ F:	drivers/clocksource/timer-prima2.c
 F:	drivers/clocksource/timer-atlas7.c
 N:	[^a-z]sirf
 
+ARM/CZ.NIC TURRIS MOX SUPPORT
+M:	Marek Behun <[email protected]>
+W:	http://mox.turris.cz
+S:	Maintained
+F:	include/mfd/moxtet.h
+F:	drivers/mfd/moxtet.c
+
 ARM/EBSA110 MACHINE SUPPORT
 M:	Russell King <[email protected]>
 L:	linux-arm-kernel at lists.infradead.org (moderated for non-subscribers)
drivers/mfd/Kconfig +10 -0
index b860eb5aa194..395d8b15f8a8 100644
@@ -784,6 +784,16 @@ config MFD_MAX8998
 	  additional drivers must be enabled in order to use the functionality
 	  of the device.
 
+config MFD_MOXTET
+	tristate "CZ.NIC Turris Mox module configuration bus"
+	depends on SPI_MASTER && OF
+	help
+	  Say yes here to add support for the module configuration bus found
+	  on CZ.NIC's Turris Mox. This is needed for the ability to read
+	  in what order the modules are connected and to get/set some of
+	  their settings. For example the GPIOs on Mox SFP module are
+	  configured through this bus.
+
 config MFD_MT6397
 	tristate "MediaTek MT6397 PMIC Support"
 	select MFD_CORE
drivers/mfd/Makefile +1 -0
index e9fd20dba18d..1401863e27b1 100644
@@ -147,6 +147,7 @@ max8925-objs			:= max8925-core.o max8925-i2c.o
 obj-$(CONFIG_MFD_MAX8925)	+= max8925.o
 obj-$(CONFIG_MFD_MAX8997)	+= max8997.o max8997-irq.o
 obj-$(CONFIG_MFD_MAX8998)	+= max8998.o max8998-irq.o
+obj-$(CONFIG_MFD_MOXTET)	+= moxtet.o
 
 pcf50633-objs			:= pcf50633-core.o pcf50633-irq.o
 obj-$(CONFIG_MFD_PCF50633)	+= pcf50633.o
drivers/mfd/moxtet.c new file +497 -0

497 lines not on this page · raw message

include/linux/mfd/moxtet.h new file +103 -0

103 lines not on this page · raw message

Signature · 1 line
2.16.4
#4 lore raw

PATCH 2/7 drivers: mfd: Add support for Moxtet bus

I found out that there are two calls to dev_err with too many arguments, will patch in next version.

drivers/mfd/moxtet.c line 298

Quoted 12 lines · On Wed, 8 Aug 2018 17:27:01 +0200 Marek Beh?n <[email protected]> wrote:
drivers/mfd/moxtet.c new file +10 -0
+static void
+moxtet_register_devices_from_topology(struct moxtet *moxtet)
+{
+	struct moxtet_device *dev;
+	int i, ret;
+
+	for (i = 0; i < moxtet->count; ++i) {
+		dev = moxtet_alloc_device(moxtet);
+		if (!dev) {
+			dev_err(moxtet->dev, "Moxtet device %u alloc

error\n",

drivers/mfd/moxtet.c new file +1 -0
+				i, moxtet->modules[i]);

here

drivers/mfd/moxtet.c line 309

Quoted 12 lines
drivers/mfd/moxtet.c new file +10 -0
+			continue;
+		}
+
+		dev->idx = i;
+		dev->id = moxtet->modules[i];
+
+		ret = moxtet_add_device(dev);
+		if (ret && ret != -EBUSY) {
+			dev_err(moxtet->dev,
+				"Moxtet device %u register error:

%i\n", i,

drivers/mfd/moxtet.c new file +1 -0
+				moxtet->modules[i], ret);

and here

Quoted 3 lines
drivers/mfd/moxtet.c new file +3 -0
+		}
+	}
+}
#5 lore raw

PATCH 3/7 drivers: gpio: Add support for Turris Mox SFP GPIOs

In reply to Marek Behún

The SFP cage GPIOs on the SFP cage module of Turris Mox can be configured via the moxtet bus.

This driver supports those GPIOs so that they can be used by phylink.

 create mode 100644 Documentation/devicetree/bindings/gpio/gpio-moxtet-sfp.txt
 create mode 100644 drivers/gpio/gpio-moxtet-sfp.c
Documentation/devicetree/bindings/gpio/gpio-moxtet-sfp.txt new file +16 -0
new file mode 100644
index 000000000000..a8267aef7d2c
@@ -0,0 +1,16 @@
+GPIO configuration of the SFP cage found on Turris Mox (over Moxtet bus)
+
+Required properties:
+ - compatible		: Should be "cznic,moxtet-sfp".
+ - gpio-controller	: Marks the device node as a GPIO controller.
+ - #gpio-cells		: Should be two. For consumer use see gpio.txt.
+
+Example:
+
+	moxtet_sfp: moxtet-sfp at 0 {
+		compatible = "cznic,moxtet-sfp";
+		gpio-controller;
+		#gpio-cells;
+		reg = <0>;
+		moxtet,id = <1>;
+	}
MAINTAINERS +1 -0
index 27ca12e8309a..cfac1b21596c 100644
@@ -1383,6 +1383,7 @@ M:	Marek Behun <[email protected]>
 W:	http://mox.turris.cz
 S:	Maintained
 F:	include/mfd/moxtet.h
+F:	drivers/gpio/gpio-moxtet-sfp.c
 F:	drivers/mfd/moxtet.c
 
 ARM/EBSA110 MACHINE SUPPORT
drivers/gpio/Kconfig +9 -0
index 71c0ab46f216..bd6e52de99c5 100644
@@ -1048,6 +1048,15 @@ config GPIO_MAX77620
 	  driver also provides interrupt support for each of the gpios.
 	  Say yes here to enable the max77620 to be used as gpio controller.
 
+config GPIO_MOXTET_SFP
+	tristate "Turris Mox SFP GPIO expander"
+	depends on MFD_MOXTET
+	help
+	  Say yes here if you are building for the Turris Mox router.
+	  This is the driver needed for configuring the GPIOs found on the
+	  module with SFP cage of the Turris Mox router.
+	  This driver uses the Moxtet bus.
+
 config GPIO_MSIC
 	bool "Intel MSIC mixed signal gpio support"
 	depends on (X86 || COMPILE_TEST) && MFD_INTEL_MSIC
drivers/gpio/Makefile +3 -2
index 1324c8f966a7..ba464186c468 100644
@@ -84,7 +84,8 @@ obj-$(CONFIG_GPIO_MC33880)	+= gpio-mc33880.o
 obj-$(CONFIG_GPIO_MC9S08DZ60)	+= gpio-mc9s08dz60.o
 obj-$(CONFIG_GPIO_ML_IOH)	+= gpio-ml-ioh.o
 obj-$(CONFIG_GPIO_MM_LANTIQ)	+= gpio-mm-lantiq.o
-obj-$(CONFIG_GPIO_MOCKUP)      += gpio-mockup.o
+obj-$(CONFIG_GPIO_MOCKUP)	+= gpio-mockup.o
+obj-$(CONFIG_GPIO_MOXTET_SFP)	+= gpio-moxtet-sfp.o
 obj-$(CONFIG_GPIO_MPC5200)	+= gpio-mpc5200.o
 obj-$(CONFIG_GPIO_MPC8XXX)	+= gpio-mpc8xxx.o
 obj-$(CONFIG_GPIO_MSIC)		+= gpio-msic.o
@@ -100,7 +101,7 @@ obj-$(CONFIG_GPIO_PCI_IDIO_16)	+= gpio-pci-idio-16.o
 obj-$(CONFIG_GPIO_PCIE_IDIO_24)	+= gpio-pcie-idio-24.o
 obj-$(CONFIG_GPIO_PISOSR)	+= gpio-pisosr.o
 obj-$(CONFIG_GPIO_PL061)	+= gpio-pl061.o
-obj-$(CONFIG_GPIO_PMIC_EIC_SPRD)	+= gpio-pmic-eic-sprd.o
+obj-$(CONFIG_GPIO_PMIC_EIC_SPRD)+= gpio-pmic-eic-sprd.o
 obj-$(CONFIG_GPIO_PXA)		+= gpio-pxa.o
 obj-$(CONFIG_GPIO_RC5T583)	+= gpio-rc5t583.o
 obj-$(CONFIG_GPIO_RDC321X)	+= gpio-rdc321x.o
drivers/gpio/gpio-moxtet-sfp.c new file +156 -0

156 lines not on this page · raw message

Signature · 1 line
2.16.4
maintainer
#6 lore raw

PATCH 3/7 drivers: gpio: Add support for Turris Mox SFP GPIOs

Quoted 4 lines · Marek Beh?n

The SFP cage GPIOs on the SFP cage module of Turris Mox can be configured via the moxtet bus.

This driver supports those GPIOs so that they can be used by phylink.

I don't see anything in this driver which is specific to SFP. It is just a GPIO driver for 3 lines. So i would drop all references to SFP. You can then use it to blink LEDs on some other module, etc.

Andrew

#7 lore raw

PATCH 3/7 drivers: gpio: Add support for Turris Mox SFP GPIOs

Quoted 4 lines · On Wed, 8 Aug 2018 18:55:32 +0200 Andrew Lunn <[email protected]> wrote:
Quoted 3 lines · Marek Beh?n

The SFP cage GPIOs on the SFP cage module of Turris Mox can be configured via the moxtet bus.

Quoted 8 lines
Quoted 2 lines

This driver supports those GPIOs so that they can be used by phylink.

I don't see anything in this driver which is specific to SFP. It is just a GPIO driver for 3 lines. So i would drop all references to SFP. You can then use it to blink LEDs on some other module, etc.

Andrew

You are right, I shall rewrite the code to be more general. Marek

#8 lore raw

PATCH 4/7 watchdog: Add support for Armada 37xx CPU watchdog

In reply to Marek Behún

This adds support for the CPU watchdog found on Marvell Armada 37xx SoCs.

There are 4 counters which can be set as CPU watchdog counters. This driver uses the second counter (ID 1, counting from 0) (Marvell's Linux also uses second counter by default). In the future it could be adapted to use other counters, with definition in the device tree.

 create mode 100644 Documentation/devicetree/bindings/watchdog/armada-37xx-wdt.txt
 create mode 100644 drivers/watchdog/armada_37xx_wdt.c
Documentation/devicetree/bindings/watchdog/armada-37xx-wdt.txt new file +23 -0
new file mode 100644
index 000000000000..4ba9e40ad386
@@ -0,0 +1,23 @@
+* Armada 37xx CPU Watchdog Timer Controller
+
+Required properties:
+- compatible : must be "marvell,armada-3700-wdt"
+- reg : base physical address of the controller and length of memory mapped
+	region.
+- clocks : the clock feeding the watchdog timer. See clock-bindings.txt
+- marvell,system-controller : reference to syscon node for the CPU Miscellaneous
+	Registers
+
+Example:
+
+	cpu_misc: system-controller at d000 {
+		compatible = "syscon", "simple-mfd";
+		reg = <0xd000 0x1000>;
+	};
+
+	wdt: watchdog-timer at 8300 {
+		compatible = "marvell,armada-3700-wdt";
+		reg = <0x8300 0x40>;
+		marvell,system-controller = <&cpu_misc>;
+		clocks = <&xtalclk>;
+	};
MAINTAINERS +1 -0
index cfac1b21596c..e2d2955a8331 100644
@@ -1385,6 +1385,7 @@ S:	Maintained
 F:	include/mfd/moxtet.h
 F:	drivers/gpio/gpio-moxtet-sfp.c
 F:	drivers/mfd/moxtet.c
+F:	drivers/watchdog/armada_37xx_wdt.c
 
 ARM/EBSA110 MACHINE SUPPORT
 M:	Russell King <[email protected]>
drivers/watchdog/Kconfig +11 -0
index 9af07fd92763..c3d82e3fe1e1 100644
@@ -263,6 +263,17 @@ config ARM_SBSA_WATCHDOG
 	  To compile this driver as module, choose M here: The module
 	  will be called sbsa_gwdt.
 
+config ARMADA_37XX_WATCHDOG
+	tristate "Armada 37xx watchdog"
+	depends on ARCH_MVEBU || COMPILE_TEST
+	select MFD_SYSCON
+	select WATCHDOG_CORE
+	help
+	  Say Y here to include support for the watchdog timer found on
+	  Marvell Armada 37xx SoCs.
+	  To compile this driver as a module, choose M here: the
+	  module will be called armada_37xx_wdt.
+
 config ASM9260_WATCHDOG
 	tristate "Alphascale ASM9260 watchdog"
 	depends on MACH_ASM9260 || COMPILE_TEST
drivers/watchdog/Makefile +1 -0
index 1d3c6b094fe5..476d9069ed7c 100644
@@ -38,6 +38,7 @@ obj-$(CONFIG_USBPCWATCHDOG) += pcwd_usb.o
 # ARM Architecture
 obj-$(CONFIG_ARM_SP805_WATCHDOG) += sp805_wdt.o
 obj-$(CONFIG_ARM_SBSA_WATCHDOG) += sbsa_gwdt.o
+obj-$(CONFIG_ARMADA_37XX_WATCHDOG) += armada_37xx_wdt.o
 obj-$(CONFIG_ASM9260_WATCHDOG) += asm9260_wdt.o
 obj-$(CONFIG_AT91RM9200_WATCHDOG) += at91rm9200_wdt.o
 obj-$(CONFIG_AT91SAM9X_WATCHDOG) += at91sam9_wdt.o
drivers/watchdog/armada_37xx_wdt.c new file +353 -0

353 lines not on this page · raw message

Signature · 1 line
2.16.4
#9 lore raw

PATCH 5/7 arm64: dts: marvell: armada-37xx: add nodes to support watchdog

In reply to Marek Behún

This adds the system controller node for CPU Miscellaneous Registers (which is needed for the watchdog node) and the watchdog node.

arch/arm64/boot/dts/marvell/armada-37xx.dtsi +12 -0
index 3353252d78a0..b2f041a17d9d 100644
@@ -65,6 +65,18 @@
 			/* 32M internal register @ 0xd000_0000 */
 			ranges = <0x0 0x0 0xd0000000 0x2000000>;
 
+			wdt: watchdog-timer at 8300 {
+				compatible = "marvell,armada-3700-wdt";
+				reg = <0x8300 0x40>;
+				marvell,system-controller = <&cpu_misc>;
+				clocks = <&xtalclk>;
+			};
+
+			cpu_misc: system-controller at d000 {
+				compatible = "syscon", "simple-mfd";
+				reg = <0xd000 0x1000>;
+			};
+
 			spi0: spi at 10600 {
 				compatible = "marvell,armada-3700-spi";
 				#address-cells = <1>;
Signature · 1 line
2.16.4
#10 lore raw

PATCH 6/7 net: mvneta: Don't use GRO on Armada 3720

In reply to Marek Behún

For some reason on Armada 3720 boards (EspressoBin and Turris Mox) the networking driver behaves weirdly when using napi_gro_receive.

For example downloading a big file from a local network (low ping) is fast, but when downloading from a remote server (higher ping), the download speed is at first high but drops rapidly to almost nothing or absolutely nothing.

This is fixed when using netif_receive_skb instead of napi_gro_receive.

Cc Russell King - ARM Linux <[email protected]>
Cc netdev at vger.kernel.org
drivers/net/ethernet/marvell/mvneta.c +8 -2
index 0ad2f3f7da85..27f3017d94c5 100644
@@ -1959,7 +1959,10 @@ static int mvneta_rx_swbm(struct mvneta_port *pp, int rx_todo,
 
 			skb->protocol = eth_type_trans(skb, dev);
 			mvneta_rx_csum(pp, rx_status, skb);
-			napi_gro_receive(&port->napi, skb);
+			if (pp->neta_armada3700)
+				netif_receive_skb(skb);
+			else
+				napi_gro_receive(&port->napi, skb);
 
 			rcvd_pkts++;
 			rcvd_bytes += rx_bytes;
@@ -2001,7 +2004,10 @@ static int mvneta_rx_swbm(struct mvneta_port *pp, int rx_todo,
 
 		mvneta_rx_csum(pp, rx_status, skb);
 
-		napi_gro_receive(&port->napi, skb);
+		if (pp->neta_armada3700)
+			netif_receive_skb(skb);
+		else
+			napi_gro_receive(&port->napi, skb);
 	}
 
 	if (rcvd_pkts) {
Signature · 1 line
2.16.4
maintainer
#11 lore raw

Re: [PATCH v1 6/7] net: mvneta: Don't use GRO on Armada 3720

Quoted 9 lines · Marek Beh?n

For some reason on Armada 3720 boards (EspressoBin and Turris Mox) the networking driver behaves weirdly when using napi_gro_receive.

For example downloading a big file from a local network (low ping) is fast, but when downloading from a remote server (higher ping), the download speed is at first high but drops rapidly to almost nothing or absolutely nothing.

This is fixed when using netif_receive_skb instead of napi_gro_receive.

Before doing this, we should really understand what is going on. It is probably just a driver bug which needs fixing. And GRO should be good for performance, so we do want to use it, if possible.

Andrew

#12 lore raw

Re: [PATCH v1 6/7] net: mvneta: Don't use GRO on Armada 3720

Quoted 10 lines · Andrew Lunn
Quoted 8 lines · Marek Beh?n

For some reason on Armada 3720 boards (EspressoBin and Turris Mox) the networking driver behaves weirdly when using napi_gro_receive.

For example downloading a big file from a local network (low ping) is fast, but when downloading from a remote server (higher ping), the download speed is at first high but drops rapidly to almost nothing or absolutely nothing.

Quoted 5 lines
Quoted 1 line

This is fixed when using netif_receive_skb instead of napi_gro_receive.

Before doing this, we should really understand what is going on. It is probably just a driver bug which needs fixing. And GRO should be good for performance, so we do want to use it, if possible.

I'd just disable it and worry about it later. The software gro in the mvneta would batch up 64k and is one of the reasons why sch_cake does gso splitting by default. (64k unsplit, downshifted to 1mbit = ~540ms of latency). If this mvneta facility is in addition buggy, that explains some puzzling things I've seen in various benchmarks. thx for the steer as to what to look for!

IMHO: in general gro looks good on dumb single stream benchmarks, not as useful on mixed routed traffic with more entropy, and batching clutters up the mvneta receive path that otherwise could be draining the rx ring and spitting packets into the rest of the system faster. The mvneta is mostly (?) used on routing devices.

Quoted 2 lines

Andrew

Signature · 4 lines
Dave T?ht
CEO, TekLibre, LLC
http://www.teklibre.com
Tel: 1-669-226-2619
#13 lore raw

Re: [PATCH v1 6/7] net: mvneta: Don't use GRO on Armada 3720

In reply to Marek Behún

+ more people

Quoted 7 lines · On Wed, 8 Aug 2018 17:27:05 +0200 Marek Beh?n wrote:

For some reason on Armada 3720 boards (EspressoBin and Turris Mox) the networking driver behaves weirdly when using napi_gro_receive.

For example downloading a big file from a local network (low ping) is fast, but when downloading from a remote server (higher ping), the download speed is at first high but drops rapidly to almost nothing or absolutely nothing.

We also met this issue on some berlin platforms. I tried to fix the bug, but no clue so far.

Quoted 2 lines

This is fixed when using netif_receive_skb instead of napi_gro_receive.

This is a workaround. The good news is this workaround also fixes the issue we saw on berlin.

Quoted 33 lines
Cc Russell King - ARM Linux <[email protected]>
Cc netdev at vger.kernel.org
drivers/net/ethernet/marvell/mvneta.c +8 -2
index 0ad2f3f7da85..27f3017d94c5 100644
@@ -1959,7 +1959,10 @@ static int mvneta_rx_swbm(struct mvneta_port *pp, int rx_todo,
 
 			skb->protocol = eth_type_trans(skb, dev);
 			mvneta_rx_csum(pp, rx_status, skb);
-			napi_gro_receive(&port->napi, skb);
+			if (pp->neta_armada3700)
+				netif_receive_skb(skb);
+			else
+				napi_gro_receive(&port->napi, skb);
 
 			rcvd_pkts++;
 			rcvd_bytes += rx_bytes;
@@ -2001,7 +2004,10 @@ static int mvneta_rx_swbm(struct mvneta_port *pp, int rx_todo,
 
 		mvneta_rx_csum(pp, rx_status, skb);
 
-		napi_gro_receive(&port->napi, skb);
+		if (pp->neta_armada3700)
+			netif_receive_skb(skb);
+		else
+			napi_gro_receive(&port->napi, skb);
 	}
 
 	if (rcvd_pkts) {
#14 lore raw

Re: [PATCH v1 6/7] net: mvneta: Don't use GRO on Armada 3720

Hi,

Quoted 32 lines · On Thu, 9 Aug 2018 12:40:41 +0800 Jisheng Zhang wrote:

+ more people

Quoted 7 lines · On Wed, 8 Aug 2018 17:27:05 +0200 Marek Beh?n wrote:

For some reason on Armada 3720 boards (EspressoBin and Turris Mox) the networking driver behaves weirdly when using napi_gro_receive.

For example downloading a big file from a local network (low ping) is fast, but when downloading from a remote server (higher ping), the download speed is at first high but drops rapidly to almost nothing or absolutely nothing.

We also met this issue on some berlin platforms. I tried to fix the bug, but no clue so far.

Quoted 2 lines

This is fixed when using netif_receive_skb instead of napi_gro_receive.

This is a workaround. The good news is this workaround also fixes the issue we saw on berlin.

Quoted 11 lines
Cc Russell King - ARM Linux <[email protected]>
Cc netdev at vger.kernel.org
drivers/net/ethernet/marvell/mvneta.c +0 -0
index 0ad2f3f7da85..27f3017d94c5 100644
@@ -1959,7 +1959,10 @@ static int mvneta_rx_swbm(struct mvneta_port *pp, int rx_todo,
 
Quoted 7 lines

drivers/net/ethernet/marvell/mvneta.c static int mvneta_rx_swbm(struct mvneta_port *pp, int rx_todo, · line 1965

Quoted 7 lines
drivers/net/ethernet/marvell/mvneta.c +4 -1
 			skb->protocol = eth_type_trans(skb, dev);
 			mvneta_rx_csum(pp, rx_status, skb);
-			napi_gro_receive(&port->napi, skb);
+			if (pp->neta_armada3700)
+				netif_receive_skb(skb);
+			else
+				napi_gro_receive(&port->napi, skb);

I think I found the root cause, if neta_armada3700 is true, the port got from this_cpu_ptr(pp->ports) is invalid, this is bug... I'll cook a patch for this

Thanks

Quoted 16 lines
Quoted 15 lines
drivers/net/ethernet/marvell/mvneta.c +4 -1
 			rcvd_pkts++;
 			rcvd_bytes += rx_bytes;
@@ -2001,7 +2004,10 @@ static int mvneta_rx_swbm(struct mvneta_port *pp, int rx_todo,
 
 		mvneta_rx_csum(pp, rx_status, skb);
 
-		napi_gro_receive(&port->napi, skb);
+		if (pp->neta_armada3700)
+			netif_receive_skb(skb);
+		else
+			napi_gro_receive(&port->napi, skb);
 	}
 
 	if (rcvd_pkts) {  
#15 lore raw

Re: [PATCH v1 6/7] net: mvneta: Don't use GRO on Armada 3720

Quoted 36 lines · On Thu, 9 Aug 2018 19:27:55 +0800 Jisheng Zhang wrote:

Hi,

Quoted 32 lines · On Thu, 9 Aug 2018 12:40:41 +0800 Jisheng Zhang wrote:

+ more people

Quoted 7 lines · On Wed, 8 Aug 2018 17:27:05 +0200 Marek Beh?n wrote:

For some reason on Armada 3720 boards (EspressoBin and Turris Mox) the networking driver behaves weirdly when using napi_gro_receive.

For example downloading a big file from a local network (low ping) is fast, but when downloading from a remote server (higher ping), the download speed is at first high but drops rapidly to almost nothing or absolutely nothing.

We also met this issue on some berlin platforms. I tried to fix the bug, but no clue so far.

Quoted 2 lines

This is fixed when using netif_receive_skb instead of napi_gro_receive.

This is a workaround. The good news is this workaround also fixes the issue we saw on berlin.

Quoted 11 lines
Cc Russell King - ARM Linux <[email protected]>
Cc netdev at vger.kernel.org
drivers/net/ethernet/marvell/mvneta.c +0 -0
index 0ad2f3f7da85..27f3017d94c5 100644
@@ -1959,7 +1959,10 @@ static int mvneta_rx_swbm(struct mvneta_port *pp, int rx_todo,
 
Quoted 10 lines
Quoted 7 lines

drivers/net/ethernet/marvell/mvneta.c static int mvneta_rx_swbm(struct mvneta_port *pp, int rx_todo, · line 1965

Quoted 7 lines
drivers/net/ethernet/marvell/mvneta.c +4 -1
 			skb->protocol = eth_type_trans(skb, dev);
 			mvneta_rx_csum(pp, rx_status, skb);
-			napi_gro_receive(&port->napi, skb);
+			if (pp->neta_armada3700)
+				netif_receive_skb(skb);
+			else
+				napi_gro_receive(&port->napi, skb);  

I think I found the root cause, if neta_armada3700 is true, the port got from this_cpu_ptr(pp->ports) is invalid, this is bug... I'll cook a patch for this

correct it as:

the port's(port is got from this_cpu_ptr(pp->ports) napi is invalid.

Patch is sent out. Could you please try?

Per my test, it solves the issue we saw on berlin.

Quoted 20 lines

Thanks

Quoted 16 lines
Quoted 15 lines
drivers/net/ethernet/marvell/mvneta.c +4 -1
 			rcvd_pkts++;
 			rcvd_bytes += rx_bytes;
@@ -2001,7 +2004,10 @@ static int mvneta_rx_swbm(struct mvneta_port *pp, int rx_todo,
 
 		mvneta_rx_csum(pp, rx_status, skb);
 
-		napi_gro_receive(&port->napi, skb);
+		if (pp->neta_armada3700)
+			netif_receive_skb(skb);
+		else
+			napi_gro_receive(&port->napi, skb);
 	}
 
 	if (rcvd_pkts) {    
#16 lore raw

Re: [PATCH v1 6/7] net: mvneta: Don't use GRO on Armada 3720

Quoted 53 lines · On Thu, 9 Aug 2018 20:08:32 +0800 Jisheng Zhang <[email protected]>…
Quoted 42 lines · On Thu, 9 Aug 2018 19:27:55 +0800 Jisheng Zhang wrote:

Hi,

Quoted 38 lines · On Thu, 9 Aug 2018 12:40:41 +0800 Jisheng Zhang wrote:

+ more people

Quoted 8 lines · On Wed, 8 Aug 2018 17:27:05 +0200 Marek Beh?n wrote:

For some reason on Armada 3720 boards (EspressoBin and Turris Mox) the networking driver behaves weirdly when using napi_gro_receive.

For example downloading a big file from a local network (low ping) is fast, but when downloading from a remote server (higher ping), the download speed is at first high but drops rapidly to almost nothing or absolutely nothing.

We also met this issue on some berlin platforms. I tried to fix the bug, but no clue so far.

Quoted 3 lines

This is fixed when using netif_receive_skb instead of napi_gro_receive.

This is a workaround. The good news is this workaround also fixes the issue we saw on berlin.

Quoted 15 lines
Cc Russell King - ARM Linux <[email protected]>
Cc netdev at vger.kernel.org
(quoted diff) +0 -0

b/drivers/net/ethernet/marvell/mvneta.c index 0ad2f3f7da85..27f3017d94c5 100644 --- a/drivers/net/ethernet/marvell/mvneta.c +++ b/drivers/net/ethernet/marvell/mvneta.c @@ -1959,7 +1959,10 @@ static int mvneta_rx_swbm(struct mvneta_port *pp, int rx_todo,
skb->protocol = eth_type_trans(skb,
dev); mvneta_rx_csum(pp, rx_status, skb);

(quoted diff) +1 -1
-			napi_gro_receive(&port->napi, skb);
+			if (pp->neta_armada3700)
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
drivers/net/ethernet/marvell/mvneta.c +3 -0
+				netif_receive_skb(skb);
+			else
+				napi_gro_receive(&port->napi,

skb);

I think I found the root cause, if neta_armada3700 is true, the port got from this_cpu_ptr(pp->ports) is invalid, this is bug... I'll cook a patch for this

Quoted 29 lines

correct it as:

the port's(port is got from this_cpu_ptr(pp->ports) napi is invalid.

Patch is sent out. Could you please try?

Per my test, it solves the issue we saw on berlin.

Quoted 20 lines

Thanks

Quoted 16 lines
Quoted 15 lines
drivers/net/ethernet/marvell/mvneta.c +4 -1
 			rcvd_pkts++;
 			rcvd_bytes += rx_bytes;
@@ -2001,7 +2004,10 @@ static int mvneta_rx_swbm(struct
 mvneta_port *pp, int rx_todo, 
 		mvneta_rx_csum(pp, rx_status, skb);
 
-		napi_gro_receive(&port->napi, skb);
+		if (pp->neta_armada3700)
+			netif_receive_skb(skb);
+		else
+			napi_gro_receive(&port->napi, skb);
 	}
 
 	if (rcvd_pkts) {      

Jisheng, the issue is solved with your patch. Thanks :)

#17 lore raw

PATCH 7/7 ARM64: dts: marvell: Add DTS files for Turris Mox

In reply to Marek Behún

This adds basic support for the Turris Mox board from CZ.NIC.

Turris Mox is as modular router based on the Armada 3720 SOC (same as EspressoBin).

The basic module can be extended by different modules. When those modules are connected, U-Boot shall patch this basic device-tree with nodes corresponding to those modules.

 create mode 100644 Documentation/devicetree/bindings/arm/marvell/armada-3720-turris-mox.txt
 create mode 100644 arch/arm64/boot/dts/marvell/armada-3720-turris-mox.dts
Documentation/devicetree/bindings/arm/marvell/armada-3720-turris-mox.txt new file +6 -0
new file mode 100644
index 000000000000..408fc07a9bbf
@@ -0,0 +1,6 @@
+CZ.NIC's Turris Mox SOHO router Device Tree Bindings
+----------------------------------------------------
+
+Required root node property:
+
+compatible: must contain "cznic,turris-mox"
MAINTAINERS +1 -0
index e2d2955a8331..40af94bf0b0a 100644
@@ -1382,6 +1382,7 @@ ARM/CZ.NIC TURRIS MOX SUPPORT
 M:	Marek Behun <[email protected]>
 W:	http://mox.turris.cz
 S:	Maintained
+F:	arch/arm64/boot/dts/marvell/armada-3720-turris-mox.dts
 F:	include/mfd/moxtet.h
 F:	drivers/gpio/gpio-moxtet-sfp.c
 F:	drivers/mfd/moxtet.c
arch/arm64/boot/dts/marvell/Makefile +1 -0
index ea9d49f2a911..1dd89d1bf8ad 100644
@@ -2,6 +2,7 @@
 # Mvebu SoC Family
 dtb-$(CONFIG_ARCH_MVEBU) += armada-3720-db.dtb
 dtb-$(CONFIG_ARCH_MVEBU) += armada-3720-espressobin.dtb
+dtb-$(CONFIG_ARCH_MVEBU) += armada-3720-turris-mox.dtb
 dtb-$(CONFIG_ARCH_MVEBU) += armada-7040-db.dtb
 dtb-$(CONFIG_ARCH_MVEBU) += armada-8040-db.dtb
 dtb-$(CONFIG_ARCH_MVEBU) += armada-8040-mcbin.dtb
arch/arm64/boot/dts/marvell/armada-3720-turris-mox.dts new file +229 -0

229 lines not on this page · raw message

Signature · 1 line
2.16.4