MediaTek SoC flow offload improvements + wireless support
Subsystems (5)
Felix Fietkau
maintainerv2 cover MediaTek SoC flow offload improvements + wireless support
This series contains the following improvements to mediatek ethernet flow offload support:
- support dma-coherent on ethernet to improve performance
- add ipv6 offload support
- rework hardware flow table entry handling to improve dealing with hash collisions and competing flows
- support creating offload entries from user space
- support creating offload entries with just source/destination mac address, vlan and output device information
- add driver changes for supporting the Wireless Ethernet Dispatch core, which can be used to offload flows from ethernet to MT7915 PCIe WLAN devices
Changes in v2:
- add missing dt-bindings patches
David Bentham (1):
net: ethernet: mtk_eth_soc: add ipv6 flow offload support
Felix Fietkau (10):
net: ethernet: mtk_eth_soc: add support for coherent DMA
arm64: dts: mediatek: mt7622: add support for coherent DMA
net: ethernet: mtk_eth_soc: add support for Wireless Ethernet Dispatch
(WED)
net: ethernet: mtk_eth_soc: implement flow offloading to WED devices
arm64: dts: mediatek: mt7622: introduce nodes for Wireless Ethernet
Dispatch
net: ethernet: mtk_eth_soc: support TC_SETUP_BLOCK for PPE offload
net: ethernet: mtk_eth_soc: allocate struct mtk_ppe separately
net: ethernet: mtk_eth_soc: rework hardware flow table management
net: ethernet: mtk_eth_soc: remove bridge flow offload type entry
support
net: ethernet: mtk_eth_soc: support creating mac address based offload
entries
Lorenzo Bianconi (3):
dt-bindings: net: mediatek: add optional properties for the SoC
ethernet core
dt-bindings: arm: mediatek: document WED binding for MT7622
dt-bindings: arm: mediatek: document the pcie mirror node on MT7622
.../mediatek/mediatek,mt7622-pcie-mirror.yaml | 42 + .../arm/mediatek/mediatek,mt7622-wed.yaml | 50 + .../devicetree/bindings/net/mediatek-net.txt | 10 + arch/arm64/boot/dts/mediatek/mt7622.dtsi | 32 +- drivers/net/ethernet/mediatek/Kconfig | 4 + drivers/net/ethernet/mediatek/Makefile | 5 + drivers/net/ethernet/mediatek/mtk_eth_soc.c | 131 ++- drivers/net/ethernet/mediatek/mtk_eth_soc.h | 14 +- drivers/net/ethernet/mediatek/mtk_ppe.c | 368 +++++++- drivers/net/ethernet/mediatek/mtk_ppe.h | 89 +- .../net/ethernet/mediatek/mtk_ppe_debugfs.c | 1 - .../net/ethernet/mediatek/mtk_ppe_offload.c | 189 +++- drivers/net/ethernet/mediatek/mtk_wed.c | 875 ++++++++++++++++++ drivers/net/ethernet/mediatek/mtk_wed.h | 135 +++ .../net/ethernet/mediatek/mtk_wed_debugfs.c | 175 ++++ drivers/net/ethernet/mediatek/mtk_wed_ops.c | 8 + drivers/net/ethernet/mediatek/mtk_wed_regs.h | 251 +++++ include/linux/netdevice.h | 7 + include/linux/soc/mediatek/mtk_wed.h | 131 +++ net/core/dev.c | 4 + 20 files changed, 2396 insertions(+), 125 deletions(-) create mode 100644 Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-pcie-mirror.yaml create mode 100644 Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-wed.yaml create mode 100644 drivers/net/ethernet/mediatek/mtk_wed.c create mode 100644 drivers/net/ethernet/mediatek/mtk_wed.h create mode 100644 drivers/net/ethernet/mediatek/mtk_wed_debugfs.c create mode 100644 drivers/net/ethernet/mediatek/mtk_wed_ops.c create mode 100644 drivers/net/ethernet/mediatek/mtk_wed_regs.h create mode 100644 include/linux/soc/mediatek/mtk_wed.h
Signature · 7 lines
2.35.1 _______________________________________________ Linux-mediatek mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-mediatek
Felix Fietkau
maintainerPATCH v2 1/14 dt-bindings: net: mediatek: add optional properties for the SoC ethernet core
From: Lorenzo Bianconi <[email protected]>
Introduce dma-coherent, cci-control and hifsys optional properties to the mediatek ethernet controller bindings
--- Documentation/devicetree/bindings/net/mediatek-net.txt | 6 ++++++ 1 file changed, 6 insertions(+)
Documentation/devicetree/bindings/net/mediatek-net.txt +6 -0
@@ -41,6 +41,12 @@ Required properties: - mediatek,pctl: phandle to the syscon node that handles the ports slew rate and driver current: only for MT2701 and MT7623 SoC +Optional properties: +- dma-coherent: present if dma operations are coherent +- mediatek,cci-control: phandle to the cache coherent interconnect node +- mediatek,hifsys: phandle to the mediatek hifsys controller used to provide + various clocks and reset to the system. + * Ethernet MAC node Required properties:
Signature · 7 lines
2.35.1 _______________________________________________ Linux-mediatek mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-mediatek
Rob Herring
maintainerRe: [PATCH v2 01/14] dt-bindings: net: mediatek: add optional properties for the SoC ethernet core
Quoted 16 lines · Felix Fietkau
From: Lorenzo Bianconi <[email protected]>
Introduce dma-coherent, cci-control and hifsys optional properties to the mediatek ethernet controller bindings
Signed-off-by Lorenzo Bianconi <[email protected]>Signed-off-by Felix Fietkau <[email protected]>--- Documentation/devicetree/bindings/net/mediatek-net.txt | 6 ++++++ 1 file changed, 6 insertions(+)Documentation/devicetree/bindings/net/mediatek-net.txt +0
-0
Documentation/devicetree/bindings/net/mediatek-net.txt Required properties: · line 46
Quoted 6 lines
Documentation/devicetree/bindings/net/mediatek-net.txt +3
-0@@ -41,6 +41,12 @@ Required properties: - mediatek,pctl: phandle to the syscon node that handles the ports slew rate and driver current: only for MT2701 and MT7623 SoC +Optional properties: +- dma-coherent: present if dma operations are coherent +- mediatek,cci-control: phandle to the cache coherent interconnect node
There's a common property for this already. See CCI-400 binding.
Documentation/devicetree/bindings/net/mediatek-net.txt Required properties: · line 49
Quoted 3 lines
Documentation/devicetree/bindings/net/mediatek-net.txt +3
-0+- mediatek,hifsys: phandle to the mediatek hifsys controller used to provide + various clocks and reset to the system. +
This series is adding a handful of new properties. Please convert the binding to DT schema first.
Rob
Lorenzo Bianconi
maintainerRe: [PATCH v2 01/14] dt-bindings: net: mediatek: add optional properties for the SoC ethernet core
Quoted 26 lines · Felix Fietkau
Quoted 16 lines
From: Lorenzo Bianconi <[email protected]>
Introduce dma-coherent, cci-control and hifsys optional properties to the mediatek ethernet controller bindings
Signed-off-by Lorenzo Bianconi <[email protected]>Signed-off-by Felix Fietkau <[email protected]>--- Documentation/devicetree/bindings/net/mediatek-net.txt | 6 ++++++ 1 file changed, 6 insertions(+)Documentation/devicetree/bindings/net/mediatek-net.txt +0
-0Documentation/devicetree/bindings/net/mediatek-net.txt Required properties: · line 46
Quoted 6 lines
Documentation/devicetree/bindings/net/mediatek-net.txt +3
-0@@ -41,6 +41,12 @@ Required properties: - mediatek,pctl: phandle to the syscon node that handles the ports slew rate and driver current: only for MT2701 and MT7623 SoC +Optional properties: +- dma-coherent: present if dma operations are coherent +- mediatek,cci-control: phandle to the cache coherent interconnect nodeThere's a common property for this already. See CCI-400 binding.
Quoted 6 lines
Documentation/devicetree/bindings/net/mediatek-net.txt Required properties: · line 49
Quoted 3 lines
Documentation/devicetree/bindings/net/mediatek-net.txt +3
-0+- mediatek,hifsys: phandle to the mediatek hifsys controller used to provide + various clocks and reset to the system. +This series is adding a handful of new properties. Please convert the binding to DT schema first.
ack, I will converti this file to yaml format.
Regards,
Lorenzo
Quoted 2 lines
Rob
Felix Fietkau
maintainerPATCH v2 2/14 net: ethernet: mtk_eth_soc: add support for coherent DMA
In reply to Felix FietkauIt improves performance by eliminating the need for a cache flush on rx and tx In preparation for supporting WED (Wireless Ethernet Dispatch), also add a function for disabling coherent DMA at runtime.
--- drivers/net/ethernet/mediatek/mtk_eth_soc.c | 95 +++++++++++++++------ drivers/net/ethernet/mediatek/mtk_eth_soc.h | 9 ++ 2 files changed, 80 insertions(+), 24 deletions(-)
drivers/net/ethernet/mediatek/mtk_eth_soc.c +71 -24
@@ -9,6 +9,7 @@#include <linux/of_device.h> #include <linux/of_mdio.h> #include <linux/of_net.h> +#include <linux/of_address.h> #include <linux/mfd/syscon.h> #include <linux/regmap.h> #include <linux/clk.h>
@@ -786,7 +787,7 @@ static int mtk_init_fq_dma(struct mtk_eth *eth)dma_addr_t dma_addr;int i; - eth->scratch_ring =dma_alloc_coherent (eth->dev, + eth->scratch_ring =dma_alloc_coherent (eth->dma_dev, cnt *sizeof (struct mtk_tx_dma ), ð->phy_scratch_ring,GFP_ATOMIC );
@@ -798,10 +799,10 @@ static int mtk_init_fq_dma(struct mtk_eth *eth)if (unlikely (!eth->scratch_head))return -ENOMEM ; - dma_addr =dma_map_single (eth->dev, + dma_addr =dma_map_single (eth->dma_dev, eth->scratch_head, cnt *MTK_QDMA_PAGE_SIZE ,DMA_FROM_DEVICE ); -if (unlikely (dma_mapping_error (eth->dev, dma_addr))) +if (unlikely (dma_mapping_error (eth->dma_dev, dma_addr)))return -ENOMEM ; phy_ring_tail = eth->phy_scratch_ring +
@@ -855,26 +856,26 @@ static void mtk_tx_unmap(struct mtk_eth *eth, struct mtk_tx_buf *tx_buf, {if (MTK_HAS_CAPS (eth->soc->caps,MTK_QDMA )) {if (tx_buf->flags &MTK_TX_FLAGS_SINGLE0 ) { -dma_unmap_single (eth->dev, +dma_unmap_single (eth->dma_dev,dma_unmap_addr (tx_buf, dma_addr0),dma_unmap_len (tx_buf, dma_len0),DMA_TO_DEVICE ); }else if (tx_buf->flags &MTK_TX_FLAGS_PAGE0 ) { -dma_unmap_page (eth->dev, +dma_unmap_page (eth->dma_dev,dma_unmap_addr (tx_buf, dma_addr0),dma_unmap_len (tx_buf, dma_len0),DMA_TO_DEVICE ); } }else {if (dma_unmap_len (tx_buf, dma_len0)) { -dma_unmap_page (eth->dev, +dma_unmap_page (eth->dma_dev,dma_unmap_addr (tx_buf, dma_addr0),dma_unmap_len (tx_buf, dma_len0),DMA_TO_DEVICE ); }if (dma_unmap_len (tx_buf, dma_len1)) { -dma_unmap_page (eth->dev, +dma_unmap_page (eth->dma_dev,dma_unmap_addr (tx_buf, dma_addr1),dma_unmap_len (tx_buf, dma_len1),DMA_TO_DEVICE );
@@ -952,9 +953,9 @@ static int mtk_tx_map(struct sk_buff *skb, struct net_device *dev,if (skb_vlan_tag_present (skb)) txd4 |=TX_DMA_INS_VLAN |skb_vlan_tag_get (skb); - mapped_addr =dma_map_single (eth->dev, skb->data, + mapped_addr =dma_map_single (eth->dma_dev, skb->data,skb_headlen (skb),DMA_TO_DEVICE ); -if (unlikely (dma_mapping_error (eth->dev, mapped_addr))) +if (unlikely (dma_mapping_error (eth->dma_dev, mapped_addr)))return -ENOMEM ;WRITE_ONCE (itxd->txd1, mapped_addr);
@@ -993,10 +994,10 @@ static int mtk_tx_map(struct sk_buff *skb, struct net_device *dev, frag_map_size =min (frag_size,MTK_TX_DMA_BUF_LEN ); - mapped_addr =skb_frag_dma_map (eth->dev, frag, offset, + mapped_addr =skb_frag_dma_map (eth->dma_dev, frag, offset, frag_map_size,DMA_TO_DEVICE ); -if (unlikely (dma_mapping_error (eth->dev, mapped_addr))) +if (unlikely (dma_mapping_error (eth->dma_dev, mapped_addr))) goto err_dma; if (i == nr_frags - 1 &&
@@ -1274,18 +1275,18 @@ static int mtk_poll_rx(struct napi_struct *napi, int budget, netdev->stats.rx_dropped++; goto release_desc; } - dma_addr =dma_map_single (eth->dev, + dma_addr =dma_map_single (eth->dma_dev, new_data +NET_SKB_PAD + eth->ip_align, ring->buf_size,DMA_FROM_DEVICE ); -if (unlikely (dma_mapping_error (eth->dev, dma_addr))) { +if (unlikely (dma_mapping_error (eth->dma_dev, dma_addr))) {skb_free_frag (new_data); netdev->stats.rx_dropped++; goto release_desc; } -dma_unmap_single (eth->dev, trxd.rxd1, +dma_unmap_single (eth->dma_dev, trxd.rxd1, ring->buf_size,DMA_FROM_DEVICE );/* receive data */
@@ -1558,7 +1559,7 @@ static int mtk_tx_alloc(struct mtk_eth *eth)if (!ring->buf) goto no_tx_mem; - ring->dma =dma_alloc_coherent (eth->dev,MTK_DMA_SIZE * sz, + ring->dma =dma_alloc_coherent (eth->dma_dev,MTK_DMA_SIZE * sz, &ring->phys,GFP_ATOMIC );if (!ring->dma) goto no_tx_mem;
@@ -1576,7 +1577,7 @@ static int mtk_tx_alloc(struct mtk_eth *eth)* descriptors in ring->dma_pdma. */ if (!MTK_HAS_CAPS (eth->soc->caps,MTK_QDMA )) { - ring->dma_pdma =dma_alloc_coherent (eth->dev,MTK_DMA_SIZE * sz, + ring->dma_pdma =dma_alloc_coherent (eth->dma_dev,MTK_DMA_SIZE * sz, &ring->phys_pdma,GFP_ATOMIC );if (!ring->dma_pdma)
@@ -1635,7 +1636,7 @@ static void mtk_tx_clean(struct mtk_eth *eth) }if (ring->dma) { -dma_free_coherent (eth->dev, +dma_free_coherent (eth->dma_dev,MTK_DMA_SIZE *sizeof (*ring->dma), ring->dma, ring->phys);
@@ -1643,7 +1644,7 @@ static void mtk_tx_clean(struct mtk_eth *eth) }if (ring->dma_pdma) { -dma_free_coherent (eth->dev, +dma_free_coherent (eth->dma_dev,MTK_DMA_SIZE *sizeof (*ring->dma_pdma), ring->dma_pdma, ring->phys_pdma);
@@ -1688,18 +1689,18 @@ static int mtk_rx_alloc(struct mtk_eth *eth, int ring_no, int rx_flag)return -ENOMEM ; } - ring->dma =dma_alloc_coherent (eth->dev, + ring->dma =dma_alloc_coherent (eth->dma_dev, rx_dma_size *sizeof (*ring->dma), &ring->phys,GFP_ATOMIC );if (!ring->dma)return -ENOMEM ;for (i =0 ; i < rx_dma_size; i++) { -dma_addr_t dma_addr =dma_map_single (eth->dev, +dma_addr_t dma_addr =dma_map_single (eth->dma_dev, ring->data[i] +NET_SKB_PAD + eth->ip_align, ring->buf_size,DMA_FROM_DEVICE ); -if (unlikely (dma_mapping_error (eth->dev, dma_addr))) +if (unlikely (dma_mapping_error (eth->dma_dev, dma_addr)))return -ENOMEM ; ring->dma[i].rxd1 = (unsigned int )dma_addr;
@@ -1735,7 +1736,7 @@ static void mtk_rx_clean(struct mtk_eth *eth, struct mtk_rx_ring *ring)continue ;if (!ring->dma[i].rxd1)continue ; -dma_unmap_single (eth->dev, +dma_unmap_single (eth->dma_dev, ring->dma[i].rxd1, ring->buf_size,DMA_FROM_DEVICE );
@@ -1746,7 +1747,7 @@ static void mtk_rx_clean(struct mtk_eth *eth, struct mtk_rx_ring *ring) }if (ring->dma) { -dma_free_coherent (eth->dev, +dma_free_coherent (eth->dma_dev, ring->dma_size *sizeof (*ring->dma), ring->dma, ring->phys);
@@ -2099,7 +2100,7 @@ static void mtk_dma_free(struct mtk_eth *eth)if (eth->netdev[i])netdev_reset_queue (eth->netdev[i]);if (eth->scratch_ring) { -dma_free_coherent (eth->dev, +dma_free_coherent (eth->dma_dev,MTK_DMA_SIZE *sizeof (struct mtk_tx_dma ), eth->scratch_ring, eth->phy_scratch_ring);
@@ -2448,6 +2449,8 @@ static void mtk_dim_tx(struct work_struct *work)static int mtk_hw_init (struct mtk_eth *eth) { +u32 dma_mask =ETHSYS_DMA_AG_MAP_PDMA |ETHSYS_DMA_AG_MAP_QDMA | +ETHSYS_DMA_AG_MAP_PPE ;int i, val, ret;if (test_and_set_bit (MTK_HW_INIT , ð->state))
@@ -2460,6 +2463,10 @@ static int mtk_hw_init(struct mtk_eth *eth)if (ret) goto err_disable_pm; +if (eth->ethsys) +regmap_update_bits (eth->ethsys,ETHSYS_DMA_AG_MAP , dma_mask, +of_dma_is_coherent (eth->dma_dev->of_node) * dma_mask); +if (MTK_HAS_CAPS (eth->soc->caps,MTK_SOC_MT7628 )) { ret =device_reset (eth->dev);if (ret) {
@@ -3040,6 +3047,35 @@ static int mtk_add_mac(struct mtk_eth *eth, struct device_node *np)return err; } +void mtk_eth_set_dma_device (struct mtk_eth *eth,struct device *dma_dev) +{ +struct net_device *dev, *tmp; +LIST_HEAD (dev_list); +int i; + +rtnl_lock (); + +for (i =0 ; i <MTK_MAC_COUNT ; i++) { + dev = eth->netdev[i]; + +if (!dev || !(dev->flags &IFF_UP )) +continue ; + +list_add_tail (&dev->close_list, &dev_list); + } + +dev_close_many (&dev_list, false); + + eth->dma_dev = dma_dev; + +list_for_each_entry_safe (dev, tmp, &dev_list, close_list) { +list_del_init (&dev->close_list); +dev_open (dev,NULL ); + } + +rtnl_unlock (); +} +static int mtk_probe (struct platform_device *pdev) {struct device_node *mac_np;
@@ -3053,6 +3089,7 @@ static int mtk_probe(struct platform_device *pdev) eth->soc =of_device_get_match_data (&pdev->dev); eth->dev = &pdev->dev; + eth->dma_dev = &pdev->dev; eth->base =devm_platform_ioremap_resource (pdev,0 );if (IS_ERR (eth->base))return PTR_ERR (eth->base);
@@ -3101,6 +3138,16 @@ static int mtk_probe(struct platform_device *pdev) } } +if (of_dma_is_coherent (pdev->dev.of_node)) { +struct regmap *cci; + + cci =syscon_regmap_lookup_by_phandle (pdev->dev.of_node, +"mediatek,cci-control" ); +/* enable CPU/bus coherency */ +if (!IS_ERR (cci)) +regmap_write (cci,0 ,3 ); + } +if (MTK_HAS_CAPS (eth->soc->caps,MTK_SGMII )) { eth->sgmii =devm_kzalloc (eth->dev,sizeof (*eth->sgmii),GFP_KERNEL );
drivers/net/ethernet/mediatek/mtk_eth_soc.h +9 -0
@@ -465,6 +465,12 @@#define RSTCTRL_FE BIT(6)#define RSTCTRL_PPE BIT(31) +/* ethernet dma channel agent map */ +#define ETHSYS_DMA_AG_MAP 0x408 +#define ETHSYS_DMA_AG_MAP_PDMA BIT(0) +#define ETHSYS_DMA_AG_MAP_QDMA BIT(1) +#define ETHSYS_DMA_AG_MAP_PPE BIT(2) +/* SGMII subsystem config registers */ /* Register to auto-negotiation restart */ #define SGMSYS_PCS_CONTROL_1 0x0
@@ -882,6 +888,7 @@ struct mtk_sgmii {/* struct mtk_eth - This is the main datasructure for holding the state * of the driver * @dev: The device pointer +* @dev: The device pointer used for dma mapping/alloc * @base: The mapped register i/o base * @page_lock: Make sure that register operations are atomic * @tx_irq__lock: Make sure that IRQ register operations are atomic
@@ -925,6 +932,7 @@ struct mtk_sgmii {struct mtk_eth {struct device *dev; +struct device *dma_dev;void __iomem *base;spinlock_t page_lock;spinlock_t tx_irq_lock;
@@ -1023,6 +1031,7 @@ int mtk_gmac_rgmii_path_setup(struct mtk_eth *eth, int mac_id);int mtk_eth_offload_init (struct mtk_eth *eth);int mtk_eth_setup_tc (struct net_device *dev,enum tc_setup_type type,void *type_data); +void mtk_eth_set_dma_device (struct mtk_eth *eth,struct device *dma_dev);#endif /* MTK_ETH_H */
Signature · 7 lines
2.35.1 _______________________________________________ Linux-mediatek mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-mediatek
Felix Fietkau
maintainerPATCH v2 3/14 arm64: dts: mediatek: mt7622: add support for coherent DMA
In reply to Felix FietkauIt improves performance by eliminating the need for a cache flush on rx and tx
--- arch/arm64/boot/dts/mediatek/mt7622.dtsi | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
arch/arm64/boot/dts/mediatek/mt7622.dtsi +3 -1
@@ -357,7 +357,7 @@ cci_control1: slave-if@4000 { }; cci_control2: slave-if@5000 { - compatible = "arm,cci-400-ctrl-if"; + compatible = "arm,cci-400-ctrl-if", "syscon"; interface-type = "ace"; reg = <0x5000 0x1000>; };
@@ -945,6 +945,8 @@ eth: ethernet@1b100000 { power-domains = <&scpsys MT7622_POWER_DOMAIN_ETHSYS>; mediatek,ethsys = <ðsys>; mediatek,sgmiisys = <&sgmiisys>; + mediatek,cci-control = <&cci_control2>; + dma-coherent; #address-cells = <1>; #size-cells = <0>; status = "disabled";
Signature · 7 lines
2.35.1 _______________________________________________ Linux-mediatek mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-mediatek
Felix Fietkau
maintainerPATCH v2 4/14 dt-bindings: arm: mediatek: document WED binding for MT7622
In reply to Felix FietkauFrom: Lorenzo Bianconi <[email protected]>
Document the binding for the Wireless Ethernet Dispatch core on the MT7622 SoC, which is used for Ethernet->WLAN offloading
Add related info in mediatek-net bindings.
--- .../arm/mediatek/mediatek,mt7622-wed.yaml | 50 +++++++++++++++++++ .../devicetree/bindings/net/mediatek-net.txt | 2 + 2 files changed, 52 insertions(+) create mode 100644 Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-wed.yaml
Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-wed.yaml new file +50 -0
@@ -0,0 +1,50 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: "http://devicetree.org/schemas/arm/mediatek/mediatek,mt7622-wed.yaml#" +$schema: "http://devicetree.org/meta-schemas/core.yaml#" + +title: MediaTek Wireless Ethernet Dispatch Controller for MT7622 + +maintainers: + - Lorenzo Bianconi <[email protected]> + - Felix Fietkau <[email protected]> + +description: + The mediatek wireless ethernet dispatch controller can be configured to + intercept and handle access to the WLAN DMA queues and PCIe interrupts + and implement hardware flow offloading from ethernet to WLAN. + +properties: + compatible: + items: + - enum: + - mediatek,mt7622-wed + - const: syscon + + reg: + maxItems: 1 + + interrupts: + maxItems: 1 + +required: + - compatible + - reg + - interrupts + +additionalProperties: false + +examples: + - | + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/interrupt-controller/irq.h> + soc { + #address-cells = <2>; + #size-cells = <2>; + wed0: wed@1020a000 { + compatible = "mediatek,mt7622-wed","syscon"; + reg = <0 0x1020a000 0 0x1000>; + interrupts = <GIC_SPI 214 IRQ_TYPE_LEVEL_LOW>; + }; + };
Documentation/devicetree/bindings/net/mediatek-net.txt +2 -0
@@ -46,6 +46,8 @@ Optional properties: - mediatek,cci-control: phandle to the cache coherent interconnect node - mediatek,hifsys: phandle to the mediatek hifsys controller used to provide various clocks and reset to the system. +- mediatek,wed: a list of phandles to wireless ethernet dispatch nodes for + MT7622 SoC. * Ethernet MAC node
Signature · 7 lines
2.35.1 _______________________________________________ Linux-mediatek mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-mediatek
Re: [PATCH v2 04/14] dt-bindings: arm: mediatek: document WED binding for MT7622
Quoted 8 lines · Felix Fietkau
From: Lorenzo Bianconi <[email protected]>
Document the binding for the Wireless Ethernet Dispatch core on the MT7622 SoC, which is used for
Ethernet->WLANoffloading
Add related info in mediatek-net bindings.Signed-off-by Lorenzo Bianconi <[email protected]>Signed-off-by Felix Fietkau <[email protected]>
Thank you for your patch. There is something to discuss/improve.
Quoted 5 lines
--- .../arm/mediatek/mediatek,mt7622-wed.yaml | 50 +++++++++++++++++++ .../devicetree/bindings/net/mediatek-net.txt | 2 + 2 files changed, 52 insertions(+) create mode 100644 Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-wed.yaml
Don't store drivers in arm directory. See:
https://lore.kernel.org/linux-devicetree/[email protected]/
Isn't this a network offload engine? If yes, then probably it should be in "net/".
Quoted 42 lines
Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-wed.yaml new file +35
-0@@ -0,0 +1,50 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: "http://devicetree.org/schemas/arm/mediatek/mediatek,mt7622-wed.yaml#" +$schema: "http://devicetree.org/meta-schemas/core.yaml#" + +title: MediaTek Wireless Ethernet Dispatch Controller for MT7622 + +maintainers: + - Lorenzo Bianconi <[email protected]> + - Felix Fietkau <[email protected]> + +description: + The mediatek wireless ethernet dispatch controller can be configured to + intercept and handle access to the WLAN DMA queues and PCIe interrupts + and implement hardware flow offloading from ethernet to WLAN. + +properties: + compatible: + items: + - enum: + - mediatek,mt7622-wed + - const: syscon + + reg: + maxItems: 1 + + interrupts: + maxItems: 1 + +required: + - compatible + - reg + - interrupts +
Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-wed.yaml line 45
Quoted 10 lines
Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-wed.yaml new file +10
-0+additionalProperties: false + +examples: + - | + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/interrupt-controller/irq.h> + soc { + #address-cells = <2>; + #size-cells = <2>; + wed0: wed@1020a000 {
Generic node name, "wed" is specific. Maybe "network-offload"? Or "network-accelerator"? You probably know better what this device does, so maybe come with some generic name?
The same in DTS patch.
The bindings themself look ok.
Best regards,
Krzysztof
_______________________________________________
Linux-mediatek mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-mediatek
Felix Fietkau
maintainerRe: [PATCH v2 04/14] dt-bindings: arm: mediatek: document WED binding for MT7622
Quoted 18 lines · Krzysztof Kozlowski
Quoted 8 lines · Felix Fietkau
From: Lorenzo Bianconi <[email protected]>
Document the binding for the Wireless Ethernet Dispatch core on the MT7622 SoC, which is used for
Ethernet->WLANoffloading
Add related info in mediatek-net bindings.Signed-off-by Lorenzo Bianconi <[email protected]>Signed-off-by Felix Fietkau <[email protected]>Thank you for your patch. There is something to discuss/improve.
Quoted 5 lines
--- .../arm/mediatek/mediatek,mt7622-wed.yaml | 50 +++++++++++++++++++ .../devicetree/bindings/net/mediatek-net.txt | 2 + 2 files changed, 52 insertions(+) create mode 100644 Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-wed.yaml
Quoted 5 lines
Don't store drivers in arm directory. See:
https://lore.kernel.org/linux-devicetree/[email protected]/Isn't this a network offload engine? If yes, then probably it should be in "
net/".
It's not a network offload engine by itself. It's a SoC component that connects to the offload engine and controls a MTK PCIe WLAN device, intercepting interrupts and DMA rings in order to be able to inject packets coming in from the offload engine. Do you think it still belongs in net, or maybe in soc instead?
Quoted 47 lines
Quoted 47 lines
Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-wed.yaml new file +41
-0@@ -0,0 +1,50 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: "http://devicetree.org/schemas/arm/mediatek/mediatek,mt7622-wed.yaml#" +$schema: "http://devicetree.org/meta-schemas/core.yaml#" + +title: MediaTek Wireless Ethernet Dispatch Controller for MT7622 + +maintainers: + - Lorenzo Bianconi <[email protected]> + - Felix Fietkau <[email protected]> + +description: + The mediatek wireless ethernet dispatch controller can be configured to + intercept and handle access to the WLAN DMA queues and PCIe interrupts + and implement hardware flow offloading from ethernet to WLAN. + +properties: + compatible: + items: + - enum: + - mediatek,mt7622-wed + - const: syscon + + reg: + maxItems: 1 + + interrupts: + maxItems: 1 + +required: + - compatible + - reg + - interrupts + +additionalProperties: false + +examples: + - | + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/interrupt-controller/irq.h>
Quoted 8 lines
Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-wed.yaml line 45
Quoted 4 lines
Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-wed.yaml new file +4
-0+ soc { + #address-cells = <2>; + #size-cells = <2>; + wed0: wed@1020a000 {Generic node name, "wed" is specific. Maybe "network-offload"? Or "network-accelerator"? You probably know better what this device does, so maybe come with some generic name?
wed stands for "wireless ethernet dispatch". Both network-offload and network-accelerator don't really fit. Would it make sense to spell it out, or do you have any better naming ideas?
Thanks,
- Felix
_______________________________________________
Linux-mediatek mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-mediatek
Re: [PATCH v2 04/14] dt-bindings: arm: mediatek: document WED binding for MT7622
Quoted 22 lines · Felix Fietkau
Quoted 21 lines · Krzysztof Kozlowski
Quoted 8 lines · Felix Fietkau
From: Lorenzo Bianconi <[email protected]>
Document the binding for the Wireless Ethernet Dispatch core on the MT7622 SoC, which is used for
Ethernet->WLANoffloading
Add related info in mediatek-net bindings.Signed-off-by Lorenzo Bianconi <[email protected]>Signed-off-by Felix Fietkau <[email protected]>Thank you for your patch. There is something to discuss/improve.
Quoted 5 lines
--- .../arm/mediatek/mediatek,mt7622-wed.yaml | 50 +++++++++++++++++++ .../devicetree/bindings/net/mediatek-net.txt | 2 + 2 files changed, 52 insertions(+) create mode 100644 Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-wed.yamlDon't store drivers in arm directory. See:
https://lore.kernel.org/linux-devicetree/[email protected]/
Quoted 7 lines
Quoted 2 lines
Isn't this a network offload engine? If yes, then probably it should be in "
net/".It's not a network offload engine by itself. It's a SoC component that connects to the offload engine and controls a MTK PCIe WLAN device, intercepting interrupts and DMA rings in order to be able to inject packets coming in from the offload engine.
Do you think it still belongs in net, or maybe in soc instead?
I think it belongs into drivers/net/. Presumably this has some kind of user interface to configure which packets are forwarded? I would not want to maintain that in a SoC driver as this clearly needs to communicate with both of the normal network devices in some form.
Arnd
_______________________________________________
Linux-mediatek mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-mediatek
Felix Fietkau
maintainerRe: [PATCH v2 04/14] dt-bindings: arm: mediatek: document WED binding for MT7622
Quoted 31 lines · Arnd Bergmann
Quoted 22 lines · Felix Fietkau
Quoted 21 lines · Krzysztof Kozlowski
Quoted 8 lines · Felix Fietkau
From: Lorenzo Bianconi <[email protected]>
Document the binding for the Wireless Ethernet Dispatch core on the MT7622 SoC, which is used for
Ethernet->WLANoffloading
Add related info in mediatek-net bindings.Signed-off-by Lorenzo Bianconi <[email protected]>Signed-off-by Felix Fietkau <[email protected]>Thank you for your patch. There is something to discuss/improve.
Quoted 5 lines
--- .../arm/mediatek/mediatek,mt7622-wed.yaml | 50 +++++++++++++++++++ .../devicetree/bindings/net/mediatek-net.txt | 2 + 2 files changed, 52 insertions(+) create mode 100644 Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-wed.yamlDon't store drivers in arm directory. See:
https://lore.kernel.org/linux-devicetree/[email protected]/Quoted 7 lines
Quoted 2 lines
Isn't this a network offload engine? If yes, then probably it should be in "
net/".It's not a network offload engine by itself. It's a SoC component that connects to the offload engine and controls a MTK PCIe WLAN device, intercepting interrupts and DMA rings in order to be able to inject packets coming in from the offload engine.
Do you think it still belongs in net, or maybe in soc instead?
Quoted 4 lines
I think it belongs into
drivers/net/. Presumably this has some kind of user interface to configure which packets are forwarded? I would not want to maintain that in a SoC driver as this clearly needs to communicate with both of the normal network devices in some form.
The WLAN driver attaches to WED in order to deal with the intercepted DMA rings, but other than that, WED itself has no user configuration.
Offload is controlled by the PPE code in the ethernet driver (which is already upstream), and WED simply provides a destination port for PPE, which allows packets to flow to the wireless device.
- Felix
_______________________________________________
Linux-mediatek mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-mediatek
Re: [PATCH v2 04/14] dt-bindings: arm: mediatek: document WED binding for MT7622
Quoted 17 lines · Felix Fietkau
Quoted 16 lines · Arnd Bergmann
Quoted 14 lines · Felix Fietkau
Quoted 13 lines · Krzysztof Kozlowski
Quoted 8 lines · Felix Fietkau
From: Lorenzo Bianconi <[email protected]>
Document the binding for the Wireless Ethernet Dispatch core on the MT7622 SoC, which is used for
Ethernet->WLANoffloading Add related info in mediatek-net bindings.Signed-off-by Lorenzo Bianconi <[email protected]>Signed-off-by Felix Fietkau <[email protected]>Thank you for your patch. There is something to
discuss/improve.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
---
.../arm/mediatek/mediatek,mt7622-wed.yaml| 50
+++++++++++++++++++.../devicetree/bindings/net/mediatek-net.txt| 2 + 2 files changed, 52 insertions(+) create mode 100644Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-wed.yaml
Don't store drivers in arm directory. See:
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Isn't this a network offload engine? If yes, then probably it should be
Quoted 11 lines
Quoted 11 lines
Quoted 6 lines
Quoted 1 line
in "
net/".It's not a network offload engine by itself. It's a SoC component that connects to the offload engine and controls a MTK PCIe WLAN device, intercepting interrupts and DMA rings in order to be able to inject packets coming in from the offload engine. Do you think it still belongs in net, or maybe in soc instead?
I think it belongs into
drivers/net/. Presumably this has some kind of user interface to configure which packets are forwarded? I would not want to maintain that in a SoC driver as this clearly needs to communicate with both of the normal network devices in some form.
Quoted 6 lines
The WLAN driver attaches to WED in order to deal with the intercepted
DMA rings, but other than that, WED itself has no user
configuration. Offload is controlled by the PPE code in the ethernet driver (which is already upstream), and WED simply provides a destination port for PPE, which allows packets to flow to the wireless device.
Thanks for clarification. I still wonder about the missing drivers as I responded to your second bindings:
https://lore.kernel.org/all/[email protected]/T/#m6d108c644f0c05cd12c05e56abe2ef75760c6cef
Both of these compatibles - WED and PCIe - are not actually used. Now everything is done inside your Ethernet driver which pokes WED and PCIe-mirror address space via regmap/syscon.
Separate bindings might have sense if WED/PCIe mirror were ever converted to real drivers.
Best regards,
Krzysztof
_______________________________________________
Linux-mediatek mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-mediatek
Felix Fietkau
maintainerRe: [PATCH v2 04/14] dt-bindings: arm: mediatek: document WED binding for MT7622
Quoted 10 lines · Krzysztof Kozlowski
Thanks for clarification. I still wonder about the missing drivers as I responded to your second bindings:
https://lore.kernel.org/all/[email protected]/T/#m6d108c644f0c05cd12c05e56abe2ef75760c6cefBoth of these compatibles - WED and PCIe - are not actually used. Now everything is done inside your Ethernet driver which pokes WED and PCIe-mirror address space via regmap/syscon.
Separate bindings might have sense if WED/PCIe mirror were ever
converted to real drivers.I think in terms of hardware description it makes more sense to have
separate nodes, even if the implementation uses them in one driver at the moment.
- Felix
_______________________________________________
Linux-mediatek mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-mediatek
Re: [PATCH v2 04/14] dt-bindings: arm: mediatek: document WED binding for MT7622
In reply to Felix FietkauQuoted 6 lines
Quoted 2 lines
Isn't this a network offload engine? If yes, then probably it should be in "
net/".It's not a network offload engine by itself. It's a SoC component that connects to the offload engine and controls a MTK PCIe WLAN device, intercepting interrupts and DMA rings in order to be able to inject packets coming in from the offload engine.
Hi Felix
Maybe turn the question around. Can it be used for something other than networking? If not, then somewhere under net seems reasonable.
Andrew
Felix Fietkau
maintainerRe: [PATCH v2 04/14] dt-bindings: arm: mediatek: document WED binding for MT7622
Quoted 11 lines · Andrew Lunn
Quoted 6 lines
Quoted 2 lines
Isn't this a network offload engine? If yes, then probably it should be in "
net/".It's not a network offload engine by itself. It's a SoC component that connects to the offload engine and controls a MTK PCIe WLAN device, intercepting interrupts and DMA rings in order to be able to inject packets coming in from the offload engine.
Hi Felix
Maybe turn the question around. Can it be used for something other than networking? If not, then somewhere under net seems reasonable.
I'm fine with moving this to net.
- Felix
Felix Fietkau
maintainerPATCH v2 5/14 dt-bindings: arm: mediatek: document the pcie mirror node on MT7622
In reply to Felix FietkauFrom: Lorenzo Bianconi <[email protected]>
This patch adds the pcie mirror document bindings for MT7622 SoC. The feature is used for intercepting PCIe MMIO access for the WED core Add related info in mediatek-net bindings.
--- .../mediatek/mediatek,mt7622-pcie-mirror.yaml | 42 +++++++++++++++++++ .../devicetree/bindings/net/mediatek-net.txt | 2 + 2 files changed, 44 insertions(+) create mode 100644 Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-pcie-mirror.yaml
Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-pcie-mirror.yaml new file +42 -0
@@ -0,0 +1,42 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: "http://devicetree.org/schemas/arm/mediatek/mediatek,mt7622-pcie-mirror.yaml#" +$schema: "http://devicetree.org/meta-schemas/core.yaml#" + +title: MediaTek PCIE Mirror Controller for MT7622 + +maintainers: + - Lorenzo Bianconi <[email protected]> + - Felix Fietkau <[email protected]> + +description: + The mediatek PCIE mirror provides a configuration interface for PCIE + controller on MT7622 soc. + +properties: + compatible: + items: + - enum: + - mediatek,mt7622-pcie-mirror + - const: syscon + + reg: + maxItems: 1 + +required: + - compatible + - reg + +additionalProperties: false + +examples: + - | + soc { + #address-cells = <2>; + #size-cells = <2>; + pcie_mirror: pcie-mirror@10000400 { + compatible = "mediatek,mt7622-pcie-mirror", "syscon"; + reg = <0 0x10000400 0 0x10>; + }; + };
Documentation/devicetree/bindings/net/mediatek-net.txt +2 -0
@@ -48,6 +48,8 @@ Optional properties: various clocks and reset to the system. - mediatek,wed: a list of phandles to wireless ethernet dispatch nodes for MT7622 SoC. +- mediatek,pcie-mirror: phandle to the mediatek pcie-mirror controller for + MT7622 SoC. * Ethernet MAC node
Signature · 7 lines
2.35.1 _______________________________________________ Linux-mediatek mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-mediatek
Re: [PATCH v2 05/14] dt-bindings: arm: mediatek: document the pcie mirror node on MT7622
Quoted 10 lines · Felix Fietkau
From: Lorenzo Bianconi <[email protected]>
This patch adds the pcie mirror document bindings for MT7622 SoC. The feature is used for intercepting PCIe MMIO access for the WED core Add related info in mediatek-net bindings.
Signed-off-by Lorenzo Bianconi <[email protected]>Signed-off-by Felix Fietkau <[email protected]>--- .../mediatek/mediatek,mt7622-pcie-mirror.yaml | 42 +++++++++++++++++++
Eh, I wanted to ask to not put it inside arm/, but judging by your usage
- you did not create drivers for both of these (WED and PCIe mirror).
You only need them to expose address spaces via syscon.
This actually looks hacky. Either WED and PCIe mirror are part of network driver, then add the address spaces via "reg". If they are not, but instead they are separate blocks, why you don't have drivers for them?
Best regards,
Krzysztof
_______________________________________________
Linux-mediatek mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-mediatek
Felix Fietkau
maintainerRe: [PATCH v2 05/14] dt-bindings: arm: mediatek: document the pcie mirror node on MT7622
Quoted 12 lines · Krzysztof Kozlowski
Quoted 10 lines · Felix Fietkau
From: Lorenzo Bianconi <[email protected]>
This patch adds the pcie mirror document bindings for MT7622 SoC. The feature is used for intercepting PCIe MMIO access for the WED core Add related info in mediatek-net bindings.
Signed-off-by Lorenzo Bianconi <[email protected]>Signed-off-by Felix Fietkau <[email protected]>--- .../mediatek/mediatek,mt7622-pcie-mirror.yaml | 42 +++++++++++++++++++
Quoted 8 lines
Eh, I wanted to ask to not put it inside arm/, but judging by your usage
- you did not create drivers for both of these (WED and PCIe mirror).You only need them to expose address spaces via syscon.
This actually looks hacky. Either WED and PCIe mirror are part of network driver, then add the address spaces via "reg". If they are not, but instead they are separate blocks, why you don't have drivers for them?
The code that uses the WED block is built into the Ethernet driver, but not all SoCs that use this ethernet core have it. Also, there are two WED blocks, and I'm not sure if future SoCs might have a different number of them at some point.
The WED code also needs to access registers of the ethernet MAC.
One reason for having a separate device is this:
As long as WED is not in use, ethernet supports coherent DMA for increased performance. When the first wireless device attaches to WED, IO coherency gets disabled and the ethernet DMA rings are cleaned up and allocated again, this time with the struct device of WED (which doesn't have the dma-coherent property).
- Felix
_______________________________________________
Linux-mediatek mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-mediatek
Rob Herring
maintainerRe: [PATCH v2 05/14] dt-bindings: arm: mediatek: document the pcie mirror node on MT7622
Quoted 22 lines · Felix Fietkau
Quoted 12 lines · Krzysztof Kozlowski
Quoted 10 lines · Felix Fietkau
From: Lorenzo Bianconi <[email protected]>
This patch adds the pcie mirror document bindings for MT7622 SoC. The feature is used for intercepting PCIe MMIO access for the WED core Add related info in mediatek-net bindings.
Signed-off-by Lorenzo Bianconi <[email protected]>Signed-off-by Felix Fietkau <[email protected]>--- .../mediatek/mediatek,mt7622-pcie-mirror.yaml | 42 +++++++++++++++++++Quoted 8 lines
Eh, I wanted to ask to not put it inside arm/, but judging by your usage
- you did not create drivers for both of these (WED and PCIe mirror).You only need them to expose address spaces via syscon.
This actually looks hacky. Either WED and PCIe mirror are part of network driver, then add the address spaces via "reg". If they are not, but instead they are separate blocks, why you don't have drivers for them?
Quoted 11 lines
The code that uses the WED block is built into the Ethernet driver, but not all SoCs that use this ethernet core have it. Also, there are two WED blocks, and I'm not sure if future SoCs might have a different number of them at some point.
The WED code also needs to access registers of the ethernet MAC. One reason for having a separate device is this:
As long as WED is not in use, ethernet supports coherent DMA for increased performance. When the first wireless device attaches to WED, IO coherency gets disabled and the ethernet DMA rings are cleaned up and allocated again, this time with the struct device of WED (which doesn't have the dma-coherent property).
I'm pretty sure there are assumptions in the driver core that coherency is not changing on the fly. In any case, if it is, using 'dma-coherent' is not appropriate. You obviously have another method to determine whether you are coherent or not.
Rob
_______________________________________________
Linux-mediatek mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-mediatek
Felix Fietkau
maintainerRe: [PATCH v2 05/14] dt-bindings: arm: mediatek: document the pcie mirror node on MT7622
Quoted 35 lines · Rob Herring
Quoted 22 lines · Felix Fietkau
Quoted 12 lines · Krzysztof Kozlowski
Quoted 10 lines · Felix Fietkau
From: Lorenzo Bianconi <[email protected]>
This patch adds the pcie mirror document bindings for MT7622 SoC. The feature is used for intercepting PCIe MMIO access for the WED core Add related info in mediatek-net bindings.
Signed-off-by Lorenzo Bianconi <[email protected]>Signed-off-by Felix Fietkau <[email protected]>--- .../mediatek/mediatek,mt7622-pcie-mirror.yaml | 42 +++++++++++++++++++Quoted 8 lines
Eh, I wanted to ask to not put it inside arm/, but judging by your usage
- you did not create drivers for both of these (WED and PCIe mirror).You only need them to expose address spaces via syscon.
This actually looks hacky. Either WED and PCIe mirror are part of network driver, then add the address spaces via "reg". If they are not, but instead they are separate blocks, why you don't have drivers for them?
Quoted 11 lines
The code that uses the WED block is built into the Ethernet driver, but not all SoCs that use this ethernet core have it. Also, there are two WED blocks, and I'm not sure if future SoCs might have a different number of them at some point.
The WED code also needs to access registers of the ethernet MAC. One reason for having a separate device is this:
As long as WED is not in use, ethernet supports coherent DMA for increased performance. When the first wireless device attaches to WED, IO coherency gets disabled and the ethernet DMA rings are cleaned up and allocated again, this time with the struct device of WED (which doesn't have the dma-coherent property).
Quoted 4 lines
I'm pretty sure there are assumptions in the driver core that coherency is not changing on the fly. In any case, if it is, using 'dma-coherent' is not appropriate. You obviously have another method to determine whether you are coherent or not.
It's not really on the fly. Before changing coherency, all DMA memory is freed, and the subsequent reallocation uses the struct device of the WED core, which does not have the dma-coherent property.
- Felix
_______________________________________________
Linux-mediatek mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-mediatek
Rob Herring
maintainerRe: [PATCH v2 05/14] dt-bindings: arm: mediatek: document the pcie mirror node on MT7622
In reply to Felix FietkauQuoted 32 lines · Felix Fietkau
From: Lorenzo Bianconi <[email protected]>
This patch adds the pcie mirror document bindings for MT7622 SoC. The feature is used for intercepting PCIe MMIO access for the WED core Add related info in mediatek-net bindings.
Signed-off-by Lorenzo Bianconi <[email protected]>Signed-off-by Felix Fietkau <[email protected]>--- .../mediatek/mediatek,mt7622-pcie-mirror.yaml | 42 +++++++++++++++++++ .../devicetree/bindings/net/mediatek-net.txt | 2 + 2 files changed, 44 insertions(+) create mode 100644 Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-pcie-mirror.yamlDocumentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-pcie-mirror.yaml new file +12
-0@@ -0,0 +1,42 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: "http://devicetree.org/schemas/arm/mediatek/mediatek,mt7622-pcie-mirror.yaml#" +$schema: "http://devicetree.org/meta-schemas/core.yaml#" + +title: MediaTek PCIE Mirror Controller for MT7622 + +maintainers: + - Lorenzo Bianconi <[email protected]> + - Felix Fietkau <[email protected]> +
Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-pcie-mirror.yaml line 22
Quoted 10 lines
Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-pcie-mirror.yaml new file +10
-0+description: + The mediatek PCIE mirror provides a configuration interface for PCIE + controller on MT7622 soc. + +properties: + compatible: + items: + - enum: + - mediatek,mt7622-pcie-mirror + - const: syscon
This doesn't sound like a syscon to me. Are there multiple clients or functions in this block? A 'syscon' property is not the only way to create a regmap if that's what you need.
Rob
Felix Fietkau
maintainerRe: [PATCH v2 05/14] dt-bindings: arm: mediatek: document the pcie mirror node on MT7622
Quoted 39 lines · Rob Herring
Quoted 38 lines · Felix Fietkau
From: Lorenzo Bianconi <[email protected]>
This patch adds the pcie mirror document bindings for MT7622 SoC. The feature is used for intercepting PCIe MMIO access for the WED core Add related info in mediatek-net bindings.
Signed-off-by Lorenzo Bianconi <[email protected]>Signed-off-by Felix Fietkau <[email protected]>--- .../mediatek/mediatek,mt7622-pcie-mirror.yaml | 42 +++++++++++++++++++ .../devicetree/bindings/net/mediatek-net.txt | 2 + 2 files changed, 44 insertions(+) create mode 100644 Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-pcie-mirror.yamlDocumentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-pcie-mirror.yaml new file +18
-0@@ -0,0 +1,42 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: "http://devicetree.org/schemas/arm/mediatek/mediatek,mt7622-pcie-mirror.yaml#" +$schema: "http://devicetree.org/meta-schemas/core.yaml#" + +title: MediaTek PCIE Mirror Controller for MT7622 + +maintainers: + - Lorenzo Bianconi <[email protected]> + - Felix Fietkau <[email protected]> + +description: + The mediatek PCIE mirror provides a configuration interface for PCIE + controller on MT7622 soc. + +properties: + compatible:
Quoted 8 lines
Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-pcie-mirror.yaml line 22
Quoted 4 lines
Documentation/devicetree/bindings/arm/mediatek/mediatek,mt7622-pcie-mirror.yaml new file +4
-0+ items: + - enum: + - mediatek,mt7622-pcie-mirror + - const: sysconThis doesn't sound like a syscon to me. Are there multiple clients or functions in this block? A 'syscon' property is not the only way to create a regmap if that's what you need.
It's used only by the WED code in the ethernet driver, but there are multiple WED instances and only a single pcie-mirror block containing configuration for them.
- Felix
Felix Fietkau
maintainerPATCH v2 6/14 net: ethernet: mtk_eth_soc: add support for Wireless Ethernet Dispatch (WED)
In reply to Felix FietkauThe Wireless Ethernet Dispatch subsystem on the MT7622 SoC can be configured to intercept and handle access to the DMA queues and PCIe interrupts for a MT7615/MT7915 wireless card.
It can manage the internal WDMA (Wireless DMA) controller, which allows ethernet packets to be passed from the packet switch engine (PSE) to the wireless card, bypassing the CPU entirely.
This can be used to implement hardware flow offloading from ethernet to WLAN.
--- drivers/net/ethernet/mediatek/Kconfig | 4 + drivers/net/ethernet/mediatek/Makefile | 5 + drivers/net/ethernet/mediatek/mtk_eth_soc.c | 17 + drivers/net/ethernet/mediatek/mtk_eth_soc.h | 3 + drivers/net/ethernet/mediatek/mtk_wed.c | 875 ++++++++++++++++++ drivers/net/ethernet/mediatek/mtk_wed.h | 128 +++ .../net/ethernet/mediatek/mtk_wed_debugfs.c | 175 ++++ drivers/net/ethernet/mediatek/mtk_wed_ops.c | 8 + drivers/net/ethernet/mediatek/mtk_wed_regs.h | 251 +++++ include/linux/soc/mediatek/mtk_wed.h | 131 +++ 10 files changed, 1597 insertions(+) create mode 100644 drivers/net/ethernet/mediatek/mtk_wed.c create mode 100644 drivers/net/ethernet/mediatek/mtk_wed.h create mode 100644 drivers/net/ethernet/mediatek/mtk_wed_debugfs.c create mode 100644 drivers/net/ethernet/mediatek/mtk_wed_ops.c create mode 100644 drivers/net/ethernet/mediatek/mtk_wed_regs.h create mode 100644 include/linux/soc/mediatek/mtk_wed.h
drivers/net/ethernet/mediatek/Kconfig +4 -0
@@ -7,6 +7,10 @@ config NET_VENDOR_MEDIATEK if NET_VENDOR_MEDIATEK +config NET_MEDIATEK_SOC_WED + depends on ARCH_MEDIATEK || COMPILE_TEST + def_bool NET_MEDIATEK_SOC != n + config NET_MEDIATEK_SOC tristate "MediaTek SoC Gigabit Ethernet support" depends on NET_DSA || !NET_DSA
drivers/net/ethernet/mediatek/Makefile +5 -0
@@ -5,4 +5,9 @@ obj-$(CONFIG_NET_MEDIATEK_SOC) += mtk_eth.o mtk_eth-y := mtk_eth_soc.o mtk_sgmii.o mtk_eth_path.o mtk_ppe.o mtk_ppe_debugfs.o mtk_ppe_offload.o +mtk_eth-$(CONFIG_NET_MEDIATEK_SOC_WED) += mtk_wed.o +ifdef CONFIG_DEBUG_FS +mtk_eth-$(CONFIG_NET_MEDIATEK_SOC_WED) += mtk_wed_debugfs.o +endif +obj-$(CONFIG_NET_MEDIATEK_SOC_WED) += mtk_wed_ops.o obj-$(CONFIG_NET_MEDIATEK_STAR_EMAC) += mtk_star_emac.o
drivers/net/ethernet/mediatek/mtk_eth_soc.c +17 -0
@@ -24,6 +24,7 @@#include <net/dsa.h> #include "mtk_eth_soc.h" +#include "mtk_wed.h" static int mtk_msg_level =-1 ;module_param_named (msg_level, mtk_msg_level, int,0 );
@@ -3170,6 +3171,22 @@ static int mtk_probe(struct platform_device *pdev) } } +for (i =0 ;; i++) { +struct device_node *np =of_parse_phandle (pdev->dev.of_node, +"mediatek,wed" , i); +static const u32 wdma_regs[] = { +MTK_WDMA0_BASE , +MTK_WDMA1_BASE + }; +void __iomem *wdma; + +if (!np || i >=ARRAY_SIZE (wdma_regs)) +break ; + + wdma = eth->base + wdma_regs[i]; +mtk_wed_add_hw (np, eth, wdma, i); + } +for (i =0 ; i <3 ; i++) {if (MTK_HAS_CAPS (eth->soc->caps,MTK_SHARED_INT ) && i >0 ) eth->irq[i] = eth->irq[0 ];
drivers/net/ethernet/mediatek/mtk_eth_soc.h +3 -0
@@ -295,6 +295,9 @@#define MTK_GDM1_TX_GPCNT 0x2438#define MTK_STAT_OFFSET 0x40 +#define MTK_WDMA0_BASE 0x2800 +#define MTK_WDMA1_BASE 0x2c00 +/* QDMA descriptor txd4 */ #define TX_DMA_CHKSUM (0x7 << 29)#define TX_DMA_TSO BIT(28)
drivers/net/ethernet/mediatek/mtk_wed.c new file +875 -0
875 lines not on this page · raw message
drivers/net/ethernet/mediatek/mtk_wed.h new file +128 -0
128 lines not on this page · raw message
drivers/net/ethernet/mediatek/mtk_wed_debugfs.c new file +175 -0
175 lines not on this page · raw message
drivers/net/ethernet/mediatek/mtk_wed_ops.c new file +8 -0
@@ -0,0 +1,8 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (C) 2020 Felix Fietkau <[email protected]> */ + +#include <linux/kernel.h> +#include <linux/soc/mediatek/mtk_wed.h> + +const struct mtk_wed_ops __rcu *mtk_soc_wed_ops; +EXPORT_SYMBOL_GPL (mtk_soc_wed_ops);
drivers/net/ethernet/mediatek/mtk_wed_regs.h new file +251 -0
251 lines not on this page · raw message
include/linux/soc/mediatek/mtk_wed.h new file +131 -0
131 lines not on this page · raw message
Signature · 7 lines
2.35.1 _______________________________________________ Linux-mediatek mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-mediatek
Felix Fietkau
maintainerPATCH v2 7/14 net: ethernet: mtk_eth_soc: implement flow offloading to WED devices
In reply to Felix FietkauThis allows hardware flow offloading from Ethernet to WLAN on MT7622 SoC
--- drivers/net/ethernet/mediatek/mtk_ppe.c | 18 ++++++ drivers/net/ethernet/mediatek/mtk_ppe.h | 14 +++-- .../net/ethernet/mediatek/mtk_ppe_offload.c | 56 ++++++++++++++++++- drivers/net/ethernet/mediatek/mtk_wed.h | 7 +++ include/linux/netdevice.h | 7 +++ net/core/dev.c | 4 ++ 6 files changed, 98 insertions(+), 8 deletions(-)
drivers/net/ethernet/mediatek/mtk_ppe.c +18 -0
@@ -329,6 +329,24 @@ int mtk_foe_entry_set_pppoe(struct mtk_foe_entry *entry, int sid)return 0 ; } +int mtk_foe_entry_set_wdma (struct mtk_foe_entry *entry,int wdma_idx,int txq, +int bss,int wcid) +{ +struct mtk_foe_mac_info *l2 =mtk_foe_entry_l2 (entry); +u32 *ib2 =mtk_foe_entry_ib2 (entry); + + *ib2 &= ~MTK_FOE_IB2_PORT_MG ; + *ib2 |=MTK_FOE_IB2_WDMA_WINFO ; +if (wdma_idx) + *ib2 |=MTK_FOE_IB2_WDMA_DEVIDX ; + + l2->vlan2 =FIELD_PREP (MTK_FOE_VLAN2_WINFO_BSS , bss) | +FIELD_PREP (MTK_FOE_VLAN2_WINFO_WCID , wcid) | +FIELD_PREP (MTK_FOE_VLAN2_WINFO_RING , txq); + +return 0 ; +} +static inline bool mtk_foe_entry_usable (struct mtk_foe_entry *entry) { return !(entry->ib1 & MTK_FOE_IB1_STATIC) &&
drivers/net/ethernet/mediatek/mtk_ppe.h +8 -6
@@ -48,9 +48,9 @@ enum {#define MTK_FOE_IB2_DEST_PORT GENMASK(7, 5)#define MTK_FOE_IB2_MULTICAST BIT(8) -#define MTK_FOE_IB2_WHNAT_QID2 GENMASK(13, 12) -#define MTK_FOE_IB2_WHNAT_DEVIDX BIT(16) -#define MTK_FOE_IB2_WHNAT_NAT BIT(17) +#define MTK_FOE_IB2_WDMA_QID2 GENMASK(13, 12) +#define MTK_FOE_IB2_WDMA_DEVIDX BIT(16) +#define MTK_FOE_IB2_WDMA_WINFO BIT(17)#define MTK_FOE_IB2_PORT_MG GENMASK(17, 12)
@@ -58,9 +58,9 @@ enum {#define MTK_FOE_IB2_DSCP GENMASK(31, 24) -#define MTK_FOE_VLAN2_WHNAT_BSS GEMMASK(5, 0) -#define MTK_FOE_VLAN2_WHNAT_WCID GENMASK(13, 6) -#define MTK_FOE_VLAN2_WHNAT_RING GENMASK(15, 14) +#define MTK_FOE_VLAN2_WINFO_BSS GENMASK(5, 0) +#define MTK_FOE_VLAN2_WINFO_WCID GENMASK(13, 6) +#define MTK_FOE_VLAN2_WINFO_RING GENMASK(15, 14)enum {MTK_FOE_STATE_INVALID ,
@@ -281,6 +281,8 @@ int mtk_foe_entry_set_ipv6_tuple(struct mtk_foe_entry *entry,int mtk_foe_entry_set_dsa (struct mtk_foe_entry *entry,int port);int mtk_foe_entry_set_vlan (struct mtk_foe_entry *entry,int vid);int mtk_foe_entry_set_pppoe (struct mtk_foe_entry *entry,int sid); +int mtk_foe_entry_set_wdma (struct mtk_foe_entry *entry,int wdma_idx,int txq, +int bss,int wcid);int mtk_foe_entry_commit (struct mtk_ppe *ppe,struct mtk_foe_entry *entry,u16 timestamp);int mtk_ppe_debugfs_init (struct mtk_ppe *ppe);
drivers/net/ethernet/mediatek/mtk_ppe_offload.c +54 -2
122 lines not on this page · raw message
drivers/net/ethernet/mediatek/mtk_wed.h +7 -0
@@ -7,6 +7,7 @@#include <linux/soc/mediatek/mtk_wed.h> #include <linux/debugfs.h> #include <linux/regmap.h> +#include <linux/netdevice.h> struct mtk_eth ;
@@ -27,6 +28,12 @@ struct mtk_wed_hw {int index; }; +struct mtk_wdma_info { +u8 wdma_idx; +u8 queue; +u16 wcid; +u8 bss; +};#ifdef CONFIG_NET_MEDIATEK_SOC_WED static inline void
include/linux/netdevice.h +7 -0
@@ -862,6 +862,7 @@ enum net_device_path_type {DEV_PATH_BRIDGE ,DEV_PATH_PPPOE ,DEV_PATH_DSA , + DEV_PATH_MTK_WDMA, }; struct net_device_path {
@@ -887,6 +888,12 @@ struct net_device_path {int port;u16 proto; } dsa; +struct { +u8 wdma_idx; +u8 queue; +u16 wcid; +u8 bss; + } mtk_wdma; }; };
net/core/dev.c +4 -0
@@ -701,6 +701,10 @@ int dev_fill_forward_path(const struct net_device *dev, const u8 *daddr,if (WARN_ON_ONCE (last_dev == ctx.dev))return -1 ; } + +if (!ctx.dev) +return ret; + path =dev_fwd_path (stack);if (!path)return -1 ;
Signature · 7 lines
2.35.1 _______________________________________________ Linux-mediatek mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-mediatek
Felix Fietkau
maintainerPATCH v2 8/14 arm64: dts: mediatek: mt7622: introduce nodes for Wireless Ethernet Dispatch
In reply to Felix FietkauIntroduce wed0 and wed1 nodes in order to enable offloading forwarding between ethernet and wireless devices on the mt7622 chipset.
--- arch/arm64/boot/dts/mediatek/mt7622.dtsi | 28 ++++++++++++++++++++++++ 1 file changed, 28 insertions(+)
arch/arm64/boot/dts/mediatek/mt7622.dtsi +28 -0
@@ -901,6 +901,11 @@ sata_port: sata-phy@1a243000 { }; }; + hifsys: syscon@1af00000 { + compatible = "mediatek,mt7622-hifsys", "syscon"; + reg = <0 0x1af00000 0 0x70>; + }; + ethsys: syscon@1b000000 { compatible = "mediatek,mt7622-ethsys", "syscon";
@@ -919,6 +924,26 @@ hsdma: dma-controller@1b007000 { #dma-cells = <1>; }; + pcie_mirror: pcie-mirror@10000400 { + compatible = "mediatek,mt7622-pcie-mirror", + "syscon"; + reg = <0 0x10000400 0 0x10>; + }; + + wed0: wed@1020a000 { + compatible = "mediatek,mt7622-wed", + "syscon"; + reg = <0 0x1020a000 0 0x1000>; + interrupts = <GIC_SPI 214 IRQ_TYPE_LEVEL_LOW>; + }; + + wed1: wed@1020b000 { + compatible = "mediatek,mt7622-wed", + "syscon"; + reg = <0 0x1020b000 0 0x1000>; + interrupts = <GIC_SPI 215 IRQ_TYPE_LEVEL_LOW>; + }; + eth: ethernet@1b100000 { compatible = "mediatek,mt7622-eth", "mediatek,mt2701-eth",
@@ -946,6 +971,9 @@ eth: ethernet@1b100000 { mediatek,ethsys = <ðsys>; mediatek,sgmiisys = <&sgmiisys>; mediatek,cci-control = <&cci_control2>; + mediatek,wed = <&wed0>, <&wed1>; + mediatek,pcie-mirror = <&pcie_mirror>; + mediatek,hifsys = <&hifsys>; dma-coherent; #address-cells = <1>; #size-cells = <0>;
Signature · 7 lines
2.35.1 _______________________________________________ Linux-mediatek mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-mediatek
Felix Fietkau
maintainerPATCH v2 9/14 net: ethernet: mtk_eth_soc: add ipv6 flow offload support
In reply to Felix FietkauFrom: David Bentham <[email protected]>
Add the missing IPv6 flow offloading support for routing only. Hardware flow offloading is done by the packet processing engine (PPE) of the Ethernet MAC and as it doesn't support mangling of IPv6 packets, IPv6 NAT cannot be supported.
--- .../net/ethernet/mediatek/mtk_ppe_offload.c | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+)
drivers/net/ethernet/mediatek/mtk_ppe_offload.c +28 -0
@@ -6,6 +6,7 @@#include <linux/if_ether.h> #include <linux/rhashtable.h> #include <linux/ip.h> +#include <linux/ipv6.h> #include <net/flow_offload.h> #include <net/pkt_cls.h> #include <net/dsa.h>
@@ -20,6 +21,11 @@ struct mtk_flow_data {__be32 src_addr;__be32 dst_addr; } v4; + +struct { +struct in6_addr src_addr; +struct in6_addr dst_addr; + } v6; };__be16 src_port;
@@ -65,6 +71,14 @@ mtk_flow_set_ipv4_addr(struct mtk_foe_entry *foe, struct mtk_flow_data *data, data->v4.dst_addr, data->dst_port); } +static int +mtk_flow_set_ipv6_addr (struct mtk_foe_entry *foe,struct mtk_flow_data *data) +{ +return mtk_foe_entry_set_ipv6_tuple (foe, + data->v6.src_addr.s6_addr32, data->src_port, + data->v6.dst_addr.s6_addr32, data->dst_port); +} +static void mtk_flow_offload_mangle_eth (const struct flow_action_entry *act,void *eth) {
@@ -296,6 +310,9 @@ mtk_flow_offload_replace(struct mtk_eth *eth, struct flow_cls_offload *f)case FLOW_DISSECTOR_KEY_IPV4_ADDRS : offload_type =MTK_PPE_PKT_TYPE_IPV4_HNAPT ;break ; +case FLOW_DISSECTOR_KEY_IPV6_ADDRS : + offload_type =MTK_PPE_PKT_TYPE_IPV6_ROUTE_5T ; +break ;default :return -EOPNOTSUPP ; }
@@ -331,6 +348,17 @@ mtk_flow_offload_replace(struct mtk_eth *eth, struct flow_cls_offload *f)mtk_flow_set_ipv4_addr (&foe, &data, false); } +if (addr_type ==FLOW_DISSECTOR_KEY_IPV6_ADDRS ) { +struct flow_match_ipv6_addrs addrs; + +flow_rule_match_ipv6_addrs (rule, &addrs); + + data.v6.src_addr = addrs.key->src; + data.v6.dst_addr = addrs.key->dst; + +mtk_flow_set_ipv6_addr (&foe, &data); + } +flow_action_for_each (i, act, &rule->action) {if (act->id !=FLOW_ACTION_MANGLE )continue ;
Signature · 7 lines
2.35.1 _______________________________________________ Linux-mediatek mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-mediatek
Felix Fietkau
maintainerPATCH v2 10/14 net: ethernet: mtk_eth_soc: support TC_SETUP_BLOCK for PPE offload
In reply to Felix FietkauThis allows offload entries to be created from user space
--- drivers/net/ethernet/mediatek/mtk_ppe_offload.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
drivers/net/ethernet/mediatek/mtk_ppe_offload.c +6 -3
@@ -563,10 +563,13 @@ mtk_eth_setup_tc_block(struct net_device *dev, struct flow_block_offload *f)int mtk_eth_setup_tc (struct net_device *dev,enum tc_setup_type type,void *type_data) { -if (type ==TC_SETUP_FT ) +switch (type) { +case TC_SETUP_BLOCK : +case TC_SETUP_FT :return mtk_eth_setup_tc_block (dev, type_data); - -return -EOPNOTSUPP ; +default : +return -EOPNOTSUPP ; + } }int mtk_eth_offload_init (struct mtk_eth *eth)
Signature · 7 lines
2.35.1 _______________________________________________ Linux-mediatek mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-mediatek
Felix Fietkau
maintainerPATCH v2 11/14 net: ethernet: mtk_eth_soc: allocate struct mtk_ppe separately
In reply to Felix FietkauPreparation for adding more data to it, which will increase its size.
--- drivers/net/ethernet/mediatek/mtk_eth_soc.c | 11 ++++++----- drivers/net/ethernet/mediatek/mtk_eth_soc.h | 2 +- drivers/net/ethernet/mediatek/mtk_ppe.c | 11 ++++++++--- drivers/net/ethernet/mediatek/mtk_ppe.h | 3 +-- drivers/net/ethernet/mediatek/mtk_ppe_offload.c | 12 ++++++------ 5 files changed, 22 insertions(+), 17 deletions(-)
drivers/net/ethernet/mediatek/mtk_eth_soc.c +6 -5
@@ -2269,7 +2269,7 @@ static int mtk_open(struct net_device *dev)if (err)return err; -if (eth->soc->offload_version &&mtk_ppe_start (ð->ppe) ==0 ) +if (eth->soc->offload_version &&mtk_ppe_start (eth->ppe) ==0 ) gdm_config =MTK_GDMA_TO_PPE ;mtk_gdm_config (eth, gdm_config);
@@ -2343,7 +2343,7 @@ static int mtk_stop(struct net_device *dev)mtk_dma_free (eth);if (eth->soc->offload_version) -mtk_ppe_stop (ð->ppe); +mtk_ppe_stop (eth->ppe);return 0 ; }
@@ -3262,10 +3262,11 @@ static int mtk_probe(struct platform_device *pdev) }if (eth->soc->offload_version) { - err =mtk_ppe_init (ð->ppe, eth->dev, - eth->base +MTK_ETH_PPE_BASE ,2 ); -if (err) + eth->ppe =mtk_ppe_init (eth->dev, eth->base +MTK_ETH_PPE_BASE ,2 ); +if (!eth->ppe) { + err = -ENOMEM ; goto err_free_dev; + } err =mtk_eth_offload_init (eth);if (err)
drivers/net/ethernet/mediatek/mtk_eth_soc.h +1 -1
@@ -985,7 +985,7 @@ struct mtk_eth {u32 rx_dma_l4_valid;int ip_align; -struct mtk_ppe ppe; +struct mtk_ppe *ppe;struct rhashtable flow_table; };
drivers/net/ethernet/mediatek/mtk_ppe.c +8 -3
@@ -384,10 +384,15 @@ int mtk_foe_entry_commit(struct mtk_ppe *ppe, struct mtk_foe_entry *entry,return hash; } -int mtk_ppe_init (struct mtk_ppe *ppe,struct device *dev,void __iomem *base, +struct mtk_ppe *mtk_ppe_init (struct device *dev,void __iomem *base,int version) {struct mtk_foe_entry *foe; +struct mtk_ppe *ppe; + + ppe =devm_kzalloc (dev,sizeof (*ppe),GFP_KERNEL ); +if (!ppe) +return NULL ;/* need to allocate a separate device, since it PPE DMA access is * not coherent.
@@ -399,13 +404,13 @@ int mtk_ppe_init(struct mtk_ppe *ppe, struct device *dev, void __iomem *base, foe =dmam_alloc_coherent (ppe->dev,MTK_PPE_ENTRIES *sizeof (*foe), &ppe->foe_phys,GFP_KERNEL );if (!foe) -return -ENOMEM ; +return NULL ; ppe->foe_table = foe;mtk_ppe_debugfs_init (ppe); -return 0 ; +return ppe; }static void mtk_ppe_init_foe_table (struct mtk_ppe *ppe)
drivers/net/ethernet/mediatek/mtk_ppe.h +1 -2
@@ -246,8 +246,7 @@ struct mtk_ppe {void *acct_table; }; -int mtk_ppe_init (struct mtk_ppe *ppe,struct device *dev,void __iomem *base, -int version); +struct mtk_ppe *mtk_ppe_init (struct device *dev,void __iomem *base,int version);int mtk_ppe_start (struct mtk_ppe *ppe);int mtk_ppe_stop (struct mtk_ppe *ppe);
drivers/net/ethernet/mediatek/mtk_ppe_offload.c +6 -6
@@ -411,7 +411,7 @@ mtk_flow_offload_replace(struct mtk_eth *eth, struct flow_cls_offload *f) entry->cookie = f->cookie; timestamp =mtk_eth_timestamp (eth); - hash =mtk_foe_entry_commit (ð->ppe, &foe, timestamp); + hash =mtk_foe_entry_commit (eth->ppe, &foe, timestamp);if (hash <0 ) { err = hash; goto free;
@@ -426,7 +426,7 @@ mtk_flow_offload_replace(struct mtk_eth *eth, struct flow_cls_offload *f)return 0 ; clear_flow: -mtk_foe_entry_clear (ð->ppe, hash); +mtk_foe_entry_clear (eth->ppe, hash); free:kfree (entry);if (wed_index >=0 )
@@ -444,7 +444,7 @@ mtk_flow_offload_destroy(struct mtk_eth *eth, struct flow_cls_offload *f)if (!entry)return -ENOENT ; -mtk_foe_entry_clear (ð->ppe, entry->hash); +mtk_foe_entry_clear (eth->ppe, entry->hash);rhashtable_remove_fast (ð->flow_table, &entry->node, mtk_flow_ht_params);if (entry->wed_index >=0 )
@@ -466,7 +466,7 @@ mtk_flow_offload_stats(struct mtk_eth *eth, struct flow_cls_offload *f)if (!entry)return -ENOENT ; - timestamp =mtk_foe_entry_timestamp (ð->ppe, entry->hash); + timestamp =mtk_foe_entry_timestamp (eth->ppe, entry->hash);if (timestamp <0 )return -ETIMEDOUT ;
@@ -522,7 +522,7 @@ mtk_eth_setup_tc_block(struct net_device *dev, struct flow_block_offload *f)struct flow_block_cb *block_cb;flow_setup_cb_t *cb; -if (!eth->ppe.foe_table) +if (!eth->ppe || !eth->ppe->foe_table)return -EOPNOTSUPP ;if (f->binder_type !=FLOW_BLOCK_BINDER_TYPE_CLSACT_INGRESS )
@@ -574,7 +574,7 @@ int mtk_eth_setup_tc(struct net_device *dev, enum tc_setup_type type,int mtk_eth_offload_init (struct mtk_eth *eth) { -if (!eth->ppe.foe_table) +if (!eth->ppe || !eth->ppe->foe_table)return 0 ;return rhashtable_init (ð->flow_table, &mtk_flow_ht_params);
Signature · 7 lines
2.35.1 _______________________________________________ Linux-mediatek mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-mediatek
Felix Fietkau
maintainerPATCH v2 12/14 net: ethernet: mtk_eth_soc: rework hardware flow table management
In reply to Felix FietkauThe hardware was designed to handle flow detection and creation of flow entries by itself, relying on the software primarily for filling in egress routing information.
When there is a hash collision between multiple flows, this allows the hardware to maintain the entry for the most active flow.
Additionally, the hardware only keeps offloading active for entries with at least 30 packets per second.
With this rework, the code no longer creates a hardware entries directly. Instead, the hardware entry is only created when the PPE reports a matching unbound flow with the minimum target rate.
In order to reduce CPU overhead, looking for flows belonging to a hash entry is rate limited to once every 100ms.
This rework is also used as preparation for emulating bridge offload by managing L4 offload entries on demand.
--- drivers/net/ethernet/mediatek/mtk_eth_soc.c | 10 +- drivers/net/ethernet/mediatek/mtk_ppe.c | 134 ++++++++++++++++-- drivers/net/ethernet/mediatek/mtk_ppe.h | 38 ++++- .../net/ethernet/mediatek/mtk_ppe_offload.c | 42 ++---- 4 files changed, 170 insertions(+), 54 deletions(-)
drivers/net/ethernet/mediatek/mtk_eth_soc.c +8 -2
@@ -21,6 +21,7 @@#include <linux/pinctrl/devinfo.h> #include <linux/phylink.h> #include <linux/jhash.h> +#include <linux/bitfield.h> #include <net/dsa.h> #include "mtk_eth_soc.h"
@@ -1239,7 +1240,7 @@ static int mtk_poll_rx(struct napi_struct *napi, int budget,struct net_device *netdev;unsigned int pktlen;dma_addr_t dma_addr; -u32 hash; +u32 hash, reason;int mac; ring =mtk_get_rx_ring (eth);
@@ -1315,6 +1316,11 @@ static int mtk_poll_rx(struct napi_struct *napi, int budget,skb_set_hash (skb, hash,PKT_HASH_TYPE_L4 ); } + reason =FIELD_GET (MTK_RXD4_PPE_CPU_REASON , trxd.rxd4); +if (reason ==MTK_PPE_CPU_REASON_HIT_UNBIND_RATE_REACHED ) +mtk_ppe_check_skb (eth->ppe, skb, + trxd.rxd4 &MTK_RXD4_FOE_ENTRY ); +if (netdev->features &NETIF_F_HW_VLAN_CTAG_RX && (trxd.rxd2 &RX_DMA_VTAG )) __vlan_hwaccel_put_tag(skb, htons(ETH_P_8021Q),
@@ -3262,7 +3268,7 @@ static int mtk_probe(struct platform_device *pdev) }if (eth->soc->offload_version) { - eth->ppe =mtk_ppe_init (eth->dev, eth->base +MTK_ETH_PPE_BASE ,2 ); + eth->ppe =mtk_ppe_init (eth, eth->base +MTK_ETH_PPE_BASE ,2 );if (!eth->ppe) { err = -ENOMEM ; goto err_free_dev;
drivers/net/ethernet/mediatek/mtk_ppe.c +120 -14
180 lines not on this page · raw message
drivers/net/ethernet/mediatek/mtk_ppe.h +32 -6
@@ -235,7 +235,17 @@ enum { MTK_PPE_CPU_REASON_INVALID = 0x1f, }; +struct mtk_flow_entry { +struct rhash_head node; +struct hlist_node list; +unsigned long cookie; +struct mtk_foe_entry data; +u16 hash; +s8 wed_index; +}; +struct mtk_ppe { +struct mtk_eth *eth;struct device *dev;void __iomem *base;int version;
@@ -243,18 +253,33 @@ struct mtk_ppe {struct mtk_foe_entry *foe_table;dma_addr_t foe_phys; +u16 foe_check_time[MTK_PPE_ENTRIES ]; +struct hlist_head foe_flow[MTK_PPE_ENTRIES /2 ]; +void *acct_table; }; -struct mtk_ppe *mtk_ppe_init (struct device *dev,void __iomem *base,int version); +struct mtk_ppe *mtk_ppe_init (struct mtk_eth *eth,void __iomem *base,int version);int mtk_ppe_start (struct mtk_ppe *ppe);int mtk_ppe_stop (struct mtk_ppe *ppe); +void __mtk_ppe_check_skb (struct mtk_ppe *ppe,struct sk_buff *skb,u16 hash); +static inline void -mtk_foe_entry_clear (struct mtk_ppe *ppe,u16 hash) +mtk_ppe_check_skb (struct mtk_ppe *ppe,struct sk_buff *skb,u16 hash) { - ppe->foe_table[hash].ib1 =0 ; -dma_wmb (); +u16 now, diff; + +if (!ppe) +return ; + + now = (u16 )jiffies; + diff = now - ppe->foe_check_time[hash]; +if (diff <HZ /10 ) +return ; + + ppe->foe_check_time[hash] = now; +__mtk_ppe_check_skb (ppe, skb, hash); } static inline int
@@ -282,8 +307,9 @@ int mtk_foe_entry_set_vlan(struct mtk_foe_entry *entry, int vid);int mtk_foe_entry_set_pppoe (struct mtk_foe_entry *entry,int sid);int mtk_foe_entry_set_wdma (struct mtk_foe_entry *entry,int wdma_idx,int txq,int bss,int wcid); -int mtk_foe_entry_commit (struct mtk_ppe *ppe,struct mtk_foe_entry *entry, -u16 timestamp); +int mtk_foe_entry_commit (struct mtk_ppe *ppe,struct mtk_flow_entry *entry); +void mtk_foe_entry_clear (struct mtk_ppe *ppe,struct mtk_flow_entry *entry); +int mtk_foe_entry_idle_time (struct mtk_ppe *ppe,struct mtk_flow_entry *entry);int mtk_ppe_debugfs_init (struct mtk_ppe *ppe);#endif
drivers/net/ethernet/mediatek/mtk_ppe_offload.c +10 -32
93 lines not on this page · raw message
Signature · 7 lines
2.35.1 _______________________________________________ Linux-mediatek mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-mediatek
Felix Fietkau
maintainerPATCH v2 13/14 net: ethernet: mtk_eth_soc: remove bridge flow offload type entry support
In reply to Felix FietkauAccording to MediaTek, this feature is not supported in current hardware
--- drivers/net/ethernet/mediatek/mtk_ppe.c | 8 -------- drivers/net/ethernet/mediatek/mtk_ppe_debugfs.c | 1 - 2 files changed, 9 deletions(-)
drivers/net/ethernet/mediatek/mtk_ppe.c +0 -8
@@ -84,13 +84,6 @@ static u32 mtk_ppe_hash_entry(struct mtk_foe_entry *e)u32 hash;switch (FIELD_GET (MTK_FOE_IB1_PACKET_TYPE , e->ib1)) { -case MTK_PPE_PKT_TYPE_BRIDGE : - hv1 = e->bridge.src_mac_lo; - hv1 ^= ((e->bridge.src_mac_hi &0xffff ) <<16 ); - hv2 = e->bridge.src_mac_hi >>16 ; - hv2 ^= e->bridge.dest_mac_lo; - hv3 = e->bridge.dest_mac_hi; -break ;case MTK_PPE_PKT_TYPE_IPV4_ROUTE :case MTK_PPE_PKT_TYPE_IPV4_HNAPT : hv1 = e->ipv4.orig.ports;
@@ -572,7 +565,6 @@ int mtk_ppe_start(struct mtk_ppe *ppe)MTK_PPE_FLOW_CFG_IP4_NAT |MTK_PPE_FLOW_CFG_IP4_NAPT |MTK_PPE_FLOW_CFG_IP4_DSLITE | -MTK_PPE_FLOW_CFG_L2_BRIDGE |MTK_PPE_FLOW_CFG_IP4_NAT_FRAG ;ppe_w32 (ppe,MTK_PPE_FLOW_CFG , val);
drivers/net/ethernet/mediatek/mtk_ppe_debugfs.c +0 -1
@@ -32,7 +32,6 @@ static const char *mtk_foe_pkt_type_str(int type)static const char *const type_str[] = { [MTK_PPE_PKT_TYPE_IPV4_HNAPT ] ="IPv4 5T" , [MTK_PPE_PKT_TYPE_IPV4_ROUTE ] ="IPv4 3T" , - [MTK_PPE_PKT_TYPE_BRIDGE ] ="L2" , [MTK_PPE_PKT_TYPE_IPV4_DSLITE ] ="DS-LITE" , [MTK_PPE_PKT_TYPE_IPV6_ROUTE_3T ] ="IPv6 3T" , [MTK_PPE_PKT_TYPE_IPV6_ROUTE_5T ] ="IPv6 5T" ,
Signature · 7 lines
2.35.1 _______________________________________________ Linux-mediatek mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-mediatek
Felix Fietkau
maintainerPATCH v2 14/14 net: ethernet: mtk_eth_soc: support creating mac address based offload entries
In reply to Felix FietkauThis will be used to implement a limited form of bridge offloading. Since the hardware does not support flow table entries with just source and destination MAC address, the driver has to emulate it.
The hardware automatically creates entries entries for incoming flows, even when they are bridged instead of routed, and reports when packets for these flows have reached the minimum PPS rate for offloading.
After this happens, we look up the L2 flow offload entry based on the MAC header and fill in the output routing information in the flow table. The dynamically created per-flow entries are automatically removed when either the hardware flowtable entry expires, is replaced, or if the offload rule they belong to is removed
--- drivers/net/ethernet/mediatek/mtk_ppe.c | 241 ++++++++++++++++-- drivers/net/ethernet/mediatek/mtk_ppe.h | 44 +++- .../net/ethernet/mediatek/mtk_ppe_offload.c | 60 ++++- 3 files changed, 299 insertions(+), 46 deletions(-)
drivers/net/ethernet/mediatek/mtk_ppe.c +219 -22
@@ -6,12 +6,22 @@#include <linux/iopoll.h> #include <linux/etherdevice.h> #include <linux/platform_device.h> +#include <linux/if_ether.h> +#include <linux/if_vlan.h> +#include <net/dsa.h> #include "mtk_eth_soc.h" #include "mtk_ppe.h" #include "mtk_ppe_regs.h" static DEFINE_SPINLOCK (ppe_lock); +static const struct rhashtable_params mtk_flow_l2_ht_params = { + .head_offset = offsetof(struct mtk_flow_entry , l2_node), + .key_offset = offsetof(struct mtk_flow_entry, data.bridge), + .key_len = offsetof(struct mtk_foe_bridge , key_end), + .automatic_shrinking = true, +}; +static void ppe_w32 (struct mtk_ppe *ppe,u32 reg,u32 val) {writel (val, ppe->base + reg);
@@ -123,6 +133,9 @@ mtk_foe_entry_l2(struct mtk_foe_entry *entry) {int type =FIELD_GET (MTK_FOE_IB1_PACKET_TYPE , entry->ib1); +if (type ==MTK_PPE_PKT_TYPE_BRIDGE ) +return &entry->bridge.l2; +if (type >=MTK_PPE_PKT_TYPE_IPV4_DSLITE )return &entry->ipv6.l2;
@@ -134,6 +147,9 @@ mtk_foe_entry_ib2(struct mtk_foe_entry *entry) {int type =FIELD_GET (MTK_FOE_IB1_PACKET_TYPE , entry->ib1); +if (type ==MTK_PPE_PKT_TYPE_BRIDGE ) +return &entry->bridge.ib2; +if (type >=MTK_PPE_PKT_TYPE_IPV4_DSLITE )return &entry->ipv6.ib2;
@@ -168,7 +184,12 @@ int mtk_foe_entry_prepare(struct mtk_foe_entry *entry, int type, int l4proto,if (type ==MTK_PPE_PKT_TYPE_IPV6_ROUTE_3T ) entry->ipv6.ports = ports_pad; -if (type >=MTK_PPE_PKT_TYPE_IPV4_DSLITE ) { +if (type ==MTK_PPE_PKT_TYPE_BRIDGE ) { +ether_addr_copy (entry->bridge.src_mac, src_mac); +ether_addr_copy (entry->bridge.dest_mac, dest_mac); + entry->bridge.ib2 = val; + l2 = &entry->bridge.l2; + }else if (type >=MTK_PPE_PKT_TYPE_IPV4_DSLITE ) { entry->ipv6.ib2 = val; l2 = &entry->ipv6.l2; }else {
@@ -371,6 +392,84 @@ mtk_flow_entry_match(struct mtk_flow_entry *entry, struct mtk_foe_entry *data)return !memcmp (&entry->data.data, &data->data, len -4 ); } +static void +__mtk_foe_entry_clear (struct mtk_ppe *ppe,struct mtk_flow_entry *entry) +{ +struct hlist_head *head; +struct hlist_node *tmp; + +if (entry->type ==MTK_FLOW_TYPE_L2 ) { +rhashtable_remove_fast (&ppe->l2_flows, &entry->l2_node, + mtk_flow_l2_ht_params); + + head = &entry->l2_flows; +hlist_for_each_entry_safe (entry, tmp, head, l2_data.list) +__mtk_foe_entry_clear (ppe, entry); +return ; + } + +hlist_del_init (&entry->list); +if (entry->hash !=0xffff ) { + ppe->foe_table[entry->hash].ib1 &= ~MTK_FOE_IB1_STATE ; + ppe->foe_table[entry->hash].ib1 |=FIELD_PREP (MTK_FOE_IB1_STATE , +MTK_FOE_STATE_BIND ); +dma_wmb (); + } + entry->hash =0xffff ; + +if (entry->type !=MTK_FLOW_TYPE_L2_SUBFLOW ) +return ; + +hlist_del_init (&entry->l2_data.list); +kfree (entry); +} + +static int __mtk_foe_entry_idle_time (struct mtk_ppe *ppe,u32 ib1) +{ +u16 timestamp; +u16 now; + + now =mtk_eth_timestamp (ppe->eth) &MTK_FOE_IB1_BIND_TIMESTAMP ; + timestamp = ib1 &MTK_FOE_IB1_BIND_TIMESTAMP ; + +if (timestamp > now) +return MTK_FOE_IB1_BIND_TIMESTAMP +1 - timestamp + now; +else +return now - timestamp; +} + +static void +mtk_flow_entry_update_l2 (struct mtk_ppe *ppe,struct mtk_flow_entry *entry) +{ +struct mtk_flow_entry *cur; +struct mtk_foe_entry *hwe; +struct hlist_node *tmp; +int idle; + + idle =__mtk_foe_entry_idle_time (ppe, entry->data.ib1); +hlist_for_each_entry_safe (cur, tmp, &entry->l2_flows, l2_data.list) { +int cur_idle; +u32 ib1; + + hwe = &ppe->foe_table[cur->hash]; + ib1 =READ_ONCE (hwe->ib1); + +if (FIELD_GET (MTK_FOE_IB1_STATE , ib1) !=MTK_FOE_STATE_BIND ) { + cur->hash =0xffff ; +__mtk_foe_entry_clear (ppe, cur); +continue ; + } + + cur_idle =__mtk_foe_entry_idle_time (ppe, ib1); +if (cur_idle >= idle) +continue ; + + idle = cur_idle; + entry->data.ib1 &= ~MTK_FOE_IB1_BIND_TIMESTAMP ; + entry->data.ib1 |= hwe->ib1 &MTK_FOE_IB1_BIND_TIMESTAMP ; + } +} +static void mtk_flow_entry_update (struct mtk_ppe *ppe,struct mtk_flow_entry *entry) {
@@ -378,6 +477,12 @@ mtk_flow_entry_update(struct mtk_ppe *ppe, struct mtk_flow_entry *entry)struct mtk_foe_entry foe;spin_lock_bh (&ppe_lock); + +if (entry->type ==MTK_FLOW_TYPE_L2 ) { +mtk_flow_entry_update_l2 (ppe, entry); + goto out; + } +if (entry->hash ==0xffff ) goto out;
@@ -419,21 +524,28 @@ __mtk_foe_entry_commit(struct mtk_ppe *ppe, struct mtk_foe_entry *entry,void mtk_foe_entry_clear (struct mtk_ppe *ppe,struct mtk_flow_entry *entry) {spin_lock_bh (&ppe_lock); -hlist_del_init (&entry->list); -if (entry->hash !=0xffff ) { - ppe->foe_table[entry->hash].ib1 &= ~MTK_FOE_IB1_STATE ; - ppe->foe_table[entry->hash].ib1 |=FIELD_PREP (MTK_FOE_IB1_STATE , -MTK_FOE_STATE_BIND ); -dma_wmb (); - } - entry->hash =0xffff ; +__mtk_foe_entry_clear (ppe, entry);spin_unlock_bh (&ppe_lock); } +static int +mtk_foe_entry_commit_l2 (struct mtk_ppe *ppe,struct mtk_flow_entry *entry) +{ + entry->type =MTK_FLOW_TYPE_L2 ; + +return rhashtable_insert_fast (&ppe->l2_flows, &entry->l2_node, + mtk_flow_l2_ht_params); +} +int mtk_foe_entry_commit (struct mtk_ppe *ppe,struct mtk_flow_entry *entry) { -u32 hash =mtk_ppe_hash_entry (&entry->data); +int type =FIELD_GET (MTK_FOE_IB1_PACKET_TYPE , entry->data.ib1); +u32 hash; + +if (type ==MTK_PPE_PKT_TYPE_BRIDGE ) +return mtk_foe_entry_commit_l2 (ppe, entry); + hash =mtk_ppe_hash_entry (&entry->data); entry->hash =0xffff ;spin_lock_bh (&ppe_lock);hlist_add_head (&entry->list, &ppe->foe_flow[hash /2 ]);
@@ -442,18 +554,72 @@ int mtk_foe_entry_commit(struct mtk_ppe *ppe, struct mtk_flow_entry *entry)return 0 ; } +static void +mtk_foe_entry_commit_subflow (struct mtk_ppe *ppe,struct mtk_flow_entry *entry, +u16 hash) +{ +struct mtk_flow_entry *flow_info; +struct mtk_foe_entry foe, *hwe; +struct mtk_foe_mac_info *l2; +u32 ib1_mask =MTK_FOE_IB1_PACKET_TYPE |MTK_FOE_IB1_UDP ; +int type; + + flow_info = kzalloc(offsetof(struct mtk_flow_entry, l2_data.end), +GFP_ATOMIC ); +if (!flow_info) +return ; + + flow_info->l2_data.base_flow = entry; + flow_info->type =MTK_FLOW_TYPE_L2_SUBFLOW ; + flow_info->hash = hash; +hlist_add_head (&flow_info->list, &ppe->foe_flow[hash /2 ]); +hlist_add_head (&flow_info->l2_data.list, &entry->l2_flows); + + hwe = &ppe->foe_table[hash]; +memcpy (&foe, hwe,sizeof (foe)); + foe.ib1 &= ib1_mask; + foe.ib1 |= entry->data.ib1 & ~ib1_mask; + + l2 =mtk_foe_entry_l2 (&foe); +memcpy (l2, &entry->data.bridge.l2,sizeof (*l2)); + + type =FIELD_GET (MTK_FOE_IB1_PACKET_TYPE , foe.ib1); +if (type ==MTK_PPE_PKT_TYPE_IPV4_HNAPT ) +memcpy (&foe.ipv4.new, &foe.ipv4.orig,sizeof (foe.ipv4.new)); +else if (type >=MTK_PPE_PKT_TYPE_IPV6_ROUTE_3T && l2->etype ==ETH_P_IP ) + l2->etype =ETH_P_IPV6 ; + + *mtk_foe_entry_ib2 (&foe) = entry->data.bridge.ib2; + +__mtk_foe_entry_commit (ppe, &foe, hash); +} +void __mtk_ppe_check_skb (struct mtk_ppe *ppe,struct sk_buff *skb,u16 hash) {struct hlist_head *head = &ppe->foe_flow[hash /2 ]; -struct mtk_flow_entry *entry;struct mtk_foe_entry *hwe = &ppe->foe_table[hash]; +struct mtk_flow_entry *entry; +struct mtk_foe_bridge key = {}; +struct ethhdr *eh;bool found = false; - -if (hlist_empty (head)) -return ; +u8 *tag;spin_lock_bh (&ppe_lock); + +if (FIELD_GET (MTK_FOE_IB1_STATE , hwe->ib1) ==MTK_FOE_STATE_BIND ) + goto out; +hlist_for_each_entry (entry, head, list) { +if (entry->type ==MTK_FLOW_TYPE_L2_SUBFLOW ) { +if (unlikely (FIELD_GET (MTK_FOE_IB1_STATE , hwe->ib1) == +MTK_FOE_STATE_BIND )) +continue ; + + entry->hash =0xffff ; +__mtk_foe_entry_clear (ppe, entry); +continue ; + } +if (found || !mtk_flow_entry_match (entry, hwe)) {if (entry->hash !=0xffff ) entry->hash =0xffff ;
@@ -464,21 +630,50 @@ void __mtk_ppe_check_skb(struct mtk_ppe *ppe, struct sk_buff *skb, u16 hash)__mtk_foe_entry_commit (ppe, &entry->data, hash); found = true; } + +if (found) + goto out; + + eh =eth_hdr (skb); +ether_addr_copy (key.dest_mac, eh->h_dest); +ether_addr_copy (key.src_mac, eh->h_source); + tag = skb->data -2 ; + key.vlan =0 ; +switch (skb->protocol) { +#if IS_ENABLED (CONFIG_NET_DSA ) +case htons (ETH_P_XDSA ): +if (!netdev_uses_dsa (skb->dev) || + skb->dev->dsa_ptr->tag_ops->proto !=DSA_TAG_PROTO_MTK ) + goto out; + + tag +=4 ; +if (get_unaligned_be16 (tag) !=ETH_P_8021Q ) +break ; + + fallthrough; +#endif +case htons (ETH_P_8021Q ): + key.vlan =get_unaligned_be16 (tag +2 ) &VLAN_VID_MASK ; +break ; +default : +break ; + } + + entry =rhashtable_lookup_fast (&ppe->l2_flows, &key, mtk_flow_l2_ht_params); +if (!entry) + goto out; + +mtk_foe_entry_commit_subflow (ppe, entry, hash); + +out:spin_unlock_bh (&ppe_lock); }int mtk_foe_entry_idle_time (struct mtk_ppe *ppe,struct mtk_flow_entry *entry) { -u16 now =mtk_eth_timestamp (ppe->eth) &MTK_FOE_IB1_BIND_TIMESTAMP ; -u16 timestamp; -mtk_flow_entry_update (ppe, entry); - timestamp = entry->data.ib1 &MTK_FOE_IB1_BIND_TIMESTAMP ; -if (timestamp > now) -return MTK_FOE_IB1_BIND_TIMESTAMP +1 - timestamp + now; -else -return now - timestamp; +return __mtk_foe_entry_idle_time (ppe, entry->data.ib1); } struct mtk_ppe *mtk_ppe_init(struct mtk_eth *eth, void __iomem *base,
@@ -492,6 +687,8 @@ struct mtk_ppe *mtk_ppe_init(struct mtk_eth *eth, void __iomem *base,if (!ppe)return NULL ; +rhashtable_init (&ppe->l2_flows, &mtk_flow_l2_ht_params); +/* need to allocate a separate device, since it PPE DMA access is * not coherent. */
drivers/net/ethernet/mediatek/mtk_ppe.h +32 -12
76 lines not on this page · raw message
drivers/net/ethernet/mediatek/mtk_ppe_offload.c +48 -12
102 lines not on this page · raw message
Signature · 7 lines
2.35.1 _______________________________________________ Linux-mediatek mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-mediatek
Re: [PATCH v2 14/14] net: ethernet: mtk_eth_soc: support creating mac address based offload entries
Quoted 32 lines · Felix Fietkau
This will be used to implement a limited form of bridge offloading. Since the hardware does not support flow table entries with just source and destination MAC address, the driver has to emulate it.
The hardware automatically creates entries entries for incoming flows, even when they are bridged instead of routed, and reports when packets for these flows have reached the minimum PPS rate for offloading.
After this happens, we look up the L2 flow offload entry based on the MAC header and fill in the output routing information in the flow table. The dynamically created per-flow entries are automatically removed when either the hardware flowtable entry expires, is replaced, or if the offload rule they belong to is removed
drivers/net/ethernet/mediatek/mtk_ppe.c +18
-0+ +if (found) + goto out; + + eh =eth_hdr (skb); +ether_addr_copy (key.dest_mac, eh->h_dest); +ether_addr_copy (key.src_mac, eh->h_source); + tag = skb->data -2 ; + key.vlan =0 ; +switch (skb->protocol) { +#if IS_ENABLED (CONFIG_NET_DSA ) +case htons (ETH_P_XDSA ): +if (!netdev_uses_dsa (skb->dev) || + skb->dev->dsa_ptr->tag_ops->proto !=DSA_TAG_PROTO_MTK ) + goto out; + + tag +=4 ; +if (get_unaligned_be16 (tag) !=ETH_P_8021Q )
drivers/net/ethernet/mediatek/mtk_ppe.c void __mtk_ppe_check_skb(struct mtk_ppe *ppe, struct sk_buff *skb, u16 hash) · line 660
Quoted 10 lines
drivers/net/ethernet/mediatek/mtk_ppe.c +10
-0+break ; + + fallthrough; +#endif +case htons (ETH_P_8021Q ): + key.vlan =get_unaligned_be16 (tag +2 ) &VLAN_VID_MASK ; +break ; +default : +break ; + }
I'm trying to understand the architecture here.
We have an Ethernet interface and a Wireless interface. The slow path is that frames ingress from one of these interfaces, Linux decides what to do with them, either L2 or L3, and they then egress probably out the other interface.
The hardware will look at the frames and try to spot flows? It will then report any it finds. You can then add an offload, telling it for a flow it needs to perform L2 or L3 processing, and egress out a specific port? Linux then no longer sees the frame, the hardware handles it, until the flow times out?
So i'm wondering what is going on here. So is this a frame which has ingressed, either from the WiFi, or another switch port, gone to the software bridge, bridges to a DSA slave interface, the DSA tagger has added a tag and now it is in the master interface? Can you accelerate such frames? What is adding the DSA tag on the fast path? And in the opposite direction, frames which egress the switch which have a DSA tag and are heading to the WiFi, what is removing the tag? Does the accelerator also understand the tag and know what to do with it?
Andrew
Felix Fietkau
maintainerRe: [PATCH v2 14/14] net: ethernet: mtk_eth_soc: support creating mac address based offload entries
Quoted 51 lines · Andrew Lunn
Quoted 32 lines · Felix Fietkau
This will be used to implement a limited form of bridge offloading. Since the hardware does not support flow table entries with just source and destination MAC address, the driver has to emulate it.
The hardware automatically creates entries entries for incoming flows, even when they are bridged instead of routed, and reports when packets for these flows have reached the minimum PPS rate for offloading.
After this happens, we look up the L2 flow offload entry based on the MAC header and fill in the output routing information in the flow table. The dynamically created per-flow entries are automatically removed when either the hardware flowtable entry expires, is replaced, or if the offload rule they belong to is removed
drivers/net/ethernet/mediatek/mtk_ppe.c +18
-0+ +if (found) + goto out; + + eh =eth_hdr (skb); +ether_addr_copy (key.dest_mac, eh->h_dest); +ether_addr_copy (key.src_mac, eh->h_source); + tag = skb->data -2 ; + key.vlan =0 ; +switch (skb->protocol) { +#if IS_ENABLED (CONFIG_NET_DSA ) +case htons (ETH_P_XDSA ): +if (!netdev_uses_dsa (skb->dev) || + skb->dev->dsa_ptr->tag_ops->proto !=DSA_TAG_PROTO_MTK ) + goto out; + + tag +=4 ; +if (get_unaligned_be16 (tag) !=ETH_P_8021Q )drivers/net/ethernet/mediatek/mtk_ppe.c void __mtk_ppe_check_skb(struct mtk_ppe *ppe, struct sk_buff *skb, u16 hash) · line 660
Quoted 10 lines
drivers/net/ethernet/mediatek/mtk_ppe.c +10
-0+break ; + + fallthrough; +#endif +case htons (ETH_P_8021Q ): + key.vlan =get_unaligned_be16 (tag +2 ) &VLAN_VID_MASK ; +break ; +default : +break ; + }I'm trying to understand the architecture here.
We have an Ethernet interface and a Wireless interface. The slow path is that frames ingress from one of these interfaces, Linux decides what to do with them, either L2 or L3, and they then egress probably out the other interface.
Quoted 5 lines
The hardware will look at the frames and try to spot flows? It will then report any it finds. You can then add an offload, telling it for a flow it needs to perform L2 or L3 processing, and egress out a specific port? Linux then no longer sees the frame, the hardware handles it, until the flow times out?
Yes, the hw handles it until either the flow times out, or the corresponding offload entry is removed.
For OpenWrt I also wrote a daemon that uses tc classifier BPF to accelerate the software bridge and create hardware offload entries as well via hardware TC flower rules: https://github.com/nbd168/bridger
It works in combination with these changes.
Quoted 8 lines
So i'm wondering what is going on here. So is this a frame which has
ingressed, either from the WiFi, or another switch port, gone to the
software bridge, bridges to a DSA slave interface, the DSA tagger has
added a tag and now it is in the master interface? Can you accelerate
such frames? What is adding the DSA tag on the fast path? And in the
opposite direction, frames which egress the switch which have a DSA
tag and are heading to the WiFi, what is removing the tag? Does the
accelerator also understand the tag and know what to do with it?WiFi -> Ethernet is not supported by MT7622, but will be added for newer
SoCs like MT7986. The PPE supports both parsing and inserting MT7530 compatible DSA tags. For Ethernet->WiFi flows, the PPE will also add required metadata that is parsed by the MT7915 WiFi Firmware in order to figure out what vif/station the packets were meant for.
- Felix
Re: [PATCH v2 14/14] net: ethernet: mtk_eth_soc: support creating mac address based offload entries
Quoted 58 lines · Felix Fietkau
Quoted 51 lines · Andrew Lunn
Quoted 32 lines · Felix Fietkau
This will be used to implement a limited form of bridge offloading. Since the hardware does not support flow table entries with just source and destination MAC address, the driver has to emulate it.
The hardware automatically creates entries entries for incoming flows, even when they are bridged instead of routed, and reports when packets for these flows have reached the minimum PPS rate for offloading.
After this happens, we look up the L2 flow offload entry based on the MAC header and fill in the output routing information in the flow table. The dynamically created per-flow entries are automatically removed when either the hardware flowtable entry expires, is replaced, or if the offload rule they belong to is removed
drivers/net/ethernet/mediatek/mtk_ppe.c +18
-0+ +if (found) + goto out; + + eh =eth_hdr (skb); +ether_addr_copy (key.dest_mac, eh->h_dest); +ether_addr_copy (key.src_mac, eh->h_source); + tag = skb->data -2 ; + key.vlan =0 ; +switch (skb->protocol) { +#if IS_ENABLED (CONFIG_NET_DSA ) +case htons (ETH_P_XDSA ): +if (!netdev_uses_dsa (skb->dev) || + skb->dev->dsa_ptr->tag_ops->proto !=DSA_TAG_PROTO_MTK ) + goto out; + + tag +=4 ; +if (get_unaligned_be16 (tag) !=ETH_P_8021Q )drivers/net/ethernet/mediatek/mtk_ppe.c void __mtk_ppe_check_skb(struct mtk_ppe *ppe, struct sk_buff *skb, u16 hash) · line 660
Quoted 10 lines
drivers/net/ethernet/mediatek/mtk_ppe.c +10
-0+break ; + + fallthrough; +#endif +case htons (ETH_P_8021Q ): + key.vlan =get_unaligned_be16 (tag +2 ) &VLAN_VID_MASK ; +break ; +default : +break ; + }I'm trying to understand the architecture here.
We have an Ethernet interface and a Wireless interface. The slow path is that frames ingress from one of these interfaces, Linux decides what to do with them, either L2 or L3, and they then egress probably out the other interface.
Quoted 5 lines
The hardware will look at the frames and try to spot flows? It will then report any it finds. You can then add an offload, telling it for a flow it needs to perform L2 or L3 processing, and egress out a specific port? Linux then no longer sees the frame, the hardware handles it, until the flow times out?
Quoted 7 lines
Yes, the hw handles it until either the flow times out, or the corresponding offload entry is removed.
For OpenWrt I also wrote a daemon that uses tc classifier BPF to accelerate the software bridge and create hardware offload entries as well via hardware TC flower rules: https://github.com/nbd168/bridger
It works in combination with these changes.
What about the bridge? In Linux, it is the software bridge which controls all this at L2, and it should be offloading the flows, via switchdev. The egress port you derive here is from the software bridge FDB?
Quoted 10 lines
Quoted 9 lines
So i'm wondering what is going on here. So is this a frame which has ingressed, either from the WiFi, or another switch port, gone to the software bridge, bridges to a DSA slave interface, the DSA tagger has added a tag and now it is in the master interface? Can you accelerate such frames? What is adding the DSA tag on the fast path? And in the opposite direction, frames which egress the switch which have a DSA tag and are heading to the WiFi, what is removing the tag? Does the accelerator also understand the tag and know what to do with it?WiFi -> Ethernet is not supported by MT7622, but will be added for newer
Quoted 4 lines
SoCs like MT7986. The PPE supports both parsing and inserting MT7530 compatible DSA tags. For
Ethernet->WiFiflows, the PPE will also add required metadata that is parsed by the MT7915 WiFi Firmware in order to figure out what vif/station the packets were meant for.
O.K. What about IGMP and multicast? Does the accelerate match on IGMP and forwards it to the CPU, rather than follow the flow rules? Can you set multiple egress destinations for multicast so that it can go both to the switch and the host, when the host has a local interest in the traffic?
Andrew
Felix Fietkau
maintainerRe: [PATCH v2 14/14] net: ethernet: mtk_eth_soc: support creating mac address based offload entries
Quoted 67 lines · Andrew Lunn
Quoted 58 lines · Felix Fietkau
Quoted 51 lines · Andrew Lunn
Quoted 32 lines · Felix Fietkau
This will be used to implement a limited form of bridge offloading. Since the hardware does not support flow table entries with just source and destination MAC address, the driver has to emulate it.
The hardware automatically creates entries entries for incoming flows, even when they are bridged instead of routed, and reports when packets for these flows have reached the minimum PPS rate for offloading.
After this happens, we look up the L2 flow offload entry based on the MAC header and fill in the output routing information in the flow table. The dynamically created per-flow entries are automatically removed when either the hardware flowtable entry expires, is replaced, or if the offload rule they belong to is removed
drivers/net/ethernet/mediatek/mtk_ppe.c +18
-0+ +if (found) + goto out; + + eh =eth_hdr (skb); +ether_addr_copy (key.dest_mac, eh->h_dest); +ether_addr_copy (key.src_mac, eh->h_source); + tag = skb->data -2 ; + key.vlan =0 ; +switch (skb->protocol) { +#if IS_ENABLED (CONFIG_NET_DSA ) +case htons (ETH_P_XDSA ): +if (!netdev_uses_dsa (skb->dev) || + skb->dev->dsa_ptr->tag_ops->proto !=DSA_TAG_PROTO_MTK ) + goto out; + + tag +=4 ; +if (get_unaligned_be16 (tag) !=ETH_P_8021Q )drivers/net/ethernet/mediatek/mtk_ppe.c void __mtk_ppe_check_skb(struct mtk_ppe *ppe, struct sk_buff *skb, u16 hash) · line 660
Quoted 10 lines
drivers/net/ethernet/mediatek/mtk_ppe.c +10
-0+break ; + + fallthrough; +#endif +case htons (ETH_P_8021Q ): + key.vlan =get_unaligned_be16 (tag +2 ) &VLAN_VID_MASK ; +break ; +default : +break ; + }I'm trying to understand the architecture here.
We have an Ethernet interface and a Wireless interface. The slow path is that frames ingress from one of these interfaces, Linux decides what to do with them, either L2 or L3, and they then egress probably out the other interface.
Quoted 5 lines
The hardware will look at the frames and try to spot flows? It will then report any it finds. You can then add an offload, telling it for a flow it needs to perform L2 or L3 processing, and egress out a specific port? Linux then no longer sees the frame, the hardware handles it, until the flow times out?
Quoted 7 lines
Yes, the hw handles it until either the flow times out, or the corresponding offload entry is removed.
For OpenWrt I also wrote a daemon that uses tc classifier BPF to accelerate the software bridge and create hardware offload entries as well via hardware TC flower rules: https://github.com/nbd168/bridger
It works in combination with these changes.
Quoted 4 lines
What about the bridge? In Linux, it is the software bridge which controls all this at L2, and it should be offloading the flows, via switchdev. The egress port you derive here is from the software bridge FDB?
My code uses netlink to fetch and monitor the bridge configuration, including fdb, port state, vlans, etc. and it uses that for the offload path - no extra configuration needed.
Quoted 15 lines
Quoted 10 lines
Quoted 9 lines
So i'm wondering what is going on here. So is this a frame which has ingressed, either from the WiFi, or another switch port, gone to the software bridge, bridges to a DSA slave interface, the DSA tagger has added a tag and now it is in the master interface? Can you accelerate such frames? What is adding the DSA tag on the fast path? And in the opposite direction, frames which egress the switch which have a DSA tag and are heading to the WiFi, what is removing the tag? Does the accelerator also understand the tag and know what to do with it?WiFi -> Ethernet is not supported by MT7622, but will be added for newer
Quoted 4 lines
SoCs like MT7986. The PPE supports both parsing and inserting MT7530 compatible DSA tags. For
Ethernet->WiFiflows, the PPE will also add required metadata that is parsed by the MT7915 WiFi Firmware in order to figure out what vif/station the packets were meant for.
Quoted 5 lines
O.K. What about IGMP and multicast? Does the accelerate match on IGMP and forwards it to the CPU, rather than follow the flow rules? Can you set multiple egress destinations for multicast so that it can go both to the switch and the host, when the host has a local interest in the traffic?
IGMP/multicast isn't handled yet at the moment. I still need to do some research on what can be offloaded and how. The offload only handles unicast and everything else is going through the CPU.
- Felix
_______________________________________________
Linux-mediatek mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-mediatek
Re: [PATCH v2 14/14] net: ethernet: mtk_eth_soc: support creating mac address based offload entries
Quoted 20 lines
Quoted 20 lines
Quoted 19 lines
Quoted 2 lines
Quoted 2 lines
I'm trying to understand the architecture here.
We have an Ethernet interface and a Wireless interface. The slowpath
Quoted 4 lines
is that frames ingress from one of these interfaces, Linux decides what to do with them, either L2 or L3, and they then egress probably out the other interface.
Quoted 1 line
The hardware will look at the frames and try to spot flows? It
will
Quoted 4 lines
then report any it finds. You can then add an offload, telling it for a flow it needs to perform L2 or L3 processing, and egress out a specific port? Linux then no longer sees the frame, the hardware handles it, until the flow times out?
Yes, the hw handles it until either the flow times out, or the corresponding offload entry is removed.
For OpenWrt I also wrote a daemon that uses tc classifier BPF to accelerate the software bridge and create hardware offload entries as well via hardware TC flower rules: https://github.com/nbd168/bridger
It works in combination with these changes.
Quoted 8 lines
Quoted 4 lines
What about the bridge? In Linux, it is the software bridge which controls all this at L2, and it should be offloading the flows, via switchdev. The egress port you derive here is from the software bridge FDB?
My code uses netlink to fetch and monitor the bridge configuration, including fdb, port state, vlans, etc. and it uses that for the offload path
- no extra configuration needed.
So this is where we get into architecture issues. Do we really want Linux to have two ways for setting up L2 networking? It was decided that users should not need to know about how to use an accelerator, they should not use additional tools, it should just look like linux. The user should just add the WiFi netdev to the bridge and switchdev will do the rest to offload L2 switching to the hardware.
You appear to be saying you need a daemon in userspace. That is not how every other accelerate works in Linux networking.
We the Linux network community need to decided if we want this?
Andrew
_______________________________________________
Linux-mediatek mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-mediatek
Felix Fietkau
maintainerRe: [PATCH v2 14/14] net: ethernet: mtk_eth_soc: support creating mac address based offload entries
Quoted 36 lines · Andrew Lunn
Quoted 20 lines
Quoted 20 lines
Quoted 19 lines
Quoted 2 lines
Quoted 2 lines
I'm trying to understand the architecture here.
We have an Ethernet interface and a Wireless interface. The slowpath
Quoted 4 lines
is that frames ingress from one of these interfaces, Linux decides what to do with them, either L2 or L3, and they then egress probably out the other interface.
Quoted 1 line
The hardware will look at the frames and try to spot flows? It
will
Quoted 4 lines
then report any it finds. You can then add an offload, telling it for a flow it needs to perform L2 or L3 processing, and egress out a specific port? Linux then no longer sees the frame, the hardware handles it, until the flow times out?
Yes, the hw handles it until either the flow times out, or the corresponding offload entry is removed.
For OpenWrt I also wrote a daemon that uses tc classifier BPF to accelerate the software bridge and create hardware offload entries as well via hardware TC flower rules: https://github.com/nbd168/bridger
It works in combination with these changes.Quoted 8 lines
Quoted 4 lines
What about the bridge? In Linux, it is the software bridge which controls all this at L2, and it should be offloading the flows, via switchdev. The egress port you derive here is from the software bridge FDB?
My code uses netlink to fetch and monitor the bridge configuration, including fdb, port state, vlans, etc. and it uses that for the offload path
- no extra configuration needed.So this is where we get into architecture issues. Do we really want Linux to have two ways for setting up L2 networking? It was decided that users should not need to know about how to use an accelerator, they should not use additional tools, it should just look like linux. The user should just add the WiFi netdev to the bridge and switchdev will do the rest to offload L2 switching to the hardware.
Quoted 4 lines
You appear to be saying you need a daemon in userspace. That is not how every other accelerate works in Linux networking.
We the Linux network community need to decided if we want this?
The problem here is that it can't be fully transparent. Enabling hardware offload for LAN -> WiFi comes at a cost of bypassing airtime fairness and mac80211's bufferbloat mitigation. Some people want this anyway (often but not always for benchmark/marketing purposes), but it's not something that I would want to have enabled by default simply by a wifi netdev to a bridge.
Initially, I wanted to put more of the state tracking code in the kernel. I made the first implementation of my acceleration code as a patch to the network bridge - speeding up bridge unicast forwarding significantly for any device regardless of hardware support. I wanted to build on that to avoid putting a lot of FDB/VLAN related tracking directly into the driver.
That approach was immediately rejected and I was told to use BPF instead.
That said, I really don't think it's a good idea to put all the code for tracking the bridge state, and all possible forwarding destinations into the driver directly.
I believe the combination of doing the bridge state tracking in user space + using the standard TC API for programming offloading rules into the hardware is a reasonable compromise.
- Felix
Re: [PATCH v2 14/14] net: ethernet: mtk_eth_soc: support creating mac address based offload entries
Quoted 40 lines · Felix Fietkau
Quoted 38 lines · Andrew Lunn
Quoted 20 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
I'm trying to understand the architecture here.
We have an Ethernet interface and a Wireless interface. The slowpath
Quoted 4 lines
is that frames ingress from one of these interfaces, Linux decides what to do with them, either L2 or L3, and they then egress probably out the other interface.
Quoted 1 line
The hardware will look at the frames and try to spot flows? It
Quoted 8 lines
Quoted 8 lines
will
Quoted 4 lines
then report any it finds. You can then add an offload, telling it for a flow it needs to perform L2 or L3 processing, and egress out a specific port? Linux then no longer sees the frame, the hardware handles it, until the flow times out?
Yes, the hw handles it until either the flow times out, or the corresponding offload entry is removed.
Quoted 1 line
Quoted 1 line
For OpenWrt I also wrote a daemon that uses tc classifier
BPF to accelerate
Quoted 4 lines
Quoted 4 lines
the software bridge and create hardware offload entries as well via hardware TC flower rules: https://github.com/nbd168/bridger
It works in combination with these changes.
What about the bridge? In Linux, it is the software bridge whichQuoted 7 lines
Quoted 3 lines
controls all this at L2, and it should be offloading the flows, via switchdev. The egress port you derive here is from the software bridge FDB?
My code uses netlink to fetch and monitor the bridge configuration, including fdb, port state, vlans, etc. and it uses that for the offload path
- no extra configuration needed.So this is where we get into architecture issues. Do we really want Linux to have two ways for setting up L2 networking? It was decided that users should not need to know about how to use an accelerator, they should not use additional tools, it should just look like linux. The user should just add the WiFi netdev to the bridge and switchdev will do the rest to offload L2 switching to the hardware.
You appear to be saying you need a daemon in userspace. That is not how every other accelerate works in Linux networking.
Quoted 8 lines
Quoted 1 line
We the Linux network community need to decided if we want this?
The problem here is that it can't be fully transparent. Enabling hardware offload for LAN -> WiFi comes at a cost of bypassing airtime fairness and mac80211's bufferbloat mitigation.
Some people want this anyway (often but not always for benchmark/marketing purposes), but it's not something that I would want to have enabled by default simply by a wifi netdev to a bridge.
So this sounds like a generic issue. How does IPA handle this? Looping in Alex Elder.
There is already something partially in this direction in the bridge. You can add a static entry with our without self. This controls if a specific static entry in the FDB is offloaded to the accelerate or not. Maybe you can add an attribute to a port which determines if dynamic entries are self or not, so you can decide if frames egressing out a specific interface are accelerated or not, depending on user choice. Since such a change should not touch the fast path, it has a better chance of being merged.
Quoted 5 lines
Initially, I wanted to put more of the state tracking code in the kernel. I made the first implementation of my acceleration code as a patch to the network bridge - speeding up bridge unicast forwarding significantly for any device regardless of hardware support. I wanted to build on that to avoid putting a lot of FDB/VLAN related tracking directly into the driver.
But the driver is the correct place for this. How generic is the state tracking? Do you expect any other hardware to need the same state tracking? IPA? Some other accelerate your know of?
Andrew
Felix Fietkau
maintainerRe: [PATCH v2 14/14] net: ethernet: mtk_eth_soc: support creating mac address based offload entries
Quoted 53 lines · Andrew Lunn
Quoted 40 lines · Felix Fietkau
Quoted 38 lines · Andrew Lunn
Quoted 20 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
I'm trying to understand the architecture here.
We have an Ethernet interface and a Wireless interface. The slowpath
Quoted 4 lines
is that frames ingress from one of these interfaces, Linux decides what to do with them, either L2 or L3, and they then egress probably out the other interface.
Quoted 1 line
The hardware will look at the frames and try to spot flows? It
Quoted 8 lines
Quoted 8 lines
will
Quoted 4 lines
then report any it finds. You can then add an offload, telling it for a flow it needs to perform L2 or L3 processing, and egress out a specific port? Linux then no longer sees the frame, the hardware handles it, until the flow times out?
Yes, the hw handles it until either the flow times out, or the corresponding offload entry is removed.
Quoted 1 line
Quoted 1 line
For OpenWrt I also wrote a daemon that uses tc classifier
BPF to accelerate
Quoted 4 lines
Quoted 4 lines
the software bridge and create hardware offload entries as well via hardware TC flower rules: https://github.com/nbd168/bridger
It works in combination with these changes.
What about the bridge? In Linux, it is the software bridge whichQuoted 7 lines
Quoted 3 lines
controls all this at L2, and it should be offloading the flows, via switchdev. The egress port you derive here is from the software bridge FDB?
My code uses netlink to fetch and monitor the bridge configuration, including fdb, port state, vlans, etc. and it uses that for the offload path
- no extra configuration needed.So this is where we get into architecture issues. Do we really want Linux to have two ways for setting up L2 networking? It was decided that users should not need to know about how to use an accelerator, they should not use additional tools, it should just look like linux. The user should just add the WiFi netdev to the bridge and switchdev will do the rest to offload L2 switching to the hardware.
You appear to be saying you need a daemon in userspace. That is not how every other accelerate works in Linux networking.
Quoted 8 lines
Quoted 1 line
We the Linux network community need to decided if we want this?
The problem here is that it can't be fully transparent. Enabling hardware offload for LAN -> WiFi comes at a cost of bypassing airtime fairness and mac80211's bufferbloat mitigation.
Some people want this anyway (often but not always for benchmark/marketing purposes), but it's not something that I would want to have enabled by default simply by a wifi netdev to a bridge.So this sounds like a generic issue. How does IPA handle this? Looping in Alex Elder.
Quoted 8 lines
There is already something partially in this direction in the bridge. You can add a static entry with our without self. This controls if a specific static entry in the FDB is offloaded to the accelerate or not. Maybe you can add an attribute to a port which determines if dynamic entries are self or not, so you can decide if frames egressing out a specific interface are accelerated or not, depending on user choice. Since such a change should not touch the fast path, it has a better chance of being merged.
Sounds interesting. If there is some overlap and if we can get some common code in place, it might be worth looking into for the MTK offload as well. I do think this will take more time though.
Quoted 9 lines
Quoted 5 lines
Initially, I wanted to put more of the state tracking code in the kernel. I made the first implementation of my acceleration code as a patch to the network bridge - speeding up bridge unicast forwarding significantly for any device regardless of hardware support. I wanted to build on that to avoid putting a lot of FDB/VLAN related tracking directly into the driver.
But the driver is the correct place for this. How generic is the state tracking? Do you expect any other hardware to need the same state tracking? IPA? Some other accelerate your know of?
It basically has to keep track of all possible destination ports, their STP state, all their fdb entries, member VLANs of all ports. It has to quickly react to changes in any of these.
In order to implement this properly, I would also need to make more changes to mac80211. Right now, mac80211 drivers do not have access to the net_device pointer of virtual interfaces. So mac80211 itself would likely need to implement the switchdev ops and handle some of this.
There are also some other issues where I don't know how this is supposed to be solved properly:
On MT7622 most of the bridge ports are connected to a MT7531 switch using DSA. Offloading (lan->wlan bridging or L3/L4 NAT/routing) is not handled by the switch itself, it is handled by a packet processing engine in the SoC, which knows how to handle the DSA tags of the MT7531 switch.
So if I were to handle this through switchdev implemented on the wlan and ethernet devices, it would technically not be part of the same switch, since it's a behind a different component with a different driver.
Also, is switchdev able to handle the situation where only parts of the traffic is offloaded and the rest (e.g. multicast) is handled through the regular software path?
In my opinion, handling it through the TC offload has a number of advantages:
- It's a lot simpler
- It uses the same kind of offloading rules that my software fastpath already uses
- It allows more fine grained control over which traffic should be offloaded (src mac -> destination MAC tuple)
I also plan on extending my software fast path code to support emulating bridging of WiFi client mode interfaces. This involves doing some MAC address translation with some IP address tracking. I want that to support hardware offload as well.
I really don't think that desire for supporting switchdev based offload should be a blocker for accepting this code now, especially since my implementation relies on existing Linux network APIs without inventing any new ones, and there are valid use cases for using it, even with switchdev support in place.
- Felix
Re: [PATCH v2 14/14] net: ethernet: mtk_eth_soc: support creating mac address based offload entries
Quoted 3 lines
It basically has to keep track of all possible destination ports, their STP state, all their fdb entries, member VLANs of all ports. It has to quickly react to changes in any of these.
switchdev gives you all of those i think. DSA does not make use of them all, in particularly the fdb entries, because of the low bandwidth management link to the switch. But look at the Mellanox switch, it keeps its hardware fdb entries in sync with the software fdb.
And you get every quick access to these, sometimes too quick in that it is holding a spinlock when it calls the switchdev functions, and you need to defer the handling in your driver if you want to use a mutex, perform blocking IO etc.
Quoted 4 lines
In order to implement this properly, I would also need to make more changes to mac80211. Right now, mac80211 drivers do not have access to the
net_devicepointer of virtual interfaces. So mac80211 itself would likely need to implement the switchdev ops and handle some of this.
So this again sounds like something which would be shared by IPA, and any other hardware which can accelerate forwarding between WiFi and some other sort of interface.
Quoted 10 lines
There are also some other issues where I don't know how this is supposed to be solved properly:
On MT7622 most of the bridge ports are connected to a MT7531 switch using DSA. Offloading (lan->wlanbridging or L3/L4 NAT/routing) is not handled by the switch itself, it is handled by a packet processing engine in the SoC, which knows how to handle the DSA tags of the MT7531 switch.So if I were to handle this through switchdev implemented on the wlan and ethernet devices, it would technically not be part of the same switch, since it's a behind a different component with a different driver.
What is important here is the user experience. The user is not expected to know there is an accelerate being used. You setup the bridge just as normal, using iproute2. You add routes in the normal way, either by iproute2, or frr can add routes from OSPF, BGP, RIP or whatever, via zebra. I'm not sure anybody has yet accelerated NAT, but the same principle should be used, using iptables in the normal way, and the accelerate is then informed and should accelerate it if possible.
switchdev gives you notification of when anything changes. You can have multiple receivers of these notifications, so the packet processor can act on them as well as the DSA switch.
Quoted 3 lines
Also, is switchdev able to handle the situation where only parts of the traffic is offloaded and the rest (e.g. multicast) is handled through the regular software path?
Yes, that is not a problem. I deliberately use the term accelerator. We accelerate what Linux can already do. If the accelerator hardware is not capable of something, Linux still is, so just pass it the frames and it will do the right thing. Multicast is a good example of this, many of the DSA switch drivers don't accelerate it.
Quoted 13 lines
In my opinion, handling it through the TC offload has a number of advantages:
- It's a lot simpler
- It uses the same kind of offloading rules that my software fastpath already uses
- It allows more fine grained control over which traffic should be offloaded (src mac -> destination MAC tuple)I also plan on extending my software fast path code to support emulating bridging of WiFi client mode interfaces. This involves doing some MAC address translation with some IP address tracking. I want that to support hardware offload as well.
Quoted 5 lines
I really don't think that desire for supporting switchdev based offload should be a blocker for accepting this code now, especially since my implementation relies on existing Linux network APIs without inventing any new ones, and there are valid use cases for using it, even with switchdev support in place.
What we need to avoid is fragmentation of the way we do things. It has been decided that switchdev is how we use accelerators, and the user should not really know anything about the accelerator. No other in kernel network accelerator needs a user space component listening to netlink notifications and programming the accelerator from user space. Do we really want two ways to do this?
Andrew
Felix Fietkau
maintainerRe: [PATCH v2 14/14] net: ethernet: mtk_eth_soc: support creating mac address based offload entries
Quoted 15 lines · Andrew Lunn
Quoted 3 lines
It basically has to keep track of all possible destination ports, their STP state, all their fdb entries, member VLANs of all ports. It has to quickly react to changes in any of these.
switchdev gives you all of those i think. DSA does not make use of them all, in particularly the fdb entries, because of the low bandwidth management link to the switch. But look at the Mellanox switch, it keeps its hardware fdb entries in sync with the software fdb.
And you get every quick access to these, sometimes too quick in that it is holding a spinlock when it calls the switchdev functions, and you need to defer the handling in your driver if you want to use a mutex, perform blocking IO etc.
Quoted 8 lines
Quoted 4 lines
In order to implement this properly, I would also need to make more changes to mac80211. Right now, mac80211 drivers do not have access to the
net_devicepointer of virtual interfaces. So mac80211 itself would likely need to implement the switchdev ops and handle some of this.So this again sounds like something which would be shared by IPA, and any other hardware which can accelerate forwarding between WiFi and some other sort of interface.
I would really like to see an example of how this should be done.
Is there a work in progress tree for IPA with offloading? Because the code that I see upstream doesn't seem to have any of that - or did I look in the wrong place?
Quoted 11 lines
Quoted 10 lines
There are also some other issues where I don't know how this is supposed to be solved properly:
On MT7622 most of the bridge ports are connected to a MT7531 switch using DSA. Offloading (lan->wlanbridging or L3/L4 NAT/routing) is not handled by the switch itself, it is handled by a packet processing engine in the SoC, which knows how to handle the DSA tags of the MT7531 switch.So if I were to handle this through switchdev implemented on the wlan and ethernet devices, it would technically not be part of the same switch, since it's a behind a different component with a different driver.
Quoted 8 lines
What is important here is the user experience. The user is not expected to know there is an accelerate being used. You setup the bridge just as normal, using iproute2. You add routes in the normal way, either by iproute2, or frr can add routes from OSPF, BGP, RIP or whatever, via zebra. I'm not sure anybody has yet accelerated NAT, but the same principle should be used, using iptables in the normal way, and the accelerate is then informed and should accelerate it if possible.
Accelerated NAT on MT7622 is already present in the upstream code for a while. It's there for ethernet, and with my patches it also works for ethernet -> wlan.
Quoted 8 lines
switchdev gives you notification of when anything changes. You can have multiple receivers of these notifications, so the packet processor can act on them as well as the DSA switch.
Quoted 3 lines
Also, is switchdev able to handle the situation where only parts of the traffic is offloaded and the rest (e.g. multicast) is handled through the regular software path?
Quoted 6 lines
Yes, that is not a problem. I deliberately use the term accelerator. We accelerate what Linux can already do. If the accelerator hardware is not capable of something, Linux still is, so just pass it the frames and it will do the right thing. Multicast is a good example of this, many of the DSA switch drivers don't accelerate it.
Don't get me wrong, I'm not against switchdev support at all. I just don't know how to do it yet, and the code that I put in place is useful for non-switchdev use cases as well.
Quoted 19 lines
Quoted 13 lines
In my opinion, handling it through the TC offload has a number of advantages:
- It's a lot simpler
- It uses the same kind of offloading rules that my software fastpath already uses
- It allows more fine grained control over which traffic should be offloaded (src mac -> destination MAC tuple)I also plan on extending my software fast path code to support emulating bridging of WiFi client mode interfaces. This involves doing some MAC address translation with some IP address tracking. I want that to support hardware offload as well.
Quoted 5 lines
I really don't think that desire for supporting switchdev based offload should be a blocker for accepting this code now, especially since my implementation relies on existing Linux network APIs without inventing any new ones, and there are valid use cases for using it, even with switchdev support in place.
Quoted 6 lines
What we need to avoid is fragmentation of the way we do things. It has been decided that switchdev is how we use accelerators, and the user should not really know anything about the accelerator. No other in kernel network accelerator needs a user space component listening to netlink notifications and programming the accelerator from user space. Do we really want two ways to do this?
There's always some overlap in what the APIs can do. And when it comes to the "client mode bridge" use case that I mentioned, I would also need exactly the same API that I put in place here. And this is not something that can (or even should) be done using switchdev. mac80211 prevents adding client mode interfaces to bridges for a reason.
- Felix
This series was applied to netdev/net-next.git (master) by David S. Miller <[email protected]>:
Hello:
This series was applied to netdev/net-next.git (master) by David S. Miller <[email protected]>:
Quoted 14 lines · On Tue, 5 Apr 2022 21:57:41 +0200 you wrote:
This series contains the following improvements to mediatek ethernet flow offload support:
- support dma-coherent on ethernet to improve performance
- add ipv6 offload support
- rework hardware flow table entry handling to improve dealing with hash collisions and competing flows
- support creating offload entries from user space
- support creating offload entries with just source/destination mac address, vlan and output device information
- add driver changes for supporting the Wireless Ethernet Dispatch core, which can be used to offload flows from ethernet to MT7915 PCIe WLAN devices
Quoted 1 line
[...]
Here is the summary with links:
- [v2,01/14] dt-bindings: net: mediatek: add optional properties for the SoC ethernet core
https://git.kernel.org/netdev/net-next/c/1dafd0d60703 - [v2,02/14] net: ethernet: mtk_eth_soc: add support for coherent DMA https://git.kernel.org/netdev/net-next/c/d776a57e4a28 - [v2,03/14] arm64: dts: mediatek: mt7622: add support for coherent DMA https://git.kernel.org/netdev/net-next/c/3abd063019b6 - [v2,04/14] dt-bindings: arm: mediatek: document WED binding for MT7622 https://git.kernel.org/netdev/net-next/c/55c1c4e945fa - [v2,05/14] dt-bindings: arm: mediatek: document the pcie mirror node on MT7622 https://git.kernel.org/netdev/net-next/c/f14ac41b785f - [v2,06/14] net: ethernet: mtk_eth_soc: add support for Wireless Ethernet Dispatch (WED) https://git.kernel.org/netdev/net-next/c/804775dfc288 - [v2,07/14] net: ethernet: mtk_eth_soc: implement flow offloading to WED devices https://git.kernel.org/netdev/net-next/c/a333215e10cb - [v2,08/14] arm64: dts: mediatek: mt7622: introduce nodes for Wireless Ethernet Dispatch https://git.kernel.org/netdev/net-next/c/e9b65ecb7c30 - [v2,09/14] net: ethernet: mtk_eth_soc: add ipv6 flow offload support https://git.kernel.org/netdev/net-next/c/817b2fdf1667 - [v2,10/14] net: ethernet: mtk_eth_soc: support TC_SETUP_BLOCK for PPE offload https://git.kernel.org/netdev/net-next/c/bb14c19122b7 - [v2,11/14] net: ethernet: mtk_eth_soc: allocate struct mtk_ppe separately https://git.kernel.org/netdev/net-next/c/1ccc723b5829 - [v2,12/14] net: ethernet: mtk_eth_soc: rework hardware flow table management https://git.kernel.org/netdev/net-next/c/c4f033d9e03e - [v2,13/14] net: ethernet: mtk_eth_soc: remove bridge flow offload type entry support https://git.kernel.org/netdev/net-next/c/8ff25d377445 - [v2,14/14] net: ethernet: mtk_eth_soc: support creating mac address based offload entries https://git.kernel.org/netdev/net-next/c/33fc42de3327
You are awesome, thank you!
Signature · 9 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html _______________________________________________ Linux-mediatek mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-mediatek
Quoted 19 lines · On Wed, Apr 06, 2022 at 01:30:17PM +0000, [email protected]…
Hello:
This series was applied to netdev/net-next.git (master) by David S. Miller <[email protected]>:
Quoted 13 lines · On Tue, 5 Apr 2022 21:57:41 +0200 you wrote:
This series contains the following improvements to mediatek ethernet flow offload support:
- support dma-coherent on ethernet to improve performance
- add ipv6 offload support
- rework hardware flow table entry handling to improve dealing with hash collisions and competing flows
- support creating offload entries from user space
- support creating offload entries with just source/destination mac address, vlan and output device information
- add driver changes for supporting the Wireless Ethernet Dispatch core, which can be used to offload flows from ethernet to MT7915 PCIe WLAN devices
Hi David
It seems very early to merge this. The discussion of if the files are even in the right places has not even finished. And Arnd seems to not want parts of this in his subsystem. And there are some major architecture issues which need discussing...
I think you should revert this.
Andrew
Felix Fietkau
maintainerQuoted 21 lines · Andrew Lunn
Quoted 19 lines · On Wed, Apr 06, 2022 at 01:30:17PM +0000, [email protected]…
Hello:
This series was applied to netdev/net-next.git (master) by David S. Miller <[email protected]>:
Quoted 13 lines · On Tue, 5 Apr 2022 21:57:41 +0200 you wrote:
This series contains the following improvements to mediatek ethernet flow offload support:
- support dma-coherent on ethernet to improve performance
- add ipv6 offload support
- rework hardware flow table entry handling to improve dealing with hash collisions and competing flows
- support creating offload entries from user space
- support creating offload entries with just source/destination mac address, vlan and output device information
- add driver changes for supporting the Wireless Ethernet Dispatch core, which can be used to offload flows from ethernet to MT7915 PCIe WLAN devices
Quoted 8 lines
Hi David
It seems very early to merge this. The discussion of if the files are even in the right places has not even finished. And Arnd seems to not want parts of this in his subsystem. And there are some major architecture issues which need discussing...
I think you should revert this.
How about I simply send follow-up patches that move the relevant pieces to net?
- Felix
Quoted 23 lines · Felix Fietkau
Quoted 22 lines · Andrew Lunn
Quoted 18 lines · On Wed, Apr 06, 2022 at 01:30:17PM +0000, [email protected]…
Hello:
This series was applied to netdev/net-next.git (master) by David S. Miller <[email protected]>:
Quoted 12 lines · On Tue, 5 Apr 2022 21:57:41 +0200 you wrote:
This series contains the following improvements to mediatek ethernet flow offload support:
Quoted 1 line
- support dma-coherent on ethernet to improve performance
- add ipv6 offload support
- rework hardware flow table entry handling to improve dealing with hash collisions and competing flows
- support creating offload entries from user space
- support creating offload entries with just source/destination mac address, vlan and output device information
- add driver changes for supporting the Wireless Ethernet Dispatch core, which can be used to offload flows from ethernet to MT7915 PCIe WLAN devicesHi David
Quoted 8 lines
Quoted 6 lines
It seems very early to merge this. The discussion of if the files are even in the right places has not even finished. And Arnd seems to not want parts of this in his subsystem. And there are some major architecture issues which need discussing...
I think you should revert this.
How about I simply send follow-up patches that move the relevant pieces to net?
There has just been comments from Rob about the binding. I've not yet looked at the code, but if i remember correctly, v1 had some interaction with the DSA tagger, so i do want to look at it.
I'm also wondering if there is anything common here with IPA. It is an accelerator which sits between the WiFi and the mobile phone baseband device.
I really would prefer that a proper review of this code was made, by netdev people, and the bigger architecture questions looked at. So far, all the reviewers have been from outside netdev.
Andrew
_______________________________________________
Linux-mediatek mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-mediatek
Felix Fietkau
maintainerQuoted 33 lines · Andrew Lunn
Quoted 23 lines · Felix Fietkau
Quoted 22 lines · Andrew Lunn
Quoted 18 lines · On Wed, Apr 06, 2022 at 01:30:17PM +0000, [email protected]…
Hello:
This series was applied to netdev/net-next.git (master) by David S. Miller <[email protected]>:
Quoted 12 lines · On Tue, 5 Apr 2022 21:57:41 +0200 you wrote:
This series contains the following improvements to mediatek ethernet flow offload support:
Quoted 1 line
- support dma-coherent on ethernet to improve performance
- add ipv6 offload support
- rework hardware flow table entry handling to improve dealing with hash collisions and competing flows
- support creating offload entries from user space
- support creating offload entries with just source/destination mac address, vlan and output device information
- add driver changes for supporting the Wireless Ethernet Dispatch core, which can be used to offload flows from ethernet to MT7915 PCIe WLAN devicesHi David
Quoted 8 lines
Quoted 6 lines
It seems very early to merge this. The discussion of if the files are even in the right places has not even finished. And Arnd seems to not want parts of this in his subsystem. And there are some major architecture issues which need discussing...
I think you should revert this.
How about I simply send follow-up patches that move the relevant pieces to net?
Quoted 7 lines
There has just been comments from Rob about the binding. I've not yet looked at the code, but if i remember correctly, v1 had some interaction with the DSA tagger, so i do want to look at it.
I'm also wondering if there is anything common here with IPA. It is an accelerator which sits between the WiFi and the mobile phone baseband device.
I don't think it has much in common with IPA. WED doesn't completely sit between WiFi and the ethernet MAC, it just captures some (but not all) of the DMA rings and intercepts IRQs.
Quoted 3 lines
I really would prefer that a proper review of this code was made, by netdev people, and the bigger architecture questions looked at. So far, all the reviewers have been from outside netdev.
Sure. I will definitely quickly fix any issues that show up. I did submit this series once already a while back, and there hasn't really been any feedback on it.
- Felix