net: dwc_eth_qos: Fully rewrite RX descriptor field 3
PATCH 1/6 net: dwc_eth_qos: Fully rewrite RX descriptor field 3
The RX descriptor field 3 should contain only OWN and BUF1V bits before being used for receiving data by the DMA engine. However, right now, if the descriptor was already used for receiving data and is being cleared, the field 3 is only modified and the aforementioned two bits are ORRed into the field. This could lead to a residual dirty bits being left in the field 3 from previous transfer, and it generally does. Fully set the field 3 instead to clear those residual dirty bits.
--- drivers/net/dwc_eth_qos.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/net/dwc_eth_qos.c +2 -2
@@ -1242,7 +1242,7 @@ static int eqos_start(struct udevice *dev)struct eqos_desc *rx_desc = &(eqos->rx_descs[i]); rx_desc->des0 = (u32)(ulong)(eqos->rx_dma_buf + (i *EQOS_MAX_PACKET_SIZE )); - rx_desc->des3 |=EQOS_DESC3_OWN |EQOS_DESC3_BUF1V ; + rx_desc->des3 =EQOS_DESC3_OWN |EQOS_DESC3_BUF1V ; } eqos->config->ops->eqos_flush_desc (eqos->descs);
@@ -1436,7 +1436,7 @@ static int eqos_free_pkt(struct udevice *dev, uchar *packet, int length)* writes to the rest of the descriptor too. */ mb (); - rx_desc->des3 |=EQOS_DESC3_OWN |EQOS_DESC3_BUF1V ; + rx_desc->des3 =EQOS_DESC3_OWN |EQOS_DESC3_BUF1V ; eqos->config->ops->eqos_flush_desc (rx_desc);writel ((ulong )rx_desc, &eqos->dma_regs->ch0_rxdesc_tail_pointer);
Signature · 1 line
2.25.1
PATCH 1/6 net: dwc_eth_qos: Fully rewrite RX descriptor field 3
Quoted 8 lines · Marek Vasut
The RX descriptor field 3 should contain only OWN and BUF1V bits before being used for receiving data by the DMA engine. However, right now, if the descriptor was already used for receiving data and is being cleared, the field 3 is only modified and the aforementioned two bits are ORRed into the field. This could lead to a residual dirty bits being left in the field 3 from previous transfer, and it generally does. Fully set the field 3 instead to clear those residual dirty bits.
Quoted 6 lines
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>
Joe, this was posted for a month, can this go in ?
PATCH 2/6 net: dwc_eth_qos: Correctly wrap around TX descriptor tail pointer
In reply to Marek VasutThis code programs the next descriptor in the TX descriptor ring into the hardware as the last valid TX descriptor. The problem is that if the currenty descriptor is the last one in the array, the code will not wrap around correctly and use TX descriptor 0 again, but instead will use TX descriptor at address right past the TX descriptor ring, which is the first descriptor in the RX ring.
Fix this by adding the necessary wrap-around.
--- drivers/net/dwc_eth_qos.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/net/dwc_eth_qos.c +2 -1
@@ -1373,7 +1373,8 @@ static int eqos_send(struct udevice *dev, void *packet, int length) tx_desc->des3 =EQOS_DESC3_OWN |EQOS_DESC3_FD |EQOS_DESC3_LD | length; eqos->config->ops->eqos_flush_desc (tx_desc); -writel ((ulong)(tx_desc +1 ), &eqos->dma_regs->ch0_txdesc_tail_pointer); +writel ((ulong)(&(eqos->tx_descs[eqos->tx_desc_idx])), + &eqos->dma_regs->ch0_txdesc_tail_pointer);for (i =0 ; i <1000000 ; i++) { eqos->config->ops->eqos_inval_desc (tx_desc);
Signature · 1 line
2.25.1
PATCH 2/6 net: dwc_eth_qos: Correctly wrap around TX descriptor tail pointer
Quoted 35 lines · Marek Vasut
This code programs the next descriptor in the TX descriptor ring into the hardware as the last valid TX descriptor. The problem is that if the currenty descriptor is the last one in the array, the code will not wrap around correctly and use TX descriptor 0 again, but instead will use TX descriptor at address right past the TX descriptor ring, which is the first descriptor in the RX ring.
Fix this by adding the necessary wrap-around.
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>--- drivers/net/dwc_eth_qos.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/net/dwc_eth_qos.c +2
-2@@ -1373,7 +1373,8 @@ static int eqos_send(struct udevice *dev, void *packet, int length) tx_desc->des3 =EQOS_DESC3_OWN |EQOS_DESC3_FD |EQOS_DESC3_LD | length; eqos->config->ops->eqos_flush_desc (tx_desc); -writel ((ulong)(tx_desc +1 ), &eqos->dma_regs->ch0_txdesc_tail_pointer); +writel ((ulong)(&(eqos->tx_descs[eqos->tx_desc_idx])), + &eqos->dma_regs->ch0_txdesc_tail_pointer);for (i =0 ; i <1000000 ; i++) { eqos->config->ops->eqos_inval_desc (tx_desc); --
Quoted 2 lines
2.25.1
PATCH 2/6 net: dwc_eth_qos: Correctly wrap around TX descriptor tail pointer
In reply to Marek VasutDear Marek,
Quoted 11 lines
From: Marek Vasut <[email protected]>
Sent: lundi 23 mars 2020 02:45This code programs the next descriptor in the TX descriptor ring into the hardware as the last valid TX descriptor. The problem is that if the currenty descriptor is the last one in the array, the code will not wrap around correctly and use TX descriptor 0 again, but instead will use TX descriptor at address right past the TX descriptor ring, which is the first descriptor in the RX ring.
Fix this by adding the necessary wrap-around.
Quoted 7 lines
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>---
Thanks
Patrick
PATCH 3/6 net: dwc_eth_qos: Flush the RX descriptors on init
In reply to Marek VasutCurrently the code only flushes the first RX descriptor, not every entry in the RX descriptor ring. Fix this, to make sure the DMA engine can pick the RX descriptors correctly.
--- drivers/net/dwc_eth_qos.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/dwc_eth_qos.c +1 -1
@@ -1243,8 +1243,8 @@ static int eqos_start(struct udevice *dev) rx_desc->des0 = (u32)(ulong)(eqos->rx_dma_buf + (i *EQOS_MAX_PACKET_SIZE )); rx_desc->des3 =EQOS_DESC3_OWN |EQOS_DESC3_BUF1V ; + eqos->config->ops->eqos_flush_desc (rx_desc); } - eqos->config->ops->eqos_flush_desc (eqos->descs);writel (0 , &eqos->dma_regs->ch0_txdesc_list_haddress);writel ((ulong )eqos->tx_descs, &eqos->dma_regs->ch0_txdesc_list_address);
Signature · 1 line
2.25.1
PATCH 3/6 net: dwc_eth_qos: Flush the RX descriptors on init
Quoted 30 lines · Marek Vasut
Currently the code only flushes the first RX descriptor, not every entry in the RX descriptor ring. Fix this, to make sure the DMA engine can pick the RX descriptors correctly.
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>--- drivers/net/dwc_eth_qos.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/net/dwc_eth_qos.c +1
-2@@ -1243,8 +1243,8 @@ static int eqos_start(struct udevice *dev) rx_desc->des0 = (u32)(ulong)(eqos->rx_dma_buf + (i *EQOS_MAX_PACKET_SIZE )); rx_desc->des3 =EQOS_DESC3_OWN |EQOS_DESC3_BUF1V ; + eqos->config->ops->eqos_flush_desc (rx_desc); } - eqos->config->ops->eqos_flush_desc (eqos->descs);writel (0 , &eqos->dma_regs->ch0_txdesc_list_haddress);writel ((ulong )eqos->tx_descs, &eqos->dma_regs->ch0_txdesc_list_address); --
Quoted 2 lines
2.25.1
PATCH 3/6 net: dwc_eth_qos: Flush the RX descriptors on init
In reply to Marek VasutDear Marek,
Quoted 14 lines
From: Marek Vasut <[email protected]>
Sent: lundi 23 mars 2020 02:45
To: u-boot at lists.denx.de
Cc: Marek Vasut <[email protected]>; Joe Hershberger <[email protected]>; Patrice CHOTARD <[email protected]>; Patrick DELAUNAY <[email protected]>; Ramon Fried <[email protected]>; Stephen Warren <[email protected]>
Subject: [PATCH 3/6] net:dwc_eth_qos: Flush the RX descriptors on init
Importance: HighCurrently the code only flushes the first RX descriptor, not every entry in the RX descriptor ring. Fix this, to make sure the DMA engine can pick the RX descriptors correctly.
Quoted 7 lines
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>---
Thanks
Patrick
PATCH 4/6 net: dwc_eth_qos: Invalidate RX descriptor before reading
In reply to Marek VasutThe current code polls the RX desciptor ring for new packets by reading the RX descriptor status. This works by accident, as the RX descriptors are often in non-cacheable memory. However, the driver does support use of RX descriptors in cacheable memory.
This patch adds a missing RX descriptor invalidation, which assures the CPU will read a fresh copy of the RX descriptor instead of a cached one.
--- drivers/net/dwc_eth_qos.c | 1 + 1 file changed, 1 insertion(+)
drivers/net/dwc_eth_qos.c +1 -0
@@ -1397,6 +1397,7 @@ static int eqos_recv(struct udevice *dev, int flags, uchar **packetp)debug ("%s(dev=%p, flags=%x):\n" , __func__, dev, flags); rx_desc = &(eqos->rx_descs[eqos->rx_desc_idx]); + eqos->config->ops->eqos_inval_desc (rx_desc);if (rx_desc->des3 &EQOS_DESC3_OWN ) {debug ("%s: RX packet not available\n" , __func__);return -EAGAIN ;
Signature · 1 line
2.25.1
PATCH 4/6 net: dwc_eth_qos: Invalidate RX descriptor before reading
Quoted 32 lines · Marek Vasut
The current code polls the RX desciptor ring for new packets by reading the RX descriptor status. This works by accident, as the RX descriptors are often in non-cacheable memory. However, the driver does support use of RX descriptors in cacheable memory.
This patch adds a missing RX descriptor invalidation, which assures the CPU will read a fresh copy of the RX descriptor instead of a cached one.
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>--- drivers/net/dwc_eth_qos.c | 1 + 1 file changed, 1 insertion(+)drivers/net/dwc_eth_qos.c +1
-1@@ -1397,6 +1397,7 @@ static int eqos_recv(struct udevice *dev, int flags, uchar **packetp)debug ("%s(dev=%p, flags=%x):\n" , __func__, dev, flags); rx_desc = &(eqos->rx_descs[eqos->rx_desc_idx]); + eqos->config->ops->eqos_inval_desc (rx_desc);if (rx_desc->des3 &EQOS_DESC3_OWN ) {debug ("%s: RX packet not available\n" , __func__);return -EAGAIN ; --
Quoted 2 lines
2.25.1
PATCH 4/6 net: dwc_eth_qos: Invalidate RX descriptor before reading
In reply to Marek VasutDear Marek,
Quoted 11 lines
From: Marek Vasut <[email protected]>
Sent: lundi 23 mars 2020 02:45The current code polls the RX desciptor ring for new packets by reading the RX descriptor status. This works by accident, as the RX descriptors are often in non- cacheable memory. However, the driver does support use of RX descriptors in cacheable memory.
This patch adds a missing RX descriptor invalidation, which assures the CPU will read a fresh copy of the RX descriptor instead of a cached one.
Quoted 7 lines
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>---
Patrick
PATCH 5/6 net: dwc_eth_qos: Invalidate RX packet DMA buffer
In reply to Marek VasutThis patch prevents an issue where the RX packet might have been accessed by the CPU, which now has cached data from the packet in the caches and possibly various write buffers, and these data may be evicted from the caches into the DRAM while the buffer is also written by the DMA.
By invalidating the buffer after the CPU accessed it and before the DMA populates the buffer, it is assured that the buffer will not be corrupted.
--- drivers/net/dwc_eth_qos.c | 6 ++++++ 1 file changed, 6 insertions(+)
drivers/net/dwc_eth_qos.c +6 -0
@@ -1430,6 +1430,9 @@ static int eqos_free_pkt(struct udevice *dev, uchar *packet, int length) } rx_desc = &(eqos->rx_descs[eqos->rx_desc_idx]); + + eqos->config->ops->eqos_inval_buffer (packet, length); + rx_desc->des0 = (u32 )(ulong )packet; rx_desc->des1 =0 ; rx_desc->des2 =0 ;
@@ -1492,6 +1495,9 @@ static int eqos_probe_resources_core(struct udevice *dev) }debug ("%s: rx_pkt=%p\n" , __func__, eqos->rx_pkt); + eqos->config->ops->eqos_inval_buffer (eqos->rx_dma_buf, +EQOS_MAX_PACKET_SIZE *EQOS_DESCRIPTORS_RX ); +debug ("%s: OK\n" , __func__);return 0 ;
Signature · 1 line
2.25.1
PATCH 5/6 net: dwc_eth_qos: Invalidate RX packet DMA buffer
Quoted 46 lines · Marek Vasut
This patch prevents an issue where the RX packet might have been accessed by the CPU, which now has cached data from the packet in the caches and possibly various write buffers, and these data may be evicted from the caches into the DRAM while the buffer is also written by the DMA.
By invalidating the buffer after the CPU accessed it and before the DMA populates the buffer, it is assured that the buffer will not be corrupted.
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>--- drivers/net/dwc_eth_qos.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/net/dwc_eth_qos.c +6
-1@@ -1430,6 +1430,9 @@ static int eqos_free_pkt(struct udevice *dev, uchar *packet, int length) } rx_desc = &(eqos->rx_descs[eqos->rx_desc_idx]); + + eqos->config->ops->eqos_inval_buffer (packet, length); + rx_desc->des0 = (u32 )(ulong )packet; rx_desc->des1 =0 ; rx_desc->des2 =0 ;@@ -1492,6 +1495,9 @@ static int eqos_probe_resources_core(struct udevice *dev) }debug ("%s: rx_pkt=%p\n" , __func__, eqos->rx_pkt); + eqos->config->ops->eqos_inval_buffer (eqos->rx_dma_buf, +EQOS_MAX_PACKET_SIZE *EQOS_DESCRIPTORS_RX ); +debug ("%s: OK\n" , __func__);return 0 ; --
Quoted 2 lines
2.25.1
PATCH 5/6 net: dwc_eth_qos: Invalidate RX packet DMA buffer
In reply to Marek VasutHi Marek,
Quoted 11 lines
From: Marek Vasut <[email protected]>
Sent: lundi 23 mars 2020 02:45This patch prevents an issue where the RX packet might have been accessed by the CPU, which now has cached data from the packet in the caches and possibly various write buffers, and these data may be evicted from the caches into the DRAM while the buffer is also written by the DMA.
By invalidating the buffer after the CPU accessed it and before the DMA populates the buffer, it is assured that the buffer will not be corrupted.
Quoted 7 lines
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>---
Patrick
PATCH 1/6 net: dwc_eth_qos: Fully rewrite RX descriptor field 3
In reply to Marek VasutQuoted 42 lines · Marek Vasut
The RX descriptor field 3 should contain only OWN and BUF1V bits before being used for receiving data by the DMA engine. However, right now, if the descriptor was already used for receiving data and is being cleared, the field 3 is only modified and the aforementioned two bits are ORRed into the field. This could lead to a residual dirty bits being left in the field 3 from previous transfer, and it generally does. Fully set the field 3 instead to clear those residual dirty bits.
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>--- drivers/net/dwc_eth_qos.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/net/dwc_eth_qos.c +2
-3@@ -1242,7 +1242,7 @@ static int eqos_start(struct udevice *dev)struct eqos_desc *rx_desc = &(eqos->rx_descs[i]); rx_desc->des0 = (u32)(ulong)(eqos->rx_dma_buf + (i *EQOS_MAX_PACKET_SIZE )); - rx_desc->des3 |=EQOS_DESC3_OWN |EQOS_DESC3_BUF1V ; + rx_desc->des3 =EQOS_DESC3_OWN |EQOS_DESC3_BUF1V ; } eqos->config->ops->eqos_flush_desc (eqos->descs);@@ -1436,7 +1436,7 @@ static int eqos_free_pkt(struct udevice *dev, uchar *packet, int length)* writes to the rest of the descriptor too. */ mb (); - rx_desc->des3 |=EQOS_DESC3_OWN |EQOS_DESC3_BUF1V ; + rx_desc->des3 =EQOS_DESC3_OWN |EQOS_DESC3_BUF1V ; eqos->config->ops->eqos_flush_desc (rx_desc);writel ((ulong )rx_desc, &eqos->dma_regs->ch0_rxdesc_tail_pointer); --
Quoted 2 lines
2.25.1
PATCH 1/6 net: dwc_eth_qos: Fully rewrite RX descriptor field 3
In reply to Marek VasutQuoted 7 lines · Marek Vasut
The RX descriptor field 3 should contain only OWN and BUF1V bits before being used for receiving data by the DMA engine. However, right now, if the descriptor was already used for receiving data and is being cleared, the field 3 is only modified and the aforementioned two bits are ORRed into the field. This could lead to a residual dirty bits being left in the field 3 from previous transfer, and it generally does. Fully set the field 3 instead to clear those residual dirty bits.
The series,
(On Jetson, using test/py tests)
PATCH 1/6 net: dwc_eth_qos: Fully rewrite RX descriptor field 3
In reply to Marek VasutDear Marek,
Quoted 21 lines
From: Marek Vasut <[email protected]>
Sent: lundi 23 mars 2020 02:45The RX descriptor field 3 should contain only OWN and BUF1V bits before being used for receiving data by the DMA engine. However, right now, if the descriptor was already used for receiving data and is being cleared, the field 3 is only modified and the aforementioned two bits are ORRed into the field. This could lead to a residual dirty bits being left in the field 3 from previous transfer, and it generally does. Fully set the field 3 instead to clear those residual dirty bits.
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>--- drivers/net/dwc_eth_qos.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/net/dwc_eth_qos.c index renamed from drivers/net/dwc_eth_qos.c +0
-0
Quoted 2 lines
0564bebf76..4bce6d4290100644
---a/drivers/net/dwc_eth_qos.c
Thanks
Patrick
PATCH 1/6 net: dwc_eth_qos: Fully rewrite RX descriptor field 3
In reply to Marek VasutDear Marek,
Quoted 10 lines
From: Marek Vasut <[email protected]>
Sent: lundi 23 mars 2020 02:45The RX descriptor field 3 should contain only OWN and BUF1V bits before being used for receiving data by the DMA engine. However, right now, if the descriptor was already used for receiving data and is being cleared, the field 3 is only modified and the aforementioned two bits are ORRed into the field. This could lead to a residual dirty bits being left in the field 3 from previous transfer, and it generally does. Fully set the field 3 instead to clear those residual dirty bits.
Quoted 7 lines
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>---
For the series,
Regards
Patrick
PATCH 1/6 net: dwc_eth_qos: Fully rewrite RX descriptor field 3
Dear Marek,
Quoted 24 lines
From: Patrick DELAUNAY
Sent: lundi 6 avril 2020 17:11Dear Marek,
Quoted 9 lines
From: Marek Vasut <[email protected]>
Sent: lundi 23 mars 2020 02:45The RX descriptor field 3 should contain only OWN and BUF1V bits before being used for receiving data by the DMA engine. However, right now, if the descriptor was already used for receiving data and is being cleared, the field 3 is only modified and the aforementioned two bits are ORRed into the field. This could lead to a residual dirty bits being left in the field 3 from previous transfer, and it generally does. Fully
set the field 3 instead to clear those residual dirty bits.
Quoted 8 lines
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>---
Quoted 3 lines
For the series,
manual test: dhcp and run
bootcmd_pxe)
To complete my test and to check the cache management in the driver,
I test the sequence (CONFIG_SYS_NONCACHED_MEMORY is activated):
1) ping with dcache ON: Always OK
STM32MP> dhcp
STM32MP> ping 91.162.57.93
2) ping with dcache OFF : Always OK
STM32MP> dcache off
STM32MP> ping 91.162.57.93
3) ping with dcache ON : Failed
STM32MP> dcache on
STM32MP> ping 91.162.57.93
Failed with "eqos_send: TX timeout"
and after "eqos_recv: RX packet not available"
4) ping with dcache OFF : Always OK
STM32MP> dcache on
STM32MP> ping 91.162.57.93
I don't sure this sequence is really valid or if this issue show a problem in cache management.
As I don't see any obvious issue in eqos_send, do you idea on reason of this issue ?
Quoted 3 lines
Regards
Patrick
PATCH 1/6 net: dwc_eth_qos: Fully rewrite RX descriptor field 3
Quoted 1 line · Patrick DELAUNAY
Dear Marek,
Hi,
Quoted 28 lines
To complete my test and to check the cache management in the driver,
I test the sequence (
CONFIG_SYS_NONCACHED_MEMORYis activated):1) ping with dcache ON: Always OK
STM32MP> dhcp
STM32MP> ping 91.162.57.932) ping with dcache OFF : Always OK
STM32MP> dcache off
STM32MP> ping 91.162.57.933) ping with dcache ON : Failed
STM32MP> dcache on
STM32MP> ping 91.162.57.93Failed with "
eqos_send: TX timeout"
and after "eqos_recv: RX packet not available"4) ping with dcache OFF : Always OK
STM32MP> dcache on
STM32MP> ping 91.162.57.93
Quoted 5 lines
I don't sure this sequence is really valid or if this issue show a problem in cache management.
As I don't see any obvious issue in
eqos_send, do you idea on reason of this issue ?
So the test is basically:
dhcp
while true ; do
ping 192.168.1.300 ;
dcache off ;
ping 192.168.1.300 ;
dcache on ;
done
right ?
I can tell you what you're likely gonna see if you debug it further.
Use the board on an isolated network segment (get some USB ethernet or whatever), disable dhcp server, avahi and all other such stuff. Then run wireshark on that interface.
Then, patch the dwmac driver to dump every packet it sends and receives, the first 16 or so bytes are enough, so you'd see the header. Then run your test.
After a while, you will see that the DWMAC transmits some valid packet, but receives a packet with the exact same source MAC it just transmitted , the source MAC of the stm32 board. But if I recall it correctly, you won't see that packet on the line in wireshark, it just somehow loops back in.
You can even accelerate that process by using arping from the host PC, just arping the stm32 and the error will happen faster.
I was hoping I managed to fix all of these problems in this series, but apparently there is more :-( Do you feel like debugging this further?
PATCH 1/6 net: dwc_eth_qos: Fully rewrite RX descriptor field 3
Dear,
Quoted 52 lines
From: Marek Vasut <[email protected]>
Sent: mardi 7 avril 2020 12:04Quoted 1 line · Patrick DELAUNAY
Dear Marek,
Hi,
Quoted 28 lines
To complete my test and to check the cache management in the driver,
I test the sequence (
CONFIG_SYS_NONCACHED_MEMORYis activated):1) ping with dcache ON: Always OK
STM32MP> dhcp
STM32MP> ping 91.162.57.932) ping with dcache OFF : Always OK
STM32MP> dcache off
STM32MP> ping 91.162.57.933) ping with dcache ON : Failed
STM32MP> dcache on
STM32MP> ping 91.162.57.93Failed with "
eqos_send: TX timeout"
and after "eqos_recv: RX packet not available"4) ping with dcache OFF : Always OK
STM32MP> dcache on
STM32MP> ping 91.162.57.93Quoted 5 lines
I don't sure this sequence is really valid or if this issue show a problem in cache management.
As I don't see any obvious issue in
eqos_send, do you idea on reason of this issue ?So the test is basically:
dhcp
while true ; do
ping 192.168.1.300 ;
dcache off ;
ping 192.168.1.300 ;
dcache on ;
done
Quoted 1 line
right ?
Yes, exactly and it is more clear
Quoted 15 lines
I can tell you what you're likely gonna see if you debug it further.
Use the board on an isolated network segment (get some USB ethernet or whatever), disable dhcp server, avahi and all other such stuff. Then run wireshark on that interface.
Then, patch the dwmac driver to dump every packet it sends and receives, the first 16 or so bytes are enough, so you'd see the header. Then run your test.
After a while, you will see that the DWMAC transmits some valid packet, but receives a packet with the exact same source MAC it just transmitted , the source MAC of the stm32 board. But if I recall it correctly, you won't see that packet on the line in wireshark, it just somehow loops back in.
Quoted 5 lines
You can even accelerate that process by using arping from the host PC, just arping the stm32 and the error will happen faster.
I was hoping I managed to fix all of these problems in this series, but apparently there is more :-( Do you feel like debugging this further?
Yes it is my fear.
Thanks for advices...
I will try to debug it
but my bandwith is limited for the next week, so I can't give a guarantee.
Regards
Patrick
PATCH 1/6 net: dwc_eth_qos: Fully rewrite RX descriptor field 3
Quoted 76 lines · Patrick DELAUNAY
Dear,
Quoted 52 lines
From: Marek Vasut <[email protected]>
Sent: mardi 7 avril 2020 12:04Quoted 1 line · Patrick DELAUNAY
Dear Marek,
Hi,
Quoted 28 lines
To complete my test and to check the cache management in the driver,
I test the sequence (
CONFIG_SYS_NONCACHED_MEMORYis activated):1) ping with dcache ON: Always OK
STM32MP> dhcp
STM32MP> ping 91.162.57.932) ping with dcache OFF : Always OK
STM32MP> dcache off
STM32MP> ping 91.162.57.933) ping with dcache ON : Failed
STM32MP> dcache on
STM32MP> ping 91.162.57.93Failed with "
eqos_send: TX timeout"
and after "eqos_recv: RX packet not available"4) ping with dcache OFF : Always OK
STM32MP> dcache on
STM32MP> ping 91.162.57.93Quoted 5 lines
I don't sure this sequence is really valid or if this issue show a problem in cache management.
As I don't see any obvious issue in
eqos_send, do you idea on reason of this issue ?So the test is basically:
dhcp
while true ; do
ping 192.168.1.300 ;
dcache off ;
ping 192.168.1.300 ;
dcache on ;
doneQuoted 1 line
right ?
Yes, exactly and it is more clear
Quoted 18 lines
I can tell you what you're likely gonna see if you debug it further.
Use the board on an isolated network segment (get some USB ethernet or whatever), disable dhcp server, avahi and all other such stuff. Then run wireshark on that interface.
Then, patch the dwmac driver to dump every packet it sends and receives, the first 16 or so bytes are enough, so you'd see the header. Then run your test.
After a while, you will see that the DWMAC transmits some valid packet, but receives a packet with the exact same source MAC it just transmitted , the source MAC of the stm32 board. But if I recall it correctly, you won't see that packet on the line in wireshark, it just somehow loops back in.
You can even accelerate that process by using arping from the host PC, just arping the stm32 and the error will happen faster.
Quoted 8 lines
Quoted 2 lines
I was hoping I managed to fix all of these problems in this series, but apparently there is more :-( Do you feel like debugging this further?
Yes it is my fear.
Thanks for advices...
I will try to debug it
but my bandwith is limited for the next week, so I can't give a guarantee.
I sent a patch to pad the descriptors to cacheline size just now, however I still didn't get Joe to review/apply any of the networking patches. I hope he's just busy with other stuff ...
PATCH 6/6 net: dwc_eth_qos: Prevent DMA from writing updated RX DMA descriptor
In reply to Marek VasutThe DMA may attempt to write a DMA descriptor in the ring while it is being updated. By writing the DMA descriptor buffer address to 0, it is assured the DMA will not use such a buffer and the buffer can be updated without any interference.
--- drivers/net/dwc_eth_qos.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/net/dwc_eth_qos.c +3 -1
@@ -1431,8 +1431,10 @@ static int eqos_free_pkt(struct udevice *dev, uchar *packet, int length) rx_desc = &(eqos->rx_descs[eqos->rx_desc_idx]); + rx_desc->des0 =0 ; +mb (); + eqos->config->ops->eqos_flush_desc (rx_desc); eqos->config->ops->eqos_inval_buffer (packet, length); - rx_desc->des0 = (u32 )(ulong )packet; rx_desc->des1 =0 ; rx_desc->des2 =0 ;
Signature · 1 line
2.25.1
PATCH 6/6 net: dwc_eth_qos: Prevent DMA from writing updated RX DMA descriptor
Quoted 21 lines · Marek Vasut
The DMA may attempt to write a DMA descriptor in the ring while it is being updated. By writing the DMA descriptor buffer address to 0, it is assured the DMA will not use such a buffer and the buffer can be updated without any interference.
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>--- drivers/net/dwc_eth_qos.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)drivers/net/dwc_eth_qos.c +0
-0
drivers/net/dwc_eth_qos.c static int eqos_free_pkt(struct udevice *dev, uchar *packet, int length) · line 1435
Quoted 5 lines
drivers/net/dwc_eth_qos.c +2
-0@@ -1431,8 +1431,10 @@ static int eqos_free_pkt(struct udevice *dev, uchar *packet, int length) rx_desc = &(eqos->rx_descs[eqos->rx_desc_idx]); + rx_desc->des0 =0 ; +mb ();
Better use wmb() for better understanding your goal here ?
Quoted 9 lines
drivers/net/dwc_eth_qos.c +1
-2+ eqos->config->ops->eqos_flush_desc (rx_desc); eqos->config->ops->eqos_inval_buffer (packet, length); - rx_desc->des0 = (u32 )(ulong )packet; rx_desc->des1 =0 ; rx_desc->des2 =0 ; --2.25.1
PATCH 6/6 net: dwc_eth_qos: Prevent DMA from writing updated RX DMA descriptor
Quoted 23 lines · Ramon Fried
Quoted 22 lines · Marek Vasut
The DMA may attempt to write a DMA descriptor in the ring while it is being updated. By writing the DMA descriptor buffer address to 0, it is assured the DMA will not use such a buffer and the buffer can be updated without any interference.
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>--- drivers/net/dwc_eth_qos.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)drivers/net/dwc_eth_qos.c +0
-0@@ -1431,8 +1431,10 @@ static int eqos_free_pkt(struct udevice *dev, uchar *packet, int length)
Quoted 5 lines
drivers/net/dwc_eth_qos.c static int eqos_free_pkt(struct udevice *dev, uchar *packet, int length) · line 1435
Quoted 4 lines
drivers/net/dwc_eth_qos.c +2
-0rx_desc = &(eqos->rx_descs[eqos->rx_desc_idx]); + rx_desc->des0 =0 ; +mb ();Better use
wmb()for better understanding your goal here ?
The driver uses mb() all over the place, so I'm just keeping it consistent here.
But I wonder whether we even need all those barriers in this driver at all.
PATCH 6/6 net: dwc_eth_qos: Prevent DMA from writing updated RX DMA descriptor
Quoted 27 lines · Marek Vasut
Quoted 25 lines · Ramon Fried
Quoted 24 lines · Marek Vasut
The DMA may attempt to write a DMA descriptor in the ring while it is being updated. By writing the DMA descriptor buffer address to 0, it is assured the DMA will not use such a buffer and the buffer can be updated without any interference.
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>--- drivers/net/dwc_eth_qos.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)drivers/net/dwc_eth_qos.c +0
-0@@ -1431,8 +1431,10 @@ static int eqos_free_pkt(struct udevice *dev, uchar *packet, int length) rx_desc = &(eqos->rx_descs[eqos->rx_desc_idx]);
Quoted 8 lines
Quoted 3 lines
drivers/net/dwc_eth_qos.c static int eqos_free_pkt(struct udevice *dev, uchar *packet, int length) · line 1435
Quoted 2 lines
drivers/net/dwc_eth_qos.c +2
-0+ rx_desc->des0 =0 ; +mb ();Better use
wmb()for better understanding your goal here ?The driver uses
mb()all over the place, so I'm just keeping it consistent here.But I wonder whether we even need all those barriers in this driver at all.
In that case that's fine :)
PATCH 6/6 net: dwc_eth_qos: Prevent DMA from writing updated RX DMA descriptor
Quoted 32 lines · Ramon Fried
Quoted 31 lines · Marek Vasut
Quoted 28 lines · Ramon Fried
Quoted 21 lines · Marek Vasut
The DMA may attempt to write a DMA descriptor in the ring while it is being updated. By writing the DMA descriptor buffer address to 0, it is assured the DMA will not use such a buffer and the buffer can be updated without any interference.
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>--- drivers/net/dwc_eth_qos.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)drivers/net/dwc_eth_qos.c +0
-0drivers/net/dwc_eth_qos.c static int eqos_free_pkt(struct udevice *dev, uchar *packet, int length) · line 1435
Quoted 5 lines
drivers/net/dwc_eth_qos.c +2
-0@@ -1431,8 +1431,10 @@ static int eqos_free_pkt(struct udevice *dev, uchar *packet, int length) rx_desc = &(eqos->rx_descs[eqos->rx_desc_idx]); + rx_desc->des0 =0 ; +mb ();Better use
wmb()for better understanding your goal here ?
Quoted 5 lines
Quoted 4 lines
The driver uses
mb()all over the place, so I'm just keeping it consistent here.But I wonder whether we even need all those barriers in this driver at all.
In that case that's fine :)
I mean, I am asking about the need for those barriers here, that's my question here.
PATCH 6/6 net: dwc_eth_qos: Prevent DMA from writing updated RX DMA descriptor
Quoted 34 lines · Marek Vasut
Quoted 32 lines · Ramon Fried
Quoted 31 lines · Marek Vasut
Quoted 28 lines · Ramon Fried
Quoted 21 lines · Marek Vasut
The DMA may attempt to write a DMA descriptor in the ring while it is being updated. By writing the DMA descriptor buffer address to 0, it is assured the DMA will not use such a buffer and the buffer can be updated without any interference.
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>--- drivers/net/dwc_eth_qos.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)drivers/net/dwc_eth_qos.c +0
-0drivers/net/dwc_eth_qos.c static int eqos_free_pkt(struct udevice *dev, uchar *packet, int length) · line 1435
Quoted 5 lines
drivers/net/dwc_eth_qos.c +2
-0@@ -1431,8 +1431,10 @@ static int eqos_free_pkt(struct udevice *dev, uchar *packet, int length) rx_desc = &(eqos->rx_descs[eqos->rx_desc_idx]); + rx_desc->des0 =0 ; +mb ();Better use
wmb()for better understanding your goal here ?
Quoted 8 lines
Quoted 5 lines
Quoted 4 lines
The driver uses
mb()all over the place, so I'm just keeping it consistent here.But I wonder whether we even need all those barriers in this driver at all.
In that case that's fine :)
I mean, I am asking about the need for those barriers here, that's my question here.
Better safe than sorry. unless you have the means to test it to see if it doesn't break.
Thanks,
Ramon.
PATCH 6/6 net: dwc_eth_qos: Prevent DMA from writing updated RX DMA descriptor
Quoted 38 lines · Ramon Fried
Quoted 37 lines · Marek Vasut
Quoted 35 lines · Ramon Fried
Quoted 34 lines · Marek Vasut
Quoted 23 lines · Ramon Fried
Quoted 22 lines · Marek Vasut
The DMA may attempt to write a DMA descriptor in the ring while it is being updated. By writing the DMA descriptor buffer address to 0, it is assured the DMA will not use such a buffer and the buffer can be updated without any interference.
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>--- drivers/net/dwc_eth_qos.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)drivers/net/dwc_eth_qos.c +0
-0@@ -1431,8 +1431,10 @@ static int eqos_free_pkt(struct udevice *dev, uchar *packet, int length)Quoted 5 lines
drivers/net/dwc_eth_qos.c static int eqos_free_pkt(struct udevice *dev, uchar *packet, int length) · line 1435
Quoted 4 lines
drivers/net/dwc_eth_qos.c +2
-0rx_desc = &(eqos->rx_descs[eqos->rx_desc_idx]); + rx_desc->des0 =0 ; +mb ();Better use
wmb()for better understanding your goal here ?The driver uses
mb()all over the place, so I'm just keeping it consistent here.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Quoted 1 line
But I wonder whether we even need all those barriers in this driver at all.
In that case that's fine :)
I mean, I am asking about the need for those barriers here, that's my question here.
Better safe than sorry. unless you have the means to test it to see if it doesn't break.
On STM32MP1, yes. Not on the Tegra platform.
PATCH 6/6 net: dwc_eth_qos: Prevent DMA from writing updated RX DMA descriptor
In reply to Marek VasutQuoted 33 lines · Marek Vasut
The DMA may attempt to write a DMA descriptor in the ring while it is being updated. By writing the DMA descriptor buffer address to 0, it is assured the DMA will not use such a buffer and the buffer can be updated without any interference.
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>--- drivers/net/dwc_eth_qos.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)drivers/net/dwc_eth_qos.c +3
-2@@ -1431,8 +1431,10 @@ static int eqos_free_pkt(struct udevice *dev, uchar *packet, int length) rx_desc = &(eqos->rx_descs[eqos->rx_desc_idx]); + rx_desc->des0 =0 ; +mb (); + eqos->config->ops->eqos_flush_desc (rx_desc); eqos->config->ops->eqos_inval_buffer (packet, length); - rx_desc->des0 = (u32 )(ulong )packet; rx_desc->des1 =0 ; rx_desc->des2 =0 ; --
Quoted 2 lines
2.25.1
PATCH 6/6 net: dwc_eth_qos: Prevent DMA from writing updated RX DMA descriptor
In reply to Marek VasutDear Marek,
Quoted 7 lines
From: Marek Vasut <[email protected]>
Sent: lundi 23 mars 2020 02:45The DMA may attempt to write a DMA descriptor in the ring while it is being updated. By writing the DMA descriptor buffer address to 0, it is assured the DMA will not use such a buffer and the buffer can be updated without any interference.
Quoted 7 lines
Signed-off-by Marek Vasut <[email protected]>Cc Joe Hershberger <[email protected]>Cc Patrice Chotard <[email protected]>Cc Patrick Delaunay <[email protected]>Cc Ramon Fried <[email protected]>Cc Stephen Warren <[email protected]>---