sfc: Restrict PIO for 64bit arch in order to avoid data corruption
Subsystems (3)
cover sfc: Restrict PIO for 64bit arch in order to avoid data corruption
This patch series
The linux net driver uses memcpy_toio() in order to copy into the PIO buffers.
Even on a 64bit machine this causes 32bit accesses to a write- combined memory region.
There are hardware limitations that mean that only
64bit naturally aligned accesses are safe in all cases. Due to being write-combined memory region two 32bit accesses may be coalesced to form a 64bit non 64bit aligned access.
Solution was to open-code the memory copy routines using pointers and to only enable PIO for x86_64 machines.
This bug fix applies to v3.13 and v3.14 stable branches.
Jon Cooper (2):
sfc: use 64-bit writes for PIO.
sfc: Restrict PIO to 64-bit architectures
drivers/net/ethernet/sfc/io.h | 8 ++++++++ drivers/net/ethernet/sfc/tx.c | 24 +++++++++++++++++++----- 2 files changed, 27 insertions(+), 5 deletions(-)
The information contained in this message is confidential and is intended for the addressee(s) only. If you have received this message in error, please notify the sender immediately and delete the message. Unless you are an addressee (or authorized to receive for an addressee), you may not use, copy or disclose to anyone this message or any information contained in this message. The unauthorized use, disclosure, copying or alteration of this message is strictly prohibited.
PATCH 1/2 sfc: use 64-bit writes for PIO.
From: Jon Cooper <[email protected]>
The linux net driver uses memcpy_toio() in order to copy into the PIO buffers.
Even on a 64bit machine this causes 32bit accesses to a write- combined memory region.
There are hardware limitations that mean that only
64bit naturally aligned accesses are safe in all cases. Due to being write-combined memory region two 32bit accesses may be coalesced to form a 64bit non 64bit aligned access.
Solution was to open-code the memory copy routines using pointers and to only enable PIO for x86_64 machines.
Patch to open-code the memory copy routines. 32bit writes over the PCI bus causes data corruption.
orig-hg-hash: 853f313def1e5e9c733f980f2b4e6330a7d063ef
--- drivers/net/ethernet/sfc/tx.c | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-)
drivers/net/ethernet/sfc/tx.c +19 -5
@@ -189,6 +189,20 @@ struct efx_short_copy_buffer {u8 buf[L1_CACHE_BYTES ]; }; +/* Copy in explicit 64-bit writes. */ +static void efx_memcpy_64 (void *dest,void *src,size_t len) +{ +uint64_t *src64 = src, *dest64 = dest; +size_t i, l64 = len /8 ; + +WARN_ON_ONCE (len %8 !=0 ); +WARN_ON_ONCE (((u8 *)dest - (u8 *)0 ) %8 !=0 ); +BUILD_BUG_ON (sizeof (uint64_t ) !=8 ); + +for (i =0 ; i < l64; ++i) + dest64[i] = src64[i]; +} +/* Copy to PIO, respecting that writes to PIO buffers must be dword aligned. * Advances piobuf pointer. Leaves additional data in the copy buffer. */
@@ -198,7 +212,7 @@ static void efx_memcpy_toio_aligned(struct efx_nic *efx, u8 __iomem **piobuf, {int block_len = len & ~(sizeof (copy_buf->buf) -1 ); -memcpy_toio (*piobuf, data, block_len); +efx_memcpy_64 (*piobuf, data, block_len); *piobuf += block_len; len -= block_len;
@@ -230,7 +244,7 @@ static void efx_memcpy_toio_aligned_cb(struct efx_nic *efx, u8 __iomem **piobuf,if (copy_buf->used <sizeof (copy_buf->buf))return ; -memcpy_toio (*piobuf, copy_buf->buf,sizeof (copy_buf->buf)); +efx_memcpy_64 (*piobuf, copy_buf->buf,sizeof (copy_buf->buf)); *piobuf +=sizeof (copy_buf->buf); data += copy_to_buf; len -= copy_to_buf;
@@ -245,7 +259,7 @@ static void efx_flush_copy_buffer(struct efx_nic *efx, u8 __iomem *piobuf, {/* if there's anything in it, write the whole buffer, including junk */ if (copy_buf->used) -memcpy_toio (piobuf, copy_buf->buf,sizeof (copy_buf->buf)); +efx_memcpy_64 (piobuf, copy_buf->buf,sizeof (copy_buf->buf)); }/* Traverse skb structure and copy fragments in to PIO buffer.
@@ -304,8 +318,8 @@ efx_enqueue_skb_pio(struct efx_tx_queue *tx_queue, struct sk_buff *skb)*/ BUILD_BUG_ON (L1_CACHE_BYTES >SKB_DATA_ALIGN (sizeof (struct skb_shared_info ))); -memcpy_toio (tx_queue->piobuf, skb->data, -ALIGN (skb->len,L1_CACHE_BYTES )); +efx_memcpy_64 (tx_queue->piobuf, skb->data, +ALIGN (skb->len,L1_CACHE_BYTES )); } EFX_POPULATE_QWORD_5(buffer->option,
The information contained in this message is confidential and is intended for the addressee(s) only. If you have received this message in error, please notify the sender immediately and delete the message. Unless you are an addressee (or authorized to receive for an addressee), you may not use, copy or disclose to anyone this message or any information contained in this message. The unauthorized use, disclosure, copying or alteration of this message is strictly prohibited.
PATCH 2/2 sfc: Restrict PIO to 64-bit architectures
In reply to Shradha ShahFrom: Jon Cooper <[email protected]>
The linux net driver uses memcpy_toio() in order to copy into the PIO buffers.
Even on a 64bit machine this causes 32bit accesses to a write- combined memory region.
There are hardware limitations that mean that only
64bit naturally aligned accesses are safe in all cases. Due to being write-combined memory region two 32bit accesses may be coalesced to form a 64bit non 64bit aligned access.
Solution was to open-code the memory copy routines using pointers and to only enable PIO for x86_64 machines.
Enable PIO for x86_64 architecture only.
Not tested on platforms other than x86_64.
orig-hg-hash: 53dc43553d9bbe1c5a64c49e580ca571ddc470ae
--- drivers/net/ethernet/sfc/io.h | 8 ++++++++ 1 file changed, 8 insertions(+)
drivers/net/ethernet/sfc/io.h +8 -0
@@ -66,10 +66,18 @@#define EFX_USE_QWORD_IO 1#endif +/* PIO only works on 64-bit architectures */ +#if BITS_PER_LONG ==64 +/* not strictly necessary to restrict to x86 arch, but done for safety +* since unusual write combining behaviour can break PIO. +*/ +#ifdef CONFIG_X86_64 /* PIO is a win only if write-combining is possible */ #ifdef ARCH_HAS_IOREMAP_WC #define EFX_USE_PIO 1#endif +#endif +#endif #ifdef EFX_USE_QWORD_IO static inline void _efx_writeq(struct efx_nic *efx, __le64 value,
The information contained in this message is confidential and is intended for the addressee(s) only. If you have received this message in error, please notify the sender immediately and delete the message. Unless you are an addressee (or authorized to receive for an addressee), you may not use, copy or disclose to anyone this message or any information contained in this message. The unauthorized use, disclosure, copying or alteration of this message is strictly prohibited.
David Miller
maintainerFrom: Shradha Shah <[email protected]>
Date: Tue, 20 May 2014 08:44:45 +0100
Quoted 1 line
The information contained in this message is confidential and is intended for the addressee(s) only. If you have received this message in error, please notify the sender immediately and delete the message. Unless you are an addressee (or authorized to receive for an addressee), you may not use, copy or disclose to anyone this message or any information contained in this message. The unauthorized use, disclosure, copying or alteration of this message is strictly prohibited.
With this notification, I cannot read your email, and furthermore they make absolutely no sense at all on a public mailing list.
Please disable this notice and resubmit your patches if you want me to apply them.
Thank you.