e1000e: use page pool
Get series with b4
b4 am [email protected]Versions newest first
- v1 Latest8 patches 5 repliesIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/8 | e1000e: add jumbo Rx CRC strippingdrivers/ | +11 | 3 replies | |
| 2/8 | e1000e: dump pages for jumbo Rx buffersdrivers/ | +16 | 0 replies | |
| 3/8 | e1000e: prevent race between PM and reset taskdrivers/ | +21 | 0 replies | |
| 4/8 | e1000e: remove packet-split Rx pathdrivers/ drivers/ drivers/ | +81 | 0 replies | |
| 5/8 | e1000e: always use jumbo Rx pathdrivers/ drivers/ | +18 | 0 replies | |
| 6/8 | e1000e: disable NAPI while interface is downdrivers/ | +3 | 2 replies | |
| 7/8 | e1000e: use libeth page_pool for RxDocumentation/ drivers/ drivers/ +2 more | +332 | 0 replies | |
| 8/8 | e1000e: return skbs to NAPI cachedrivers/ | +10 | 0 replies |
Reviews trailers per patch
1 of 8 patches has a Reviewed-by or Acked-by.
| Patch | Aleksandr Loktionov |
|---|---|
| 1/8 | R-b |
| 2/8 | |
| 3/8 | |
| 4/8 | |
| 5/8 | |
| 6/8 | |
| 7/8 | |
| 8/8 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH iwl-next 0/8] e1000e: use page pool
This series converges the three e1000e Rx paths into one and then makes it use a page pool. The jumbo path is chosen as the golden path because it is the only one that can handle every combination of MTU and page size. Any performance regression caused by using that path for all configurations is remedied by eliminating allocations and DMA mapping from the hot path, as shown in benchmarks below.
There are some general modernizations in this conversion, intended to make this driver match conventions found in other Intel drivers. The copybreak parameter is removed. The NAPI skb cache is used. The netmem API is used for chained frames. Prefetch is used optimistically in the cleaner loop to pull in the next descriptor and the frame headers.
User-visible changes:
* The copybreak module parameter and its documentation are removed.
* The rx_header_split ethtool stat is removed.
* Small packets are charged at the buffer's full 2 KB truesize at standard MTU.
* Jumbo MTU frames are delivered as 2 KB chunks. At MTU > 2026 this increases userspace copy overhead by gathering more fragments and increases truesize accounting per frame. This is described in more detail below.
The series begins by fixing some existing bugs which would either be more reachable or have worse consequences after the conversion. Missing support for CrcStripping=0 is added. The page dump routine's mis-read of jumbo buffers is remedied. A race between the reset task and runtime PM is guarded.
Then the standard and packet-split paths are removed, making the jumbo path the only way to Rx. At this point standard-MTU Rx allocates and maps a page per packet, until the page pool conversion removes that cost.
Next NAPI is disabled while the adapter is down, which is required to pass assertions when the page pool is destroyed.
Next is the main event, the singular Rx path is converted to use a page pool with support from libeth.
Finally, a small change to Tx to return skbs to the NAPI cache, which the Rx side now pulls from when constructing frame heads.
Benchmarks were run on an I218-V and Xeon E3-1240v3 pinned at 3.8GHz. NAPI was isolated on one core and iperf3 on another. ITR was fixed at 4000/s because the adaptive scaling can take a few minutes to settle (to be fixed separately). C-states above C1 and EEE were disabled. Kernel version was 7.1.12. Direct link from sender to receiver.
The sender was an 82574L in a Xeon E3-1270 pinned at 3.4GHz, NIC vectors and iperf3 -s pinned to separate cores, ITR fixed.
For cycle measurements, each cell is perf stat cycles on the core divided by packets received over 30s, averaged over three runs. IOMMU "lazy" is the default DMA-FQ. Throughput was identical between stock and series, with one exception noted below. The stock driver is configured at the default copybreak=256. Note that the stock copybreak features is flawed; it always remaps DMA when sending an already-mapped buffer back to the h/w.
Bulk TCP, segments sized to the MTU (line rate at both MTUs):
NAPI core cycles/pkt consumer core cycles/pkt
MTU IOMMU kpps stock series change stock series change 1500 off 81.4 2523 1784 -29% 2193 2193 0% 1500 lazy 81.4 5170 1938 -63% 2369 2323 -2% 9000 off 13.9 11590 7574 -35% 11764 12792 +9% 9000 lazy 13.9 21781 8423 -61% 12129 13396 +10%
The increase in consumer cost at MTU 9000 is probably because the frame arrives as five 2 KB chunks instead of three pieces from the legacy packet-split path. The copy to userspace walks more fragments and GRO merges fewer frames per skb.
TCP with an 88-byte MSS (iperf3 -M 88, ~142-byte frames):
NAPI core cycles/pkt consumer core cycles/pkt
MTU IOMMU kpps stock series change stock series change 1500 off 754.3 1453 793 -45% 446 484 +8% 1500 lazy 754.3 3819 809 -79% 387 496 +28% 9000 off 754.4 2231 780 -65% 498 509 +2% 9000 lazy 520/754 6920 798 -88% 430 523 +22%
At MTU 9000 with IOMMU on, the stock driver can't keep up with the stream at line rate. It polls continuously at 95% of the core and can't send descriptors back to h/w fast enough, so it sends pause frames and flow control throttles the sender.
Differences in consumer cost are likely explained by two things:
* The stock copybreak=256 leaves the payload in the NAPI core's cache.
* The high IOMMU cost in the stock driver causes it to return more bytes per read.
UDP, one full-size datagram per frame (1472/8972 bytes, line rate):
NAPI core cycles/pkt consumer core cycles/pkt
MTU IOMMU kpps stock series change stock series change 1500 off 81.4 4991 4417 -11% 6409 6381 0% 1500 lazy 81.4 7426 4397 -41% 6619 6395 -3% 9000 off 13.9 12070 8286 -31% 12861 13624 +6% 9000 lazy 13.9 21494 8289 -61% 12566 13772 +10%
Here we see a similar tax on the jumbo MTU consumer due to gathering more fragments.
UDP, 64-byte datagrams paced at 150 kpps:
NAPI core cycles/pkt consumer core cycles/pkt
MTU IOMMU kpps stock series change stock series change 1500 off 150.2 4202 3939 -6% 5541 5525 0% 1500 lazy 150.3 6663 3919 -41% 6151 5465 -11% 9000 off 150.2 4376 3956 -10% 5535 5428 -2% 9000 lazy 150.2 6821 3953 -42% 6154 5453 -11%
At MTU 9000, this rate of datagrams runs up against the socket receive buffer's limits in the series, causing a 0.003% drop rate. Because each datagram sits in a full page buffer, the default 213 KB socket buffer only holds 48 vs. stock's ~275. This can be remedied by increasing SO_RCVBUF or rmem_default. This is another example of copybreak truesize compression value in the stock driver.
Payload Density:
This table shows pieces per frame and density at every MTU breakpoint. Stock is the standard path at 1500 and packet-split above it.
stock (std / packet split) series (2 KB chunks)
MTU pieces truesize density pieces truesize density
<= 1500 1 2688 0.56 1 2304 0.66 1501-2026 1 4864 0.31-0.42 1 4352 0.35-0.47 2027-4074 1 4864 0.42-0.84 2 8448 0.24-0.48 4075-4096 1 4864 0.84 3 12544 0.33 4097-6122 2 8960 0.46-0.68 3 12544 0.33-0.49 6123-8170 2 8960 0.68-0.91 4 16640 0.37-0.49 8171-8192 2 8960 0.91-0.92 5 20736 0.39-0.40 8193-9212 3 13056 0.63-0.71 5 20736 0.40-0.44
Series chunk counts assume the worst-case frame (MTU + 22: VLAN tag and FCS present).
This series is hungrier for socket buffer size under two conditions:
* At higher-than-standard MTU, some page size is wasted because RCTL.BSIZE only accepts powers of two, unlike the more granular settings of later h/w.
* For small packets at any MTU, copybreak no longer moves the payload into an appropriately-sized buffer.
Matt Vollrath (8):
e1000e: add jumbo Rx CRC stripping
e1000e: dump pages for jumbo Rx buffers
e1000e: prevent race between PM and reset task
e1000e: remove packet-split Rx path
e1000e: always use jumbo Rx path
e1000e: disable NAPI while interface is down
e1000e: use libeth page_pool for Rx
e1000e: return skbs to NAPI cache
.../device_drivers/ethernet/intel/e1000e.rst | 15 - drivers/net/ethernet/intel/Kconfig | 1 + drivers/net/ethernet/intel/e1000e/e1000.h | 53 +- drivers/net/ethernet/intel/e1000e/ethtool.c | 1 - drivers/net/ethernet/intel/e1000e/netdev.c | 1374 +++++------------ drivers/net/ethernet/intel/e1000e/param.c | 6 - 6 files changed, 382 insertions(+), 1068 deletions(-)
base-commit: 1b78070aaef63512688aebfbc82365ef9d6660f1
Signature · 1 line
2.43.0