Skip to contentSkip to conversation

pktgen: tricks

PATCH In review Stephen Hemminger 5 participants 11 messages Sep 23, 2009 – Oct 1, 2009

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

I thought others want to know how to get maximum speed of pktgen.

1. Run nothing else (even X11), just a command line
2. Make sure ethernet flow control is disabled
ethtool -A eth0 autoneg off rx off tx off
3. Make sure clocksource is TSC. On my old SMP Opteron's needed to get patch since in 2.6.30 or later, the clock guru's decided to remove it on all non Intel machines. Look for patch than enables "tsc=reliable"
4. Compile Ethernet drivers in, the overhead of the indirect function call required for modules (or cache footprint), slows things down.
5. Increase transmit ring size to 1000
ethtool -G eth0 tx 1000

Result: OK: 70408581(c70405979+d2602) nsec, 100000000 (60byte,0frags)
1420281pps 681Mb/sec (681734880bps) errors: 0

#1 lore raw
Quoted 15 lines · On Tue, 22 Sep 2009 22:49:02 -0700 Stephen Hemminger <[email protected]>…

I thought others want to know how to get maximum speed of pktgen.

1. Run nothing else (even X11), just a command line
2. Make sure ethernet flow control is disabled
ethtool -A eth0 autoneg off rx off tx off
3. Make sure clocksource is TSC. On my old SMP Opteron's needed to get patch since in 2.6.30 or later, the clock guru's decided to remove it on all non Intel machines. Look for patch than enables "tsc=reliable"
4. Compile Ethernet drivers in, the overhead of the indirect function call required for modules (or cache footprint), slows things down.
5. Increase transmit ring size to 1000
ethtool -G eth0 tx 1000

Quoted 2 lines

Result: OK: 70408581(c70405979+d2602) nsec, 100000000 (60byte,0frags)
1420281pps 681Mb/sec (681734880bps) errors: 0

Other kernel config help:
- turn off lock dependency checker, kmecheck, page alloc debug basically anything that slows stuff down
- turn off content group scheduler

#2 lore raw
Quoted 19 lines · Stephen Hemminger
Quoted 15 lines · On Tue, 22 Sep 2009 22:49:02 -0700 Stephen Hemminger <[email protected]>…

I thought others want to know how to get maximum speed of pktgen.

1. Run nothing else (even X11), just a command line
2. Make sure ethernet flow control is disabled
ethtool -A eth0 autoneg off rx off tx off
3. Make sure clocksource is TSC. On my old SMP Opteron's needed to get patch since in 2.6.30 or later, the clock guru's decided to remove it on all non Intel machines. Look for patch than enables "tsc=reliable"
4. Compile Ethernet drivers in, the overhead of the indirect function call required for modules (or cache footprint), slows things down.
5. Increase transmit ring size to 1000
ethtool -G eth0 tx 1000

Quoted 8 lines
Quoted 2 lines

Result: OK: 70408581(c70405979+d2602) nsec, 100000000 (60byte,0frags)
1420281pps 681Mb/sec (681734880bps) errors: 0

Other kernel config help:
- turn off lock dependency checker, kmecheck, page alloc debug basically anything that slows stuff down
- turn off content group scheduler

I and thought netperf was getting away from real-world?-)

rick jones

#3 lore raw
In reply to Stephen Hemminger
Quoted 4 lines · On Thursday 24 September 2009 03:41:41 Stephen Hemminger wrote:

Other kernel config help:
- turn off lock dependency checker, kmecheck, page alloc debug basically anything that slows stuff down
- turn off content group scheduler

Maybe, but i'm not sure (i can't test it):
Disable randomize VA space? On embedded boards it was helping.
In some case disabling SMP helped, when various SMP locks involved, but maybe not for pktgen.

Quoted 4 lines

--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw
Quoted 11 lines · Denys Fedoryschenko
Quoted 4 lines · On Thursday 24 September 2009 03:41:41 Stephen Hemminger wrote:

Other kernel config help:
- turn off lock dependency checker, kmecheck, page alloc debug basically anything that slows stuff down
- turn off content group scheduler

Maybe, but i'm not sure (i can't test it):
Disable randomize VA space? On embedded boards it was helping.
In some case disabling SMP helped, when various SMP locks involved, but maybe not for pktgen.

pktgen is a kernel module, and is not affected by randomize VA space.

But of course, disabling SMP must help, as long as your machine needs one cpu only :)

#5 lore raw

PATCH 1/1 pktgen: Avoid dirtying skb->users when txq is full

In reply to Stephen Hemminger
Quoted 5 lines · Stephen Hemminger
Quoted 2 lines · On Tue, 22 Sep 2009 22:49:02 -0700 Stephen Hemminger <[email protected]>…

I thought others want to know how to get maximum speed of pktgen.

Quoted 13 lines
Quoted 4 lines

1. Run nothing else (even X11), just a command line
2. Make sure ethernet flow control is disabled ethtool -A eth0 autoneg off rx off tx off
3. Make sure clocksource is TSC. On my old SMP Opteron's

Quoted 9 lines

needed to get patch since in 2.6.30 or later, the clock guru's decided to remove it on all non Intel machines. Look for patch than enables "tsc=reliable"
4. Compile Ethernet drivers in, the overhead of the indirect function call required for modules (or cache footprint), slows things down.
5. Increase transmit ring size to 1000
ethtool -G eth0 tx 1000

Thanks a lot Stephen.

I did some pktgen session tonight and found one contention on skb->users field that following patch avoids.

Before patch :
------------------------------------------------------------------------------
PerfTop: 5187 irqs/sec kernel:100.0% [100000 cycles], (all, cpu: 0)
------------------------------------------------------------------------------

             samples    pcnt   kernel function
             _______   _____   _______________

            16688.00 - 50.9% : consume_skb
             6541.00 - 20.0% : skb_dma_unmap
             3277.00 - 10.0% : tg3_poll
             1968.00 -  6.0% : mwait_idle
              651.00 -  2.0% : irq_entries_start
              466.00 -  1.4% : _spin_lock
              442.00 -  1.3% : mix_pool_bytes_extract
              373.00 -  1.1% : tg3_msi
              353.00 -  1.1% : read_tsc
              177.00 -  0.5% : sched_clock_local
              176.00 -  0.5% : sched_clock
              137.00 -  0.4% : tick_nohz_stop_sched_tick

After patch:
------------------------------------------------------------------------------
PerfTop: 3530 irqs/sec kernel:99.9% [100000 cycles], (all, cpu: 0)
------------------------------------------------------------------------------

             samples    pcnt   kernel function
             _______   _____   _______________

            17127.00 - 34.0% : tg3_poll
            12691.00 - 25.2% : consume_skb
             5299.00 - 10.5% : skb_dma_unmap
             4179.00 -  8.3% : mwait_idle
             1583.00 -  3.1% : irq_entries_start
             1288.00 -  2.6% : mix_pool_bytes_extract
             1239.00 -  2.5% : tg3_msi
             1062.00 -  2.1% : read_tsc
              583.00 -  1.2% : _spin_lock
              432.00 -  0.9% : sched_clock
              416.00 -  0.8% : sched_clock_local
              360.00 -  0.7% : tick_nohz_stop_sched_tick
              329.00 -  0.7% : ktime_get
              263.00 -  0.5% : _spin_lock_irqsave

I believe we could go further, batching the atomic_inc(&skb->users) we do all the time, competing with the atomic_dec() done by tx completion handler (possibly run on other cpu): Reserve XXXXXXX units to the skb->users, and decrement a pktgen local variable and refill the reserve if necessary, once in a while...

[PATCH] pktgen: Avoid dirtying skb->users when txq is full

We can avoid two atomic ops on skb->users if packet is not going to be sent to the device (because hardware txqueue is full)

---

net/core/pktgen.c +7 -4
index 4d11c28..6a9ab28 100644
@@ -3439,12 +3439,14 @@ static void pktgen_xmit(struct pktgen_dev *pkt_dev)
 	txq = netdev_get_tx_queue(odev, queue_map);
 
 	__netif_tx_lock_bh(txq);
-	atomic_inc(&(pkt_dev->skb->users));
 
-	if (unlikely(netif_tx_queue_stopped(txq) || netif_tx_queue_frozen(txq)))
+	if (unlikely(netif_tx_queue_stopped(txq) || netif_tx_queue_frozen(txq))) {
 		ret = NETDEV_TX_BUSY;
-	else
-		ret = (*xmit)(pkt_dev->skb, odev);
+		pkt_dev->last_ok = 0;
+		goto unlock;
+	}
+	atomic_inc(&(pkt_dev->skb->users));
+	ret = (*xmit)(pkt_dev->skb, odev);
 
 	switch (ret) {
 	case NETDEV_TX_OK:
@@ -3466,6 +3468,7 @@ static void pktgen_xmit(struct pktgen_dev *pkt_dev)
 		atomic_dec(&(pkt_dev->skb->users));
 		pkt_dev->last_ok = 0;
 	}
+unlock:
 	__netif_tx_unlock_bh(txq);
 
 	/* If pkt_dev->count is zero, then run forever */
author Acked-by on v1 1/1
#6 lore raw

Re: [PATCH] pktgen: Avoid dirtying skb->users when txq is full

Quoted 70 lines · On Thu, 01 Oct 2009 01:03:33 +0200 Eric Dumazet <[email protected]> wrote:
Quoted 5 lines · Stephen Hemminger
Quoted 2 lines · On Tue, 22 Sep 2009 22:49:02 -0700 Stephen Hemminger <[email protected]>…

I thought others want to know how to get maximum speed of pktgen.

Quoted 13 lines
Quoted 4 lines

1. Run nothing else (even X11), just a command line
2. Make sure ethernet flow control is disabled ethtool -A eth0 autoneg off rx off tx off
3. Make sure clocksource is TSC. On my old SMP Opteron's

Quoted 9 lines

needed to get patch since in 2.6.30 or later, the clock guru's decided to remove it on all non Intel machines. Look for patch than enables "tsc=reliable"
4. Compile Ethernet drivers in, the overhead of the indirect function call required for modules (or cache footprint), slows things down.
5. Increase transmit ring size to 1000
ethtool -G eth0 tx 1000

Thanks a lot Stephen.

I did some pktgen session tonight and found one contention on skb->users field that following patch avoids.

Before patch :
------------------------------------------------------------------------------
PerfTop: 5187 irqs/sec kernel:100.0% [100000 cycles], (all, cpu: 0)
------------------------------------------------------------------------------

             samples    pcnt   kernel function
             _______   _____   _______________

            16688.00 - 50.9% : consume_skb
             6541.00 - 20.0% : skb_dma_unmap
             3277.00 - 10.0% : tg3_poll
             1968.00 -  6.0% : mwait_idle
              651.00 -  2.0% : irq_entries_start
              466.00 -  1.4% : _spin_lock
              442.00 -  1.3% : mix_pool_bytes_extract
              373.00 -  1.1% : tg3_msi
              353.00 -  1.1% : read_tsc
              177.00 -  0.5% : sched_clock_local
              176.00 -  0.5% : sched_clock
              137.00 -  0.4% : tick_nohz_stop_sched_tick

After patch:
------------------------------------------------------------------------------
PerfTop: 3530 irqs/sec kernel:99.9% [100000 cycles], (all, cpu: 0)
------------------------------------------------------------------------------

             samples    pcnt   kernel function
             _______   _____   _______________

            17127.00 - 34.0% : tg3_poll
            12691.00 - 25.2% : consume_skb
             5299.00 - 10.5% : skb_dma_unmap
             4179.00 -  8.3% : mwait_idle
             1583.00 -  3.1% : irq_entries_start
             1288.00 -  2.6% : mix_pool_bytes_extract
             1239.00 -  2.5% : tg3_msi
             1062.00 -  2.1% : read_tsc
              583.00 -  1.2% : _spin_lock
              432.00 -  0.9% : sched_clock
              416.00 -  0.8% : sched_clock_local
              360.00 -  0.7% : tick_nohz_stop_sched_tick
              329.00 -  0.7% : ktime_get
              263.00 -  0.5% : _spin_lock_irqsave
Quoted 4 lines

I believe we could go further, batching the atomic_inc(&skb->users) we do all the time, competing with the atomic_dec() done by tx completion handler (possibly run on other cpu): Reserve XXXXXXX units to the skb->users, and decrement a pktgen local variable and refill the reserve if necessary, once in a while...

When buffer is allocated we know the number of times it will be cloned. So why not set it there, would need to cleanup on interrupt but that should be possible.

Alternatively, just change skb->destructor on last packet and use a proper completion mechanism.

Quoted 32 lines

[PATCH] pktgen: Avoid dirtying skb->users when txq is full

We can avoid two atomic ops on skb->users if packet is not going to be sent to the device (because hardware txqueue is full)

---

net/core/pktgen.c +6 -4
index 4d11c28..6a9ab28 100644
@@ -3439,12 +3439,14 @@ static void pktgen_xmit(struct pktgen_dev *pkt_dev)
 	txq = netdev_get_tx_queue(odev, queue_map);
 
 	__netif_tx_lock_bh(txq);
-	atomic_inc(&(pkt_dev->skb->users));
 
-	if (unlikely(netif_tx_queue_stopped(txq) || netif_tx_queue_frozen(txq)))
+	if (unlikely(netif_tx_queue_stopped(txq) || netif_tx_queue_frozen(txq))) {
 		ret = NETDEV_TX_BUSY;
-	else
-		ret = (*xmit)(pkt_dev->skb, odev);
+		pkt_dev->last_ok = 0;
+		goto unlock;
+	}
+	atomic_inc(&(pkt_dev->skb->users));
+	ret = (*xmit)(pkt_dev->skb, odev);
 
 	switch (ret) {
 	case NETDEV_TX_OK:

net/core/pktgen.c static void pktgen_xmit(struct pktgen_dev *pkt_dev) · line 3474

Quoted 7 lines
net/core/pktgen.c +1 -0
@@ -3466,6 +3468,7 @@ static void pktgen_xmit(struct pktgen_dev *pkt_dev)
 		atomic_dec(&(pkt_dev->skb->users));
 		pkt_dev->last_ok = 0;
 	}
+unlock:
 	__netif_tx_unlock_bh(txq);
 
 	/* If pkt_dev->count is zero, then run forever */
#7 lore raw

PATCH 1/1 pktgen: Fix delay handling

After last pktgen changes, delay handling is wrong.

pktgen actually sends packets at full line speed.

Fix is to update pkt_dev->next_tx even if spin() returns early, so that next spin() calls have a chance to see a positive delay.

---
 net/core/pktgen.c |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)
net/core/pktgen.c +4 -2
index 0bcecbf..1a0682e 100644
@@ -2106,15 +2106,17 @@ static void pktgen_setup_inject(struct pktgen_dev *pkt_dev)
 static void spin(struct pktgen_dev *pkt_dev, ktime_t spin_until)
 {
 	ktime_t start;
-	s32 remaining;
+	s64 remaining;
 	struct hrtimer_sleeper t;
 
 	hrtimer_init_on_stack(&t.timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
 	hrtimer_set_expires(&t.timer, spin_until);
 
 	remaining = ktime_to_us(hrtimer_expires_remaining(&t.timer));
-	if (remaining <= 0)
+	if (remaining <= 0) {
+		pkt_dev->next_tx = ktime_add_ns(spin_until, pkt_dev->delay);
 		return;
+	}
 
 	start = ktime_now();
 	if (remaining < 100)
#8 lore raw

Re: [PATCH] pktgen: Fix delay handling

Quoted 8 lines · Eric Dumazet

After last pktgen changes, delay handling is wrong.

pktgen actually sends packets at full line speed.

Fix is to update pkt_dev->next_tx even if spin() returns early, so that next spin() calls have a chance to see a positive delay.

Oh well, I hit this bug on linux-2.6 git tree, but I did the patch on net-next-2.6

But it appears net/core/pktgen.c is different on net-next-2.6

Stephen, David, I am a bit lost here, something went wrong in a merge process ?

In any case, here is the patch against Linus tree, where bug is present.

Thanks

[PATCH] pktgen: Fix delay handling

After last pktgen changes, delay handling is wrong.

pktgen actually sends packets at full line speed.

Fix is to update pkt_dev->next_tx even if spin() returns early, so that next spin() calls have a chance to see a positive delay.

---

net/core/pktgen.c +4 -2
index 4d11c28..b694552 100644
@@ -2105,15 +2105,17 @@ static void pktgen_setup_inject(struct pktgen_dev *pkt_dev)
 static void spin(struct pktgen_dev *pkt_dev, ktime_t spin_until)
 {
 	ktime_t start_time, end_time;
-	s32 remaining;
+	s64 remaining;
 	struct hrtimer_sleeper t;
 
 	hrtimer_init_on_stack(&t.timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
 	hrtimer_set_expires(&t.timer, spin_until);
 
 	remaining = ktime_to_us(hrtimer_expires_remaining(&t.timer));
-	if (remaining <= 0)
+	if (remaining <= 0) {
+		pkt_dev->next_tx = ktime_add_ns(spin_until, pkt_dev->delay);
 		return;
+	}
 
 	start_time = ktime_now();
 	if (remaining < 100)
maintainer
#9 lore raw

Re: [PATCH] pktgen: Fix delay handling

From: Eric Dumazet <[email protected]>
Date: Thu, 01 Oct 2009 12:04:41 +0200

Quoted 4 lines

But it appears net/core/pktgen.c is different on net-next-2.6

Stephen, David, I am a bit lost here, something went wrong in a merge process ?

net-next-2.6 is just a stale old tree, there is no new networking work in there and it is simply Linus's tree as of a few weeks ago.

It's only there so Stephen Rothwell has something to do a 'nop' pull from into his linux-next tree.

I'll apply your fix, thanks!

#10 lore raw

Re: [PATCH] pktgen: Fix delay handling

Quoted 8 lines · David Miller

net-next-2.6 is just a stale old tree, there is no new networking work in there and it is simply Linus's tree as of a few weeks ago.

It's only there so Stephen Rothwell has something to do a 'nop' pull from into his linux-next tree.

I'll apply your fix, thanks!

Thanks for the explanation David.