Skip to contentSkip to conversation

zerocopy patch against 2.4.2-pre2

UPDATE David S. Miller 3 participants 5 messages Feb 9, 2001 – Feb 14, 2001

lore
Share & tools
lore
#0 lore raw

As usual:

ftp://ftp.kernel.org/pub/linux/kernel/people/davem/zerocopy-2.4.2p2-1.diff.gz

It's updated to be against the latest (2.4.2-pre2) and I've removed the non-zerocopy related fixes from the patch (because I've sent them under seperate cover to Linus).

Enjoy. As usual, I am very seriously interested in any bugs or performance problems introduced by this patch. Thanks.

Later,
David S. Miller
[email protected]
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

#1 lore raw
Quoted 9 lines · David S. Miller

As usual:

ftp://ftp.kernel.org/pub/linux/kernel/people/davem/zerocopy-2.4.2p2-1.diff.gz

It's updated to be against the latest (2.4.2-pre2) and I've removed the non-zerocopy related fixes from the patch (because I've sent them under seperate cover to Linus).

Changing the memory copy function did make some difference in my setup. But the performance drop on send(8k) is only approx 10%, partly because I changed the way I'm testing it - `cyclesoak' is now penalised more heavily by cache misses, and amount of cache missing which networking causes cyclesoak is basically the same, whether or not the ZC patch is applied.

I tried a number of things to try to optimise this situation on an SG-capable NIC with the ZC patch:

	while (more_to_send) {
		read(fd, buf, 8192);
		send(sock, buf, 8192);
	}

Things I tried:

- Use the csum_copy() functions

- Use copy_from_user()

- Use copy_from_user if src and dest are 8-byte aligned, else use csum_copy.

- Force data alignment.

Explain: If an application sends a few bytes to a connection (say, some headers) and then starts pumping bulk data down the same connection, we end up in the situation where the source of a copy_from_user is poorly aligned, and it *stays* that way for the whole operation. This is because new, incoming data is always tacked onto the end of the socket write buffer.

Copying from a poorly aligned source address takes 1.5 to 2 times as long, depending upon the combination of source-cached and dest-cached.

So I special-cased this in tcp_sendmsg: if we see a large write from userspace and we're poorly aligned then just send out a single undersized frame so we can drop back into alignment.

This didn't make a lot of difference, which perhaps indicates that the dominating factor is misses, not alignment. If it _is_ misses, they're probably due to aliasing - Ingo said his toy has 2 megs of full-speed L2.

- skbuff_cache.

Explain: When we build an skbuff for ZC transmit it is always the same size - it only holds the headers. The data is put into the fragment buffer. So I created a slab cache for skbuffs whose data length is <= 256 bytes, and used that.

This didn't make much difference.

send(8k), no SG                                             19.2%
send(8k), SG, csum_copy                                     20.3%
send(8k), SG, copy_from_user                                20.9%
send(8k), SG, choose copy                                   20.6%     (huh?)
send(8k), SG, page-aligned, choose copy                     20.3%
send(8k), SG, page-aligned, csum_copy                       20.2%
send(8k), SG, csum_copy, skbuff_cache                       20.5%     (huh?)
send(8k), SG, csum_copy, skbuff_cache, page-aligned         20.2%
send(8k), SG, copy_from_user, skbuff_cache, page-aligned    20.2%

That's all pretty uninteresting, except for the observation that not using Pentium string ops on un-8byte-aligned is the biggest win. And the two huhs, the first of which is bizarre. I've checked that code over and over:

if (((long)_from | (long)_to) & 7)

	csum_and_copy()

else
copy_from_user()

and it's slower than an unconditional csum_and_copy(). Wierd.

The profiles are more interesting:

send(8k), no SG 18.2%
=========================================

c0224734 tcp_transmit_skb                             47   0.0347
c01127dc schedule                                     54   0.0340
c021599c ip_output                                    54   0.1688
c010a768 handle_IRQ_event                             55   0.4583
c02041ec skb_release_data                             60   0.5357
c0211068 ip_route_input                               69   0.1938
c022abac tcp_v4_rcv                                   75   0.0470
c0215adc ip_queue_xmit                                76   0.0571
c0204410 skb_clone                                    85   0.1986
c0219a54 tcp_sendmsg_copy                             99   0.0270
c02209fc tcp_clean_rtx_queue                         101   0.1153
c02042c4 __kfree_skb                                 113   0.3404
c024a3cc csum_partial_copy_generic                   436   1.7581
c0125580 file_read_actor                             548   6.5238
00000000 total                                      2874   0.0021

send(8k), SG, csum copy 20.3%
=========================================

c0211068 ip_route_input                               47   0.1320
c011be60 del_timer                                    49   0.6806
c021599c ip_output                                    49   0.1531
c010a768 handle_IRQ_event                             56   0.4667
c022abac tcp_v4_rcv                                   66   0.0414
c02041ec skb_release_data                             69   0.6161
c0215adc ip_queue_xmit                                69   0.0518
c0204410 skb_clone                                    70   0.1636
c02042c4 __kfree_skb                                  96   0.2892
c02209fc tcp_clean_rtx_queue                         100   0.1142
c021b6fc tcp_sendmsg                                 109   0.0439
c021a8ac do_tcp_sendpages                            152   0.0440
c024a3cc csum_partial_copy_generic                   520   2.0968
c0125580 file_read_actor                             615   7.3214
00000000 total                                      3222   0.0024

Note that in each and every profile which I've taken with the ZC patch, file_read_actor() took a big hit. That's
the read from the kernel into userspace. Which confirms
that ZC is more cache-hungry.

Also, note that the sum of ZC's do_tcp_sendpages and tcp_sendmsg is considerably higher than non-ZC's tcp_sendmsg_copy.

These profiles also show us the aggregate impact of the ZC patch: 3222/2874 = 1.12. Twelve percent. The device driver cost isn't shown here because it's in a module.

Here is a profile with a non-modular driver:

c020833c skb_release_data                             54   0.3857
c0218d4c ip_queue_xmit                                63   0.0497
c010a768 handle_IRQ_event                             70   0.5833
c024d4f8 __strncpy_from_user                          74   2.0556
c021e53c tcp_sendmsg                                  75   0.0302
c022d9ec tcp_v4_rcv                                   77   0.0482
c022383c tcp_clean_rtx_queue                          90   0.1027
c0208434 __kfree_skb                                 104   0.4000
c021d6ec do_tcp_sendpages                            122   0.0353
c01d3a08 boomerang_rx                                150   0.1276
c01d32ac boomerang_interrupt                         287   0.2781
c01d2c94 boomerang_start_xmit                        324   0.4475
c024d1cc csum_partial_copy_generic                   443   1.7863
c01255f0 file_read_actor                             643   7.6548
00000000 total                                      3852   0.0028

Device driver cost is 15-20% of the kernel occupancy.

We can pull ZC's 12% cost down a little bit by using a slab cache on the skbuffs, and by forcing better copy alignment (not sure how though). Also perhaps by doing some colouring on the data structures which are being used. These differences are all quite small, which makes this stuff pretty tricky.

I now need to go off and hunt down a Pentium performance counter patch which take less than a year to understand, find out where these alleged misses are happening.

-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

#2 lore raw
Quoted 6 lines · Andrew Morton

Changing the memory copy function did make some difference in my setup. But the performance drop on send(8k) is only approx 10%, partly because I changed the way I'm testing it - `cyclesoak' is now penalised more heavily by cache misses, and amount of cache missing which networking causes cyclesoak is basically the same, whether or not the ZC patch is applied.

Ok ok ok, but are we at the point where there are no sizable "over the wire" performance anomalies anymore? That is what is important, what are the localhost bandwidth measurements looking like for you now with/without the patch applied?

I want to reach a known state where we can conclude "over the wire is about as good or better than before, but there is a cpu/cache usage penalty from the zerocopy stuff".

This is important. It lets us get to the next stage which is to use your tools, numbers, and some profiling to see if we can get some of that cpu overhead back.

Later,
David S. Miller
[email protected]

#3 lore raw
In reply to Andrew Morton
Quoted 9 lines · David S. Miller
Quoted 6 lines · Andrew Morton

Changing the memory copy function did make some difference in my setup. But the performance drop on send(8k) is only approx 10%, partly because I changed the way I'm testing it - `cyclesoak' is now penalised more heavily by cache misses, and amount of cache missing which networking causes cyclesoak is basically the same, whether or not the ZC patch is applied.

Quoted 4 lines

Ok ok ok, but are we at the point where there are no sizable "over the wire" performance anomalies anymore? That is what is important, what are the localhost bandwidth measurements looking like for you now with/without the patch applied?

Using 2.4.2-pre3 + zerocopy-2.4.2p3-1.diff

All numbers in megs/sec

zcc/zcs is doing read(8k)/send(8k) to localhost.

On the dual 500MHz PII:

zcc/zcs bw_tcp

  Unpatched:       70                       66     
  Patched:         67                       66

Single 500MHz PII:

  Unpatched:       58                       54
  Patched:         49                       52 

Single 650MHz PIII Coppermine:

  Unpatched:       140                      180-250 
  Patched:         107                      159  

With or without ZC, there is Wierd Stuff happening with local networking. Throughput is all over the place.

- With zcs reporting throughput once per second, the numbers were jumping around by +/-10%. Had to bump the averaging period to 5 seconds to
make much sense of it. With a real network, they're rock solid.

- The difference between the PII and PIII is far beyond anything I see with any other workload.

- The difference between zcc/zcs and bw_tcp on the PIII is interesting. It's still apparent when zcc/zcs uses a 64k transfer buffer, like bw_tcp. zcc/zcs is doing file system reads, whereas bw_tcp isn't. But the discrepancy isn't there on the PII.

- On the unpatched kernel, I saw one bw_tcp run after a reboot report 410 Mbytes/sec. Thereafter it's around 210. err.. make that 180. No, make that 254. WTF?

Amongst all the noise it seems there's a problem on the PIII but not the PII.

It's getting very lonely testing this stuff. It would be useful if someone else could help out - at least running the bw_tcp tests. It's pretty simple:

bw_tcp -s ; bw_tcp 0

Quoted 7 lines

I want to reach a known state where we can conclude "over the wire is about as good or better than before, but there is a cpu/cache usage penalty from the zerocopy stuff".

This is important. It lets us get to the next stage which is to use your tools, numbers, and some profiling to see if we can get some of that cpu overhead back.

Seems, with the 100baseT NIC the performance drop on the Coppermine is only half that of the Mendocino. I _think_ the Mendocino is only 4-way associative, but reports vary on this. Coppermine is 8-way.

-

#4 lore raw
In reply to Andrew Morton
Quoted 6 lines · Andrew Morton

It's getting very lonely testing this stuff. It would be useful if someone else could help out - at least running the bw_tcp tests. It's pretty simple:

bw_tcp -s ; bw_tcp 0

OK, here's my bw_tcp results on a K6-2 450. I ran bw_tcp 10 times, then averaged the results.

bw_tcp
2.4.2-pre3 57.0
2.4.2-pre3zc 52.6

-Dave