Deactivating TCP checksumming

DISCUSSION shesha bhushan 7 participants 14 messages Apr 1, 2003 – Apr 4, 2003

Share & tools

shesha bhushan

author

2003-04-01 12:00 UTC #0 lore raw

I get that. I can talk with the driver vendor. But to gain the usefulness of caculation of CSUM in HW we need to disable the software CSUM calculation in TCP layer in the kernel. Am I correct? I am trying to find that and I ma stuck there. How to disble the software TCP CSUM calculation? and later I can talk with driver vendor to enable it in hardware. I wanted help from linux gurus in disabling TCP csum calculation in the kernel.

Thanking You
Shesha

Quoted 45 lines

From: Matti Aarnio <[email protected]>
To: shesha bhushan <[email protected]>
Subject: Re: Deactivating TCP checksumming
Date: Tue, 1 Apr 2003 15:00:08 +0300
Quoted 4 lines · shesha bhushan
Ok I will. Is there any other material which I can reffer? I am using Intel pro1000 GBE
Thank you very much for providing me the below information.
You will need to talk with the driver author then. In 2.4.20 kernels (at least RedHat version) there is directory
drivers/net/e1000/
in which the driver code resides.
The e1000_main.c has:
        if(adapter->hw.mac_type >= e1000_82543) {
                netdev->features = NETIF_F_SG |
                                   NETIF_F_HW_CSUM |
                                   NETIF_F_HW_VLAN_TX |
                                   NETIF_F_HW_VLAN_RX |
                                   NETIF_F_HW_VLAN_FILTER;
        } else {
                netdev->features = NETIF_F_SG;
        }
... so, if your card isn't with that chip, then perhaps that is
the reason for not doing checksums in HW ?
Existence of that kind of test is telling to me, that "E1000" name is used to refer to a series of cards with varying properties. (Like there are a whole family of cards driven by 3c59x driver,
and even larger one referred as "tulip".)
/Matti Aarnio
Quoted 6 lines
Quoted 6 lines
From: Matti Aarnio <[email protected]>
To: shesha bhushan <[email protected]>
CC: [email protected], [email protected]
Subject: Re: Deactivating TCP checksumming
Date: Tue, 1 Apr 2003 13:58:21 +0300

Quoted 20 lines

Quoted 3 lines
Quoted 3 lines · shesha bhushan
Quoted 2 lines
Hello all,
I am trying to de-activate the TCP checksumming and allow hardware
(GBE
Quoted 16 lines
Quoted 16 lines
to
Quoted 1 line
compute it for me). But can any one let me know how to do it.
GBE ? Likely device feature flags are wrong -- See examples
from drivers/net/sunhme.c, acenic.c, tg3.c for various ways
to use NETIF_F_*_CSUM feature flags.
For (some of) explanations: include/linux/netdevice.h (for NETIF_F_* flags)
Quoted 4 lines
All suggestion are highly apperciated.
Thanking You
Shesha
/Matti Aarnio

_________________________________________________________________

Matti Aarnio

2003-04-01 12:17 UTC #1 lore raw

Quoted 6 lines · shesha bhushan

I get that. I can talk with the driver vendor. But to gain the usefulness of caculation of CSUM in HW we need to disable the software CSUM calculation in TCP layer in the kernel. Am I correct? I am trying to find that and I ma stuck there. How to disble the software TCP CSUM calculation? and later I can talk with driver vendor to enable it in hardware. I wanted help from linux gurus in disabling TCP csum calculation in the kernel.

The kernel code is already smart enough of detect that the outbound device will handle the checksum calculations all by itself, and not do it in that case.

Testing of dev->features is done in files:
net/core/dev.c
net/ipv4/tcp.c
(depending what protocol is in question.)
in the latter case, actually in common tcp path with route-cached route_caps flags.

I did
egrep 'NETIF_F_.._CSUM' net/*/*.c
to find those.
(and a number of other subset searches finding nothing)

Grep is your friend.

This whole "zero-copy" infastructure was implemented during development in 2.3 series.

Quoted 2 lines

Thanking You
Shesha

/Matti Aarnio

Dennis Cook

2003-04-02 20:20 UTC #2 lore raw

Using RH Linux kernel 2.4.18, setting "features" bit NETIF_F_IP_CSUM does not appear
to keep a valid IP checksum from being computed in packets presented to my driver
for transmission. So having HW compute outgoing checksum buys nothing. Checked this
by suppressing HW checksum computation. Packets are still accepted by peer.

Dennis Cook
Sandgate Technologies

"Matti Aarnio" <[email protected]> wrote in message news:[email protected]...

Quoted 2 lines · shesha bhushan

Quoted 1 line
I get that. I can talk with the driver vendor. But to gain the

usefulness

Quoted 2 lines

Quoted 2 lines
of caculation of CSUM in HW we need to disable the software CSUM calculation in TCP layer in the kernel. Am I correct? I am trying to

find

Quoted 1 line

Quoted 1 line
that and I ma stuck there. How to disble the software TCP CSUM

calculation?

Quoted 1 line

Quoted 1 line
and later I can talk with driver vendor to enable it in hardware. I

wanted

Quoted 27 lines

Quoted 1 line
help from linux gurus in disabling TCP csum calculation in the kernel.
The kernel code is already smart enough of detect that the outbound device will handle the checksum calculations all by itself, and not do it in that case.
Testing of dev->features is done in files:
   net/core/dev.c
   net/ipv4/tcp.c
(depending what protocol is in question.)
in the latter case, actually in common tcp path with route-cached route_caps flags.
I did
egrep 'NETIF_F_.._CSUM' net/*/*.c
to find those.
(and a number of other subset searches finding nothing)
Grep is your friend.
This whole "zero-copy" infastructure was implemented during development in 2.3 series.
Quoted 2 lines
Thanking You
Shesha
/Matti Aarnio

Jeff Garzik

2003-04-02 20:25 UTC #3 lore raw

Quoted 5 lines · Dennis Cook

Using RH Linux kernel 2.4.18, setting "features" bit NETIF_F_IP_CSUM does not appear
to keep a valid IP checksum from being computed in packets presented to my driver
for transmission. So having HW compute outgoing checksum buys nothing.

You are not using sendfile(2), which is required to activate h/w csum.

Jeff

Dennis Cook

2003-04-02 20:36 UTC #4 lore raw

What I was looking for is a general capability to keep the SW transport stack from
computing outgoing TCP/UDP/IP checksums so that the HW can be allowed to do it,
similar to Windows checksum offload capability.

"Jeff Garzik" <[email protected]> wrote in message news:[email protected]...

Quoted 2 lines · Dennis Cook

Quoted 1 line
Using RH Linux kernel 2.4.18, setting "features" bit NETIF_F_IP_CSUM

does

Quoted 2 lines

Quoted 2 lines
not appear
to keep a valid IP checksum from being computed in packets presented to

Quoted 9 lines

Quoted 2 lines
driver
for transmission. So having HW compute outgoing checksum buys nothing.
You are not using sendfile(2), which is required to activate h/w csum.
Jeff

Jeff Garzik

2003-04-02 20:47 UTC #5 lore raw

Quoted 5 lines · Dennis Cook

What I was looking for is a general capability to keep the SW transport stack from
computing outgoing TCP/UDP/IP checksums so that the HW can be allowed to do it,
similar to Windows checksum offload capability.

If you are not using sendfile(2), it is _more expensive_ to offload checksums, because we already checksum and copy at the same time.

Hardware checksum offload is only a win when a copy is eliminated.

Therefore, _always_ offloading checksum is actually slower in some cases, because of the unneeded additional HW csum setup that would be performed.

Jeff

Dennis Cook

2003-04-03 20:24 UTC #6 lore raw

Based on various feedback, on my RH Linux 2.4.18 kernel I tried the following:

Set "features" bit NETIF_F_IP_CSUM set (the only feature bit set). In my network driver start-transmit check for "CHECKSUM_HW" in ip_summed. Using a small test program, use "sendfile" to copy a file to a network socket FD.
Result is none of the packets presented to my network adapter driver have ip_summed set to CHECKSUM_HW, so the SW IP stack has already computed checksums.

Is this mechanism possibly broken on kernel 2.4?

"Jeff Garzik" <[email protected]> wrote in message news:[email protected]...

Quoted 4 lines · Dennis Cook

Quoted 3 lines
What I was looking for is a general capability to keep the SW transport stack from
computing outgoing TCP/UDP/IP checksums so that the HW can be allowed to

Quoted 15 lines

Quoted 2 lines
it,
similar to Windows checksum offload capability.
If you are not using sendfile(2), it is _more expensive_ to offload checksums, because we already checksum and copy at the same time.
Hardware checksum offload is only a win when a copy is eliminated.
Therefore, _always_ offloading checksum is actually slower in some cases, because of the unneeded additional HW csum setup that would be performed.
Jeff

Jeff Garzik

2003-04-03 20:35 UTC #7 lore raw

Quoted 10 lines · Dennis Cook

Based on various feedback, on my RH Linux 2.4.18 kernel I tried the following:
Set "features" bit NETIF_F_IP_CSUM set (the only feature bit set). In my network driver start-transmit check for "CHECKSUM_HW" in ip_summed. Using a small test program, use "sendfile" to copy a file to a network socket FD.
Result is none of the packets presented to my network adapter driver have ip_summed set to CHECKSUM_HW, so the SW IP stack has already computed checksums.

CHECKSUM_HW is for receive, not transmit. Read the comments at the top of include/linux/skbuff.h.

Quoted 1 line

Is this mechanism possibly broken on kernel 2.4?

it works quite well.

Jeff

Dennis Cook

2003-04-03 20:47 UTC #8 lore raw

In the 3c59x.c, e1000, and other adapter drivers, ip_summed is what is being checked for value CHECKSUM_HW when sending a packet.

Quoted 36 lines

-----Original Message-----
From: Jeff Garzik [mailto:[email protected]]
Sent: Thursday, April 03, 2003 03:47 PM
To: Dennis Cook
Cc: [email protected]; [email protected]
Subject: Re: Deactivating TCP checksumming
Quoted 5 lines · Dennis Cook
Based on various feedback, on my RH Linux 2.4.18 kernel I tried the following:
Set "features" bit NETIF_F_IP_CSUM set (the only feature bit set). In my network driver start-transmit check for "CHECKSUM_HW" in
ip_summed.
Quoted 3 lines
Using a small test program, use "sendfile" to copy a file to a network socket FD.
Result is none of the packets presented to my network adapter
driver have
Quoted 2 lines
ip_summed set to CHECKSUM_HW, so the SW IP stack has already computed checksums.
CHECKSUM_HW is for receive, not transmit. Read the comments at the top of include/linux/skbuff.h.
Quoted 1 line
Is this mechanism possibly broken on kernel 2.4?
it works quite well.
Jeff

Jeff Garzik

2003-04-04 14:15 UTC #9 lore raw

Quoted 3 lines · Dennis Cook

In the 3c59x.c, e1000, and other adapter drivers, ip_summed is what is being checked for value CHECKSUM_HW when sending a packet.

Yep; my mistake on that one.

Regardless, Ion and Manish picked up on what you were missed - the NETIF_F_SG bit. And if your hardware can do 64-bit DMA, you'll want to set NETIF_F_HIGHDMA too.

Jeff

Abhishek Agrawal

2003-04-04 14:38 UTC #10 lore raw

In reply to Jeff Garzik

Quoted 4 lines · Jeff Garzik

CHECKSUM_HW is for receive, not transmit. Read the comments at the top of include/linux/skbuff.h.

Actually CHECKSUM_HW can be set at either of the "producer" ends. At least this is what I gather from tcp_output.c

// tcp_output.c:454
if (!skb_shinfo(skb)->nr_frags && skb->ip_summed != CHECKSUM_HW) {
//...

        }
else {
                skb->ip_summed = CHECKSUM_HW;
                skb_split(skb, buff, len);
     }

AND

//1014 dev.c:dev_queue_xmit()

        /* If packet is not checksummed and device does not support
         * checksumming for this protocol, complete checksumming here.
         */
        if (skb->ip_summed == CHECKSUM_HW &&
            (!(dev->features&(NETIF_F_HW_CSUM|NETIF_F_NO_CSUM)) &&
             (!(dev->features&NETIF_F_IP_CSUM) ||
              skb->protocol != htons(ETH_P_IP)))) {
                if ((skb = skb_checksum_help(skb)) == NULL)
                        return -ENOMEM;
        }

Ion Badulescu

2003-04-04 04:31 UTC #11 lore raw

In reply to Dennis Cook

Quoted 11 lines · Dennis Cook

Based on various feedback, on my RH Linux 2.4.18 kernel I tried the following:
Set "features" bit NETIF_F_IP_CSUM set (the only feature bit set). In my network driver start-transmit check for "CHECKSUM_HW" in ip_summed. Using a small test program, use "sendfile" to copy a file to a network socket FD.
Result is none of the packets presented to my network adapter driver have ip_summed set to CHECKSUM_HW, so the SW IP stack has already computed checksums.

Quoted 1 line

Is this mechanism possibly broken on kernel 2.4?

No, but you also need the scatter-gather bit to be set. Otherwise the network needs to perform at least one copy to linearize the skb, thus it will take the opportunity to checksum it at the same time so the hardware capability of the board is not used.

Ion

Signature · 2 lines

  It is better to keep your mouth shut and be thought a fool,
            than to open it and remove all doubt.

Richard B. Johnson

2003-04-02 20:50 UTC #12 lore raw

In reply to Dennis Cook

Quoted 5 lines · Dennis Cook

What I was looking for is a general capability to keep the SW transport stack from
computing outgoing TCP/UDP/IP checksums so that the HW can be allowed to do it,
similar to Windows checksum offload capability.

REALLY? Who are you kidding. Windows has no such capability.

Check \WINDOWS\SYSTEM32\DRIVERS\ETC\* and see who they stole the TCP/IP stack from!

Further, when you perform normal user->TCP/IP operations, you get checksumming for free as part of the copy operation. It's only when you don't even copy data that you can get any advantage of not checksumming. That's why sendfile disables it.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.20 on an i686 machine (797.90 BogoMips). Why is the government concerned about the lunatic fringe? Think about it.

Dennis Cook

2003-04-02 21:11 UTC #13 lore raw

Re: Windows support of checksum offloading (not kidding).

Following from Windows DDK

===============================================

To achieve a significant performance boost, the Microsoft TCP/IP transport can offload one or more of the following tasks to a NIC that has the appropriate task-offload capabilities:

a.. Checksum tasks
The TCP/IP transport can offload the calculation and/or validation of IP
and/or TCP checksums. The initial release of Windows� 2000 does not support UDP checksum offloads; however, future service packs and update releases of Windows 2000 and later versions may support UDP checksum offloads.

===============================================

Win2K SP3 and WinXP both indicate to my driver that TCP and IP checksums are being offloaded
on packets to be sent provided the driver advertises that the associated HW is capable of computing
the checksums. I haven't established that the SW transport stack actually skips computing the checksums.

"Richard B. Johnson" <[email protected]> wrote in message news:Pine.LNX.4.53.0304021555160.32710@chaos...

Quoted 5 lines · Dennis Cook

Quoted 4 lines
What I was looking for is a general capability to keep the SW transport stack from
computing outgoing TCP/UDP/IP checksums so that the HW can be allowed to

Quoted 17 lines

Quoted 2 lines
it,
similar to Windows checksum offload capability.
REALLY? Who are you kidding. Windows has no such capability.
Check \WINDOWS\SYSTEM32\DRIVERS\ETC\* and see who they stole the TCP/IP stack from!
Further, when you perform normal user->TCP/IP operations, you get checksumming for free as part of the copy operation. It's only when you don't even copy data that you can get any advantage of not checksumming. That's why sendfile disables it.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.20 on an i686 machine (797.90 BogoMips). Why is the government concerned about the lunatic fringe? Think about it.