Skip to contentSkip to conversation

NETIF_F_SG question

DISCUSSION Joakim Tjernlund 4 participants 10 messages Feb 1, 2003 – Feb 3, 2003

lore
Share & tools
lore
#0 lore raw

I am thinking of implementing harware scatter/gatter support( NETIF_F_SG) in my ethernet driver. The network device cannot do HW checksuming.

Will the IP stack make use of the SG support and will there be any significant performance improvement?

Jocke

#1 lore raw
Quoted 4 lines · Joakim Tjernlund

I am thinking of implementing harware scatter/gatter support( NETIF_F_SG) in my ethernet driver. The network device cannot do HW checksuming.

Will the IP stack make use of the SG support and will there be any significant performance improvement?

No; you need HW checksumming for NETIF_F_SG to be useful.

If HW checksumming is not available, scatter-gather is useless, because the net stack must always make a pass over the data to checksum it. Since it must do that, it can linearize the skb at the same time, eliminating the need for SG.

Jeff

#2 lore raw
Quoted 8 lines
Quoted 5 lines · Joakim Tjernlund

I am thinking of implementing harware scatter/gatter support( NETIF_F_SG) in my ethernet driver. The network device cannot do HW checksuming.

Will the IP stack make use of the SG support and will there be any significant performance improvement?

Quoted 8 lines

No; you need HW checksumming for NETIF_F_SG to be useful.

If HW checksumming is not available, scatter-gather is useless, because the net stack must always make a pass over the data to checksum it. Since it must do that, it can linearize the skb at the same time, eliminating the need for SG.

Jeff

linearize = copy small buffers inte one big buffer, or? Surley the copy operation will cost something?

Jocke

#4 lore raw
Quoted 10 lines

Joakim Tjernlund <[email protected]> :
[...]

Quoted 1 line

Surley the copy operation will cost something?

Cost is hidden while checksumming.

--
Ueimor

Yes, I realize that you can make the cost much smaller by checksumming and copying at the same time. However for every word that is checksummed one must execute an extra store. That store will also hit the L1 cache, flushing more important data to memory, so maybe the cost is visible?

Jocke

#5 lore raw
In reply to Jeff Garzik
Quoted 8 lines
Quoted 5 lines · Joakim Tjernlund

I am thinking of implementing harware scatter/gatter support( NETIF_F_SG) in my ethernet driver. The network device cannot do HW checksuming.

Will the IP stack make use of the SG support and will there be any significant performance improvement?

Quoted 8 lines

No; you need HW checksumming for NETIF_F_SG to be useful.

If HW checksumming is not available, scatter-gather is useless, because the net stack must always make a pass over the data to checksum it. Since it must do that, it can linearize the skb at the same time, eliminating the need for SG.

Jeff

I think HW checksumming and SG are independent. Either one of them should not require the other one in any context.

Zero copy sendfile() does not require HW checksum to do zero copy, right? If HW checksum is present, then you get some extra performance as a bonus.

(hmm, one could make SG mandatory and the devices that don't support it can
implement it in their driver. Just an idea)

Jocke

#6 lore raw
Quoted 3 lines · Joakim Tjernlund

I think HW checksumming and SG are independent. Either one of them should not require the other one in any context.

They should be independent in general, but they aren't when the particular case of TCP/IPv4 is concerned.

Quoted 1 line

Zero copy sendfile() does not require HW checksum to do zero copy, right?

Wrong...

Quoted 1 line

If HW checksum is present, then you get some extra performance as a bonus.

You get zerocopy, yes. :-) No HW cksum, no zerocopy.

Don't let this stop you, however. It's always possible that other networking stacks will eventually make use of SG while not requiring HW TCP/UDP cksums. None of them do right now, but...

Quoted 2 lines

(hmm, one could make SG mandatory and the devices that don't support it can implement it in their driver. Just an idea)

Not really, that way lies driver madness. The less complexity in the driver, the better.

Ion
[starfire driver maintainer]

Signature · 2 lines
  It is better to keep your mouth shut and be thought a fool,
            than to open it and remove all doubt.
#7 lore raw
Quoted 8 lines · Joakim Tjernlund
Quoted 3 lines

I think HW checksumming and SG are independent. Either one of them should not require the other one in any context.

They should be independent in general, but they aren't when the particular case of TCP/IPv4 is concerned.

Quoted 7 lines
Quoted 1 line

Zero copy sendfile() does not require HW checksum to do zero copy, right?

Wrong...

Quoted 1 line

If HW checksum is present, then you get some extra performance as a bonus.

You get zerocopy, yes. :-) No HW cksum, no zerocopy.

OK, but it should be easy to remove HW cksum as a condition to do zerocopy?

Quoted 4 lines

Don't let this stop you, however. It's always possible that other networking stacks will eventually make use of SG while not requiring HW TCP/UDP cksums. None of them do right now, but...

zerocopy without requiring HW cksums only OR could for instance the forwarding procdure also benefit from SG without requiring HW cksums?

Quoted 6 lines
Quoted 3 lines

(hmm, one could make SG mandatory and the devices that don't support it can implement it in their driver. Just an idea)

Not really, that way lies driver madness. The less complexity in the driver, the better.

Just a wild idea, forget it. You are right

Joakim

Quoted 3 lines

Ion
[starfire driver maintainer]

#8 lore raw
Quoted 3 lines · Joakim Tjernlund
Quoted 1 line

You get zerocopy, yes. :-) No HW cksum, no zerocopy.

OK, but it should be easy to remove HW cksum as a condition to do zerocopy?

Nope. You're looking at this the wrong way: the goal is not zero copy, but zero data access by CPU. Once you realize that, it's clear that SG alone is no good.

This is not necessarily the only approach, but it is the current approach in the Linux IPv4 stack. It's not worth the effort to re-engineer the code in order to support the fast-disappearing hardware which supports SG but not cksums.

Quoted 2 lines

zerocopy without requiring HW cksums only OR could for instance the forwarding procdure also benefit from SG without requiring HW cksums?

The forwarding procedure is already dealing with linear buffers because 99.99% of the network cards on the market receive packets into one linear buffer. So again SG is useless for that.

Ion

Signature · 2 lines
  It is better to keep your mouth shut and be thought a fool,
            than to open it and remove all doubt.
#9 lore raw
Quoted 6 lines · Joakim Tjernlund
Quoted 4 lines
Quoted 2 lines

You get zerocopy, yes. :-) No HW cksum, no zerocopy.

OK, but it should be easy to remove HW cksum as a condition to do zerocopy?

Quoted 8 lines

Nope. You're looking at this the wrong way: the goal is not zero copy, but zero data access by CPU. Once you realize that, it's clear that SG alone is no good.

This is not necessarily the only approach, but it is the current approach in the Linux IPv4 stack. It's not worth the effort to re-engineer the code in order to support the fast-disappearing hardware which supports SG but not cksums.

Agreed.

Quoted 7 lines
Quoted 3 lines

zerocopy without requiring HW cksums only OR could for instance the forwarding procdure also benefit from SG without requiring HW cksums?

The forwarding procedure is already dealing with linear buffers because 99.99% of the network cards on the market receive packets into one linear buffer. So again SG is useless for that.

I see, thanks for your patience with me.

Joakim

Quoted 2 lines

Ion