jumbo frames and memory fragmentation
Subsystems (2)
I'm running a system with multiple e1000 devices, using 9KB jumbo frames. I'm running a modified 2.6.10 with e1000 driver 5.5.4-k2.
I'm a bit concerned about the behaviour of this driver with jumbo frames. We ask for 9KB. The driver then bumps that up to a power-of-two, so it calls dev_alloc_skb(16384). That then bumps it up a bit to allow for its own overhead, so it appears that we end up asking for 32KB of physically contiguous memory for every packet coming in. Ouch.
Add to that the fact that this version of the driver doesn't do copybreak, and it means that after we're up for a few days it starts complaining about not being able to allocate buffers.
Anyone have any suggestions on how to improve this? Upgrading kernels isn't an option. I could port back the copybreak stuff fairly easily.
Back in 2.4 some of the drivers used to retry buffer allocations using GFP_KERNEL once interrupts were reenabled. I don't see many of them doing that anymore--would there be any benefit to that?
Thanks,
Chris
Quoted 3 lines · Chris Friesen
Anyone have any suggestions on how to improve this? Upgrading kernels isn't an option. I could port back the copybreak stuff fairly easily.
Either upgrade your kernel or backport the page-splitting code in the current tree. That's really the only sane solution for jumbo packets.
Cheers,
Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txtQuoted 2 lines · Herbert Xu
Either upgrade your kernel or backport the page-splitting code in the current tree. That's really the only sane solution for jumbo packets.
Looking at the page-splitting code, it says "82571 and greater support packet-split...". We're running the 82546GB device. Looks like it won't help me.
Chris
Quoted 6 lines · Chris Friesen
Quoted 2 lines
Either upgrade your kernel or backport the page-splitting code in the current tree. That's really the only sane solution for jumbo packets.
Looking at the page-splitting code, it says "82571 and greater support packet-split...". We're running the 82546GB device. Looks like it won't help me.
It definitely will.
Packet split in hardware means separating data and headers into different pages in different reads, while software page split means that skb has a list of fragments where part of the packet will be DMAed, so jumbo frame will be converted into several pages.
Signature · 1 line
Evgeniy Polyakov
Quoted 4 lines · Evgeniy Polyakov
It definitely will.
Packet split in hardware means separating data and headers into
different pages in different reads, while software page split means that skb has a list of fragments where part of the packet will be DMAed, so jumbo frame will be converted into several pages.
Maybe I'm looking at the wrong code then. Can you point me to where this "software page split" is handled?
Chris
Quoted 10 lines · Chris Friesen
Quoted 5 lines · Evgeniy Polyakov
It definitely will.
Packet split in hardware means separating data and headers into different pages in different reads, while software page split means that skb has a list of fragments where part of the packet will be DMAed, so jumbo frame will be converted into several pages.Maybe I'm looking at the wrong code then. Can you point me to where this "software page split" is handled?
You're right, it isn't supported for your hardware. to solve the problem (I got this from mailing lists) you probably need to adjust the /proc/sys/vm/min_free_kbytes
let us know if this helps for you.
Quoted 10 lines · Chris Friesen
Quoted 5 lines · Evgeniy Polyakov
It definitely will.
Packet split in hardware means separating data and headers into different pages in different reads, while software page split means that skb has a list of fragments where part of the packet will be DMAed, so jumbo frame will be converted into several pages.Maybe I'm looking at the wrong code then. Can you point me to where this "software page split" is handled?
Yes, you are correct, e1000 only supports receiving page split for 82571 and greater adapters.
Quoted 1 line
Chris
Signature · 1 line
Evgeniy Polyakov
Quoted 8 lines · Chris Friesen
Quoted 2 lines · Herbert Xu
Either upgrade your kernel or backport the page-splitting code in the current tree. That's really the only sane solution for jumbo packets.
Looking at the page-splitting code, it says "82571 and greater support packet-split...". We're running the 82546GB device. Looks like it won't help me.
Well, time to fork out for a new card then :)
Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txtQuoted 6 lines · Herbert Xu
Quoted 1 line · Chris Friesen
Looking at the page-splitting code, it says "82571 and greater support packet-split...". We're running the 82546GB device. Looks like it won't help me.
Well, time to fork out for a new card then :)
I wish. This is an embedded ATCA board. I'm not going to get a respin just because it doesn't deal well with jumbo frames.
Chris