Skip to contentSkip to conversation

jumbo frames and memory fragmentation

DISCUSSION Chris Friesen 4 participants 9 messages Jun 29, 2006 – Jul 17, 2006

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

I'm running a system with multiple e1000 devices, using 9KB jumbo frames. I'm running a modified 2.6.10 with e1000 driver 5.5.4-k2.

I'm a bit concerned about the behaviour of this driver with jumbo frames. We ask for 9KB. The driver then bumps that up to a power-of-two, so it calls dev_alloc_skb(16384). That then bumps it up a bit to allow for its own overhead, so it appears that we end up asking for 32KB of physically contiguous memory for every packet coming in. Ouch.

Add to that the fact that this version of the driver doesn't do copybreak, and it means that after we're up for a few days it starts complaining about not being able to allocate buffers.

Anyone have any suggestions on how to improve this? Upgrading kernels isn't an option. I could port back the copybreak stuff fairly easily.

Back in 2.4 some of the drivers used to retry buffer allocations using GFP_KERNEL once interrupts were reenabled. I don't see many of them doing that anymore--would there be any benefit to that?

Thanks,

Chris

#1 lore raw
Quoted 3 lines · Chris Friesen

Anyone have any suggestions on how to improve this? Upgrading kernels isn't an option. I could port back the copybreak stuff fairly easily.

Either upgrade your kernel or backport the page-splitting code in the current tree. That's really the only sane solution for jumbo packets.

Cheers,

Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#2 lore raw
Quoted 2 lines · Herbert Xu

Either upgrade your kernel or backport the page-splitting code in the current tree. That's really the only sane solution for jumbo packets.

Looking at the page-splitting code, it says "82571 and greater support packet-split...". We're running the 82546GB device. Looks like it won't help me.

Chris

#3 lore raw
Quoted 6 lines · Chris Friesen
Quoted 2 lines

Either upgrade your kernel or backport the page-splitting code in the current tree. That's really the only sane solution for jumbo packets.

Looking at the page-splitting code, it says "82571 and greater support packet-split...". We're running the 82546GB device. Looks like it won't help me.

It definitely will.
Packet split in hardware means separating data and headers into different pages in different reads, while software page split means that skb has a list of fragments where part of the packet will be DMAed, so jumbo frame will be converted into several pages.

Signature · 1 line
	Evgeniy Polyakov
#4 lore raw
Quoted 4 lines · Evgeniy Polyakov

It definitely will.
Packet split in hardware means separating data and headers into
different pages in different reads, while software page split means that skb has a list of fragments where part of the packet will be DMAed, so jumbo frame will be converted into several pages.

Maybe I'm looking at the wrong code then. Can you point me to where this "software page split" is handled?

Chris

#5 lore raw
Quoted 10 lines · Chris Friesen
Quoted 5 lines · Evgeniy Polyakov

It definitely will.
Packet split in hardware means separating data and headers into different pages in different reads, while software page split means that skb has a list of fragments where part of the packet will be DMAed, so jumbo frame will be converted into several pages.

Maybe I'm looking at the wrong code then. Can you point me to where this "software page split" is handled?

You're right, it isn't supported for your hardware. to solve the problem (I got this from mailing lists) you probably need to adjust the /proc/sys/vm/min_free_kbytes

let us know if this helps for you.

#6 lore raw
In reply to Chris Friesen
Quoted 10 lines · Chris Friesen
Quoted 5 lines · Evgeniy Polyakov

It definitely will.
Packet split in hardware means separating data and headers into different pages in different reads, while software page split means that skb has a list of fragments where part of the packet will be DMAed, so jumbo frame will be converted into several pages.

Maybe I'm looking at the wrong code then. Can you point me to where this "software page split" is handled?

Yes, you are correct, e1000 only supports receiving page split for 82571 and greater adapters.

Quoted 1 line

Chris

Signature · 1 line
	Evgeniy Polyakov
#7 lore raw
In reply to Chris Friesen
Quoted 8 lines · Chris Friesen
Quoted 2 lines · Herbert Xu

Either upgrade your kernel or backport the page-splitting code in the current tree. That's really the only sane solution for jumbo packets.

Looking at the page-splitting code, it says "82571 and greater support packet-split...". We're running the 82546GB device. Looks like it won't help me.

Well, time to fork out for a new card then :)

Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#8 lore raw
Quoted 6 lines · Herbert Xu
Quoted 1 line · Chris Friesen

Looking at the page-splitting code, it says "82571 and greater support packet-split...". We're running the 82546GB device. Looks like it won't help me.

Well, time to fork out for a new card then :)

I wish. This is an embedded ATCA board. I'm not going to get a respin just because it doesn't deal well with jumbo frames.

Chris