Skip to contentSkip to conversation

NS83820 2.6.0-test5 driver seems unstable on IA64

DISCUSSION Andi Kleen 5 participants 6 messages Sep 26, 2003 – Sep 27, 2003 3 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

Andi Kleen

author
#3 lore raw
Quoted 1 line · David Mosberger

Why would that be? Slower, yes, but very slow?

It depends on your architecture I guess. On K8/x86-64 it isn't that big a deal (one cycle penalty for unaligned accesses), but if you take an exception for each unaligned read it will be incredible slow.

Of course the copy in the driver has the same problem, so it's not much better.

My point was basically that the header access are peanuts compared to the unaligned copy. It would be much better to optimize the copy than the header access. The proposal of just copying the header does not address this. And copying the packet in the driver has the same problem, so IMHO it's useless.

The solution proposed by Ivan sounds much better. The basic problem is that the Ethernet header is not a multiple of 4 and that misaligns everything after it. Use one receive descriptor for the MAC header and another for the rest (IP/TCP/payload) In the rest everything is aligned and there are no problems with misaligned data types (ignoring exceptional cases like broken timestamp options which can be handled slowly). Fixing the stack to handle separate MAC headers should not be that much work, the code is fairly limited. Drawback is that it requires bigger changes to the network drivers and maybe some special case code for non standard MAC headers.

-Andi

#4 lore raw
Quoted 3 lines · Andi Kleen

The solution proposed by Ivan sounds much better. The basic problem is that the Ethernet header is not a multiple of 4 and that misaligns everything after it.

At least some hardware will offset the incoming packet by two bytes to align everything. Whatever mechanism we end up using, it would be nice if it could make use of the hardware that is capable of this.

Chris

Signature · 4 lines
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: [email protected]
#5 lore raw
In reply to Andi Kleen
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Andi Kleen
Quoted 1 line · Andi> David Mosberger <[email protected]> writes:
Quoted 1 line

Why would that be? Slower, yes, but very slow?

Andi> It depends on your architecture I guess. On K8/x86-64 it isn't Andi> that big a deal (one cycle penalty for unaligned accesses), Andi> but if you take an exception for each unaligned read it will Andi> be incredible slow.

What are you talking about? Arches that don't like/support unaligned accesses will certainly have a copy_user() which handles misalignment just fine. For example, on ia64, the copy will run slightly slower because of an additional shift in the loop, but that penalty only shows on fully cached data. If the data is not cached, the copy speed for aligned vs. misaligned data is virtually the same.

Andi> Use one receive descriptor for the MAC header and another for Andi> the rest (IP/TCP/payload) In the rest everything is aligned Andi> and there are no problems with misaligned data types (ignoring Andi> exceptional cases like broken timestamp options which can be Andi> handled slowly).

That's what I proposed.

Andi> Fixing the stack to handle separate MAC headers should not be Andi> that much work, the code is fairly limited. Drawback is that Andi> it requires bigger changes to the network drivers and maybe Andi> some special case code for non standard MAC headers.

I suspect you'd be better off just copying the 14 bytes of the Ethernet header so that the entire packet is contiguous.

	--david
#6 lore raw
Quoted 5 lines · On Fri, 26 Sep 2003 10:47:12 -0700 David Mosberger <[email protected]>…

Arches that don't like/support unaligned accesses will certainly have a copy_user() which handles misalignment just fine. For example, on ia64, the copy will run slightly slower because of an additional shift in the loop, but that penalty only shows on fully cached data.

Exactly correct, and on many platforms (sparc64 happens to be one) there is _ZERO_ performance penalty for misalignment or source or destination buffer during a memcpy/memmove.

#7 lore raw
In reply to David Mosberger
Quoted 11 lines · David Mosberger
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Andi Kleen
Quoted 1 line · Andi> David Mosberger <[email protected]> writes:
Quoted 1 line

Why would that be? Slower, yes, but very slow?

Andi> It depends on your architecture I guess. On K8/x86-64 it isn't Andi> that big a deal (one cycle penalty for unaligned accesses), Andi> but if you take an exception for each unaligned read it will Andi> be incredible slow.

What are you talking about? Arches that don't like/support unaligned

They don't in this case ;-)

See my reply to David Miller for details.

Quoted 8 lines

Andi> Use one receive descriptor for the MAC header and another for Andi> the rest (IP/TCP/payload) In the rest everything is aligned Andi> and there are no problems with misaligned data types (ignoring Andi> exceptional cases like broken timestamp options which can be Andi> handled slowly).

That's what I proposed.

Hmm, must have missed that sorry. I saw it in Ivan's post only.

Quoted 7 lines

Andi> Fixing the stack to hadle separate MAC headers should not be Andi> that much work, the code is fairly limited. Drawback is that Andi> it requires bigger changes to the network drivers and maybe Andi> some special case code for non standard MAC headers.

I suspect you'd be better off just copying the 14 bytes of the Ethernet header so that the entire packet is contiguous.

Good point. So it can be even handled transparently in drivers without any core stack changes.

Only drawback is that more RX descriptors also usually require more PCI writes, which can be also very slow. Needs to be benchmarked carefully if it's a worthy trade-off.

-Andi

#8 lore raw
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Andi Kleen

Andi> Use one receive descriptor for the MAC header and another for Andi> the rest (IP/TCP/payload) In the rest everything is aligned Andi> and there are no problems with misaligned data types (ignoring Andi> exceptional cases like broken timestamp options which can be Andi> handled slowly).

Quoted 1 line
Quoted 1 line

That's what I proposed.

Andi> Hmm, must have missed that sorry. I saw it in Ivan's post Andi> only.

Hmmh, I wonder whether I accidentally replied via the newsgroup. I see my post in the newsgroup, but not on the linux-kernel archive. Anyhow, I attached the original mail below.

Andi> Fixing the stack to hadle separate MAC headers should not be Andi> that much work, the code is fairly limited. Drawback is that Andi> it requires bigger changes to the network drivers and maybe Andi> some special case code for non standard MAC headers.

Quoted 2 lines
Quoted 2 lines

I suspect you'd be better off just copying the 14 bytes of the
Ethernet header so that the entire packet is contiguous.

Andi> Good point. So it can be even handled transparently in drivers Andi> without any core stack changes.

Andi> Only drawback is that more RX descriptors also usually require Andi> more PCI writes, which can be also very slow. Needs to be Andi> benchmarked carefully if it's a worthy trade-off.

Yes, I totally agree on this one. Descriptor fetching is usually quite expensive, though depending on how smart the PCI controller is, some of the cost could be hidden.

	--david

---------------------------------------------------------------------
From: David Mosberger <[email protected]>
Subject: Re: NS83820 2.6.0-test5 driver seems unstable on IA64
Newsgroups: linux.kernel
Date: Fri, 26 Sep 2003 08:50:09 +0200
Reply-To: [email protected]
Organization: linux.* mail to news gateway

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · David S. Miller

David> On Fri, 26 Sep 2003 08:16:36 +0200 Manfred Spraul

Quoted 2 lines · David> <[email protected]> wrote:
Quoted 2 lines

Is that really the right solution? Add a full-packet copy to every driver?

David> In the short term, yes it is.

I know nothing about the ns83820, but page 22 of the data sheet
(http://www.national.com/pf/DP/DP83820.html) suggests that it would be possible to setup _two_ descriptors for each incoming packet: the first one would cover the Ethernet header (14 bytes), the second the rest. That way, IP-header would be 8-byte aligned (since the ns83820's DMA engine seems to require 8-byte alignment for incoming data).

If so, this would let you get the IP-header and on aligned at the cost of extra descriptor fetching. The cost of fetching the extra descriptor will be big enough that you'd only want to do this when unaligned accesses are expensive, but hopefully it would be cheaper than copying the entire packet.

	--david