Skip to contentSkip to conversation

NS83820 2.6.0-test5 driver seems unstable on IA64

DISCUSSION Peter Chubb 18 participants 67 messages Sep 19, 2003 – Sep 26, 2003

lore
Share & tools
lore
#0 lore raw

Hi,
I'm having some problems with the NS83820 driver on Itanium-1.

1. I see many many `unaligned accesses' when the IP header is accessed (ip_input.c:410 and 423)
It looks as if the IP header is two byte aligned, when it ought to be 4-byte aligned.

The error message is, e.g.,
kernel unaligned access to 0xe000000128d8881e, ip=0xe0000000047ba2d1

0xe0000000047ba2d1 is
ip_input:410

		   if (iph->ihl < 5 || iph->version != 4)
		                goto inhdr_error; 

0xe000000128d8881e is what's calculated as &iph->ihl

The obvious approach of realigning the SKB by 2 bytes seems not to work.

Any ideas?

--
Dr Peter Chubb http://www.gelato.unsw.edu.au [email protected]
You are lost in a maze of BitKeeper repositories, all slightly different.

#1 lore raw
Quoted 2 lines · Peter Chubb

The obvious approach of realigning the SKB by 2 bytes seems not to work.

Could you be more detailed about the "obvious approach"? ie show the diff of what you changed.

Several other NIC driver "alias" (as davidm describes it) the buffer by reserving two bytes at the beginning of the recieve buffer where header and payload data are DMAd on inbound traffic.

I look to my favorite driver, tulip, for an example and promptly get confused. interrupt.c:tulip_rx() calls skb_reserve(xx,2) to force alignment when replenishing RX buffers while a note in tulip_core.c:tulip_init_ring() clearly says not to. I haven't looked further to figure out the difference.

hth,
grant

#2 lore raw
In reply to Peter Chubb
Quoted 10 lines · Grant Grundler
Quoted 2 lines · Peter Chubb

The obvious approach of realigning the SKB by 2 bytes seems not to work.

Could you be more detailed about the "obvious approach"? ie show the diff of what you changed.

Several other NIC driver "alias" (as davidm describes it) the buffer by reserving two bytes at the beginning of the recieve buffer where header and payload data are DMAd on inbound traffic.

It is a mixed blessing, because the result is a non cache line aligned buffer. Some NIC chipsets don't like this because they have to do a read-modify-write cycle for the first cache line and cannot burst the full packet.

-Andi

#3 lore raw
In reply to Peter Chubb
Quoted 7 lines · Grant Grundler
Quoted 3 lines · Peter Chubb

The
obvious approach of realigning the SKB by 2 bytes seems not to work.

Could you be more detailed about the "obvious approach"? ie show the diff of what you changed.

It doesn't work as in no error messages, no pings, no interrupts to the driver. And the kernel hangs after a short while.

This is what I changed:

=== drivers/net/ns83820.c 1.19 vs edited ==--- 1.19/drivers/net/ns83820.c Thu Jun 5 13:50:00 2003
+++ edited/drivers/net/ns83820.c Fri Sep 19 13:49:23 2003
@@ -567,7 +567,7 @@

                res = (long)skb->tail & 0xf;
                res = 0x10 - res;
                res &= 0xf;

- skb_reserve(skb, res);
+ skb_reserve(skb, res+2);

                skb->dev = &dev->net_dev;
                if (gfp != GFP_ATOMIC)

--
Dr Peter Chubb http://www.gelato.unsw.edu.au peterc AT gelato.unsw.edu.au
You are lost in a maze of BitKeeper repositories, all slightly different.

#4 lore raw
In reply to Peter Chubb
Quoted 4 lines · Andi Kleen

It is a mixed blessing, because the result is a non cache line aligned buffer. Some NIC chipsets don't like this because they have to do a read-modify-write cycle for the first cache line and cannot burst the full packet.

yeah, that reminds me...tulip can only DMA to word aligned addresses. I looked it up again in DEC 21143 HWREF (page 113 of the pdf):
Table 4-3. RDES2 Bit Field Description
Field Description
31:0 Buffer Address 1
Indicates the physical address of buffer 1. The buffer must be longword aligned (RDES2<1:0> = 00).

Same is true for TX/RX descriptor addresses.
Behavior is undefined if the addresses for DMA are not 4-byte aligned.

Anyone know if that's true for NS83820?
I couldn't find which driver controls that chip/NIC via a quick grep.

grant

#5 lore raw
In reply to Peter Chubb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Grant" = Grant Grundler <[email protected]> writes:

Grant> On Fri, Sep 19, 2003 at 06:43:15AM +0200, Andi Kleen wrote:

Grant> Same is true for TX/RX descriptor addresses. Behavior is Grant> undefined if the addresses for DMA are not 4-byte aligned.

Grant> Anyone know if that's true for NS83820? I couldn't find which Grant> driver controls that chip/NIC via a quick grep.

I believe it is, based on the experiments I've done here, and on looking at the BSD driver.

Driver is drivers/net/ns83820.c

Peter C

#6 lore raw
In reply to Peter Chubb

Review line 570

Quoted 10 lines

This is what I changed:

=== drivers/net/ns83820.c 1.19 vs edited ==> --- 1.19/drivers/net/ns83820.c Thu Jun 5 13:50:00 2003
+++ edited/drivers/net/ns83820.c Fri Sep 19 13:49:23 2003

(quoted diff) +1 -1
@@ -567,7 +567,7 @@
                res = (long)skb->tail & 0xf;
                res = 0x10 - res;
                res &= 0xf;
-               skb_reserve(skb, res);
+               skb_reserve(skb, res+2);

You must add the 2 bytes in dev_alloc_skb() too.

(it also add 16 bytes by itself, but they are used for other things)

-Andi

#7 lore raw
In reply to Peter Chubb
Quoted 3 lines · Andi Kleen

You must add the 2 bytes in dev_alloc_skb() too.

(it also add 16 bytes by itself, but they are used for other things)

Sorry, the chipset does not support packet rx into unaligned buffers. stoopid. Fwiw, the copy code got removed by davem -- isn't the core dealubg wuth unakugned packets now?

-ben

Signature · 3 lines
"The software industry today survives only through an unstated agreement 
not to stir things up too much.  We must hope this lawsuit [by SCO] isn't 
the big stirring spoon." -- Ian Lance Taylor, Linux Journal, June 2003
#8 lore raw
In reply to Peter Chubb

OK, a patch for the driver in 2.6.0-test5 is appended.

I suspect that there are other architectures that don't like unaligned accesses... feel free to add them to the #ifdef.
This is based on code I found in the revision history. It seems to work.

Without this patch, the console messages saying `unaligned access' would come out fast enough and often enough to delay and or miss interrupts, leading to an eventual machine hangup on I2000.

=== drivers/net/ns83820.c 1.30 vs edited ==--- 1.30/drivers/net/ns83820.c Thu Sep 11 09:46:45 2003
+++ edited/drivers/net/ns83820.c Mon Sep 22 12:49:18 2003
@@ -793,6 +793,25 @@
}
}

+#if defined(__ia64__)
+static inline struct sk_buff *skb_realign_iphdr(struct sk_buff *skb, int len, struct ns83820 *dev) +{
+ struct sk_buff *tmp = __dev_alloc_skb(len+2, GFP_ATOMIC);
+ if (!tmp)
+ return NULL;
+ tmp->dev = &dev->net_dev;
+ skb_reserve(tmp, 2);
+ memcpy(skb_put(tmp, len), skb->data, len);
+ kfree_skb(skb);
+ return tmp;
+}
+#else
+static inline struct sk_buff *skb_realign_iphdr(struct sk_buff *skb, int len, struct ns83820 *dev) +{
+ return skb;
+}
+#endif
+
static void FASTCALL(ns83820_rx_kick(struct ns83820 *dev));
static void ns83820_rx_kick(struct ns83820 *dev)
{
@@ -862,6 +881,7 @@
if (likely(CMDSTS_OK & cmdsts)) {
int len = cmdsts & 0xffff;
skb_put(skb, len);
+ skb = skb_realign_iphdr(skb, len, dev);
if (unlikely(!skb))
goto netdev_mangle_me_harder_failed;
if (cmdsts & CMDSTS_DEST_MULTI)

#9 lore raw
In reply to Peter Chubb

Denied. Dave, please explain.

-ben

Quoted 48 lines · Peter Chubb

OK, a patch for the driver in 2.6.0-test5 is appended.

I suspect that there are other architectures that don't like unaligned accesses... feel free to add them to the #ifdef.
This is based on code I found in the revision history. It seems to work.

Without this patch, the console messages saying `unaligned access' would come out fast enough and often enough to delay and or miss interrupts, leading to an eventual machine hangup on I2000.

=== drivers/net/ns83820.c 1.30 vs edited ==> --- 1.30/drivers/net/ns83820.c Thu Sep 11 09:46:45 2003
+++ edited/drivers/net/ns83820.c Mon Sep 22 12:49:18 2003

(quoted diff) +20 -0
@@ -793,6 +793,25 @@
 	}
 }
 
+#if defined(__ia64__)
+static inline struct sk_buff *skb_realign_iphdr(struct sk_buff *skb, int len, struct ns83820 *dev)
+{
+	struct sk_buff *tmp = __dev_alloc_skb(len+2, GFP_ATOMIC);
+	if (!tmp)
+		return NULL;
+	tmp->dev = &dev->net_dev;
+	skb_reserve(tmp, 2);
+	memcpy(skb_put(tmp, len), skb->data, len);
+	kfree_skb(skb);
+	return tmp;
+}
+#else
+static inline  struct sk_buff *skb_realign_iphdr(struct sk_buff *skb, int len, struct ns83820 *dev)
+{
+	return skb;
+}
+#endif
+
 static void FASTCALL(ns83820_rx_kick(struct ns83820 *dev));
 static void ns83820_rx_kick(struct ns83820 *dev)
 {
@@ -862,6 +881,7 @@
 		if (likely(CMDSTS_OK & cmdsts)) {
 			int len = cmdsts & 0xffff;
 			skb_put(skb, len);
+			skb = skb_realign_iphdr(skb, len, dev);
 			if (unlikely(!skb))
 				goto netdev_mangle_me_harder_failed;
 			if (cmdsts & CMDSTS_DEST_MULTI)
Signature · 3 lines
"The software industry today survives only through an unstated agreement 
not to stir things up too much.  We must hope this lawsuit [by SCO] isn't 
the big stirring spoon." -- Ian Lance Taylor, Linux Journal, June 2003
#10 lore raw
In reply to Peter Chubb
Quoted 1 line · On Mon, 22 Sep 2003 20:36:29 -0400 Benjamin LaHaise <[email protected]> wrote:

Denied. Dave, please explain.

Why should I have anything to explain? :-)

The fact that ia64 is doing a printk for an unaligned kernel load or store is what you should be asking questions about :)

It's one thing if ia64 keeps track of unaligned accesses as a counter statistic, but emitting a printk for everyone is pretty anti-social.

Unaligned accesses in the kernel are perfectly normal, and are absolutely going to happen in various kinds of networking setups.

#11 lore raw
In reply to Peter Chubb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"David" = David S Miller <[email protected]> writes:

David> On Mon, 22 Sep 2003 20:36:29 -0400 Benjamin LaHaise

Quoted 1 line · David> <[email protected]> wrote:
Quoted 1 line

Denied. Dave, please explain.

David> Why should I have anything to explain? :-)

David> The fact that ia64 is doing a printk for an unaligned kernel David> load or store is what you should be asking questions about :)

How expensive is it to take the trap and do a fix up, compared to making an aligned copy? As it involves raising and handling a fault disassembling the instruction that caused the fault, etc., I'd be surprised if it's much less than 1000 cycles, even without the printk, although I haven't measured it yet, and can't find enough info in the architecture manuals to know what it is.

Even if it's only 500 cycles, you can copy and realign a large packet in that time.

--
Dr Peter Chubb http://www.gelato.unsw.edu.au peterc AT gelato.unsw.edu.au
You are lost in a maze of BitKeeper repositories, all slightly different.

#12 lore raw
In reply to Peter Chubb
Quoted 6 lines · On Tue, 23 Sep 2003 20:40:05 +1000 Peter Chubb <[email protected]> wrote:

How expensive is it to take the trap and do a fix up, compared to making an aligned copy? As it involves raising and handling a fault disassembling the instruction that caused the fault, etc., I'd be surprised if it's much less than 1000 cycles, even without the printk, although I haven't measured it yet, and can't find enough info in the architecture manuals to know what it is.

A cache miss can cause 100 or so cycles. :)

And unlike the fixup trap, the printk wakes up a process and causes disk activity as syslogd writes to the kernel message log file.

#13 lore raw
In reply to Peter Chubb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · David S. Miller

David> On Tue, 23 Sep 2003 20:40:05 +1000 Peter Chubb

Quoted 6 lines · David> <[email protected]> wrote:
Quoted 6 lines

How expensive is it to take the trap and do a fix up, compared to making an aligned copy? As it involves raising and handling a fault disassembling the instruction that caused the fault, etc., I'd be surprised if it's much less than 1000 cycles, even without the printk, although I haven't measured it yet, and can't find enough info in the architecture manuals to know what it is.

David> A cache miss can cause 100 or so cycles. :)

David> And unlike the fixup trap, the printk wakes up a process and David> causes disk activity as syslogd writes to the kernel message David> log file.

The printk() is rate-controlled and doesn't happen for every unaligned access. It's average cost can be made as low as we want to, by adjusting the rate.

	--david
#14 lore raw
In reply to Peter Chubb
Quoted 3 lines · On Tue, 23 Sep 2003 07:59:11 -0700 David Mosberger <[email protected]>…

The printk() is rate-controlled and doesn't happen for every unaligned access. It's average cost can be made as low as we want to, by adjusting the rate.

But if the event is normal, you shouldn't be logging it as if it weren't.

Anyone who tries to use IP over appletalk or certain protocols over PPP are going to see your silly messages.

As I understand it, you even do this stupid printk for user apps as well, that makes it more than rediculious. I'd be surprised if anyone can find any useful kernel messages on an ia64 system in the dmesg output with all the unaligned access crap there.

#15 lore raw
In reply to Peter Chubb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · David S. Miller

DaveM> On Tue, 23 Sep 2003 07:59:11 -0700

Quoted 3 lines · DaveM> David Mosberger <[email protected]> wrote:
Quoted 3 lines

The printk() is rate-controlled and doesn't happen for every unaligned access. It's average cost can be made as low as we want to, by adjusting the rate.

DaveM> But if the event is normal, you shouldn't be logging it as if DaveM> it weren't.

An event that causes a slow-down of 500 times or so is not "normal". On Alpha, we did have just a counter. Guess what, nobody ever noticed when things ran much slower than they should have.

DaveM> Anyone who tries to use IP over appletalk or certain protocols DaveM> over PPP are going to see your silly messages.

DaveM> As I understand it, you even do this stupid printk for user apps DaveM> as well, that makes it more than rediculious. I'd be surprised DaveM> if anyone can find any useful kernel messages on an ia64 system DaveM> in the dmesg output with all the unaligned access crap there.

Ever heard of prctl --unalign=silent?
I don't normaly do that and I still get very few unaligned warning messages.

	--david
#16 lore raw
In reply to Peter Chubb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · David S. Miller

DaveM> On Tue, 23 Sep 2003 07:59:11 -0700

Quoted 3 lines · DaveM> David Mosberger <[email protected]> wrote:
Quoted 3 lines

The printk() is rate-controlled and doesn't happen for every unaligned access. It's average cost can be made as low as we want to, by adjusting the rate.

DaveM> But if the event is normal, you shouldn't be logging it as if DaveM> it weren't.

An event that causes a slow-down of 500 times or so is not "normal". On Alpha, we did have just a counter. Guess what, nobody ever noticed when things ran much slower than they should have.

DaveM> Anyone who tries to use IP over appletalk or certain protocols DaveM> over PPP are going to see your silly messages.

DaveM> As I understand it, you even do this stupid printk for user apps DaveM> as well, that makes it more than rediculious. I'd be surprised DaveM> if anyone can find any useful kernel messages on an ia64 system DaveM> in the dmesg output with all the unaligned access crap there.

Ever heard of prctl --unalign=silent?
I don't normaly do that and I still get very few unaligned warning messages.

	--david
#17 lore raw
In reply to Peter Chubb
Quoted 2 lines · Kevin

Rate-limited unaligned loads in user space make a lot more sense, since they _may_ point out issues in the code.

That's what generates the bulk of the noise in people's ia64 dmesg output.

#18 lore raw
In reply to Peter Chubb
Quoted 6 lines
Quoted 3 lines

The printk() is rate-controlled and doesn't happen for every unaligned access. It's average cost can be made as low as we want to, by adjusting the rate.

But if the event is normal, you shouldn't be logging it as if it weren't.

That's my view on the fpswa printk's (handle_fpu_swa): they are normal, expected, and there is absolutely nothing that can be done about them -- so why print a "warning" about them (even if it is only 5 per second)? If nothing else, toggle the meaning for IA64_THREAD_FPEMU_NOPRINT: turn it ON for special apps.

Rate-limited unaligned loads in user space make a lot more sense, since they _may_ point out issues in the code.

Kevin

#19 lore raw
In reply to Peter Chubb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Kevin

Van> That's my view on the fpswa printk's (handle_fpu_swa): they are Van> normal, expected, and there is absolutely nothing that can be Van> done about them -- so why print a "warning" about them (even if Van> it is only 5 per second)? If nothing else, toggle the meaning Van> for IA64_THREAD_FPEMU_NOPRINT: turn it ON for special apps.

So what's wrong with doign prctl --fpemul=silent in the init process? The flags are inherited across fork().

	--david
#20 lore raw
In reply to Peter Chubb
Quoted 4 lines

As I understand it, you even do this stupid printk for user apps as well, that makes it more than rediculious. I'd be surprised if anyone can find any useful kernel messages on an ia64 system in the dmesg output with all the unaligned access crap there.

I don't think that it is "normal" for applications to do unaligned memory access. It means that:

a) the programmer is playing fast and loose with types and/or casts.
b) the end-user is going to be disappointed with the performance.

Looking at a couple of ia64 build servers here I see zero unaligned access messages in the logs.

-Tony

#21 lore raw
In reply to Peter Chubb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · David S. Miller

DaveM> On Tue, 23 Sep 2003 12:58:59 -0500

Quoted 2 lines · Kevin
Quoted 2 lines

Rate-limited unaligned loads in user space make a lot more sense, since they _may_ point out issues in the code.

DaveM> That's what generates the bulk of the noise in people's ia64 DaveM> dmesg output.

At the moment, there are two ways to control the unaligned message printing:

- use the dmesg command to lower the printing threshold below KERN_WARNING ("dmesg -n4", IIRC)

- use prctl --unalign=silent to turn off unaligned printing for a particular task and its children

If these mechanisms are not sufficient, there is nothing to stop anyone from cooking up a patch. For my puposes, what's there works nicely so I myself have no plans to make changes in this area.

	--david
#22 lore raw
In reply to Peter Chubb
Quoted 2 lines · Tony

Looking at a couple of ia64 build servers here I see zero unaligned access messages in the logs.

ip options can still be an odd number of bytes. Having itanic spew bogus log messages is just plain wrong (yet another hardware issue pushed off on software for no significant gain).

-ben

Signature · 3 lines
"The software industry today survives only through an unstated agreement 
not to stir things up too much.  We must hope this lawsuit [by SCO] isn't 
the big stirring spoon." -- Ian Lance Taylor, Linux Journal, June 2003
#23 lore raw
In reply to Peter Chubb
Quoted 7 lines · On Tue, 23 Sep 2003 14:29:25 -0400 Benjamin LaHaise <[email protected]> wrote:
Quoted 2 lines · Tony

Looking at a couple of ia64 build servers here I see zero unaligned access messages in the logs.

ip options can still be an odd number of bytes. Having itanic spew bogus log messages is just plain wrong (yet another hardware issue pushed off on software for no significant gain).

Also, some TCP implementations sometimes don't align the timestamp options in the headers as well.

Also, as I mentioned, try IP over appletalk.

Unaligned accesses are perfectly normal, _especially_ in the kernel. This is an axiom of the kernel networking decided a long time ago, and printing out a silly message when it happens doesn't make that any less true.

#24 lore raw
In reply to Peter Chubb
Quoted 2 lines · On Tue, 23 Sep 2003 11:27:13 -0700 David Mosberger <[email protected]>…

- use the dmesg command to lower the printing threshold below KERN_WARNING ("dmesg -n4", IIRC)

Not terribly useful if all the unaligned messages made the one I'm interested in get kicked out of the logs.

Quoted 2 lines

- use prctl --unalign=silent to turn off unaligned printing for a particular task and its children

What about for the kernel?

No other port is so obstinate about printing out unaligned kernel access messages, why does ia64 have to be so different?

Unaligned accesses are not only perfectly fine in the kernel, they are in fact expected in certain circumstances.

#25 lore raw
In reply to Peter Chubb

On Tue, Sep 23, 2003 at 02:29:25PM -0400, Benjamin LaHaise wrote:
...

Quoted 1 line

(yet another hardware issue pushed off on software for no significant gain).

Even x86 pays at least a one cycle penalty for every misaligned access. In general, open source code has no excuse for using misaligned fields. It's (mostly) avoidable. TCP/IP headers are the historical exception.

One could make the same arguement that a modern NIC should not require 16 byte alignment for DMA. It's a tradeoff one way or the other. Just a matter of perspective.

grant

#26 lore raw
In reply to Peter Chubb
Quoted 1 line · On Tue, 23 Sep 2003 11:51:04 -0700 Grant Grundler <[email protected]> wrote:

Even x86 pays at least a one cycle penalty for every misaligned access.

Yes, one cycle, and it's completely lost in the noise when it happens.

Quoted 2 lines

In general, open source code has no excuse for using misaligned fields. It's (mostly) avoidable. TCP/IP headers are the historical exception.

It's not the TCP/IP headers intrinsically, it's what they are embedded inside of.

For example, if the ethernet driver (as nearly all of our's do which can) optimized for an ethernet header followed by an IP header, guess what? That causes ethernet header followed by appletalk followed by IP to do unaligned accesses.

It is an unavoidable axoim in the kernel networking. Unaligned accesses will happen, and they aren't a bug and therefore not worthy of mention in the kernel logs any more than "page was freed" :-)

#27 lore raw
In reply to Peter Chubb
Quoted 1 line · On Tue, 23 Sep 2003 20:54:50 +0200 Andreas Schwab <[email protected]> wrote:

Unaligned access are a BUG.

Wrong, they've been allowed in the kernel networking from day one and there is nothing that can be done to avoid the cases for which they occur.

Stop spreading fud.

#28 lore raw
In reply to Peter Chubb
Quoted 7 lines · Benjamin LaHaise
Quoted 2 lines · Tony

Looking at a couple of ia64 build servers here I see zero unaligned access messages in the logs.

ip options can still be an odd number of bytes. Having itanic spew bogus log messages is just plain wrong (yet another hardware issue pushed off on software for no significant gain).

Unaligned access are a BUG.

Andreas.

Signature · 4 lines
Andreas Schwab, SuSE Labs, [email protected]
SuSE Linux AG, Deutschherrnstr. 15-19, D-90429 Nürnberg
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."
#29 lore raw
In reply to Peter Chubb
Quoted 3 lines · On Tue, 23 Sep 2003 21:09:47 +0200 Andreas Schwab <[email protected]> wrote:

The compiler is allowed to take advantage that there are no unaligned accesses. You need to use compiler extensions (like attribute packed) to stop it from doing this.

That's correct, and if the address is misaligned the cpu "traps" and the kernel fixes up the load/store access to fix it up.

This unaligned trap handling is required for a port of Linux to a given cpu architecture.

That's what we're talking about here.

#30 lore raw
In reply to Peter Chubb
Quoted 7 lines · Grant Grundler

On Tue, Sep 23, 2003 at 02:29:25PM -0400, Benjamin LaHaise wrote:
...

Quoted 1 line

(yet another hardware issue pushed off on software for no significant gain).

Even x86 pays at least a one cycle penalty for every misaligned access. In general, open source code has no excuse for using misaligned fields. It's (mostly) avoidable. TCP/IP headers are the historical exception.

The other point to keep in mind is that apparently the .09u rev of the P4 includes additional hardware for handling unaligned accesses because they are so common (gzip is a good example of an app where it is faster to do an unaligned access for the benefit of fetching the data in one instruction instead of 3+, and there is no way to improve on it).

Quoted 3 lines

One could make the same arguement that a modern NIC should not require 16 byte alignment for DMA. It's a tradeoff one way or the other. Just a matter of perspective.

I consider the 83820 buggy in this regard, too. That said, the fix does not belong in the driver layer, as having it duplicated in a dozen drivers is more stupid than fixing up the arch code which is required anyways.

-ben

Signature · 3 lines
"The software industry today survives only through an unstated agreement 
not to stir things up too much.  We must hope this lawsuit [by SCO] isn't 
the big stirring spoon." -- Ian Lance Taylor, Linux Journal, June 2003
#31 lore raw
In reply to Peter Chubb
Quoted 2 lines · On Tue, 23 Sep 2003 21:16:33 +0200 Andreas Schwab <[email protected]> wrote:

Or the compiler generates code to take advantage of the fact that the lower address bits are zero.

The only place where I can se it doing this legally is for structure offsets. For example where a "load 4 byte word" instruction takes an offsetable address composed of a reg and an integer offset where the integer offset must be a multiple of 4.

This rule we do abide by in the kernel, because PARISC requires this.

Anything more is asking for trouble, I wouldn't want to use such a compiler in the real world :)

#32 lore raw
In reply to Peter Chubb
Quoted 8 lines · David S. Miller
Quoted 1 line · On Tue, 23 Sep 2003 20:54:50 +0200 Andreas Schwab <[email protected]> wrote:

Unaligned access are a BUG.

Wrong, they've been allowed in the kernel networking from day one and there is nothing that can be done to avoid the cases for which they occur.

The compiler is allowed to take advantage that there are no unaligned accesses. You need to use compiler extensions (like attribute packed) to stop it from doing this.

Quoted 1 line

Stop spreading fud.

Nothing like this.

Andreas.

Signature · 4 lines
Andreas Schwab, SuSE Labs, [email protected]
SuSE Linux AG, Deutschherrnstr. 15-19, D-90429 Nürnberg
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."
#34 lore raw
In reply to Peter Chubb
Quoted 2 lines · On Tue, 23 Sep 2003 12:17:21 -0700 David Mosberger <[email protected]>…

Look, this may be difficult for you to understand, but different people find different policies useful.

You, sure. But you are not the only user of the ia64 port just as I am not the only user of the sparc64 port and if sparc64 generated diagnostic messages not useful to people other than me I'd have to quiet them by default.

All I'm suggesting is provide a way to control the kernel unaligned accesses message and default it to off.

#35 lore raw
In reply to Peter Chubb
Quoted 2 lines · On Tue, 23 Sep 2003 11:06:04 -0700 David Mosberger <[email protected]>…

So what's wrong with doign prctl --fpemul=silent in the init process? The flags are inherited across fork().

Great, can we get such a setting for the kernel bits too?

#36 lore raw
In reply to Peter Chubb
Quoted 2 lines · Jeff Garzik

Traditionally, if the NIC driver is accessing unaligned data, it should be using get/put_unaligned().

The driver isn't, the networking stack is (it has to). For more examples where this behaviour is required: PPPoE, VLAN...

-ben

Signature · 3 lines
"The software industry today survives only through an unstated agreement 
not to stir things up too much.  We must hope this lawsuit [by SCO] isn't 
the big stirring spoon." -- Ian Lance Taylor, Linux Journal, June 2003
#37 lore raw
In reply to Peter Chubb
Quoted 9 lines · David S. Miller
Quoted 3 lines · On Tue, 23 Sep 2003 21:09:47 +0200 Andreas Schwab <[email protected]> wrote:

The compiler is allowed to take advantage that there are no unaligned accesses. You need to use compiler extensions (like attribute packed) to stop it from doing this.

That's correct, and if the address is misaligned the cpu "traps" and the kernel fixes up the load/store access to fix it up.

Or the compiler generates code to take advantage of the fact that the lower address bits are zero.

Quoted 1 line

That's what we're talking about here.

Of course, the kernel language is not ISO C, and never will be.

Andreas.

Signature · 4 lines
Andreas Schwab, SuSE Labs, [email protected]
SuSE Linux AG, Deutschherrnstr. 15-19, D-90429 Nürnberg
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."
#38 lore raw
In reply to Peter Chubb
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · David S. Miller

DaveM> No other port is so obstinate about printing out unaligned DaveM> kernel access messages, why does ia64 have to be so DaveM> different?

Look, this may be difficult for you to understand, but different people find different policies useful. I absolutely want to see when unaligned accesses happen, because it's almost always a performance issue, if not an outright bug. If you don't like the current mechanisms to control the policy for handling unaligned accesses, make them better. Don't try to tell me that the policy I want is "wrong"---that will get you nowhere.

	--david
#39 lore raw
In reply to Peter Chubb
Quoted 2 lines · On Tue, 23 Sep 2003 20:28:04 +0100 Matthew Wilcox <[email protected]> wrote:

That's not true; they could be avoided by using get_unaligned() and put_unaligned(). You just don't want to because they'd make sparc suck.

Not only sparc would be effected by this. Using {get,put}_unaligned() all over the networking would incur a penalty for many platforms, not just sparc.

#40 lore raw
In reply to Peter Chubb
Quoted 6 lines · Benjamin LaHaise
Quoted 2 lines · Jeff Garzik

Traditionally, if the NIC driver is accessing unaligned data, it should be using get/put_unaligned().

The driver isn't, the networking stack is (it has to). For more examples where this behaviour is required: PPPoE, VLAN...

Oh, no argument.... agreed 100%

Jeff

#41 lore raw
In reply to Peter Chubb
Quoted 8 lines · David S. Miller
Quoted 1 line · On Tue, 23 Sep 2003 20:54:50 +0200 Andreas Schwab <[email protected]> wrote:

Unaligned access are a BUG.

Wrong, they've been allowed in the kernel networking from day one and there is nothing that can be done to avoid the cases for which they occur.

That's not true; they could be avoided by using get_unaligned() and put_unaligned(). You just don't want to because they'd make sparc suck.

Signature · 3 lines
"It's not Hollywood.  War is real, war is primarily not about defeat or
victory, it is about death.  I've seen thousands and thousands of dead bodies.
Do you think I want to have an academic debate on this subject?" -- Robert Fisk
#42 lore raw
In reply to Peter Chubb
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line · David S. Miller

Look, this may be difficult for you to understand, but different people find different policies useful.

David> You, sure. But you are not the only user of the ia64 port just David> as I am not the only user of the sparc64 port and if sparc64 generated David> diagnostic messages not useful to people other than me I'd have to David> quiet them by default.

But they _are_ useful. Peter certainly noticed very quickly that there was a problem with the ns83820 driver. Do you think it would have been better for the kernel to run silently at greatly degraded performance? I think not.

The optimistic assumption that the network stack is making about headers being aligned works as long as unaligned headers are relatively rare. If unaligned headers are common, we'll have to do something about that for ia64 (and Alpha, SPARC, PA-RISC, MIPS, etc.), because performance will otherwise suck.

BTW: Peter, I'm not quite sure I understand why your machine got into real problems because of the printks. The messages are supposed to be rated limited to at most 5 per 5 second interval. Each message is about 80 bytes in size, so we're talking about 80 bytes/s. Do you know what really went wrong here? Did the rate limiting not work as expected or is there something strange about your setup?

	--david
#43 lore raw
In reply to Peter Chubb
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line · David S. Miller

Look, this may be difficult for you to understand, but different people find different policies useful.

David> You, sure. But you are not the only user of the ia64 port just David> as I am not the only user of the sparc64 port and if sparc64 generated David> diagnostic messages not useful to people other than me I'd have to David> quiet them by default.

But they _are_ useful. Peter certainly noticed very quickly that there was a problem with the ns83820 driver. Do you think it would have been better for the kernel to run silently at greatly degraded performance? I think not.

The optimistic assumption that the network stack is making about headers being aligned works as long as unaligned headers are relatively rare. If unaligned headers are common, we'll have to do something about that for ia64 (and Alpha, SPARC, PA-RISC, MIPS, etc.), because performance will otherwise suck.

BTW: Peter, I'm not quite sure I understand why your machine got into real problems because of the printks. The messages are supposed to be rated limited to at most 5 per 5 second interval. Each message is about 80 bytes in size, so we're talking about 80 bytes/s. Do you know what really went wrong here? Did the rate limiting not work as expected or is there something strange about your setup?

	--david
#44 lore raw
In reply to Peter Chubb
Quoted 3 lines · David S. Miller
Quoted 1 line

Even x86 pays at least a one cycle penalty for every misaligned access.

Yes, one cycle, and it's completely lost in the noise when it happens.

Depends on the app - for the networking stack, I agree.

To revisit Ben's comment: if we know something is likely to be misaligned, a RISC processor can efficiently load both parts and merge them (one cycle penalty vs a regular aligned load). Given misaligned accesses are infrequent enough to affect performance, it makes sense to do this in SW because it reduces cost of the HW design/test/mfg cycles.

...

Quoted 3 lines

It is an unavoidable axoim in the kernel networking. Unaligned accesses will happen, and they aren't a bug and therefore not worthy of mention in the kernel logs any more than "page was freed" :-)

Ok. If the kernel networking stack used get_unaligned() in the one place Peter originally found, x86/sparc64?/et al wouldn't see a difference. It would avoid traps on ia64 and parisc. Bad idea?
Any other arches it might help/hurt on?

grant

#45 lore raw
In reply to Peter Chubb
Quoted 3 lines · On Tue, 23 Sep 2003 13:38:19 -0700 Grant Grundler <[email protected]> wrote:

Given misaligned accesses are infrequent enough to affect performance, it makes sense to do this in SW because it reduces cost of the HW design/test/mfg cycles.

Intel actually optimizes this on the P4, what is your response to that? Is Intel wasting they time? :-)

Quoted 4 lines

Ok. If the kernel networking stack used get_unaligned() in the one place Peter originally found, x86/sparc64?/et al wouldn't see a difference. It would avoid traps on ia64 and parisc. Bad idea?
Any other arches it might help/hurt on?

It's needed on every access to every TCP and IP header portion for the case we're talking about in this thread, where the network device driver gives the networking a packet that ends up with unaligned IP and TCP headers.

I once considered adding some get_unaligned() uses to the TCP option parsing code, guess who rejected that patch? It wasn't me, it was Linus himself and I came to learn that he's right on this one.

#46 lore raw
In reply to Peter Chubb
Quoted 2 lines · Tony

a) the programmer is playing fast and loose with types and/or casts.
b) the end-user is going to be disappointed with the performance.

c) the programmer is being clever and knows the unaligned access is cheaper on average than the cost of making sure it cant happen

Quoted 2 lines

Looking at a couple of ia64 build servers here I see zero unaligned access messages in the logs.

Anyone who can deliver network traffic to your box can soon fix that...

#47 lore raw
In reply to Peter Chubb
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line · Alan Cox

Looking at a couple of ia64 build servers here I see zero unaligned access messages in the logs.

Alan> Anyone who can deliver network traffic to your box can soon fix that...

Not if he's running Red Hat. This is on a Red Hat 9 machine (x86, just so you can't argue it's ia64-specific...):

 $ fgrep LOGLEVEL /etc/rc.sysinit
 /bin/dmesg -n $LOGLEVEL
 $ fgrep LOGLEVEL /etc/sysconfig/*
 /etc/sysconfig/init:LOGLEVEL=3

Red Hat users won't be bothered by unaligned messages.

	--david
#48 lore raw
In reply to Peter Chubb
Quoted 2 lines · David S. Miller

Intel actually optimizes this on the P4, what is your response to that? Is Intel wasting they time? :-)

nono...but Intel doesn't have a choice on x86. They have to optimize for the binaries that are out there. Compatibility is everything in that market space.

And someone at Intel obviously agrees the newer architectures should support misaligned access in SW since ever RISC chip they've built (starting with i860, ~1989) does it that way.

Quoted 4 lines

It's needed on every access to every TCP and IP header portion for the case we're talking about in this thread, where the network device driver gives the networking a packet that ends up with unaligned IP and TCP headers.

Yeah, I don't use most LAN features (PPPoE, VLAN, Appletalk, etc). I naively thought there must be a subset everyone uses...but defining that subset sounds like a rat hole I shouldn't go near.

Quoted 3 lines

I once considered adding some get_unaligned() uses to the TCP option parsing code, guess who rejected that patch? It wasn't me, it was Linus himself and I came to learn that he's right on this one.

I'm not totally comfortable with that. The NICs I care about seem to "bias the buffer address" to compensate for some "common case". Seems like those cases would be cheaper (and more portable) to add the get_unaligned() calls in the networking stack....I don't know though really.

thanks,
grant

#49 lore raw
In reply to Peter Chubb
Quoted 6 lines · Alan Cox
Quoted 2 lines · Tony

a) the programmer is playing fast and loose with types and/or casts.
b) the end-user is going to be disappointed with the performance.

c) the programmer is being clever and knows the unaligned access is cheaper on average than the cost of making sure it cant happen

Which is great until the "cleverly written" program is fed a data set that pushes into the unaligned case far more frequently than the programmer anticipated.

Quoted 5 lines
Quoted 2 lines

Looking at a couple of ia64 build servers here I see zero unaligned access messages in the logs.

Anyone who can deliver network traffic to your box can soon fix that...

See answer above :-)

-Tony Luck

#50 lore raw
In reply to Peter Chubb
Quoted 2 lines · David S. Miller

As I understand it, you even do this stupid printk for user apps as well, that makes it more than rediculious.

Just as a point of interest, as an application programmer I think it's the type of thing you want to know about quite loudly. The only benchmark style figure on this I've ever seen is in this paper

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dv_vstechart/html/vcconWindowsDataAlignmentOnIPFX86X86-64.asp

(forgive the long url) which, if you scroll down to the little graph, shows OS fixup is about 450 times slower that an aligned access. That sucks, and if I didn't realise I'd done it, it would suck even more.

Oh, and I think fixing it up automatically & warning is much more useful than sending a (by default) terminating signal to the program; though I'm sure others might disagree.

-i
[email protected]
http://www.gelato.unsw.edu.au

#51 lore raw
In reply to Peter Chubb
Quoted 7 lines · David S. Miller
Quoted 3 lines

That's not true; they could be avoided by using get_unaligned() and put_unaligned(). You just don't want to because they'd make sparc suck.

Not only sparc would be effected by this. Using {get,put}_unaligned() all over the networking would incur a penalty for many platforms, not just sparc.

Really? I'd have thought that get/put_unaligned would be a simple load/store for architectures which wish to implement it in that manner.

Other architectures could take it as an optimisation hint, to avoid taking a trap. They'd probably still need to implement the fixup, but if a few of these hints could reduce the trap frequency significantly then it may be worth doing?

I guess it depends on how many of these hints would be needed at the source level to avoid "most" of the traps.

#52 lore raw
In reply to Peter Chubb
Quoted 3 lines · Tony

Which is great until the "cleverly written" program is fed a data set that pushes into the unaligned case far more frequently than the programmer anticipated.

Which is why the people who work on the networking are well aware of the issues and will make sure the common case never triggers these unaligned accesses.

People writing protocol stacks _don't_ feed these data unaligned cases out onto the wire, because like us they want the networking to go fast. Why in the world do you think they specify in the very RFCs that define the protocols that one should use NOP options in the TCP option area in order to align TCP timestamps on a 32-bit boundry?

Do you think they say this so people can go ahead and use memmove()'s and byte loads all over the place anyways?

No, rather, they specify things so that unless you do something absolutely stupid all the shit is aligned properly.

It is absurdly stupid to do byte loads of TCP and IP header bits just because one tenth of one hundredths of one percent of systems have some configuration where word and half-word loads of these things will be unaligned _AND_ be slow on that cpu.

#53 lore raw
In reply to Peter Chubb
Quoted 2 lines · On Wed, 24 Sep 2003 09:04:12 +1000 Ian Wienand <[email protected]> wrote:

Just as a point of interest, as an application programmer I think it's the type of thing you want to know about quite loudly.

Sure, and I agree that you should have the choice to see these messages at the verbosity you desire, and ia64 has the capability to control things in this way.

What I disagree with is spitting out log messages about such things happening in the kernel. Because that is remotely triggerable. As Alan mentioned, if I can send packets to your ia64 box I can make it spew system log messages.

#54 lore raw
In reply to Peter Chubb
Quoted 3 lines · On Tue, 23 Sep 2003 15:35:40 -0700 Grant Grundler <[email protected]> wrote:

And someone at Intel obviously agrees the newer architectures should support misaligned access in SW since ever RISC chip they've built (starting with i860, ~1989) does it that way.

That's a amusing coincidence since at least some people think ia64 will end up the same way the i860 did :-)

In the past I did always advocate things the way you are right now, but these days I think I've been wrong the whole time and Intel on x86 is doing the right thing.

They do everything in hardware and this makes the software so much simpler. Sure, there's a lot of architectually inherited complexity in the x86 family, but their engineering priorities mean there is so much other stuff you simply never have to think about as a programmer.

#55 lore raw
In reply to Peter Chubb
Quoted 11 lines · On Tue, 23 Sep 2003 16:15:29 -0700 Andrew Morton <[email protected]> wrote:
Quoted 7 lines · David S. Miller
Quoted 3 lines

That's not true; they could be avoided by using get_unaligned() and put_unaligned(). You just don't want to because they'd make sparc suck.

Not only sparc would be effected by this. Using {get,put}_unaligned() all over the networking would incur a penalty for many platforms, not just sparc.

Really? I'd have thought that get/put_unaligned would be a simple load/store for architectures which wish to implement it in that manner.

Only on systems that have the "load upper/lower-unaligned" instructions. On others it's a memmove().

#56 lore raw
In reply to Peter Chubb
Quoted 3 lines · On Tue, 23 Sep 2003 17:13:33 -0700 Grant Grundler <[email protected]> wrote:

It might. I be happy to share what I know about i860/i960 over pizza. I worked on ATT SVR4 port to i860 in 1989/90 and things were quite different then...

Q: How does an OS context switch work on the i860?
A: The user sneezes and the kernel cleans up after it.

I've hung out with i860 hackers already in my past :)

#57 lore raw
In reply to Peter Chubb
Quoted 2 lines · On Wed, 24 Sep 2003 01:14:56 +0100 Matthew Wilcox <[email protected]> wrote:

(I can see this descending into get_unaligned_likely() and get_aligned_unlikely() which i'd rather avoid ...)

get_unaligned_unlikely() is exactly what just loading the 32-bit value as if it were aligned is, ie. what we're doing right now.

People who want top performance should not put a networking card into their machine that can only DMA packets to 32-byte or whatever boundaries. That's exactly the limitation NS83820 has and therefore why it should be avoided like the plague by performance conscious folks.

#58 lore raw
In reply to Peter Chubb
Quoted 9 lines · David S. Miller
Quoted 3 lines · On Tue, 23 Sep 2003 15:35:40 -0700 Grant Grundler <[email protected]> wrote:

And someone at Intel obviously agrees the newer architectures should support misaligned access in SW since ever RISC chip they've built (starting with i860, ~1989) does it that way.

That's a amusing coincidence since at least some people think ia64 will end up the same way the i860 did :-)

It might. I be happy to share what I know about i860/i960 over pizza. I worked on ATT SVR4 port to i860 in 1989/90 and things were quite different then...

Quoted 3 lines

In the past I did always advocate things the way you are right now, but these days I think I've been wrong the whole time and Intel on x86 is doing the right thing.

I'm not so interested in "right" or "wrong". I'd just like to see other arches besides x86 work well (ia64, parisc in particular) and that includes how linux deals with unaligned accesses. If x86 is the gold standard for "the right way", we'd be using bounce buffers for DMA to highmem (well, PAE support would get added somehow) and 64-bit kernels wouldn't have happened...but linux so far seems to accomodate other arches *when reasonable*. I'll follow Andrew Morton's comments...

Quoted 4 lines

They do everything in hardware and this makes the software so much simpler. Sure, there's a lot of architectually inherited complexity in the x86 family, but their engineering priorities mean there is so much other stuff you simply never have to think about as a programmer.

Yes, true. But I think we are digressing from the original thread here... (/me works his way around another really big rat hole :^)

thanks,
grant

#59 lore raw
In reply to Peter Chubb
Quoted 9 lines · David S. Miller
Quoted 6 lines
Quoted 3 lines

Not only sparc would be effected by this. Using {get,put}_unaligned() all over the networking would incur a penalty for many platforms, not just sparc.

Really? I'd have thought that get/put_unaligned would be a simple load/store for architectures which wish to implement it in that manner.

Only on systems that have the "load upper/lower-unaligned" instructions. On others it's a memmove().

It is at the moment, but why should it be? Why can't it be implemented as load-and-trap if that's the fastest way to do it?

(I can see this descending into get_unaligned_likely() and get_aligned_unlikely() which i'd rather avoid ...)

Signature · 3 lines
"It's not Hollywood.  War is real, war is primarily not about defeat or
victory, it is about death.  I've seen thousands and thousands of dead bodies.
Do you think I want to have an academic debate on this subject?" -- Robert Fisk
#61 lore raw
In reply to Peter Chubb
Quoted 9 lines · David S. Miller

That's a amusing coincidence since at least some people think ia64 will end up the same way the i860 did :-)
In the past I did always advocate things the way you are right now, but these days I think I've been wrong the whole time and Intel on x86 is doing the right thing.
They do everything in hardware and this makes the software so much simpler. Sure, there's a lot of architectually inherited complexity in the x86 family, but their engineering priorities mean there is so much other stuff you simply never have to think about as a programmer.

Several of the x86 "hardware assists" need some rather hefty hacks codewise to cope with their concomitant data structure proliferations under industrial workloads, and generally have me begging for RISC's system-level features instead (which, of course, require various undoings of Linux' x86 crossdressings to exploit).

Given the reactions in prior threads, this message clearly needs to wait a long while before it will ever be heard.

-- wli

#62 lore raw
In reply to Peter Chubb
Quoted 1 line · On Tue, 23 Sep 2003 10:56:33 -0700 David Mosberger <[email protected]>…

An event that causes a slow-down of 500 times or so is not "normal".

For the slow path of IP option processing it is. When using IP embedded in appletalk, it is.

It's all slow path stuff.

Why is this so hard to understand?

#63 lore raw
In reply to Peter Chubb
Quoted 4 lines · On Tue, 23 Sep 2003 13:00:03 -0700 David Mosberger <[email protected]>…

But they _are_ useful. Peter certainly noticed very quickly that there was a problem with the ns83820 driver. Do you think it would have been better for the kernel to run silently at greatly degraded performance? I think not.

It's not even decided that the copying version is faster.

I bet for bulk data transfers it isn't faster at all, because instead of spinning in the unaligned trap handler your spinning swiping the L2 cache clean with the copies. I'll take some traps over twice the data footprint any day, because I can optimize the traps but I can't make the memory overhead go away.

Also, how much have you optimized your unaligned trap handler? If it's a thousand cycles now, do you think you could cut that overhead say in half? I bet you could, and then some. I bet a clever handler could be done on ia64 or sparc64 in a hundred cycles or so, maybe even faster.

Furthermore, like I said, if you want performance you aren't going to stick a NS83820 in your box. Even the author of the driver admits this and it's not like this chip is the only ~$30USD gigabit chip in town anymore :)

#64 lore raw
In reply to Peter Chubb
Quoted 11 lines · Tony
Quoted 6 lines · Alan Cox
Quoted 2 lines · Tony

a) the programmer is playing fast and loose with types and/or casts.
b) the end-user is going to be disappointed with the performance.

c) the programmer is being clever and knows the unaligned access is cheaper on average than the cost of making sure it cant happen

Which is great until the "cleverly written" program is fed a data set that pushes into the unaligned case far more frequently than the programmer anticipated.

Didn't we recently fix a DoS attack on the TCP stack that was basically this sort of thing with hashes? And crafting an attack against this would be a lot simpler than the hash-function attack.....

#65 lore raw
In reply to Peter Chubb
Quoted 1 line · David S. Miller

It's not even decided that the copying version is faster.

On Alpha it is - I did some experimentation yesterday with
the tulip driver and "rx_copybreak=0" vs default "rx_copybreak18". That's true, copying the large packet is very expensive due to constant cache misses. On UP1500 it costs more than 2500 cycles per 1.5Kb copy on large ftp transfers.
OTOH, minimal time to handle unaligned load is ~200 cycles (ev6 cpu), but only when the trap handler itself and its data are in L1 cache (like doing unaligned load in a tight loop). In the real life average time is ~400 cycles due to cache misses.
Further, with "rx_copybreak=0" I've got 20 (!) unaligned traps per TCP packet, i.e. penalty was 8000 vs 2500 cycles...
Note that my network setup is as simple as possible - no routing/filtering etc., just a networked workstation. I guess one with a more complex routing/firewalling configuration would have a LOT more unaligned traps per packet.

Ivan.

#66 lore raw
In reply to Peter Chubb
Quoted 11 lines · On Thu, 25 Sep 2003 20:11:27 +0400 Ivan Kokshaysky <[email protected]>…

On Alpha it is - I did some experimentation yesterday with the tulip driver and "rx_copybreak=0" vs default "rx_copybreak18". That's true, copying the large packet is very expensive due to constant cache misses. On UP1500 it costs more than 2500 cycles per 1.5Kb copy on large ftp transfers.
OTOH, minimal time to handle unaligned load is ~200 cycles (ev6 cpu), but only when the trap handler itself and its data are in L1 cache (like doing unaligned load in a tight loop). In the real life average time is ~400 cycles due to cache misses.
Further, with "rx_copybreak=0" I've got 20 (!) unaligned traps per TCP packet, i.e. penalty was 8000 vs 2500 cycles...

Fine, then we should have something like an rx_copybreak scheme in the ns83820 driver too.

Thanks for your data.