Skip to contentSkip to conversation

arm: Kirkwood: Add memory barriers to kwgbe_send/recv

PATCH New Simon Kagstrom 6 participants 10 messages Jul 1, 2009 – Jul 4, 2009 Network drivers

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 arm: Kirkwood: Add memory barriers to kwgbe_send/recv

Add memory barriers to kwgbe_send/recv

kwgbe_send/recv both have loops waiting for the hardware to set a bit. GCC 4.3.3 cleverly optimizes this to ... a while(1); loop. This patch introduces memory barriers to force re-loading of the transmit descriptor.

mb() wasn't defined for arm, but perhaps it should?

---
 drivers/net/kirkwood_egiga.c |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)
drivers/net/kirkwood_egiga.c +4 -0
index 3c5db19..abedf77 100644
@@ -513,6 +513,8 @@ static int kwgbe_send(struct eth_device *dev, volatile void *dataptr,
 			printf("Err..(%s) in xmit packet\n", __FUNCTION__);
 			return -1;
 		}
+		/* Memory barrier to see to it that cmd_sts is re-read */
+		asm volatile("" : : : "memory");
 	};
 	return 0;
 }
@@ -531,6 +533,8 @@ static int kwgbe_recv(struct eth_device *dev)
 			debug("%s time out...\n", __FUNCTION__);
 			return -1;
 		}
+		/* Memory barrier to see to it that cmd_sts is re-read */
+		asm volatile("" : : : "memory");
 	} while (p_rxdesc_curr->cmd_sts & KWGBE_BUFFER_OWNED_BY_DMA);
 
 	if (p_rxdesc_curr->byte_cnt != 0) {
Signature · 1 line
1.6.0.4
#1 lore raw

PATCH 1/1 arm: Kirkwood: Add memory barriers to kwgbe_send/recv

Hi Simon,

Quoted 7 lines

Add memory barriers to kwgbe_send/recv

kwgbe_send/recv both have loops waiting for the hardware to set a bit. GCC 4.3.3 cleverly optimizes this to ... a while(1); loop. This patch introduces memory barriers to force re-loading of the transmit descriptor.

mb() wasn't defined for arm, but perhaps it should?

You should rather use read/write accessor macros which "do the right thing". Try using readl() for the loop. Memory barriers do not belong into "upper level" code.

Cheers
Detlev

Signature · 7 lines
X-Windows has to be the most expensive way ever of popping up an Emacs
window.
                                          -- The UNIX Haters Handbook
--
DENX Software Engineering GmbH,      MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich,  Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: dzu at denx.de
#2 lore raw

PATCH 1/1 arm: Kirkwood: Add memory barriers to kwgbe_send/recv

Hi Detlev,

Quoted 7 lines · On Wednesday 01 July 2009 18:56:18 Detlev Zundel wrote:
Quoted 7 lines

Add memory barriers to kwgbe_send/recv

kwgbe_send/recv both have loops waiting for the hardware to set a bit. GCC 4.3.3 cleverly optimizes this to ... a while(1); loop. This patch introduces memory barriers to force re-loading of the transmit descriptor.

Quoted 5 lines
Quoted 1 line

mb() wasn't defined for arm, but perhaps it should?

You should rather use read/write accessor macros which "do the right thing". Try using readl() for the loop. Memory barriers do not belong into "upper level" code.

I know that we advertised the usage of the io accessor functions for quite some time now. But I'm not so sure here anymore. One disadvantage of this usage could be speed penalty by the usage of too many unnecessary barriers (included via the accessor functions on some platforms).

I remember a discussion either on the linuxppc-dev, linux-arm-kernel or the nextdev list about removing the accessor functions from the Linux version of this Marvell ethernet drivers (mv643xx_eth.c) at some time because of this speed penalty. Sorry but I don't have a link for this mlist discussion.

Just my 0.02$.

Best regards,
Stefan

=====================================================================
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-0 Fax: +49-8142-66989-80 Email: office at denx.de
=====================================================================

#3 lore raw

PATCH 1/1 arm: Kirkwood: Add memory barriers to kwgbe_send/recv

Quoted 4 lines · Stefan Roese

I know that we advertised the usage of the io accessor functions for quite some time now. But I'm not so sure here anymore. One disadvantage of this usage could be speed penalty by the usage of too many unnecessary barriers (included via the accessor functions on some platforms).

Hot paths (or tiny bootstraps) can be optimized if needed, but why invite screwups for the majority of code?

-Scott

#4 lore raw

PATCH 1/1 arm: Kirkwood: Add memory barriers to kwgbe_send/recv

Hi Stefan & Scott,

Quoted 8 lines
Quoted 4 lines · Stefan Roese

I know that we advertised the usage of the io accessor functions for quite some time now. But I'm not so sure here anymore. One disadvantage of this usage could be speed penalty by the usage of too many unnecessary barriers (included via the accessor functions on some platforms).

Hot paths (or tiny bootstraps) can be optimized if needed, but why invite screwups for the majority of code?

I fully agree with Scott here. Experience has shown that robustness pays off much more than premature optimizations. If you happen to have performance problems, then I also seriously advise not to optimize without actual performance data collected by a profiling run.

This will not be easy for U-Boot but still I've seen so many examples of effort put into "optimizations" which in the end turned out not to be relevant for the total performance at all.

Cheers
Detlev

Signature · 7 lines
Programming X-Windows is like trying to find the square root of pi
using roman numerals.
                                          -- The UNIX Haters Handbook
--
DENX Software Engineering GmbH,      MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich,  Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: dzu at denx.de
#5 lore raw

PATCH 1/1 arm: Kirkwood: Add memory barriers to kwgbe_send/recv

In reply to Simon Kagstrom
Quoted 10 lines

-----Original Message-----
From: u-boot-bounces at lists.denx.de [mailto:u-boot-bounces at lists.denx.de] On Behalf Of Simon Kagstrom
Sent: Wednesday, July 01, 2009 8:46 PM
To: U-Boot ML
Subject: [U-Boot] [PATCH] arm: Kirkwood: Add memory barriers to kwgbe_send/recv

Quoted 5 lines

Add memory barriers to kwgbe_send/recv

kwgbe_send/recv both have loops waiting for the hardware to set a bit.
GCC 4.3.3 cleverly optimizes this to ... a while(1); loop.

Struct used is defined as volatile here.
Is this not sufficient?

Regards..
Prafulla . .

Quoted 4 lines

This patch introduces memory barriers to force re-loading of the transmit descriptor.

mb() wasn't defined for arm, but perhaps it should?

#6 lore raw

PATCH 1/1 arm: Kirkwood: Add memory barriers to kwgbe_send/recv

Quoted 5 lines · On Wed, 1 Jul 2009 09:57:10 -0700 Prafulla Wadaskar <[email protected]>…
Quoted 3 lines

kwgbe_send/recv both have loops waiting for the hardware to set a bit.
GCC 4.3.3 cleverly optimizes this to ... a while(1); loop.

Struct used is defined as volatile here.
Is this not sufficient?

Right, but that was only for recv - send isn't volatile. I'll send a new patch with the required changes. The question is which :-):

1. Make structure volatile in send as well

2. Use readl() where needed

3. Keep the memory barrier approach (and maybe add a mb() macro to arm as well)

I'm happy to get suggestions!

I'll prepare the openrd_base board patches a bit more and send them to the list later today.

// Simon

#7 lore raw

PATCH 1/1 arm: Kirkwood: Add memory barriers to kwgbe_send/recv

Quoted 16 lines

-----Original Message-----
From: Simon Kagstrom [mailto:simon.kagstrom at netinsight.net]
Sent: Thursday, July 02, 2009 12:17 PM
To: Prafulla Wadaskar
Cc: U-Boot ML
Subject: Re: [U-Boot] [PATCH] arm: Kirkwood: Add memory barriers to kwgbe_send/recv

Quoted 1 line · On Wed, 1 Jul 2009 09:57:10 -0700 Prafulla Wadaskar <[email protected]>…
Quoted 1 line

kwgbe_send/recv both have loops waiting for the hardware

to set a

Quoted 2 lines
Quoted 2 lines

bit.
GCC 4.3.3 cleverly optimizes this to ... a while(1); loop.

Quoted 7 lines
Quoted 2 lines

Struct used is defined as volatile here.
Is this not sufficient?

Right, but that was only for recv - send isn't volatile. I'll send a new patch with the required changes. The question is which :-):

1. Make structure volatile in send as well

Hi Simon
I suggest to implement this change

Regards..
Prafulla . .

#8 lore raw

PATCH 1/1 arm: Kirkwood: Add memory barriers to kwgbe_send/recv

Hi Prafulla,

Quoted 17 lines
Quoted 17 lines

-----Original Message-----
From: Simon Kagstrom [mailto:simon.kagstrom at netinsight.net]
Sent: Thursday, July 02, 2009 12:17 PM
To: Prafulla Wadaskar
Cc: U-Boot ML
Subject: Re: [U-Boot] [PATCH] arm: Kirkwood: Add memory barriers to kwgbe_send/recv

Quoted 1 line · On Wed, 1 Jul 2009 09:57:10 -0700 Prafulla Wadaskar <[email protected]>…
Quoted 1 line

kwgbe_send/recv both have loops waiting for the hardware

to set a

Quoted 4 lines
Quoted 2 lines

bit.
GCC 4.3.3 cleverly optimizes this to ... a while(1); loop.

Struct used is defined as volatile here.
Is this not sufficient?

Quoted 6 lines
Quoted 4 lines

Right, but that was only for recv - send isn't volatile. I'll send a new patch with the required changes. The question is which :-):

1. Make structure volatile in send as well

Hi Simon
I suggest to implement this change

Did you read Documentation/volatile-considered-harmful.txt in a Linux source tree?

Cheers
Detlev

Signature · 5 lines
Practice random senselessness and act kind of beautiful.
--
DENX Software Engineering GmbH,      MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich,  Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: dzu at denx.de
#9 lore raw

PATCH 1/1 arm: Kirkwood: Add memory barriers to kwgbe_send/recv

In reply to Simon Kagstrom

Dear Simon Kagstrom,

Quoted 11 lines · In message <[email protected]> you wrote:
Quoted 5 lines · On Wed, 1 Jul 2009 09:57:10 -0700 Prafulla Wadaskar <[email protected]>…
Quoted 3 lines

kwgbe_send/recv both have loops waiting for the hardware to set a bit.
GCC 4.3.3 cleverly optimizes this to ... a while(1); loop.

Struct used is defined as volatile here.
Is this not sufficient?

Right, but that was only for recv - send isn't volatile. I'll send a new patch with the required changes. The question is which :-):

Quoted 8 lines

1. Make structure volatile in send as well

2. Use readl() where needed

3. Keep the memory barrier approach (and maybe add a mb() macro to arm as well)

I'm happy to get suggestions!

There is a pretty clear policy in U-Boot now: please always use accessor functions.

Best regards,

Wolfgang Denk

Signature · 7 lines
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
The shortest unit of time in the multiverse is the News York  Second,
defined  as  the  period  of  time between the traffic lights turning
green and the cab behind you honking.
                                - Terry Pratchett, _Lords and Ladies_