Skip to contentSkip to conversation

s2io ppc64 fix for readq/writeq

PATCH In review Benjamin Herrenschmidt 3 participants 13 messages Nov 6, 2006 – Nov 7, 2006

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 s2io ppc64 fix for readq/writeq

The s2io driver is redefining it's own readq/writeq based on readl/writel when the platform doesn't provide native ones. However, it currently does so by testing #ifndef readq. While that works for now, we are about to change ppc64 to use inline functions rather that macros for all those IO accessors which will break that test. This fixes it. I don't have anything less ugly at hand unfortunately.

---

The patch changing ppc64 own definition is scheduled to go in 2.6.20 when the merge window opens, so it would be nice if this patch could go in a similar timeframe, provided that you agree with it of course. It can go earlier as it won't break current ppc64.

drivers/net/s2io.h +5 -3
Index: linux-cell/drivers/net/s2io.h
===================================================================
@@ -862,8 +862,10 @@ struct s2io_nic {
 #define RESET_ERROR 1;
 #define CMD_ERROR   2;
 
-/*  OS related system calls */
-#ifndef readq
+/* OS related system calls. Note that ppc64 has readq defined as
+ * an inline, not a macro
+ */
+#if !defined(CONFIG_PPC64) && !defined(readq)
 static inline u64 readq(void __iomem *addr)
 {
 	u64 ret = 0;
@@ -875,7 +877,7 @@ static inline u64 readq(void __iomem *ad
 }
 #endif
 
-#ifndef writeq
+#if !defined(CONFIG_PPC64) && !defined(writeq)
 static inline void writeq(u64 val, void __iomem *addr)
 {
 	writel((u32) (val), addr);
#1 lore raw
Quoted 33 lines · Benjamin Herrenschmidt

The s2io driver is redefining it's own readq/writeq based on readl/writel when the platform doesn't provide native ones. However, it currently does so by testing #ifndef readq. While that works for now, we are about to change ppc64 to use inline functions rather that macros for all those IO accessors which will break that test. This fixes it. I don't have anything less ugly at hand unfortunately.

---

The patch changing ppc64 own definition is scheduled to go in 2.6.20 when the merge window opens, so it would be nice if this patch could go in a similar timeframe, provided that you agree with it of course. It can go earlier as it won't break current ppc64.

Index: linux-cell/drivers/net/s2io.h
===================================================================

drivers/net/s2io.h +4 -2
@@ -862,8 +862,10 @@ struct s2io_nic {
 #define RESET_ERROR 1;
 #define CMD_ERROR   2;
 
-/*  OS related system calls */
-#ifndef readq
+/* OS related system calls. Note that ppc64 has readq defined as
+ * an inline, not a macro
+ */
+#if !defined(CONFIG_PPC64) && !defined(readq)
 static inline u64 readq(void __iomem *addr)
 {
 	u64 ret = 0;

drivers/net/s2io.h static inline u64 readq(void __iomem *ad · line 881

Quoted 6 lines
drivers/net/s2io.h +1 -1
@@ -875,7 +877,7 @@ static inline u64 readq(void __iomem *ad
 }
 #endif
 
-#ifndef writeq
+#if !defined(CONFIG_PPC64) && !defined(writeq)
 static inline void writeq(u64 val, void __iomem *addr)

This seems a bit ugly. Could you add

	#define readq readq

to your platform instead?

I generally think it's a bug in the kernel-wide API, if use of said API requires arch-specific ifdefs.

Or maybe the problem could be solved another way, by guaranteeing that a "good enough for drivers" readq() and writeq() exist on all platforms, even 32-bit platforms where the operation isn't inherently atomic.

Jeff

#2 lore raw
Quoted 6 lines

This seems a bit ugly. Could you add

	#define readq readq

to your platform instead?

That's ugly too imho but I suppose I can do it :-)

Quoted 2 lines

I generally think it's a bug in the kernel-wide API, if use of said API requires arch-specific ifdefs.

Yes. I agree. In that specific case, I suppose what you propose is the least ugly of the solutions. HAVE_ARCH_* is pretty much out of fascion (and I tend to agree with Linus that it's not pretty anyway).

Actually, I tend to think in that specific case that the driver defining something called readq and writeq based on a pair of readl's and writel's is fairly bogus though.

Quoted 3 lines

Or maybe the problem could be solved another way, by guaranteeing that a "good enough for drivers" readq() and writeq() exist on all platforms, even 32-bit platforms where the operation isn't inherently atomic.

I'd rather not provide readq/writeq if they aren't atomic.

Ben.

#3 lore raw
Quoted 14 lines · Benjamin Herrenschmidt
Quoted 5 lines

This seems a bit ugly. Could you add

	#define readq readq

to your platform instead?

That's ugly too imho but I suppose I can do it :-)

Quoted 1 line

I generally think it's a bug in the kernel-wide API, if use of said API requires arch-specific ifdefs.

Yes. I agree. In that specific case, I suppose what you propose is the least ugly of the solutions. HAVE_ARCH_* is pretty much out of fascion (and I tend to agree with Linus that it's not pretty anyway).

Quoted 7 lines

Actually, I tend to think in that specific case that the driver defining something called readq and writeq based on a pair of readl's and writel's is fairly bogus though.

Quoted 1 line

Or maybe the problem could be solved another way, by guaranteeing that a "good enough for drivers" readq() and writeq() exist on all platforms, even 32-bit platforms where the operation isn't inherently atomic.

I'd rather not provide readq/writeq if they aren't atomic.

This is why I said "good enough for drivers". This is _key_.

I have run into several [PCI] devices with 64-bit registers, and __none__ of them had requirements such that the Linux platform code -must- provide an atomic readq/writeq. Probably because everybody wants to support 32-bit platforms with their devices.

What you call "fairly bogus" is precisely what drivers need. These devices with 64-bit registers just don't need the atomicity that arch developers harp about :)

Jeff

#4 lore raw
Quoted 10 lines

This is why I said "good enough for drivers". This is _key_.

I have run into several [PCI] devices with 64-bit registers, and __none__ of them had requirements such that the Linux platform code -must- provide an atomic readq/writeq. Probably because everybody wants to support 32-bit platforms with their devices.

What you call "fairly bogus" is precisely what drivers need. These devices with 64-bit registers just don't need the atomicity that arch developers harp about :)

Is there any consistency in that case in which half need to be read/written first ? Or none of these ever had side effects ?

Ben.

#5 lore raw
Quoted 10 lines · Benjamin Herrenschmidt
Quoted 7 lines

This is why I said "good enough for drivers". This is _key_.

I have run into several [PCI] devices with 64-bit registers, and __none__ of them had requirements such that the Linux platform code -must- provide an atomic readq/writeq. Probably because everybody wants
to support 32-bit platforms with their devices.

What you call "fairly bogus" is precisely what drivers need. These devices with 64-bit registers just don't need the atomicity that arch developers harp about :)

Is there any consistency in that case in which half need to be read/written first ? Or none of these ever had side effects ?

Generally the kernel code should write the two 32-bit chunks to the memory-mapped region in order (low dword first), and let things take care of themselves from there.

That's pretty much the implementation that -every- driver copies, when they need readq/writeq to work on a 32-bit platform.

Jeff

#7 lore raw
In reply to Jeff Garzik
Quoted 6 lines · Jeff Garzik

This seems a bit ugly. Could you add

	#define readq readq

to your platform instead?

Heartily agreed. MUCH better than adding unrelated #if defined() stuff, whether arch-related or otherwise.

Linus

#8 lore raw
Quoted 11 lines · Linus Torvalds
Quoted 6 lines · Jeff Garzik

This seems a bit ugly. Could you add

	#define readq readq

to your platform instead?

Heartily agreed. MUCH better than adding unrelated #if defined() stuff, whether arch-related or otherwise.

I agree it's less ugly, though I still don't like it much :-)

Anyway, what do you think of Jeff proposal to just implement them as two 32 bits operations ? My arch guy side screams at the idea, but if, indeed, drivers generally cope fine with it, I suppose that's ok.

Ben.

#9 lore raw
Quoted 4 lines · Benjamin Herrenschmidt

Anyway, what do you think of Jeff proposal to just implement them as two 32 bits operations ? My arch guy side screams at the idea, but if, indeed, drivers generally cope fine with it, I suppose that's ok.

Last I saw, that's how normal PCI will split the IO anyway, so I guess it makes sense.

Linus

#10 lore raw
Quoted 9 lines · Linus Torvalds
Quoted 4 lines · Benjamin Herrenschmidt

Anyway, what do you think of Jeff proposal to just implement them as two 32 bits operations ? My arch guy side screams at the idea, but if, indeed, drivers generally cope fine with it, I suppose that's ok.

Last I saw, that's how normal PCI will split the IO anyway, so I guess it makes sense.

Hrm.. true indeed. I'll implement them that way for ppc32 then.

Ben.

#11 lore raw
Quoted 8 lines · Benjamin Herrenschmidt
Quoted 5 lines · Linus Torvalds
Quoted 3 lines · Benjamin Herrenschmidt

Anyway, what do you think of Jeff proposal to just implement them as two 32 bits operations ? My arch guy side screams at the idea, but if, indeed, drivers generally cope fine with it, I suppose that's ok.

Last I saw, that's how normal PCI will split the IO anyway, so I guess it makes sense.

Hrm.. true indeed. I'll implement them that way for ppc32 then.

Bonus points if you want to find-and-kill where individual drivers did

	#ifndef readq
	implement readq and writeq by hand...
	#endif

:)

#12 lore raw
Quoted 9 lines · Jeff Garzik
Quoted 8 lines · Benjamin Herrenschmidt
Quoted 6 lines · Linus Torvalds
Quoted 3 lines · Benjamin Herrenschmidt

Anyway, what do you think of Jeff proposal to just implement them as two 32 bits operations ? My arch guy side screams at the idea, but if, indeed, drivers generally cope fine with it, I suppose that's ok.

Last I saw, that's how normal PCI will split the IO anyway, so I guess it makes sense.

Quoted 7 lines
Quoted 1 line

Hrm.. true indeed. I'll implement them that way for ppc32 then.

Bonus points if you want to find-and-kill where individual drivers did

	#ifndef readq
	implement readq and writeq by hand...
	#endif

Yes, well, we would have to make sure all archs have them defined first though, but I suppose I can have a look later this week, maybe tomorrow. Shouldn't be too hard :)

Ben.

#6 lore raw
In reply to Jeff Garzik
Quoted 7 lines

Generally the kernel code should write the two 32-bit chunks to the memory-mapped region in order (low dword first), and let things take care of themselves from there.

That's pretty much the implementation that -every- driver copies, when they need readq/writeq to work on a 32-bit platform.

What do you mean by low dword first ? For example, the implementation in the s2io driver does:

static inline u64 readq(void __iomem *addr)

{
	u64 ret = 0;
	ret = readl(addr + 4);
	ret <<= 32;
	ret |= readl(addr);

	return ret;
}

static inline void writeq(u64 val, void __iomem *addr)

{
	writel((u32) (val), addr);
	writel((u32) (val >> 32), (addr + 4));
}

As you can see, it reads the -second- dword first (high order dword in little endian), but writes the first dword first (low order dword in little endian).

If there is any logic here, it's card specific.

Or is this really what PCI does when doing 64 bits accesses on a 32 bits PCI bus ? I would have expected the later (what write does) but this driver does it reverse on reads.

I'm tempted to go to the simple

#define readq readq for now until we clear that up.

Ben.