Modpost warning on Alchemy

Somebody with a clue on the Alchemy stuff may want to look into this mostpost warning:

MODPOST vmlinux.o
WARNING: vmlinux.o(.text+0x1e32dc): Section mismatch: reference to .init.text:add_wired_entry (between 'config_access' and 'config_write')
LD vmlinux

All the PCI config space accessors on Alchemy will call arch/mips/pci/ops-au1000.c:config_access which in turn calls add_wired_entry add_wired_entry in turn is an __init function so it's only a matter of luck if the PCI code doesn't explode on Alchemy.

So could somebody Alchemist try to rewrite this to use ioremap() instead?

Thanks,

Ralf

Quoted 1 line

So could somebody Alchemist try to rewrite this to use ioremap() instead?

Will ioremap() work for 4 GB range? I guess not.

Quoted 3 lines

Quoted 1 line

So could somebody Alchemist try to rewrite this to use ioremap() instead?

Will ioremap() work for 4 GB range? I guess not.

Of course it will. It shall work with whatever physical address space is
supported by your MMU. As long as the MMU is handled correctly that is, but I guess I may have omitted this clarification as obvious.

Maciej

Quoted 6 lines

Quoted 3 lines

Quoted 1 line

So could somebody Alchemist try to rewrite this to use ioremap() instead?

Will ioremap() work for 4 GB range? I guess not.

Of course it will. It shall work with whatever physical address
space is supported by your MMU. As long as the MMU is handled correctly that is, but I guess I may have omitted this clarification as obvious.

Even on a CPU with 36-bit physical address? ;-)

Quoted 1 line

Maciej

Quoted 5 lines

Quoted 3 lines

Of course it will. It shall work with whatever physical address space is supported by your MMU. As long as the MMU is handled correctly that is, but I guess I may have omitted this clarification as obvious.

Even on a CPU with 36-bit physical address? ;-)

WTF... I know I've typed "32-bit CPU"! :-/

Quoted 7 lines

Quoted 5 lines

Quoted 3 lines

Of course it will. It shall work with whatever physical address space is supported by your MMU. As long as the MMU is handled correctly that is, but I guess I may have omitted this clarification as obvious.

Even on a CPU with 36-bit physical address? ;-)

WTF... I know I've typed "32-bit CPU"! :-/

It does not matter. The physical address of an I/O resource can be

treated as a cookie that is converted, typically though an MMU, to another cookie that can be used with {read,write}{b,w,l}(). Of course you may have troubles ioremap()ping say 4GB of I/O space on a processor with a 32-bit virtual address space, but that is a corner case and typically your I/O space will be sparsely occupied.

On a MIPS32 processor you have 512MB of KSEG0/1 unmapped virtual address
space available for I/O devices located within the first 512MB of the

physical address space plus 1GB of KSEG2 mapped virtual address space available for I/O devices located anywhere in the physical address space. That gives you from 1GB to 1.5GB of virtual address space for I/O which is enough for all the usual cases.

Maciej

PCI config. space is mapped at 0x600000000, well beyond KGSEG0/1.

Thanks for wasting time on my education about MIPS. ;-)

Quoted 1 line

PCI config. space is mapped at 0x600000000, well beyond KGSEG0/1.

It is still just fine with ioremap() -- it will simply use KSEG2 in this
case. You cannot bypass the TLB here with a 32-bit processor no matter what.

And regarding what you have written above and the size issue you
mentioned in another e-mail (do you map the whole PCI config space linearly in the physical address space of the CPU or suchlike?) -- PCI

config space accesses are rare (by design rather than chance), so

performance is a non-issue and it should be absolutely fine for you to call ioremap() and iounmap() in code specific for your PCI host bridge for the required fragment upon every access. There is no need for a permanent

map here. You probably waste more performance by taking away a TLB entry to wire it anyway.

Quoted 1 line

Thanks for wasting time on my education about MIPS. ;-)

Well, more about Linux perhaps than MIPS in general. :-)

Maciej

Quoted 3 lines

And regarding what you have written above and the size issue you mentioned in another e-mail (do you map the whole PCI config space linearly in the physical address space of the CPU or suchlike?) -- PCI

No, I don't. But that was why the original code preferred the wired entry
approach over ioremap() -- not to map a whole range...

Quoted 1 line

config space accesses are rare (by design rather than chance), so

That depends on the drivers used (some IDE drivers access it really often).

Quoted 3 lines

performance is a non-issue and it should be absolutely fine for you to call ioremap() and iounmap() in code specific for your PCI host bridge for the required fragment upon every access. There is no need for a permanent

That's an idea -- however, as the currecnt code uses a cached mapping, this
part would certainly need to be saved in the new implementaion -- if someone will go and fix it eventually. :-)

Quoted 1 line

Well, more about Linux perhaps than MIPS in general. :-)

Let's say that was about Linux/MIPS. But the key word was "wasting". ;-)

Quoted 1 line

That depends on the drivers used (some IDE drivers access it really often).

It is their problem I would say -- there is a design problem either in
these drivers or the hardware handled. The PCI spec is very explicit that the config space is meant to be seldom accessed only. Device initialization/shutdown and bus error recovery are the normal places.

An awful lot of vendors get it horribly wrong and many end up needing configuration space access even in IRQ handlers. Dishonourable mentions to ATI for example ;)

Quoted 6 lines

Quoted 3 lines

And regarding what you have written above and the size issue you mentioned in another e-mail (do you map the whole PCI config space linearly in the physical address space of the CPU or suchlike?) -- PCI

No, I don't. But that was why the original code preferred the wired entry
approach over ioremap() -- not to map a whole range...

So what is the issue with the size then? How big is the area?

Quoted 7 lines

Quoted 3 lines

performance is a non-issue and it should be absolutely fine for you to call ioremap() and iounmap() in code specific for your PCI host bridge for the required fragment upon every access. There is no need for a permanent

That's an idea -- however, as the currecnt code uses a cached mapping, this
part would certainly need to be saved in the new implementaion -- if someone will go and fix it eventually. :-)

Well, cached mapping does not seem particularly wise with PCI
configuration registers, but you have got the ioremap_cachable() call if you insist. ;-)

Quoted 3 lines

Quoted 1 line

Well, more about Linux perhaps than MIPS in general. :-)

Let's say that was about Linux/MIPS. But the key word was "wasting". ;-)

I reckon the key is how you look at it. ;-)

Maciej

Quoted 1 line

So what is the issue with the size then? How big is the area?

I've already said: 4 gigs! At least in theory, actually it's 2 gigs due to

a device # being limited to 0 thru 19 (address bits 11 thru 30 are used as IDSELx).

I meant that the implementation "caches" the 8 KiB mapping used for the
last config. access.

There was no need to tell me about how KSEG0/1/2 work -- that's why I
cosidered it wasting time. :-)

Quoted 3 lines

Quoted 1 line

So what is the issue with the size then? How big is the area?

I've already said: 4 gigs! At least in theory, actually it's 2 gigs due to

Oh, I mistook it for the base physical address, sorry.

Quoted 2 lines

a device # being limited to 0 thru 19 (address bits 11 thru 30 are used as IDSELx).

It does not help too much with a 32-bit virtual address space indeed.
Though I gather it has to be very sparsely populated as 16MiB is enough to cover the whole configuration space of a single PCI bus tree. Thus it has

to be another example where the chip designer "forgot" to talk to software people. Or a shifter was traded for software performance and complexity. ;-)

Quoted 2 lines

There was no need to tell me about how KSEG0/1/2 work -- that's why I
cosidered it wasting time. :-)

As I say -- the key is how you look at it. There are other readers on
the list who may benefit; it is archived too. In this sense I can hardly consider it a waste of time. And it was fun to explain and no fun shall be ever considered of no use. :-)

Maciej

Quoted 3 lines

It does not help too much with a 32-bit virtual address space indeed. Though I gather it has to be very sparsely populated as 16MiB is enough to cover the whole configuration space of a single PCI bus tree. Thus it has

Hm, maybe 16 MiB would be enough indeed, as the Alchemy CPUs are known to
not support bus masters behind PCI bridges...

Quoted 6 lines

Quoted 3 lines

It does not help too much with a 32-bit virtual address space indeed. Though I gather it has to be very sparsely populated as 16MiB is enough to cover the whole configuration space of a single PCI bus tree. Thus it has

Hm, maybe 16 MiB would be enough indeed, as the Alchemy CPUs are known to
not support bus masters behind PCI bridges...

That is unrelated -- for configuration accesses (assuming the basic

configuration space) you need: 8 bits for the bus number + 5 bits for the device number + 3 bits for the function number + 8 bits for the register number. The total is 24 bits.

It is up to hardware to sort it out and put the right bits on the bus.

Maciej

Yeah, that the format of type 1 cycles.

Unfortunately, Alchemy designers were too lazy to implement a simple
translation scheme for type 0 cycles. They probably though that with 36-bit bus the may not limit themselves... :-)

Quoted 5 lines

It is still just fine with ioremap() -- it will simply use KSEG2 in
this case. You cannot bypass the TLB here with a 32-bit processor no matter what.

And regarding what you have written above and the size issue you
mentioned in another e-mail (do you map the whole PCI config space linearly in the physical address space of the CPU or suchlike?) -- PCI

No, I don't. But that was why the original code preferred the wired
entry approach over ioremap() -- not to map a whole range...

Quoted 1 line

config space accesses are rare (by design rather than chance), so

That depends on the drivers used (some IDE drivers access it really
often).

Quoted 1 line

performance is a non-issue and it should be absolutely fine for you to call ioremap() and iounmap() in code specific for your PCI host bridge for the required fragment upon every access. There is no need for a permanent

That's an idea -- however, as the currecnt code uses a cached

Quoted 2 lines

No, I don't. But that was why the original code preferred the wired
entry approach over ioremap() -- not to map a whole range...

Not the only one: dynamic ioremap() seems to be impossible in interrupt
context.

Quoted 5 lines

Quoted 2 lines

No, I don't. But that was why the original code preferred the wired
entry approach over ioremap() -- not to map a whole range...

Not the only one: dynamic ioremap() seems to be impossible in interrupt
context.

Well, ioremap() may sleep indeed. How about using a softirq then?
Broken hardware (=one that requires PCI configuration accesses from the IRQ context) is not an excuse to extend the breakage over to software.

Lockdep would trigger on ioremap from an interrupt almost immediately. But I guess not a whole lot of people are using it, probably because they think they're safe from locking problems on uniprocessors ...

Quoted 5 lines

Quoted 2 lines

No, I don't. But that was why the original code preferred the wired
entry approach over ioremap() -- not to map a whole range...

Not the only one: dynamic ioremap() seems to be impossible in interrupt
context.

Well, ioremap() may sleep indeed.

How about using a softirq then?

Broken hardware (=one that requires PCI configuration accesses from the IRQ context) is not an excuse to extend the breakage over to software.

Maciej

Quoted 8 lines

Quoted 6 lines

Quoted 2 lines

No, I don't. But that was why the original code preferred the wired
entry approach over ioremap() -- not to map a whole range...

Not the only one: dynamic ioremap() seems to be impossible in
interrupt
context.

Well, ioremap() may sleep indeed.

So, the only viable option of using sofirq() is mapping all the
sparce 2KiB regions at the __init time -- that will waste every half of page though...

Quoted 1 line

How about using a softirq then?

Using softirq for what?

Quoted 3 lines

Quoted 1 line

How about using a softirq then?

Using softirq for what?

For what you would otherwise want to do in the interrupt context.

If you want an example, then please have a look at phylib, where in the

interrupt handler you cannot even access the interrupt status register of the originating device (there may be a number of them sharing the same line too), because in general it is impossible to access the MDIO bus from the interrupt context.

Maciej

Quoted 1 line

If you want an example, then please have a look at phylib, where in the

No. I don't feel like rewriting IDE/libata cores. ;-)

Quoted 7 lines · Maciej W. Rozycki

Quoted 3 lines

Quoted 1 line

So could somebody Alchemist try to rewrite this to use ioremap() instead?

Will ioremap() work for 4 GB range? I guess not.

Of course it will. It shall work with whatever physical address space is
supported by your MMU. As long as the MMU is handled correctly that is, but I guess I may have omitted this clarification as obvious.

Even on a CPU with 36-bit physical address? ;-)

Quoted 5 lines

Quoted 3 lines

Of course it will. It shall work with whatever physical address space is
supported by your MMU. As long as the MMU is handled correctly that is, but I guess I may have omitted this clarification as obvious.

Even on a CPU with 36-bit physical address? ;-)

Nope. This is one problem for example with ioremap on a Pentium Pro.

Quoted 3 lines

Quoted 1 line

Even on a CPU with 36-bit physical address? ;-)

Nope. This is one problem for example with ioremap on a Pentium Pro.

Well, but we only consider MIPS processors here which do not have such
odd restrictions resulting from bad design decisions in the past. ;-) The 32-bit TLB entry format allows for up to 36 bits of the physical address space (34 bits if support for the page size of 1kB has been

enabled). For anything beyond that you need a 64-bit MIPS processor using the 64-bit TLB entry format.

Quoted 5 lines

Quoted 3 lines

Quoted 1 line

Even on a CPU with 36-bit physical address? ;-)

Nope. This is one problem for example with ioremap on a Pentium Pro.

Well, but we only consider MIPS processors here which do not have such odd restrictions resulting from bad design decisions in the past. ;-) The 32-bit TLB entry format allows for up to 36 bits of the physical address space (34 bits if support for the page size of 1kB has been

So does the Pentium Pro. We can map 36bit physical addresses.

Quoted 1 line

enabled). For anything beyond that you need a 64-bit MIPS processor using the 64-bit TLB entry format.

Your problem is a little higher up the stack. ioremap takes an unsigned long, which on a 32bit system usually means you can't give it a 36bit bus address to remap.

Alan

Quoted 2 lines

enabled). For anything beyond that you need a 64-bit MIPS processor using the 64-bit TLB entry format.

Your problem is a little higher up the stack. ioremap takes an unsigned long, which on a 32bit system usually means you can't give it a 36bit bus address to remap.

#ifdef CONFIG_64BIT_PHYS_ADDR
typedef unsigned long long phys_t;
#else
typedef unsigned long phys_t;
#endif

so no problem here as long as you enable CONFIG_64BIT_PHYS_ADDR which is implied in such a case.

Quoted 2 lines

so no problem here as long as you enable CONFIG_64BIT_PHYS_ADDR which is implied in such a case.

Which happens to be the solution that is Linus-incompatible so I may eventually have to change it ;-)

Quoted 2 lines

Which happens to be the solution that is Linus-incompatible so I may eventually have to change it ;-)

Well, however we do it, the width of the physical address cookie used
with ioremap() should not be forced to be related to the width of the virtual address space in any way. I see no reason for us to be crippled by limitations of some other architectures or, worse yet, by ones of some code specific to some other platform.

It's the physical not virtual address and as you say yourself it's a cookie so could potencially be some opaque value that isn't a physical address at all. And I wouldn't quite call it crippling that's certainly way exagerated.