handle access to non-present IO ports on 8xx
RFC 1/1 handle access to non-present IO ports on 8xx
Hi,
Accessing non present "IO ports" on 8xx generates MCE's. The exception is easily triggered during insertion/removal/suspension of PCMCIA cards.
The following (against ancient v2.4) adds exception table entries for I/O instructions on 8xx (copied from the original Paul's PowerMac code), and changes MachineCheckException() slightly to cover 8xx specific's (on 8xx the MCE can be generated while executing the IO access instruction itself, which is not the case on PowerMac's, as the comment on traps.c details).
A few things I'm wondering:
1) why does the current PowerMac version covers only inb() and not outb() ? I had to add outb() exception table entries for 8xx.
2) Is the same wanted for other embedded PPC's?
3) How to make the misc.S exception entries and additional instructions selectable only on the platform who need it? #ifdef does not sound a good idea.
Nevermind the "#ifdef CONFIG_ALL_PPC" crap - that needs to be done properly.
Index: arch/ppc/kernel/misc.S
===================================================================
RCS file: /mnt/test1/tslinux_mv21/linux-216/arch/ppc/kernel/misc.S,v retrieving revision 1.2
ppc/kernel/misc.S +152 -29
@@ -736,8 +736,23 @@ subi r4,r4,1 blelr- 00: lbz r5,0(r3) - eieio - stbu r5,1(r4) +01: eieio +02: stbu r5,1(r4) +03: twi 0, r5, 0 +04: isync +05: nop +06: .section .fixup,"ax" +07: b 9f + .text +08: .section __ex_table, "a" + .align 2 + .long 00b, 07b + .long 01b, 07b + .long 02b, 07b + .long 03b, 07b + .long 04b, 07b + .long 05b, 07b + .text bdnz 00b blr
@@ -747,10 +762,27 @@ subi r4,r4,1 blelr- 00: lbzu r5,1(r4) - stb r5,0(r3) - eieio +01: stb r5,0(r3) +02: eieio +03: twi 0, r5, 0 +04: isync +05: nop +06: .section .fixup,"ax" +07: b 9f + .text +08: .section __ex_table, "a" + .align 2 + .long 00b, 07b + .long 01b, 07b + .long 02b, 07b + .long 03b, 07b + .long 04b, 07b + .long 05b, 07b + .text + bdnz 00b - blr +9: blr + _GLOBAL(_insw) cmpwi 0,r5,0
@@ -758,10 +790,25 @@ subi r4,r4,2 blelr- 00: lhbrx r5,0,r3 - eieio - sthu r5,2(r4) +01: eieio +02: sthu r5,2(r4) +03: twi 0, r5, 0 +04: isync +05: nop +06: .section .fixup,"ax" +07: b 9f + .text +08: .section __ex_table, "a" + .align 2 + .long 00b, 07b + .long 01b, 07b + .long 02b, 07b + .long 03b, 07b + .long 04b, 07b + .long 05b, 07b + .text bdnz 00b - blr +9: blr _GLOBAL(_outsw) cmpwi 0,r5,0
@@ -769,10 +816,25 @@ subi r4,r4,2 blelr- 00: lhzu r5,2(r4) - eieio - sthbrx r5,0,r3 +01: eieio +02: sthbrx r5,0,r3 +03: twi 0, r5, 0 +04: isync +05: nop +06: .section .fixup,"ax" +07: b 9f + .text +08: .section __ex_table, "a" + .align 2 + .long 00b, 07b + .long 01b, 07b + .long 02b, 07b + .long 03b, 07b + .long 04b, 07b + .long 05b, 07b + .text bdnz 00b - blr +9: blr _GLOBAL(_insl) cmpwi 0,r5,0
@@ -780,10 +842,25 @@ subi r4,r4,4 blelr- 00: lwbrx r5,0,r3 - eieio - stwu r5,4(r4) +01: eieio +02: stwu r5,4(r4) +03: twi 0, r5, 0 +04: isync +05: nop +06: .section .fixup,"ax" +07: b 9f + .text +08: .section __ex_table, "a" + .align 2 + .long 00b, 07b + .long 01b, 07b + .long 02b, 07b + .long 03b, 07b + .long 04b, 07b + .long 05b, 07b + .text bdnz 00b - blr +9: blr _GLOBAL(_outsl) cmpwi 0,r5,0
@@ -791,10 +868,26 @@ subi r4,r4,4 blelr- 00: lwzu r5,4(r4) - stwbrx r5,0,r3 - eieio +01: stwbrx r5,0,r3 +02: eieio +02: stwu r5,4(r4) +03: twi 0, r5, 0 +04: isync +05: nop +06: .section .fixup,"ax" +07: b 9f + .text +08: .section __ex_table, "a" + .align 2 + .long 00b, 07b + .long 01b, 07b + .long 02b, 07b + .long 03b, 07b + .long 04b, 07b + .long 05b, 07b + .text bdnz 00b - blr +9: blr _GLOBAL(ide_insw) _GLOBAL(_insw_ns)
@@ -803,10 +896,25 @@ subi r4,r4,2 blelr- 00: lhz r5,0(r3) - eieio - sthu r5,2(r4) +01: eieio +02: sthu r5,2(r4) +03: twi 0, r5, 0 +04: isync +05: nop +06: .section .fixup,"ax" +07: blr + .previous +08: .section __ex_table, "a" + .align 2 + .long 00b, 07b + .long 01b, 07b + .long 02b, 07b + .long 03b, 07b + .long 04b, 07b + .long 05b, 07b + .previous bdnz 00b - blr +9: blr _GLOBAL(ide_outsw) _GLOBAL(_outsw_ns)
@@ -815,10 +923,25 @@ subi r4,r4,2 blelr- 00: lhzu r5,2(r4) - sth r5,0(r3) - eieio +01: sth r5,0(r3) +02: eieio +03: twi 0, r5, 0 +04: isync +05: nop +06: .section .fixup,"ax" +07: b 9f + .text +08: .section __ex_table, "a" + .align 2 + .long 00b, 07b + .long 01b, 07b + .long 02b, 07b + .long 03b, 07b + .long 04b, 07b + .long 05b, 07b + .text bdnz 00b - blr +9: blr _GLOBAL(_insl_ns) cmpwi 0,r5,0
@@ -826,10 +949,10 @@ subi r4,r4,4 blelr- 00: lwz r5,0(r3) - eieio - stwu r5,4(r4) +01: eieio +02: stwu r5,4(r4) bdnz 00b - blr +9: blr _GLOBAL(_outsl_ns) cmpwi 0,r5,0
@@ -837,10 +960,10 @@ subi r4,r4,4 blelr- 00: lwzu r5,4(r4) - stw r5,0(r3) - eieio +01: stw r5,0(r3) +02: eieio bdnz 00b - blr +9: blr /* * Extended precision shifts.
Index: arch/ppc/kernel/traps.c
===================================================================
RCS file: /mnt/test1/tslinux_mv21/linux-216/arch/ppc/kernel/traps.c,v retrieving revision 1.1.1.1
ppc/kernel/traps.c +9 -6
@@ -130,9 +130,7 @@void MachineCheckException (struct pt_regs *regs) { -#ifdef CONFIG_ALL_PPC unsigned long fixup; -#endif /* CONFIG_ALL_PPC */ unsigned long msr = regs->msr;if (user_mode (regs)) {
@@ -150,7 +148,7 @@return ; } -#ifdef CONFIG_ALL_PPC +//#ifdef CONFIG_ALL_PPC /* * I/O accesses can cause machine checks on powermacs. * Check if the NIP corresponds to the address of a sync
@@ -176,20 +174,25 @@ nip -=2 ; else if (*nip == 0x4c00012c) /* isync */ --nip; -if (*nip ==0x7c0004ac || (*nip >>26 ) ==3 ) { +/* eieio from I/O string functions */ + else if ((*nip) == 0x7c0006ac || *(nip+1) == 0x7c0006ac) + nip +=2 ; + +if (*nip ==0x7c0004ac || (*(nip+1 ) >>26 ) ==3 || + (*nip >>26 ) ==3 ) {/* sync or twi */ unsigned int rb; --nip; rb = (*nip >>11 ) &0x1f ; -printk (KERN_DEBUG "%s bad port %lx at %p\n" , +printk (KERN_ERR "%s bad port %lx at %p\n" , (*nip &0x100 )?"OUT to" :"IN from" , regs->gpr[rb] - _IO_BASE, nip); regs->nip = fixup;return ; } } -#endif /* CONFIG_ALL_PPC */ +//#endif /* CONFIG_ALL_PPC */ printk ("Machine check in kernel mode.\n" );printk ("Caused by (from SRR1=%lx): " , msr);switch (msr &0x601F0000 ) {
Index: include/asm-ppc/io.h
===================================================================
RCS file: /mnt/test1/tslinux_mv21/linux-216/include/asm-ppc/io.h,v retrieving revision 1.1.1.1
asm-ppc/io.h +15 -6
@@ -82,7 +82,7 @@#define insl (port, buf, nl) _insl_ns((u32 *)((port)+_IO_BASE), (buf), (nl))#define outsl (port, buf, nl) _outsl_ns((u32 *)((port)+_IO_BASE), (buf), (nl)) -#ifdef CONFIG_ALL_PPC +#ifdef CONFIG_8xx/* * On powermacs, we will get a machine check exception if we * try to read data from a non-existent I/O port. Because the
@@ -105,7 +105,7 @@ { \unsigned int x; \ __asm__ __volatile__( \ - op" %0,0,%1\n" \ +"0:" op" %0,0,%1\n" \"1: twi 0,%0,0\n" \"2: isync\n" \ "3: nop\n" \
@@ -116,6 +116,7 @@".previous\n" \".section __ex_table,\"a\"\n" \" .align 2\n" \ +" .long 0b,5b\n" \" .long 1b,5b\n" \" .long 2b,5b\n" \" .long 3b,5b\n" \
@@ -129,12 +130,20 @@extern __inline__void name (unsigned int val,unsigned int port) \ { \ __asm__ __volatile__( \ - op" %0,0,%1\n" \ -"1: sync\n" \ -"2:\n" \ +"0:" op" %0,0,%1\n" \ +"1: twi 0, %0, 0\n" \ +"2: isync\n" \ +"3: nop\n" \ +"4:\n" \ +".section .fixup,\"ax\"\n" \ +"5: b 4b\n" \ +".previous\n" \".section __ex_table,\"a\"\n" \" .align 2\n" \ -" .long 1b,2b\n" \ +" .long 0b,5b\n" \ +" .long 1b,5b\n" \ +" .long 2b,5b\n" \ +" .long 3b,5b\n" \".previous" \ : :"r" (val),"r" (port + _IO_BASE)); \ }
On Wed, 2005-06-01 at 13:50 -0300, Marcelo Tosatti wrote:
Hrm... removing a PCMCIA card triggers mchecks ? that is bad... With "proper" PCMCIA controllers, those are swallowed properly when the card is removed. The eating of the machine check is a bit too hackish to my taste... Better is to "not do that" by making sure the legacy crap isn't trying to tap unexisting ports, but then, if PCMCIA is also a problem...
Quoted 2 lines
1) why does the current PowerMac version covers only
inb()and notoutb()? I had to addoutb()exception table entries for 8xx.
Not sure, maybe historical loss ? :) You should CC paulus
Quoted 1 line
2) Is the same wanted for other embedded PPC's?
It's up to you. It slows down those IOs, but on the other hand, inX/outX aren't supposed to be very common anymore, at least not with "fast" devices, and the IO itself is usually an order of magnitude slower than doing those syncs...
Quoted 16 lines
3) How to make the
misc.Sexception entries and additional instructions selectable only on the platform who need it? #ifdef does not sound a good idea.Nevermind the "#ifdef
CONFIG_ALL_PPC" crap - that needs to be done properly.Index:
arch/ppc/kernel/misc.S
===================================================================
RCS file:/mnt/test1/tslinux_mv21/linux-216/arch/ppc/kernel/misc.S,vretrieving revision 1.2
diff -u -r1.2misc.S
ppc/kernel/misc.S line 743
Quoted 10 lines
ppc/kernel/misc.S +5
-2@@ -736,8 +736,23 @@ subi r4,r4,1 blelr- 00: lbz r5,0(r3) - eieio - stbu r5,1(r4) +01: eieio +02: stbu r5,1(r4) +03: twi 0, r5, 0 +04: isync +05: nop
Hrm... the "twi/isync" stuff is not valid for a store. The reason for doing twi here is to trigger a data dependency on the result of the load (to make sure the load actually happens) and the isync to make sure that twi has actually completed. For stores, you are stuffed... either your HW don't do machine checks on stores, or you put a sync here and "hope" you don't have write posting along the IO chain that would cause your machine check to arrive too late...
Ben.
Quoted 9 lines · Benjamin Herrenschmidt
On Wed, 2005-06-01 at 13:50 -0300, Marcelo Tosatti wrote:
Hrm... removing a PCMCIA card triggers mchecks ? that is bad... With "proper" PCMCIA controllers, those are swallowed properly when the card is removed. The eating of the machine check is a bit too hackish to my taste... Better is to "not do that" by making sure the legacy crap isn't trying to tap unexisting ports, but then, if PCMCIA is also a problem...
8xx is not proper in any way whatsoever :)
There's no way to fix this thing with simple software hacks.
For example take a PCMCIA driver that's minding it's own businees, when someone yanks the card out.
cli()
...
inb(xxx)
...
<----- card is yanked here
...
inb(yyy)
...
<----- MCE heresti()
Quoted 7 lines
Quoted 2 lines
1) why does the current PowerMac version covers only
inb()and notoutb()? I had to addoutb()exception table entries for 8xx.Not sure, maybe historical loss ? :) You should CC paulus
Quoted 8 lines
Quoted 1 line
2) Is the same wanted for other embedded PPC's?
It's up to you. It slows down those IOs, but on the other hand, inX/outX aren't supposed to be very common anymore, at least not with "fast" devices, and the IO itself is usually an order of magnitude slower than doing those syncs...
Fast ISA I/O. :)
Quoted 2 lines
Quoted 2 lines
3) How to make the
misc.Sexception entries and additional instructions selectable only on the platform who need it? #ifdef does not sound a good idea.
I wouldn't mind...
Quoted 5 lines
Quoted 5 lines
Nevermind the "#ifdef
CONFIG_ALL_PPC" crap - that needs to be done properly.
Regards
Pantelis
P.S. Good job marcello :)
Hi Ben,
Quoted 8 lines · Benjamin Herrenschmidt
On Wed, 2005-06-01 at 13:50 -0300, Marcelo Tosatti wrote:
Hrm... removing a PCMCIA card triggers mchecks ? that is bad... With "proper" PCMCIA controllers, those are swallowed properly when the card is removed. The eating of the machine check is a bit too hackish to my taste... Better is to "not do that" by making sure the legacy crap isn't trying to tap unexisting ports, but then, if PCMCIA is also a problem...
Well, cardmgr calls the driver's shutdown/close routine as soon as the card is removed. Some of those methods write to IO registers in the process (eg net/pcmcia/pcnet_cs.c).
I dont see any elegant change that could be done in PCMCIA.
One possibility that comes to mind would be to enforce the rule that drivers should not writeout in case of a removal event, but that does not sound feasible to me: It just looks too messy, however I have no good argument against it.
Well, its restricted to 8xx (maybe 32-bit PPC) with PCMCIA, so its just not worth the trouble?
Quoted 36 lines
Quoted 2 lines
1) why does the current PowerMac version covers only
inb()and notoutb()? I had to addoutb()exception table entries for 8xx.Not sure, maybe historical loss ? :) You should CC paulus
Quoted 1 line
2) Is the same wanted for other embedded PPC's?
It's up to you. It slows down those IOs, but on the other hand, inX/outX aren't supposed to be very common anymore, at least not with "fast" devices, and the IO itself is usually an order of magnitude slower than doing those syncs...
Quoted 24 lines
3) How to make the
misc.Sexception entries and additional instructions selectable only on the platform who need it? #ifdef does not sound a good idea.Nevermind the "#ifdef
CONFIG_ALL_PPC" crap - that needs to be done properly.Index:
arch/ppc/kernel/misc.S
===================================================================
RCS file:/mnt/test1/tslinux_mv21/linux-216/arch/ppc/kernel/misc.S,vretrieving revision 1.2
diff -u -r1.2misc.Sppc/kernel/misc.S +3
-2@@ -736,8 +736,23 @@ subi r4,r4,1 blelr- 00: lbz r5,0(r3) - eieio - stbu r5,1(r4) +01: eieio +02: stbu r5,1(r4) +03: twi 0, r5, 0
Quoted 8 lines
ppc/kernel/misc.S line 743
Quoted 2 lines
ppc/kernel/misc.S +2
-0+04: isync +05: nopHrm... the "twi/isync" stuff is not valid for a store. The reason for doing twi here is to trigger a data dependency on the result of the load (to make sure the load actually happens) and the isync to make sure that twi has actually completed. For stores, you are stuffed... either your HW don't do machine checks on stores,
Does not sound a like a good idea :)
Quoted 3 lines
or you put a sync here and "hope"
you don't have write posting along the IO chain that would cause your machine check to arrive too late...
Hmmm... I'll look that up. My limited testing on HW in question does not exhibit longer MCE delays than "twi/isync/nop" sequence - actually, it is always triggered during the IO access instruction itself.
Where can I find information about write posting on 8xx? Is there a term for it used in PPC documentation? I can't find anything...
Thanks!
Quoted 12 lines · Marcelo Tosatti
Hi Ben,
Quoted 8 lines · Benjamin Herrenschmidt
On Wed, 2005-06-01 at 13:50 -0300, Marcelo Tosatti wrote:
Hrm... removing a PCMCIA card triggers mchecks ? that is bad... With "proper" PCMCIA controllers, those are swallowed properly when the card is removed. The eating of the machine check is a bit too hackish to my taste... Better is to "not do that" by making sure the legacy crap isn't trying to tap unexisting ports, but then, if PCMCIA is also a problem...
Quoted 5 lines
Well, cardmgr calls the driver's shutdown/close routine as soon as the card is removed. Some of those methods write to IO registers in the process (eg
net/pcmcia/pcnet_cs.c).I dont see any elegant change that could be done in PCMCIA.
I know, the thing is, on platforms with a "classical" PCI<->PCMCIA bridge, the bridge will not issue machine checks when the card is removed. I don't know if that is possible with your HW setup, I suppose you are hooking PCMCIA directly to the CPU IO bus ...
Quoted 2 lines
One possibility that comes to mind would be to enforce the rule that drivers should not writeout in case of a removal event,
They can't know, they get the removal even too late. The driver may be in the middle of writing when you remove the card. If there is no HW way to prevent the machine check, then you'll need software recovery.
Quoted 5 lines
but that does not sound feasible to me: It just looks too messy, however I have no good argument against it.
Well, its restricted to 8xx (maybe 32-bit PPC) with PCMCIA, so its just not worth the trouble?
Hehe... dunno.
Quoted 3 lines
Hmmm... I'll look that up. My limited testing on HW in question does not exhibit longer MCE delays than "twi/isync/nop" sequence - actually, it is always triggered during the IO access instruction itself.
I suppose the 8xx may actually be quite synchronous on IOs then...
Quoted 2 lines
Where can I find information about write posting on 8xx? Is there a term for it used in PPC documentation? I can't find anything...
Well, do you have a store queue ? What does the 8xx manual says about ordering & queueing of non cacheable loads & stores ? Maybe the chip is fully synchronous on IOs ?
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Hrm... removing a PCMCIA card triggers mchecks ? that is bad... With "proper" PCMCIA controllers, those are swallowed properly when the
card
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
is removed. The eating of the machine check is a bit too hackish to my taste... Better is to "not do that" by making sure the legacy crap
isn't
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
trying to tap unexisting ports, but then, if PCMCIA is also a problem...
Well, cardmgr calls the driver's shutdown/close routine as soon as the card is removed. Some of those methods write to IO registers in the process (eg
net/pcmcia/pcnet_cs.c).
Quoted 7 lines
Quoted 1 line
I dont see any elegant change that could be done in PCMCIA.
I know, the thing is, on platforms with a "classical" PCI<->PCMCIA bridge, the bridge will not issue machine checks when the card is removed. I don't know if that is possible with your HW setup, I suppose you are hooking PCMCIA directly to the CPU IO bus ...
I forget if I pointed this out already or not, but there's a good chance the mchecks you are seeing are from a bad PCMCIA implementation, not the 8xx itself. PCMCIA uses WAIT, which is a negative ACK really, and that must be pulled up externally, so if you remove the card any active WAIT goes away. I'd be glad to take a look at the schematics if available and tell you if I think there might be a problem there.
Then again, maybe it doesn't matter where the mchecks are coming from, you just need to handle them...
Mark Chambers
Quoted 17 lines · Benjamin Herrenschmidt
Quoted 16 lines · Marcelo Tosatti
Hi Ben,
Quoted 8 lines · Benjamin Herrenschmidt
On Wed, 2005-06-01 at 13:50 -0300, Marcelo Tosatti wrote:
Hrm... removing a PCMCIA card triggers mchecks ? that is bad... With "proper" PCMCIA controllers, those are swallowed properly when the card is removed. The eating of the machine check is a bit too hackish to my taste... Better is to "not do that" by making sure the legacy crap isn't trying to tap unexisting ports, but then, if PCMCIA is also a problem...
Well, cardmgr calls the driver's shutdown/close routine as soon as the card is removed. Some of those methods write to IO registers in the process (eg
net/pcmcia/pcnet_cs.c).
Quoted 6 lines
Quoted 1 line
I dont see any elegant change that could be done in PCMCIA.
I know, the thing is, on platforms with a "classical" PCI<->PCMCIA bridge, the bridge will not issue machine checks when the card is removed. I don't know if that is possible with your HW setup, I suppose you are hooking PCMCIA directly to the CPU IO bus ...
I think the PCMCIA bus is hooked directly to the CPU, yes.
Quoted 19 lines
Quoted 2 lines
One possibility that comes to mind would be to enforce the rule that drivers should not writeout in case of a removal event,
They can't know, they get the removal even too late. The driver may be in the middle of writing when you remove the card. If there is no HW way to prevent the machine check, then you'll need software recovery.
Quoted 5 lines
but that does not sound feasible to me: It just looks too messy, however I have no good argument against it.
Well, its restricted to 8xx (maybe 32-bit PPC) with PCMCIA, so its just not worth the trouble?
Hehe... dunno.
Quoted 3 lines
Hmmm... I'll look that up. My limited testing on HW in question does not exhibit longer MCE delays than "twi/isync/nop" sequence - actually, it is always triggered during the IO access instruction itself.
Quoted 8 lines
I suppose the 8xx may actually be quite synchronous on IOs then...
Quoted 2 lines
Where can I find information about write posting on 8xx? Is there a term for it used in PPC documentation? I can't find anything...
Well, do you have a store queue ? What does the 8xx manual says about ordering & queueing of non cacheable loads & stores ? Maybe the chip is fully synchronous on IOs ?
It seems so, yes.
Section 6.1 of MPC860UM.pdf:
"All exceptions associated with memory are implemented as precise, which means that a load/store instruction is not complete until all possible error indications are sampled from the load/store bus.
This also imples that a store or nonspeculative load instruction is not issued to the load/store bus until all previous instructions have completed."
I'll prepare a new patch reflecting this information, thanks again Ben.