Skip to contentSkip to conversation

MCA/INIT: need to wire CURRENT_STACK?

DISCUSSION Keith Owens 3 participants 6 messages Sep 21, 2005

lore
Share & tools
lore
#0 lore raw

I put this comment in arch/ia64/kernel/mca_asm.S, where we switch to the MCA/INIT stack. The code runs fine without wiring IA64_KR_CURRENT_STACK and IA64_TR_CURRENT_STACK. Is that coincidence or do the extra registers really need to be updated? The MCA/INIT stacks are initialized early in boot, they are in region 7.

Remember that the MCA/INIT handlers run with interrupts disabled.

ia64_set_kernel_registers:

        add temp3=MCA_SP_OFFSET, r3
        add temp4=MCA_SOS_OFFSET+IA64_SAL_OS_STATE_OS_GP_OFFSET, r3
        mov b0=r2               // save return address
        GET_IA64_MCA_DATA(temp1)
        ;;
        add temp4=temp4, temp1  // &struct ia64_sal_os_state.os_gp
        add r12=temp1, temp3    // kernel stack pointer on MCA/INIT stack
        add r13=temp1, r3       // set current to start of MCA/INIT stack
        ;;
        ld8 r1=[temp4]          // OS GP from SAL OS state
        ;;
        DATA_PA_TO_VA(r1,temp1)
        DATA_PA_TO_VA(r12,temp2)
        DATA_PA_TO_VA(r13,temp3)
        ;;
        mov IA64_KR(CURRENT)=r13

        // FIXME: do I need to wire IA64_KR_CURRENT_STACK and IA64_TR_CURRENT_STACK?

        br.sptk b0
#1 lore raw
Quoted 5 lines

I put this comment in arch/ia64/kernel/mca_asm.S, where we switch to the MCA/INIT stack. The code runs fine without wiring IA64_KR_CURRENT_STACK and IA64_TR_CURRENT_STACK. Is that coincidence or do the extra registers really need to be updated? The MCA/INIT stacks are initialized early in boot, they are in region 7.

Those registers keep track of which address we have pinned with DTR[2] so that context switch code (ia64_switch_to) can figure out whether it needs to reload DTR[2] to map the stack of the new process. Since you aren't going to context switch from your MCA code (I hope!) this should't be an issue.

BUT ... it raises the question of whether you will do anything in your MCA handler that will run into problems because the stack is perhaps[1] not pinned ... just mapped by a DTC entry. I forget what the exact issue is that led to pinning the stack with DTR[2], but there is a correctness issue involved. David: memory jog time??

-Tony

[1] It might be pinned if it is allocated from the same granule as the stack of the most recently running process on this cpu, since DTR[2] just maps a whole granule at a time. You might also have a DTC mapping active, so skip lightly past the problem in simple tests.

#2 lore raw
In reply to Keith Owens
Quoted 5 lines

BUT ... it raises the question of whether you will do anything in your MCA handler that will run into problems because the stack is perhaps[1] not pinned ... just mapped by a DTC entry. I forget what the exact issue is that led to pinning the stack with DTR[2], but there is a correctness issue involved. David: memory jog time??

Any virtually mapped memory that needs to be accessed with PSR.IC=0 needs to be pinned because otherwise you'd get a nested TLB fault and things would go south from there real fast.

--david
--
Mosberger Consulting LLC, voice/fax: 510-744-9372,
http://www.mosberger-consulting.com/
35706 Runckel Lane, Fremont, CA 94536

#3 lore raw
In reply to Keith Owens
Quoted 9 lines · David Mosberger-Tang
Quoted 5 lines

BUT ... it raises the question of whether you will do anything in your MCA handler that will run into problems because the stack is perhaps[1] not pinned ... just mapped by a DTC entry. I forget what the exact issue is that led to pinning the stack with DTR[2], but there is a correctness issue involved. David: memory jog time??

Any virtually mapped memory that needs to be accessed with PSR.IC=0 needs to be pinned because otherwise you'd get a nested TLB fault and things would go south from there real fast.

The virtual mode handlers are entered with psr.ic=1 and psr.i=0. External interrupts are blocked until the handlers resume to the original context. Internal interrupts like KDB_ENTER() use the handler's stack in virtual mode with psr.ic=0, but by the time it gets to ivt.S::non_syscall, the C code has already accessed the handler stack with psr.ic=1. Worst case, the first access to the handler stack in mca.c (which is with psr.ic) will take an alt DTLB fault and drop in a DTC for the handler stack.

There is a theoretical problem case. After the DTC for the handler stack has been loaded, the handler does enough data accesses that its own DTC is flushed. Then the handler, without accessing its own stack, causes an internal interrupt, e.g. KDB_ENTER(). That could cause a nested DTLB fault.

I can change the mca_asm code to save, modify and restore IA64_KR_CURRENT_STACK and IA64_TR_CURRENT_STACK around the C handlers. But that gets awkward if the C code wants to resume to a new context, the C code would have to set new values for IA64_KR_CURRENT_STACK and IA64_TR_CURRENT_STACK while it was still using its own stack, which might open the same problem.

#4 lore raw
In reply to Keith Owens
Quoted 3 lines · Keith Owens

On Wed, 21 Sep 2005 11:12:58 -0700,

The virtual mode handlers are entered with psr.ic=1 and psr.i=0.

I'm not sure what you mean by virtual mode handlers, but execution in the IVT starts with PSR.IC=0.

--david

#5 lore raw
In reply to Keith Owens
Quoted 6 lines · David Mosberger-Tang
Quoted 3 lines · Keith Owens

On Wed, 21 Sep 2005 11:12:58 -0700,

The virtual mode handlers are entered with psr.ic=1 and psr.i=0.

I'm not sure what you mean by virtual mode handlers,

The C code that starts with ia64_mca_handler and ia64_init_handler. mca_asm.S is entered in physical mode with psr.ic=0, it switches to virtual with psr.ic=1 and psr.i=0 before calling the handlers.

Quoted 2 lines

but execution in
the IVT starts with PSR.IC=0.

Right, so the real question is - is there any path that causes an interrupt before the virtual mode handlers have accessed their stacks and the required DTC has been created? Alternatively can the DTC for the MCA/INIT stacks be lost once the handlers have started running?

Given the uncertainty and the fact the the TLB error recovery in mca_asm.S assumes that CURRENT_STACK is defined, I need to wire CURRENT_STACK around the C code handlers. Patch coming soon.

BTW David, are you aware that your mailer is using quoted printable, even for plain text messages? All the '=' signs come out as '='.