Skip to contentSkip to conversation

SAL_MC_RENDEZ logic

RFC Hidetoshi Seto 3 participants 4 messages Sep 12, 2005

lore
Share & tools
lore
#0 lore raw

Hi all,

I'm now testing the MCA codes on brand-new system, and bump into a problem that slave processors infinitely loop in ia64_mca_wakeup_ipi_wait().

The cause was that the SAL clears the IRR bit just after its spin in SAL_MC_RENDEZ procedure, and OS spins again until the IRR bit be set in ia64_mca_wakeup_ipi_wait().

According to the SAL spec, it says:
(SAL_MC_RENDEZ:)
When this procedure returns, it is the responsibility of the operating system to clear the IRR bits for the MC_rendezvous interrupt and the wake up interrupt, if any.

I'm not sure but it seems "if any" means that SAL can clear the IRR bits on behalf of OS. So OS shouldn't expect the IRR always be set on returning from SAL_MC_RENDEZ, is this right?

I found a archive 2 years ago, from Keith:
http://marc.theaimsgroup.com/?l=linux-ia64&m5590709805820
However there was no responce...

I don't know whether there is any old SAL never spins in SAL_MC_RENDEZ or not. Or is this the beginning of nightmare, having different MCA codes depend on the SAL version?

Thanks,
H.Seto

#1 lore raw
Quoted 8 lines · Hidetoshi Seto

I'm now testing the MCA codes on brand-new system, and bump into a problem that slave processors infinitely loop in ia64_mca_wakeup_ipi_wait().

The cause was that the SAL clears the IRR bit just after its spin in SAL_MC_RENDEZ procedure, and OS spins again until the IRR bit be set in ia64_mca_wakeup_ipi_wait().

Quoted 5 lines

According to the SAL spec, it says:
(SAL_MC_RENDEZ:)
When this procedure returns, it is the responsibility of the operating system to clear the IRR bits for the MC_rendezvous interrupt and the wake up interrupt, if any.

The IRR bits are read only. The OS clears them by reading cr.ivr, in the external interrupt vector. The only reason that mca.c tests IRR directly is because at that point interrupts are disabled.

Quoted 3 lines

I'm not sure but it seems "if any" means that SAL can clear the IRR bits on behalf of OS. So OS shouldn't expect the IRR always be set on returning from SAL_MC_RENDEZ, is this right?

The phrase "if any" is quite ambiguous, it is not clear what it means here.

Quoted 3 lines

I don't know whether there is any old SAL never spins in SAL_MC_RENDEZ or not. Or is this the beginning of nightmare, having different MCA codes depend on the SAL version?

I hope not. In any case my MCA/INIT rewrite removes the spin in mca.c waiting for IRR to be set. Instead the slave comes out of SAL due to a wake up call, waits for the monarch to exit then the slaves all exit.

Once a slave resumes to its normal context and interrupts are enabled again, then the external interrupt vector clears the wake up bit and calls ia64_mca_wakeup_int_handler() which is a no-op. The rendezvous IRR bit is cleared when we read cr.ivr prior to calling ia64_mca_rendez_int_handler(), i.e. this bit is already clear when we rendezvous.

In your case I would say that SAL is wrong. I would argue that SAL should not be reading cr.ivr at all, it should leave that to the OS. The existing (2.6.13) code will not work with that SAL. My rewrite (hopefully in 2.6.14-rc1) will work with that SAL.

#2 lore raw
In reply to Hidetoshi Seto

Thank you for your reply, Keith.

Quoted 3 lines · Keith Owens

The IRR bits are read only. The OS clears them by reading cr.ivr, in the external interrupt vector. The only reason that mca.c tests IRR directly is because at that point interrupts are disabled.

I forgot to mention, the SAL actually reads cr.ivr and writes cr.eoi.

Quoted 6 lines
Quoted 3 lines

I'm not sure but it seems "if any" means that SAL can clear the IRR bits on behalf of OS. So OS shouldn't expect the IRR always be set on returning from SAL_MC_RENDEZ, is this right?

The phrase "if any" is quite ambiguous, it is not clear what it means here.

I agree. It should be written in full sentence.

Quoted 15 lines
Quoted 3 lines

I don't know whether there is any old SAL never spins in SAL_MC_RENDEZ or not. Or is this the beginning of nightmare, having different MCA codes depend on the SAL version?

I hope not. In any case my MCA/INIT rewrite removes the spin in mca.c waiting for IRR to be set. Instead the slave comes out of SAL due to a wake up call, waits for the monarch to exit then the slaves all exit.

Once a slave resumes to its normal context and interrupts are enabled again, then the external interrupt vector clears the wake up bit and calls ia64_mca_wakeup_int_handler() which is a no-op. The rendezvous IRR bit is cleared when we read cr.ivr prior to calling ia64_mca_rendez_int_handler(), i.e. this bit is already clear when we rendezvous.

Quoted 4 lines

In your case I would say that SAL is wrong. I would argue that SAL should not be reading cr.ivr at all, it should leave that to the OS. The existing (2.6.13) code will not work with that SAL. My rewrite (hopefully in 2.6.14-rc1) will work with that SAL.

I appreciate your work very well.
I'll argue off this problem with developers of the SAL instead of you.

Thanks,
H.Seto

#3 lore raw
In reply to Hidetoshi Seto

MCA experts,

Itanium 2 processor datasheet mentions about ETM generated CMCI but none of the PAL/SAL spec/Itanium error handling guide has that info. PAL_MC_ERROR_INFO/SAL_GET_STATE_INFO for processor errors do not have any entries related to thermal event.

When CMCI is sent, I presume there's an error record of it. Where can I find the info of ETM-CMC related error record format/doc?

Itanium 2 processor datasheet reads:
#5.1.2 Enhanced Thermal Management
...Once the thermal sensing device observes the temperature rise above the thermal entry point, the processor will enter a low power mode of execution and notify the system by sending a Correctable Machine Check Interrupt (CMCI). ...

Thanks,
John Ik Lee (J.I.)
Sr. Staff Engineer
Platform Solutions, Inc

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Hidetoshi Seto
Sent: Monday, September 12, 2005 1:28 AM
To: Keith Owens
Cc: [email protected]
Subject: Re: [RFC] SAL_MC_RENDEZ logic

Thank you for your reply, Keith.

Quoted 3 lines · Keith Owens

The IRR bits are read only. The OS clears them by reading cr.ivr, in the external interrupt vector. The only reason that mca.c tests IRR directly is because at that point interrupts are disabled.

I forgot to mention, the SAL actually reads cr.ivr and writes cr.eoi.

Quoted 6 lines
Quoted 3 lines

I'm not sure but it seems "if any" means that SAL can clear the IRR bits on behalf of OS. So OS shouldn't expect the IRR always be set on returning from SAL_MC_RENDEZ, is this right?

The phrase "if any" is quite ambiguous, it is not clear what it means here.

I agree. It should be written in full sentence.

Quoted 6 lines
Quoted 3 lines

I don't know whether there is any old SAL never spins in SAL_MC_RENDEZ or not. Or is this the beginning of nightmare, having different MCA codes depend on the SAL version?

I hope not. In any case my MCA/INIT rewrite removes the spin in mca.c waiting for IRR to be set. Instead the slave comes out of SAL due to

a

Quoted 9 lines

wake up call, waits for the monarch to exit then the slaves all exit.

Once a slave resumes to its normal context and interrupts are enabled again, then the external interrupt vector clears the wake up bit and calls ia64_mca_wakeup_int_handler() which is a no-op. The rendezvous IRR bit is cleared when we read cr.ivr prior to calling ia64_mca_rendez_int_handler(), i.e. this bit is already clear when we rendezvous.

Quoted 4 lines

In your case I would say that SAL is wrong. I would argue that SAL should not be reading cr.ivr at all, it should leave that to the OS. The existing (2.6.13) code will not work with that SAL. My rewrite (hopefully in 2.6.14-rc1) will work with that SAL.

I appreciate your work very well.
I'll argue off this problem with developers of the SAL instead of you.

Thanks,
H.Seto

-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html