ia64: reset console_loglevel so INIT output always goes to

Reset console_loglevel early in INIT handler. Otherwise, if it has been turned down (i.e., with "dmesg -n1"), the user may see no effect at all from issuing an INIT. We're never going to run any more user code, so there won't be any opportunity for anything to collect the output from the dmesg buffer.

=== arch/ia64/kernel/mca.c 1.71 vs edited ==--- 1.71/arch/ia64/kernel/mca.c 2004-11-11 11:04:30 -07:00
+++ edited/arch/ia64/kernel/mca.c 2005-01-07 16:22:54 -07:00
@@ -1133,6 +1133,7 @@
pal_min_state_area_t *ms;

oops_in_progress = 1; /* avoid deadlock in printk, but it makes recovery dodgy */
+ console_loglevel = 15; /* make sure printks make it to console */

printk(KERN_INFO "Entered OS INIT handler. PSP=%lx\n",
ia64_sal_to_os_handoff_state.proc_state_param);

That is fine for the current code base. It will have to be revisited when we get per cpu INIT stacks and the slave INIT handler actually does something.

We are slowly but steadily moving to recovery from some MCA events. If one of the cpus is spinning disabled when an MCA occurs then the disabled cpu will get a slave INIT event as part of the MCA rendezvous. If the MCA is recoverable then the slave INIT event will also be recoverable and will eventually return to user space.

That change is still some way off, but bear it in mind when changing the existing code.

As Keith pointed out, we are working on returning from an INIT.

Would it be reasonable to save the current console_loglevel, set console_loglevel up to 15, call init_handler_platform(), and then set console_loglevel back to the original level?

Right now init_handler_platform() never returns. When the code is improved to return, the console_loglevel will get restored to the correct level.

Russ Anderson, OS RAS/Partitioning Project Lead  
SGI - Silicon Graphics Inc          [email protected]

Good points, Keith.

There are a number of changes that will be needed now that MCAs and INITs are becoming recoverable. A disabled CPU should not receive an INIT as part of MCA rendezvous. Some of the changes will require changes in the MCA and SAL specs.

For example, having SAL rendezvous all the CPUs before calling OS_MCA may have been reasonable when linux lacked the ability to recover from an MCA. But now that is changing, the descision to rendezvous CPUs should get made later, in linux, if it cannot recover from the MCA. Does it really make sense to rendezvous 512 CPUs just because one CPU happened to hit a memory uncorrectable in a user application (and recovers by killing the appication and discarding the page)?

Does it still make sense to have only one call into OS_MCA at a time? Or is it more reasonable to support multiple OS_MCAs and let the linux MCA code coordinate processing of the OS_MCA, when needed? As the code progresses, it should be reasonable move more of the decision & coordination code further into the recovery code (or at least not prevent that from happening) so that, for example, multiple independent MCAs can be recovered in parallel.

As I said, this will require changes in the MCA & SAL specs. Some are simply clearing up ambiguities in the specs, as Keith found in MCA logging of recovered errors. Some will be more fundamental changes to support better recovery. The code has reached the point where we need to start making enhancements to those specs.

Thanks,

Russ Anderson, OS RAS/Partitioning Project Lead  
SGI - Silicon Graphics Inc          [email protected]

I do not see any alternative. SAL has no idea if the OS can recover from a memory MCA or not, that decision has to be made by the OS. Leaving the rendezvous decision to the OS would significantly complicate the OS/SAL interface, it requires another SAL call by the OS, changes to every SAL version and code in the OS to work out if the current prom supports the SAL change or not. If memory is failing, we want the other cpus to keep off that physical memory while we work around the problem and decide if we can recover, so we need to stop the other cpus anyway.

MCA/INIT events are very rare and, in most cases, the rendezvous is a standard interrupt which is reasonably fast. I will take simplicity of OS coding over speed every time for this case. The simplest method is for SAL to rendezvous all the cpus unless SAL knows unequivocally that the error will be recovered. If there is any uncertainty about whether the OS will recover or not, then do the full rendezvous.

I believe that option is already allowed by the SAL specification. Linux has never supported it because it never had the per cpu infrastructure.

Sure. I expect that we will one day be able to continue after an INIT, and I considered doing the save/restore in the current patch. But I preferred the simplest possible patch, partly because this is an issue for distributions, and I'd like to see them pick this up quickly

I agree with Keith. Although it might seem a bit of an overkill to rendezvous all CPUs on large systems, I think it greatly enhances the chances of a clean recovery. Based on my error handing experience on MIPS based systems with similar CPU counts, you still might have external interventions etc.

Getting the system into a known state for the rare recovery case is certainly a big advantage and avoids a lot of corner cases which will be extremly hard to test.

Thanks

Matthias Fouquet-Lapar  Core Platform Software    [email protected]  VNET 521-8213
Principal Engineer      Silicon Graphics          Home Office (+33) 1 3047 4127