Skip to contentSkip to conversation

2.6.0-test9 pal/sal/salinfo/mca

PATCH David Mosberger 6 participants 8 messages Nov 25, 2003 – Dec 4, 2003

lore
Share & tools
lore
#0 lore raw

Keith> This converts 2.6 to use salinfo instead of printing Keith> CMC/CPE/MCA/INIT records in the kernel. It makes the two Keith> kernel versions as close together as possible.

The patch is fine with me, but given it's size, it's 2.6.1 material.

	--david
#1 lore raw
Quoted 6 lines · David Mosberger

Keith> This converts 2.6 to use salinfo instead of printing Keith> CMC/CPE/MCA/INIT records in the kernel. It makes the two Keith> kernel versions as close together as possible.

The patch is fine with me, but given it's size, it's 2.6.1 material.

Agreed, I just wanted it up for review.

#2 lore raw
In reply to David Mosberger
Quoted 6 lines · Keith Owens

Forward port the recent changes to pal.h, sal.h, mca.h, salinfo.c and mca.c from 2.4.23-rc2 to 2.6.0-test9.

This converts 2.6 to use salinfo instead of printing CMC/CPE/MCA/INIT records in the kernel. It makes the two kernel versions as close together as possible.

I'd like to inquire a bit more into the state of MCA in 2.4 and 2.6. We are assembling a 1000 node ia64 cluster out of intel Tiger 4 servers and we want to make sure that MCA works well enough that we can at least get a good count of the ECC SBE's and panic if we get a MBE.

We are currently basing our kernel off of the Red Hat Enterprise Linux 3 kernel and we discovered that the implementation of MCA included with it does not work for us. The most obvious problem is that it never calls ia64_sal_clear_state_info after fetching a SAL record. Thus the CPE reasserts itself and the machine effectively locks up infinitely printing out the same CPE to the console.

So what we are trying to do is improve the state of the MCA handling in our kernel. I managed a backport of the MCA code from 2.6.0-test9 to 2.4 and it works much better. However, there are a couple of problems with it that could probably be sorted out by someone who understands the code better. Keith your message sort of hints that the possibility that the 2.4 kernel's MCA code is further advanced than the 2.6 code. This led us initially to believe that we could backport the 2.4.23 kernel's MCA code and have it work. However, taking a look at the 2.4.23 kernel from kernel.org, it is quickly evident that it doesn't make the needed call to sal_clear_state_info.

So my question is: should we continue forward with our backport of the 2.6 MCA code or is the 2.4 code actually functional enough to support our needs and we are missing something in our quick inspection of the code.

Also if the 2.6 backport is the way to go, would other people be interested in having the 2.6 MCA code backport made available? Once we get it working satisfactorily here, I'm going to push for it to be integrated into the Red Hat kernel. Is this something that, would be worthwhile to push upstream?

-ben

#3 lore raw
In reply to David Mosberger
Quoted 13 lines · Ben Woodard
Quoted 6 lines · Keith Owens

Forward port the recent changes to pal.h, sal.h, mca.h, salinfo.c and mca.c from 2.4.23-rc2 to 2.6.0-test9.

This converts 2.6 to use salinfo instead of printing CMC/CPE/MCA/INIT records in the kernel. It makes the two kernel versions as close together as possible.

I'd like to inquire a bit more into the state of MCA in 2.4 and 2.6. We are assembling a 1000 node ia64 cluster out of intel Tiger 4 servers and we want to make sure that MCA works well enough that we can at least get a good count of the ECC SBE's and panic if we get a MBE.

Quoted 6 lines

We are currently basing our kernel off of the Red Hat Enterprise Linux 3 kernel and we discovered that the implementation of MCA included with it does not work for us. The most obvious problem is that it never calls ia64_sal_clear_state_info after fetching a SAL record. Thus the CPE reasserts itself and the machine effectively locks up infinitely printing out the same CPE to the console.

2.4.23 ia64 BK tree has these lines in ia64_mca_log_sal_error_record().

        salinfo_log_wakeup(sal_info_type, buffer, size);
        platform_err = ia64_log_print(sal_info_type, (prfunc_t)printk);
        /* Clear logs from corrected errors in case there's no user-level logger */
        if (sal_info_type = SAL_INFO_TYPE_CPE || sal_info_type = SAL_INFO_TYPE_CMC)
                ia64_sal_clear_state_info(sal_info_type);

so you should be clearing CPE records immediately. AS 3.0 is probably out of date in its MCA handling.

Quoted 6 lines

So what we are trying to do is improve the state of the MCA handling in our kernel. I managed a backport of the MCA code from 2.6.0-test9 to 2.4 and it works much better. However, there are a couple of problems with it that could probably be sorted out by someone who understands the code better. Keith your message sort of hints that the possibility that the 2.4 kernel's MCA code is further advanced than the 2.6 code.

With my 2.6 patch of 2003-11-25, 2.4 and 2.6 MCA handling is the same, and it works for CPE. Grab these files from ia64 2.4 BK and merge them with the AS 3.0 files, if there is ay doubt, use the ia64 2.4 BK version.

include/asm-ia64/sal.h
include/asm-ia64/pal.h
include/asm-ia64/mca.h
arch/ia64/Kconfig
arch/ia64/kernel/Makefile
arch/ia64/kernel/salinfo.c
arch/ia64/kernel/mca.c

#4 lore raw
In reply to David Mosberger

Ben,

Quoted 3 lines · Ben Woodard

Also if the 2.6 backport is the way to go, would other people be interested in having the 2.6 MCA code backport made available? Once we

Yes, it is important to have the full MCA support in RHEL kernel. For instance, the MCA INIT handler is a very useful piece of code to debug deadlocks when it is coupled to the kernel stack unwinder (which is th default on 2.6). I wish I had this feature today in RHEL-3.0.

Signature · 1 line
-Stephane
#5 lore raw
In reply to David Mosberger
Quoted 6 lines · Ben Woodard

We are currently basing our kernel off of the Red Hat Enterprise Linux 3 kernel and we discovered that the implementation of MCA included with it does not work for us. The most obvious problem is that it never calls ia64_sal_clear_state_info after fetching a SAL record. Thus the CPE reasserts itself and the machine effectively locks up infinitely printing out the same CPE to the console.

I believe this is bug #104667 in the Red Hat bugzilla. The patch should be attached to the bug report. I can't get bugzilla to show me the report, so I don't know the current status.

Quoted 4 lines

So my question is: should we continue forward with our backport of the 2.6 MCA code or is the 2.4 code actually functional enough to support our needs and we are missing something in our quick inspection of the code.

As Keith mentioned, the current 2.4 and 2.6 MCA code should be essentially identical.

Bjorn

#6 lore raw
In reply to David Mosberger
Quoted 3 lines

As Keith mentioned, the current 2.4 and 2.6 MCA code should be essentially identical.

David deferred Keith's last MCA patch for 2.6.1 ... so the current checked in 2.6 code doesn't have the latest goodies.

Keith's patch can be found here:
http://marc.theaimsgroup.com/?l=linux-ia64&m6974968032730&w=2

Don't forget these all require a user level daemon to pick up CMC and CPE reports and log them in all their glory.

Keith: did you post salinfo-0.4 yet? I haven't seen it.

-Tony

#7 lore raw
In reply to David Mosberger
Quoted 6 lines · Tony
Quoted 3 lines

As Keith mentioned, the current 2.4 and 2.6 MCA code should be essentially identical.

David deferred Keith's last MCA patch for 2.6.1 ... so the current checked in 2.6 code doesn't have the latest goodies.

Oops, you're right. I forgot about that.

Quoted 7 lines

Keith's patch can be found here:
http://marc.theaimsgroup.com/?l=linux-ia64&m6974968032730&w=2

Don't forget these all require a user level daemon to pick up CMC and CPE reports and log them in all their glory.

Keith: did you post salinfo-0.4 yet? I haven't seen it.

He sent it to me last week, and I just now dropped it here:

ftp://ftp.kernel.org/pub/linux/kernel/people/helgaas/salinfo-0.4.tar.gz

(Keith, this includes the patch you sent me on Tuesday). Keith's done some Red Hat packaging, too; anybody want to contribute Debian packaging?

Bjorn