qla1280: failing mbox check

Trying to get 2.6.17-rc3 up and running on my SGI Octane, I've hit an odd quirk, and I'm looking for some clarification on whether the bug may potentially lie in the SCSI layer, or possibly more in the mips core someplace.

Booting 2.6.17, the QLogic 1040 chip is detected, but upon sending the mbox command, it timeouts and fails, reporting this message:

scsi(0): mailbox timed out, mailbox0 4000, ictrl 0006, istatus 0004
qla1280: Failed mbox check
scsi(0): mailbox timed out, mailbox0 4000, ictrl 0006, istatus 0004

The last line will then repeat indefinitely.

Now I know between 2.6.16 and 2.6.17-rc3, a lot of changes occured across a number of archs to consolidate some of the bitops functions into the generic section of the tree, I'm not sure if this is one possible area of breakage or not. I tried turning all the debug options for qla1280, but didn't get any additional output that was meaningful. Are there other ways to debug this? I stuck a printk to see what the returned values of mb[] are, and got nothing but 0x0, so I can only assume something bad is happening to them someplace.

Any ideas where to start looking or things to try? 2.6.16.* works flawlessly, and I didn't see anything odd done to qla1280.c itself that could be the point of breakage. So I can only assume that the error is in a higher-level somewheres.

--Kumba

Gentoo/MIPS Team Lead
Gentoo Foundation Board of Trustees

"Such is oft the course of deeds that move the wheels of the world: small hands do them because they must, while the eyes of the great are elsewhere."  --Elrond

Kumba> Any ideas where to start looking or things to try? 2.6.16.* Kumba> works flawlessly, and I didn't see anything odd done to Kumba> qla1280.c itself that could be the point of breakage. So I can Kumba> only assume that the error is in a higher-level somewheres.

'git bisect' is your friend here.

Cheers,
Jes

That was the answer I was afraid of.... Already tried using git bisect to track down a bootmem allocation bug in SGI O2 systems (2.6.17 just broke everything it seems...), but the bisection point happened to be around the time when all the bitops changes were occuring (moving duplicate functions from asm-mips -> asm-generic), resulting in a tree that wouldn't even compile.

Having to patch up a broken tree so it even builds to determine if the bisection is good or bad might obscure the bug, but well, guess that's something to try on the weekend.

Offhand, though, is there anything I should keep an eye out for that can do things like setting mb[] to 0x0 (i.e., things you guys might've seen happen in the past?). And what do those numbers for 'mailbox0', 'ictrl', and 'istatus' mean (I'm guessing not much, because they're probably qla1040-specific error codes that require chip docs to understand).

--Kumba

Gentoo/MIPS Team Lead
Gentoo Foundation Board of Trustees

"Such is oft the course of deeds that move the wheels of the world: small hands do them because they must, while the eyes of the great are elsewhere."  --Elrond

Kumba> That was the answer I was afraid of.... Already tried using Kumba> git bisect to track down a bootmem allocation bug in SGI O2 Kumba> systems (2.6.17 just broke everything it seems...), but the Kumba> bisection point happened to be around the time when all the Kumba> bitops changes were occuring (moving duplicate functions from Kumba> asm-mips -> asm-generic), resulting in a tree that wouldn't Kumba> even compile.

Kumba> Having to patch up a broken tree so it even builds to determine Kumba> if the bisection is good or bad might obscure the bug, but Kumba> well, guess that's something to try on the weekend.

Uh oh!, bitops changing half way? Maybe the thing to try first is to revert all the bitops mangling.

Kumba> Offhand, though, is there anything I should keep an eye out for Kumba> that can do things like setting mb[] to 0x0 (i.e., things you Kumba> guys might've seen happen in the past?). And what do those Kumba> numbers for 'mailbox0', 'ictrl', and 'istatus' mean (I'm Kumba> guessing not much, because they're probably qla1040-specific Kumba> error codes that require chip docs to understand).

I believe ICTRL == interrupt control, ISTATUS == interrupt status. Don't have the docts in front of me though.

Cheers,
Jes

Yeah, thought about that, as some functions that were duplicated in the tree in many of the arch ports got moved from their respective asm-$arch to asm-generic, and appropriate tweaks added to compensate for these changes. I remember something from bitops stuff changed, but I'll be blasted if I can remember what.
Probably gonna have to get some help from the Octane port author and some of
the other mips hackers to try and trace down some of these changes (i.e., seems mips asm-offsets.c gets something weird done to it by gcc now when I run a git bisect and try to build it, which breaks a few of the asm files -- makes testing the bisect somewhat fun).

--Kumba

Gentoo/MIPS Team Lead
Gentoo Foundation Board of Trustees

"Such is oft the course of deeds that move the wheels of the world: small hands do them because they must, while the eyes of the great are elsewhere."  --Elrond