Software IO-TLB Kernel panic
Hi,
I am reproduceably getting kernel panics when accessing discs on an Adaptec 39160 adapter (SCSI host 1 after the built-in QLA1280). I am using the "new" aic7xxx driver on a 2.4.4 IA64 kernel. I configured the kernel for DIG-compliant C0-stepping hardware.
The kernel panic occurs in map_single (arch/ia64/lib/swiotlb.c:171).
I have a 2-CPU Lion with C0-stepping CPUs. The requested IO TLB size is 8192 when the panic occurs.
After the crash, I always have severe corruption on the filesystem that was being accessed during the crash. e2fsck reports ~10 illegal blocks in one inode, and many follow-up errors.
The only TLB related boot message I see is
kernel: Placing software IO TLB between 0xe000000000100000 - 0xe000000000300000
I am looking into this myself right now, but I would be grateful for hints where to start. Any help appreciated,
Martin
Signature · 2 lines
Martin Wilck <[email protected]> FSC EP PS DS1, Paderborn Tel. +49 5251 8 15113
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Martin Wilck
Martin> I am reproduceably getting kernel panics when accessing Martin> discs on an Adaptec 39160 adapter (SCSI host 1 after the Martin> built-in QLA1280). I am using the "new" aic7xxx driver on a Martin> 2.4.4 IA64 kernel. I configured the kernel for Martin> DIG-compliant C0-stepping hardware.
Martin> The kernel panic occurs in map_single
Martin> (arch/ia64/lib/swiotlb.c:171).
Sounds like the driver is trying to map buffers that are bigger than (1 << IO_TLB_SHIFT) (currently 2KB).
--david
Quoted 6 lines
Martin> The kernel panic occurs in
map_single
Martin> (arch/ia64/lib/swiotlb.c:171).Sounds like the driver is trying to map buffers that are bigger than (1 <<
IO_TLB_SHIFT) (currently 2KB).
I looked at the code and thought that bigger buffers than 2kB can (in principle) be created by joining subsequent buffers.
Of course, it could happen that no such contiguous space is available. Am I wrong ?
Martin
Signature · 2 lines
Martin Wilck <[email protected]> FSC EP PS DS1, Paderborn Tel. +49 5251 8 15113
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line · Martin Wilck
Martin> The kernel panic occurs in map_single
Martin> (arch/ia64/lib/swiotlb.c:171).
Quoted 2 lines
Quoted 2 lines
Sounds like the driver is trying to map buffers that are bigger
than (1 <<IO_TLB_SHIFT) (currently 2KB).
Martin> I looked at the code and thought that bigger buffers than Martin> 2kB can (in principle) be created by joining subsequent Martin> buffers. Of course, it could happen that no such contiguous Martin> space is available. Am I wrong ?
Yes, you're right. I forgot about the joining.
--david
[Linux-ia64] Software IO-TLB Kernel panic - preliminary analysis
In reply to Martin WilckThis problem is really hard to hunt down, as even kdb will not respond anymore after the crash happens. Also, my system logs are truncated.
What I have seen, though, is that IO-TLBs are allocated very quickly immediately before the crash. By using some printk's, I saw 133 allocations of 8192-byte chunks in a row without a single deallocation immediately before the machine came down. This alone accounts for about half of the bounce buffer space, without any space that was allocated before and without any further allocations that I may have lost due to the lost lines in the log.
By inspecting elements of the pci_dev structure passed to the routine, I am now 99% convinced that the Adptect 7899a controller
is the "guilty" device. This fits well to the finding that the crashes always occur after (!) a file system on that card was activated a little more heavily.
It seems that the problem does not occur with the "old" aic7xxx driver. On the contrary, that driver seems to deallocate every buffer immediately after allocation.
Thus, for the time being I'd recommend to use the aic7xxx_old driver. But it looks like a problem that ought to be solved.
Should I perhaps approach the aic7xxx maintainers?
Regards,
Martin
Signature · 2 lines
Martin Wilck <[email protected]> FSC EP PS DS1, Paderborn Tel. +49 5251 8 15113