b44 driver causes panic when using swiotlb
The b44 driver is triggering this panic in swiotlb_map_page():
if (!dma_capable(dev, dev_addr, size))
panic("map_single: bounce buffer is not DMA'ble");
The kernel log says the bounce buffers are at 0xdb400000, but b44 can only do DMA to the first 1GB of memory:
PCI-DMA: Using software bounce buffering for IO (SWIOTLB) Placing 64MB software IO TLB between ffff8800db400000 - ffff8800df400000 software IO TLB at phys 0xdb400000 - 0xdf400000
Quoted 7 lines · Chuck Ebbert
The b44 driver is triggering this panic in
swiotlb_map_page():if (!dma_capable(dev,
dev_addr, size))
panic("map_single: bounce buffer is not DMA'ble");The kernel log says the bounce buffers are at
0xdb400000, but b44 can only do DMA to the first 1GB of memory:
b44 needs to use GFP_DMA then and do its own custom bouncing. The standard pci_map_* bounce buffering is only designed for at least 32bit capable devices.
BTW I'm pretty sure this worked at some point, must have regressed somehow.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only
Quoted 6 lines · Andi Kleen
Quoted 5 lines · Chuck Ebbert
The b44 driver is triggering this panic in
swiotlb_map_page():if (!dma_capable(dev,
dev_addr, size))
panic("map_single: bounce buffer is not DMA'ble");
Quoted 6 lines
Quoted 2 lines
The kernel log says the bounce buffers are at
0xdb400000, but b44 can only do DMA to the first 1GB of memory:b44 needs to use
GFP_DMAthen and do its own custom bouncing. The standard pci_map_* bounce buffering is only designed for at least 32bit capable devices.
That seems wrong - it's a documented API and that restriction isn't documented. Either it should comply with the request or return a failure if it can't accomodate it, not just blow up internally. There's no reason the driver should have to deal with this on its own.
In this case the DMA mapping code should really be falling back to GFP_DMA automatically if the IOMMU aperture is outside the DMA mask of the device.
Quoted 3 lines
BTW I'm pretty sure this worked at some point, must have regressed somehow.
Quoted 14 lines · On Mon, 31 Jan 2011 18:54:21 -0600 Robert Hancock <[email protected]> wrote:
Quoted 6 lines · Andi Kleen
Quoted 5 lines · Chuck Ebbert
The b44 driver is triggering this panic in
swiotlb_map_page():if (!dma_capable(dev,
dev_addr, size))
panic("map_single: bounce buffer is not DMA'ble");Quoted 6 lines
Quoted 2 lines
The kernel log says the bounce buffers are at
0xdb400000, but b44 can only do DMA to the first 1GB of memory:b44 needs to use
GFP_DMAthen and do its own custom bouncing. The standard pci_map_* bounce buffering is only designed for at least 32bit capable devices.
Quoted 8 lines
That seems wrong - it's a documented API and that restriction isn't documented. Either it should comply with the request or return a failure if it can't accomodate it, not just blow up internally. There's no reason the driver should have to deal with this on its own.
In this case the DMA mapping code should really be falling back to
GFP_DMAautomatically if the IOMMU aperture is outside the DMA mask of the device.
swiotlb allocates the bounce buffer when a system boots up. We can't allocate much in GFP_DMA. swiotlb uses somewhere under 4GB. So it can't help devices that have odd dma_mask (that is, except for 4GB).
Unfortunately, Such device needs to do own custom bouncing or needs their subsystem to does that.
Some ideas to implement something that works for such device were discussed. Seems that the conclusion is that it's doesn't worth making the common code complicated for such minor and insane devices.
Quoted 30 lines · FUJITA Tomonori
Quoted 14 lines · On Mon, 31 Jan 2011 18:54:21 -0600 Robert Hancock<[email protected]> wrote:
Quoted 6 lines · Andi Kleen
Quoted 5 lines · Chuck Ebbert
The b44 driver is triggering this panic in
swiotlb_map_page():if (!dma_capable(dev,
dev_addr, size))
panic("map_single: bounce buffer is not DMA'ble");Quoted 6 lines
Quoted 2 lines
The kernel log says the bounce buffers are at
0xdb400000, but b44 can only do DMA to the first 1GB of memory:b44 needs to use
GFP_DMAthen and do its own custom bouncing. The standard pci_map_* bounce buffering is only designed for at least 32bit capable devices.Quoted 8 lines
That seems wrong - it's a documented API and that restriction isn't documented. Either it should comply with the request or return a failure if it can't accomodate it, not just blow up internally. There's no reason the driver should have to deal with this on its own.
In this case the DMA mapping code should really be falling back to
GFP_DMAautomatically if the IOMMU aperture is outside the DMA mask of the device.swiotlb allocates the bounce buffer when a system boots up. We can't allocate much in
GFP_DMA. swiotlb uses somewhere under 4GB. So it can't help devices that have odddma_mask(that is, except for 4GB).
Quoted 6 lines
Unfortunately, Such device needs to do own custom bouncing or needs their subsystem to does that.
Some ideas to implement something that works for such device were discussed. Seems that the conclusion is that it's doesn't worth making the common code complicated for such minor and insane devices.
I don't think this is the only device that has sub-32-bit DMA restrictions, this will just lead to a bunch of duplicated code. In particular, how is LPC DMA supposed to work?
At the very least we should be allowing the driver to deal with the failure instead of panicing the system. Otherwise we are just leaving a land mine for people to trip over.
Quoted 6 lines · On Mon, 31 Jan 2011 21:22:23 -0600 Robert Hancock <[email protected]> wrote:
Quoted 3 lines
Some ideas to implement something that works for such device were discussed. Seems that the conclusion is that it's doesn't worth making the common code complicated for such minor and insane devices.
I don't think this is the only device that has sub-32-bit DMA restrictions, this will just lead to a bunch of duplicated code.
Yeah, not only device but not many.
The block layer has the own bouncing mechanism. Some network drivers have the similar bouncing code. I don't know if there are other kinds of drivers that have the own bouncing code.
I thought that we can make mm/bounce.c (used for block drivers now) work any drivers without complicating it. We could make swiotlb to do but it's too complicated and it doesn't worth.
Quoted 2 lines
In
particular, how is LPC DMA supposed to work?
LPC DMA can't do 32bit dma?
Quoted 3 lines
At the very least we should be allowing the driver to deal with the failure instead of panicing the system. Otherwise we are just leaving a land mine for people to trip over.
Agreed. swiotlb shouldn't panic in this case. I'll take care of it.
Quoted 21 lines · FUJITA Tomonori
Quoted 6 lines · On Mon, 31 Jan 2011 21:22:23 -0600 Robert Hancock<[email protected]> wrote:
Quoted 3 lines
Some ideas to implement something that works for such device were discussed. Seems that the conclusion is that it's doesn't worth making the common code complicated for such minor and insane devices.
I don't think this is the only device that has sub-32-bit DMA restrictions, this will just lead to a bunch of duplicated code.
Yeah, not only device but not many.
The block layer has the own bouncing mechanism. Some network drivers have the similar bouncing code. I don't know if there are other kinds of drivers that have the own bouncing code.
I thought that we can make
mm/bounce.c(used for block drivers now) work any drivers without complicating it. We could make swiotlb to do but it's too complicated and it doesn't worth.
Quoted 4 lines
Quoted 2 lines
In
particular, how is LPC DMA supposed to work?LPC DMA can't do 32bit dma?
At least not if it's using ISA-style 3rd-party DMA. Some devices may do bus-mastering and be able to do 32-bit DMA, but I've never seen one, at least.
Quoted 7 lines
Quoted 5 lines
At the very least we should be allowing the driver to deal with the failure instead of panicing the system. Otherwise we are just leaving a land mine for people to trip over.
Agreed. swiotlb shouldn't panic in this case. I'll take care of it.
Quoted 7 lines · On Tue, 1 Feb 2011 10:28:00 +0900 FUJITA Tomonori…
swiotlb allocates the bounce buffer when a system boots up. We can't allocate much in
GFP_DMA. swiotlb uses somewhere under 4GB. So it can't help devices that have odddma_mask(that is, except for 4GB).Unfortunately, Such device needs to do own custom bouncing or needs their subsystem to does that.
I think we're chasing the wrong problem here.
swiotlb uses alloc_bootmem_low_pages() to try to get buffers as low in memory as possible. I asked someone who is hitting this bug to try 2.6.36 and he reports the buffers really are low there:
2.6.36: 5c00000
2.6.37: db600000
So something happened very early in the 2.6.37-rc cycle that changed this behavior. I tried looking at the bootmem code but could not see the problem. The only related option I could find in .config was this:
# CONFIG_NO_BOOTMEM is not set
It was set this way in both .36 and .37.
Quoted 16 lines · On Tue, 1 Feb 2011 16:18:59 -0500 Chuck Ebbert <[email protected]> wrote:
Quoted 6 lines
swiotlb allocates the bounce buffer when a system boots up. We can't allocate much in
GFP_DMA. swiotlb uses somewhere under 4GB. So it can't help devices that have odddma_mask(that is, except for 4GB).Unfortunately, Such device needs to do own custom bouncing or needs their subsystem to does that.
I think we're chasing the wrong problem here.
swiotlb uses
alloc_bootmem_low_pages()to try to get buffers as low in memory as possible. I asked someone who is hitting this bug to try 2.6.36 and he reports the buffers really are low there:2.6.36: 5c00000
2.6.37:db600000
Quoted 7 lines
So something happened very early in the 2.6.37-rc cycle that changed this behavior. I tried looking at the bootmem code but could not see the problem. The only related option I could find in .config was this:
#
CONFIG_NO_BOOTMEMis not setIt was set this way in both .36 and .37.
I don't think this matters because we can't guarantee that swiotlb can't allocate ZONE_DMA. Obviously, the default swiotlb bouncing buffer size, 64MB, is too large for ZONE_DMA.
If you are lucky, you sometimes can allocate lower memory if swiotlb allocates lower memory at the boot time. But you can't assume that. swiotlb isn't designed to handle such.
Can you try the patch that I sent you? As I wrote, b44 has the own bouncing mechanism, so we should be fine if swiotlb returns an appropriate error instead of just panicking.
Quoted 23 lines · On Wed, 2 Feb 2011 09:06:34 +0900 FUJITA Tomonori…
Quoted 20 lines · On Tue, 1 Feb 2011 16:18:59 -0500 Chuck Ebbert <[email protected]> wrote:
Quoted 6 lines
swiotlb allocates the bounce buffer when a system boots up. We can't allocate much in
GFP_DMA. swiotlb uses somewhere under 4GB. So it can't help devices that have odddma_mask(that is, except for 4GB).Unfortunately, Such device needs to do own custom bouncing or needs their subsystem to does that.
I think we're chasing the wrong problem here.
swiotlb uses
alloc_bootmem_low_pages()to try to get buffers as low in memory as possible. I asked someone who is hitting this bug to try 2.6.36 and he reports the buffers really are low there:2.6.36: 5c00000
2.6.37:db600000So something happened very early in the 2.6.37-rc cycle that changed this behavior. I tried looking at the bootmem code but could not see the problem. The only related option I could find in .config was this:
Quoted 6 lines
Quoted 3 lines
#
CONFIG_NO_BOOTMEMis not setIt was set this way in both .36 and .37.
I don't think this matters because we can't guarantee that swiotlb can't allocate
ZONE_DMA.
Oops, I meant that, we can't guarantee that swiotlb allocates ZONE_DMA.
Quoted 15 lines
Obviously, the default swiotlb bouncing
buffer size, 64MB, is too large forZONE_DMA.If you are lucky, you sometimes can allocate lower memory if swiotlb allocates lower memory at the boot time. But you can't assume that. swiotlb isn't designed to handle such.
Can you try the patch that I sent you? As I wrote, b44 has the own bouncing mechanism, so we should be fine if swiotlb returns an appropriate error instead of just panicking.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 7 lines · Robert Hancock
Quoted 6 lines · Andi Kleen
Quoted 5 lines · Chuck Ebbert
The b44 driver is triggering this panic in
swiotlb_map_page():if (!dma_capable(dev,
dev_addr, size))
panic("map_single: bounce buffer is not DMA'ble");
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
The kernel log says the bounce buffers are at
0xdb400000, but b44 can only do DMA to the first 1GB of memory:b44 needs to use
GFP_DMAthen and do its own custom bouncing. The standard pci_map_* bounce buffering is only designed for at least 32bit capable devices.That seems wrong - it's a documented API and that restriction isn'
Please read the documentation. When PCI DMA cannot handle it it returns failure. That is what happened here.
Besides historically PCI-DMA never handled < 4GB.
Quoted 2 lines
documented. Either it should comply with the request or return a failure if it can't accomodate it, not just blow up internally.
It does return failure, that is what breaks the driver because it doesn't handle it.
Quoted 2 lines
There's no reason the driver should have to deal with this on its own.
Even if swiotlb could handle it, other implementations of PCI-DMA can not
Quoted 3 lines
In this case the DMA mapping code should really be falling back to
GFP_DMAautomatically if the IOMMU aperture is outside the DMA mask of the device.
No, no -- you cannot allocate memory in DMA API. It's forbidden. Some paths that use it cannot tolerate it.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only
Quoted 11 lines · Andi Kleen
Quoted 10 lines · Robert Hancock
Quoted 9 lines · Andi Kleen
Quoted 7 lines · Chuck Ebbert
The b44 driver is triggering this panic in
swiotlb_map_page():if (!dma_capable(dev,
dev_addr, size))panic("map_single: bounce buffer is not DMA'ble");The kernel log says the bounce buffers are at
0xdb400000, but b44 can only do DMA to the first 1GB of memory:
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
b44 needs to use
GFP_DMAthen and do its own custom bouncing. The standard pci_map_* bounce buffering is only designed for at least 32bit capable devices.That seems wrong - it's a documented API and that restriction isn'
Please read the documentation. When PCI DMA cannot handle it it returns failure. That is what happened here.
Apparently not, given that it hit a panic inside the swiotlb code..
Quoted 7 lines · On Mon, 31 Jan 2011 10:54:12 -0500 Chuck Ebbert <[email protected]> wrote:
The b44 driver is triggering this panic in
swiotlb_map_page():if (!dma_capable(dev,
dev_addr, size))
panic("map_single: bounce buffer is not DMA'ble");The kernel log says the bounce buffers are at
0xdb400000, but b44 can only do DMA to the first 1GB of memory:
Can you try this? b44 has the own bouncing mechanism so if swiotlb returns an error, the driver can allocate an appropriate buffer.
lib/swiotlb.c +4 -2
@@ -686,8 +686,10 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,/* * Ensure that the address returned is DMA'ble */ -if (!dma_capable (dev, dev_addr, size)) -panic ("map_single: bounce buffer is not DMA'ble" ); +if (!dma_capable (dev, dev_addr, size)) { +swiotlb_tbl_unmap_single (dev, map, size, dir); + dev_addr =swiotlb_virt_to_bus (dev, io_tlb_overflow_buffer); + }return dev_addr; }
Quoted 21 lines · On Tue, 1 Feb 2011 22:54:56 +0900 FUJITA Tomonori…
Quoted 7 lines · On Mon, 31 Jan 2011 10:54:12 -0500 Chuck Ebbert <[email protected]> wrote:
The b44 driver is triggering this panic in
swiotlb_map_page():if (!dma_capable(dev,
dev_addr, size))
panic("map_single: bounce buffer is not DMA'ble");The kernel log says the bounce buffers are at
0xdb400000, but b44 can only do DMA to the first 1GB of memory:Can you try this? b44 has the own bouncing mechanism so if swiotlb returns an error, the driver can allocate an appropriate buffer.
lib/swiotlb.c +0
-0@@ -686,8 +686,10 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,/* * Ensure that the address returned is DMA'ble
lib/swiotlb.c dma_addr_t swiotlb_map_page(struct device *dev, struct page *page, · line 695
Quoted 10 lines
lib/swiotlb.c +4
-2*/ -if (!dma_capable (dev, dev_addr, size)) -panic ("map_single: bounce buffer is not DMA'ble" ); +if (!dma_capable (dev, dev_addr, size)) { +swiotlb_tbl_unmap_single (dev, map, size, dir); + dev_addr =swiotlb_virt_to_bus (dev, io_tlb_overflow_buffer); + }return dev_addr; }
Ping, any chance to try this?
Thanks,
Quoted 17 lines · FUJITA Tomonori
Quoted 14 lines · On Tue, 1 Feb 2011 22:54:56 +0900 FUJITA Tomonori…
Quoted 7 lines · On Mon, 31 Jan 2011 10:54:12 -0500 Chuck Ebbert <[email protected]> wrote:
The b44 driver is triggering this panic in
swiotlb_map_page():if (!dma_capable(dev,
dev_addr, size))
panic("map_single: bounce buffer is not DMA'ble");The kernel log says the bounce buffers are at
0xdb400000, but b44 can only do DMA to the first 1GB of memory:Can you try this? b44 has the own bouncing mechanism so if swiotlb returns an error, the driver can allocate an appropriate buffer.
Quoted 14 lines
Quoted 5 lines
lib/swiotlb.c +0
-0lib/swiotlb.c dma_addr_t swiotlb_map_page(struct device *dev, · line 692
Quoted 9 lines
lib/swiotlb.c +3
-2@@ -686,8 +686,10 @@ dma_addr_t swiotlb_map_page(struct device *dev,struct page *page,/* * Ensure that the address returned is DMA'ble */ - if (!dma_capable(dev, dev_addr, size)) - panic("map_single: bounce buffer is not DMA'ble"); + if (!dma_capable(dev, dev_addr, size)) { +swiotlb_tbl_unmap_single (dev ,map ,size ,dir ); + dev_addr = swiotlb_virt_to_bus(dev,
io_tlb_overflow_buffer);
Quoted 6 lines
Quoted 4 lines
lib/swiotlb.c +1
-0+ }return dev_addr;}
Ping, any chance to try this?
Friend machine oopsed on network with b44 on 2.6.37, so we tried this patch and network card was detected nicely and was actually usable. The oops started to happen after he upgraded memory from 2GB to 4GB in his dell vostro 1500 notebook.
03:00.0 Ethernet controller: Broadcom Corporation BCM4401-B0 100Base-TX (rev
02)
# grep b44 dmesg.log [ 8.524297] b44 0000:03:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17 [ 8.576838] b44: b44.c:v2.0 [ 8.593920] b44 ssb1:0: eth0: Broadcom 44xx/47xx 10/100BaseT Ethernet 00:1c:23:9b:3f:5f [ 55.350228] b44 ssb1:0: eth0: Link is up at 100 Mbps, full duplex [ 55.350233] b44 ssb1:0: eth0: Flow control is off for TX and off for RX
Quoted 1 line
Thanks,
Signature · 2 lines
Arkadiusz Miśkiewicz PLD/Linux Team arekm / maven.pl http://ftp.pld-linux.org/
Quoted 36 lines · On Mon, 14 Feb 2011 17:41:54 +0100 Arkadiusz Mi$(D+\(Bkiewicz…
Quoted 17 lines · FUJITA Tomonori
Quoted 14 lines · On Tue, 1 Feb 2011 22:54:56 +0900 FUJITA Tomonori…
Quoted 7 lines · On Mon, 31 Jan 2011 10:54:12 -0500 Chuck Ebbert <[email protected]> wrote:
The b44 driver is triggering this panic in
swiotlb_map_page():if (!dma_capable(dev,
dev_addr, size))
panic("map_single: bounce buffer is not DMA'ble");The kernel log says the bounce buffers are at
0xdb400000, but b44 can only do DMA to the first 1GB of memory:Can you try this? b44 has the own bouncing mechanism so if swiotlb returns an error, the driver can allocate an appropriate buffer.
Quoted 14 lines
Quoted 5 lines
lib/swiotlb.c +0
-0lib/swiotlb.c dma_addr_t swiotlb_map_page(struct device *dev, · line 692
Quoted 9 lines
lib/swiotlb.c +3
-2@@ -686,8 +686,10 @@ dma_addr_t swiotlb_map_page(struct device *dev,struct page *page,/* * Ensure that the address returned is DMA'ble */ - if (!dma_capable(dev, dev_addr, size)) - panic("map_single: bounce buffer is not DMA'ble"); + if (!dma_capable(dev, dev_addr, size)) { +swiotlb_tbl_unmap_single (dev ,map ,size ,dir ); + dev_addr = swiotlb_virt_to_bus(dev,
io_tlb_overflow_buffer);Quoted 2 lines
Quoted 2 lines
lib/swiotlb.c +1
-0+ }
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
lib/swiotlb.c +0
-0return dev_addr;}
Ping, any chance to try this?
Friend machine oopsed on network with b44 on 2.6.37, so we tried this patch and network card was detected nicely and was actually usable. The oops
Thanks for the confirmation.
Quoted 2 lines
started to happen after he upgraded memory from 2GB to 4GB in his dell vostro 1500 notebook.
Yeah, a system with 2GB memory doesn't use swiotlb so you don't hit this bug.
Thanks,
Quoted 18 lines · On Mon, 14 Feb 2011 20:59:43 +0900 FUJITA Tomonori…
Quoted 18 lines
Can you try this? b44 has the own bouncing mechanism so if swiotlb returns an error, the driver can allocate an appropriate buffer.
lib/swiotlb.c +4
-2@@ -686,8 +686,10 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,/* * Ensure that the address returned is DMA'ble */ -if (!dma_capable (dev, dev_addr, size)) -panic ("map_single: bounce buffer is not DMA'ble" ); +if (!dma_capable (dev, dev_addr, size)) { +swiotlb_tbl_unmap_single (dev, map, size, dir); + dev_addr =swiotlb_virt_to_bus (dev, io_tlb_overflow_buffer); + }
Quoted 4 lines
lib/swiotlb.c dma_addr_t swiotlb_map_page(struct device *dev, struct page *page, · line 695
Quoted 2 lines
lib/swiotlb.c +0
-0return dev_addr; }Ping, any chance to try this?
The user who reported the bug tried it and says that it works.