Skip to contentSkip to conversation

ISP1760 driver crashes

DISCUSSION Alan Stern 4 participants 12 messages Nov 19, 2008 – Nov 21, 2008 1 message not in the archive

Subsystems (3)
lore
Share & tools
lore

Alan Stern

author maintainer
#1 lore raw
In reply to a message not in the archive
Quoted 7 lines · Sebastian Andrzej Siewior
Quoted 5 lines · Intelligent Platforms)

I indeed have HIGHMEM enabled in my configuration.
I recompiled the kernel without HIGHMEM and it works. I don't think that this a satisfying solution for a board with up to 2GB of RAM and considerable amount of VMALLOC space, but at least this works. If you have any more ideas how to circumvent this, please let me know.

Quoted 8 lines

Sure, this is not a sollution but atleast now I know what happens:
- The kernel allocates memory for transfer
- the memory is highmem and not in kernel so the buffer is NULL
- we don't have a dma-mask and therefore the dma address is 0
- boom

The sollution would be probably to prevent the usb-storage core to allocate memory from HIGHMEM.

usb-storage doesn't allocate the memory. The memory is allocated by the block layer or the filesystem.

Quoted 4 lines

Now I don't if there is a flag for something
like that and I am not using that. On the other hand this may be broken for a long time and you are the first one which has that much memory with no DMA-capable USB controller.

Jens, is there any way to tell the kernel that a device uses PIO and therefore its buffers shouldn't be allocated in high memory? For example, shouldn't a NULL dma_mask do this?

If not, are there standard routines to set up bounce buffers for such devices?

Alan Stern

#2 lore raw
Quoted 21 lines · Alan Stern
Quoted 7 lines · Sebastian Andrzej Siewior
Quoted 5 lines · Intelligent Platforms)

I indeed have HIGHMEM enabled in my configuration.
I recompiled the kernel without HIGHMEM and it works. I don't think that this a satisfying solution for a board with up to 2GB of RAM and considerable amount of VMALLOC space, but at least this works. If you have any more ideas how to circumvent this, please let me know.

Quoted 8 lines

Sure, this is not a sollution but atleast now I know what happens:
- The kernel allocates memory for transfer
- the memory is highmem and not in kernel so the buffer is NULL
- we don't have a dma-mask and therefore the dma address is 0
- boom

The sollution would be probably to prevent the usb-storage core to allocate memory from HIGHMEM.

usb-storage doesn't allocate the memory. The memory is allocated by the block layer or the filesystem.

Quoted 8 lines
Quoted 4 lines

Now I don't if there is a flag for something
like that and I am not using that. On the other hand this may be broken for a long time and you are the first one which has that much memory with no DMA-capable USB controller.

Jens, is there any way to tell the kernel that a device uses PIO and therefore its buffers shouldn't be allocated in high memory? For example, shouldn't a NULL dma_mask do this?

Sure, just use blk_queue_bounce_limit(q, BLK_BOUNCE_HIGH), then you are certain that you will always have a virtual mapping for the IO you receive.

Or you can use the bio kmap/kunmap helpers to get such a mapping temporarily if you wish. But if your pio condition is permanent, you may as well just use bouncing.

Signature · 1 line
Jens Axboe
author maintainer
#3 lore raw
Quoted 31 lines · Jens Axboe
Quoted 21 lines · Alan Stern
Quoted 7 lines · Sebastian Andrzej Siewior
Quoted 5 lines · Intelligent Platforms)

I indeed have HIGHMEM enabled in my configuration.
I recompiled the kernel without HIGHMEM and it works. I don't think that this a satisfying solution for a board with up to 2GB of RAM and considerable amount of VMALLOC space, but at least this works. If you have any more ideas how to circumvent this, please let me know.

Quoted 8 lines

Sure, this is not a sollution but atleast now I know what happens:
- The kernel allocates memory for transfer
- the memory is highmem and not in kernel so the buffer is NULL
- we don't have a dma-mask and therefore the dma address is 0
- boom

The sollution would be probably to prevent the usb-storage core to allocate memory from HIGHMEM.

usb-storage doesn't allocate the memory. The memory is allocated by the block layer or the filesystem.

Quoted 8 lines
Quoted 4 lines

Now I don't if there is a flag for something
like that and I am not using that. On the other hand this may be broken for a long time and you are the first one which has that much memory with no DMA-capable USB controller.

Jens, is there any way to tell the kernel that a device uses PIO and therefore its buffers shouldn't be allocated in high memory? For example, shouldn't a NULL dma_mask do this?

Quoted 7 lines

Sure, just use blk_queue_bounce_limit(q, BLK_BOUNCE_HIGH), then you are certain that you will always have a virtual mapping for the IO you receive.

Or you can use the bio kmap/kunmap helpers to get such a mapping temporarily if you wish. But if your pio condition is permanent, you may as well just use bouncing.

Thank you.

Thomas, the blk_queue_bounce_limit() routine is called in drivers/scsi/scsi_lib.c:__scsi_alloc_queue(). The value it passes is computed by scsi_calculate_bounce_limit(), and in that routine host_dev->dma_mask should be NULL (since isp1760-hcd sets the mask to NULL). Therefore the bounce limit should be 0xffffffff.

Now maybe this value isn't correct. You can try the patch below to see if it helps. If it doesn't, add a printk in __scsi_alloc_queue() to see what bounce limit value is getting used.

Alan Stern

drivers/scsi/scsi_lib.c +1 -1
Index: usb-2.6/drivers/scsi/scsi_lib.c
===================================================================
@@ -1684,7 +1684,7 @@ static void scsi_request_fn(struct reque
 u64 scsi_calculate_bounce_limit(struct Scsi_Host *shost)
 {
 	struct device *host_dev;
-	u64 bounce_limit = 0xffffffff;
+	u64 bounce_limit = BLK_BOUNCE_HIGH;
 
 	if (shost->unchecked_isa_dma)
 		return BLK_BOUNCE_ISA;
#4 lore raw
Quoted 63 lines · Alan Stern
Quoted 31 lines · Jens Axboe
Quoted 21 lines · Alan Stern
Quoted 7 lines · Sebastian Andrzej Siewior
Quoted 5 lines · Intelligent Platforms)

I indeed have HIGHMEM enabled in my configuration.
I recompiled the kernel without HIGHMEM and it works. I don't think that this a satisfying solution for a board with up to 2GB of RAM and considerable amount of VMALLOC space, but at least this works. If you have any more ideas how to circumvent this, please let me know.

Quoted 8 lines

Sure, this is not a sollution but atleast now I know what happens:
- The kernel allocates memory for transfer
- the memory is highmem and not in kernel so the buffer is NULL
- we don't have a dma-mask and therefore the dma address is 0
- boom

The sollution would be probably to prevent the usb-storage core to allocate memory from HIGHMEM.

usb-storage doesn't allocate the memory. The memory is allocated by the block layer or the filesystem.

Quoted 8 lines
Quoted 4 lines

Now I don't if there is a flag for something
like that and I am not using that. On the other hand this may be broken for a long time and you are the first one which has that much memory with no DMA-capable USB controller.

Jens, is there any way to tell the kernel that a device uses PIO and therefore its buffers shouldn't be allocated in high memory? For example, shouldn't a NULL dma_mask do this?

Quoted 7 lines

Sure, just use blk_queue_bounce_limit(q, BLK_BOUNCE_HIGH), then you are certain that you will always have a virtual mapping for the IO you receive.

Or you can use the bio kmap/kunmap helpers to get such a mapping temporarily if you wish. But if your pio condition is permanent, you may as well just use bouncing.

Thank you.

Thomas, the blk_queue_bounce_limit() routine is called in drivers/scsi/scsi_lib.c:__scsi_alloc_queue(). The value it passes is computed by scsi_calculate_bounce_limit(), and in that routine host_dev->dma_mask should be NULL (since isp1760-hcd sets the mask to NULL). Therefore the bounce limit should be 0xffffffff.

Now maybe this value isn't correct. You can try the patch below to see if it helps. If it doesn't, add a printk in __scsi_alloc_queue() to see what bounce limit value is getting used.

Alan Stern

Index: usb-2.6/drivers/scsi/scsi_lib.c
===================================================================

drivers/scsi/scsi_lib.c static void scsi_request_fn(struct reque · line 1690

Quoted 8 lines
drivers/scsi/scsi_lib.c +1 -1
@@ -1684,7 +1684,7 @@ static void scsi_request_fn(struct reque
 u64 scsi_calculate_bounce_limit(struct Scsi_Host *shost)
 {
 	struct device *host_dev;
-	u64 bounce_limit = 0xffffffff;
+	u64 bounce_limit = BLK_BOUNCE_HIGH;
 
 	if (shost->unchecked_isa_dma)
 		return BLK_BOUNCE_ISA;

The best solution is probably to either provide a "doesn't do highmem" in the scsi host template, or provide an appropriate DMA mask for the pci device to indicate it through that setting instead.

Signature · 1 line
Jens Axboe
#12 lore raw
In reply to Alan Stern
Quoted 61 lines · Jens Axboe
Quoted 9 lines · Alan Stern
Quoted 7 lines · Sebastian Andrzej Siewior
Quoted 6 lines
Quoted 3 lines · Intelligent Platforms)

I indeed have HIGHMEM enabled in my configuration. I recompiled the kernel without HIGHMEM and it works. I don't think that this a satisfying solution for a board

with up to 2GB

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

of RAM and considerable amount of VMALLOC space, but

at least this works.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

If you have any more ideas how to circumvent this,

please let me know.

Quoted 1 line
Quoted 1 line
Quoted 1 line

Sure, this is not a sollution but atleast now I know

what happens:

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

- The kernel allocates memory for transfer
- the memory is highmem and not in kernel so the buffer is NULL
- we don't have a dma-mask and therefore the dma address is 0
- boom

The sollution would be probably to prevent the

usb-storage core to

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

allocate memory from HIGHMEM.

usb-storage doesn't allocate the memory. The memory is

allocated by

Quoted 5 lines
Quoted 5 lines

the block layer or the filesystem.

Quoted 3 lines

Now I don't if there is a flag for something like that and I am not using that. On the other hand this may be broken for a long time and you are the first one which has that much

memory with no

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

DMA-capable USB controller.

Jens, is there any way to tell the kernel that a device

uses PIO and

Quoted 5 lines
Quoted 2 lines

therefore its buffers shouldn't be allocated in high memory? For example, shouldn't a NULL dma_mask do this?

Sure, just use blk_queue_bounce_limit(q, BLK_BOUNCE_HIGH), then you are certain that you will always have a virtual mapping for

the IO you

Quoted 4 lines

receive.

Or you can use the bio kmap/kunmap helpers to get such a mapping temporarily if you wish. But if your pio condition is

permanent, you

Quoted 1 line

may as well just use bouncing.

Thank you.

Thomas, the blk_queue_bounce_limit() routine is called in drivers/scsi/scsi_lib.c:__scsi_alloc_queue(). The value it passes is computed by scsi_calculate_bounce_limit(), and in that routine host_dev->dma_mask should be NULL (since isp1760-hcd sets the mask to NULL). Therefore the bounce limit should be 0xffffffff.

Quoted 6 lines

Now maybe this value isn't correct. You can try the patch below to see if it helps. If it doesn't, add a printk in __scsi_alloc_queue() to see what bounce limit value is getting used.

Alan Stern

Thanks Alan,
This seems to fix the problem.

Thomas

Quoted 16 lines

Index: usb-2.6/drivers/scsi/scsi_lib.c
===================================================================

drivers/scsi/scsi_lib.c +1 -1
@@ -1684,7 +1684,7 @@ static void scsi_request_fn(struct reque
 u64 scsi_calculate_bounce_limit(struct Scsi_Host *shost)  {
 	struct device *host_dev;
-	u64 bounce_limit = 0xffffffff;
+	u64 bounce_limit = BLK_BOUNCE_HIGH;
 
 	if (shost->unchecked_isa_dma)
 		return BLK_BOUNCE_ISA;
 
author maintainer
#5 lore raw
In reply to Jens Axboe
Quoted 9 lines · Jens Axboe
Quoted 9 lines
drivers/scsi/scsi_lib.c +1 -1
@@ -1684,7 +1684,7 @@ static void scsi_request_fn(struct reque
 u64 scsi_calculate_bounce_limit(struct Scsi_Host *shost)
 {
 	struct device *host_dev;
-	u64 bounce_limit = 0xffffffff;
+	u64 bounce_limit = BLK_BOUNCE_HIGH;
 
Quoted 7 lines

drivers/scsi/scsi_lib.c static void scsi_request_fn(struct reque · line 1690

Quoted 3 lines
drivers/scsi/scsi_lib.c +0 -0
 	if (shost->unchecked_isa_dma)
 		return BLK_BOUNCE_ISA;

The best solution is probably to either provide a "doesn't do highmem" in the scsi host template, or provide an appropriate DMA mask for the pci device to indicate it through that setting instead.

The DMA mask is currently set to NULL. Is that not appropriate for a device that can't do DMA? If not, then what would be appropriate?

Also, is the patch above not correct?

Alan Stern

#6 lore raw
Quoted 19 lines · Alan Stern
Quoted 9 lines · Jens Axboe
Quoted 9 lines
drivers/scsi/scsi_lib.c +1 -1
@@ -1684,7 +1684,7 @@ static void scsi_request_fn(struct reque
 u64 scsi_calculate_bounce_limit(struct Scsi_Host *shost)
 {
 	struct device *host_dev;
-	u64 bounce_limit = 0xffffffff;
+	u64 bounce_limit = BLK_BOUNCE_HIGH;
 
Quoted 7 lines

drivers/scsi/scsi_lib.c static void scsi_request_fn(struct reque · line 1690

Quoted 3 lines
drivers/scsi/scsi_lib.c +0 -0
 	if (shost->unchecked_isa_dma)
 		return BLK_BOUNCE_ISA;

The best solution is probably to either provide a "doesn't do highmem" in the scsi host template, or provide an appropriate DMA mask for the pci device to indicate it through that setting instead.

Quoted 2 lines

The DMA mask is currently set to NULL. Is that not appropriate for a device that can't do DMA? If not, then what would be appropriate?

It's changing behaviour. There's no current rule that says if you don't have a dma mask set, we only do PIO (even if such a rule DOES make sense). Additionally, you don't HAVE to bounce for PIO. As I wrote earlier, it's perfectly feasible to use bio kmap'ings to do the transfer.

Quoted 1 line

Also, is the patch above not correct?

It'll certainly work in the sense that if you don't have a dma_mask set, you only get lowmem pages. Whether the new behaviour is something we want, not sure. Check with James what he thinks, it's his domain.

Signature · 1 line
Jens Axboe
#7 lore raw
Quoted 29 lines · On Wed, 19 Nov 2008 18:21:25 +0100 Jens Axboe <[email protected]> wrote:
Quoted 19 lines · Alan Stern
Quoted 9 lines · Jens Axboe
Quoted 9 lines
drivers/scsi/scsi_lib.c +1 -1
@@ -1684,7 +1684,7 @@ static void scsi_request_fn(struct reque
 u64 scsi_calculate_bounce_limit(struct Scsi_Host *shost)
 {
 	struct device *host_dev;
-	u64 bounce_limit = 0xffffffff;
+	u64 bounce_limit = BLK_BOUNCE_HIGH;
 
Quoted 7 lines

drivers/scsi/scsi_lib.c static void scsi_request_fn(struct reque · line 1690

Quoted 3 lines
drivers/scsi/scsi_lib.c +0 -0
 	if (shost->unchecked_isa_dma)
 		return BLK_BOUNCE_ISA;

The best solution is probably to either provide a "doesn't do highmem" in the scsi host template, or provide an appropriate DMA mask for the pci device to indicate it through that setting instead.

Quoted 2 lines

The DMA mask is currently set to NULL. Is that not appropriate for a device that can't do DMA? If not, then what would be appropriate?

It's changing behaviour. There's no current rule that says if you don't have a dma mask set, we only do PIO (even if such a rule DOES make sense). Additionally, you don't HAVE to bounce for PIO. As I wrote earlier, it's perfectly feasible to use bio kmap'ings to do the transfer.

Quoted 5 lines
Quoted 1 line

Also, is the patch above not correct?

It'll certainly work in the sense that if you don't have a dma_mask set, you only get lowmem pages. Whether the new behaviour is something we want, not sure. Check with James what he thinks, it's his domain.

We have been used 4GB for long time if dma_mask is zero (I guess we use 4GB as kinda the default dma address limit at several places). The majority of drivers (such as pci) sets properly dev->dma_mask so the patch might not change anything but suddenly changing the long-standing rule in an odd way (use BLK_BOUNCE_HIGH if dma_mask is zero) doesn't sound a good idea to me.

Why not calling blk_queue_bounce_limit() in the slave_configure hook? I think that it's the common way for SCSI LLDs with odd bounce limit.

#8 lore raw
Quoted 45 lines · FUJITA Tomonori
Quoted 29 lines · On Wed, 19 Nov 2008 18:21:25 +0100 Jens Axboe <[email protected]> wrote:
Quoted 19 lines · Alan Stern
Quoted 9 lines · Jens Axboe
Quoted 9 lines
drivers/scsi/scsi_lib.c +1 -1
@@ -1684,7 +1684,7 @@ static void scsi_request_fn(struct reque
 u64 scsi_calculate_bounce_limit(struct Scsi_Host *shost)
 {
 	struct device *host_dev;
-	u64 bounce_limit = 0xffffffff;
+	u64 bounce_limit = BLK_BOUNCE_HIGH;
 
Quoted 7 lines

drivers/scsi/scsi_lib.c static void scsi_request_fn(struct reque · line 1690

Quoted 3 lines
drivers/scsi/scsi_lib.c +0 -0
 	if (shost->unchecked_isa_dma)
 		return BLK_BOUNCE_ISA;

The best solution is probably to either provide a "doesn't do highmem" in the scsi host template, or provide an appropriate DMA mask for the pci device to indicate it through that setting instead.

Quoted 2 lines

The DMA mask is currently set to NULL. Is that not appropriate for a device that can't do DMA? If not, then what would be appropriate?

It's changing behaviour. There's no current rule that says if you don't have a dma mask set, we only do PIO (even if such a rule DOES make sense). Additionally, you don't HAVE to bounce for PIO. As I wrote earlier, it's perfectly feasible to use bio kmap'ings to do the transfer.

Quoted 5 lines
Quoted 1 line

Also, is the patch above not correct?

It'll certainly work in the sense that if you don't have a dma_mask set, you only get lowmem pages. Whether the new behaviour is something we want, not sure. Check with James what he thinks, it's his domain.

We have been used 4GB for long time if dma_mask is zero (I guess we use 4GB as kinda the default dma address limit at several places). The majority of drivers (such as pci) sets properly dev->dma_mask so the patch might not change anything but suddenly changing the long-standing rule in an odd way (use BLK_BOUNCE_HIGH if dma_mask is zero) doesn't sound a good idea to me.

Quoted 2 lines

Why not calling blk_queue_bounce_limit() in the slave_configure hook? I think that it's the common way for SCSI LLDs with odd bounce limit.

Yep, that should indeed work just fine and is preferable to changing this logic.

Signature · 1 line
Jens Axboe
author maintainer
#9 lore raw
In reply to FUJITA Tomonori
Quoted 9 lines · FUJITA Tomonori

We have been used 4GB for long time if dma_mask is zero (I guess we use 4GB as kinda the default dma address limit at several places). The majority of drivers (such as pci) sets properly dev->dma_mask so the patch might not change anything but suddenly changing the long-standing rule in an odd way (use BLK_BOUNCE_HIGH if dma_mask is zero) doesn't sound a good idea to me.

Why not calling blk_queue_bounce_limit() in the slave_configure hook? I think that it's the common way for SCSI LLDs with odd bounce limit.

Thomas, here's a patch to do what Tomonori suggests. Try replacing the old patch with this one.

Alan Stern

drivers/usb/storage/scsiglue.c +8 -0
Index: usb-2.6/drivers/usb/storage/scsiglue.c
===================================================================
@@ -129,6 +129,14 @@ static int slave_configure(struct scsi_d
 					      max_sectors);
 	}
 
+	/* Some USB host controllers can't do DMA; they have to use PIO.
+	 * They indicate this by setting their dma_mask to NULL.  For
+	 * such controllers we need to make sure the block layer sets
+	 * up bounce buffers in addressable memory.
+	 */
+	if (!us->pusb_dev->bus->controller->dma_mask)
+		blk_queue_bounce_limit(sdev->request_queue, BLK_BOUNCE_HIGH);
+
 	/* We can't put these settings in slave_alloc() because that gets
 	 * called before the device type is known.  Consequently these
 	 * settings can't be overridden via the scsi devinfo mechanism. */
#10 lore raw
Quoted 28 lines · Alan Stern
Quoted 9 lines · FUJITA Tomonori

We have been used 4GB for long time if dma_mask is zero (I guess we use 4GB as kinda the default dma address limit at several places). The majority of drivers (such as pci) sets properly dev->dma_mask so the patch might not change anything but suddenly changing the long-standing rule in an odd way (use BLK_BOUNCE_HIGH if dma_mask is zero) doesn't sound a good idea to me.

Why not calling blk_queue_bounce_limit() in the slave_configure hook? I think that it's the common way for SCSI LLDs with odd bounce limit.

Thomas, here's a patch to do what Tomonori suggests. Try replacing the old patch with this one.

Alan Stern

Index: usb-2.6/drivers/usb/storage/scsiglue.c
===================================================================

drivers/usb/storage/scsiglue.c +1 -0
@@ -129,6 +129,14 @@ static int slave_configure(struct scsi_d
 					      max_sectors);
 	}
 
+	/* Some USB host controllers can't do DMA; they have to use PIO.

drivers/usb/storage/scsiglue.c static int slave_configure(struct scsi_d · line 142

Quoted 10 lines
drivers/usb/storage/scsiglue.c +7 -0
+	 * They indicate this by setting their dma_mask to NULL.  For
+	 * such controllers we need to make sure the block layer sets
+	 * up bounce buffers in addressable memory.
+	 */
+	if (!us->pusb_dev->bus->controller->dma_mask)
+		blk_queue_bounce_limit(sdev->request_queue, BLK_BOUNCE_HIGH);
+
 	/* We can't put these settings in slave_alloc() because that gets
 	 * called before the device type is known.  Consequently these
 	 * settings can't be overridden via the scsi devinfo mechanism. */

That looks like a good fix. You can add my Acked-by: to that, if you wish.

Signature · 1 line
Jens Axboe
#11 lore raw
In reply to Alan Stern
Quoted 13 lines

Alan Stern wrote:

Quoted 2 lines · FUJITA Tomonori

We have been used 4GB for long time if dma_mask is zero (I guess we use 4GB as kinda the default dma address limit at several

places). The

Quoted 1 line

majority of drivers (such as pci) sets properly

dev->dma_mask so the

Quoted 4 lines

patch might not change anything but suddenly changing the long-standing rule in an odd way (use BLK_BOUNCE_HIGH if dma_mask is zero) doesn't sound a good idea to me.

Quoted 8 lines
Quoted 1 line

Why not calling blk_queue_bounce_limit() in the

slave_configure hook?

Quoted 1 line

I think that it's the common way for SCSI LLDs with odd

bounce limit.

Thomas, here's a patch to do what Tomonori suggests. Try replacing the old patch with this one.

Thanks, I've tried the new patch and everything works ok. Thomas

Quoted 27 lines

Alan Stern

Index: usb-2.6/drivers/usb/storage/scsiglue.c
===================================================================

drivers/usb/storage/scsiglue.c +8 -0
@@ -129,6 +129,14 @@ static int slave_configure(struct scsi_d
 					      max_sectors);
 	}
 
+	/* Some USB host controllers can't do DMA; they have to use PIO.
+	 * They indicate this by setting their dma_mask to NULL.  For
+	 * such controllers we need to make sure the block layer sets
+	 * up bounce buffers in addressable memory.
+	 */
+	if (!us->pusb_dev->bus->controller->dma_mask)
+		blk_queue_bounce_limit(sdev->request_queue, 
 BLK_BOUNCE_HIGH);
+
 	/* We can't put these settings in slave_alloc() because 

that gets

drivers/usb/storage/scsiglue.c +0 -0
 	 * called before the device type is known.  Consequently these
 	 * settings can't be overridden via the scsi devinfo 

mechanism. */