New: SCSI host adapter's scatter-gather list size Issue

http://bugzilla.kernel.org/show_bug.cgi?id=15007

Summary: SCSI host adapter's scatter-gather list size Issue
Product: IO/Storage
Version: 2.5
Platform: All
OS/Version: Linux
Tree: Mainline
Status: NEW
Severity: normal
Priority: P1
Component: SCSI
AssignedTo: [email protected]
ReportedBy: [email protected]
CC: [email protected]
Regression: No

I found if I set scatter-gather table size as 1024 Areca's RAID controller, ARC1680, and then execute "mkfs.ext3 /dev/sdX" on a sas/sata volume which is formated as NTFS, it will kill the controller and let it restart.
This issue pops up easily on Intel server board with multi-processors and so
far can not be reproduced on a volume with a original Linux filesystem such as, ext3.
I also found the issue is on the write command with 128 sg entries, each with
4K byte in length. As the command comes from the upper layer, it lacks data in the 128th sg entry.
I exits from 2.6.18 to 2.6.32.
I try to track kernel source code but so far there is no finding.
The driver is ftp://ftp.areca.com.tw/RaidCards/AP_Drivers/Linux/DRIVER/SourceCode/arcmsr.1.20.0X.15-90605.zip. Anybody can give it a try??
Thanks,

Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

http://bugzilla.kernel.org/show_bug.cgi?id=15007

nick cheng <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

                 CC|                            |[email protected]
     Kernel Version|                            |2.6.32

Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

You mean that setting sg_tablesize in scsi_host_template to 1024?

Can your controller hardware and firmware handle 1024 scatter gather entries?

The device driver really get 128 entries?

How can you issue a write command with 128 sg entries?

Even if your user-space application issues a write command with 128 sg entries, the device driver could get less sg entries.

If the device driver gets a command including 128 sg entries (cmd->sdb.table.nents is set to 128) and 128th sg entry doesn't have data, then probably we have a bug in scatter gather or block layer code.

http://bugzilla.kernel.org/show_bug.cgi?id=15007

--- Comment #1 from Anonymous Emailer <[email protected]> 2010-01-14 08:25:25 ---
Reply-To: [email protected]

You mean that setting sg_tablesize in scsi_host_template to 1024?

Can your controller hardware and firmware handle 1024 scatter gather entries?

The device driver really get 128 entries?

How can you issue a write command with 128 sg entries?

Even if your user-space application issues a write command with 128 sg entries, the device driver could get less sg entries.

If the device driver gets a command including 128 sg entries (cmd->sdb.table.nents is set to 128) and 128th sg entry doesn't have data, then probably we have a bug in scatter gather or block layer code.

Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

http://bugzilla.kernel.org/show_bug.cgi?id=15007

--- Comment #2 from nick cheng <[email protected]> 2010-01-14 10:54:42 --- Thanks for reply.
I am sure that hardware and firmware can handle 1024 scatter gather entries.
I can not limit the AP to dispatch a write command with 128 sg entries. I just execute mkfs.ext3 /dev/sdX and print it out as the sg entry number is
128. I found the 128th entry indeed has no data.

Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

So 'no data sg entry' is the cause of hardware reset?

handling.

static int arcmsr_build_ccb(struct AdapterControlBlock *acb,struct CommandControlBlock *ccb,struct scsi_cmnd *pcmd)
{
(snip)...

					arccdbsize += sizeof(struct SG32ENTRY);
				}else{
					struct SG64ENTRY *pdma_sg = (struct SG64ENTRY *)psge;

					pdma_sg->addresshigh = address_hi;
					pdma_sg->address = address_lo;
					pdma_sg->length = length|cpu_to_le32(IS_SG64_ADDR);
					psge += sizeof(struct SG64ENTRY);
					arccdbsize += sizeof(struct SG64ENTRY);
				}
				sl++;

The last line doesn't work. Use sl = sg_next(sl); Using scsi_for_each_sg() is even better (as the driver in mainline does). With this bug, 129th sg entry is broken. If your analysis is correct, there should be other bugs..

http://bugzilla.kernel.org/show_bug.cgi?id=15007

--- Comment #3 from Anonymous Emailer <[email protected]> 2010-01-14 11:12:21 ---
Reply-To: [email protected]

So 'no data sg entry' is the cause of hardware reset?

handling.

static int arcmsr_build_ccb(struct AdapterControlBlock *acb,struct CommandControlBlock *ccb,struct scsi_cmnd *pcmd)
{
(snip)...

                    arccdbsize += sizeof(struct SG32ENTRY);
                }else{
                    struct SG64ENTRY *pdma_sg = (struct SG64ENTRY *)psge;

                    pdma_sg->addresshigh = address_hi;
                    pdma_sg->address = address_lo;
                    pdma_sg->length = length|cpu_to_le32(IS_SG64_ADDR);
                    psge += sizeof(struct SG64ENTRY);
                    arccdbsize += sizeof(struct SG64ENTRY);
                }
                sl++;

The last line doesn't work. Use sl = sg_next(sl); Using scsi_for_each_sg() is even better (as the driver in mainline does). With this bug, 129th sg entry is broken. If your analysis is correct, there should be other bugs..

Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

http://bugzilla.kernel.org/show_bug.cgi?id=15007

--- Comment #4 from nick cheng <[email protected]> 2010-01-14 12:27:11 --- Sorry, is it a really bug?
Does it bug anything? I can't figure it out.
I ever used scsi_for_each_sg() and found the performance is inferior to the original version. I tracked the source code for scsi_for_each_sg()and there is no two much difference to each other. I doubt it is compiler issue to make it worst.
But it is my guess. Do you know why?
Anyway, will you or anybody take care of this issue?
Thank you again,

Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

Yes because if sg entries are larger than 127, they are not continuous. So sg++ doesn't work. We chain sg entries. The last sg entry is a pointer to the next sg list.

For example, a command has 128 entries, it has two sg lists. The first sg list has 127 sg entries. The last entry in the first sg list is a pointer to the second sg list. The second sg list has one sg entry. The command has 128 entries in total.

My previous mail is partially wrong. As I explained above, if sg entries is 128, this bug leads to bogus 128th sg entry. So I think that you hit this bug.

http://bugzilla.kernel.org/show_bug.cgi?id=15007

--- Comment #5 from Anonymous Emailer <[email protected]> 2010-01-14 12:49:54 ---
Reply-To: [email protected]

Yes because if sg entries are larger than 127, they are not continuous. So sg++ doesn't work. We chain sg entries. The last sg entry is a pointer to the next sg list.

For example, a command has 128 entries, it has two sg lists. The first sg list has 127 sg entries. The last entry in the first sg list is a pointer to the second sg list. The second sg list has one sg entry. The command has 128 entries in total.

My previous mail is partially wrong. As I explained above, if sg entries is 128, this bug leads to bogus 128th sg entry. So I think that you hit this bug.

Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

http://bugzilla.kernel.org/show_bug.cgi?id=15007

--- Comment #6 from nick cheng <[email protected]> 2010-01-15 02:49:48 --- OK. I got it.
But based on 128 sg entries issue on my original driver I found it bugs every time while doing mkfs.ext3 on a default NTFS filesystem, but if I do it on a ext3 filesystem, it works smoothly.
Besides, it bugs on SMP platforms always but doesn't on non-SMP plaform. Can your inference explain it?
Thanks so much for your help.

Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

http://bugzilla.kernel.org/show_bug.cgi?id=15007

--- Comment #7 from nick cheng <[email protected]> 2010-01-15 12:23:13 --- Hi,
I got one more question.
What does sg_tablesize in scsi_host_template mean?
It means the max sg entry number which the HW can handle once?
I collect the sg table size from upper layer while I do mkfs.ext3. The maximum number is circa one eighth of sg_tablesize.
Thanks for your reply in advance.

Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

Do you mean that 128 sg entries don't cause any problems during performing mkfs on ext3? If so, I have no idea.

Do you mean that 128 sg entries don't cause any problems on non SMP boxes? If so, I have no idea. Your driver should hit this bug with any boxes.

Yes. A driver should set it to hardware limit.

Can you close this bug report on bugzilla.kernel.org? The bug is not in mainline. If you have any further questions related with this topic, please send to linux-scsi mailing list instead of using the kernel bug tracker.

Thanks,

http://bugzilla.kernel.org/show_bug.cgi?id=15007

--- Comment #8 from Anonymous Emailer <[email protected]> 2010-01-18 04:18:08 ---
Reply-To: [email protected]

Do you mean that 128 sg entries don't cause any problems during performing mkfs on ext3? If so, I have no idea.

Do you mean that 128 sg entries don't cause any problems on non SMP boxes? If so, I have no idea. Your driver should hit this bug with any boxes.

Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

http://bugzilla.kernel.org/show_bug.cgi?id=15007

--- Comment #9 from Anonymous Emailer <[email protected]> 2010-01-18 04:18:16 ---
Reply-To: [email protected]

Yes. A driver should set it to hardware limit.

Can you close this bug report on bugzilla.kernel.org? The bug is not in mainline. If you have any further questions related with this topic, please send to linux-scsi mailing list instead of using the kernel bug tracker.

Thanks,

Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

http://bugzilla.kernel.org/show_bug.cgi?id=15007

nick cheng <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

             Status|NEW                         |RESOLVED
         Resolution|                            |WILL_NOT_FIX
         Regression|No                          |Yes

--- Comment #10 from nick cheng <[email protected]> 2010-01-18 09:41:58 --- (In reply to comment #9)

OK

Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

http://bugzilla.kernel.org/show_bug.cgi?id=15007

--- Comment #11 from nick cheng <[email protected]> 2010-01-18 09:45:32 --- (In reply to comment #8)

Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.