New: SCSI Generic queueing completes commands in reverse order

http://bugzilla.kernel.org/show_bug.cgi?id=15565

           Summary: SCSI Generic queueing completes commands in reverse
                    order
           Product: IO/Storage
           Version: 2.5
    Kernel Version: 2.6.18-2.6.32
          Platform: All
        OS/Version: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: SCSI
        AssignedTo: [email protected]
        ReportedBy: [email protected]
        Regression: No

I've noticed after queueing the first command, subsequent commands appear to be executed and complete in reverse order. The SCSI Generic HOWTO says "By default, read() will return the oldest completed request that is queued up."

This could also be a performance defect if it's what's really happening since it isn't desirable behavior if, for example, ios are typically ordered by lba and issued one at a time by kernel to a non queueing block device.

This reverse order behavior is trivial to reproduce; just queue 16 concurrent INQUIRY commands. The following are typical results I get from initially queueing 16 READ_10 commands.

   Completion Command #  hdr.driver_duration (us)
   Order
   1          22         14979
   2          20         14981
   3          19         14982
   4          17         14984
   5          16         14985
   6          15         14986
   7          14         14988
   8          12         14990
   9          11         14991
   10         10         14992

http://bugzilla.kernel.org/show_bug.cgi?id=15565

           Summary: SCSI Generic queueing completes commands in reverse
                    order
           Product: IO/Storage
           Version: 2.5
    Kernel Version: 2.6.18-2.6.32
          Platform: All
        OS/Version: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: SCSI
        AssignedTo: [email protected]
        ReportedBy: [email protected]
        Regression: No

I've noticed after queueing the first command, subsequent commands appear to be executed and complete in reverse order. The SCSI Generic HOWTO says "By default, read() will return the oldest completed request that is queued up."

This could also be a performance defect if it's what's really happening since it isn't desirable behavior if, for example, ios are typically ordered by lba and issued one at a time by kernel to a non queueing block device.

This reverse order behavior is trivial to reproduce; just queue 16 concurrent INQUIRY commands. The following are typical results I get from initially queueing 16 READ_10 commands.

   Completion Command #  hdr.driver_duration (us)
   Order
   1          22         14979
   2          20         14981
   3          19         14982
   4          17         14984
   5          16         14985
   6          15         14986
   7          14         14988
   8          12         14990
   9          11         14991
   10         10         14992

Quoted 4 lines · [email protected] wrote:

http://bugzilla.kernel.org/show_bug.cgi?id=15565

Summary: SCSI Generic queueing completes commands in reverse
order

I have been told that is a feature :-)

The SCSI mid level processes commands from pass-throughs (e.g. sg and bsg) in LIFO order. For certain types of error processing it makes sense. For READs and WRITEs it makes no sense.

Quoted 33 lines · [email protected] wrote:

http://bugzilla.kernel.org/show_bug.cgi?id=15565

           Summary: SCSI Generic queueing completes commands in reverse
                    order
           Product: IO/Storage
           Version: 2.5
    Kernel Version: 2.6.18-2.6.32
          Platform: All
        OS/Version: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: SCSI
        AssignedTo: [email protected]
        ReportedBy: [email protected]
        Regression: No

I've noticed after queueing the first command, subsequent commands appear to be executed and complete in reverse order. The SCSI Generic HOWTO says "By default, read() will return the oldest completed request that is queued up."

This could also be a performance defect if it's what's really happening since it isn't desirable behavior if, for example, ios are typically ordered by lba and issued one at a time by kernel to a non queueing block device.

This reverse order behavior is trivial to reproduce; just queue 16 concurrent INQUIRY commands. The following are typical results I get from initially queueing 16 READ_10 commands.

Quoted 13 lines

   Completion Command #  hdr.driver_duration (us)
   Order
   1          22         14979
   2          20         14981
   3          19         14982
   4          17         14984
   5          16         14985
   6          15         14986
   7          14         14988
   8          12         14990
   9          11         14991
   10         10         14992

I have been told that is a feature :-)

The SCSI mid level processes commands from pass-throughs (e.g. sg and bsg) in LIFO order. For certain types of error processing it makes sense. For READs and WRITEs it makes no sense.

Doug Gilbert

Quoted 33 lines · [email protected] wrote:

http://bugzilla.kernel.org/show_bug.cgi?id=15565

           Summary: SCSI Generic queueing completes commands in reverse
                    order
           Product: IO/Storage
           Version: 2.5
    Kernel Version: 2.6.18-2.6.32
          Platform: All
        OS/Version: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: SCSI
        AssignedTo: [email protected]
        ReportedBy: [email protected]
        Regression: No

I've noticed after queueing the first command, subsequent commands appear to be executed and complete in reverse order. The SCSI Generic HOWTO says "By default, read() will return the oldest completed request that is queued up."

This could also be a performance defect if it's what's really happening since it isn't desirable behavior if, for example, ios are typically ordered by lba and issued one at a time by kernel to a non queueing block device.

This reverse order behavior is trivial to reproduce; just queue 16 concurrent INQUIRY commands. The following are typical results I get from initially queueing 16 READ_10 commands.

Quoted 13 lines

   Completion Command #  hdr.driver_duration (us)
   Order
   1          22         14979
   2          20         14981
   3          19         14982
   4          17         14984
   5          16         14985
   6          15         14986
   7          14         14988
   8          12         14990
   9          11         14991
   10         10         14992

I have been told that is a feature :-)

The SCSI mid level processes commands from pass-throughs (e.g. sg and bsg) in LIFO order. For certain types of error processing it makes sense. For READs and WRITEs it makes no sense.

Doug Gilbert

I have fixed that for bsg long ago. There is a flag that you put:

    sg.flags = BSG_FLAG_Q_AT_TAIL;

Which will do the by order, queue at tail. Zero was kept compatible to sg meaning "queue at head" i.e. jump the line.

It could be added to sg as well if needed

http://bugzilla.kernel.org/show_bug.cgi?id=15565

--- Comment #4 from Mike Hayward <[email protected]> 2010-03-18 17:21:05 --- Hi Boaz,

Quoted 6 lines

I have fixed that for bsg long ago. There is a flag that you put:

    sg.flags = BSG_FLAG_Q_AT_TAIL;

Which will do the by order, queue at tail. Zero was kept compatible to sg meaning "queue at head" i.e. jump the line.

Quoted 1 line

It could be added to sg as well if needed

I think that would be a huge improvement for disk drives.

Isn't BSG driven with a blocking ioctl? Maybe I'm using old docs, but from the HOWTO's I've read, I was under the impression it would issue one SCSI command at a time and block until it completed. Queueing would effectively only be possible then with multiple threads, right?

Since sg allows multiple (only 16 right now) ios to be queued nonblocking to a block device from a single threaded process, an SG_FLAG_Q_AT_TAIL flag would seem particularly relevant.

Nonblocking behavior is the primary reason why I am using sg... it seems really inefficient and complex to use dozens of threads and ipc just to try to submit a queue of ios to the kernel and hence a device.

If bsg actually allowed tasks to be queued nonblocking like sg, I'd switch in a heart beat since having two filesystem names for the same device introduces locking problems, name translation complexity, and confusion in general for administrators.

- Mike

http://bugzilla.kernel.org/show_bug.cgi?id=15565

--- Comment #4 from Mike Hayward <[email protected]> 2010-03-18 17:21:05 --- Hi Boaz,

Quoted 6 lines

I have fixed that for bsg long ago. There is a flag that you put:

    sg.flags = BSG_FLAG_Q_AT_TAIL;

Which will do the by order, queue at tail. Zero was kept compatible to sg meaning "queue at head" i.e. jump the line.

Quoted 1 line

It could be added to sg as well if needed

I think that would be a huge improvement for disk drives.

Isn't BSG driven with a blocking ioctl? Maybe I'm using old docs, but from the HOWTO's I've read, I was under the impression it would issue one SCSI command at a time and block until it completed. Queueing would effectively only be possible then with multiple threads, right?

Since sg allows multiple (only 16 right now) ios to be queued nonblocking to a block device from a single threaded process, an SG_FLAG_Q_AT_TAIL flag would seem particularly relevant.

Nonblocking behavior is the primary reason why I am using sg... it seems really inefficient and complex to use dozens of threads and ipc just to try to submit a queue of ios to the kernel and hence a device.

If bsg actually allowed tasks to be queued nonblocking like sg, I'd switch in a heart beat since having two filesystem names for the same device introduces locking problems, name translation complexity, and confusion in general for administrators.

- Mike

Please switch to mail ?

Quoted 15 lines · On 03/18/2010 07:21 PM, [email protected] wrote:

http://bugzilla.kernel.org/show_bug.cgi?id=15565

--- Comment #4 from Mike Hayward <[email protected]> 2010-03-18 17:21:05 --- Hi Boaz,

Quoted 6 lines

I have fixed that for bsg long ago. There is a flag that you put:

    sg.flags = BSG_FLAG_Q_AT_TAIL;

Which will do the by order, queue at tail. Zero was kept compatible to sg meaning "queue at head" i.e. jump the line.

Quoted 9 lines

Quoted 1 line

It could be added to sg as well if needed

I think that would be a huge improvement for disk drives.

Isn't BSG driven with a blocking ioctl? Maybe I'm using old docs, but from the HOWTO's I've read, I was under the impression it would issue one SCSI command at a time and block until it completed. Queueing would effectively only be possible then with multiple threads, right?

There is a way to do write with multiple ios at once and then do reads to get return status. (I have untested code that does that but never needed it so badly as to finish it. But I know of other people that did it)

bsg does not have all these mmap/shared buffers options like sg does, though. Hell not even AIOs.

Quoted 4 lines

Since sg allows multiple (only 16 right now) ios to be queued nonblocking to a block device from a single threaded process, an SG_FLAG_Q_AT_TAIL flag would seem particularly relevant.

Yes. Currently the queue of 16 does nothing because in effect the Kernel will completely drain the Q on each "Q_AT_HEAD". So the Q does nothing for IO with current sg.

Quoted 4 lines

Nonblocking behavior is the primary reason why I am using sg... it seems really inefficient and complex to use dozens of threads and ipc just to try to submit a queue of ios to the kernel and hence a device.

There are surely other ways to do AIO, no?
(The A in AIO stands for Asynchronous)

Quoted 5 lines

If bsg actually allowed tasks to be queued nonblocking like sg, I'd switch in a heart beat since having two filesystem names for the same device introduces locking problems, name translation complexity, and confusion in general for administrators.

Well bsg has it's own naming translation. You do not use /dev/sdX1. You need to open the bsg char-device corresponding the particular scsi-device

Please switch to mail ?

Quoted 15 lines · On 03/18/2010 07:21 PM, [email protected] wrote:

http://bugzilla.kernel.org/show_bug.cgi?id=15565

--- Comment #4 from Mike Hayward <[email protected]> 2010-03-18 17:21:05 --- Hi Boaz,

Quoted 6 lines

I have fixed that for bsg long ago. There is a flag that you put:

    sg.flags = BSG_FLAG_Q_AT_TAIL;

Which will do the by order, queue at tail. Zero was kept compatible to sg meaning "queue at head" i.e. jump the line.

Quoted 9 lines

Quoted 1 line

It could be added to sg as well if needed

I think that would be a huge improvement for disk drives.

Isn't BSG driven with a blocking ioctl? Maybe I'm using old docs, but from the HOWTO's I've read, I was under the impression it would issue one SCSI command at a time and block until it completed. Queueing would effectively only be possible then with multiple threads, right?

There is a way to do write with multiple ios at once and then do reads to get return status. (I have untested code that does that but never needed it so badly as to finish it. But I know of other people that did it)

bsg does not have all these mmap/shared buffers options like sg does, though. Hell not even AIOs.

Quoted 4 lines

Since sg allows multiple (only 16 right now) ios to be queued nonblocking to a block device from a single threaded process, an SG_FLAG_Q_AT_TAIL flag would seem particularly relevant.

Yes. Currently the queue of 16 does nothing because in effect the Kernel will completely drain the Q on each "Q_AT_HEAD". So the Q does nothing for IO with current sg.

Quoted 4 lines

Nonblocking behavior is the primary reason why I am using sg... it seems really inefficient and complex to use dozens of threads and ipc just to try to submit a queue of ios to the kernel and hence a device.

There are surely other ways to do AIO, no?
(The A in AIO stands for Asynchronous)

Quoted 5 lines

If bsg actually allowed tasks to be queued nonblocking like sg, I'd switch in a heart beat since having two filesystem names for the same device introduces locking problems, name translation complexity, and confusion in general for administrators.

Well bsg has it's own naming translation. You do not use /dev/sdX1. You need to open the bsg char-device corresponding the particular scsi-device

Quoted 40 lines

Quoted 8 lines

Hi Boaz,

Quoted 6 lines

I have fixed that for bsg long ago. There is a flag that you put:

    sg.flags = BSG_FLAG_Q_AT_TAIL;

Which will do the by order, queue at tail. Zero was kept compatible to sg meaning "queue at head" i.e. jump the line.

Quoted 9 lines

Quoted 1 line

It could be added to sg as well if needed

I think that would be a huge improvement for disk drives.

Isn't BSG driven with a blocking ioctl? Maybe I'm using old docs, but from the HOWTO's I've read, I was under the impression it would issue one SCSI command at a time and block until it completed. Queueing would effectively only be possible then with multiple threads, right?

There is a way to do write with multiple ios at once and then do reads to get return status. (I have untested code that does that but never needed it so badly as to finish it. But I know of other people that did it)

bsg does not have all these mmap/shared buffers options like sg does, though. Hell not even AIOs.

Quoted 4 lines

Since sg allows multiple (only 16 right now) ios to be queued nonblocking to a block device from a single threaded process, an SG_FLAG_Q_AT_TAIL flag would seem particularly relevant.

Yes. Currently the queue of 16 does nothing because in effect the Kernel will completely drain the Q on each "Q_AT_HEAD". So the Q does nothing for IO with current sg.

Quoted 4 lines

Nonblocking behavior is the primary reason why I am using sg... it seems really inefficient and complex to use dozens of threads and ipc just to try to submit a queue of ios to the kernel and hence a device.

There are surely other ways to do AIO, no?
(The A in AIO stands for Asynchronous)

Quoted 7 lines

Quoted 5 lines

If bsg actually allowed tasks to be queued nonblocking like sg, I'd switch in a heart beat since having two filesystem names for the same device introduces locking problems, name translation complexity, and confusion in general for administrators.

Well bsg has it's own naming translation. You do not use /dev/sdX1. You need to open the bsg char-device corresponding the particular scsi-device

What might be useful is to tell the sd driver not to hook a particular logical unit with peripheral device type 0 (i.e. a disk). Perhaps this could be done via the LU UUID. That way udev (?) would be less tempted to send crap commands to the device at random times.

After that you only have to worry about the device node duality between bsg and sg :-)
They are pretty well behaved, only sending commands when they are asked to.

Adding a SG_FLAG_Q_AT_TAIL flag to sg seems like a good idea.

Quoted 40 lines

Quoted 8 lines

Hi Boaz,

Quoted 6 lines

I have fixed that for bsg long ago. There is a flag that you put:

    sg.flags = BSG_FLAG_Q_AT_TAIL;

Which will do the by order, queue at tail. Zero was kept compatible to sg meaning "queue at head" i.e. jump the line.

Quoted 9 lines

Quoted 1 line

It could be added to sg as well if needed

I think that would be a huge improvement for disk drives.

Isn't BSG driven with a blocking ioctl? Maybe I'm using old docs, but from the HOWTO's I've read, I was under the impression it would issue one SCSI command at a time and block until it completed. Queueing would effectively only be possible then with multiple threads, right?

There is a way to do write with multiple ios at once and then do reads to get return status. (I have untested code that does that but never needed it so badly as to finish it. But I know of other people that did it)

bsg does not have all these mmap/shared buffers options like sg does, though. Hell not even AIOs.

Quoted 4 lines

Since sg allows multiple (only 16 right now) ios to be queued nonblocking to a block device from a single threaded process, an SG_FLAG_Q_AT_TAIL flag would seem particularly relevant.

Yes. Currently the queue of 16 does nothing because in effect the Kernel will completely drain the Q on each "Q_AT_HEAD". So the Q does nothing for IO with current sg.

Quoted 4 lines

Nonblocking behavior is the primary reason why I am using sg... it seems really inefficient and complex to use dozens of threads and ipc just to try to submit a queue of ios to the kernel and hence a device.

There are surely other ways to do AIO, no?
(The A in AIO stands for Asynchronous)

Quoted 7 lines

Quoted 5 lines

If bsg actually allowed tasks to be queued nonblocking like sg, I'd switch in a heart beat since having two filesystem names for the same device introduces locking problems, name translation complexity, and confusion in general for administrators.

Well bsg has it's own naming translation. You do not use /dev/sdX1. You need to open the bsg char-device corresponding the particular scsi-device

What might be useful is to tell the sd driver not to hook a particular logical unit with peripheral device type 0 (i.e. a disk). Perhaps this could be done via the LU UUID. That way udev (?) would be less tempted to send crap commands to the device at random times.

After that you only have to worry about the device node duality between bsg and sg :-)
They are pretty well behaved, only sending commands when they are asked to.

Adding a SG_FLAG_Q_AT_TAIL flag to sg seems like a good idea.

Quoted 53 lines · Douglas Gilbert, on 03/18/2010 10:30 PM wrote:

Quoted 40 lines

Quoted 8 lines

Hi Boaz,

Quoted 6 lines

I have fixed that for bsg long ago. There is a flag that you put:

    sg.flags = BSG_FLAG_Q_AT_TAIL;

Which will do the by order, queue at tail. Zero was kept compatible to sg meaning "queue at head" i.e. jump the line.

Quoted 9 lines

Quoted 1 line

It could be added to sg as well if needed

I think that would be a huge improvement for disk drives.

Isn't BSG driven with a blocking ioctl? Maybe I'm using old docs, but from the HOWTO's I've read, I was under the impression it would issue one SCSI command at a time and block until it completed. Queueing would effectively only be possible then with multiple threads, right?

There is a way to do write with multiple ios at once and then do reads to get return status. (I have untested code that does that but never needed it so badly as to finish it. But I know of other people that did it)

bsg does not have all these mmap/shared buffers options like sg does, though. Hell not even AIOs.

Quoted 4 lines

Since sg allows multiple (only 16 right now) ios to be queued nonblocking to a block device from a single threaded process, an SG_FLAG_Q_AT_TAIL flag would seem particularly relevant.

Yes. Currently the queue of 16 does nothing because in effect the Kernel will completely drain the Q on each "Q_AT_HEAD". So the Q does nothing for IO with current sg.

Quoted 4 lines

Nonblocking behavior is the primary reason why I am using sg... it seems really inefficient and complex to use dozens of threads and ipc just to try to submit a queue of ios to the kernel and hence a device.

There are surely other ways to do AIO, no?
(The A in AIO stands for Asynchronous)

Quoted 7 lines

Quoted 5 lines

If bsg actually allowed tasks to be queued nonblocking like sg, I'd switch in a heart beat since having two filesystem names for the same device introduces locking problems, name translation complexity, and confusion in general for administrators.

Well bsg has it's own naming translation. You do not use /dev/sdX1. You need to open the bsg char-device corresponding the particular scsi-device

What might be useful is to tell the sd driver not to hook a particular logical unit with peripheral device type 0 (i.e. a disk). Perhaps this could be done via the LU UUID. That way udev (?) would be less tempted to send crap commands to the device at random times.

Quoted 8 lines

After that you only have to worry about the device node duality between bsg and sg :-)
They are pretty well behaved, only sending commands when they are asked to.

Adding a SG_FLAG_Q_AT_TAIL flag to sg seems like a good idea.

Great so see the problem confirmed and it's going to move! I and other people have been asking this for 3 (or 4?) years and have had a silence in reply. It hasn't even been confirmed, so I'd prepare a patch.

Eventually, this should be fixed.

Vlad

Quoted 53 lines · Douglas Gilbert, on 03/18/2010 10:30 PM wrote:

Quoted 40 lines

Quoted 8 lines

Hi Boaz,

Quoted 6 lines

I have fixed that for bsg long ago. There is a flag that you put:

    sg.flags = BSG_FLAG_Q_AT_TAIL;

Which will do the by order, queue at tail. Zero was kept compatible to sg meaning "queue at head" i.e. jump the line.

Quoted 9 lines

Quoted 1 line

It could be added to sg as well if needed

I think that would be a huge improvement for disk drives.

Isn't BSG driven with a blocking ioctl? Maybe I'm using old docs, but from the HOWTO's I've read, I was under the impression it would issue one SCSI command at a time and block until it completed. Queueing would effectively only be possible then with multiple threads, right?

There is a way to do write with multiple ios at once and then do reads to get return status. (I have untested code that does that but never needed it so badly as to finish it. But I know of other people that did it)

bsg does not have all these mmap/shared buffers options like sg does, though. Hell not even AIOs.

Quoted 4 lines

Since sg allows multiple (only 16 right now) ios to be queued nonblocking to a block device from a single threaded process, an SG_FLAG_Q_AT_TAIL flag would seem particularly relevant.

Yes. Currently the queue of 16 does nothing because in effect the Kernel will completely drain the Q on each "Q_AT_HEAD". So the Q does nothing for IO with current sg.

Quoted 4 lines

Nonblocking behavior is the primary reason why I am using sg... it seems really inefficient and complex to use dozens of threads and ipc just to try to submit a queue of ios to the kernel and hence a device.

There are surely other ways to do AIO, no?
(The A in AIO stands for Asynchronous)

Quoted 7 lines

Quoted 5 lines

If bsg actually allowed tasks to be queued nonblocking like sg, I'd switch in a heart beat since having two filesystem names for the same device introduces locking problems, name translation complexity, and confusion in general for administrators.

Well bsg has it's own naming translation. You do not use /dev/sdX1. You need to open the bsg char-device corresponding the particular scsi-device

What might be useful is to tell the sd driver not to hook a particular logical unit with peripheral device type 0 (i.e. a disk). Perhaps this could be done via the LU UUID. That way udev (?) would be less tempted to send crap commands to the device at random times.

Quoted 8 lines

After that you only have to worry about the device node duality between bsg and sg :-)
They are pretty well behaved, only sending commands when they are asked to.

Adding a SG_FLAG_Q_AT_TAIL flag to sg seems like a good idea.

Great so see the problem confirmed and it's going to move! I and other people have been asking this for 3 (or 4?) years and have had a silence in reply. It hasn't even been confirmed, so I'd prepare a patch.

Eventually, this should be fixed.

Vlad

Great so see the problem confirmed and it's going to move! I and other people have been asking this for 3 (or 4?) years and have had a silence in reply. It hasn't even been confirmed, so I'd prepare a patch.

Quoted 63 lines · Vladislav Bolkhovitin

Quoted 53 lines · Douglas Gilbert, on 03/18/2010 10:30 PM wrote:

Quoted 40 lines

Quoted 8 lines

Hi Boaz,

Quoted 6 lines

I have fixed that for bsg long ago. There is a flag that you put:

    sg.flags = BSG_FLAG_Q_AT_TAIL;

Which will do the by order, queue at tail. Zero was kept compatible to sg meaning "queue at head" i.e. jump the line.

Quoted 9 lines

Quoted 1 line

It could be added to sg as well if needed

I think that would be a huge improvement for disk drives.

Isn't BSG driven with a blocking ioctl? Maybe I'm using old docs, but from the HOWTO's I've read, I was under the impression it would issue one SCSI command at a time and block until it completed. Queueing would effectively only be possible then with multiple threads, right?

There is a way to do write with multiple ios at once and then do reads to get return status. (I have untested code that does that but never needed it so badly as to finish it. But I know of other people that did it)

bsg does not have all these mmap/shared buffers options like sg does, though. Hell not even AIOs.

Quoted 4 lines

Since sg allows multiple (only 16 right now) ios to be queued nonblocking to a block device from a single threaded process, an SG_FLAG_Q_AT_TAIL flag would seem particularly relevant.

Yes. Currently the queue of 16 does nothing because in effect the Kernel will completely drain the Q on each "Q_AT_HEAD". So the Q does nothing for IO with current sg.

Quoted 4 lines

Nonblocking behavior is the primary reason why I am using sg... it seems really inefficient and complex to use dozens of threads and ipc just to try to submit a queue of ios to the kernel and hence a device.

There are surely other ways to do AIO, no?
(The A in AIO stands for Asynchronous)

Quoted 7 lines

Quoted 5 lines

If bsg actually allowed tasks to be queued nonblocking like sg, I'd switch in a heart beat since having two filesystem names for the same device introduces locking problems, name translation complexity, and confusion in general for administrators.

Well bsg has it's own naming translation. You do not use /dev/sdX1. You need to open the bsg char-device corresponding the particular scsi-device

What might be useful is to tell the sd driver not to hook a particular logical unit with peripheral device type 0 (i.e. a disk). Perhaps this could be done via the LU UUID. That way udev (?) would be less tempted to send crap commands to the device at random times.

Quoted 8 lines

After that you only have to worry about the device node duality between bsg and sg :-)
They are pretty well behaved, only sending commands when they are asked to.

Adding a SG_FLAG_Q_AT_TAIL flag to sg seems like a good idea.

Quoted 7 lines

Great so see the problem confirmed and it's going to move! I and other people have been asking this for 3 (or 4?) years and have had a silence in reply. It hasn't even been confirmed, so I'd prepare a patch.

Eventually, this should be fixed.

Vlad

We're looking at supporting a non-disk scsi device which can effectively process a queue depth of nine in parallel. Having our application access the hardware via the "sg" device is our first choice on how to proceed.

My reading of this thread indicates that the 9 parallel commands we'll issue will be effectively single threaded by the kernel due to their being queued at the head of the queue.

I'd like to encourage the implementation of SG_FLAG_Q_AT_TAIL for sg devices.

Mike