Skip to contentSkip to conversation

SCSI io_request_lock patch

DISCUSSION Jonathan Lahr 2 participants 7 messages Nov 12, 2001 – Nov 15, 2001

lore
Share & tools
lore
#0 lore raw

This is a request for comments on the patch described below which implements a revised approach to reducing io_request_lock contention in 2.4.

This new version of the io_request_lock patch (siorl-v0) is available at http://sourceforge.net/projects/lse/. It employs the same concurrent request queueing scheme as the iorlv0 patch but isolates code changes to the SCSI subsystem and engages the new locking scheme only for SCSI drivers which explicitly request it. I took this more restricted approach after additional development based on comments from Jens and others indicated that iorlv0 impacted the IDE subsystem and was unnecessarily broad in general.

The siorl-v0 patch allows drivers to enable concurrent queueing through the concurrent_queue field in the Scsi_Host_Template which is copied to the request queue. It creates SCSI-specific versions of generic block i/o functions used by the SCSI subsystem and modifies them to conditionally engage the new locking scheme based on this field. It allows control over which drivers use concurrent queueing and preserves original block i/o behavior by default.

I tested this patch with aic7xxx and lpfc drivers, and regression tested it with IDE disk and CDROM drivers. Any feedback would be appreciated.

Thanks,
Jonathan

Signature · 5 lines
Jonathan Lahr
IBM Linux Technology Center
Beaverton, Oregon
[email protected]
503-578-3385
#1 lore raw
Quoted 14 lines · Jonathan Lahr

This is a request for comments on the patch described below which implements a revised approach to reducing io_request_lock contention in 2.4.

This new version of the io_request_lock patch (siorl-v0) is available at http://sourceforge.net/projects/lse/. It employs the same concurrent request queueing scheme as the iorlv0 patch but isolates code changes to the SCSI subsystem and engages the new locking scheme only for SCSI drivers which explicitly request it. I took this more restricted approach after additional development based on comments from Jens and others indicated that iorlv0 impacted the IDE subsystem and was unnecessarily broad in general.

Quoted 7 lines

The siorl-v0 patch allows drivers to enable concurrent queueing through the concurrent_queue field in the Scsi_Host_Template which is copied to the request queue. It creates SCSI-specific versions of generic block i/o functions used by the SCSI subsystem and modifies them to conditionally engage the new locking scheme based on this field. It allows control over which drivers use concurrent queueing and preserves original block i/o behavior by default.

Sorry Jonathan, but this is even more broken than the last patch. In different ways. In no particular order:

o You are duplicating way too much code and exporting block internals
o You are breaking SCSI merge completely, why on earth are you suddenly using ll_*_merge functions for SCSI?!
o scsi_make_request need not worry about head active
o scsi_make_request can safe the q->*_merge indirect
o scsi_dispatch_cmd() io_request_lock removal looks racy

At least you are not breaking anything other than SCSI this time...

Signature · 1 line
Jens Axboe
#2 lore raw
In reply to Jonathan Lahr
Quoted 23 lines · Jens Axboe [[email protected]] wrote:
Quoted 14 lines · Jonathan Lahr

This is a request for comments on the patch described below which implements a revised approach to reducing io_request_lock contention in 2.4.

This new version of the io_request_lock patch (siorl-v0) is available at http://sourceforge.net/projects/lse/. It employs the same concurrent request queueing scheme as the iorlv0 patch but isolates code changes to the SCSI subsystem and engages the new locking scheme only for SCSI drivers which explicitly request it. I took this more restricted approach after additional development based on comments from Jens and others indicated that iorlv0 impacted the IDE subsystem and was unnecessarily broad in general.

Quoted 7 lines

The siorl-v0 patch allows drivers to enable concurrent queueing through the concurrent_queue field in the Scsi_Host_Template which is copied to the request queue. It creates SCSI-specific versions of generic block i/o functions used by the SCSI subsystem and modifies them to conditionally engage the new locking scheme based on this field. It allows control over which drivers use concurrent queueing and preserves original block i/o behavior by default.

Quoted 4 lines

Sorry Jonathan, but this is even more broken than the last patch. In different ways. In no particular order:

o You are duplicating way too much code and exporting block internals

The duplication is a reasonable starting point for SCSI-specific functions. The block i/o design provides for exactly this type of tailoring through function pointers installed in request_queue.

What problem you do see with exporting block internals?

Quoted 5 lines

o You are breaking SCSI merge completely, why on earth are you suddenly using ll_*_merge functions for SCSI?!
o scsi_make_request need not worry about head active
o scsi_make_request can safe the q->*_merge indirect
o scsi_dispatch_cmd() io_request_lock removal looks racy

I will investigate the above comments further.

Quoted 1 line

At least you are not breaking anything other than SCSI this time...

Do you think the separation of SCSI from generic block i/o code and the driver-activated control of concurrent queueing provides a path for future work to reduce io_request_lock contention in SCSI/FC?

Signature · 5 lines
Jonathan Lahr
IBM Linux Technology Center
Beaverton, Oregon
[email protected]
503-578-3385
#3 lore raw
In reply to Jonathan Lahr
Quoted 28 lines · Jonathan Lahr
Quoted 27 lines · Jens Axboe [[email protected]] wrote:
Quoted 14 lines · Jonathan Lahr

This is a request for comments on the patch described below which implements a revised approach to reducing io_request_lock contention in 2.4.

This new version of the io_request_lock patch (siorl-v0) is available at http://sourceforge.net/projects/lse/. It employs the same concurrent request queueing scheme as the iorlv0 patch but isolates code changes to the SCSI subsystem and engages the new locking scheme only for SCSI drivers which explicitly request it. I took this more restricted approach after additional development based on comments from Jens and others indicated that iorlv0 impacted the IDE subsystem and was unnecessarily broad in general.

Quoted 8 lines

The siorl-v0 patch allows drivers to enable concurrent queueing through the concurrent_queue field in the Scsi_Host_Template which is copied to the request queue. It creates SCSI-specific versions of generic block i/o functions used by the SCSI subsystem and modifies them to conditionally engage the new locking scheme based on this field. It allows control over which drivers use concurrent queueing and preserves original block i/o behavior by default.

Sorry Jonathan, but this is even more broken than the last patch. In different ways. In no particular order:

Quoted 6 lines
Quoted 2 lines

o You are duplicating way too much code and exporting block internals

The duplication is a reasonable starting point for SCSI-specific functions. The block i/o design provides for exactly this type of tailoring through function pointers installed in request_queue.

Yes I know, I wrote most of said code :-)

Quoted 1 line

What problem you do see with exporting block internals?

It's absolutely worthless. Look, it ties in with the points I made below. You are exporting the merge functions for instance, and setting them in the queue. This will cause scsi_merge not to use it's own functions, broken.

The make_request_fn addition could be ok, just needs to be cleaned a bit.

Quoted 6 lines
Quoted 5 lines

o You are breaking SCSI merge completely, why on earth are you suddenly using ll_*_merge functions for SCSI?! o scsi_make_request need not worry about head active o scsi_make_request can safe the q->*_merge indirect o scsi_dispatch_cmd() io_request_lock removal looks racy

Quoted 7 lines

I will investigate the above comments further.

Quoted 1 line

At least you are not breaking anything other than SCSI this time...

Do you think the separation of SCSI from generic block i/o code and the driver-activated control of concurrent queueing provides a path for future work to reduce io_request_lock contention in SCSI/FC?

Not really, but I do think it could be a viable 2.4 alternative. For 2.5 we still want to do this the right way.

Signature · 1 line
Jens Axboe
#4 lore raw
In reply to Jonathan Lahr
Quoted 29 lines · Jens Axboe [[email protected]] wrote:
Quoted 28 lines · Jonathan Lahr
Quoted 27 lines · Jens Axboe [[email protected]] wrote:
Quoted 14 lines · Jonathan Lahr

This is a request for comments on the patch described below which implements a revised approach to reducing io_request_lock contention in 2.4.

This new version of the io_request_lock patch (siorl-v0) is available at http://sourceforge.net/projects/lse/. It employs the same concurrent request queueing scheme as the iorlv0 patch but isolates code changes to the SCSI subsystem and engages the new locking scheme only for SCSI drivers which explicitly request it. I took this more restricted approach after additional development based on comments from Jens and others indicated that iorlv0 impacted the IDE subsystem and was unnecessarily broad in general.

Quoted 8 lines

The siorl-v0 patch allows drivers to enable concurrent queueing through the concurrent_queue field in the Scsi_Host_Template which is copied to the request queue. It creates SCSI-specific versions of generic block i/o functions used by the SCSI subsystem and modifies them to conditionally engage the new locking scheme based on this field. It allows control over which drivers use concurrent queueing and preserves original block i/o behavior by default.

Sorry Jonathan, but this is even more broken than the last patch. In different ways. In no particular order:

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

o You are duplicating way too much code and exporting block internals

The duplication is a reasonable starting point for SCSI-specific functions. The block i/o design provides for exactly this type of tailoring through function pointers installed in request_queue.

Yes I know, I wrote most of said code :-)

And this approach makes good use of it.

Quoted 6 lines
Quoted 1 line

What problem you do see with exporting block internals?

It's absolutely worthless. Look, it ties in with the points I made below. You are exporting the merge functions for instance, and setting them in the queue. This will cause scsi_merge not to use it's own functions, broken.

As in the baseline, initialize_merge_fn overwrites these pointers:

     q->back_merge_fn = scsi_back_merge_fn_;
     q->front_merge_fn = scsi_front_merge_fn_;
     q->merge_requests_fn = scsi_merge_requests_fn_;
Quoted 6 lines
Quoted 3 lines

Do you think the separation of SCSI from generic block i/o code and the driver-activated control of concurrent queueing provides a path for future work to reduce io_request_lock contention in SCSI/FC?

Not really, but I do think it could be a viable 2.4 alternative. For 2.5 we still want to do this the right way.

I'll try to stay apprised of the 2.5 work as it progresses.

Signature · 5 lines
Jonathan Lahr
IBM Linux Technology Center
Beaverton, Oregon
[email protected]
503-578-3385
#5 lore raw
In reply to Jonathan Lahr
Quoted 9 lines · Jonathan Lahr
Quoted 4 lines

It's absolutely worthless. Look, it ties in with the points I made below. You are exporting the merge functions for instance, and setting them in the queue. This will cause scsi_merge not to use it's own functions, broken.

As in the baseline, initialize_merge_fn overwrites these pointers:

     q->back_merge_fn = scsi_back_merge_fn_;
     q->front_merge_fn = scsi_front_merge_fn_;
     q->merge_requests_fn = scsi_merge_requests_fn_;

I had forgotten I had #if 0 out the check for already set back_merge etc in scsi_merge -- however that's still beside the point. _Why_ are you exporting the ll_rw_blk functions and setting them just to have them overridden? Makes no sense.

Don't export the merge functions ever, define your own if you really need them. You don't, though.

As I've mentioned before, go ahead with the make_request_fn replacement. That is indeed what it is there for.

Signature · 1 line
Jens Axboe
#6 lore raw
In reply to Jonathan Lahr
Quoted 15 lines · Jens Axboe [[email protected]] wrote:
Quoted 9 lines · Jonathan Lahr
Quoted 4 lines

It's absolutely worthless. Look, it ties in with the points I made below. You are exporting the merge functions for instance, and setting them in the queue. This will cause scsi_merge not to use it's own functions, broken.

As in the baseline, initialize_merge_fn overwrites these pointers:

     q->back_merge_fn = scsi_back_merge_fn_;
     q->front_merge_fn = scsi_front_merge_fn_;
     q->merge_requests_fn = scsi_merge_requests_fn_;

I had forgotten I had #if 0 out the check for already set back_merge etc in scsi_merge -- however that's still beside the point. _Why_ are you exporting the ll_rw_blk functions and setting them just to have them overridden? Makes no sense.

...

Quoted 2 lines

Don't export the merge functions ever, define your own if you really need them. You don't, though.

That resulted merely from basing scsi_init_queue on blk_init_queue. I'll simplify the code by removing these unnecessary assignments.

Signature · 5 lines
Jonathan Lahr
IBM Linux Technology Center
Beaverton, Oregon
[email protected]
503-578-3385