SCSI io_request_lock patch
This is a request for comments on the patch described below which implements a revised approach to reducing io_request_lock contention in 2.4.
This new version of the io_request_lock patch (siorl-v0) is available at http://sourceforge.net/projects/lse/. It employs the same concurrent request queueing scheme as the iorlv0 patch but isolates code changes to the SCSI subsystem and engages the new locking scheme only for SCSI drivers which explicitly request it. I took this more restricted approach after additional development based on comments from Jens and others indicated that iorlv0 impacted the IDE subsystem and was unnecessarily broad in general.
The siorl-v0 patch allows drivers to enable concurrent queueing through the concurrent_queue field in the Scsi_Host_Template which is copied to the request queue. It creates SCSI-specific versions of generic block i/o functions used by the SCSI subsystem and modifies them to conditionally engage the new locking scheme based on this field. It allows control over which drivers use concurrent queueing and preserves original block i/o behavior by default.
I tested this patch with aic7xxx and lpfc drivers, and regression tested it with IDE disk and CDROM drivers. Any feedback would be appreciated.
Thanks,
Jonathan
Signature · 5 lines
Jonathan Lahr IBM Linux Technology Center Beaverton, Oregon [email protected] 503-578-3385
Quoted 14 lines · Jonathan Lahr
This is a request for comments on the patch described below which implements a revised approach to reducing
io_request_lockcontention in 2.4.This new version of the
io_request_lockpatch (siorl-v0) is available at http://sourceforge.net/projects/lse/. It employs the same concurrent request queueing scheme as the iorlv0 patch but isolates code changes to the SCSI subsystem and engages the new locking scheme only for SCSI drivers which explicitly request it. I took this more restricted approach after additional development based on comments from Jens and others indicated that iorlv0 impacted the IDE subsystem and was unnecessarily broad in general.
Quoted 7 lines
The siorl-v0 patch allows drivers to enable concurrent queueing through the
concurrent_queuefield in theScsi_Host_Templatewhich is copied to the request queue. It creates SCSI-specific versions of generic block i/o functions used by the SCSI subsystem and modifies them to conditionally engage the new locking scheme based on this field. It allows control over which drivers use concurrent queueing and preserves original block i/o behavior by default.
Sorry Jonathan, but this is even more broken than the last patch. In different ways. In no particular order:
o You are duplicating way too much code and exporting block internals
o You are breaking SCSI merge completely, why on earth are you suddenly using ll_*_merge functions for SCSI?!
o scsi_make_request need not worry about head active
o scsi_make_request can safe the q->*_merge indirect
o scsi_dispatch_cmd() io_request_lock removal looks racy
At least you are not breaking anything other than SCSI this time...
Signature · 1 line
Jens Axboe
Quoted 23 lines · Jens Axboe [[email protected]] wrote:
Quoted 14 lines · Jonathan Lahr
This is a request for comments on the patch described below which implements a revised approach to reducing
io_request_lockcontention in 2.4.This new version of the
io_request_lockpatch (siorl-v0) is available at http://sourceforge.net/projects/lse/. It employs the same concurrent request queueing scheme as the iorlv0 patch but isolates code changes to the SCSI subsystem and engages the new locking scheme only for SCSI drivers which explicitly request it. I took this more restricted approach after additional development based on comments from Jens and others indicated that iorlv0 impacted the IDE subsystem and was unnecessarily broad in general.Quoted 7 lines
The siorl-v0 patch allows drivers to enable concurrent queueing through the
concurrent_queuefield in theScsi_Host_Templatewhich is copied to the request queue. It creates SCSI-specific versions of generic block i/o functions used by the SCSI subsystem and modifies them to conditionally engage the new locking scheme based on this field. It allows control over which drivers use concurrent queueing and preserves original block i/o behavior by default.
Quoted 4 lines
Sorry Jonathan, but this is even more broken than the last patch. In different ways. In no particular order:
o You are duplicating way too much code and exporting block internals
The duplication is a reasonable starting point for SCSI-specific functions. The block i/o design provides for exactly this type of tailoring through function pointers installed in request_queue.
What problem you do see with exporting block internals?
Quoted 5 lines
o You are breaking SCSI merge completely, why on earth are you suddenly using ll_*_merge functions for SCSI?!
oscsi_make_requestneed not worry about head active
oscsi_make_requestcan safe the q->*_merge indirect
oscsi_dispatch_cmd()io_request_lockremoval looks racy
I will investigate the above comments further.
Quoted 1 line
At least you are not breaking anything other than SCSI this time...
Do you think the separation of SCSI from generic block i/o code and the driver-activated control of concurrent queueing provides a path for future work to reduce io_request_lock contention in SCSI/FC?
Signature · 5 lines
Jonathan Lahr IBM Linux Technology Center Beaverton, Oregon [email protected] 503-578-3385
Quoted 28 lines · Jonathan Lahr
Quoted 27 lines · Jens Axboe [[email protected]] wrote:
Quoted 14 lines · Jonathan Lahr
This is a request for comments on the patch described below which implements a revised approach to reducing
io_request_lockcontention in 2.4.This new version of the
io_request_lockpatch (siorl-v0) is available at http://sourceforge.net/projects/lse/. It employs the same concurrent request queueing scheme as the iorlv0 patch but isolates code changes to the SCSI subsystem and engages the new locking scheme only for SCSI drivers which explicitly request it. I took this more restricted approach after additional development based on comments from Jens and others indicated that iorlv0 impacted the IDE subsystem and was unnecessarily broad in general.Quoted 8 lines
The siorl-v0 patch allows drivers to enable concurrent queueing through the
concurrent_queuefield in theScsi_Host_Templatewhich is copied to the request queue. It creates SCSI-specific versions of generic block i/o functions used by the SCSI subsystem and modifies them to conditionally engage the new locking scheme based on this field. It allows control over which drivers use concurrent queueing and preserves original block i/o behavior by default.Sorry Jonathan, but this is even more broken than the last patch. In different ways. In no particular order:
Quoted 6 lines
Quoted 2 lines
o You are duplicating way too much code and exporting block internals
The duplication is a reasonable starting point for SCSI-specific functions. The block i/o design provides for exactly this type of tailoring through function pointers installed in
request_queue.
Yes I know, I wrote most of said code :-)
Quoted 1 line
What problem you do see with exporting block internals?
It's absolutely worthless. Look, it ties in with the points I made below. You are exporting the merge functions for instance, and setting them in the queue. This will cause scsi_merge not to use it's own functions, broken.
The make_request_fn addition could be ok, just needs to be cleaned a bit.
Quoted 6 lines
Quoted 5 lines
o You are breaking SCSI merge completely, why on earth are you suddenly using ll_*_merge functions for SCSI?! o
scsi_make_requestneed not worry about head active oscsi_make_requestcan safe the q->*_merge indirect oscsi_dispatch_cmd()io_request_lockremoval looks racy
Quoted 7 lines
I will investigate the above comments further.
Quoted 1 line
At least you are not breaking anything other than SCSI this time...
Do you think the separation of SCSI from generic block i/o code and the driver-activated control of concurrent queueing provides a path for future work to reduce
io_request_lockcontention in SCSI/FC?
Not really, but I do think it could be a viable 2.4 alternative. For 2.5 we still want to do this the right way.
Signature · 1 line
Jens Axboe
Quoted 29 lines · Jens Axboe [[email protected]] wrote:
Quoted 28 lines · Jonathan Lahr
Quoted 27 lines · Jens Axboe [[email protected]] wrote:
Quoted 14 lines · Jonathan Lahr
This is a request for comments on the patch described below which implements a revised approach to reducing
io_request_lockcontention in 2.4.This new version of the
io_request_lockpatch (siorl-v0) is available at http://sourceforge.net/projects/lse/. It employs the same concurrent request queueing scheme as the iorlv0 patch but isolates code changes to the SCSI subsystem and engages the new locking scheme only for SCSI drivers which explicitly request it. I took this more restricted approach after additional development based on comments from Jens and others indicated that iorlv0 impacted the IDE subsystem and was unnecessarily broad in general.Quoted 8 lines
The siorl-v0 patch allows drivers to enable concurrent queueing through the
concurrent_queuefield in theScsi_Host_Templatewhich is copied to the request queue. It creates SCSI-specific versions of generic block i/o functions used by the SCSI subsystem and modifies them to conditionally engage the new locking scheme based on this field. It allows control over which drivers use concurrent queueing and preserves original block i/o behavior by default.Sorry Jonathan, but this is even more broken than the last patch. In different ways. In no particular order:
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
o You are duplicating way too much code and exporting block internals
The duplication is a reasonable starting point for SCSI-specific functions. The block i/o design provides for exactly this type of tailoring through function pointers installed in
request_queue.Yes I know, I wrote most of said code :-)
And this approach makes good use of it.
Quoted 6 lines
Quoted 1 line
What problem you do see with exporting block internals?
It's absolutely worthless. Look, it ties in with the points I made below. You are exporting the merge functions for instance, and setting them in the queue. This will cause
scsi_mergenot to use it's own functions, broken.
As in the baseline, initialize_merge_fn overwrites these pointers:
q->back_merge_fn = scsi_back_merge_fn_;
q->front_merge_fn = scsi_front_merge_fn_;
q->merge_requests_fn = scsi_merge_requests_fn_;Quoted 6 lines
Quoted 3 lines
Do you think the separation of SCSI from generic block i/o code and the driver-activated control of concurrent queueing provides a path for future work to reduce
io_request_lockcontention in SCSI/FC?Not really, but I do think it could be a viable 2.4 alternative. For 2.5 we still want to do this the right way.
I'll try to stay apprised of the 2.5 work as it progresses.
Signature · 5 lines
Jonathan Lahr IBM Linux Technology Center Beaverton, Oregon [email protected] 503-578-3385
Quoted 9 lines · Jonathan Lahr
Quoted 4 lines
It's absolutely worthless. Look, it ties in with the points I made below. You are exporting the merge functions for instance, and setting them in the queue. This will cause
scsi_mergenot to use it's own functions, broken.As in the baseline,
initialize_merge_fnoverwrites these pointers:q->back_merge_fn = scsi_back_merge_fn_; q->front_merge_fn = scsi_front_merge_fn_; q->merge_requests_fn = scsi_merge_requests_fn_;
I had forgotten I had #if 0 out the check for already set back_merge etc in scsi_merge -- however that's still beside the point. _Why_ are you exporting the ll_rw_blk functions and setting them just to have them overridden? Makes no sense.
Don't export the merge functions ever, define your own if you really need them. You don't, though.
As I've mentioned before, go ahead with the make_request_fn replacement. That is indeed what it is there for.
Signature · 1 line
Jens Axboe
Quoted 15 lines · Jens Axboe [[email protected]] wrote:
Quoted 9 lines · Jonathan Lahr
Quoted 4 lines
It's absolutely worthless. Look, it ties in with the points I made below. You are exporting the merge functions for instance, and setting them in the queue. This will cause
scsi_mergenot to use it's own functions, broken.As in the baseline,
initialize_merge_fnoverwrites these pointers:q->back_merge_fn = scsi_back_merge_fn_; q->front_merge_fn = scsi_front_merge_fn_; q->merge_requests_fn = scsi_merge_requests_fn_;I had forgotten I had #if 0 out the check for already set
back_mergeetc inscsi_merge-- however that's still beside the point. _Why_ are you exporting thell_rw_blkfunctions and setting them just to have them overridden? Makes no sense.
...
Quoted 2 lines
Don't export the merge functions ever, define your own if you really need them. You don't, though.
That resulted merely from basing scsi_init_queue on blk_init_queue. I'll simplify the code by removing these unnecessary assignments.
Signature · 5 lines
Jonathan Lahr IBM Linux Technology Center Beaverton, Oregon [email protected] 503-578-3385