Beginning of some updates to scsi mid layer
A message that is not in the archive (sent off-list or not yet received): [email protected]
Review
Quoted 8 lines
So, when Justin Gibbs' driver allocates a queue depth of 253
commands on drive X, then finds out that drive X has a hard limit of 64, then we aren't continuing to waste 189 commands worth of allocated kernel memory that can never be effectively used.(quoted diff) +3
-0+ diff = p->dev_lun_queue_depth[tindex] - + p->dev_active_cmds[tindex]; + p->dev_lun_queue_depth[tindex] -= diff;
I think this is not enough, or backwards. We hit a situation with EMC when attaching hundreds of disks depleted atomic memory with queue request arrays before swapper had a chance to replenish it. I would much appreciate if the initial allocation was concervative, if low performing, and was adjusted upwards at slave_attach time.
For 2.5 I would appreciate if we made some SCSI allocations non-atomic. The probe time things better be GFP_KERNEL.
-- Pete
Quoted 10 lines
I think this is not enough, or backwards. We hit a situation with EMC when attaching hundreds of disks depleted atomic memory with queue request arrays before swapper had a chance to replenish it. I would much appreciate if the initial allocation was concervative, if low performing, and was adjusted upwards at
slave_attachtime.For 2.5 I would appreciate if we made some SCSI allocations non-atomic. The probe time things better be
GFP_KERNEL.
I'd go further and say that you only need probe your boot device. After that, GFP_KERNEL is more than sufficient for anything.
Quoted 5 lines · Pete Zaitcev
Quoted 4 lines
So, when Justin Gibbs' driver allocates a queue depth of 253
commands on drive X, then finds out that drive X has a hard limit of 64, then we aren't continuing to waste 189 commands worth of allocated kernel memory that can never be effectively used.
Quoted 9 lines
Review
Quoted 3 lines
(quoted diff) +3
-0+ diff = p->dev_lun_queue_depth[tindex] - + p->dev_active_cmds[tindex]; + p->dev_lun_queue_depth[tindex] -= diff;I think this is not enough, or backwards. We hit a situation with EMC when attaching hundreds of disks depleted atomic memory with queue request arrays before swapper had a chance to replenish it. I would much appreciate if the initial allocation was concervative, if low performing, and was adjusted upwards at
slave_attachtime.
This is actually from the runtime code, part of my interrupt handler. It notices that we might get a lot of QUEUE_FULLs from a device all at the same queue depth, and in that case it reduces the queue depth to the new depth. So, in short, it's not backwards, it's actually taking the depth down based upon solid feedback from the drive.
Quoted 2 lines
For 2.5 I would appreciate if we made some SCSI allocations non-atomic. The probe time things better be
GFP_KERNEL.
Noted, I'll change that soon. As long as we have at least 1 command block, then the other allocations can be at our leisure since the change to scsi_release_command() will attempt to allocate 1 command block per completed command until they catch up to the intended queue depth. And, since that isn't done from an interrupt, all make it GFP_KERNEL as well and if we fail, then we simply wait and do the allocation later.
Allocations on the aic7xxx_old driver are a different thing. Don't pay too much attention to them. This work I'm doing here has pointed out to me that the tagged queue handling in the driver is totally busted in regards to multi-lun disk devices. I would actually have to do a rather large overhaul to get it right, and that's not on the plate until I've got some of this other stuff out of the way.
Signature · 4 lines
Doug Ledford <[email protected]> 919-754-3700 x44233 Red Hat, Inc. 1801 Varsity Dr. Raleigh, NC 27606
Quoted 6 lines · Doug Ledford
This is actually from the runtime code, part of my interrupt handler. It notices that we might get a lot of
QUEUE_FULLsfrom a device all at the same queue depth, and in that case it reduces the queue depth to the new depth. So, in short, it's not backwards, it's actually taking the depth down based upon solid feedback from the drive.
Is there any accounting for the following condition:
An array with multiple logical units can accept 256 commands (for argument purposes) from all initiators for all luns. Therefore, the number of commands that you can queue to sdX is dependent on how many commands queued by this initiator to other sds, as well as how many commands other initiators have issued to this sd and others.
This may not be an issue so much with parallel SCSI RAIDs, but it does come up with Fibrechannel.
It is theoretically possible to get QUEUE_FULL when the host has no other commands in progress on a logical unit. Would the host driver then believe that it could issue 0 or 1 command only, or is there some lower limit?
jeremy
Quoted 8 lines · Jeremy Higdon
Is there any accounting for the following condition:
An array with multiple logical units can accept 256 commands (for argument purposes) from all initiators for all luns. Therefore, the number of commands that you can queue to sdX is dependent on how many commands queued by this initiator to other sds, as well as how many commands other initiators have issued to this sd and others.
That's (currently) on a per driver basis. My driver would account for this situation just fine. Others might not. That is one of the things I plan to change in the mid layer very soon now.
Quoted 8 lines
This may not be an issue so much with parallel SCSI RAIDs, but it does come up with Fibrechannel.
It is theoretically possible to get
QUEUE_FULLwhen the host has no other commands in progress on a logical unit. Would the host driver then believe that it could issue 0 or 1 command only, or is there some lower limit?jeremy
Signature · 4 lines
Doug Ledford <[email protected]> 919-754-3700 x44233 Red Hat, Inc. 1801 Varsity Dr. Raleigh, NC 27606