Ghost devices being reported with AIC7XXX version 6.2.6
Subsystems (2)
PATCH 1/1 Ghost devices being reported with AIC7XXX version 6.2.6
Hi,
It seems the kernel reintroduced a bug that I fixed a year ago regarding phantom devices being reported by the aic7xxx driver. The problem is that the AIC driver defines the maximum LUs it supports as 64 (aic7xxx.h:#define AHC_NUM_LUNS 64). Symmetrix reports SCSI_2 devices which according to the spec means no target supports more than 8 LUs (LUN 0-7) per target id. However, since the Symmetrix can be configured to support more than 8 LUs per target id, its tagged in the blacklist with the BLIST_LARGELUN flag, which trips the scanning code into treating the Symmetrix as a SCSI_3 device:
drivers/scsi/scsi_scan.c:
{"EMC", "SYMMETRIX", "*", BLIST_SPARSELUN | BLIST_LARGELUN | BLIST_FORCELUN},
..........
/* don't probe further for luns > 7 for targets <= SCSI_2 */
if ((lun0_sl < SCSI_3) && (lun > 7))
break;
if (!scan_scsis_single(channel, order_dev, lun, lun0_sl,
&max_dev_lun, &sparse_lun, &SDpnt, shpnt,
scsi_result) && !sparse_lun)
break; /* break means don't probe further for luns!=0 */
if (SDpnt && (0 == lun)) {
int bflags = get_device_flags (scsi_result);
if (bflags & BLIST_LARGELUN)
lun0_sl = SCSI_3; /* treat as SCSI 3 */
else
lun0_sl = SDpnt->scsi_level;This isn't a problem so far until we start scanning LUN > 31. The SCSI_3 IDENTIFY Message Format defines 6 bits for LUN, but since Symmetrix reports its devices as SCSI_2, we follow the SCSI_2 IDENTIFY Message Format which defines 3 bits for the LUNTRN, and we also use the 2 reserved bits which gives us 5 bits or 32 LUs. So, when the SCSI subsystem begins scanning for LUNs > 31, the aic7xxx driver reports devices that are actually (LUN & 0x0000001f) and LUNs that have already been reported are re-reported, hence the phantom device problem.
Justin can you change AHC_NUM_LUNS to 32 or suggest an alternative fix?
Alternatively a new BLIST_FLAG could be defined. This also has its problems because Fibre attached Symm's work with LUN > 31 without problems since the IDENTIFY Message is unique to Parallel. Perhaps another option is to add a "ahc_num_luns=xxx" module parameter allowing the decision to be made at runtime.
Please cc: me on linux replies as I'm not currently subscribed.
--------CUT HERE---------
drivers/scsi/aic7xxx/aic7xxx.h renamed from drivers/scsi/aic7xxx/aic7xxx.h.orig +1 -1
@@ -122,7 +122,7 @@* You can have 2^64 luns when information unit transfers are enabled, * but it is doubtful this driver will ever support IUTs. */ -#define AHC_NUM_LUNS 64 +#define AHC_NUM_LUNS 32/* * The maximum transfer per S/G segment.
--------CUT HERE---------
-Michael F. Brown, EMC Corp.
Email: [email protected] EMC Tie Line: x43416 External Line: (508) 249-3416
"5 years from now everyone will be running free
GNU on their 200 MIPS, 64M SPARCstation-5." -Andrew Tanenbaum '92
(In late 1997 I installed Linux for the first time on my 200Mhz, 128M
AMD x86 clone)
Quoted 1 line
Justin can you change
AHC_NUM_LUNSto 32 or suggest an alternative fix?
The aic7xxx driver is not at fault and so should not be changed. The aic7xxx driver really does support all 64 luns. In my opinion, there are three problems here:
1) The Symetrix should treat a non-zero LUNTAR bit as an invalid selection. This would prevent these higher luns from showing up as valid devices.
2) Linux should never use more than 5 bits of lun for a device that reports itself as SCSI-2. If the lun is larger than what will fit in 3 bits, byte 1 of the CDB should not be altered even if the device is SCSI2.
3) Linux should be using the Report Luns command to determine how many luns are on a device and at what locations. This might even allow you to never probe more than 8 luns on a SCSI 2 device unless they support this command. Does the Symetrix support it? Going beyond 8 luns on a SCSI2 device is really a hack otherwise.
--
Justin
Quoted 17 lines
Quoted 3 lines
Justin can you change
AHC_NUM_LUNSto 32 or suggest an alternative fix?The aic7xxx driver is not at fault and so should not be changed. The aic7xxx driver really does support all 64 luns. In my opinion, there are three problems here:
1) The Symetrix should treat a non-zero LUNTAR bit as an invalid selection. This would prevent these higher luns from showing up as valid devices.
2) Linux should never use more than 5 bits of lun for a device that reports itself as SCSI-2. If the lun is larger than what will fit in 3 bits, byte 1 of the CDB should not be altered even if the device is SCSI2.
Quoted 6 lines
3) Linux should be using the Report Luns command to determine how many luns are on a device and at what locations. This might even allow you to never probe more than 8 luns on a SCSI 2 device unless they support this command. Does the Symetrix support it? Going beyond 8 luns on a SCSI2 device is really a hack otherwise.
Absolutely.
Quoted 8 lines · Matthew Jacob
Quoted 6 lines
3) Linux should be using the Report Luns command to determine how many luns are on a device and at what locations. This might even allow you to never probe more than 8 luns on a SCSI 2 device unless they support this command. Does the Symetrix support it? Going beyond 8 luns on a SCSI2 device is really a hack otherwise.
Absolutely.
Now is the right time to fix this properly for 2.5
Quoted 10 lines · Matthew Jacob
Quoted 7 lines
Quoted 6 lines
3) Linux should be using the Report Luns command to determine how many luns are on a device and at what locations. This might even allow you to never probe more than 8 luns on a SCSI 2 device unless they support this command. Does the Symetrix support it? Going beyond 8 luns on a SCSI2 device is really a hack otherwise.
Quoted 5 lines
Quoted 1 line
Absolutely.
Now is the right time to fix this properly for 2.5
Well, yes.
Quoted 10 lines · Matthew Jacob
Quoted 10 lines · Matthew Jacob
Quoted 7 lines
Quoted 6 lines
3) Linux should be using the Report Luns command to determine how many luns are on a device and at what locations. This might even allow you to never probe more than 8 luns on a SCSI 2 device unless they support this command. Does the Symetrix support it? Going beyond 8 luns on a SCSI2 device is really a hack otherwise.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Absolutely.
Now is the right time to fix this properly for 2.5
Well, yes.
2.5 does this already.
Signature · 4 lines
Doug Ledford <[email protected]> 919-754-3700 x44233 Red Hat, Inc. 1801 Varsity Dr. Raleigh, NC 27606
Quoted 14 lines · Matthew Jacob
Quoted 12 lines
Quoted 12 lines · Matthew Jacob
Quoted 8 lines
Quoted 6 lines
3) Linux should be using the Report Luns command to determine how many luns are on a device and at what locations. This might even allow you to never probe more than 8 luns on a SCSI 2 device unless they support this command. Does the Symetrix support it? Going beyond 8 luns on a SCSI2 device is really a hack otherwise.
Absolutely.
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Now is the right time to fix this properly for 2.5
Well, yes.
2.5 does this already.
*blush*
I was actually going to start finally working 2.5 next week (don't hit me! don't hit me)
Quoted 5 lines · Matthew Jacob
*blush*
I was actually going to start finally working 2.5 next week (don't hit me! don't hit me)
Hehehe! Going to start after the feature freeze is already over, makes it kind of hard to get anything new in doesn't it ;-)
Signature · 4 lines
Doug Ledford <[email protected]> 919-754-3700 x44233 Red Hat, Inc. 1801 Varsity Dr. Raleigh, NC 27606
Quoted 9 lines · Matthew Jacob
Quoted 5 lines
*blush*
I was actually going to start finally working 2.5 next week (don't hit me! don't hit me)
Hehehe! Going to start after the feature freeze is already over, makes it kind of hard to get anything new in doesn't it ;-)
Well, look- doing feature freezes early is all well and swell, but remember that a lot of us out here, if we get any work stuff at all for Linux, has us concentrate on 2.4, or even 2.2.
The only people to ever mention 2.5 recently did so because they're planning for next year.
Early feature freezes are great for those who are *on* the long term train and are comfortable with shoving things out a year. This, btw, is one of things I detested about Sun when I worked there. Drivers and driver subsystems are candidates for late binding and late changing because they are typically isolated areas.
It's also true that I haven't really been involved other than kibitzing for 2.5. It would have required me to be a lot more tightly involved in the Linux world than I have been. My best hope is to comment with perspective outside of Linux and debug and offer alternatives where possible- that way I can remain positive and perhaps helpful when I cannot put in all the time that I either can or would like to do.
Hmph. If we'd actually connected and I'd gone to work for RH 18mos ago, this all would be a moot discussion.
-matt
Quoted 2 lines
The only people to ever mention 2.5 recently did so because they're
v "to me"
Quoted 1 line
planning for next year.
Quoted 7 lines · Matthew Jacob
Quoted 7 lines · Matthew Jacob
Quoted 5 lines
*blush*
I was actually going to start finally working 2.5 next week (don't hit me! don't hit me)
Quoted 6 lines
Quoted 2 lines
Hehehe! Going to start after the feature freeze is already over, makes it kind of hard to get anything new in doesn't it ;-)
Well, look- doing feature freezes early is all well and swell, but remember that a lot of us out here, if we get any work stuff at all for Linux, has us concentrate on 2.4, or even 2.2.
Yes, I know, that's why there was a wink-smiley there.
Quoted 2 lines
driver subsystems are candidates for late binding and late changing because they are typically isolated areas.
As well as they are here. At least driver updates anyway. Driver subsystem changes are a bit harder to get in, especially if they stand a chance of breaking other working drivers.
Signature · 4 lines
Doug Ledford <[email protected]> 919-754-3700 x44233 Red Hat, Inc. 1801 Varsity Dr. Raleigh, NC 27606
Quoted 9 lines · Doug Ledford
Quoted 5 lines · Matthew Jacob
*blush*
I was actually going to start finally working 2.5 next week (don't hit me! don't hit me)
Hehehe! Going to start after the feature freeze is already over, makes it kind of hard to get anything new in doesn't it ;-)
Its not a major new feature its just some more scsi bug fixing
Quoted 11 lines · Alan Cox
Quoted 8 lines · Matthew Jacob
Quoted 6 lines
3) Linux should be using the Report Luns command to determine how many luns are on a device and at what locations. This might even allow you to never probe more than 8 luns on a SCSI 2 device unless they support this command. Does the Symetrix support it? Going beyond 8 luns on a SCSI2 device is really a hack otherwise.
Absolutely.
Now is the right time to fix this properly for 2.5
REPORT LUNS is a SCSI-3 command, not SCSI-2, so it can't solve the problem. Right?
You should be able to configure any recent EMC box (and AFAIK any recent disk array hardware, at least IBM Shark, IBM fastt) to report back as a SCSI-3 device.
-- Patrick Mansfield
Quoted 6 lines · Justin T. Gibbs
Quoted 1 line
Justin can you change
AHC_NUM_LUNSto 32 or suggest an alternative fix?The aic7xxx driver is not at fault and so should not be changed. The aic7xxx driver really does support all 64 luns. In my opinion, there are three problems here:
Quoted 8 lines
1) The Symetrix should treat a non-zero LUNTAR bit as an invalid selection. This would prevent these higher luns from showing up as valid devices.
2) Linux should never use more than 5 bits of lun for a device that reports itself as SCSI-2. If the lun is larger than what will fit in 3 bits, byte 1 of the CDB should not be altered even if the device is SCSI2.
The whole "support more than 8 luns on SCSI-2 devices" is a hack anyway, I would be more likely to just remove it instead of changing other things.
Quoted 6 lines
3) Linux should be using the Report Luns command to determine how many luns are on a device and at what locations. This might even allow you to never probe more than 8 luns on a SCSI 2 device unless they support this command. Does the Symetrix support it? Going beyond 8 luns on a SCSI2 device is really a hack otherwise.
We currently only do report luns on SCSI-3 devices. I didn't think this was a supported command on SCSI-2, but I hadn't checked the spec recently...hmmm...nope, not supported in SCSI-2, so using report luns on SCSI-2 devices is just as much a hack as going above lun 7 on SCSI-2 devices without specific knowledge of transfer interconnect.
Quoted 6 lines
--
Justin
-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 4 lines
Doug Ledford <[email protected]> 919-754-3700 x44233 Red Hat, Inc. 1801 Varsity Dr. Raleigh, NC 27606
Quoted 6 lines
We currently only do report luns on SCSI-3 devices. I didn't think this was a supported command on SCSI-2, but I hadn't checked the spec recently...hmmm...nope, not supported in SCSI-2, so using report luns on SCSI-2 devices is just as much a hack as going above lun 7 on SCSI-2 devices without specific knowledge of transfer interconnect.
It's transitional in that a number of devices report as SCSI-2 but do things like REPORT LUNs. It's changing very fast though.
Quoted 9 lines · Matthew Jacob
Quoted 6 lines
We currently only do report luns on SCSI-3 devices. I didn't think this was a supported command on SCSI-2, but I hadn't checked the spec recently...hmmm...nope, not supported in SCSI-2, so using report luns on SCSI-2 devices is just as much a hack as going above lun 7 on SCSI-2 devices without specific knowledge of transfer interconnect.
It's transitional in that a number of devices report as SCSI-2 but do things like REPORT LUNs. It's changing very fast though.
And we recently added INQUIRY with the EVPD bit set and an EVPD page 0.
Quoted 1 line
From the reports I've seen, I'm pretty sure it's causing a more than
trivial number of devices to lock up on boot. I suspect blindly pulling a REPORT_LUNS on SCSI-2 devices would likely cause similar problems.
Signature · 4 lines
Doug Ledford <[email protected]> 919-754-3700 x44233 Red Hat, Inc. 1801 Varsity Dr. Raleigh, NC 27606
Quoted 5 lines
And we recently added INQUIRY with the EVPD bit set and an EVPD page 0. From the reports I've seen, I'm pretty sure it's causing a more than trivial number of devices to lock up on boot. I suspect blindly pulling a
REPORT_LUNSon SCSI-2 devices would likely cause similar problems.
That's less likely than the EVPD stuff which seems to make more devices choke than not.
I would suggest the following for REPORT LUNS usage on SCSI-2 devices:
+ a config option (so it goes away in a year) to enable REPORT LUNS on SCSI-2, or set an uppper SCSI-2 lun scan limit.
+ REPORT LUNS will only be used on SCSI-2 devices that are DIRECT ACCESS (sorry, ATL && other tape and tape emulator folks)
Quoted 4 lines
And we recently added INQUIRY with the EVPD bit set and an EVPD page 0.
Quoted 1 line
From the reports I've seen, I'm pretty sure it's causing a more than
trivial number of devices to lock up on boot. I suspect blindly pulling a
REPORT_LUNSon SCSI-2 devices would likely cause similar problems.
What about adding BLIST_FORCE_REPORT_LUNS?
-Michael F. Brown, EMC Corp.
Email: [email protected] EMC Tie Line: x43416 External Line: (508) 249-3416
"5 years from now everyone will be running free
GNU on their 200 MIPS, 64M SPARCstation-5." -Andrew Tanenbaum '92
(In late 1997 I installed Linux for the first time on my 200Mhz, 128M
AMD x86 clone)
Yeah, but lets get out of the tailchasing exercise of list updates if we can...
Quoted 19 lines
Quoted 4 lines
And we recently added INQUIRY with the EVPD bit set and an EVPD page 0.
Quoted 1 line
From the reports I've seen, I'm pretty sure it's causing a more than
trivial number of devices to lock up on boot. I suspect blindly pulling a
REPORT_LUNSon SCSI-2 devices would likely cause similar problems.What about adding
BLIST_FORCE_REPORT_LUNS?-Michael F. Brown, EMC Corp.
Email: [email protected] EMC Tie Line: x43416 External Line: (508) 249-3416"5 years from now everyone will be running free
GNU on their 200 MIPS, 64M SPARCstation-5." -Andrew Tanenbaum '92(In late 1997 I installed Linux for the first time on my 200Mhz, 128M
AMD x86 clone)
I'd rather have to add a list entry for one of a few dozen storage array vendors then get hundreds bug reports from noname USB block devices that blew up when we sent REPORT_LUNS. Considering a lot of them blow up if you ask for more than a 36 byte INQUIRY response, I think REPORT_LUNS should be special cased in the blacklist.
I haven't been tracking 2.5, but I knew there were plans to push push the scanning code and the blacklist stuff into a userspace scsid. Did this make it?
Quoted 24 lines · Matthew Jacob
Yeah, but lets get out of the tailchasing exercise of list updates if we can...
Quoted 19 lines
Quoted 4 lines
And we recently added INQUIRY with the EVPD bit set and an EVPD page 0.
Quoted 1 line
From the reports I've seen, I'm pretty sure it's causing a more than
trivial number of devices to lock up on boot. I suspect blindly pulling a
REPORT_LUNSon SCSI-2 devices would likely cause similar problems.What about adding
BLIST_FORCE_REPORT_LUNS?-Michael F. Brown, EMC Corp.
Email: [email protected] EMC Tie Line: x43416 External Line: (508) 249-3416"5 years from now everyone will be running free
GNU on their 200 MIPS, 64M SPARCstation-5." -Andrew Tanenbaum '92(In late 1997 I installed Linux for the first time on my 200Mhz, 128M
AMD x86 clone)
-Michael F. Brown, EMC Corp.
Email: [email protected] EMC Tie Line: x43416 External Line: (508) 249-3416
"5 years from now everyone will be running free
GNU on their 200 MIPS, 64M SPARCstation-5." -Andrew Tanenbaum '92
(In late 1997 I installed Linux for the first time on my 200Mhz, 128M
AMD x86 clone)
Quoted 3 lines · [email protected] said:
I haven't been tracking 2.5, but I knew there were plans to push push the scanning code and the blacklist stuff into a userspace scsid. Did this make it?
That was my plan, but I spent too much time scraping out the innards. I hope to do it for 2.7
James
Hi,
Quoted 2 lines
The whole "support more than 8 luns on SCSI-2 devices" is a hack anyway, I would be more likely to just remove it instead of changing other things.
This would break production environments. Symmetrix isn't the only storage array that supports this hack. DG, Dell, HP, and others also support this hack. If its taken out in 2.5 and replaced with REPORT_LUNS that'd be ok, but taking it out of 2.4 without replacing it with REPORT_LUNS is going to break people.
Quoted 7 lines
Quoted 6 lines
3) Linux should be using the Report Luns command to determine how many luns are on a device and at what locations. This might even allow you to never probe more than 8 luns on a SCSI 2 device unless they support this command. Does the Symetrix support it? Going beyond 8 luns on a SCSI2 device is really a hack otherwise.
Quoted 5 lines
We currently only do report luns on SCSI-3 devices. I didn't think this was a supported command on SCSI-2, but I hadn't checked the spec recently...hmmm...nope, not supported in SCSI-2, so using report luns on SCSI-2 devices is just as much a hack as going above lun 7 on SCSI-2 devices without specific knowledge of transfer interconnect.
Symmetrix supports Report Luns despite reporting SCSI_2 Units. The problem is that because Symmetrix in the device table with a BLIST_LARGELUN flag, this block gets tripped in the scanning code:
if (SDpnt && (0 == lun)) {
int bflags = get_device_flags (scsi_result);
if (bflags & BLIST_LARGELUN)
lun0_sl = SCSI_3; /* treat as SCSI 3 */
else
lun0_sl = SDpnt->scsi_level;
}The quick fix is to change the AIC driver's max lun value to 32, but Justin's right, its not the AIC driver with the problem. I don't know what the right fix is for 2.4. We need a hack akin to something like:
if ((bflags & BLIST_LARGELUN) && !shpnt->is_fibre && lun > 31) /* IDENTIFY only has 5 bits for LUN including reserved hack */ break; else if ((lun0_sl < SCSI_3) && (lun > 7)) /* don't probe further for luns > 7 for targets <= SCSI_2 */ break; if (!scan_scsis_single(channel, order_dev, lun, lun0_sl, &max_dev_lun, &sparse_lun, &SDpnt, shpnt, scsi_result) && !sparse_lun) break; /* break means don't probe further for luns!=0 */ if (SDpnt && (0 == lun)) lun0_sl = SDpnt->scsi_level;
and to get rid of this logic:
if (SDpnt && (0 == lun)) {
int bflags = get_device_flags (scsi_result);
if (bflags & BLIST_LARGELUN)
lun0_sl = SCSI_3; /* treat as SCSI 3 */
else
lun0_sl = SDpnt->scsi_level;Comments? Again please cc: me for list replies.
-Michael F. Brown, EMC Corp.
Email: [email protected]
"5 years from now everyone will be running free
GNU on their 200 MIPS, 64M SPARCstation-5." -Andrew Tanenbaum '92
(In late 1997 I installed Linux for the first time on my 200Mhz, 128M
AMD x86 clone)
Quoted 10 lines · Mike Brown
Hi,
Quoted 2 lines
The whole "support more than 8 luns on SCSI-2 devices" is a hack anyway, I would be more likely to just remove it instead of changing other things.
This would break production environments. Symmetrix isn't the only storage array that supports this hack. DG, Dell, HP, and others also support this hack. If its taken out in 2.5 and replaced with
REPORT_LUNSthat'd be ok, but taking it out of 2.4 without replacing it withREPORT_LUNSis going to break people.
Note, these hacks have caused more more bug reports and more crap in the last 2 years than anything else when it comes to device scanning in linux. If it were up to me, I'm so fed up with them I'd yank *all* the hacks that make us scan for non-compliant devices and instead tell the vendors to fix their firmware to be SCSI compliant.
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
3) Linux should be using the Report Luns command to determine how many luns are on a device and at what locations. This might even allow you to never probe more than 8 luns on a SCSI 2 device unless they support this command. Does the Symetrix support it? Going beyond 8 luns on a SCSI2 device is really a hack otherwise.
Quoted 7 lines
Quoted 5 lines
We currently only do report luns on SCSI-3 devices. I didn't think this was a supported command on SCSI-2, but I hadn't checked the spec recently...hmmm...nope, not supported in SCSI-2, so using report luns on SCSI-2 devices is just as much a hack as going above lun 7 on SCSI-2 devices without specific knowledge of transfer interconnect.
Symmetrix supports Report Luns despite reporting
SCSI_2Units. The
Can you comment on why Symmetrix reports SCSI-2 units then? Report Luns is an optional SCSI-3 command. If you don't have all the required SCSI-3 items implemented yet so that you can't report SCSI-3, then I would think you need to not worry about optional commands until the required ones are done. On the other hand, if you've got all the required SCSI-3 items implemented, then why aren't you reporting SCSI-3?
Quoted 2 lines
I don't know
what the right fix is for 2.4. We need a hack akin to something like:
The right fix doesn't exist. Hack is exactly what it's going to take to fix this. Once again, it's a case of some vendor wanting SCSI-3 features on a device that they only report as being SCSI-2 causing breakage.
I'm currently working in 2.5, not 2.4. This will be fixed in 2.5 by using report luns. My goal for 2.5 is to make the whitelist as small as possible though, so fixing up your firmware to report the proper SCSI version would be appreciated. If not, then it will have to be a report luns whitelist entry. For 2.4, I don't have an answer and I'm not sure when I would get around to looking at it personally. Not this week anyway, maybe next.
Signature · 4 lines
Doug Ledford <[email protected]> 919-754-3700 x44233 Red Hat, Inc. 1801 Varsity Dr. Raleigh, NC 27606