MPTSAS: ("activate status led" | "find disc") in external enclosure

Hallo!

As some of you might already now I have to set up a (for us) bigger SAN with Linux. I'm simulating disc failure right now. To find a defective disc in the enclosure is unresolved yet. I already had a look at several web pages and searched for it but without luck.

The manual of the enclosure says (PROMISE VTRAK J610S):
-----------------------------------------------------------------------
LEDs State

                | Dark     | Steady Green  | Flashing Green
Power/Activity* | No Drive | Drive Present | Activity
Status**        | HBA/RAID controller determines the LED behavior

[...]
** Refer to the user documentation for your HBA or RAID controller for this
information.
-----------------------------------------------------------------------

Is there a standard way to lit the led in the enclosure with linux means or anything alike? The enclosure does not tell the serial numbers of the included disc.
The HBA is an LSI SAS 3801X.

Thanks
Lars

                            Informationstechnologie
Berlin-Brandenburgische Akademie der Wissenschaften
Jägerstrasse 22-23                     10117 Berlin
Tel.: +49 30 20370-352           http://www.bbaw.de
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

It depends on whether you have the HBA configured in RAID mode or not. If you have, then it takes absolute control of the enclosure management at the moment and I don't think you can get to it.

If you run the HBA as a pure SAS one, then the promise website says this enclosure is ses2 compliant, so you should just be able to insert the ses module and have enclosure management (2.6.24+ kernels). This relies on the HBA exposing the enclosure device, but I think mptsas only eats it for RAID configurations.

to see what's in the enclosure, just do

ls /sys/class/enclosure/*/

each of the subdirectories has a slot label (name taken from SES) to locate the enclosure, just echo 1 to the locate file and it will flash the lights. If it has a disk, there should be a device link in the slot directory (might not be, this depends on ses working correctly and VPD inquiries identifying the right disk).

James

--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi James,

our HBA has no RAID option.

I tried 2.6.24.5 but the ses seems to be introduced in 2.6.25.

# ls /sys/class/enclosure/*/
/sys/class/enclosure/6:0:16:0/:
0  10  12  14  2  4  6  8  components  subsystem
1  11  13  15  3  5  7  9  device      uevent

/sys/class/enclosure/6:0:33:0/:

0  10  12  14  2  4  6  8  components  subsystem
1  11  13  15  3  5  7  9  device      uevent

This is two times the same enclosure.

monosan:/sys/class/enclosure/6:0:16:0/0 # ls -l
insgesamt 0
-rw-r--r-- 1 root root 4096 21. Apr 10:48 active
-rw-r--r-- 1 root root 4096 21. Apr 10:48 fault
-rw-r--r-- 1 root root 4096 21. Apr 10:48 locate
-rw-r--r-- 1 root root 4096 21. Apr 10:48 status
lrwxrwxrwx 1 root root 0 21. Apr 10:46 subsystem -> ../../../enclosure_component
-r--r--r-- 1 root root 4096 21. Apr 10:48 type
--w------- 1 root root 4096 21. Apr 10:48 uevent

There seems to be no device link if I understood you correctly. Is the subdirctory numbering somehow in relation to the SCSI ID?

Thanks
Lars
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

OK ... it's hard for me to keep track of what went when. It must have gone in after 2.6.24 in the merge window.

That's unusual .. to have two devices managing the same enclosure, but if you only have 16 slots and not 32, I suppose it must be so. I'd also guess the enclosure information doesn't provide slot names, so ses automatically provided the numbering.

I have a 12 slot vitesse enclosure with two enclosure devices but they each manage half the slots:

/sys/class/enclosure/0:0:1:0/:

components  power/     SLOT 007/  SLOT 009/  SLOT 011/   uevent
device@     SLOT 006/  SLOT 008/  SLOT 010/  subsystem@

/sys/class/enclosure/0:0:3:0/:

components  power/     SLOT 001/  SLOT 003/  SLOT 005/   uevent
device@     SLOT 000/  SLOT 002/  SLOT 004/  subsystem@

No ... the subdirectory numbering is taken from the enclosure information (if available)... it should be the slot numbers of the enclosure. If the slot is populated a device link should appear (this is the tricky bit because the ses driver uses VPD inquiries to match up the reported disk, assuming the enclosure reports it, to the actual one. These work for SAS disks, but not for SATA ones). Empty slots have no device link (but you can still flash the lights using the files).

If you look at my enclosure, I only have two devices:

sparkweed:~# ls -l /sys/class/enclosure/*/*/device
lrwxrwxrwx 1 root root 0 2008-04-21 10:16 /sys/class/enclosure/0:0:1:0/SLOT 006/device -> ../../../../../../../port-0:0:0/end_device-0:0:0/target0:0:0/0:0:0:0
lrwxrwxrwx 1 root root 0 2008-04-21 10:16 /sys/class/enclosure/0:0:3:0/SLOT 001/device -> ../../../../../../../port-0:1:11/end_device-0:1:11/target0:0:2/0:0:2:0

James

James

--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

I use both external connections to both expanderboards of the enclosure just for redundancy.

So this will never work for the SATA discs in our enclosure, right? Or is there a chance with asking LSI to do something in their firmware?

Seems I have no chance to get to know which slot contains the failed drive without a shutdown. Bad luck.

But thanks alot anyway.
Best regards
Lars
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

If you want to upgrade your controller fw, I need the contents of all the attributes in /sys/class/scsi_host/host#. Just yesterday we got Warner Wakko upgraded, and I believe it might of solved his problems. I'm not sure what problems your experiencing, as this thread has been hard to follow.

I would like to point out there is a firmware request for sending SEP request, for instance setting LEDS. Here is a copy of the structure. We call this function to set LED when there is SMART errors (for non-raid devices). For RAID, the firmware handles it. Is this what your looking for?

Function = MPI_FUNCTION_SCSI_ENCLOSURE_PROCESSOR:

typedef struct _MSG_SEP_REQUEST
{

    U8                      TargetID;           /* 00h */
    U8                      Bus;                /* 01h */
    U8                      ChainOffset;        /* 02h */
    U8                      Function;           /* 03h */
    U8                      Action;             /* 04h */
    U8                      Flags;              /* 05h */
    U8                      Reserved1;          /* 06h */
    U8                      MsgFlags;           /* 07h */
    U32                     MsgContext;         /* 08h */
    U32                     SlotStatus;         /* 0Ch */
    U32                     Reserved2;          /* 10h */
    U32                     Reserved3;          /* 14h */
    U32                     Reserved4;          /* 18h */
    U16                     Slot;               /* 1Ch */
    U16                     EnclosureHandle;    /* 1Eh */

} MSG_SEP_REQUEST, MPI_POINTER PTR_MSG_SEP_REQUEST,
SEPRequest_t, MPI_POINTER pSEPRequest_t;

/* Action defines */
#define MPI_SEP_REQ_ACTION_WRITE_STATUS (0x00)
#define MPI_SEP_REQ_ACTION_READ_STATUS (0x01)

/* Flags defines */
#define MPI_SEP_REQ_FLAGS_ENCLOSURE_SLOT_ADDRESS (0x01)
#define MPI_SEP_REQ_FLAGS_BUS_TARGETID_ADDRESS (0x00)

/* SlotStatus bits for MSG_SEP_REQUEST */

#define MPI_SEP_REQ_SLOTSTATUS_NO_ERROR                 (0x00000001)
#define MPI_SEP_REQ_SLOTSTATUS_DEV_FAULTY               (0x00000002)
#define MPI_SEP_REQ_SLOTSTATUS_DEV_REBUILDING           (0x00000004)
#define MPI_SEP_REQ_SLOTSTATUS_IN_FAILED_ARRAY          (0x00000008)
#define MPI_SEP_REQ_SLOTSTATUS_IN_CRITICAL_ARRAY        (0x00000010)
#define MPI_SEP_REQ_SLOTSTATUS_PARITY_CHECK             (0x00000020)
#define MPI_SEP_REQ_SLOTSTATUS_PREDICTED_FAULT          (0x00000040)
#define MPI_SEP_REQ_SLOTSTATUS_UNCONFIGURED             (0x00000080)
#define MPI_SEP_REQ_SLOTSTATUS_HOT_SPARE                (0x00000100)
#define MPI_SEP_REQ_SLOTSTATUS_REBUILD_STOPPED          (0x00000200)
#define MPI_SEP_REQ_SLOTSTATUS_REQ_CONSISTENCY_CHECK    (0x00001000)
#define MPI_SEP_REQ_SLOTSTATUS_DISABLE                  (0x00002000)
#define MPI_SEP_REQ_SLOTSTATUS_REQ_RESERVED_DEVICE      (0x00004000)
#define MPI_SEP_REQ_SLOTSTATUS_IDENTIFY_REQUEST         (0x00020000)
#define MPI_SEP_REQ_SLOTSTATUS_REQUEST_REMOVE           (0x00040000)
#define MPI_SEP_REQ_SLOTSTATUS_REQUEST_INSERT           (0x00080000)
#define MPI_SEP_REQ_SLOTSTATUS_DO_NOT_MOVE              (0x00400000)
#define MPI_SEP_REQ_SLOTSTATUS_ACTIVE                   (0x00800000)
#define MPI_SEP_REQ_SLOTSTATUS_B_ENABLE_BYPASS          (0x04000000)
#define MPI_SEP_REQ_SLOTSTATUS_A_ENABLE_BYPASS          (0x08000000)
#define MPI_SEP_REQ_SLOTSTATUS_DEV_OFF                  (0x10000000)
#define MPI_SEP_REQ_SLOTSTATUS_SWAP_RESET               (0x80000000)

--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Hallo Eric,

I hoped I had the newet version of the firmware on the HBA. monosan:~ # cat /sys/class/scsi_host/host6/version_fw 01.24.01.00
monosan:~ # cat /sys/class/scsi_host/host6/version_mpi 105
monosan:~ # cat /sys/class/scsi_host/host6/version_nvdata_default 2d02h
monosan:~ # cat /sys/class/scsi_host/host6/version_nvdata_persistent 2d02h
monosan:~ # cat /sys/class/scsi_host/host6/version_product LSISAS1068 B1
monosan:~ # cat /sys/class/scsi_host/host6/board_name SAS3801X

I'm not firm enough with scsi to understand all your words. But the problem I have is not that something doesn't work. It works with 2.6.25. I can get the LEDs to blink. I can choose which one should do. I simply don't know which one it should be.

The problem is, I wanted to simultate a faulty drive without being one faulty.
Therefore I choosed a (working) disk to be faulty by its mutlipath-name: dm-9
Then I found out which disk it is in sd? notation. Then I read the serial number of this disk with the smartmontools.
Now I want this LED of the bay containing the disk with this serial number to blink. But I can't find out which LED matches which serial number of the containing disk.

I know my english is not the best. Ask and I try to explain again.

Regards
Lars

                            Informationstechnologie
Berlin-Brandenburgische Akademie der Wissenschaften
Jägerstrasse 22-23                     10117 Berlin
Tel.: +49 30 20370-352           http://www.bbaw.de
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

This is code was release back in December, so its fairly new.

The current driver will set the faulty LED when there is sense ASC = 0x5D, meaning smart error. This will occurr from the device driver for devices not belonging to a RAID volume. If its in a RAID volume, the firmware will handle setting the LED. You can verify this by adding a printk inside the source code: From function mptscsih_copy_sense_data(), scroll down to "Log Smart Data", add it here after the check on 0x5D. Also mptscish_issue_sep_command() is called to send the SEP command to set the LED. This is found in /usr/src/linux/drivers/message/fusion/mptscsih.c.

Also, you should be able to figure out which slot your device inside your enclosure. Each enclousre will have a unique enclosoure logical id, which is simailar to a sas address. Each device that is in inside the enclosure is assigned a slot id, which is the same as a bay indentifier. If you have a SAS device, it will have the sas_address on the sticker. Then with that sas_address, you can search inside /sys/class/sas_device, that will tell you which end device it is. Within that, you will see the bay identifier, which is the slot id. The slot id I guess is assigned from 0 to the largest number you have of slots in your enclosure, probally from left to right. If you look inside the expanders, you will see the enslosure logical id. If you have more than one expander in the enclosure, they all have the same enclosure logical id.

Eric
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

I don't think its an LSI issue. The enclosure itself collects WWNs from the inserted drives. SAS drives have an agreed format which they present in a VPD INQUIRY, so they're easy to match up. SATA drives have a defined way of manufacturing this information (defined in the SAT standard). I just need to plug one of these in and make sure my enclosure does this correctly. Once that's done, I have to persuade the ATA people to add the correct form of translation for the VPD inquiry to libata's SAT layer ... then it should all "just work". So it's basically just some development work.

You could start by asking the sg_ses tools what you have. You need to find the sg device for your enclosure and then do

sg_ses --page=10 /dev/sg<n>

It will print out a list of what the enclosure thinks the addresses for the devices are. Then I just need to verify they're SAT format and away we can go.

James

--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Hallo James,

I just did it and attached the output
monosan:~ # sg_ses --page=10 /dev/sg19 > sg19.ses
<<<additional: response too short>>>

Thanks.
Lars

                            Informationstechnologie
Berlin-Brandenburgische Akademie der Wissenschaften
Jägerstrasse 22-23                     10117 Berlin
Tel.: +49 30 20370-352           http://www.bbaw.de