Skip to contentSkip to conversation

HP SW support take 3

PATCH In review Mike Christie 2 participants 5 messages Jun 12, 2005 – Jun 14, 2005

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 HP SW support take 3

Fix locking bug. Will have to send the block layer stuff to Jens Axboe when we get hp-sw support finished. For now I included everything in one patch (reminder - don't use things like rd, dm, md, aoe, or anything without a real queue under dm-mpath if you are using a hw_hanlder that injects a failover command).

This was only tested with scsi_debug.

drivers/block/ll_rw_blk.c +3 -1
diff -Naurp linux-2.6.12-rc5.orig/drivers/block/ll_rw_blk.c linux-2.6.12-rc5/drivers/block/ll_rw_blk.c
@@ -2369,7 +2369,7 @@ void disk_round_stats(struct gendisk *di
 /*
  * queue lock must be held
  */
-static void __blk_put_request(request_queue_t *q, struct request *req)
+void __blk_put_request(request_queue_t *q, struct request *req)
 {
 	struct request_list *rl = req->rl;
 
@@ -2398,6 +2398,8 @@ static void __blk_put_request(request_qu
 	}
 }
 
+EXPORT_SYMBOL(__blk_put_request);
+
 void blk_put_request(struct request *req)
 {
 	/*
drivers/md/dm-hp-sw.c +160 -0

160 lines not on this page · raw message

drivers/md/Kconfig +9 -0
diff -Naurp linux-2.6.12-rc5.orig/drivers/md/Kconfig linux-2.6.12-rc5/drivers/md/Kconfig
@@ -236,5 +236,14 @@ config DM_MULTIPATH_EMC
 	---help---
 	  Multipath support for EMC CX/AX series hardware.
 
+config DM_MULTIPATH_HP_SW
+	tristate "HP StorageWorks and FSC FibreCat (EXPERIMENTAL)"
+	depends on DM_MULTIPATH && BLK_DEV_DM && EXPERIMENTAL
+	---help---
+	  Multipath support for HP StorageWorks and FSC FibreCat.
+          This was created with no knowledge of the HW other than
+          that it uses a START_STOP command for failover so it
+          is marked EXPERIMENTAL.
+
 endmenu
 
drivers/md/Makefile +1 -0
diff -Naurp linux-2.6.12-rc5.orig/drivers/md/Makefile linux-2.6.12-rc5/drivers/md/Makefile
@@ -33,6 +33,7 @@ obj-$(CONFIG_BLK_DEV_DM)	+= dm-mod.o
 obj-$(CONFIG_DM_CRYPT)		+= dm-crypt.o
 obj-$(CONFIG_DM_MULTIPATH)	+= dm-multipath.o dm-round-robin.o
 obj-$(CONFIG_DM_MULTIPATH_EMC)	+= dm-emc.o
+obj-$(CONFIG_DM_MULTIPATH_HP_SW) += dm-hp-sw.o
 obj-$(CONFIG_DM_SNAPSHOT)	+= dm-snapshot.o
 obj-$(CONFIG_DM_MIRROR)		+= dm-mirror.o
 obj-$(CONFIG_DM_ZERO)		+= dm-zero.o
include/linux/blkdev.h +1 -0
diff -Naurp linux-2.6.12-rc5.orig/include/linux/blkdev.h linux-2.6.12-rc5/include/linux/blkdev.h
@@ -540,6 +540,7 @@ extern void blk_unregister_queue(struct 
 extern void register_disk(struct gendisk *dev);
 extern void generic_make_request(struct bio *bio);
 extern void blk_put_request(struct request *);
+extern void __blk_put_request(request_queue_t *, struct request *);
 extern void blk_end_sync_rq(struct request *rq);
 extern void blk_attempt_remerge(request_queue_t *, struct request *);
 extern void __blk_attempt_remerge(request_queue_t *, struct request *);
#1 lore raw

I confirm a robust behaviour now with take 3.
Here is a take 4 that move to an appropriate "best effort mode", ie always return success.

The point is, whenever DM start submitting io the an asleep controler, precede with a START command. If it fails ... too bad, the consecutive io will fail and DM will try another PG. Had to do that because take 3 interprets an error where the controler switch have correctly happened.

I guess we could suppress the sense buffer allocation if this "best effort mode" is agreed on.

I also added path->dev->name to DMINFO() calls to make for a more informative output.
And packaged as an out-of-tree module, though it needs the blk patch you sent applied to the tree. (left bellow)

http://christophe.varoqui.free.fr/multipath-tools/dm-hp-sw.tar.bz2

Regards,
cvaroqui

Quoted 42 lines · Mike Christie

Fix locking bug. Will have to send the block layer stuff to Jens Axboe when we get hp-sw support finished. For now I included everything in one patch (reminder - don't use things like rd, dm, md, aoe, or anything without a real queue under dm-mpath if you are using a hw_hanlder that injects a failover command).

This was only tested with scsi_debug.

diff -Naurp linux-2.6.12-rc5.orig/drivers/block/ll_rw_blk.c linux-2.6.12-rc5/drivers/block/ll_rw_blk.c

drivers/block/ll_rw_blk.c +3 -1
@@ -2369,7 +2369,7 @@ void disk_round_stats(struct gendisk *di
 /*
  * queue lock must be held
  */
-static void __blk_put_request(request_queue_t *q, struct request *req)
+void __blk_put_request(request_queue_t *q, struct request *req)
 {
 	struct request_list *rl = req->rl;
 
@@ -2398,6 +2398,8 @@ static void __blk_put_request(request_qu
 	}
 }
 
+EXPORT_SYMBOL(__blk_put_request);
+
 void blk_put_request(struct request *req)
 {
 	/*

diff -Naurp linux-2.6.12-rc5.orig/include/linux/blkdev.h linux-2.6.12-rc5/include/linux/blkdev.h

include/linux/blkdev.h +1 -0
@@ -540,6 +540,7 @@ extern void blk_unregister_queue(struct 
 extern void register_disk(struct gendisk *dev);
 extern void generic_make_request(struct bio *bio);
 extern void blk_put_request(struct request *);
+extern void __blk_put_request(request_queue_t *, struct request *);
 extern void blk_end_sync_rq(struct request *rq);
 extern void blk_attempt_remerge(request_queue_t *, struct request *);
 extern void __blk_attempt_remerge(request_queue_t *, struct request *);
#2 lore raw
Quoted 5 lines · Christophe Varoqui

I confirm a robust behaviour now with take 3.
Here is a take 4 that move to an appropriate "best effort mode", ie always return success.

The point is, whenever DM start submitting io the an asleep controler, precede with a START command. If it fails ... too bad, the consecutive io will fail and DM will try another PG. Had to do that because take 3 interprets an error where the controler switch have correctly happened.

ok so is it if the controller is already failed over and we send the START command, rq->errors had some value so I ended up failing the operation, right? Let me send a patch that will dump the sense and rq->errors info out so we can see if there is some nice ASC/ASCQ value that will tell us this occured like with the LSI boxes.

Quoted 3 lines

I guess we could suppress the sense buffer allocation if this "best effort mode" is agreed on.

I also added path->dev->name to DMINFO() calls to make for a more informative output.

ok thanks I will add more output like that to other print statements.

#3 lore raw
Quoted 9 lines · Mike Christie
Quoted 5 lines · Christophe Varoqui

I confirm a robust behaviour now with take 3.
Here is a take 4 that move to an appropriate "best effort mode", ie always return success.

The point is, whenever DM start submitting io the an asleep controler, precede with a START command. If it fails ... too bad, the consecutive io will fail and DM will try another PG. Had to do that because take 3 interprets an error where the controler switch have correctly happened.

ok so is it if the controller is already failed over and we send the START command, rq->errors had some value so I ended up failing the operation, right? Let me send a patch that will dump the sense and rq->errors info out so we can see if there is some nice ASC/ASCQ value that will tell us this occured like with the LSI boxes.

oh wait, when I reread this I guess you are saying that I was always gettting a rq->error value when I send a START_STOP and it succeeds?

#4 lore raw
Quoted 13 lines · Mike Christie
Quoted 12 lines · Mike Christie
Quoted 9 lines · Christophe Varoqui

I confirm a robust behaviour now with take 3.
Here is a take 4 that move to an appropriate "best effort mode", ie always return success.

The point is, whenever DM start submitting io the an asleep controler, precede with a START command. If it fails ... too bad, the consecutive io will fail and DM will try another PG. Had to do that because take 3 interprets an error where the controler switch have correctly happened.

Quoted 9 lines
Quoted 5 lines

ok so is it if the controller is already failed over and we send the START command, rq->errors had some value so I ended up failing the operation, right? Let me send a patch that will dump the sense and rq->errors info out so we can see if there is some nice ASC/ASCQ value that will tell us this occured like with the LSI boxes.

oh wait, when I reread this I guess you are saying that I was always gettting a rq->error value when I send a START_STOP and it succeeds?

Here is a "sg_start -s 1" output that successfully activated the controler driving sdc :

[root@s64p17bibrn ~]# sg_start -s 1 /dev/sdc
sync_cache: scsi status: Check Condition
Current, Sense key: Not Ready
[valid=0] Info fld=0x9000000, Additional sense: Logical unit not ready, initializing cmd. required Raw sense data (in hex):
70 00 02 09 00 00 00 0a 00 00 00 00 04 02 00 93 01 00
plus...: Driver_status=0x08 (DRIVER_SENSE,SUGGEST_OK)
start_stop: scsi status: Check Condition
Current, Sense key: Not Ready
[valid=0] Info fld=0x9000000, Additional sense: Logical unit not ready, initializing cmd. required Raw sense data (in hex):
70 00 02 09 00 00 00 0a 00 00 00 00 04 02 00 93 01 00
plus...: Driver_status=0x08 (DRIVER_SENSE,SUGGEST_OK)

Note subsequent sg_start don't spit any output, the controler is already activated :

[root@s64p17bibrn ~]# sg_start -s 1 /dev/sdc [root@s64p17bibrn ~]#

Reactivating the first controler outs same sense as before :

[root@s64p17bibrn ~]# sg_start -s 1 /dev/sda
sync_cache: scsi status: Check Condition
Current, Sense key: Not Ready
[valid=0] Info fld=0x9000000, Additional sense: Logical unit not ready, initializing cmd. required Raw sense data (in hex):
70 00 02 09 00 00 00 0a 00 00 00 00 04 02 00 93 01 00
plus...: Driver_status=0x08 (DRIVER_SENSE,SUGGEST_OK)
start_stop: scsi status: Check Condition
Current, Sense key: Not Ready
[valid=0] Info fld=0x9000000, Additional sense: Logical unit not ready, initializing cmd. required Raw sense data (in hex):
70 00 02 09 00 00 00 0a 00 00 00 00 04 02 00 93 01 00
plus...: Driver_status=0x08 (DRIVER_SENSE,SUGGEST_OK)

Hope it clarifies a bit.
Let me know if I can provide more.

I'm surprised HP guys don't join the party.
Decoding the sense would be easier for them.
Lacking they knowledge, I went for the best effort mode, though I agree it would be better to decode the sense.

Regards,
cvaroqui