nfs-utils: rework device scanning in blkmapd
Subsystems (2)
cover nfs-utils: rework device scanning in blkmapd
This series reworks the way device scanning is done in blkmapd as a result of test results and discussions at the Fall 2010 Bakeathon.
We get rid of the config file and instead examine all block devices as listed in /sys/block and /proc/partitions (this is the way fdisk does it). Since we're looking for pnfs signatures, it usually doesn't hurt to examine a device that isn't part of a layout, and in general it's hard to tell ahead of time which devices should be considered, so I think having a config file does more harm than good.
We get rid of periodic (once a minute) device rescanning, and instead rescan only when the kernel asks for a layout. This will suffice for the common case, and works with all current known server implementations. A later patch will re-introduce rescanning during the life of a layout, but will be triggered by configuration change notifications from udev, rather than by expiration of an arbitrary time interval.
We get rid of the test for active/passive devices. This test really only works for EMC servers, and isn't even needed because the passive devices will be skipped over during signature detection. Just remove the detection code.
Jim Rees (4):
blkmapd: get rid of config file and instead examine all block devices
blkmapd: don't rescan periodically
blkmapd: don't use atomicio() where it's not needed
blkmapd: don't try to distinguish between active/passive devices
utils/blkmapd/Makefile.am | 2 - utils/blkmapd/cfg.c | 248 -------------------------------------- utils/blkmapd/cfg.h | 47 ------- utils/blkmapd/device-discovery.c | 141 ++++++++-------------- utils/blkmapd/device-discovery.h | 3 - utils/blkmapd/device-inq.c | 40 ++----- utils/blkmapd/device-process.c | 2 +- 7 files changed, 60 insertions(+), 423 deletions(-) delete mode 100644 utils/blkmapd/cfg.c delete mode 100644 utils/blkmapd/cfg.h
PATCH 1/4 blkmapd: get rid of config file and instead examine all block devices
Only examine actual block devices rather than everything in /dev. Since we're looking for pnfs signatures, it usually doesn't hurt to examine a device that isn't part of a layout, and in general it's hard to tell ahead of time which devices should be considered, so I think having a config file does more harm than good.
It may be useful later on to be able to blacklist some devices but I'd rather not do this through yet another config file. Maybe this can be done via udev rules.
--- utils/blkmapd/Makefile.am | 2 - utils/blkmapd/cfg.c | 248 -------------------------------------- utils/blkmapd/cfg.h | 47 ------- utils/blkmapd/device-discovery.c | 103 ++++------------ 4 files changed, 26 insertions(+), 374 deletions(-) delete mode 100644 utils/blkmapd/cfg.c delete mode 100644 utils/blkmapd/cfg.h
utils/blkmapd/Makefile.am +0 -2
@@ -7,13 +7,11 @@ sbin_PROGRAMS = blkmapd blkmapd_SOURCES = \ atomicio.c \ - cfg.c \ device-discovery.c \ device-inq.c \ device-process.c \ dm-device.c \ \ - cfg.h \ device-discovery.h blkmapd_LDADD = -ldevmapper ../../support/nfs/libnfs.a
utils/blkmapd/cfg.c deleted +0 -248
248 lines not on this page · raw message
utils/blkmapd/cfg.h deleted +0 -47
47 lines not on this page · raw message
utils/blkmapd/device-discovery.c +26 -77
156 lines not on this page · raw message
Signature · 1 line
1.7.0.4
PATCH 2/4 blkmapd: don't rescan periodically
In reply to Jim ReesPeriodic device rescanning is always going to be wrong, since the interval will either be too long and miss an important change, or too short and waste resources. It's not even needed in the normal case, only when the devices change during the life of a layout. For now, just rescan when the kernel asks for a layout. A later patch will re-introduce rescanning during the life of a layout, but will be triggered by configuration change notifications from udev, rather than by expiration of an arbitrary time interval.
--- utils/blkmapd/device-discovery.c | 14 ++++++++++---- utils/blkmapd/device-discovery.h | 2 -- 2 files changed, 10 insertions(+), 6 deletions(-)
utils/blkmapd/device-discovery.c +10 -4
@@ -308,6 +308,15 @@ int bl_disk_inquiry_process(int fd) head->status =BL_DEVICE_REQUEST_PROC ;switch (head->type) {case BL_DEVICE_MOUNT : +/* +* It shouldn't be necessary to discover devices here, since +* process_deviceinfo() will re-discover if it can't find +* the devices it needs. But in the case of multipath +* devices (ones that appear more than once, for example an +* active and a standby LUN), this will re-order them in the +* correct priority. +*/ +bl_discover_devices ();if (!process_deviceinfo (buf, buflen, &major, &minor)) { head->status =BL_DEVICE_REQUEST_ERR ; goto out;
@@ -329,7 +338,6 @@ int bl_disk_inquiry_process(int fd)case BL_DEVICE_UMOUNT :if (!dm_device_remove_all ((uint64_t *) buf)) head->status =BL_DEVICE_REQUEST_ERR ; -bl_discover_devices ();break ;default : head->status =BL_DEVICE_REQUEST_ERR ;
@@ -357,7 +365,6 @@ unsigned int bl_process_stop;int bl_run_disk_inquiry_process (int fd) {fd_set rset; -struct timeval tv;int ret; bl_process_stop =0 ;
@@ -368,8 +375,7 @@ int bl_run_disk_inquiry_process(int fd)FD_ZERO (&rset);FD_SET (fd, &rset); ret =0 ; - tv.tv_sec =BL_DEVICE_DISCOVERY_INTERVAL ; -switch (select (fd +1 , &rset,NULL ,NULL , &tv)) { +switch (select (fd +1 , &rset,NULL ,NULL ,NULL )) {case -1 :if (errno ==EINTR )continue ;
utils/blkmapd/device-discovery.h +0 -2
@@ -27,8 +27,6 @@#ifndef BL_DEVICE_DISCOVERY_H #define BL_DEVICE_DISCOVERY_H -#define BL_DEVICE_DISCOVERY_INTERVAL 60 -#include <stdint.h> #include <syslog.h>
Signature · 1 line
1.7.0.4
PATCH 3/4 blkmapd: don't use atomicio() where it's not needed
In reply to Jim Rees--- utils/blkmapd/device-process.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
utils/blkmapd/device-process.c +1 -1
@@ -121,7 +121,7 @@ read_cmp_blk_sig(const char *dev_name, struct bl_sig_comp *comp, goto error; } -if (atomicio (read, fd, sig, comp->bs_length) != comp->bs_length) { +if (read (fd, sig, comp->bs_length) != comp->bs_length) {BL_LOG_ERR ("File %s read error\n" , dev_name); goto error; }
Signature · 1 line
1.7.0.4
PATCH 4/4 blkmapd: don't try to distinguish between active/passive devices
In reply to Jim ReesThis test really only works for EMC servers, and isn't even needed because the passive devices will be skipped over during signature detection. Just remove the detection code.
--- utils/blkmapd/device-discovery.c | 24 ++++++++++++---------- utils/blkmapd/device-discovery.h | 1 - utils/blkmapd/device-inq.c | 40 +++++++++---------------------------- 3 files changed, 23 insertions(+), 42 deletions(-)
utils/blkmapd/device-discovery.c +13 -11
@@ -26,24 +26,25 @@* THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ -#include <stdlib.h> -#include <stdio.h> -#include <string.h> -#include <dirent.h> -#include <ctype.h> -#include <linux/kdev_t.h> #include <sys/types.h> #include <sys/stat.h> #include <sys/ioctl.h> #include <sys/mount.h> #include <sys/select.h> +#include <linux/kdev_t.h> +#include <scsi/scsi.h> +#include <scsi/scsi_ioctl.h> +#include <scsi/sg.h> + +#include <stdlib.h> +#include <stdio.h> +#include <string.h> +#include <dirent.h> +#include <ctype.h> #include <fcntl.h> #include <unistd.h> #include <libgen.h> #include <errno.h> -#include <scsi/scsi.h> -#include <scsi/scsi_ioctl.h> -#include <scsi/sg.h> #include "device-discovery.h"
@@ -127,7 +128,7 @@ void bl_add_disk(char *filepath)struct stat sb;off_t size =0 ;struct bl_serial *serial =NULL ; -enum bl_path_state_e ap_state =BL_PATH_STATE_PASSIVE ; +enum bl_path_state_e ap_state;struct bl_disk_path *diskpath =NULL , *path =NULL ;dev_t dev;
@@ -169,7 +170,6 @@ void bl_add_disk(char *filepath)return ; } -bldev_read_ap_state (fd, &ap_state);close (fd);BL_LOG_ERR ("%s: %s\n" , __func__, filepath);
@@ -180,6 +180,8 @@ void bl_add_disk(char *filepath)*/ if (strncmp (filepath,"/dev/mapper" ,11 ) ==0 ) ap_state =BL_PATH_STATE_PSEUDO ; +else + ap_state =BL_PATH_STATE_ACTIVE ;/* add path */ path =malloc (sizeof (struct bl_disk_path ));
utils/blkmapd/device-discovery.h +0 -1
@@ -151,7 +151,6 @@ uint64_t process_deviceinfo(const char *dev_addr_buf,extern ssize_t atomicio (ssize_t (*f) (int ,void *,size_t ),int fd,void *_s,size_t n);extern struct bl_serial *bldev_read_serial (int fd,const char *filename); -extern void bldev_read_ap_state (int fd,enum bl_path_state_e *ap_state_out);extern int bl_discover_devices (void ); #define BL_LOG_WARNING(fmt...) syslog(LOG_WARNING, fmt)
utils/blkmapd/device-inq.c +10 -30
@@ -28,23 +28,25 @@* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ -#include <stdlib.h> -#include <stdio.h> -#include <string.h> -#include <dirent.h> -#include <ctype.h> #include <sys/types.h> #include <sys/stat.h> #include <sys/ioctl.h> #include <sys/mount.h> #include <sys/select.h> +#include <scsi/scsi.h> +#include <scsi/scsi_ioctl.h> +#include <scsi/sg.h> + +#include <stdlib.h> +#include <stdio.h> +#include <string.h> +#include <dirent.h> +#include <ctype.h> #include <fcntl.h> #include <unistd.h> #include <libgen.h> #include <errno.h> -#include <scsi/scsi.h> -#include <scsi/scsi_ioctl.h> -#include <scsi/sg.h> +#include "device-discovery.h" #define DEF_ALLOC_LEN 255
@@ -147,28 +149,6 @@ int bldev_inquire_pages(int fd, int page, char **buffer)return status; } -/* For EMC multipath devices, use VPD page (0xc0) to get status. -* For other devices, return ACTIVE for now -*/ -void bldev_read_ap_state (int fd,enum bl_path_state_e *ap_state_out) -{ -int status =0 ; -char *buffer; - - *ap_state_out =BL_PATH_STATE_ACTIVE ; - - status =bldev_inquire_pages (fd,0xc0 , &buffer); -if (status) - goto out; - -if (buffer[4 ] <0x02 ) - *ap_state_out =BL_PATH_STATE_PASSIVE ; - out: -if (buffer) -free (buffer); -return ; -} -struct bl_serial *bldev_read_serial (int fd,const char *filename) {struct bl_serial *serial_out =NULL ;
Signature · 1 line
1.7.0.4
Benny Halevy
AppliedMerged, thanks!
Benny
Quoted 43 lines · Jim Rees
This series reworks the way device scanning is done in blkmapd as a result of test results and discussions at the Fall 2010 Bakeathon.
We get rid of the config file and instead examine all block devices as listed in
/sys/blockand/proc/partitions(this is the way fdisk does it). Since we're looking for pnfs signatures, it usually doesn't hurt to examine a device that isn't part of a layout, and in general it's hard to tell ahead of time which devices should be considered, so I think having a config file does more harm than good.We get rid of periodic (once a minute) device rescanning, and instead rescan only when the kernel asks for a layout. This will suffice for the common case, and works with all current known server implementations. A later patch will re-introduce rescanning during the life of a layout, but will be triggered by configuration change notifications from udev, rather than by expiration of an arbitrary time interval.
We get rid of the test for active/passive devices. This test really only works for EMC servers, and isn't even needed because the passive devices will be skipped over during signature detection. Just remove the detection code.
Jim Rees (4):
blkmapd: get rid of config file and instead examine all block devices
blkmapd: don't rescan periodically
blkmapd: don't useatomicio()where it's not needed
blkmapd: don't try to distinguish between active/passive devicesutils/blkmapd/Makefile.am | 2 - utils/blkmapd/cfg.c | 248 -------------------------------------- utils/blkmapd/cfg.h | 47 ------- utils/blkmapd/device-discovery.c | 141 ++++++++-------------- utils/blkmapd/device-discovery.h | 3 - utils/blkmapd/device-inq.c | 40 ++----- utils/blkmapd/device-process.c | 2 +- 7 files changed, 60 insertions(+), 423 deletions(-) delete mode 100644 utils/blkmapd/cfg.c delete mode 100644 utils/blkmapd/cfg.h--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html