mdadm-CI for-jes/20220620: patches for merge
cover mdadm-CI for-jes/20220620: patches for merge
Hi Jes,
The following patches are reviewed by me, and pass roughly testing by imsm array creation/stop/failure.
I post all the patches to mailing list, so that they may appear on patchwork list.
Please consider to take them to mdadm upstream.
Thanks.
Coly Li
---
Heming Zhao (1):
mdadm/super1: restore commit 45a87c2f31335 to fix clustered slot issue
Kinga Tanska (1):
util: replace ioctl use with function
Mariusz Tkaczyk (3):
imsm: introduce get_disk_slot_in_dev()
imsm: use same slot across container
imsm: block changing slots during creation
Nigel Croxon (1):
Revert "mdadm: fix coredump of mdadm --monitor -r"
ReadMe.c | 6 +- super-intel.c | 249 ++++++++++++++++++++++++++++--------------- super1.c | 12 ++- tests/09imsm-overlap | 28 ----- util.c | 2 +- 5 files changed, 181 insertions(+), 116 deletions(-) delete mode 100644 tests/09imsm-overlap
Signature · 1 line
2.35.3
PATCH 1/6 Revert "mdadm: fix coredump of mdadm --monitor -r"
From: Nigel Croxon <[email protected]>
This reverts commit 546047688e1c64638f462147c755b58119cabdc8.
mdadm: fix msg when removing a device using the short arg -r
The change from commit mdadm: fix coredump of mdadm --monitor -r broke the printing of the return message when passing -r to mdadm --manage, the removal of a device from an array.
If the current code reverts this commit, both issues are still fixed.
The original problem reported that the fix tried to address
was: The --monitor -r option requires a parameter, otherwise a null pointer will be manipulated when converting to integer data, and a core dump will appear.
The original problem was really fixed with:60815698c0a Refactor parse_num and use it to parse optarg. Which added a check for NULL in 'optarg' before moving it to the 'increments' variable.
New issue: When trying to remove a device using the short argument -r, instead of the long argument --remove, the output is empty. The problem started when commit 546047688e1c was added.
Steps to Reproduce:
1. create/assemble /dev/md0 device
2. mdadm --manage /dev/md0 -r /dev/vdxx
Actual results:
Nothing, empty output, nothing happens, the device is still connected to the array.
The output should have stated "mdadm: hot remove failed for /dev/vdxx: Device or resource busy", if the device was still active. Or it should remove the device and print a message:
mdadm: set /dev/vdd faulty in /dev/md0
mdadm: hot removed /dev/vdd from /dev/md0
The following commit should be reverted as it breaks mdadm --manage -r.
commit 546047688e1c64638f462147c755b58119cabdc8
Author: Wu Guanghao <[email protected]>
Date: Mon Aug 16 15:24:51 2021 +0800
mdadm: fix coredump of mdadm --monitor -r
--- ReadMe.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
ReadMe.c +3 -3
@@ -81,11 +81,11 @@ char Version[] = "mdadm - v" VERSION " - " VERS_DATE EXTRAVERSION "\n";* found, it is started. */ -char short_options[]="-ABCDEFGIQhVXYWZ:vqbc:i:l:p:m:r:n:x:u:c:d:z:U:N:safRSow1tye:k" ; +char short_options[]="-ABCDEFGIQhVXYWZ:vqbc:i:l:p:m:n:x:u:c:d:z:U:N:sarfRSow1tye:k:" ;char short_bitmap_options[]= -"-ABCDEFGIQhVXYWZ:vqb:c:i:l:p:m:r:n:x:u:c:d:z:U:N:sarfRSow1tye:k:" ; +"-ABCDEFGIQhVXYWZ:vqb:c:i:l:p:m:n:x:u:c:d:z:U:N:sarfRSow1tye:k:" ;char short_bitmap_auto_options[]= -"-ABCDEFGIQhVXYWZ:vqb:c:i:l:p:m:r:n:x:u:c:d:z:U:N:sa:rfRSow1tye:k:" ; +"-ABCDEFGIQhVXYWZ:vqb:c:i:l:p:m:n:x:u:c:d:z:U:N:sa:rfRSow1tye:k:" ;struct option long_options[] = { {"manage" ,0 ,0 , ManageOpt},
Signature · 1 line
2.35.3
PATCH 2/6 util: replace ioctl use with function
In reply to Coly LiFrom: Kinga Tanska <[email protected]>
Replace using of ioctl calling to get md array info with special function prepared to it.
--- util.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
util.c +1 -1
@@ -267,7 +267,7 @@ int md_array_active(int fd)* GET_ARRAY_INFO doesn't provide access to the proper state * information, so fallback to a basic check for raid_disks != 0 */ - ret =ioctl (fd,GET_ARRAY_INFO , &array); + ret =md_get_array_info (fd, &array); }return !ret;
Signature · 1 line
2.35.3
PATCH 3/6 mdadm/super1: restore commit 45a87c2f31335 to fix clustered slot issue
In reply to Coly LiFrom: Heming Zhao <[email protected]>
Commit 9d67f6496c71 ("mdadm:check the nodes when operate clustered array") modified assignment logic for st->nodes in write_bitmap1(), which introduced bitmap slot issue:
load_super1 didn't set up supertype.nodes, which made spare disk only have one slot info. Then it triggered kernel md_bitmap_load_sb to get wrong bitmap slot data.
For fixing this issue, there are two methods:
1> revert the related code of commit 9d67f6496c71. and restore the code
from former commit 45a87c2f31335 ("super1: add more checks for NodeNumUpdate option").
st->nodes value would be 0 & 1 under current code logic. i.e. When adding a spare disk, there is no place to init st->nodes, and the value is ZERO.
2> keep 9d67f6496c71, add additional ->nodes handling in load_super1(),
let load_super1 to set st->nodes when bitmap is BITMAP_MAJOR_CLUSTERED. Under current mdadm code logic, load_super1 will be called many times, any new code in load_super1 will cost mdadm running more time. And more reason is I prefer as much as possible to limit clustered code spreading in every corner.
So I used method <1> to fix this issue.
How to trigger:
dd if=/dev/zero bs=1M count=1 oflag=direct of=/dev/sda
dd if=/dev/zero bs=1M count=1 oflag=direct of=/dev/sdb
dd if=/dev/zero bs=1M count=1 oflag=direct of=/dev/sdc mdadm -C /dev/md0 -b clustered -e 1.2 -n 2 -l mirror /dev/sda /dev/sdb mdadm -a /dev/md0 /dev/sdc
mdadm /dev/md0 --fail /dev/sda
mdadm /dev/md0 --remove /dev/sda
mdadm -Ss
mdadm -A /dev/md0 /dev/sdb /dev/sdc
the output of current "mdadm -X /dev/sdc":
(there should be (by default) 4 slot info for correct output)
```
Filename : /dev/sdc
Magic : 6d746962
Version : 5
UUID : a74642f8:a6b1fba8:58e1f8db:cfe7b082
Events : 29
Events Cleared : 0
State : OK
Chunksize : 64 MB
Daemon : 5s flush period
Write Mode : Normal
Sync Size : 306176 (299.00 MiB 313.52 MB)
Bitmap : 5 bits (chunks), 5 dirty (100.0%)
```And mdadm later operations will trigger kernel output error message:
(triggered by "mdadm -A /dev/md0 /dev/sdb /dev/sdc")
``` kernel: md0: invalid bitmap file superblock: bad magic kernel: md_bitmap_copy_from_slot can't get bitmap from slot 1 kernel: md-cluster: Could not gather bitmaps from slot 1 kernel: md0: invalid bitmap file superblock: bad magic kernel: md_bitmap_copy_from_slot can't get bitmap from slot 2 kernel: md-cluster: Could not gather bitmaps from slot 2 kernel: md0: invalid bitmap file superblock: bad magic kernel: md_bitmap_copy_from_slot can't get bitmap from slot 3 kernel: md-cluster: Could not gather bitmaps from slot 3 kernel: md-cluster: failed to gather all resyn infos kernel: md0: detected capacity change from 0 to 612352 ```
--- super1.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
super1.c +11 -1
@@ -2674,7 +2674,17 @@ static int write_bitmap1(struct supertype *st, int fd, enum bitmap_update update }if (bms->version ==BITMAP_MAJOR_CLUSTERED ) { -if (__cpu_to_le32 (st->nodes) < bms->nodes) { +if (st->nodes ==1 ) { +/* the parameter for nodes is not valid */ +pr_err ("Warning: cluster-md at least needs two nodes\n" ); +return -EINVAL ; + }else if (st->nodes ==0 ) { +/* +* parameter "--nodes" is not specified, (eg, add a disk to +* clustered raid) +*/ +break ; + }else if (__cpu_to_le32 (st->nodes) < bms->nodes) {/* * Since the nodes num is not increased, no * need to check the space enough or not,
Signature · 1 line
2.35.3
PATCH 4/6 imsm: introduce get_disk_slot_in_dev()
In reply to Coly LiFrom: Mariusz Tkaczyk <[email protected]>
The routine was added to remove unnecessary get_imsm_dev() and get_imsm_map() calls, used only to determine disk slot.
Additionally, enum for IMSM return statues was added for further usage.
--- super-intel.c | 47 ++++++++++++++++++++++++++++++++++++----------- 1 file changed, 36 insertions(+), 11 deletions(-)
super-intel.c +36 -11
@@ -366,6 +366,18 @@ struct migr_record { };ASSERT_SIZE (migr_record,128 ) +/** +* enum imsm_status - internal IMSM return values representation. +* @STATUS_OK: function succeeded. +* @STATUS_ERROR: General error ocurred (not specified). +* +* Typedefed to imsm_status_t. + */ +typedef enum imsm_status { +IMSM_STATUS_ERROR =-1 , +IMSM_STATUS_OK =0 , +}imsm_status_t ; +struct md_list {/* usage marker: * 1: load metadata
@@ -1183,7 +1195,7 @@ static void set_imsm_ord_tbl_ent(struct imsm_map *map, int slot, __u32 ord) map->disk_ord_tbl[slot] =__cpu_to_le32 (ord); } -static int get_imsm_disk_slot (struct imsm_map *map,unsigned idx) +static int get_imsm_disk_slot (struct imsm_map *map,const unsigned int idx) {int slot;__u32 ord;
@@ -1194,7 +1206,7 @@ static int get_imsm_disk_slot(struct imsm_map *map, unsigned idx)return slot; } -return -1 ; +return IMSM_STATUS_ERROR ; }static int get_imsm_raid_level (struct imsm_map *map)
@@ -1209,6 +1221,23 @@ static int get_imsm_raid_level(struct imsm_map *map)return map->raid_level; } +/** +* get_disk_slot_in_dev() - retrieve disk slot from &imsm_dev. +* @super: &intel_super pointer, not NULL. +* @dev_idx: imsm device index. +* @idx: disk index. +* +* Return: Slot on success, IMSM_STATUS_ERROR otherwise. +*/ +static int get_disk_slot_in_dev (struct intel_super *super,const __u8 dev_idx, +const unsigned int idx) +{ +struct imsm_dev *dev =get_imsm_dev (super, dev_idx); +struct imsm_map *map =get_imsm_map (dev,MAP_0 ); + +return get_imsm_disk_slot (map, idx); +} +static int cmp_extent (const void *av,const void *bv) {const struct extent *a = av;
@@ -1225,13 +1254,9 @@ static int count_memberships(struct dl *dl, struct intel_super *super)int memberships =0 ;int i; -for (i =0 ; i < super->anchor->num_raid_devs; i++) { -struct imsm_dev *dev =get_imsm_dev (super, i); -struct imsm_map *map =get_imsm_map (dev,MAP_0 ); - -if (get_imsm_disk_slot (map, dl->index) >=0 ) +for (i =0 ; i < super->anchor->num_raid_devs; i++) +if (get_disk_slot_in_dev (super, i, dl->index) >=0 ) memberships++; - }return memberships; }
@@ -1941,6 +1966,7 @@ void examine_migr_rec_imsm(struct intel_super *super)/* first map under migration */ map =get_imsm_map (dev,MAP_0 ); +if (map) slot =get_imsm_disk_slot (map, super->disks->index);if (map ==NULL || slot >1 || slot <0 ) {
@@ -9655,10 +9681,9 @@ static int apply_update_activate_spare(struct imsm_update_activate_spare *u,/* count arrays using the victim in the metadata */ found =0 ;for (a = active_array; a ; a = a->next) { - dev =get_imsm_dev (super, a->info.container_member); - map =get_imsm_map (dev,MAP_0 ); +int dev_idx = a->info.container_member; -if (get_imsm_disk_slot (map, victim) >=0 ) +if (get_disk_slot_in_dev (super, dev_idx, victim) >=0 ) found++; }
Signature · 1 line
2.35.3
PATCH 5/6 imsm: use same slot across container
In reply to Coly LiFrom: Mariusz Tkaczyk <[email protected]>
Autolayout relies on drives order on super->disks list, but it is not quaranted by readdir() in sysfs_read(). As a result drive could be put in different slot in second volume.
Make it consistent by reffering to first volume, if exists.
Use enum imsm_status to unify error handling.
--- super-intel.c | 169 ++++++++++++++++++++++++++++++++------------------ 1 file changed, 108 insertions(+), 61 deletions(-)
super-intel.c +108 -61
@@ -7522,11 +7522,27 @@ static int validate_geometry_imsm_volume(struct supertype *st, int level,return 1 ; } -static int imsm_get_free_size (struct supertype *st,int raiddisks, -unsigned long long size,int chunk, -unsigned long long *freesize) +/** +* imsm_get_free_size() - get the biggest, common free space from members. +* @super: &intel_super pointer, not NULL. +* @raiddisks: number of raid disks. +* @size: requested size, could be 0 (means max size). +* @chunk: requested chunk. +* @freesize: pointer for returned size value. +* +* Return: &IMSM_STATUS_OK or &IMSM_STATUS_ERROR. +* +* @freesize is set to meaningful value, this can be @size, or calculated +* max free size. +* super->create_offset value is modified and set appropriately in +* merge_extends() for further creation. +*/ +static imsm_status_t imsm_get_free_size (struct intel_super *super, +const int raiddisks, +unsigned long long size, +const int chunk, +unsigned long long *freesize) { -struct intel_super *super = st->sb;struct imsm_super *mpb = super->anchor;struct dl *dl;int i;
@@ -7570,12 +7586,10 @@ static int imsm_get_free_size(struct supertype *st, int raiddisks,/* chunk is in K */ minsize = chunk *2 ; -if (cnt < raiddisks || - (super->orom && used && used != raiddisks) || - maxsize < minsize || - maxsize ==0 ) { +if (cnt < raiddisks || (super->orom && used && used != raiddisks) || + maxsize < minsize || maxsize ==0 ) {pr_err ("not enough devices with space to create array.\n" ); -return 0 ;/* No enough free spaces large enough */ +return IMSM_STATUS_ERROR ; }if (size ==0 ) {
@@ -7588,37 +7602,69 @@ static int imsm_get_free_size(struct supertype *st, int raiddisks, }if (mpb->num_raid_devs >0 && size && size != maxsize)pr_err ("attempting to create a second volume with size less then remaining space.\n" ); - cnt =0 ; -for (dl = super->disks; dl; dl = dl->next) -if (dl->e) - dl->raiddisk = cnt++; - *freesize = size;dprintf ("imsm: imsm_get_free_size() returns : %llu\n" , size); -return 1 ; +return IMSM_STATUS_OK ; } -static int reserve_space (struct supertype *st,int raiddisks, -unsigned long long size,int chunk, -unsigned long long *freesize) +/** +* autolayout_imsm() - automatically layout a new volume. +* @super: &intel_super pointer, not NULL. +* @raiddisks: number of raid disks. +* @size: requested size, could be 0 (means max size). +* @chunk: requested chunk. +* @freesize: pointer for returned size value. +* +* We are being asked to automatically layout a new volume based on the current +* contents of the container. If the parameters can be satisfied autolayout_imsm +* will record the disks, start offset, and will return size of the volume to +* be created. See imsm_get_free_size() for details. +* add_to_super() and getinfo_super() detect when autolayout is in progress. +* If first volume exists, slots are set consistently to it. +* +* Return: &IMSM_STATUS_OK on success, &IMSM_STATUS_ERROR otherwise. +* +* Disks are marked for creation via dl->raiddisk. +*/ +static imsm_status_t autolayout_imsm (struct intel_super *super, +const int raiddisks, +unsigned long long size,const int chunk, +unsigned long long *freesize) { -struct intel_super *super = st->sb; -struct dl *dl; -int cnt; -int rv =0 ; +int curr_slot =0 ; +struct dl *disk; +int vol_cnt = super->anchor->num_raid_devs; +imsm_status_t rv; - rv =imsm_get_free_size (st, raiddisks, size, chunk, freesize); -if (rv) { - cnt =0 ; -for (dl = super->disks; dl; dl = dl->next) -if (dl->e) - dl->raiddisk = cnt++; - rv =1 ; + rv =imsm_get_free_size (super, raiddisks, size, chunk, freesize); +if (rv !=IMSM_STATUS_OK ) +return IMSM_STATUS_ERROR ; + +for (disk = super->disks; disk; disk = disk->next) { +if (!disk->e) +continue ; + +if (curr_slot == raiddisks) +break ; + +if (vol_cnt ==0 ) { + disk->raiddisk = curr_slot; + }else { +int _slot =get_disk_slot_in_dev (super,0 , disk->index); + +if (_slot ==-1 ) { +pr_err ("Disk %s is not used in first volume, aborting\n" , + disk->devname); +return IMSM_STATUS_ERROR ; + } + disk->raiddisk = _slot; + } + curr_slot++; } -return rv; +return IMSM_STATUS_OK ; } static int validate_geometry_imsm(struct supertype *st, int level, int layout,
@@ -7654,35 +7700,35 @@ static int validate_geometry_imsm(struct supertype *st, int level, int layout, }if (!dev) { -if (st->sb) { -struct intel_super *super = st->sb; -if (!validate_geometry_imsm_orom (st->sb, level, layout, - raiddisks, chunk, size, - verbose)) +struct intel_super *super = st->sb; + +/* +* Autolayout mode, st->sb and freesize must be set. +*/ +if (!super || !freesize) { +pr_vrb ("freesize and superblock must be set for autolayout, aborting\n" ); +return 1 ; + } + +if (!validate_geometry_imsm_orom (st->sb, level, layout, + raiddisks, chunk, size, + verbose)) +return 0 ; + +if (super->orom) { +imsm_status_t rv; +int count =count_volumes (super->hba, super->orom->dpa, + verbose); +if (super->orom->vphba <= count) { +pr_vrb ("platform does not support more than %d raid volumes.\n" , + super->orom->vphba);return 0 ; -/* we are being asked to automatically layout a -* new volume based on the current contents of -* the container. If the the parameters can be -* satisfied reserve_space will record the disks, -* start offset, and size of the volume to be -* created. add_to_super and getinfo_super -* detect when autolayout is in progress. -*/ -/* assuming that freesize is always given when array is -created */ -if (super->orom && freesize) { -int count; - count =count_volumes (super->hba, - super->orom->dpa, verbose); -if (super->orom->vphba <= count) { -pr_vrb ("platform does not support more than %d raid volumes.\n" , - super->orom->vphba); -return 0 ; - } } -if (freesize) -return reserve_space (st, raiddisks, size, - *chunk, freesize); + + rv =autolayout_imsm (super, raiddisks, size, *chunk, + freesize); +if (rv !=IMSM_STATUS_OK ) +return 0 ; }return 1 ; }
@@ -11538,7 +11584,7 @@ enum imsm_reshape_type imsm_analyze_change(struct supertype *st,unsigned long long current_size;unsigned long long free_size;unsigned long long max_size; -int rv; +imsm_status_t rv;getinfo_super_imsm_volume (st, &info,NULL ); if (geo->level != info.array.level && geo->level >= 0 &&
@@ -11657,9 +11703,10 @@ enum imsm_reshape_type imsm_analyze_change(struct supertype *st, }/* check the maximum available size */ - rv =imsm_get_free_size (st, dev->vol.map->num_members, -0 , chunk, &free_size); -if (rv ==0 ) + rv =imsm_get_free_size (super, dev->vol.map->num_members, +0 , chunk, &free_size); + +if (rv !=IMSM_STATUS_OK )/* Cannot find maximum available space */ max_size =0 ;
Signature · 1 line
2.35.3
PATCH 6/6 imsm: block changing slots during creation
In reply to Coly LiFrom: Mariusz Tkaczyk <[email protected]>
If user specifies drives for array creation, then slot order across volumes is not preserved.
Ideally, it should be checked in validate_geometry() but it is not possible in current implementation (order is determined later). Add verification in add_to_super_imsm_volume() and throw error if mismatch is detected.
IMSM allows to use only same members within container. This is not hardware dependency but metadata limitation. Therefore, 09-imsm-overlap test is removed. Testing it is pointless. After this patch, creation in this scenario is blocked. Offset verification is covered in other tests.
--- super-intel.c | 33 ++++++++++++++++++++++----------- tests/09imsm-overlap | 28 ---------------------------- 2 files changed, 22 insertions(+), 39 deletions(-) delete mode 100644 tests/09imsm-overlap
super-intel.c +22 -11
@@ -5789,6 +5789,10 @@ static int add_to_super_imsm_volume(struct supertype *st, mdu_disk_info_t *dk,struct imsm_map *map;struct dl *dl, *df;int slot; +int autolayout =0 ; + +if (!is_fd_valid (fd)) + autolayout =1 ; dev =get_imsm_dev (super, super->current_vol); map =get_imsm_map (dev,MAP_0 );
@@ -5799,25 +5803,32 @@ static int add_to_super_imsm_volume(struct supertype *st, mdu_disk_info_t *dk,return 1 ; } -if (!is_fd_valid (fd)) { -/* we're doing autolayout so grab the pre-marked (in -* validate_geometry) raid_disk -*/ -for (dl = super->disks; dl; dl = dl->next) +for (dl = super->disks; dl ; dl = dl->next) { +if (autolayout) {if (dl->raiddisk == dk->raid_disk)break ; - }else { -for (dl = super->disks; dl ; dl = dl->next) -if (dl->major == dk->major && - dl->minor == dk->minor) -break ; + }else if (dl->major == dk->major && dl->minor == dk->minor) +break ; }if (!dl) { -pr_err ("%s is not a member of the same container\n" , devname); +if (!autolayout) +pr_err ("%s is not a member of the same container.\n" , + devname);return 1 ; } +if (!autolayout && super->current_vol >0 ) { +int _slot =get_disk_slot_in_dev (super,0 , dl->index); + +if (_slot != dk->raid_disk) { +pr_err ("Member %s is in %d slot for the first volume, but is in %d slot for a new volume.\n" , + dl->devname, _slot, dk->raid_disk); +pr_err ("Raid members are in different order than for the first volume, aborting.\n" ); +return 1 ; + } + } +if (mpb->num_disks ==0 )if (!get_dev_sector_size (dl->fd, dl->devname, &super->sector_size))
tests/09imsm-overlap deleted +0 -28
@@ -1,28 +0,0 @@ - -. tests/env-imsm-template - -# create raid arrays with varying degress of overlap -mdadm -CR $container -e imsm -n 6 $dev0 $dev1 $dev2 $dev3 $dev4 $dev5 -imsm_check container 6 - -size=1024 -level=1 -num_disks=2 -mdadm -CR $member0 $dev0 $dev1 -n $num_disks -l $level -z $size -mdadm -CR $member1 $dev1 $dev2 -n $num_disks -l $level -z $size -mdadm -CR $member2 $dev2 $dev3 -n $num_disks -l $level -z $size -mdadm -CR $member3 $dev3 $dev4 -n $num_disks -l $level -z $size -mdadm -CR $member4 $dev4 $dev5 -n $num_disks -l $level -z $size - -udevadm settle - -offset=0 -imsm_check member $member0 $num_disks $level $size 1024 $offset -offset=$((offset+size+4096)) -imsm_check member $member1 $num_disks $level $size 1024 $offset -offset=$((offset+size+4096)) -imsm_check member $member2 $num_disks $level $size 1024 $offset -offset=$((offset+size+4096)) -imsm_check member $member3 $num_disks $level $size 1024 $offset -offset=$((offset+size+4096)) -imsm_check member $member4 $num_disks $level $size 1024 $offset
Signature · 1 line
2.35.3