Skip to contentSkip to conversation

RAID0 DeviceDisappeared event happen when restart mdmonitor service

DISCUSSION Johnson Yan 3 participants 8 messages Sep 14, 2012 – Oct 24, 2012 MD RAID

lore
Share & tools
lore
#0 lore raw

Hi All,
When I restart mdmonitor service, a notify email will be received, which indicate DeviceDisappeared event happened for RAID0, is this phenomenon reasonable? Could anyone explain this?

The email info as below
......
Host : host3.com
Event : DeviceDisappeared
MD Device : /dev/md1
......

[root@host3 ~]# service mdmonitor restart

Killing mdmonitor:                                         [  OK  ]
Starting mdmonitor:                                        [  OK  ]

[root@host3 ~]#
[root@host3 ~]# cat /proc/mdstat
Personalities : [raid1] [raid0]
md1 : active raid0 sda1[1] sdb1[0]

      927933312 blocks super 1.2 64k chunks

md2 : active raid1 sdc1[1] sde1[0]

      292836160 blocks super 1.2 [2/2] [UU]
      bitmap: 0/2 pages [0KB], 131072KB chunk

md0 : active raid1 sdf1[0] sdg1[1]

      488254272 blocks super 1.2 [2/2] [UU]
      bitmap: 0/2 pages [0KB], 131072KB chunk

unused devices: <none>
[root@host3 ~]#
[root@host3 ~]# uname -r
3.2.28-11.el6.x86_64
[root@host3 ~]#
[root@host3 ~]# mdadm -V
mdadm - v3.2.5 - 18th May 2012
[root@host3 ~]#

Thanks in advance.
Johnson Yan

#1 lore raw

I check into code in mdmonitor.c in function check_array:

	fd = open(dev, O_RDONLY);
	if (fd < 0) {
		if (!st->err)
			alert("DeviceDisappeared", dev, NULL, ainfo);
		st->err=1;
		return 0;
	}
	fcntl(fd, F_SETFD, FD_CLOEXEC);
	if (ioctl(fd, GET_ARRAY_INFO, &array)<0) {
		if (!st->err)
			alert("DeviceDisappeared", dev, NULL, ainfo);
		st->err=1;
		close(fd);
		return 0;
	}
	/* It's much easier to list what array levels can't
	 * have a device disappear than all of them that can
	 */
	if (array.level == 0 || array.level == -1) {
		if (!st->err)
			alert("DeviceDisappeared", dev, "Wrong-Level", ainfo);
		st->err = 1;
		close(fd);
		return 0;
	}

I suspect above code lead to the DeviceDisapeared event you see, not sure why this get GET_ARRAY_INFO return array.level == 0 means Device Disappeared?

Anyone who more familiar with the logic may gave a comment?

Best regards!

Jack

2012/9/14 Johnson Yan <[email protected]>:

Quoted 48 lines

Hi All,
When I restart mdmonitor service, a notify email will be received, which indicate DeviceDisappeared event happened for RAID0, is this phenomenon reasonable? Could anyone explain this?

The email info as below
......
Host : host3.com
Event : DeviceDisappeared
MD Device : /dev/md1
......

[root@host3 ~]# service mdmonitor restart

Killing mdmonitor:                                         [  OK  ]
Starting mdmonitor:                                        [  OK  ]

[root@host3 ~]#
[root@host3 ~]# cat /proc/mdstat
Personalities : [raid1] [raid0]
md1 : active raid0 sda1[1] sdb1[0]

      927933312 blocks super 1.2 64k chunks

md2 : active raid1 sdc1[1] sde1[0]

      292836160 blocks super 1.2 [2/2] [UU]
      bitmap: 0/2 pages [0KB], 131072KB chunk

md0 : active raid1 sdf1[0] sdg1[1]

      488254272 blocks super 1.2 [2/2] [UU]
      bitmap: 0/2 pages [0KB], 131072KB chunk

unused devices: <none>
[root@host3 ~]#
[root@host3 ~]# uname -r
3.2.28-11.el6.x86_64
[root@host3 ~]#
[root@host3 ~]# mdadm -V
mdadm - v3.2.5 - 18th May 2012
[root@host3 ~]#

Thanks in advance.
Johnson Yan

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#2 lore raw

Hi Jack,
Thanks for your feedback.

This event should be caused by following code, due to the field "Wrong-Level" can be found in the notify email.

	/* It's much easier to list what array levels can't
	 * have a device disappear than all of them that can
	 */
	if (array.level == 0 || array.level == -1) {
		if (!st->err)
			alert("DeviceDisappeared", dev, "Wrong-Level", ainfo);
		st->err = 1;
		close(fd);
		return 0;
	}

Best Regards,
Johnson

-----Original Message-----
From: Jack Wang [mailto:[email protected]]
Sent: Monday, September 17, 2012 10:31 AM
To: Johnson Yan
Cc: [email protected]
Subject: Re: RAID0 DeviceDisappeared event happen when restart mdmonitor service

I check into code in mdmonitor.c in function check_array:

	fd = open(dev, O_RDONLY);
	if (fd < 0) {
		if (!st->err)
			alert("DeviceDisappeared", dev, NULL, ainfo);
		st->err=1;
		return 0;
	}
	fcntl(fd, F_SETFD, FD_CLOEXEC);
	if (ioctl(fd, GET_ARRAY_INFO, &array)<0) {
		if (!st->err)
			alert("DeviceDisappeared", dev, NULL, ainfo);
		st->err=1;
		close(fd);
		return 0;
	}
	/* It's much easier to list what array levels can't
	 * have a device disappear than all of them that can
	 */
	if (array.level == 0 || array.level == -1) {
		if (!st->err)
			alert("DeviceDisappeared", dev, "Wrong-Level", ainfo);
		st->err = 1;
		close(fd);
		return 0;
	}

I suspect above code lead to the DeviceDisapeared event you see, not sure why this get GET_ARRAY_INFO return array.level == 0 means Device Disappeared?

Anyone who more familiar with the logic may gave a comment?

Best regards!

Jack

2012/9/14 Johnson Yan <[email protected]>:

Quoted 48 lines

Hi All,
When I restart mdmonitor service, a notify email will be received, which indicate DeviceDisappeared event happened for RAID0, is this phenomenon reasonable? Could anyone explain this?

The email info as below
......
Host : host3.com
Event : DeviceDisappeared
MD Device : /dev/md1
......

[root@host3 ~]# service mdmonitor restart

Killing mdmonitor:                                         [  OK  ]
Starting mdmonitor:                                        [  OK  ]

[root@host3 ~]#
[root@host3 ~]# cat /proc/mdstat
Personalities : [raid1] [raid0]
md1 : active raid0 sda1[1] sdb1[0]

      927933312 blocks super 1.2 64k chunks

md2 : active raid1 sdc1[1] sde1[0]

      292836160 blocks super 1.2 [2/2] [UU]
      bitmap: 0/2 pages [0KB], 131072KB chunk

md0 : active raid1 sdf1[0] sdg1[1]

      488254272 blocks super 1.2 [2/2] [UU]
      bitmap: 0/2 pages [0KB], 131072KB chunk

unused devices: <none>
[root@host3 ~]#
[root@host3 ~]# uname -r
3.2.28-11.el6.x86_64
[root@host3 ~]#
[root@host3 ~]# mdadm -V
mdadm - v3.2.5 - 18th May 2012
[root@host3 ~]#

Thanks in advance.
Johnson Yan

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#3 lore raw
In reply to Jack Wang
Quoted 28 lines · On Mon, 17 Sep 2012 10:30:49 +0800 Jack Wang <[email protected]> wrote:

I check into code in mdmonitor.c in function check_array:

	fd = open(dev, O_RDONLY);
	if (fd < 0) {
		if (!st->err)
			alert("DeviceDisappeared", dev, NULL, ainfo);
		st->err=1;
		return 0;
	}
	fcntl(fd, F_SETFD, FD_CLOEXEC);
	if (ioctl(fd, GET_ARRAY_INFO, &array)<0) {
		if (!st->err)
			alert("DeviceDisappeared", dev, NULL, ainfo);
		st->err=1;
		close(fd);
		return 0;
	}
	/* It's much easier to list what array levels can't
	 * have a device disappear than all of them that can
	 */
	if (array.level == 0 || array.level == -1) {
		if (!st->err)
			alert("DeviceDisappeared", dev, "Wrong-Level", ainfo);
		st->err = 1;
		close(fd);
		return 0;
	}
Quoted 5 lines

I suspect above code lead to the DeviceDisapeared event you see, not sure why this get GET_ARRAY_INFO return array.level == 0 means Device Disappeared?

Anyone who more familiar with the logic may gave a comment?

It isn't possible or meaningful to monitor RAID0 devices - there is nothing to see, nothing that can happen, nothing to report.

mdadm will normally completely ignore RAID0 devices.

If you run
mdadm --monitor /dev/md1

when md1 is a RAID0 device, then it will be required to actively forget about it, so it reports that the device has disappeared (because it was on the list, but now isn't).

This shouldn't happen if you run "mdadm --monitor --scan" which is the standard usage. It that what is happening?

NeilBrown

Quoted 59 lines

Best regards!

Jack

2012/9/14 Johnson Yan <[email protected]>:

Quoted 40 lines

Hi All,
When I restart mdmonitor service, a notify email will be received, which indicate DeviceDisappeared event happened for RAID0, is this phenomenon reasonable? Could anyone explain this?

The email info as below
......
Host : host3.com
Event : DeviceDisappeared
MD Device : /dev/md1
......

[root@host3 ~]# service mdmonitor restart

Killing mdmonitor:                                         [  OK  ]
Starting mdmonitor:                                        [  OK  ]

[root@host3 ~]#
[root@host3 ~]# cat /proc/mdstat
Personalities : [raid1] [raid0]
md1 : active raid0 sda1[1] sdb1[0]

      927933312 blocks super 1.2 64k chunks

md2 : active raid1 sdc1[1] sde1[0]

      292836160 blocks super 1.2 [2/2] [UU]
      bitmap: 0/2 pages [0KB], 131072KB chunk

md0 : active raid1 sdf1[0] sdg1[1]

      488254272 blocks super 1.2 [2/2] [UU]
      bitmap: 0/2 pages [0KB], 131072KB chunk

unused devices: <none>
[root@host3 ~]#
[root@host3 ~]# uname -r
3.2.28-11.el6.x86_64
[root@host3 ~]#
[root@host3 ~]# mdadm -V
mdadm - v3.2.5 - 18th May 2012
[root@host3 ~]#

Quoted 8 lines

Thanks in advance.
Johnson Yan

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw

2012/9/17 NeilBrown <[email protected]>:

Quoted 46 lines
Quoted 28 lines · On Mon, 17 Sep 2012 10:30:49 +0800 Jack Wang <[email protected]> wrote:

I check into code in mdmonitor.c in function check_array:

      fd = open(dev, O_RDONLY);
      if (fd < 0) {
              if (!st->err)
                      alert("DeviceDisappeared", dev, NULL, ainfo);
              st->err=1;
              return 0;
      }
      fcntl(fd, F_SETFD, FD_CLOEXEC);
      if (ioctl(fd, GET_ARRAY_INFO, &array)<0) {
              if (!st->err)
                      alert("DeviceDisappeared", dev, NULL, ainfo);
              st->err=1;
              close(fd);
              return 0;
      }
      /* It's much easier to list what array levels can't
       * have a device disappear than all of them that can
       */
      if (array.level == 0 || array.level == -1) {
              if (!st->err)
                      alert("DeviceDisappeared", dev, "Wrong-Level", ainfo);
              st->err = 1;
              close(fd);
              return 0;
      }
Quoted 5 lines

I suspect above code lead to the DeviceDisapeared event you see, not sure why this get GET_ARRAY_INFO return array.level == 0 means Device Disappeared?

Anyone who more familiar with the logic may gave a comment?

It isn't possible or meaningful to monitor RAID0 devices - there is nothing to see, nothing that can happen, nothing to report.

mdadm will normally completely ignore RAID0 devices.

If you run
mdadm --monitor /dev/md1

Quoted 9 lines

when md1 is a RAID0 device, then it will be required to actively forget about it, so it reports that the device has disappeared (because it was on the list, but now isn't).

This shouldn't happen if you run "mdadm --monitor --scan" which is the standard usage. It that what is happening?

NeilBrown

Thanks Neil for always kindly help.
The device disappeared event report when restart mdmonitor service, which will mdadm with options PIDFILE=/var/run/mdadm/mdadm.pid PATH=/sbin:/usr/sbin:$PATH
RETVAL=0
OPTIONS="-F -y -f -s -i $PIDFILE"

And I checked man page of mdadm which clear claimed :

DeviceDisappeared
An md array which previously was configured appears to no longer be configured. (syslog priority: Critical)

If mdadm was told to monitor an array which is RAID0 or Lin- ear, then it will report DeviceDisappeared with the extra information Wrong-Level. This is because RAID0 and Linear do not support the device-failed, hot-spare and resync oper- ations which are monitored.

Thanks again for your time.

Jack

#5 lore raw

Hi Neil,
Thank you for the clarification.
I can remove the RAID0 MD's information from /etc/mdadm.conf to omit the monitor for RAID0, then so such event occurred when restart mdmonitor service, but the system won't re-assemble these RAID0 MD after every system reboot. Do you have any good advice?

Best Regards,
Johnson

Quoted 28 lines

Thanks Neil for always kindly help.
The device disappeared event report when restart mdmonitor service, which will mdadm with options PIDFILE=/var/run/mdadm/mdadm.pid PATH=/sbin:/usr/sbin:$PATH
RETVAL=0
OPTIONS="-F -y -f -s -i $PIDFILE"

And I checked man page of mdadm which clear claimed :

DeviceDisappeared
An md array which previously was configured
appears to no
longer be configured. (syslog priority: Critical)

If mdadm was told to monitor an array which is RAID0 or
Lin-
ear, then it will report DeviceDisappeared with the
extra
information Wrong-Level. This is because RAID0 and
Linear
do not support the device-failed, hot-spare and resync
oper-
ations which are monitored.

Thanks again for your time.

Jack

#6 lore raw
Quoted 4 lines · On Mon, 17 Sep 2012 11:23:40 +0800 "Johnson Yan" <[email protected]> wrote:

Hi Neil,
Thank you for the clarification.
I can remove the RAID0 MD's information from /etc/mdadm.conf to omit the monitor for RAID0, then so such event occurred when restart mdmonitor service, but the system won't re-assemble these RAID0 MD after every system reboot. Do you have any good advice?

I do now...
I've just committed the following patch which should suppress the message. It will be in mdadm-3.2.6 to be released soon.

NeilBrown

From f1661bd71bec6a8ae1919dc32a227c6fb37843f2 Mon Sep 17 00:00:00 2001
From: NeilBrown <[email protected]>
Date: Wed, 24 Oct 2012 13:09:09 +1100
Subject: [PATCH] Monitor: don't complain about non-monitorable arrays in
mdadm.conf

If we are asked to monitor a RAID0 or Linear - which cannot be monitored - we complain with "Device Disappeared .... Wrong-Level".

However if the RAID0 or Linear is being requested because it is in mdadm.conf then the message is inappropriate and confusing.

So track which arrays are added from the config file, and suppress that message in that case.

Monitor.c +3 -1
index 69cd8e0..c4d57c3 100644
@@ -37,6 +37,7 @@ struct state {
 	int err;
 	char *spare_group;
 	int active, working, failed, spare, raid;
+	int from_config;
 	int expected_spares;
 	int devstate[MAX_DISKS];
 	dev_t devid[MAX_DISKS];
@@ -178,6 +179,7 @@ int Monitor(struct mddev_dev *devlist,
 			st->next = statelist;
 			st->devnum = INT_MAX;
 			st->percent = RESYNC_UNKNOWN;
+			st->from_config = 1;
 			st->expected_spares = mdlist->spare_disks;
 			if (mdlist->spare_group)
 				st->spare_group = xstrdup(mdlist->spare_group);
@@ -475,7 +477,7 @@ static int check_array(struct state *st, struct mdstat_ent *mdstat,
 	 * have a device disappear than all of them that can
 	 */
 	if (array.level == 0 || array.level == -1) {
-		if (!st->err)
+		if (!st->err && !st->from_config)
 			alert("DeviceDisappeared", dev, "Wrong-Level", ainfo);
 		st->err = 1;
 		close(fd);
#7 lore raw

Hi Neil,
Got it, thank you for the patch.

Best Regards
Johnson

Quoted 12 lines

-----Original Message-----
From: NeilBrown [mailto:[email protected]]
Sent: Wednesday, October 24, 2012 10:11 AM
To: Johnson Yan
Cc: 'Jack Wang'; [email protected]
Subject: Re: RAID0 DeviceDisappeared event happen when restart mdmonitor service

On Mon, 17 Sep 2012 11:23:40 +0800 "Johnson Yan"
<[email protected]>
wrote:

Quoted 5 lines
Quoted 3 lines

Hi Neil,
Thank you for the clarification.
I can remove the RAID0 MD's information from /etc/mdadm.conf to omit the

monitor for RAID0, then so such event occurred when restart mdmonitor service, but the system won't re-assemble these RAID0 MD after every

system

Quoted 13 lines

reboot.

Quoted 1 line

Do you have any good advice?

I do now...
I've just committed the following patch which should suppress the message. It will be in mdadm-3.2.6 to be released soon.

NeilBrown

Quoted 8 lines

From f1661bd71bec6a8ae1919dc32a227c6fb37843f2 Mon Sep 17 00:00:00 2001
From: NeilBrown <[email protected]>
Date: Wed, 24 Oct 2012 13:09:09 +1100
Subject: [PATCH] Monitor: don't complain about non-monitorable arrays in mdadm.conf

If we are asked to monitor a RAID0 or Linear - which cannot be monitored -

we

Quoted 3 lines

complain with "Device Disappeared .... Wrong-Level".

However if the RAID0 or Linear is being requested because it is in

mdadm.conf

Quoted 21 lines

then the message is inappropriate and confusing.

So track which arrays are added from the config file, and suppress that message in that case.

Monitor.c +1 -0
index 69cd8e0..c4d57c3 100644
@@ -37,6 +37,7 @@ struct state {
 	int err;
 	char *spare_group;
 	int active, working, failed, spare, raid;
+	int from_config;
 	int expected_spares;
 	int devstate[MAX_DISKS];
 	dev_t devid[MAX_DISKS];

Monitor.c int Monitor(struct mddev_dev *devlist, · line 185

Quoted 7 lines
Monitor.c +1 -0
@@ -178,6 +179,7 @@ int Monitor(struct mddev_dev *devlist,
 			st->next = statelist;
 			st->devnum = INT_MAX;
 			st->percent = RESYNC_UNKNOWN;
+			st->from_config = 1;
 			st->expected_spares = mdlist->spare_disks;
 			if (mdlist->spare_group)
 				st->spare_group =

xstrdup(mdlist->spare_group); @@

Quoted 1 line

-475,7 +477,7 @@ static int check_array(struct state *st, struct

mdstat_ent

Monitor.c static int check_array(struct state *st, struct mdstat_ent *mdstat,

Quoted 7 lines

*mdstat,

Monitor.c +1 -1
 	 * have a device disappear than all of them that can
 	 */
 	if (array.level == 0 || array.level == -1) {
-		if (!st->err)
+		if (!st->err && !st->from_config)
 			alert("DeviceDisappeared", dev, "Wrong-Level",

ainfo);

Quoted 3 lines
Monitor.c +0 -0
 		st->err = 1;
 		close(fd);