mdadm: Fix udev rule interaction with systemd

Good morning,

I discovered a problem with the interaction between systemd, mdadm and lvm: Every 5th boot or so, my lvm physical volume on top of a raid1 array would not activate. This turned out to be a race condition with mdadm.

When activating the raid array, two uevents happen: an "ADD" event and a "CHANGE" event. In both cases, the mdadm udev rules look at the md/array_state file in sysfs. However, I observed that an array with md/array_state == clean may still be inaccessible. So, in order to analyze the issue, I looked at the contents of the "md/array_state" and the "size" sysfs files. I discovered that during activation, my raid1 was in the following three states:

1) md/array_state: inactive, size: 0
2) md/array_state: clean, size: 0
3) md/array_state: clean, size: (the real array size)

So, if the ADD event happens in state 1) and the CHANGE event happens in state 3), everything is fine. However, sometimes the ADD event happens in state 2) and the CHANGE event in state 3). In the latter case, the device is activated in systemd during the ADD event, but udev internal settings like ID_FS_TYPE are not set (since the device is inaccessible). This causes the problem described above.

What I don't know is if
a) the transition from state 1) to 3) should be atomic in the kernel.
b) state 2) is legal and this needs to be worked around in udev.

The first patch fixes a problem where ENV{SYSTEMD_READY} is never set to 1 after being set to 0. The second patch checks whether the array has non-zero size before activating the systemd device unit (in case this is not a kernel bug).

In some cases, the rules set SYSTEMD_READY to 0 in order to tell systemd to delay activation of the .device unit. It should also be set to 1 when the array is ready.

---
 udev-md-raid-arrays.rules | 2 ++
 1 file changed, 2 insertions(+)

index c95ec7b..90d1aa5 100644
@@ -17,6 +17,8 @@ TEST!="md/array_state", ENV{SYSTEMD_READY}="0", GOTO="md_end"
 ATTR{md/array_state}=="|clear|inactive", ENV{SYSTEMD_READY}="0", GOTO="md_end"
 LABEL="md_ignore_state"
 
+ENV{SYSTEMD_READY}="1"
+
 IMPORT{program}="BINDIR/mdadm --detail --export $devnode"
 ENV{DEVTYPE}=="disk", ENV{MD_NAME}=="?*", SYMLINK+="disk/by-id/md-name-$env{MD_NAME}", OPTIONS+="string_escape=replace"
 ENV{DEVTYPE}=="disk", ENV{MD_UUID}=="?*", SYMLINK+="disk/by-id/md-uuid-$env{MD_UUID}"

2.5.0

When an array is activated, it is possible for the md/array_state property to be "clean", while the array is not accessible. This leads to a situation where systemd is told to activate the block device unit while it is not ready yet.

This causes a race condition in combination with the LVM2 udev rules: Since ID_FS_TYPE is not set yet, LVM2 does not look at the device. When a change event occurs later, ID_FS_TYPE is set to LVM2_member and LVM2 sets SYSTEMD_WANTS to call a service to activate the LVM volume. However, systemd has already activated the device at that time and the change to SYSTEMD_WANTS is ignored.

To solve this, in addition to checking md/array_state, also check the size property: If size is 0, the array is not ready and should not be treated as such.

---
 udev-md-raid-arrays.rules | 1 +
 1 file changed, 1 insertion(+)

index 90d1aa5..08f3cad 100644
@@ -15,6 +15,7 @@ ENV{DEVTYPE}=="partition", GOTO="md_ignore_state"
 ATTR{md/metadata_version}=="external:[A-Za-z]*", ATTR{md/array_state}=="inactive", GOTO="md_ignore_state"
 TEST!="md/array_state", ENV{SYSTEMD_READY}="0", GOTO="md_end"
 ATTR{md/array_state}=="|clear|inactive", ENV{SYSTEMD_READY}="0", GOTO="md_end"
+ATTR{size}=="0", ENV{SYSTEMD_READY}="0", GOTO="md_end"
 LABEL="md_ignore_state"
 
 ENV{SYSTEMD_READY}="1"

2.5.0