udev-cache fixes
v3 cover udev-cache fixes
This is v3 of the udev-cache changes previously submitted by Ben Shelton and myself last summer. This is primarily a cleanup of the patches that didn't make it into dizzy; the only significant behavioral difference is that I'm now making use of udevadm control --stop-exec-queue and udevadm control --start-exec-queue to completely close the race condition between sysconf generation and udev-cache tarball generation.
The following changes since commit 219e793907406eabf632e784e3a11ab9acb77cfb:
qemu/libc-package: Fix qemu option handling (2014-12-05 17:59:42 +0000)
are available in the git repository at:
git://github.com/rtollert/openembedded-core dev/rtollert/udev-cache-9
https://github.com/rtollert/openembedded-core/tree/dev/rtollert/udev-cache-9
Richard Tollerton (9):
udev-cache: stop race between sysconf and cache generation
udev-cache: replace readfiles() with cmp
udev-cache: don't generate sysconf twice
udev-cache: refactor sysconf generation
udev-cache: Clean up message when cache is invalidated
udev-cache: always warn on console if invalidated
udev-cache: invalidate on rules.d changes
udev-cache: Update cache asynchronously
udev-cache: refactor conditionals and error handling
meta/recipes-core/udev/udev/init | 56 ++++++++++++++++++---------------- meta/recipes-core/udev/udev/udev-cache | 37 ++++++++++++++++++++-- 2 files changed, 63 insertions(+), 30 deletions(-)
Signature · 1 line
2.1.3
PATCH v3 1/9 udev-cache: stop race between sysconf and cache generation
The validity of the udev cache is determined by the sysconf file (/etc/udev/cache.data). Currently, there is a substantial delay between sysconf generation in /etc/init.d/udev and cache generation in /etc/init.d/udev-cache. If a hotplug event arrives in the middle of this, then the sysconf will be out of date with respect to the cache.
The solution is two-pronged. First, we minimize the race window by regenerating the sysconf immediately before the cache, in /etc/init.d/udev-cache. This allows us to kill the race entirely by stopping the udev event queue while the sysconf and cache are being generated.
--- meta/recipes-core/udev/udev/udev-cache | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
meta/recipes-core/udev/udev/udev-cache +18 -0
@@ -19,8 +19,22 @@ export TZ=/etc/localtime DEVCACHE_TMP="/dev/shm/udev-cache-tmp.tar" SYSCONF_CACHED="/etc/udev/cache.data" SYSCONF_TMP="/dev/shm/udev.cache" + +# A list of files which are used as a criteria to judge whether the udev cache could be reused. +CMP_FILE_LIST="/proc/version /proc/cmdline /proc/devices /proc/atags" [ -f /etc/default/udev-cache ] && . /etc/default/udev-cache +readfiles () { + READDATA="" + for filename in $@; do + if [ -r $filename ]; then + while read line; do + READDATA="$READDATA$line" + done < $filename + fi + done +} + if [ "$ROOTFS_READ_ONLY" = "yes" ]; then [ "$VERBOSE" != "no" ] && echo "udev-cache: read-only rootfs, skip generating udev-cache" exit 0
@@ -28,11 +42,15 @@ fi if [ "$DEVCACHE" != "" -a -e "$SYSCONF_TMP" ]; then echo "Populating dev cache" + udevadm control --stop-exec-queue + readfiles $CMP_FILE_LIST + echo "$READDATA" > "$SYSCONF_TMP" find /dev -xdev \( -type b -o -type c -o -type l \) | cut -c 2- \ | xargs tar cf "${DEVCACHE_TMP}" -T- gzip < "${DEVCACHE_TMP}" > "$DEVCACHE" rm -f "${DEVCACHE_TMP}" mv "$SYSCONF_TMP" "$SYSCONF_CACHED" + udevadm control --start-exec-queue fi exit 0
Signature · 1 line
2.1.3
PATCH v3 2/9 udev-cache: replace readfiles() with cmp
In reply to Richard TollertonCurrently, udev-cache system configurations are compared as shell string variables, read into memory with the readfiles() function. This is more complex, and significantly (27-41%) slower, than comparing them using cmp. (Performance was verified on both Cortex-A9 and Intel Nehalem systems.)
So just use cmp. This requires a few other small changes:
exclude /proc/atags from CMP_FILE_LIST if it doesn't exist to avoid errors in cat and cmp.
cmp -q doesn't exist in busybox, so instead, redirect output to /dev/null.
--- meta/recipes-core/udev/udev/init | 29 +++++++---------------------- meta/recipes-core/udev/udev/udev-cache | 17 +++-------------- 2 files changed, 10 insertions(+), 36 deletions(-)
meta/recipes-core/udev/udev/init +7 -22
@@ -20,17 +20,6 @@ SYSCONF_TMP="/dev/shm/udev.cache" [ -f /etc/udev/udev.conf ] && . /etc/udev/udev.conf [ -f /etc/default/rcS ] && . /etc/default/rcS -readfiles () { - READDATA="" - for filename in $@; do - if [ -r $filename ]; then - while read line; do - READDATA="$READDATA$line" - done < $filename - fi - done -} - kill_udevd () { pid=`pidof -x udevd` [ -n "$pid" ] && kill $pid
@@ -63,14 +52,12 @@ case "$1" in # Cache handling. # A list of files which are used as a criteria to judge whether the udev cache could be reused. - CMP_FILE_LIST="/proc/version /proc/cmdline /proc/devices /proc/atags" + CMP_FILE_LIST="/proc/version /proc/cmdline /proc/devices" + [ -f /proc/atags ] && CMP_FILE_LIST="$CMP_FILE_LIST /proc/atags" if [ "$DEVCACHE" != "" ]; then if [ -e $DEVCACHE ]; then - readfiles $CMP_FILE_LIST - NEWDATA="$READDATA" - readfiles "$SYSCONF_CACHED" - OLDDATA="$READDATA" - if [ "$OLDDATA" = "$NEWDATA" ]; then + cat -- "$CMP_FILE_LIST" > "$SYSCONF_TMP" + if cmp $SYSCONF_CACHED $SYSCONF_TMP >/dev/null; then tar xmf $DEVCACHE -C / -m not_first_boot=1 [ "$VERBOSE" != "no" ] && echo "udev: using cache file $DEVCACHE"
@@ -80,17 +67,15 @@ case "$1" in if [ "$VERBOSE" != "no" ]; then echo "udev: udev cache not used" echo "udev: we use $CMP_FILE_LIST as criteria to judge whether the cache /dev could be resued" - echo "udev: olddata: $OLDDATA" - echo "udev: newdata: $NEWDATA" + echo "udev: cached sysconf: $SYSCONF_CACHED" + echo "udev: current sysconf: $SYSCONF_TMP" fi - echo "$NEWDATA" > "$SYSCONF_TMP" fi else if [ "$ROOTFS_READ_ONLY" != "yes" ]; then # If rootfs is not read-only, it's possible that a new udev cache would be generated; # otherwise, we do not bother to read files. - readfiles $CMP_FILE_LIST - echo "$READDATA" > "$SYSCONF_TMP" + cat -- "$CMP_FILE_LIST" > "$SYSCONF_TMP" fi fi fi
meta/recipes-core/udev/udev/udev-cache +3 -14
@@ -21,20 +21,10 @@ SYSCONF_CACHED="/etc/udev/cache.data" SYSCONF_TMP="/dev/shm/udev.cache" # A list of files which are used as a criteria to judge whether the udev cache could be reused. -CMP_FILE_LIST="/proc/version /proc/cmdline /proc/devices /proc/atags" +CMP_FILE_LIST="/proc/version /proc/cmdline /proc/devices" +[ -f /proc/atags ] && CMP_FILE_LIST="$CMP_FILE_LIST /proc/atags" [ -f /etc/default/udev-cache ] && . /etc/default/udev-cache -readfiles () { - READDATA="" - for filename in $@; do - if [ -r $filename ]; then - while read line; do - READDATA="$READDATA$line" - done < $filename - fi - done -} - if [ "$ROOTFS_READ_ONLY" = "yes" ]; then [ "$VERBOSE" != "no" ] && echo "udev-cache: read-only rootfs, skip generating udev-cache" exit 0
@@ -43,8 +33,7 @@ fi if [ "$DEVCACHE" != "" -a -e "$SYSCONF_TMP" ]; then echo "Populating dev cache" udevadm control --stop-exec-queue - readfiles $CMP_FILE_LIST - echo "$READDATA" > "$SYSCONF_TMP" + cat -- $CMP_FILE_LIST > "$SYSCONF_TMP" find /dev -xdev \( -type b -o -type c -o -type l \) | cut -c 2- \ | xargs tar cf "${DEVCACHE_TMP}" -T- gzip < "${DEVCACHE_TMP}" > "$DEVCACHE"
Signature · 1 line
2.1.3
PATCH v3 3/9 udev-cache: don't generate sysconf twice
In reply to Richard TollertonThe udev initscript signals udev-cache to run by generating a new sysconf; but udev-cache now overwrites that with its own copy. To eliminate the needless sysconf generating in udev, we instead trigger udev-cache to run by touching a new file $DEVCACHE_REGEN.
--- meta/recipes-core/udev/udev/init | 5 ++++- meta/recipes-core/udev/udev/udev-cache | 4 +++- 2 files changed, 7 insertions(+), 2 deletions(-)
meta/recipes-core/udev/udev/init +4 -1
@@ -16,6 +16,7 @@ export TZ=/etc/localtime [ -x @UDEVD@ ] || exit 1 SYSCONF_CACHED="/etc/udev/cache.data" SYSCONF_TMP="/dev/shm/udev.cache" +DEVCACHE_REGEN="/dev/shm/udev-regen" # create to request cache regen [ -f /etc/default/udev-cache ] && . /etc/default/udev-cache [ -f /etc/udev/udev.conf ] && . /etc/udev/udev.conf [ -f /etc/default/rcS ] && . /etc/default/rcS
@@ -62,6 +63,7 @@ case "$1" in not_first_boot=1 [ "$VERBOSE" != "no" ] && echo "udev: using cache file $DEVCACHE" [ -e $SYSCONF_TMP ] && rm -f "$SYSCONF_TMP" + [ -e "$DEVCACHE_REGEN" ] && rm -f "$DEVCACHE_REGEN" else # Output detailed reason why the cached /dev is not used if [ "$VERBOSE" != "no" ]; then
@@ -70,12 +72,13 @@ case "$1" in echo "udev: cached sysconf: $SYSCONF_CACHED" echo "udev: current sysconf: $SYSCONF_TMP" fi + touch "$DEVCACHE_REGEN" fi else if [ "$ROOTFS_READ_ONLY" != "yes" ]; then # If rootfs is not read-only, it's possible that a new udev cache would be generated; # otherwise, we do not bother to read files. - cat -- "$CMP_FILE_LIST" > "$SYSCONF_TMP" + touch "$DEVCACHE_REGEN" fi fi fi
meta/recipes-core/udev/udev/udev-cache +3 -1
@@ -19,6 +19,7 @@ export TZ=/etc/localtime DEVCACHE_TMP="/dev/shm/udev-cache-tmp.tar" SYSCONF_CACHED="/etc/udev/cache.data" SYSCONF_TMP="/dev/shm/udev.cache" +DEVCACHE_REGEN="/dev/shm/udev-regen" # create to request cache regen # A list of files which are used as a criteria to judge whether the udev cache could be reused. CMP_FILE_LIST="/proc/version /proc/cmdline /proc/devices"
@@ -30,7 +31,7 @@ if [ "$ROOTFS_READ_ONLY" = "yes" ]; then exit 0 fi -if [ "$DEVCACHE" != "" -a -e "$SYSCONF_TMP" ]; then +if [ "$DEVCACHE" != "" -a -e "$DEVCACHE_REGEN" ]; then echo "Populating dev cache" udevadm control --stop-exec-queue cat -- $CMP_FILE_LIST > "$SYSCONF_TMP"
@@ -40,6 +41,7 @@ if [ "$DEVCACHE" != "" -a -e "$SYSCONF_TMP" ]; then rm -f "${DEVCACHE_TMP}" mv "$SYSCONF_TMP" "$SYSCONF_CACHED" udevadm control --start-exec-queue + rm -f "$DEVCACHE_REGEN" fi exit 0
Signature · 1 line
2.1.3
PATCH v3 4/9 udev-cache: refactor sysconf generation
In reply to Richard TollertonThe current system configuration needs to be generated both inside udev (to compare against the cached system configuration) and udev-cache (to regenerate the cached system configuration). Use a single function definition for this task, duplicated across both initscripts. This also allows administrators to modify it for machine-specific requirements.
--- meta/recipes-core/udev/udev/init | 15 +++++++++++---- meta/recipes-core/udev/udev/udev-cache | 8 +++++++- 2 files changed, 18 insertions(+), 5 deletions(-)
meta/recipes-core/udev/udev/init +11 -4
@@ -17,6 +17,16 @@ export TZ=/etc/localtime SYSCONF_CACHED="/etc/udev/cache.data" SYSCONF_TMP="/dev/shm/udev.cache" DEVCACHE_REGEN="/dev/shm/udev-regen" # create to request cache regen + +# A list of files which are used as a criteria to judge whether the udev cache could be reused. +CMP_FILE_LIST="/proc/version /proc/cmdline /proc/devices" +[ -f /proc/atags ] && CMP_FILE_LIST="$CMP_FILE_LIST /proc/atags" + +# Command to compute system configuration. +sysconf_cmd () { + cat -- $CMP_FILE_LIST +} + [ -f /etc/default/udev-cache ] && . /etc/default/udev-cache [ -f /etc/udev/udev.conf ] && . /etc/udev/udev.conf [ -f /etc/default/rcS ] && . /etc/default/rcS
@@ -52,12 +62,9 @@ case "$1" in mkdir -m 1777 -p /var/volatile/tmp # Cache handling. - # A list of files which are used as a criteria to judge whether the udev cache could be reused. - CMP_FILE_LIST="/proc/version /proc/cmdline /proc/devices" - [ -f /proc/atags ] && CMP_FILE_LIST="$CMP_FILE_LIST /proc/atags" if [ "$DEVCACHE" != "" ]; then if [ -e $DEVCACHE ]; then - cat -- "$CMP_FILE_LIST" > "$SYSCONF_TMP" + sysconf_cmd > "$SYSCONF_TMP" if cmp $SYSCONF_CACHED $SYSCONF_TMP >/dev/null; then tar xmf $DEVCACHE -C / -m not_first_boot=1
meta/recipes-core/udev/udev/udev-cache +7 -1
@@ -24,6 +24,12 @@ DEVCACHE_REGEN="/dev/shm/udev-regen" # create to request cache regen # A list of files which are used as a criteria to judge whether the udev cache could be reused. CMP_FILE_LIST="/proc/version /proc/cmdline /proc/devices" [ -f /proc/atags ] && CMP_FILE_LIST="$CMP_FILE_LIST /proc/atags" + +# Command to compute system configuration. +sysconf_cmd () { + cat -- $CMP_FILE_LIST +} + [ -f /etc/default/udev-cache ] && . /etc/default/udev-cache if [ "$ROOTFS_READ_ONLY" = "yes" ]; then
@@ -34,7 +40,7 @@ fi if [ "$DEVCACHE" != "" -a -e "$DEVCACHE_REGEN" ]; then echo "Populating dev cache" udevadm control --stop-exec-queue - cat -- $CMP_FILE_LIST > "$SYSCONF_TMP" + sysconf_cmd > "$SYSCONF_TMP" find /dev -xdev \( -type b -o -type c -o -type l \) | cut -c 2- \ | xargs tar cf "${DEVCACHE_TMP}" -T- gzip < "${DEVCACHE_TMP}" > "$DEVCACHE"
Signature · 1 line
2.1.3
PATCH v3 5/9 udev-cache: Clean up message when cache is invalidated
In reply to Richard TollertonReplace a bunch of echo's with a single cat<<EOF. Take this opportunity to more clearly communicate what is going on with the cache and what files are being looked at.
--- meta/recipes-core/udev/udev/init | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
meta/recipes-core/udev/udev/init +7 -4
@@ -74,10 +74,13 @@ case "$1" in else # Output detailed reason why the cached /dev is not used if [ "$VERBOSE" != "no" ]; then - echo "udev: udev cache not used" - echo "udev: we use $CMP_FILE_LIST as criteria to judge whether the cache /dev could be resued" - echo "udev: cached sysconf: $SYSCONF_CACHED" - echo "udev: current sysconf: $SYSCONF_TMP" + cat <<EOF +udev: Not using udev cache because of changes detected in the following files: +udev: $CMP_FILE_LIST +udev: The udev cache will be regenerated. To identify the detected changes, +udev: compare the cached sysconf at $SYSCONF_CACHED +udev: against the current sysconf at $SYSCONF_TMP +EOF fi touch "$DEVCACHE_REGEN" fi
Signature · 1 line
2.1.3
PATCH v3 6/9 udev-cache: always warn on console if invalidated
In reply to Richard TollertonFailure to use the udev cache is a significant enough impact to the boot time (possibly seconds) that it should always be reported on the console, regardless of the VERBOSE setting.
--- meta/recipes-core/udev/udev/init | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
meta/recipes-core/udev/udev/init +1 -3
@@ -73,15 +73,13 @@ case "$1" in [ -e "$DEVCACHE_REGEN" ] && rm -f "$DEVCACHE_REGEN" else # Output detailed reason why the cached /dev is not used - if [ "$VERBOSE" != "no" ]; then - cat <<EOF + cat <<EOF udev: Not using udev cache because of changes detected in the following files: udev: $CMP_FILE_LIST udev: The udev cache will be regenerated. To identify the detected changes, udev: compare the cached sysconf at $SYSCONF_CACHED udev: against the current sysconf at $SYSCONF_TMP EOF - fi touch "$DEVCACHE_REGEN" fi else
Signature · 1 line
2.1.3
PATCH v3 7/9 udev-cache: invalidate on rules.d changes
In reply to Richard TollertonPresently, the cache is not regenerated if udev rules are modified, which may cause the cache to preserve an old configuration. To fix, include the size, mtime, and filename of all udev rules in the system configuration.
This change requires stat. If busybox supplies stat, CONFIG_FEATURE_STAT_FORMAT must be enabled.
--- meta/recipes-core/udev/udev/init | 6 ++++++ meta/recipes-core/udev/udev/udev-cache | 5 +++++ 2 files changed, 11 insertions(+)
meta/recipes-core/udev/udev/init +6 -0
@@ -22,9 +22,14 @@ DEVCACHE_REGEN="/dev/shm/udev-regen" # create to request cache regen CMP_FILE_LIST="/proc/version /proc/cmdline /proc/devices" [ -f /proc/atags ] && CMP_FILE_LIST="$CMP_FILE_LIST /proc/atags" +# List of files whose metadata (size/mtime/name) will be included in cached +# system state. +META_FILE_LIST="lib/udev/rules.d/* etc/udev/rules.d/*" + # Command to compute system configuration. sysconf_cmd () { cat -- $CMP_FILE_LIST + stat -L -c '%s %Y %n' -- $META_FILE_LIST | awk -F/ '{print $1 " " $NF;}' } [ -f /etc/default/udev-cache ] && . /etc/default/udev-cache
@@ -76,6 +81,7 @@ case "$1" in cat <<EOF udev: Not using udev cache because of changes detected in the following files: udev: $CMP_FILE_LIST +udev: $META_FILE_LIST udev: The udev cache will be regenerated. To identify the detected changes, udev: compare the cached sysconf at $SYSCONF_CACHED udev: against the current sysconf at $SYSCONF_TMP
meta/recipes-core/udev/udev/udev-cache +5 -0
@@ -25,9 +25,14 @@ DEVCACHE_REGEN="/dev/shm/udev-regen" # create to request cache regen CMP_FILE_LIST="/proc/version /proc/cmdline /proc/devices" [ -f /proc/atags ] && CMP_FILE_LIST="$CMP_FILE_LIST /proc/atags" +# List of files whose metadata (size/mtime/name) will be included in cached +# system state. +META_FILE_LIST="lib/udev/rules.d/* etc/udev/rules.d/*" + # Command to compute system configuration. sysconf_cmd () { cat -- $CMP_FILE_LIST + stat -L -c '%s %Y %n' -- $META_FILE_LIST | awk -F/ '{print $1 " " $NF;}' } [ -f /etc/default/udev-cache ] && . /etc/default/udev-cache
Signature · 1 line
2.1.3
PATCH v3 8/9 udev-cache: Update cache asynchronously
In reply to Richard TollertonDon't hold up the boot while the cache is being updated.
--- meta/recipes-core/udev/udev/udev-cache | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-)
meta/recipes-core/udev/udev/udev-cache +11 -9
@@ -44,15 +44,17 @@ fi if [ "$DEVCACHE" != "" -a -e "$DEVCACHE_REGEN" ]; then echo "Populating dev cache" - udevadm control --stop-exec-queue - sysconf_cmd > "$SYSCONF_TMP" - find /dev -xdev \( -type b -o -type c -o -type l \) | cut -c 2- \ - | xargs tar cf "${DEVCACHE_TMP}" -T- - gzip < "${DEVCACHE_TMP}" > "$DEVCACHE" - rm -f "${DEVCACHE_TMP}" - mv "$SYSCONF_TMP" "$SYSCONF_CACHED" - udevadm control --start-exec-queue - rm -f "$DEVCACHE_REGEN" + ( + udevadm control --stop-exec-queue + sysconf_cmd > "$SYSCONF_TMP" + find /dev -xdev \( -type b -o -type c -o -type l \) | cut -c 2- \ + | xargs tar cf "${DEVCACHE_TMP}" -T- + gzip < "${DEVCACHE_TMP}" > "$DEVCACHE" + rm -f "${DEVCACHE_TMP}" + mv "$SYSCONF_TMP" "$SYSCONF_CACHED" + udevadm control --start-exec-queue + rm -f "$DEVCACHE_REGEN" + ) & fi exit 0
Signature · 1 line
2.1.3
PATCH v3 9/9 udev-cache: refactor conditionals and error handling
In reply to Richard TollertonMost of /etc/init.d/udev-cache is in a conditional block which can be replaced by a [ ... ] || exit 0 to reduce nesting.
This also provides an opportunity to add some additional messages when VERBOSE is set.
Capture and report errors encountered in the cache generation process, using set -e and trap EXIT. These errors were previously being ignored.
--- meta/recipes-core/udev/udev/udev-cache | 35 +++++++++++++++++++++------------- 1 file changed, 22 insertions(+), 13 deletions(-)
meta/recipes-core/udev/udev/udev-cache +22 -13
@@ -42,19 +42,28 @@ if [ "$ROOTFS_READ_ONLY" = "yes" ]; then exit 0 fi -if [ "$DEVCACHE" != "" -a -e "$DEVCACHE_REGEN" ]; then - echo "Populating dev cache" - ( - udevadm control --stop-exec-queue - sysconf_cmd > "$SYSCONF_TMP" - find /dev -xdev \( -type b -o -type c -o -type l \) | cut -c 2- \ - | xargs tar cf "${DEVCACHE_TMP}" -T- - gzip < "${DEVCACHE_TMP}" > "$DEVCACHE" - rm -f "${DEVCACHE_TMP}" - mv "$SYSCONF_TMP" "$SYSCONF_CACHED" - udevadm control --start-exec-queue - rm -f "$DEVCACHE_REGEN" - ) & +[ "$DEVCACHE" != "" ] || exit 0 +[ "${VERBOSE}" == "no" ] || echo -n "udev-cache: checking for ${DEVCACHE_REGEN}... " +if ! [ -e "$DEVCACHE_REGEN" ]; then + [ "${VERBOSE}" == "no" ] || echo "not found." + exit 0 fi +[ "${VERBOSE}" == "no" ] || echo "found." +echo "Populating dev cache" + +( + set -e + trap 'echo "udev-cache: update failed!"' EXIT + udevadm control --stop-exec-queue + sysconf_cmd > "$SYSCONF_TMP" + find /dev -xdev \( -type b -o -type c -o -type l \) | cut -c 2- \ + | xargs tar cf "${DEVCACHE_TMP}" -T- + gzip < "${DEVCACHE_TMP}" > "$DEVCACHE" + rm -f "${DEVCACHE_TMP}" + mv "$SYSCONF_TMP" "$SYSCONF_CACHED" + udevadm control --start-exec-queue + rm -f "$DEVCACHE_REGEN" + trap - EXIT +) & exit 0
Signature · 1 line
2.1.3