Skip to contentSkip to conversation

xfstests-bld: use a separate default config file for each test runner

PATCH In review Eric Biggers 2 participants 4 messages Apr 17, 2017 – Apr 21, 2017 Documentation

lore
Share & tools
lore
#0 lore raw

PATCH 1/3 xfstests-bld: use a separate default config file for each test runner

From: Eric Biggers <[email protected]>

Instead of mixing together the default configuration for kvm-xfstests, gce-xfstests, and android-xfstests in a single file "config", split it into separate files for each test runner, along with a config.common (named "xfstests-common" if overridden in ~/.config/). This will allow setting defaults for variables such as ROOT_FS, NR_CPU, and MEM that may be meaningful for more than one test runner but should have different default values. It also more closely matches how the configs in ~/.config/ work.

For now keep sourcing kvm-xfstests/config.custom if present, but suggest that the user move their settings into ~/.config/ where they aren't all mixed together for all test runners. Also keep supporting per-account configs for gce-xfstests, though really one could instead just make their ~/.config/gce-xfstests enable different settings based on their active account (it is a shell script after all).

---
 Documentation/kvm-quickstart.md     |  8 ++++----
 Documentation/kvm-xfstests.md       |  9 ++++-----
 config                              |  4 +++-
 kvm-xfstests/config.android         |  6 ++++++
 kvm-xfstests/config.common          |  7 +++++++
 kvm-xfstests/config.gce             | 15 +++++++++++++++
 kvm-xfstests/{config => config.kvm} | 35 ++++-------------------------------
 kvm-xfstests/util/get-config        | 22 +++++++++++++++++-----
 8 files changed, 60 insertions(+), 46 deletions(-)
 create mode 100644 kvm-xfstests/config.android
 create mode 100644 kvm-xfstests/config.common
 create mode 100644 kvm-xfstests/config.gce
 rename kvm-xfstests/{config => config.kvm} (54%)
Documentation/kvm-quickstart.md +4 -4
index c18a39e..8dcdc90 100644
@@ -22,10 +22,10 @@
     kernel-configs directory; pick one whose version number is close
     to the kernel version you wish to build.
 
-4.  In the fstests/kvm-xfstests directory, take a look at the "config"
-    file and either edit that file in place, or (this is preferred)
-    put override values in ~/.config/kvm-xfstests or config.custom.
-    The most common values you will likely need to override is the
+4.  In the fstests/kvm-xfstests/ directory, take a look at the
+    "config.kvm" file and either edit that file in place, or (this is
+    preferred) put override values in ~/.config/kvm-xfstests.  The
+    most common values you will likely need to override are the
     location of the compiled kernel and the preferred timezone if you
     wish the log files to display times in your local timezone.
 
Documentation/kvm-xfstests.md +4 -5
index c480e4a..72f3aae 100644
@@ -39,11 +39,10 @@ Please see the relevant documentation files for more details.
 ## Setup and configuration
 
 The configuration file for kvm-xfstests is found in the kvm-xfstests
-directory and is named config.  You can edit this file directly, but
-the better thing to do is to place override values in
-~/.config/kvm-xfstests or in kvm-xfstests/config.custom.  Please look
-at the kvm-xfstests/config file to see the shell variables you can
-set.
+directory and is named config.kvm.  You can edit this file directly,
+but the better thing to do is to place override values in
+~/.config/kvm-xfstests.  Please look at the kvm-xfstests/config.kvm
+file to see the shell variables you can set.
 
 Perhaps the most important configuration variable to set is KERNEL.
 This should point at the default location for the kernel that qemu
config +3 -1
index af0f21e..f28bf9f 100644
@@ -1,4 +1,6 @@
-# Configuration file for xfstests-bld
+#
+# Configuration file for building xfstests and root filesystem images
+#
 
 XFSTESTS_GIT=git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git
 XFSPROGS_GIT=git://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git
kvm-xfstests/config.android new file +6 -0
new file mode 100644
index 0000000..4b668f8
@@ -0,0 +1,6 @@
+#
+# Default configuration for android-xfstests
+#
+# Variables set here may be overridden in ~/.config/android-xfstests
+#
+
kvm-xfstests/config.common new file +7 -0
new file mode 100644
index 0000000..12cae0a
@@ -0,0 +1,7 @@
+#
+# Default configuration for all test runners (kvm-xfstests, gce-xfstests, etc.)
+#
+# Variables set here may be overridden in ~/.config/xfstests-common
+#
+
+PRIMARY_FSTYPE="ext4"
kvm-xfstests/config.gce new file +15 -0
new file mode 100644
index 0000000..96b64f8
@@ -0,0 +1,15 @@
+#
+# Default configuration for gce-xfstests
+#
+# Variables set here may be overridden in ~/.config/gce-xfstests
+#
+
+#
+# GS_BUCKET=gs_bucket.thunk.org
+# GCE_PROJECT=tytso-xfstests-project
+# GCE_ZONE=us-central1-c
+# GCE_KERNEL=/u1/ext4-64/arch/x86/boot/bzImage
+
+# List of firewall rules to create.  By default the gce-xfstests web interface
+# is made available to everyone over the public Internet.
+GCE_FIREWALL_RULES=("allow-http --allow tcp:80 --target-tags http-server")
kvm-xfstests/config.kvm renamed from kvm-xfstests/config +4 -31
similarity index 54%
rename from kvm-xfstests/config
rename to kvm-xfstests/config.kvm
index 994dcd3..43df96a 100644
@@ -1,27 +1,12 @@
 #
-# Customize these, or put new values in:
+# Default configuration for kvm-xfstests
 #
-#		~/.config/kvm-xfstests
-#	or
-#		~/.config/gce-xfstests
-#	or
-#		~/.config/android-xfstests
-#
-# ... to affect specific commands only;
-#
-# or in config.custom to affect all commands.
+# Variables set here may be overridden in ~/.config/kvm-xfstests
 #
 
-###############################################################################
-# Common configuration
-#
-PRIMARY_FSTYPE="ext4" # all commands
-NR_CPU=2 # kvm-xfstests and gce-xfstests
-MEM=2048 # kvm-xfstests and gce-xfstests
+NR_CPU=2
+MEM=2048
 
-###############################################################################
-# KVM configuration
-#
 #QEMU=/usr/local/bin/qemu-system-x86_64
 QEMU=/usr/bin/kvm
 KERNEL=$HOME/linux
@@ -55,15 +40,3 @@ MONITOR="-monitor telnet:localhost:7498,server,nowait"
 # interception entirely.
 CONSOLE=" -serial mon:stdio"
 #CONSOLE=" -chardev stdio,id=console,signal=off -serial chardev:console"
-
-###############################################################################
-# GCE configuration
-#
-# GS_BUCKET=gs_bucket.thunk.org
-# GCE_PROJECT=tytso-xfstests-project
-# GCE_ZONE=us-central1-c
-# GCE_KERNEL=/u1/ext4-64/arch/x86/boot/bzImage
-
-# List of firewall rules to create.  By default the gce-xfstests web interface
-# is made available to everyone over the public Internet.
-GCE_FIREWALL_RULES=("allow-http --allow tcp:80 --target-tags http-server")
kvm-xfstests/util/get-config +17 -5
index b1ffc8f..f37d8c1 100644
@@ -12,12 +12,16 @@
 #
 # . "$DIR/util/get-config"
 
-. $DIR/config
+# Source the default configs
+. $DIR/config.common
+. $DIR/config.${XFSTESTS_FLAVOR}
 
-# Source the user-provided config if present (e.g. ~/.config/kvm-xfstests)
+# Source custom configs in ~/.config/ if present
+[ -f "$HOME/.config/xfstests-common" ] && . "$HOME/.config/xfstests-common"
 [ -f "$HOME/.config/${XFSTESTS_FLAVOR}-xfstests" ] && \
 	. "$HOME/.config/${XFSTESTS_FLAVOR}-xfstests"
 
+# For gce-xfstests, source the config for the active account if present
 if test "$XFSTESTS_FLAVOR" = "gce" -a -z "$GCE_ACCOUNT" -a \
    -n "$(ls $DIR/config-* 2> /dev/null)"
 then
@@ -32,11 +36,19 @@ then
 fi
 
 if test -n "$GCE_ACCOUNT" -a -f "$DIR/config-$GCE_ACCOUNT" ; then
-   . "$DIR/config-$GCE_ACCOUNT"
+    . "$DIR/config-$GCE_ACCOUNT"
 elif test -f $DIR/config.custom ; then
-   . $DIR/config.custom
+# Else, source kvm-xfstests/config.custom if present (deprecated)
+    echo -e 1>&2 "Warning: use of kvm-xfstests/config.custom is deprecated." \
+	    "Move your settings\n         into ~/.config/kvm-xfstests," \
+	    "~/.config/gce-xfstests,\n         ~/.config/android-xfstests," \
+	    "and/or ~/.config/xfstests-common"
+    . $DIR/config.custom
 fi
 
+# Source $KVM_CONFIG if present (deprecated)
 if test -n "$KVM_CONFIG" -a -f "$KVM_CONFIG"; then
-   . $KVM_CONFIG
+    echo -e 1>&2 "Warning: use of \$KVM_CONFIG is deprecated." \
+	    "Move your settings into\n         ~/.config/kvm-xfstests"
+    . $KVM_CONFIG
 fi
Signature · 1 line
2.12.2.762.g0e3151a226-goog
#1 lore raw

PATCH 2/3 gce-xfstests: move NR_CPU, MEM, and ROOT_FS into config.gce

From: Eric Biggers <[email protected]>

This allows them to be overridden by ~/.config/gce-xfstests.

---
 Documentation/gce-xfstests.md | 12 +++++++-----
 kvm-xfstests/config.gce       | 12 ++++++++++++
 kvm-xfstests/gce-xfstests     |  9 ---------
 3 files changed, 19 insertions(+), 14 deletions(-)
Documentation/gce-xfstests.md +7 -5
index 6d053d1..5b1c9a7 100644
@@ -345,8 +345,10 @@ variable is defined:
         GCE_IMAGE_PROJECT="$GCE_PROJECT"
 
 Normally, the most recently created image in the xfstests image family
-will be used by default.  You can however override this by using the
--I option to specify a specific image file.  (For example:
-"gce-xfstests -I xfstests-201608130052 smoke".)  You can also use the
---image-project command line option to override the GCE_IMAGE_PROJECT
-setting in your configuration file.
+will be used by default.  You can however override this and use a
+specific image by setting `ROOT_FS` in your `~/.config/gce-xfstests`
+configuration file, or by using the -I option to gce-xfstests.  (For
+example: `ROOT_FS=xfstests-201608130052`, or "gce-xfstests -I
+xfstests-201608130052 smoke".)  You can also use the --image-project
+command line option to override the GCE_IMAGE_PROJECT setting in your
+configuration file.
kvm-xfstests/config.gce +12 -0
index 96b64f8..3358ae3 100644
@@ -13,3 +13,15 @@
 # List of firewall rules to create.  By default the gce-xfstests web interface
 # is made available to everyone over the public Internet.
 GCE_FIREWALL_RULES=("allow-http --allow tcp:80 --target-tags http-server")
+
+# Use different defaults for GCE because the background ruby monitoring
+# program uses extra memory --- the n1-standard-2 machine type has 7.5 GB,
+# so use that.
+NR_CPU=2
+MEM=7680
+
+# Name of the GCE image from which the test instances will be created.
+# This can also be set by the -I option.
+# If "xfstests" (the default), this is actually treated as an image family, and
+# the latest image from the xfstests family will be used.
+ROOT_FS="xfstests"
kvm-xfstests/gce-xfstests +0 -9
index 7b678bc..240467e 100755
@@ -9,15 +9,6 @@ fi
 
 . "$DIR/util/get-config"
 
-# Use different defaults for GCE because the background ruby monitoring
-# program uses extra memory --- the n1-standard-2 machine type has 7.5 GB,
-# so use that.
-NR_CPU=2
-MEM=7680
-
-# We use ROOT_FS for the GCE image; this can be set by the -I option
-ROOT_FS="xfstests"
-
 if test -r "$DIR/test-appliance/config.custom" ; then
    . "$DIR/test-appliance/config.custom"
 fi
Signature · 1 line
2.12.2.762.g0e3151a226-goog
#2 lore raw

PATCH 3/3 android-xfstests: move ROOT_FS and ROOT_FS_URL into config.android

In reply to Eric Biggers

From: Eric Biggers <[email protected]>

This allows them to be overridden by ~/.config/android-xfstests. Also start treating an empty ROOT_FS_URL as meaning that no URL is provided.

---
 Documentation/android-xfstests.md | 3 ++-
 kvm-xfstests/android-xfstests     | 9 +++------
 kvm-xfstests/config.android       | 6 ++++++
 3 files changed, 11 insertions(+), 7 deletions(-)
Documentation/android-xfstests.md +2 -1
index b309c21..5b37f9e 100644
@@ -39,7 +39,8 @@ devices.  If you encounter a problem, please submit a fix!
   [building-xfstests](building-xfstests.md).  Then, either put the
   chroot tarball in the default location of
   kvm-xfstests/test-appliance/armhf_root_fs.tar.gz, or specify it with
-  the -I option to android-xfstests.
+  ROOT_FS in your ~/.config/android-xfstests or the -I option to
+  android-xfstests.
 
 ## Procedure
 
kvm-xfstests/android-xfstests +3 -6
index 3489bc2..e33955e 100755
@@ -10,12 +10,6 @@ fi
 
 . "$DIR/util/get-config"
 
-# Path to chroot tarball; can be overridden with -I
-ROOT_FS="$DIR/test-appliance/armhf_root_fs.tar.gz"
-
-# Where to download the tarball from (at user's request) if we don't have it
-ROOT_FS_URL="https://www.kernel.org/pub/linux/kernel/people/tytso/kvm-xfstests/armhf_root_fs.tar.gz"
-
 # Chroot directory on device.
 # Note: this will be wiped clean when deploying a new chroot tarball.
 CHROOT_DIR="/data/xfstests-chroot"
@@ -142,6 +136,9 @@ setup_chroot()
     if ! [ -f "$ROOT_FS" ]; then
 	echo "The xfstests chroot tarball does not exist:"
 	echo "        $ROOT_FS"
+	if [ -z "$ROOT_FS_URL" ]; then
+	    exit 1
+	fi
 	ask_yesno "Would you like to download the latest public tarball to that location?"
 	wget -O "$ROOT_FS" "$ROOT_FS_URL"
 	echo "Finished downloading chroot tarball."
kvm-xfstests/config.android +6 -0
index 4b668f8..ed7098c 100644
@@ -4,3 +4,9 @@
 # Variables set here may be overridden in ~/.config/android-xfstests
 #
 
+# Path to chroot tarball; this can also be set by the -I option
+ROOT_FS="$DIR/test-appliance/armhf_root_fs.tar.gz"
+
+# Where to download the tarball from (at user's request) if we don't have it.
+# If you want to disable this functionality, set this to an empty string.
+ROOT_FS_URL="https://www.kernel.org/pub/linux/kernel/people/tytso/kvm-xfstests/armhf_root_fs.tar.gz"
Signature · 1 line
2.12.2.762.g0e3151a226-goog
#3 lore raw

Re: [PATCH 3/3] android-xfstests: move ROOT_FS and ROOT_FS_URL into config.android

Quoted 6 lines · Eric Biggers

From: Eric Biggers <[email protected]>

This allows them to be overridden by ~/.config/android-xfstests. Also start treating an empty ROOT_FS_URL as meaning that no URL is provided.

Thanks for the cleanup! I've applied all three patches to my tree.

- Ted