Skip to contentSkip to conversation

scripts: add a pair of scripts to set up and run Auto Upgrade Helper

PATCH In review Alexander Kanavin 3 participants 3 messages May 17, 2020 – May 27, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 scripts: add a pair of scripts to set up and run Auto Upgrade Helper

This allows automating its setup and execution on all autobuilder worker machines; previously there was a static setup on a dedicated machine, which wasn't great from maintenance perspective.

To use:

scripts/setup-auh target_dir
scripts/run-auh target_dir

(run-auh can be run several times in a directory that was previously set up)

---
 scripts/auh-config/local.conf.append   |  3 +++
 scripts/auh-config/upgrade-helper.conf | 33 ++++++++++++++++++++++++++
 scripts/run-auh                        | 32 +++++++++++++++++++++++++
 scripts/setup-auh                      | 26 ++++++++++++++++++++
 4 files changed, 94 insertions(+)
 create mode 100644 scripts/auh-config/local.conf.append
 create mode 100644 scripts/auh-config/upgrade-helper.conf
 create mode 100755 scripts/run-auh
 create mode 100755 scripts/setup-auh
scripts/auh-config/local.conf.append new file +3 -0
new file mode 100644
index 0000000..9628737
@@ -0,0 +1,3 @@
+
+INHERIT += "buildhistory"
+LICENSE_FLAGS_WHITELIST = "commercial"
scripts/auh-config/upgrade-helper.conf new file +33 -0
new file mode 100644
index 0000000..fbf5d8a
@@ -0,0 +1,33 @@
+[maintainer_override]
+# mails for recipe upgrades will go to john.doe instead of jane.doe, etc
+#[email protected][email protected]
+
+[settings]
+# recipes in blacklist will be skipped
+blacklist=linux-libc-headers linux-yocto alsa-utils-scripts build-appliance-image
+#blacklist=python python3 glibc gcc linux-libc-headers linux-yocto-rt linux-yocto linux-yocto-dev linux-yocto-tiny qt4-x11-free qt4-embedded qt4-x11-free qt4e-demo-image gnome-common gnome-desktop3 gnome-desktop-testing adt-installer build-appliance-image
+# only recipes belonging to maintainers in whitelist will be attempted
+#[email protected]
+# SMTP server
+smtp=smtp1.yoctoproject.org:25
+# from whom should the mails arrive
[email protected]
+# who should get the status mail with statistics, at the end
[email protected]
+# clean sstate directory before upgrading
+#clean_sstate=yes
+# clean tmp directory before upgrading
+#clean_tmp=yes
+# machines to test build with
+#machines=qemux86 qemux86-64 qemuarm qemumips qemuppc
+#machines=qemux86
+
+buildhistory=yes
+#testimage=yes
+#testimage_name=core-image-minimal
+
+#workdir=/home/auh/work/
+#publish_work_url=https://logs.yoctoproject.org/auh/
+
+commit_revert_policy=all
+
scripts/run-auh new file +32 -0
new file mode 100755
index 0000000..29a8044
@@ -0,0 +1,32 @@
+#!/bin/bash
+# Run Auto Upgrade Helper in a directory set up by setup_auh.
+#
+# Called with $1 - the directory where the setup was created
+
+if [ -z $1 ]; then
+  echo "Use: $0 auh_setup_dir"
+  exit 1
+fi
+
+full_dir=$(readlink -e $1)
+
+auh_dir=$full_dir/auto-upgrade-helper
+poky_dir=$full_dir/poky
+build_dir=$full_dir/build
+sstate_dir=$full_dir/build/sstate-cache
+
+pushd $poky_dir
+
+# Base the upgrades on poky master
+git fetch origin
+git checkout -B tmp-auh-upgrades origin/master
+
+source $poky_dir/oe-init-build-env $build_dir
+$auh_dir/upgradehelper.py -e all
+
+# clean up to avoid the disk filling up
+rm -rf $build_dir/tmp/
+rm -rf $build_dir/workspace/sources/*
+find $sstate_dir -atime +10 -delete
+
+popd
scripts/setup-auh new file +26 -0
new file mode 100755
index 0000000..23f3d44
@@ -0,0 +1,26 @@
+#!/bin/bash
+# Initialize Auto Upgrade Helper in a directory.
+#
+# Called with $1 - the directory to place the setup
+CONFIG_DIR=`dirname $0`/auh-config
+
+if [ -z $1 ]; then
+  echo "Use: $0 target_dir"
+  exit 1
+fi
+
+mkdir -p $1
+pushd $1
+
+git clone git://git.yoctoproject.org/poky
+pushd poky
+git config user.email [email protected]
+git config user.name "Auto Upgrade Helper"
+popd
+git clone git://git.yoctoproject.org/auto-upgrade-helper
+source poky/oe-init-build-env build
+mkdir -p upgrade-helper
+popd
+
+cp $CONFIG_DIR/upgrade-helper.conf $1/build/upgrade-helper
+cat $CONFIG_DIR/local.conf.append >> $1/build/conf/local.conf
Signature · 1 line
2.26.2
#1 lore raw
Quoted 42 lines · Alexander Kanavin

This allows automating its setup and execution on all autobuilder worker machines;
previously there was a static setup on a dedicated machine, which wasn't great from maintenance perspective.

To use:

scripts/setup-auh target_dir
scripts/run-auh target_dir

(run-auh can be run several times in a directory that was previously set up)

---
 scripts/auh-config/local.conf.append   |  3 +++
 scripts/auh-config/upgrade-helper.conf | 33 ++++++++++++++++++++++++++
 scripts/run-auh                        | 32 +++++++++++++++++++++++++
 scripts/setup-auh                      | 26 ++++++++++++++++++++
 4 files changed, 94 insertions(+)
 create mode 100644 scripts/auh-config/local.conf.append
 create mode 100644 scripts/auh-config/upgrade-helper.conf
 create mode 100755 scripts/run-auh
 create mode 100755 scripts/setup-auh
scripts/auh-config/local.conf.append new file +3 -0
new file mode 100644
index 0000000..9628737
@@ -0,0 +1,3 @@
+
+INHERIT += "buildhistory"
+LICENSE_FLAGS_WHITELIST = "commercial"
scripts/auh-config/upgrade-helper.conf new file +0 -0
new file mode 100644
index 0000000..fbf5d8a

scripts/auh-config/upgrade-helper.conf line 3

Quoted 3 lines
scripts/auh-config/upgrade-helper.conf new file +3 -0
@@ -0,0 +1,33 @@
+[maintainer_override]
+# mails for recipe upgrades will go to john.doe instead of jane.doe, etc
+#[email protected][email protected]

unrelated.. but I just spotted this ^. I am cc'ing Anibal new email address. it's in a few other places.

Quoted 110 lines
scripts/auh-config/upgrade-helper.conf new file +4 -0
+
+[settings]
+# recipes in blacklist will be skipped
+blacklist=linux-libc-headers linux-yocto alsa-utils-scripts

build-appliance-image
+#blacklist=python python3 glibc gcc linux-libc-headers linux-yocto-rt linux-yocto linux-yocto-dev linux-yocto-tiny qt4-x11-free qt4-embedded qt4-x11-free qt4e-demo-image gnome-common gnome-desktop3 gnome-desktop-testing adt-installer build-appliance-image

scripts/auh-config/upgrade-helper.conf new file +25 -0
+# only recipes belonging to maintainers in whitelist will be attempted
+#[email protected]
+# SMTP server
+smtp=smtp1.yoctoproject.org:25
+# from whom should the mails arrive
[email protected]
+# who should get the status mail with statistics, at the end
[email protected]
+# clean sstate directory before upgrading
+#clean_sstate=yes
+# clean tmp directory before upgrading
+#clean_tmp=yes
+# machines to test build with
+#machines=qemux86 qemux86-64 qemuarm qemumips qemuppc
+#machines=qemux86
+
+buildhistory=yes
+#testimage=yes
+#testimage_name=core-image-minimal
+
+#workdir=/home/auh/work/
+#publish_work_url=https://logs.yoctoproject.org/auh/
+
+commit_revert_policy=all
+
scripts/run-auh new file +32 -0
new file mode 100755
index 0000000..29a8044
@@ -0,0 +1,32 @@
+#!/bin/bash
+# Run Auto Upgrade Helper in a directory set up by setup_auh.
+#
+# Called with $1 - the directory where the setup was created
+
+if [ -z $1 ]; then
+  echo "Use: $0 auh_setup_dir"
+  exit 1
+fi
+
+full_dir=$(readlink -e $1)
+
+auh_dir=$full_dir/auto-upgrade-helper
+poky_dir=$full_dir/poky
+build_dir=$full_dir/build
+sstate_dir=$full_dir/build/sstate-cache
+
+pushd $poky_dir
+
+# Base the upgrades on poky master
+git fetch origin
+git checkout -B tmp-auh-upgrades origin/master
+
+source $poky_dir/oe-init-build-env $build_dir
+$auh_dir/upgradehelper.py -e all
+
+# clean up to avoid the disk filling up
+rm -rf $build_dir/tmp/
+rm -rf $build_dir/workspace/sources/*
+find $sstate_dir -atime +10 -delete
+
+popd
scripts/setup-auh new file +26 -1
new file mode 100755
index 0000000..23f3d44
@@ -0,0 +1,26 @@
+#!/bin/bash
+# Initialize Auto Upgrade Helper in a directory.
+#
+# Called with $1 - the directory to place the setup
+CONFIG_DIR=`dirname $0`/auh-config
+
+if [ -z $1 ]; then
+  echo "Use: $0 target_dir"
+  exit 1
+fi
+
+mkdir -p $1
+pushd $1
+
+git clone git://git.yoctoproject.org/poky
+pushd poky
+git config user.email [email protected]
+git config user.name "Auto Upgrade Helper"
+popd
+git clone git://git.yoctoproject.org/auto-upgrade-helper
+source poky/oe-init-build-env build
+mkdir -p upgrade-helper
+popd
+
+cp $CONFIG_DIR/upgrade-helper.conf $1/build/upgrade-helper
+cat $CONFIG_DIR/local.conf.append >> $1/build/conf/local.conf
--

2.26.2

#2 lore raw
Quoted 46 lines · Nicolas Dechesne
Quoted 41 lines · Alexander Kanavin

This allows automating its setup and execution on all autobuilder worker machines;
previously there was a static setup on a dedicated machine, which wasn't great from maintenance perspective.

To use:

scripts/setup-auh target_dir
scripts/run-auh target_dir

(run-auh can be run several times in a directory that was previously set up)

---
 scripts/auh-config/local.conf.append   |  3 +++
 scripts/auh-config/upgrade-helper.conf | 33 ++++++++++++++++++++++++++
 scripts/run-auh                        | 32 +++++++++++++++++++++++++
 scripts/setup-auh                      | 26 ++++++++++++++++++++
 4 files changed, 94 insertions(+)
 create mode 100644 scripts/auh-config/local.conf.append
 create mode 100644 scripts/auh-config/upgrade-helper.conf
 create mode 100755 scripts/run-auh
 create mode 100755 scripts/setup-auh
scripts/auh-config/local.conf.append new file +3 -0
new file mode 100644
index 0000000..9628737
@@ -0,0 +1,3 @@
+
+INHERIT += "buildhistory"
+LICENSE_FLAGS_WHITELIST = "commercial"
scripts/auh-config/upgrade-helper.conf new file +0 -0
new file mode 100644
index 0000000..fbf5d8a
Quoted 9 lines

scripts/auh-config/upgrade-helper.conf line 3

Quoted 4 lines
scripts/auh-config/upgrade-helper.conf new file +3 -0
@@ -0,0 +1,33 @@
+[maintainer_override]
+# mails for recipe upgrades will go to john.doe instead of jane.doe, etc
+#[email protected][email protected]

unrelated.. but I just spotted this ^. I am cc'ing Anibal new email address. it's in a few other places.

Right, Looks good to me since Alexander cleaned-up the AUH after some tinfoil improvements, he is the new maintainer :).

Cheers,
Anibal

Quoted 113 lines
Quoted 112 lines
scripts/auh-config/upgrade-helper.conf new file +4 -0
+
+[settings]
+# recipes in blacklist will be skipped
+blacklist=linux-libc-headers linux-yocto alsa-utils-scripts

build-appliance-image
+#blacklist=python python3 glibc gcc linux-libc-headers linux-yocto-rt linux-yocto linux-yocto-dev linux-yocto-tiny qt4-x11-free qt4-embedded qt4-x11-free qt4e-demo-image gnome-common gnome-desktop3 gnome-desktop-testing adt-installer build-appliance-image

scripts/auh-config/upgrade-helper.conf new file +25 -0
+# only recipes belonging to maintainers in whitelist will be attempted
+#[email protected]
+# SMTP server
+smtp=smtp1.yoctoproject.org:25
+# from whom should the mails arrive
[email protected]
+# who should get the status mail with statistics, at the end
[email protected]
+# clean sstate directory before upgrading
+#clean_sstate=yes
+# clean tmp directory before upgrading
+#clean_tmp=yes
+# machines to test build with
+#machines=qemux86 qemux86-64 qemuarm qemumips qemuppc
+#machines=qemux86
+
+buildhistory=yes
+#testimage=yes
+#testimage_name=core-image-minimal
+
+#workdir=/home/auh/work/
+#publish_work_url=https://logs.yoctoproject.org/auh/
+
+commit_revert_policy=all
+
scripts/run-auh new file +32 -0
new file mode 100755
index 0000000..29a8044
@@ -0,0 +1,32 @@
+#!/bin/bash
+# Run Auto Upgrade Helper in a directory set up by setup_auh.
+#
+# Called with $1 - the directory where the setup was created
+
+if [ -z $1 ]; then
+  echo "Use: $0 auh_setup_dir"
+  exit 1
+fi
+
+full_dir=$(readlink -e $1)
+
+auh_dir=$full_dir/auto-upgrade-helper
+poky_dir=$full_dir/poky
+build_dir=$full_dir/build
+sstate_dir=$full_dir/build/sstate-cache
+
+pushd $poky_dir
+
+# Base the upgrades on poky master
+git fetch origin
+git checkout -B tmp-auh-upgrades origin/master
+
+source $poky_dir/oe-init-build-env $build_dir
+$auh_dir/upgradehelper.py -e all
+
+# clean up to avoid the disk filling up
+rm -rf $build_dir/tmp/
+rm -rf $build_dir/workspace/sources/*
+find $sstate_dir -atime +10 -delete
+
+popd
scripts/setup-auh new file +26 -1
new file mode 100755
index 0000000..23f3d44
@@ -0,0 +1,26 @@
+#!/bin/bash
+# Initialize Auto Upgrade Helper in a directory.
+#
+# Called with $1 - the directory to place the setup
+CONFIG_DIR=`dirname $0`/auh-config
+
+if [ -z $1 ]; then
+  echo "Use: $0 target_dir"
+  exit 1
+fi
+
+mkdir -p $1
+pushd $1
+
+git clone git://git.yoctoproject.org/poky
+pushd poky
+git config user.email [email protected]
+git config user.name "Auto Upgrade Helper"
+popd
+git clone git://git.yoctoproject.org/auto-upgrade-helper
+source poky/oe-init-build-env build
+mkdir -p upgrade-helper
+popd
+
+cp $CONFIG_DIR/upgrade-helper.conf $1/build/upgrade-helper
+cat $CONFIG_DIR/local.conf.append >> $1/build/conf/local.conf
--

2.26.2