Support for Patchwork CI
v6 cover Support for Patchwork CI
Add support for patch-series validation in the patchwork ML. We use Github to schedule a trigger every 30 minutes, checking for new patche-series in parchwork which has not been tested yet.
The way we decide if a patch-series has been tested in patchwork, is by looking at its status (in particular, if it's "Needs Review / ACK"), as well as checking if test report has been uploaded to any of the series patches.
All communication to Patchwrok is done via REST API, using curl and js tools.
First, we create a script called patchwork-ci.sh that provides all the commands to read new untested patch-series, set their status and testing report. Then, we create a scheduled workflow in Gitlab, checking every 30 minutes if there are new untested patch-series. At the end, we trigger the main build workflow, used to validate LTP commits in our Github mainline. All the times we trigger the build workflow, we also provide the patch-series ID, that will be fetched and applied on the current branch before running the tests.
---
Changes in v6:
- patchwork.sh: use more "[ $stuff ] || do_something" syntax
- patchwork.sh: remove set -ex
- patchwork.sh: variables default setup
- patchwork.sh: move file creation in tmp folder
- patchwork.sh: check multiple commands exist in once
- trigger workflow now triggers only on linux-test-project/ltp repo
- Link to v5: https://patchwork.ozlabs.org/project/ltp/list/?series=452320
Changes in v5:
- patchwork is now a utility that can be used separately
- Link to v4: https://lore.kernel.org/r/[email protected]
Changes in v4:
- make use of secrets to apply patches in order to set state to needs-review-ack
- check for curl failures and eventually exit 1
- use a different format for stored file in 'patchwork-ci.sh verify' command
- define PATCHWORK_CI_FILE
- some symbols like . are not supported by patchwork checks, so slurmify the string before sending checks context
- Link to v3: https://lore.kernel.org/r/[email protected]
Changes in v3:
- secrets are not supported by if: statement, so we remove it
- Link to v2: https://lore.kernel.org/r/[email protected]
Changes in v2:
- check for secrets.PATCHWORK_TOKEN before sending tests results
- v2 to re-trigger lore after missing 3/4 patch
- Link to v1: https://lore.kernel.org/r/[email protected]
---
Andrea Cervesato (4):
ci: install dependences for patchwork-ci script
ci: add patchwork communication script
ci: add ci-patchwork-trigger workflow
ci: apply patchwork series in ci-docker-build workflow
.github/workflows/ci-docker-build.yml | 39 ++++++- .github/workflows/ci-patchwork-trigger.yml | 65 +++++++++++ ci/alpine-runtime.sh | 2 + ci/alpine.sh | 2 + ci/debian.i386.sh | 2 + ci/debian.sh | 2 + ci/fedora.sh | 2 + ci/tools/patchwork.sh | 169 +++++++++++++++++++++++++++++ ci/tumbleweed.sh | 2 + 9 files changed, 284 insertions(+), 1 deletion(-)
---
base-commit: 40c559787d15b2167ba657b3ba431e3299969d98
change-id: 20250410-patchwork_ci-7dc4ae02c40d
Best regards,
Signature · 5 lines
Andrea Cervesato <[email protected]> -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v6 1/4 ci: install dependences for patchwork-ci script
From: Andrea Cervesato <[email protected]>
--- ci/alpine-runtime.sh | 2 ++ ci/alpine.sh | 2 ++ ci/debian.i386.sh | 2 ++ ci/debian.sh | 2 ++ ci/fedora.sh | 2 ++ ci/tumbleweed.sh | 2 ++ 6 files changed, 12 insertions(+)
ci/alpine-runtime.sh +2 -0
@@ -4,6 +4,8 @@ apk add \ acl \ + curl \ + jq \ keyutils \ libaio \ libacl \
ci/alpine.sh +2 -0
@@ -9,6 +9,8 @@ apk add \ autoconf \ automake \ clang \ + curl \ + jq \ gcc \ git \ acl-dev \
ci/debian.i386.sh +2 -0
@@ -6,6 +6,8 @@ dpkg --add-architecture i386 apt update apt install -y --no-install-recommends \ + curl \ + jq \ linux-libc-dev:i386 \ gcc-multilib \ libacl1-dev:i386 \
ci/debian.sh +2 -0
@@ -23,6 +23,8 @@ pkg_minimal=" debhelper devscripts clang + curl + jq gcc git iproute2
ci/fedora.sh +2 -0
@@ -9,6 +9,8 @@ $yum \ automake \ make \ clang \ + curl \ + jq \ gcc \ git \ findutils \
ci/tumbleweed.sh +2 -0
@@ -8,6 +8,8 @@ $zyp \ autoconf \ automake \ clang \ + curl \ + jq \ findutils \ gcc \ git \
Signature · 5 lines
2.43.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v6 1/4] ci: install dependences for patchwork-ci script
Hi!
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v6 2/4 ci: add patchwork communication script
In reply to Andrea CervesatoFrom: Andrea Cervesato <[email protected]>
Add a script to communicate with patchwork. Available commands are:
- state: change patch-series state
- check: send a tests report to patchwork
- verify: will print a list of new patch-series which has not been tested in the past hour (by default)
The script can be configured defining:
- PATCHWORK_URL: patchwork url to communicate with
- PATCHWORK_TOKEN: patchwork authentication token
- PATCHWORK_SINCE: timespan in seconds where we want to fetch patch-series
--- ci/tools/patchwork.sh | 169 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 169 insertions(+)
ci/tools/patchwork.sh new file +169 -0
@@ -0,0 +1,169 @@ +#!/bin/sh -x +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Shell script to communicate with Patchwork via REST API. +# It has been mainly created for CI purposes, but it can be used in the shell +# by satisfying minimum requirements. +# +# Copyright (c) 2025 Andrea Cervesato <[email protected]> + +PATCHWORK_URL="${PATCHWORK_URL:-https://patchwork.ozlabs.org}" +PATCHWORK_SINCE="${PATCHWORK_SINCE:-3600}" + +command_exists() { + for cmd in "$@"; do + if ! command -v "$cmd" >/dev/null 2>&1; then + echo "'$1' must be present in the system" >&2 + exit 1 + fi + done +} + +command_exists "curl" "jq" + +fetch_series() { + local current_time=$(date +%s) + local since_time=$(expr $current_time - $PATCHWORK_SINCE) + local date=$(date -u -d @$since_time +"%Y-%m-%dT%H:%M:%SZ") + + curl -k -G "$PATCHWORK_URL/api/events/" \ + --data "category=series-completed" \ + --data "project=ltp" \ + --data "state=new" \ + --data "since=$date" \ + --data "archive=no" | + jq -r '.[] | "\(.payload.series.id):\(.payload.series.mbox)"' + + [ $? -eq 0 ] || exit 1 +} + +get_patches() { + local series_id="$1" + + curl -k -G "$PATCHWORK_URL/api/patches/" \ + --data "project=ltp" \ + --data "series=$series_id" | + jq -r '.[] | "\(.id)"' + + [ $? -eq 0 ] || exit 1 +} + +verify_token_exists() { + if [ -z "$PATCHWORK_TOKEN" ]; then + echo "For this feature you need \$PATCHWORK_TOKEN" + exit 1 + fi +} + +set_patch_state() { + local patch_id="$1" + local state="$2" + + verify_token_exists + + curl -k -X PATCH \ + -H "Authorization: Token $PATCHWORK_TOKEN" \ + -F "state=$state" \ + "$PATCHWORK_URL/api/patches/$patch_id/" + + [ $? -eq 0 ] || exit 1 +} + +set_series_state() { + local series_id="$1" + local state="$2" + + get_patches "$series_id" | while IFS= read -r patch_id; do + [ "$patch_id" ] && set_patch_state "$patch_id" "$state" + done +} + +get_checks() { + local patch_id="$1" + + curl -k -G "$PATCHWORK_URL/api/patches/$patch_id/checks/" | + jq -r '.[] | "\(.id)"' + + [ $? -eq 0 ] || exit 1 +} + +already_tested() { + local series_id="$1" + + get_patches "$series_id" | while IFS= read -r patch_id; do + [ "$patch_id" ] || continue + + get_checks "$patch_id" | while IFS= read -r check_id; do + if [ -n "$check_id" ]; then + echo "$check_id" + return + fi + done + done +} + +verify_new_patches() { + local tmp=$(mktemp -d) + local output="$tmp/series_ids.txt" + + echo -n '' >"$output" + + fetch_series | while IFS=: read -r series_id series_mbox; do + [ "$series_id" ] || continue + + tested=$(already_tested "$series_id") + [ "$tested" ] && continue + + echo "$series_id|$series_mbox" >>"$output" + done + + cat "$output" +} + +send_results() { + local series_id="$1" + local target_url="$2" + + verify_token_exists + + local context=$(echo "$3" | sed 's/:/_/g; s/\//-/g; s/\./-/g') + + [ "$CC" ] && context="${context}_${CC}" + [ "$ARCH" ] && context="${context}_${ARCH}" + + local result="$4" + [ "$result" == "cancelled" ] && return + + local state="fail" + [ "$result" == "success" ] && state="success" + + get_patches "$series_id" | while IFS= read -r patch_id; do + [ "$patch_id" ] || continue + + curl -k -X POST \ + -H "Authorization: Token $PATCHWORK_TOKEN" \ + -F "state=$state" \ + -F "context=$context" \ + -F "target_url=$target_url" \ + -F "description=$result" \ + "$PATCHWORK_URL/api/patches/$patch_id/checks/" + + [ $? -eq 0 ] && exit 1 + done +} + +case "$1" in +state) + set_series_state "$2" "$3" + ;; +check) + send_results "$2" "$3" "$4" "$5" + ;; +verify) + verify_new_patches + ;; +*) + echo "Available commands: state, check, verify" + exit 1 + ;; +esac
Signature · 5 lines
2.43.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v6 2/4] ci: add patchwork communication script
Hi Andrea,
TL;DR:
With fixed == and curl pipe redirect, you may add:
More info below.
Quoted 18 lines
--- ci/tools/patchwork.sh | 169 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 169 insertions(+)ci/tools/patchwork.sh new file +0
-0index 0000000000000000000000000000000000000000..bef6998b6168714e0cfaef443a4573017440bb2d
ci/tools/patchwork.sh new file +5
-0@@ -0,0 +1,169 @@ +#!/bin/sh -x +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Shell script to communicate with Patchwork via REST API. +# It has been mainly created for CI purposes, but it can be used inthe shell
ci/tools/patchwork.sh line 13
Quoted 8 lines
ci/tools/patchwork.sh new file +8
-0+# by satisfying minimum requirements. +# +# Copyright (c) 2025 Andrea Cervesato <[email protected]> + +PATCHWORK_URL="${PATCHWORK_URL:-https://patchwork.ozlabs.org}" +PATCHWORK_SINCE="${PATCHWORK_SINCE:-3600}" + +command_exists() {
Can you please add before merge?
local cmd
Quoted 23 lines
ci/tools/patchwork.sh new file +22
-0+ for cmd in "$@"; do + if ! command -v "$cmd" >/dev/null 2>&1; then + echo "'$1' must be present in the system" >&2 + exit 1 + fi + done +} + +command_exists "curl" "jq" + +fetch_series() { + local current_time=$(date +%s) + local since_time=$(expr $current_time - $PATCHWORK_SINCE) + local date=$(date -u -d @$since_time +"%Y-%m-%dT%H:%M:%SZ") + + curl -k -G "$PATCHWORK_URL/api/events/" \ + --data "category=series-completed" \ + --data "project=ltp" \ + --data "state=new" \ + --data "since=$date" \ + --data "archive=no" | + jq -r '.[] |"\(.payload.series.id):\(.payload.series.mbox)"'
ci/tools/patchwork.sh
Quoted 2 lines
ci/tools/patchwork.sh new file +2
-0+ + [ $? -eq 0 ] || exit 1
FYI $? holds exit code of the last member of pipe, 'jq -r ...' in this case. I guess you'd prefer to check curl exit code, right?
You have 2 options:
1) Redirect curl with '-o log', and call jq after $? evaluation.
2) Depend on bash and use ${PIPESTATUS[0]} (e.g. $PIPESTATUS array).
NOTE: that really requires /bin/bash shebang (e.g. not working on dash and busybox sh, thus CI will depend on bash), also will not work on zsh which uses $pipestatus array (lower case) :(. We prefer to avoid bash (unlike kernel developers) for a good reason (bash incompatibility between versions, need to install bash in container, also if arrays are needed it's a question whether use shell scripting), but it's mainly due the need tests (LTP shell API and tests) to be portable, it could be used in tools (with /bin/bash shebang).
Quoted 40 lines
ci/tools/patchwork.sh new file +40
-0+} + +get_patches() { + local series_id="$1" + + curl -k -G "$PATCHWORK_URL/api/patches/" \ + --data "project=ltp" \ + --data "series=$series_id" | + jq -r '.[] | "\(.id)"' + + [ $? -eq 0 ] || exit 1 +} + +verify_token_exists() { + if [ -z "$PATCHWORK_TOKEN" ]; then + echo "For this feature you need \$PATCHWORK_TOKEN" + exit 1 + fi +} + +set_patch_state() { + local patch_id="$1" + local state="$2" + + verify_token_exists + + curl -k -X PATCH \ + -H "Authorization: Token $PATCHWORK_TOKEN" \ + -F "state=$state" \ + "$PATCHWORK_URL/api/patches/$patch_id/" + + [ $? -eq 0 ] || exit 1 +} + +set_series_state() { + local series_id="$1" + local state="$2" + + get_patches "$series_id" | while IFS= read -r patch_id; do + [ "$patch_id" ] && set_patch_state "$patch_id"
Quoted 1 line
"$state"
FYI if you ever wish to use '-e' in a script, construct [ ... ] && ... will be broken with it. It would have to be [ ... ] || ... (or use if ...; then ...; fi).
Quoted 23 lines
ci/tools/patchwork.sh new file +19
-0+ done +} + +get_checks() { + local patch_id="$1" + + curl -k -G "$PATCHWORK_URL/api/patches/$patch_id/checks/" | + jq -r '.[] | "\(.id)"' + + [ $? -eq 0 ] || exit 1 +} + +already_tested() { + local series_id="$1" + + get_patches "$series_id" | while IFS= read -r patch_id; do + [ "$patch_id" ] || continue + + get_checks "$patch_id" | while IFS= read -r check_id;do
ci/tools/patchwork.sh new file +3
-0+ if [ -n "$check_id" ]; then + echo "$check_id" + return
ci/tools/patchwork.sh line 109
Quoted 10 lines
ci/tools/patchwork.sh new file +10
-0+ fi + done + done +} + +verify_new_patches() { + local tmp=$(mktemp -d) + local output="$tmp/series_ids.txt" + + echo -n '' >"$output"
FYI echo switches are known to have non-portable edge cases, which printf does not have, but this works (ok to use). Using printf would be simpler:
printf >"$output"
Quoted 16 lines
ci/tools/patchwork.sh new file +16
-0+ + fetch_series | while IFS=: read -r series_id series_mbox; do + [ "$series_id" ] || continue + + tested=$(already_tested "$series_id") + [ "$tested" ] && continue + + echo "$series_id|$series_mbox" >>"$output" + done + + cat "$output" +} + +send_results() { + local series_id="$1" + local target_url="$2"
ci/tools/patchwork.sh line 135
Quoted 10 lines
ci/tools/patchwork.sh new file +10
-0+ + verify_token_exists + + local context=$(echo "$3" | sed 's/:/_/g; s/\//-/g; s/\./-/g') + + [ "$CC" ] && context="${context}_${CC}" + [ "$ARCH" ] && context="${context}_${ARCH}" + + local result="$4" + [ "$result" == "cancelled" ] && return
== is a bashism, please use =
ci/tools/patchwork.sh line 138
Quoted 3 lines
ci/tools/patchwork.sh new file +3
-0+ + local state="fail" + [ "$result" == "success" ] && state="success"
And here as well.
We have checkbashisms part of make check, but that's for our LTP build system directories, here you must call it directly:
$ checkbashisms ci/tools/patchwork.sh
possible bashism in ci/tools/patchwork.sh line 135 (should be 'b = a'):
[ "$result" == "cancelled" ] && return
possible bashism in ci/tools/patchwork.sh line 138 (should be 'b = a'):
[ "$result" == "success" ] && state="success"Quoted 8 lines
ci/tools/patchwork.sh new file +8
-0+ + get_patches "$series_id" | while IFS= read -r patch_id; do + [ "$patch_id" ] || continue + + curl -k -X POST \ + -H "Authorization: Token $PATCHWORK_TOKEN" \ + -F "state=$state" \ + -F "context=$context" \
ci/tools/patchwork.sh line 156
Quoted 10 lines
ci/tools/patchwork.sh new file +10
-0+ -F "target_url=$target_url" \ + -F "description=$result" \ + "$PATCHWORK_URL/api/patches/$patch_id/checks/" + + [ $? -eq 0 ] && exit 1 + done +} + +case "$1" in +state)
I see you don't want to support running whole script in once for debugging.
Using '' would allow it (up to you of course, as most of comments):
''|state)
Kind regards,
Petr
Quoted 16 lines
ci/tools/patchwork.sh new file +13
-1+ set_series_state "$2" "$3" + ;; +check) + send_results "$2" "$3" "$4" "$5" + ;; +verify) + verify_new_patches + ;; +*) + echo "Available commands: state, check, verify" + exit 1 + ;; +esac --2.43.0
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v6 2/4] ci: add patchwork communication script
In reply to Andrea CervesatoHi!
Quoted 36 lines
Signed-off-by Andrea Cervesato <[email protected]>--- ci/tools/patchwork.sh | 169 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 169 insertions(+)ci/tools/patchwork.sh new file +25
-0@@ -0,0 +1,169 @@ +#!/bin/sh -x +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Shell script to communicate with Patchwork via REST API. +# It has been mainly created for CI purposes, but it can be used in the shell +# by satisfying minimum requirements. +# +# Copyright (c) 2025 Andrea Cervesato <[email protected]> + +PATCHWORK_URL="${PATCHWORK_URL:-https://patchwork.ozlabs.org}" +PATCHWORK_SINCE="${PATCHWORK_SINCE:-3600}" + +command_exists() { + for cmd in "$@"; do + if ! command -v "$cmd" >/dev/null 2>&1; then + echo "'$1' must be present in the system" >&2 + exit 1 + fi + done +} + +command_exists "curl" "jq" + +fetch_series() { + local current_time=$(date +%s)
ci/tools/patchwork.sh line 35
Quoted 10 lines
ci/tools/patchwork.sh new file +10
-0+ local since_time=$(expr $current_time - $PATCHWORK_SINCE) + local date=$(date -u -d @$since_time +"%Y-%m-%dT%H:%M:%SZ") + + curl -k -G "$PATCHWORK_URL/api/events/" \ + --data "category=series-completed" \ + --data "project=ltp" \ + --data "state=new" \ + --data "since=$date" \ + --data "archive=no" | + jq -r '.[] | "\(.payload.series.id):\(.payload.series.mbox)"'
^
We add ':' here and then we have to use IFS=':' to parse these values. Wouldn't this work without the need to fiddle with IFS if we just added space here instead?
If that worked we could get rid of all the IFS= assigments in the script.
Quoted 120 lines
ci/tools/patchwork.sh new file +120
-0+ [ $? -eq 0 ] || exit 1 +} + +get_patches() { + local series_id="$1" + + curl -k -G "$PATCHWORK_URL/api/patches/" \ + --data "project=ltp" \ + --data "series=$series_id" | + jq -r '.[] | "\(.id)"' + + [ $? -eq 0 ] || exit 1 +} + +verify_token_exists() { + if [ -z "$PATCHWORK_TOKEN" ]; then + echo "For this feature you need \$PATCHWORK_TOKEN" + exit 1 + fi +} + +set_patch_state() { + local patch_id="$1" + local state="$2" + + verify_token_exists + + curl -k -X PATCH \ + -H "Authorization: Token $PATCHWORK_TOKEN" \ + -F "state=$state" \ + "$PATCHWORK_URL/api/patches/$patch_id/" + + [ $? -eq 0 ] || exit 1 +} + +set_series_state() { + local series_id="$1" + local state="$2" + + get_patches "$series_id" | while IFS= read -r patch_id; do + [ "$patch_id" ] && set_patch_state "$patch_id" "$state" + done +} + +get_checks() { + local patch_id="$1" + + curl -k -G "$PATCHWORK_URL/api/patches/$patch_id/checks/" | + jq -r '.[] | "\(.id)"' + + [ $? -eq 0 ] || exit 1 +} + +already_tested() { + local series_id="$1" + + get_patches "$series_id" | while IFS= read -r patch_id; do + [ "$patch_id" ] || continue + + get_checks "$patch_id" | while IFS= read -r check_id; do + if [ -n "$check_id" ]; then + echo "$check_id" + return + fi + done + done +} + +verify_new_patches() { + local tmp=$(mktemp -d) + local output="$tmp/series_ids.txt" + + echo -n '' >"$output" + + fetch_series | while IFS=: read -r series_id series_mbox; do + [ "$series_id" ] || continue + + tested=$(already_tested "$series_id") + [ "$tested" ] && continue + + echo "$series_id|$series_mbox" >>"$output" + done + + cat "$output" +} + +send_results() { + local series_id="$1" + local target_url="$2" + + verify_token_exists + + local context=$(echo "$3" | sed 's/:/_/g; s/\//-/g; s/\./-/g') + + [ "$CC" ] && context="${context}_${CC}" + [ "$ARCH" ] && context="${context}_${ARCH}" + + local result="$4" + [ "$result" == "cancelled" ] && return + + local state="fail" + [ "$result" == "success" ] && state="success" + + get_patches "$series_id" | while IFS= read -r patch_id; do + [ "$patch_id" ] || continue + + curl -k -X POST \ + -H "Authorization: Token $PATCHWORK_TOKEN" \ + -F "state=$state" \ + -F "context=$context" \ + -F "target_url=$target_url" \ + -F "description=$result" \ + "$PATCHWORK_URL/api/patches/$patch_id/checks/" + + [ $? -eq 0 ] && exit 1 + done +} + +case "$1" in +state)
ci/tools/patchwork.sh line 166
Quoted 10 lines
ci/tools/patchwork.sh new file +10
-0+ set_series_state "$2" "$3" + ;; +check) + send_results "$2" "$3" "$4" "$5" + ;; +verify) + verify_new_patches + ;; +*) + echo "Available commands: state, check, verify"
Possibly write this to stderr (&>2)?
ci/tools/patchwork.sh line 169
Quoted 3 lines
ci/tools/patchwork.sh new file +3
-0+ exit 1 + ;; +esac
Other than these:
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v6 3/4 ci: add ci-patchwork-trigger workflow
In reply to Andrea CervesatoFrom: Andrea Cervesato <[email protected]>
Add ci-patchwork-trigger workflow that is meant to run every 30 minutes, checking for new untested LTP patches in the Mailing List and running the ci-docker-build workflow on them.
--- .github/workflows/ci-patchwork-trigger.yml | 65 ++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+)
.github/workflows/ci-patchwork-trigger.yml new file +65 -0
@@ -0,0 +1,65 @@ +# Copyright (c) 2025 Andrea Cervesato <[email protected]> + +name: "Patchwork checker" +on: + push: + schedule: + - cron: '*/15 * * * 1-5' + - cron: '*/45 * * * 6,0' + +env: + PATCHWORK_CI_FILE: patchwork-ci-output.txt + +jobs: + checker: + if: ${{ github.repository == 'linux-test-project/ltp' }} + runs-on: ubuntu-latest + steps: + - name: Git checkout + uses: actions/checkout@v1 + + - name: Verify new patches + id: verify + run: | + ./ci/tools/patchwork.sh verify > "$PATCHWORK_CI_FILE" + cat "$PATCHWORK_CI_FILE" + + - name: Run tests + if: success() + uses: actions/github-script@v7 + with: + script: | + const fs = require('fs'); + + const output = fs.readFileSync(process.env.PATCHWORK_CI_FILE, 'utf8'); + if (output.length === 0) { + console.log("'patchwork-ci.sh verify' output is empty"); + return; + } + + const lines = output.split('\n'); + if (lines.length === 0) { + console.log("No new patch-series found"); + return; + } + + for (const data of lines) { + const [series_id, series_mbox] = data.split('|'); + if (series_id.length === 0 || series_mbox.length === 0) { + console.log(`Malformed data: ${data}`); + continue; + } + + const response = await github.rest.actions.createWorkflowDispatch({ + owner: context.repo.owner, + repo: context.repo.repo, + ref: context.ref, + workflow_id: 'ci-docker-build.yml', + inputs: { + SERIES_ID: series_id, + SERIES_MBOX: series_mbox, + } + }); + + console.log(response); + }
Signature · 5 lines
2.43.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v6 3/4] ci: add ci-patchwork-trigger workflow
Hi Andrea,
3rd patch finally got to the mailing list, great :).
Quoted 5 lines · Andrea Cervesato
From: Andrea Cervesato <[email protected]>
Add ci-patchwork-trigger workflow that is meant to run every 30 minutes, checking for new untested LTP patches in the Mailing List and running the ci-docker-build workflow on them.
I guess that's the original message for some previous version. Now it runs every 15 min (4x hour) during working days (Mon-Fri) and at 0 and 45 every hour (2x hour) on weekends.
Thanks for testing github.repository.
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v6 3/4] ci: add ci-patchwork-trigger workflow
In reply to Andrea CervesatoHi!
Quoted 15 lines
Add ci-patchwork-trigger workflow that is meant to run every 30 minutes, checking for new untested LTP patches in the Mailing List and running the ci-docker-build workflow on them.
Signed-off-by Andrea Cervesato <[email protected]>--- .github/workflows/ci-patchwork-trigger.yml | 65 ++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+).github/workflows/ci-patchwork-trigger.yml new file +0
-0
.github/workflows/ci-patchwork-trigger.yml line 5
Quoted 5 lines
.github/workflows/ci-patchwork-trigger.yml new file +5
-0@@ -0,0 +1,65 @@ +# Copyright (c) 2025 Andrea Cervesato <[email protected]> + +name: "Patchwork checker" +on: + push:
Does it make sense to run the script both on schedulle and push?
.github/workflows/ci-patchwork-trigger.yml line 8
Quoted 3 lines
.github/workflows/ci-patchwork-trigger.yml new file +3
-0+ schedule: + - cron: '*/15 * * * 1-5' + - cron: '*/45 * * * 6,0'
So we run this less on weekends, I suppose that may make some sense.
Quoted 46 lines
.github/workflows/ci-patchwork-trigger.yml new file +45
-0+env: + PATCHWORK_CI_FILE: patchwork-ci-output.txt +jobs: + checker: + if: ${{ github.repository == 'linux-test-project/ltp' }} + runs-on: ubuntu-latest + steps: + - name: Git checkout + uses: actions/checkout@v1 + + - name: Verify new patches + id: verify + run: | + ./ci/tools/patchwork.sh verify > "$PATCHWORK_CI_FILE" + cat "$PATCHWORK_CI_FILE" + + - name: Run tests + if: success() + uses: actions/github-script@v7 + with: + script: | + const fs = require('fs'); + + const output = fs.readFileSync(process.env.PATCHWORK_CI_FILE, 'utf8'); + if (output.length === 0) { + console.log("'patchwork-ci.sh verify' output is empty"); + return; + } + + const lines = output.split('\n'); + if (lines.length === 0) { + console.log("No new patch-series found"); + return; + } + + for (const data of lines) { + const [series_id, series_mbox] = data.split('|'); + if (series_id.length === 0 || series_mbox.length === 0) { + console.log(`Malformed data: ${data}`); + continue; + } + + const response = await github.rest.actions.createWorkflowDispatch({ + owner: context.repo.owner, + repo: context.repo.repo,
.github/workflows/ci-patchwork-trigger.yml line 65
Quoted 10 lines
.github/workflows/ci-patchwork-trigger.yml new file +10
-0+ ref: context.ref, + workflow_id: 'ci-docker-build.yml', + inputs: { + SERIES_ID: series_id, + SERIES_MBOX: series_mbox, + } + }); + + console.log(response); + }
The code looks sane to me, though I'm not expert on github CI.
You can add my:
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v6 3/4] ci: add ci-patchwork-trigger workflow
Hi Cyril,
Quoted 4 lines · Cyril Hrubis
.github/workflows/ci-patchwork-trigger.yml line 5
Quoted 3 lines
.github/workflows/ci-patchwork-trigger.yml new file +3
-0+name: "Patchwork checker" +on: + push:Does it make sense to run the script both on schedulle and push?
It makes sense if you develop the pipeline. I can disable it tho, but then we need to remember that for each new feature on the pipeline we need to add "push:" as well.
Quoted 4 lines
.github/workflows/ci-patchwork-trigger.yml line 8
Quoted 3 lines
.github/workflows/ci-patchwork-trigger.yml new file +3
-0+ schedule: + - cron: '*/15 * * * 1-5' + - cron: '*/45 * * * 6,0'So we run this less on weekends, I suppose that may make some sense.
Yes, it makes sense if we receive patches in the weekend. Just in case, it will take a few seconds to complete the workflow anyway.
- Andrea
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
PATCH v6 4/4 ci: apply patchwork series in ci-docker-build workflow
In reply to Andrea CervesatoFrom: Andrea Cervesato <[email protected]>
Modify ci-docker-build workflow in order to apply untested new patchwork patch-series inside the current branch and to send back results in the patchwork instance.
--- .github/workflows/ci-docker-build.yml | 39 ++++++++++++++++++++++++++++++++++- 1 file changed, 38 insertions(+), 1 deletion(-)
.github/workflows/ci-docker-build.yml +38 -1
@@ -1,7 +1,19 @@ # Copyright (c) 2021-2024 Petr Vorel <[email protected]> name: "Test building in various distros in Docker" -on: [push, pull_request] +on: + push: + pull_request: + workflow_dispatch: + inputs: + SERIES_ID: + description: LTP patch series ID + required: false + default: '' + SERIES_MBOX: + description: LTP patch series URL + required: false + default: '' permissions: contents: read # to fetch code (actions/checkout)
@@ -125,6 +137,20 @@ jobs: - name: Compiler version run: $CC --version + - name: Apply Patchwork series + if: inputs.SERIES_ID != '' && inputs.SERIES_MBOX != '' + env: + PATCHWORK_TOKEN: ${{ secrets.PATCHWORK_TOKEN }} + run: | + git config --global user.name 'GitHub CI' + git config --global user.email '[email protected]' + git config --global --add safe.directory "$GITHUB_WORKSPACE" + + git checkout -b review_patch_series_"${{ inputs.SERIES_ID }}" + curl -k "${{ inputs.SERIES_MBOX }}" | git am + + ./ci/tools/patchwork.sh state "${{ inputs.SERIES_ID }}" "needs-review-ack" + - name: ver_linux run: ./ver_linux
@@ -158,3 +184,14 @@ jobs: run: | if [ "$MAKE_INSTALL" = 1 ]; then INSTALL_OPT="-i"; fi ./build.sh -r install -o ${TREE:-in} $INSTALL_OPT + + - name: Send results to Patchwork + if: always() && inputs.SERIES_ID != '' && inputs.SERIES_MBOX != '' + env: + PATCHWORK_TOKEN: ${{ secrets.PATCHWORK_TOKEN }} + run: | + ./ci/tools/patchwork.sh check \ + "${{ inputs.SERIES_ID }}" \ + "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}" \ + "${{ matrix.container }}" \ + "${{ job.status }}"
Signature · 5 lines
2.43.0 -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v6 4/4] ci: apply patchwork series in ci-docker-build workflow
Hi Andrea,
Thanks for whole patchset.
Kind regards,
Petr
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v6 4/4] ci: apply patchwork series in ci-docker-build workflow
In reply to Andrea CervesatoHi!
.github/workflows/ci-docker-build.yml jobs: · line 149
Quoted 10 lines
.github/workflows/ci-docker-build.yml +10
-0+ - name: Apply Patchwork series + if: inputs.SERIES_ID != '' && inputs.SERIES_MBOX != '' + env: + PATCHWORK_TOKEN: ${{ secrets.PATCHWORK_TOKEN }} + run: | + git config --global user.name 'GitHub CI' + git config --global user.email '[email protected]' + git config --global --add safe.directory "$GITHUB_WORKSPACE" + + git checkout -b review_patch_series_"${{ inputs.SERIES_ID }}"
So we create a branch for each CI run but I do not see any place where we get rid of it. Shouldn't we remove it after the CI run?
Quoted 12 lines
.github/workflows/ci-docker-build.yml +5
-0+ curl -k "${{ inputs.SERIES_MBOX }}" | git am + + ./ci/tools/patchwork.sh state "${{ inputs.SERIES_ID }}" "needs-review-ack" + - name: ver_linux run: ./ver_linux@@ -158,3 +184,14 @@ jobs: run: | if [ "$MAKE_INSTALL" = 1 ]; then INSTALL_OPT="-i"; fi ./build.sh -r install -o ${TREE:-in} $INSTALL_OPT +
.github/workflows/ci-docker-build.yml jobs: · line 197
Quoted 10 lines
.github/workflows/ci-docker-build.yml +10
-0+ - name: Send results to Patchwork + if: always() && inputs.SERIES_ID != '' && inputs.SERIES_MBOX != '' + env: + PATCHWORK_TOKEN: ${{ secrets.PATCHWORK_TOKEN }} + run: | + ./ci/tools/patchwork.sh check \ + "${{ inputs.SERIES_ID }}" \ + "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}" \ + "${{ matrix.container }}" \ + "${{ job.status }}"
The rest looks good, once the branch deletion is clarified or fixed:
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Re: [LTP] [PATCH v6 4/4] ci: apply patchwork series in ci-docker-build workflow
Hi Cyril,
Quoted 2 lines · Cyril Hrubis
So we create a branch for each CI run but I do not see any place where we get rid of it. Shouldn't we remove it after the CI run?
We run it inside a container, so there's no need to delete it. One container is destroyed, we destroy also the branch.
- Andrea
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp