Skip to contentSkip to conversation

ci: add Fedora 35 container in GHA

PATCH Superseded David Marchand 2 participants 9 messages Nov 16, 2021 – Apr 6, 2022

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 ci: add Fedora 35 container in GHA

Build DPDK with Fedora 35 containers.
Differences with the Ubuntu GHA vm images:
- tasks are run as root in containers, no need for sudo,
- compiler must be explicitly installed: clang is not in container images,
- GHA artifacts can't contain a ':' in their name, and must be filtered,

---
 .ci/linux-build.sh          |  3 ++
 .ci/linux-setup.sh          |  3 ++
 .github/workflows/build.yml | 67 +++++++++++++++++++++++++++++++++++--
 3 files changed, 71 insertions(+), 2 deletions(-)
.ci/linux-build.sh +3 -0
index ef0bd099be..e34a74ab64 100755
@@ -1,5 +1,8 @@
 #!/bin/sh -xe
 
+# Builds are run as root in containers, no need for sudo
+[ "$(id -u)" != '0' ] || alias sudo=
+
 on_error() {
     if [ $? = 0 ]; then
         exit
.ci/linux-setup.sh +3 -0
index ca35b4b38d..f5ef6b1b98 100755
@@ -1,5 +1,8 @@
 #!/bin/sh -xe
 
+# Builds are run as root in containers, no need for sudo
+[ "$(id -u)" != '0' ] || alias sudo=
+
 # need to install as 'root' since some of the unit tests won't run without it
 sudo python3 -m pip install --upgrade 'meson==0.49.2'
 
.github/workflows/build.yml +65 -2

79 lines not on this page · raw message

Signature · 1 line
2.23.0
Acked-by on v1 1/1
#1 lore raw
Quoted 8 lines · David Marchand

Build DPDK with Fedora 35 containers.
Differences with the Ubuntu GHA vm images:
- tasks are run as root in containers, no need for sudo,
- compiler must be explicitly installed: clang is not in container images,
- GHA artifacts can't contain a ':' in their name, and must be filtered,

---

This is awesome!

#2 lore raw
Quoted 8 lines · Aaron Conole
Quoted 7 lines

Build DPDK with Fedora 35 containers.
Differences with the Ubuntu GHA vm images:
- tasks are run as root in containers, no need for sudo,
- compiler must be explicitly installed: clang is not in container images,
- GHA artifacts can't contain a ':' in their name, and must be filtered,

Unfortunately, while testing it a bit more (having it in my tree), I got some random failures when pulling container images.
I suspect docker limits image pulls... but I am not sure yet. I'll have to investigate this.

--
David Marchand

#3 lore raw
Quoted 7 lines · David Marchand
Quoted 6 lines · Aaron Conole
Quoted 6 lines

Build DPDK with Fedora 35 containers.
Differences with the Ubuntu GHA vm images:
- tasks are run as root in containers, no need for sudo,
- compiler must be explicitly installed: clang is not in container images,
- GHA artifacts can't contain a ':' in their name, and must be filtered,

Quoted 7 lines
Quoted 2 lines
Quoted 1 line

Unfortunately, while testing it a bit more (having it in my tree), I got some random failures when pulling container images.
I suspect docker limits image pulls... but I am not sure yet. I'll have to investigate this.

I do see that there were warnings even with the github robot's run - but those builds eventually went through. I guess if it isn't stable, it makes sense to not deploy.

One alternative is to create a separate "workflow" for github action that can be called "fedora" and that will be recorded differently by the robot and then if there's a failure at least it will be a different "test." This probably isn't preferable, though - since it creates another test in the UI.

Quoted 3 lines

--
David Marchand

#4 lore raw

PATCH v2 1/1 ci: add Fedora 35 container in GHA

In reply to David Marchand

Build DPDK with Fedora 35 containers.

GHA container support does not allow caching images and docker hub seems to limit image pulls.
On the other hand, the Fedora project hub does not seem to limit them, so prefer this hub.
Nevertheless, let's try to be good citizens and cache (once a day) a prepared image for subsequent builds.

Differences with the Ubuntu GHA vm images:
- tasks are run as root in containers, no need for sudo,
- compiler must be explicitly installed,
- GHA artifacts can't contain a ':' in their name, and must be filtered,

---
Changes since v1:
- switched to Fedora project hub,
- cached container image,

---
 .ci/linux-build.sh          |   3 +
 .ci/linux-setup.sh          |   3 +
 .github/workflows/build.yml | 106 +++++++++++++++++++++++++++++++++++-
 3 files changed, 110 insertions(+), 2 deletions(-)
.ci/linux-build.sh +3 -0
index 69940dec8b..774a1441bf 100755
@@ -1,5 +1,8 @@
 #!/bin/sh -xe
 
+# Builds are run as root in containers, no need for sudo
+[ "$(id -u)" != '0' ] || alias sudo=
+
 on_error() {
     if [ $? = 0 ]; then
         exit
.ci/linux-setup.sh +3 -0
index ca35b4b38d..f5ef6b1b98 100755
@@ -1,5 +1,8 @@
 #!/bin/sh -xe
 
+# Builds are run as root in containers, no need for sudo
+[ "$(id -u)" != '0' ] || alias sudo=
+
 # need to install as 'root' since some of the unit tests won't run without it
 sudo python3 -m pip install --upgrade 'meson==0.49.2'
 
.github/workflows/build.yml +104 -2

118 lines not on this page · raw message

Signature · 1 line
2.23.0
#5 lore raw
Quoted 9 lines · David Marchand

Build DPDK with Fedora 35 containers.

GHA container support does not allow caching images and docker hub seems to limit image pulls.
On the other hand, the Fedora project hub does not seem to limit them, so prefer this hub.
Nevertheless, let's try to be good citizens and cache (once a day) a prepared image for subsequent builds.

We could move to a pipeline approach, with a first stage job preparing the image and pushing to the cache, and the second stage pulling the image from the cache and starting compilations.
Food for thoughts...

Quoted 6 lines

Differences with the Ubuntu GHA vm images:
- tasks are run as root in containers, no need for sudo,
- compiler must be explicitly installed,
- GHA artifacts can't contain a ':' in their name, and must be filtered,

Signature · 1 line
David Marchand
#6 lore raw

PATCH v3 1/1 ci: add Fedora 35 container in GHA

In reply to David Marchand

Build DPDK with Fedora 35 containers.

GHA container support does not allow caching images and docker hub seems to limit image pulls.
On the other hand, the Fedora project hub does not seem to limit them, so prefer this hub.
Nevertheless, let's try to be good citizens and cache (once a day) a prepared image for subsequent builds.
This preparation is done in a first prepare-container-images job. The rpm-container-builds job then depends on it with a 'needs:' tag.

Differences with builds in Ubuntu GHA vm images:
- tasks are run as root in containers, no need for sudo,
- compiler must be explicitly installed,
- GHA artifacts can't contain a ':' in their name, and must be filtered,
- environment variables are not inherited and must be passed explicitly,

---
Changes since v2:
- used a two stage pipeline so that container image generation is done only once,
- changed image name local:dpdk to dpdk-local (closer to conventions),

Changes since v1:
- switched to Fedora project hub,
- cached container image,

---
 .ci/linux-build.sh          |   3 +
 .ci/linux-setup.sh          |   3 +
 .github/workflows/build.yml | 142 +++++++++++++++++++++++++++++++++++-
 3 files changed, 146 insertions(+), 2 deletions(-)
.ci/linux-build.sh +3 -0
index 69940dec8b..774a1441bf 100755
@@ -1,5 +1,8 @@
 #!/bin/sh -xe
 
+# Builds are run as root in containers, no need for sudo
+[ "$(id -u)" != '0' ] || alias sudo=
+
 on_error() {
     if [ $? = 0 ]; then
         exit
.ci/linux-setup.sh +3 -0
index ca35b4b38d..f5ef6b1b98 100755
@@ -1,5 +1,8 @@
 #!/bin/sh -xe
 
+# Builds are run as root in containers, no need for sudo
+[ "$(id -u)" != '0' ] || alias sudo=
+
 # need to install as 'root' since some of the unit tests won't run without it
 sudo python3 -m pip install --upgrade 'meson==0.49.2'
 
.github/workflows/build.yml +140 -2

154 lines not on this page · raw message

Signature · 1 line
2.23.0
Acked-by on v3 1/1
#7 lore raw
Quoted 24 lines · David Marchand

Build DPDK with Fedora 35 containers.

GHA container support does not allow caching images and docker hub seems to limit image pulls.
On the other hand, the Fedora project hub does not seem to limit them, so prefer this hub.
Nevertheless, let's try to be good citizens and cache (once a day) a prepared image for subsequent builds.
This preparation is done in a first prepare-container-images job. The rpm-container-builds job then depends on it with a 'needs:' tag.

Differences with builds in Ubuntu GHA vm images:
- tasks are run as root in containers, no need for sudo,
- compiler must be explicitly installed,
- GHA artifacts can't contain a ':' in their name, and must be filtered,
- environment variables are not inherited and must be passed explicitly,

---
Changes since v2:
- used a two stage pipeline so that container image generation is done only once,
- changed image name local:dpdk to dpdk-local (closer to conventions),

Quoted 5 lines

Changes since v1:
- switched to Fedora project hub,
- cached container image,

---

This is fantastic!

author Applied
#8 lore raw
Quoted 13 lines · Aaron Conole
Quoted 11 lines · David Marchand

Build DPDK with Fedora 35 containers.

GHA container support does not allow caching images and docker hub seems to limit image pulls.
On the other hand, the Fedora project hub does not seem to limit them, so prefer this hub.
Nevertheless, let's try to be good citizens and cache (once a day) a prepared image for subsequent builds.
This preparation is done in a first prepare-container-images job. The rpm-container-builds job then depends on it with a 'needs:' tag.

Quoted 8 lines
Quoted 7 lines

Differences with builds in Ubuntu GHA vm images:
- tasks are run as root in containers, no need for sudo,
- compiler must be explicitly installed,
- GHA artifacts can't contain a ':' in their name, and must be filtered,
- environment variables are not inherited and must be passed explicitly,

Applied, thanks.

Signature · 1 line
David Marchand