gitlab-ci: Switch the 'check-patch' & 'check-dco' jobs to use python-container
RFC PATCH 1/1 gitlab-ci: Switch the 'check-patch' & 'check-dco' jobs to use python-container
The 'check-patch' and 'check-dco' jobs only need Python and git for checking the patches, so it's not really necessary to use a container here that has all the other build dependencies installed. By installing "git" in the python container, we can use this light-weight container for these jobs instead.
---
Mark as RFC since I'm not sure whether we want to have "git" in the python container or not?
.gitlab-ci.d/static_checks.yml | 8 ++++---- tests/docker/dockerfiles/python.docker | 1 + 2 files changed, 5 insertions(+), 4 deletions(-)
.gitlab-ci.d/static_checks.yml +4 -4
@@ -1,8 +1,8 @@ check-patch: stage: build - image: $CI_REGISTRY_IMAGE/qemu/centos8:latest + image: $CI_REGISTRY_IMAGE/qemu/python:latest needs: - job: amd64-centos8-container + job: python-container script: - .gitlab-ci.d/check-patch.py variables:
@@ -15,9 +15,9 @@ check-patch: check-dco: stage: build - image: $CI_REGISTRY_IMAGE/qemu/centos8:latest + image: $CI_REGISTRY_IMAGE/qemu/python:latest needs: - job: amd64-centos8-container + job: python-container script: .gitlab-ci.d/check-dco.py variables: GIT_DEPTH: 1000
tests/docker/dockerfiles/python.docker +1 -0
@@ -6,6 +6,7 @@ MAINTAINER John Snow <[email protected]> # Please keep this list sorted alphabetically ENV PACKAGES \ gcc \ + git \ make \ pipenv \ python3 \
Signature · 1 line
2.27.0
Quoted 5 lines · Thomas Huth
The 'check-patch' and 'check-dco' jobs only need Python and git for checking the patches, so it's not really necessary to use a container here that has all the other build dependencies installed. By installing "git" in the python container, we can use this light-weight container for these jobs instead.
Our python container is far from light-weight....
$ time podman pull registry.gitlab.com/qemu-project/qemu/qemu/python
real 1m52.717s
user 1m32.327s
sys 0m19.453s
vs
$ time podman pull python:3.8-alpine
real 0m4.509s
user 0m3.780s
sys 0m1.052s
It is quicker to use the alpine python container and then just install 'git' on every job, than it is to use the pre-built qemu python container
Quoted 27 lines
Signed-off-by Thomas Huth <[email protected]>---
Mark as RFC since I'm not sure whether we want to have "git" in the python container or not?.gitlab-ci.d/static_checks.yml | 8 ++++---- tests/docker/dockerfiles/python.docker | 1 + 2 files changed, 5 insertions(+), 4 deletions(-).gitlab-ci.d/static_checks.yml +2
-2@@ -1,8 +1,8 @@ check-patch: stage: build - image: $CI_REGISTRY_IMAGE/qemu/centos8:latest + image: $CI_REGISTRY_IMAGE/qemu/python:latest needs: - job: amd64-centos8-container + job: python-container script: - .gitlab-ci.d/check-patch.py variables:@@ -15,9 +15,9 @@ check-patch:
.gitlab-ci.d/static_checks.yml check-patch: · line 23
Quoted 10 lines
.gitlab-ci.d/static_checks.yml +2
-2check-dco: stage: build - image: $CI_REGISTRY_IMAGE/qemu/centos8:latest + image: $CI_REGISTRY_IMAGE/qemu/python:latest needs: - job: amd64-centos8-container + job: python-container script: .gitlab-ci.d/check-dco.py variables: GIT_DEPTH: 1000
IOW this is sufficient:
image: python:3.8-alpine
needs: []
before_script:
- apk update
- apk add git
script: ./gitlab-ci.d/check-dco.py
I expect the same would work for check-patch.py container
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 76 lines · Daniel P. Berrangé
Quoted 5 lines · Thomas Huth
The 'check-patch' and 'check-dco' jobs only need Python and git for checking the patches, so it's not really necessary to use a container here that has all the other build dependencies installed. By installing "git" in the python container, we can use this light-weight container for these jobs instead.
Our python container is far from light-weight....
$ time podman pull registry.gitlab.com/qemu-project/qemu/qemu/pythonreal 1m52.717s
user 1m32.327s
sys 0m19.453svs
$ time podman pull python:3.8-alpinereal 0m4.509s
user 0m3.780s
sys 0m1.052sIt is quicker to use the alpine python container and then just install 'git' on every job, than it is to use the pre-built qemu python container
Quoted 27 lines
Signed-off-by Thomas Huth <[email protected]>---
Mark as RFC since I'm not sure whether we want to have "git" in the python container or not?.gitlab-ci.d/static_checks.yml | 8 ++++---- tests/docker/dockerfiles/python.docker | 1 + 2 files changed, 5 insertions(+), 4 deletions(-).gitlab-ci.d/static_checks.yml +2
-2@@ -1,8 +1,8 @@ check-patch: stage: build - image: $CI_REGISTRY_IMAGE/qemu/centos8:latest + image: $CI_REGISTRY_IMAGE/qemu/python:latest needs: - job: amd64-centos8-container + job: python-container script: - .gitlab-ci.d/check-patch.py variables:@@ -15,9 +15,9 @@ check-patch:.gitlab-ci.d/static_checks.yml check-patch: · line 23
Quoted 10 lines
.gitlab-ci.d/static_checks.yml +2
-2check-dco: stage: build - image: $CI_REGISTRY_IMAGE/qemu/centos8:latest + image: $CI_REGISTRY_IMAGE/qemu/python:latest needs: - job: amd64-centos8-container + job: python-container script: .gitlab-ci.d/check-dco.py variables: GIT_DEPTH: 1000IOW this is sufficient:
image: python:3.8-alpine
needs: []
before_script:
- apk update
- apk add git
script:./gitlab-ci.d/check-dco.py
Quoted 1 line
I expect the same would work for
check-patch.pycontainer
... or would it make sense to switch tests/docker/dockerfiles/python.docker to use alpine instead of fedora?
Thomas
Quoted 76 lines · Thomas Huth
Quoted 75 lines · Daniel P. Berrangé
Quoted 5 lines · Thomas Huth
The 'check-patch' and 'check-dco' jobs only need Python and git for checking the patches, so it's not really necessary to use a container here that has all the other build dependencies installed. By installing "git" in the python container, we can use this light-weight container for these jobs instead.
Our python container is far from light-weight....
$ time podman pull registry.gitlab.com/qemu-project/qemu/qemu/pythonreal 1m52.717s
user 1m32.327s
sys 0m19.453svs
$ time podman pull python:3.8-alpinereal 0m4.509s
user 0m3.780s
sys 0m1.052sIt is quicker to use the alpine python container and then just install 'git' on every job, than it is to use the pre-built qemu python container
Quoted 26 lines
Signed-off-by Thomas Huth <[email protected]>---
Mark as RFC since I'm not sure whether we want to have "git" in the python container or not?.gitlab-ci.d/static_checks.yml | 8 ++++---- tests/docker/dockerfiles/python.docker | 1 + 2 files changed, 5 insertions(+), 4 deletions(-).gitlab-ci.d/static_checks.yml +2
-2@@ -1,8 +1,8 @@ check-patch: stage: build - image: $CI_REGISTRY_IMAGE/qemu/centos8:latest + image: $CI_REGISTRY_IMAGE/qemu/python:latest needs: - job: amd64-centos8-container + job: python-container script: - .gitlab-ci.d/check-patch.py variables:.gitlab-ci.d/static_checks.yml check-patch: · line 22
Quoted 10 lines
.gitlab-ci.d/static_checks.yml +2
-2@@ -15,9 +15,9 @@ check-patch: check-dco: stage: build - image: $CI_REGISTRY_IMAGE/qemu/centos8:latest + image: $CI_REGISTRY_IMAGE/qemu/python:latest needs: - job: amd64-centos8-container + job: python-container script: .gitlab-ci.d/check-dco.py variables: GIT_DEPTH: 1000IOW this is sufficient:
image: python:3.8-alpine
needs: []
before_script:
- apk update
- apk add git
script:./gitlab-ci.d/check-dco.py
Quoted 4 lines
Quoted 1 line
I expect the same would work for
check-patch.pycontainer... or would it make sense to switch tests/docker/dockerfiles/python.docker to use alpine instead of fedora?
That could work too
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 77 lines · Daniel P. Berrangé
Quoted 76 lines · Thomas Huth
Quoted 75 lines · Daniel P. Berrangé
Quoted 5 lines · Thomas Huth
The 'check-patch' and 'check-dco' jobs only need Python and git for checking the patches, so it's not really necessary to use a container here that has all the other build dependencies installed. By installing "git" in the python container, we can use this light-weight container for these jobs instead.
Our python container is far from light-weight....
$ time podman pull registry.gitlab.com/qemu-project/qemu/qemu/pythonreal 1m52.717s
user 1m32.327s
sys 0m19.453svs
$ time podman pull python:3.8-alpinereal 0m4.509s
user 0m3.780s
sys 0m1.052sIt is quicker to use the alpine python container and then just install 'git' on every job, than it is to use the pre-built qemu python container
Quoted 26 lines
Signed-off-by Thomas Huth <[email protected]>---
Mark as RFC since I'm not sure whether we want to have "git" in the python container or not?.gitlab-ci.d/static_checks.yml | 8 ++++---- tests/docker/dockerfiles/python.docker | 1 + 2 files changed, 5 insertions(+), 4 deletions(-).gitlab-ci.d/static_checks.yml +2
-2@@ -1,8 +1,8 @@ check-patch: stage: build - image: $CI_REGISTRY_IMAGE/qemu/centos8:latest + image: $CI_REGISTRY_IMAGE/qemu/python:latest needs: - job: amd64-centos8-container + job: python-container script: - .gitlab-ci.d/check-patch.py variables:.gitlab-ci.d/static_checks.yml check-patch: · line 22
Quoted 10 lines
.gitlab-ci.d/static_checks.yml +2
-2@@ -15,9 +15,9 @@ check-patch: check-dco: stage: build - image: $CI_REGISTRY_IMAGE/qemu/centos8:latest + image: $CI_REGISTRY_IMAGE/qemu/python:latest needs: - job: amd64-centos8-container + job: python-container script: .gitlab-ci.d/check-dco.py variables: GIT_DEPTH: 1000IOW this is sufficient:
image: python:3.8-alpine
needs: []
before_script:
- apk update
- apk add git
script:./gitlab-ci.d/check-dco.py
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
I expect the same would work for
check-patch.pycontainer... or would it make sense to switch tests/docker/dockerfiles/python.docker to use alpine instead of fedora?
That could work too
I just tried it, and it did not work out as expected: The check-python-pipenv job needs a more sophisticated environment than the alpine container can provide easily (e.g. it still needs python-3.6), so I updated my patch with your idea instead and used the python:alpine container as a base there now.
Thomas