tests/docker: ensure container command is probed at most once
PATCH v3 1/1 tests/docker: ensure container command is probed at most once
The '--engine' arg accepts either 'podman' or 'docker', which is not sufficiently granular to map directly to a command. This means that docker.py still has to then probe the exact command to use.
Meanwhile the 'probe' command prints out the full command to use but this cannot be passed back to docker.py to avoid probing again, so the caching is only useful in the few case where we run a container directly bypassing docker.py.
Address this by replacing --engine with --command for docker.py.
This in turn requires the --container-engine configure arg to be replaced with --container-command.
With these changes the container command is probed at most once during configure and never again, while running in an unconfigured tree will still probe on demand.
---
In v3:
- Avoid bashism when testing probe result
configure | 25 +++++++------ tests/docker/Makefile.include | 13 ++++--- tests/docker/docker.py | 67 ++++++++++++----------------------- 3 files changed, 40 insertions(+), 65 deletions(-)
configure +12 -13
@@ -172,7 +172,7 @@ fi # some defaults, based on the host environment # default parameters -container_engine="auto" +container_command="" cpu="" cross_compile="no" cross_prefix=""
@@ -734,7 +734,7 @@ for opt do ;; --disable-containers) use_containers="no" ;; - --container-engine=*) container_engine="$optarg" + --container-command=*) container_command="$optarg" ;; --rust-target-triple=*) rust_target_triple="$optarg" ;;
@@ -869,7 +869,7 @@ Advanced options (experts only): --enable-debug enable common debug build options --cpu=CPU Build for host CPU [$cpu] --disable-containers don't use containers for cross-building - --container-engine=TYPE which container engine to use [$container_engine] + --container-command=CMD which container command to use [autodetect] --gdb=GDB-path gdb to use for gdbstub tests [$gdb_bin] --wasm64-32bit-address-limit Restrict wasm64 address space to 32-bit (default is to use the whole 64-bit range).
@@ -1291,12 +1291,12 @@ fi ########################################## # functions to probe cross compilers -runc="no" -if test $use_containers = "yes" && (has "docker" || has "podman"); then - runc=$($python "$source_path"/tests/docker/docker.py --engine "$container_engine" probe) - if test "$runc" != "no"; then - docker_py="$python $source_path/tests/docker/docker.py --engine $container_engine" - fi +if test "$container_command" = ""; then + container_command=$($python "$source_path"/tests/docker/docker.py probe) + test "$container_command" = "no" && container_command="" +fi +if test $use_containers = "yes" && test "$container_command" != ""; then + docker_py="$python $source_path/tests/docker/docker.py --command $container_command" fi # cross compilers defaults, can be overridden with --cross-cc-ARCH
@@ -1415,7 +1415,7 @@ probe_target_compiler() { esac for host in $container_hosts; do - test "$runc" != no || continue + test "$container_command" != "" || continue test "$host" = "$cpu" || continue case $target_arch in # debian-all-test-cross architectures
@@ -1736,9 +1736,8 @@ echo all: >> $config_host_mak echo "SRC_PATH=$source_path" >> $config_host_mak echo "TARGET_DIRS=$target_list" >> $config_host_mak echo "GDB=$gdb_bin" >> $config_host_mak -if test "$runc" != no; then - echo "RUNC=$runc" >> $config_host_mak - echo "CONTAINER_ENGINE=$container_engine" >> $config_host_mak +if test "$container_command" != ""; then + echo "CONTAINER_COMMAND=$container_command" >> $config_host_mak fi echo "SUBDIRS=$subdirs" >> $config_host_mak if test "$rust" != disabled; then
tests/docker/Makefile.include +6 -7
@@ -16,9 +16,8 @@ DOCKER_DEFAULT_REGISTRY := registry.gitlab.com/qemu-project/qemu endif DOCKER_REGISTRY := $(if $(REGISTRY),$(REGISTRY),$(DOCKER_DEFAULT_REGISTRY)) -CONTAINER_ENGINE = auto -DOCKER_SCRIPT=$(SRC_PATH)/tests/docker/docker.py --engine $(CONTAINER_ENGINE) -RUNC ?= $(shell $(DOCKER_SCRIPT) probe) +CONTAINER_COMMAND ?= $(shell $(SRC_PATH)/tests/docker/docker.py probe) +DOCKER_SCRIPT=$(SRC_PATH)/tests/docker/docker.py --command "$(CONTAINER_COMMAND)" CUR_TIME := $(shell date +%Y-%m-%d-%H.%M.%S.$$$$) DOCKER_SRC_COPY := $(BUILD_DIR)/docker-src.$(CUR_TIME)
@@ -41,7 +40,7 @@ docker-qemu-src: $(DOCKER_SRC_COPY) # General rule for building docker images. docker-image-%: $(DOCKER_FILES_DIR)/%.docker $(call quiet-command, \ - DOCKER_BUILDKIT=1 $(RUNC) build \ + DOCKER_BUILDKIT=1 $(CONTAINER_COMMAND) build \ $(if $(DOCKER_V),,--quiet) \ $(if $(NOCACHE),--no-cache, \ $(if $(DOCKER_REGISTRY),--cache-from $(DOCKER_REGISTRY)/qemu/$*)) \
@@ -152,7 +151,7 @@ $(foreach i,$(filter-out $(DOCKER_PARTIAL_IMAGES),$(DOCKER_IMAGES)), \ ) docker: - @echo 'Build QEMU and run tests inside $(RUNC) containers' + @echo 'Build QEMU and run tests inside $(CONTAINER_COMMAND) containers' @echo @echo 'Available targets:' @echo
@@ -219,10 +218,10 @@ docker-run: docker-qemu-src $(IMAGE) --executable $(EXECUTABLE), \ " COPYING $(EXECUTABLE) to $(IMAGE)")) $(call quiet-command, \ - $(RUNC) run \ + $(CONTAINER_COMMAND) run \ --rm \ $(if $(NOUSER),, \ - $(if $(filter docker,$(RUNC)), \ + $(if $(filter docker,$(CONTAINER_COMMAND)), \ -u $(UID), \ --userns keep-id \ ) \
tests/docker/docker.py +22 -45
145 lines not on this page · raw message
Signature · 1 line
2.55.0
Quoted 22 lines · Daniel P. Berrangé
The '--engine' arg accepts either 'podman' or 'docker', which is not sufficiently granular to map directly to a command. This means that
docker.pystill has to then probe the exact command to use.Meanwhile the 'probe' command prints out the full command to use but this cannot be passed back to
docker.pyto avoid probing again, so the caching is only useful in the few case where we run a container directly bypassingdocker.py.Address this by replacing --engine with --command for
docker.py.This in turn requires the --container-engine configure arg to be replaced with --container-command.
With these changes the container command is probed at most once during configure and never again, while running in an unconfigured tree will still probe on demand.
Signed-off-by Daniel P. Berrangé <[email protected]>---
Quoted 9 lines
In v3:
- Avoid bashism when testing probe result
configure | 25 +++++++------ tests/docker/Makefile.include | 13 ++++--- tests/docker/docker.py | 67 ++++++++++++----------------------- 3 files changed, 40 insertions(+), 65 deletions(-)
Thanks for the update, it's working fine now.
Also, docker.py run command is now fast again when given --command argument.
I'll pull this today.
Regards,
Pierrick
Quoted 22 lines · Daniel P. Berrangé
The '--engine' arg accepts either 'podman' or 'docker', which is not sufficiently granular to map directly to a command. This means that
docker.pystill has to then probe the exact command to use.Meanwhile the 'probe' command prints out the full command to use but this cannot be passed back to
docker.pyto avoid probing again, so the caching is only useful in the few case where we run a container directly bypassingdocker.py.Address this by replacing --engine with --command for
docker.py.This in turn requires the --container-engine configure arg to be replaced with --container-command.
With these changes the container command is probed at most once during configure and never again, while running in an unconfigured tree will still probe on demand.
Signed-off-by Daniel P. Berrangé <[email protected]>---
Quoted 9 lines
In v3:
- Avoid bashism when testing probe result
configure | 25 +++++++------ tests/docker/Makefile.include | 13 ++++--- tests/docker/docker.py | 67 ++++++++++++----------------------- 3 files changed, 40 insertions(+), 65 deletions(-)
This was merged into master (43bf3a0ce969782c4394fabc94777ba880b3598d). Thank you for your contribution!
Regards,
Pierrick