Skip to contentSkip to conversation

tests/docker: Add sqlite3 module to openSUSE Leap container

PATCH Superseded Fabiano Rosas 4 participants 6 messages Jan 16, 2024 – Jan 17, 2024

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 tests/docker: Add sqlite3 module to openSUSE Leap container

Avocado needs sqlite3:

Failed to load plugin from module "avocado.plugins.journal":
ImportError("Module 'sqlite3' is not installed.
Use: sudo zypper install python311 to install it")

Include the appropriate package in the dockerfile.

From 'zypper info python311':
"This package supplies rich command line features provided by readline, and sqlite3 support for the interpreter core, thus forming a so called "extended" runtime."

---
 tests/docker/dockerfiles/opensuse-leap.docker | 1 +
 1 file changed, 1 insertion(+)
tests/docker/dockerfiles/opensuse-leap.docker +1 -0
index dc0e36ce48..cf753383a4 100644
@@ -90,6 +90,7 @@ RUN zypper update -y && \
            pcre-devel-static \
            pipewire-devel \
            pkgconfig \
+           python311 \
            python311-base \
            python311-pip \
            python311-setuptools \
Signature · 1 line
2.35.3
#1 lore raw

Hi Fabiano,

Quoted 23 lines · Fabiano Rosas

Avocado needs sqlite3:

Failed to load plugin from module "avocado.plugins.journal":
ImportError("Module 'sqlite3' is not installed.
Use: sudo zypper install python311 to install it")

Include the appropriate package in the dockerfile.

From 'zypper info python311':
"This package supplies rich command line features provided by readline, and sqlite3 support for the interpreter core, thus forming a so called "extended" runtime."

---
  tests/docker/dockerfiles/opensuse-leap.docker | 1 +
  1 file changed, 1 insertion(+)
tests/docker/dockerfiles/opensuse-leap.docker +0 -0
index dc0e36ce48..cf753383a4 100644

tests/docker/dockerfiles/opensuse-leap.docker RUN zypper update -y && \ · line 96

Quoted 7 lines
tests/docker/dockerfiles/opensuse-leap.docker +1 -0
@@ -90,6 +90,7 @@ RUN zypper update -y && \
             pcre-devel-static \
             pipewire-devel \
             pkgconfig \
+           python311 \
             python311-base \
             python311-pip \
             python311-setuptools \

See in this file header:

# THIS FILE WAS AUTO-GENERATED
#
# $ lcitool dockerfile --layers all opensuse-leap-15 qemu #
# https://gitlab.com/libvirt/libvirt-ci

libvirt-ci maintains dependencies required to build QEMU, in this case since it is a 'testing QEMU' dependency, you might add it in the generate_dockerfile("opensuse-leap") call in tests/lcitool/refresh (otherwise raise an issue in libvirt-ci about it).

Regards,

Phil.

#2 lore raw
Quoted 42 lines · Philippe Mathieu-Daudé

Hi Fabiano,

Quoted 23 lines · Fabiano Rosas

Avocado needs sqlite3:

   Failed to load plugin from module "avocado.plugins.journal":
   ImportError("Module 'sqlite3' is not installed.
   Use: sudo zypper install python311 to install it")

Include the appropriate package in the dockerfile.

 From 'zypper info python311':
   "This package supplies rich command line features provided by readline, and sqlite3 support for the interpreter core, thus forming a so called "extended" runtime."

---
  tests/docker/dockerfiles/opensuse-leap.docker | 1 +

  1 file changed, 1 insertion(+)
tests/docker/dockerfiles/opensuse-leap.docker +0 -0
index dc0e36ce48..cf753383a4 100644

tests/docker/dockerfiles/opensuse-leap.docker RUN zypper update -y && \ · line 96

Quoted 7 lines
tests/docker/dockerfiles/opensuse-leap.docker +1 -0
@@ -90,6 +90,7 @@ RUN zypper update -y && \
              pcre-devel-static \
              pipewire-devel \
              pkgconfig \
+           python311 \
              python311-base \
              python311-pip \
              python311-setuptools \

See in this file header:

  # THIS FILE WAS AUTO-GENERATED
  #
  #  $ lcitool dockerfile --layers all opensuse-leap-15 qemu #
  # https://gitlab.com/libvirt/libvirt-ci

Quoted 5 lines

libvirt-ci maintains dependencies required to build QEMU, in this case since it is a 'testing QEMU' dependency, you might add it in the generate_dockerfile("opensuse-leap") call in tests/lcitool/refresh (otherwise raise an issue in libvirt-ci about it).

This is incorrect, correct fix in this thread:
https://lore.kernel.org/qemu-devel/CABJz62OD0ZZSY16sDNRiTuPEBygy9Y6rUgRrwoA6N_yg8cgGAQ@mail.gmail.com/

#3 lore raw
In reply to Fabiano Rosas
Quoted 23 lines · Fabiano Rosas

Avocado needs sqlite3:

Failed to load plugin from module "avocado.plugins.journal":
ImportError("Module 'sqlite3' is not installed.
Use: sudo zypper install python311 to install it")

Include the appropriate package in the dockerfile.

From 'zypper info python311':
"This package supplies rich command line features provided by readline, and sqlite3 support for the interpreter core, thus forming a so called "extended" runtime."

---
  tests/docker/dockerfiles/opensuse-leap.docker | 1 +
  1 file changed, 1 insertion(+)
tests/docker/dockerfiles/opensuse-leap.docker +0 -0
index dc0e36ce48..cf753383a4 100644

tests/docker/dockerfiles/opensuse-leap.docker RUN zypper update -y && \ · line 96

Quoted 7 lines
tests/docker/dockerfiles/opensuse-leap.docker +1 -0
@@ -90,6 +90,7 @@ RUN zypper update -y && \
             pcre-devel-static \
             pipewire-devel \
             pkgconfig \
+           python311 \
             python311-base \
             python311-pip \
             python311-setuptools \

AFAIK tests/docker/dockerfiles/opensuse-leap.docker is auto-generated, so this will be lost once somebody else runs lcitool again...

I don't really have a clue, but I guess this has to be fixed in the upstream lcitool first ( https://gitlab.com/libvirt/libvirt-ci ), and then we need to update our lcitool status in QEMU afterwards. Maybe Daniel can advise for the right stteps here...?

Thomas

#4 lore raw
Quoted 7 lines · Thomas Huth
Quoted 6 lines · Fabiano Rosas

Avocado needs sqlite3:

Failed to load plugin from module "avocado.plugins.journal":
ImportError("Module 'sqlite3' is not installed.
Use: sudo zypper install python311 to install it")

Quoted 6 lines
Quoted 6 lines

Include the appropriate package in the dockerfile.

From 'zypper info python311':
"This package supplies rich command line features provided by readline, and sqlite3 support for the interpreter core, thus forming a so called "extended" runtime."

Weird choice on Python's part to have sqlite3 support as part of the standard library IMO, but that's "batteries included" for you :)

Quoted 13 lines

Review RUN zypper update -y && \ · line 96

Quoted 9 lines

+++ b/tests/docker/dockerfiles/opensuse-leap.docker

(quoted diff) +1 -0
@@ -90,6 +90,7 @@ RUN zypper update -y && \
             pcre-devel-static \
             pipewire-devel \
             pkgconfig \
+           python311 \
             python311-base \
             python311-pip \
             python311-setuptools \

AFAIK tests/docker/dockerfiles/opensuse-leap.docker is auto-generated, so this will be lost once somebody else runs lcitool again...

Quoted 4 lines

I don't really have a clue, but I guess this has to be fixed in the upstream lcitool first ( https://gitlab.com/libvirt/libvirt-ci ), and then we need to update our lcitool status in QEMU afterwards. Maybe Daniel can advise for the right stteps here...?

It looks like a bunch of mappings are maintained in tests/lcitool/mappings.yml instead of the main lcitool repository. So I think you need to apply the diff below, then run

  $ git submodule update --init tests/lcitool/libvirt-ci
  $ tests/lcitool/refresh

to propagate the changes to the generated files.

tests/lcitool/mappings.yml +4 -0
index 0b908882f1..407c03301b 100644
@@ -59,6 +59,10 @@ mappings:
     CentOSStream8:
     OpenSUSELeap15:
 
+  python3-sqlite3:
+    CentOSStream8: python38
+    OpenSUSELeap15: python311
+
   python3-tomli:
     # test using tomllib
     apk:
tests/lcitool/projects/qemu.yml +1 -0
index 82092c9f17..149b15de57 100644
@@ -97,6 +97,7 @@ packages:
  - python3-pip
  - python3-sphinx
  - python3-sphinx-rtd-theme
+ - python3-sqlite3
  - python3-tomli
  - python3-venv
  - rpm2cpio
Signature · 1 line
Andrea Bolognani / Red Hat / Virtualization
#5 lore raw
Quoted 31 lines · Andrea Bolognani
Quoted 7 lines · Thomas Huth
Quoted 6 lines · Fabiano Rosas

Avocado needs sqlite3:

Failed to load plugin from module "avocado.plugins.journal":
ImportError("Module 'sqlite3' is not installed.
Use: sudo zypper install python311 to install it")

Quoted 6 lines
Quoted 6 lines

Include the appropriate package in the dockerfile.

From 'zypper info python311':
"This package supplies rich command line features provided by readline, and sqlite3 support for the interpreter core, thus forming a so called "extended" runtime."

Weird choice on Python's part to have sqlite3 support as part of the standard library IMO, but that's "batteries included" for you :)

Quoted 13 lines

Review RUN zypper update -y && \ · line 96

Quoted 9 lines

+++ b/tests/docker/dockerfiles/opensuse-leap.docker

(quoted diff) +1 -0
@@ -90,6 +90,7 @@ RUN zypper update -y && \
              pcre-devel-static \
              pipewire-devel \
              pkgconfig \
+           python311 \
              python311-base \
              python311-pip \
              python311-setuptools \

AFAIK tests/docker/dockerfiles/opensuse-leap.docker is auto-generated, so this will be lost once somebody else runs lcitool again...

Quoted 7 lines
Quoted 4 lines

I don't really have a clue, but I guess this has to be fixed in the upstream lcitool first ( https://gitlab.com/libvirt/libvirt-ci ), and then we need to update our lcitool status in QEMU afterwards. Maybe Daniel can advise for the right stteps here...?

It looks like a bunch of mappings are maintained in tests/lcitool/mappings.yml instead of the main lcitool repository.

See commit 32c0613113 for mapping override use:

ci, docker: update CentOS and OpenSUSE Python to non-EOL versions

Python 3.6 is at end-of-life. Update the libvirt-ci module to a version that supports overrides for targets and package mappings; this way, QEMU can use the newer versions provided by CentOS 8 (Python 3.8) and OpenSUSE 15.3 (Python 3.9).

docs/devel/testing.rst +5 -0
index 11c651ca08..362a26698b 100644
@@ -479,6 +479,12 @@ first to contribute the mapping to the ``libvirt-ci`` project:
    contains the ``mappings.yml`` update.  Then add the prerequisite and
    run ``make lcitool-refresh``.
 
+For enterprise distros that default to old, end-of-life versions of the
+Python runtime, QEMU uses a separate set of mappings that work with more
+recent versions.  These can be found in ``tests/lcitool/mappings.yml``.
+Modifying this file should not be necessary unless the new pre-requisite
+is a Python library or tool.
 
Quoted 29 lines

So
I think you need to apply the diff below, then run

   $ git submodule update --init tests/lcitool/libvirt-ci
   $ tests/lcitool/refresh

to propagate the changes to the generated files.

tests/lcitool/mappings.yml +4 -0
index 0b908882f1..407c03301b 100644
@@ -59,6 +59,10 @@ mappings:
      CentOSStream8:
      OpenSUSELeap15:
 
+  python3-sqlite3:
+    CentOSStream8: python38
+    OpenSUSELeap15: python311
+
    python3-tomli:
      # test using tomllib
      apk:
tests/lcitool/projects/qemu.yml +0 -0
index 82092c9f17..149b15de57 100644

tests/lcitool/projects/qemu.yml packages: · line 103

Quoted 7 lines
tests/lcitool/projects/qemu.yml +1 -0
@@ -97,6 +97,7 @@ packages:
   - python3-pip
   - python3-sphinx
   - python3-sphinx-rtd-theme
+ - python3-sqlite3
   - python3-tomli
   - python3-venv
   - rpm2cpio

LGTM.