Skip to contentSkip to conversation

tests/vm/netbsd: Remove missing py311-expat package

RFC PATCH Superseded Philippe Mathieu-Daudé 4 participants 6 messages Jan 17, 2024 – Feb 1, 2024

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 tests/vm/netbsd: Remove missing py311-expat package

Per commits a9dbde71da ("mkvenv: add better error message for broken or missing ensurepip") and 1dee66c693 ("tests/vm: add py310-expat to NetBSD"), we need py-expat to use ensurepip.

However the py311-expat package isn't available anymore:

### Installing packages ...
processing remote summary (http://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/amd64/9.3/All)...
database for http://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/amd64/9.3/All is up-to-date py311-expat is not available in the repository
...
calculating dependencies.../py311-expat is not available in the repository pkg_install error log can be found in /var/db/pkgin/pkg_install-err.log

Dropping it from the default packages list allows creating the NetBSD VM and build / test QEMU, without error from ensurepip.

This reverts commit 1dee66c693 ("tests/vm: add py310-expat to NetBSD").

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2109

---
RFC because ensurepip seems important due to cited commits
but I'm not sure where it is used, and this fixes my CI build.

---
 tests/vm/netbsd | 1 -
 1 file changed, 1 deletion(-)
tests/vm/netbsd +0 -1
index 649fcad353..a3f6dd6b3c 100755
@@ -31,7 +31,6 @@ class NetBSDVM(basevm.BaseVM):
         "pkgconf",
         "xz",
         "python311",
-        "py311-expat",
         "ninja-build",
 
         # gnu tools
Signature · 1 line
2.41.0
Tested-by Reviewed-by on v1 1/1
#1 lore raw
Quoted 17 lines · Philippe Mathieu-Daudé

Per commits a9dbde71da ("mkvenv: add better error message for broken or missing ensurepip") and 1dee66c693 ("tests/vm: add py310-expat to NetBSD"), we need py-expat to use ensurepip.

However the py311-expat package isn't available anymore:

### Installing packages ...
processing remote summary (http://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/amd64/9.3/All)...
database for http://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/amd64/9.3/All is up-to-date py311-expat is not available in the repository
...
calculating dependencies.../py311-expat is not available in the repository pkg_install error log can be found in /var/db/pkgin/pkg_install-err.log

Dropping it from the default packages list allows creating the NetBSD VM and build / test QEMU, without error from ensurepip.

Quoted 7 lines

This reverts commit 1dee66c693 ("tests/vm: add py310-expat to NetBSD").

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2109

---
RFC because ensurepip seems important due to cited commits
but I'm not sure where it is used, and this fixes my CI build.

Thanks for tackling it, I also ran into this problem already and just did not have enough spare time yet to investigate.

I searched a little bit, and found this text here:
https://mail-index.netbsd.org/netbsd-announce/2024/01/01/msg000360.html

"
- Several packages have been folded into base packages. While the result is simpler, those updating may need to force-remove the secondary packages, depending on the update method. When doing make replace, one has to pkg_delete -f the secondary packages. pkgin handles at least the python packages correctly, removing the split package when updating python. Specific packages and the former packages now included:

* cairo: cairo-gobject
* python: py-cElementTree py-curses py-cursespanel py-expat py-readline py-sqlite3
"

So it seems like the py-expat package has simply been merged into a base package now. Thus I think your patch is fine. It also fixes the problem for me, so:

#2 lore raw
Quoted 40 lines · Thomas Huth
Quoted 17 lines · Philippe Mathieu-Daudé

Per commits a9dbde71da ("mkvenv: add better error message for broken or missing ensurepip") and 1dee66c693 ("tests/vm: add py310-expat to NetBSD"), we need py-expat to use ensurepip.

However the py311-expat package isn't available anymore:

   ### Installing packages ...
   processing remote summary (http://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/amd64/9.3/All)...
   database for http://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/amd64/9.3/All is up-to-date py311-expat is not available in the repository
   ...
   calculating dependencies.../py311-expat is not available in the repository pkg_install error log can be found in /var/db/pkgin/pkg_install-err.log

Dropping it from the default packages list allows creating the NetBSD VM and build / test QEMU, without error from ensurepip.

Quoted 7 lines

This reverts commit 1dee66c693 ("tests/vm: add py310-expat to NetBSD").

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2109

---
RFC because ensurepip seems important due to cited commits but I'm not sure where it is used, and this fixes my CI build.

Thanks for tackling it, I also ran into this problem already and just did not have enough spare time yet to investigate.

I searched a little bit, and found this text here:
https://mail-index.netbsd.org/netbsd-announce/2024/01/01/msg000360.html

"
- Several packages have been folded into base packages.  While the result is simpler, those updating may need to force-remove the secondary packages, depending on the update method.  When doing make replace, one has to pkg_delete -f the secondary packages. pkgin handles at least the python packages correctly, removing the split package when updating python.  Specific packages and the former packages now included:

Quoted 4 lines

     * cairo: cairo-gobject
     * python: py-cElementTree py-curses py-cursespanel py-expat py-readline py-sqlite3
"

Thanks for digging, I'll update the description.

Quoted 5 lines

So it seems like the py-expat package has simply been merged into a base package now. Thus I think your patch is fine. It also fixes the problem for me, so:

#3 lore raw
Quoted 48 lines · Philippe Mathieu-Daudé
Quoted 46 lines · Thomas Huth
Quoted 22 lines · Philippe Mathieu-Daudé

Per commits a9dbde71da ("mkvenv: add better error message for broken or missing ensurepip") and 1dee66c693 ("tests/vm: add py310-expat to NetBSD"), we need py-expat to use ensurepip.

However the py311-expat package isn't available anymore:

### Installing packages ...
processing remote summary
(http://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/amd64/9.3/All)...
database for
http://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/amd64/9.3/All is up-to-date
py311-expat is not available in the repository
...
calculating dependencies.../py311-expat is not available in the
repository
pkg_install error log can be found in
/var/db/pkgin/pkg_install-err.log

Dropping it from the default packages list allows creating the NetBSD VM and build / test QEMU, without error from ensurepip.

Quoted 7 lines

This reverts commit 1dee66c693 ("tests/vm: add py310-expat to NetBSD").

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2109

---
RFC because ensurepip seems important due to cited commits
but I'm not sure where it is used, and this fixes my CI build.

Thanks for tackling it, I also ran into this problem already and just did not have enough spare time yet to investigate.

I searched a little bit, and found this text here:
https://mail-index.netbsd.org/netbsd-announce/2024/01/01/msg000360.html

"
- Several packages have been folded into base packages. While the result is simpler, those updating may need to force-remove the secondary packages, depending on the update method. When doing make replace, one has to pkg_delete -f the secondary packages. pkgin handles at least the python packages correctly, removing the split package when updating python. Specific packages and the former packages now included:

Quoted 15 lines
Quoted 4 lines

* cairo: cairo-gobject
* python: py-cElementTree py-curses py-cursespanel py-expat py-readline py-sqlite3
"

Thanks for digging, I'll update the description.

Quoted 7 lines

So it seems like the py-expat package has simply been merged into a base package now. Thus I think your patch is fine. It also fixes the problem for me, so:

Quite late, but thank you both. If py-expat was folded in, then this is definitely safe. mkvenv.py could possibly use a small update to its error message hint:

"(Hint: NetBSD's pkgsrc debundles this to e.g. 'py310-expat'.)"

I don't know if this is safe to remove in its entirety just yet, because I don't know right away if any NetBSD distributions that we consider "supported" still use a de-bundled py-expat. (Maybe this error is still helpful for certain users in some cases we technically support?)

If that's not the case, then we can simplify mkvenv and remove this error handling case entirely.

--js

#4 lore raw
In reply to Philippe Mathieu-Daudé
Quoted 3 lines · Philippe Mathieu-Daudé

Per commits a9dbde71da ("mkvenv: add better error message for broken or missing ensurepip") and 1dee66c693 ("tests/vm: add py310-expat to NetBSD"), we need py-expat to use ensurepip.

Queued to testing/next, thanks.

Signature · 2 lines
Alex Bennée
Virtualisation Tech Lead @ Linaro