uutils-coreutils: move from meta-openembedded
PATCH 1/1 uutils-coreutils: move from meta-openembedded
From: Etienne Cordonnier <[email protected]>
uutils-coreutils, the Rust reimplementation of GNU coreutils, is mature enough for production use, so move the recipe from meta-oe to oe-core. It provides and rprovides coreutils, and is excluded from world unless selected as the coreutils provider.
AI-Generated: Uses Claude Sonnet 5.5
--- .../conf/distro/include/default-providers.inc | 3 + meta/conf/distro/include/maintainers.inc | 1 + .../uutils-coreutils-crates.inc | 1250 +++++++++++++++++ .../uutils-coreutils_0.12.0.bb | 86 ++ 4 files changed, 1340 insertions(+) create mode 100644 meta/recipes-core/uutils-coreutils/uutils-coreutils-crates.inc create mode 100644 meta/recipes-core/uutils-coreutils/uutils-coreutils_0.12.0.bb
meta/conf/distro/include/default-providers.inc +3 -0
@@ -38,6 +38,9 @@ VIRTUAL-RUNTIME_libsdl2 ?= "libsdl2" # # Default recipe providers # +PREFERRED_PROVIDER_coreutils ?= "coreutils" +PREFERRED_PROVIDER_coreutils-native ?= "coreutils-native" +PREFERRED_PROVIDER_nativesdk-coreutils ?= "nativesdk-coreutils" PREFERRED_PROVIDER_dbus-glib ?= "dbus-glib" PREFERRED_PROVIDER_dbus-glib-native ?= "dbus-glib-native" PREFERRED_PROVIDER_gdk-pixbuf ?= "gdk-pixbuf"
meta/conf/distro/include/maintainers.inc +1 -0
@@ -867,6 +867,7 @@ RECIPE_MAINTAINER:pn-utfcpp = "Unassigned <[email protected]>" RECIPE_MAINTAINER:pn-util-linux = "Chen Qi <[email protected]>" RECIPE_MAINTAINER:pn-util-linux-libuuid = "Chen Qi <[email protected]>" RECIPE_MAINTAINER:pn-util-macros = "Unassigned <[email protected]>" +RECIPE_MAINTAINER:pn-uutils-coreutils = "Etienne Cordonnier <[email protected]>" RECIPE_MAINTAINER:pn-v86d = "Unassigned <[email protected]>" RECIPE_MAINTAINER:pn-vala = "Unassigned <[email protected]>" RECIPE_MAINTAINER:pn-valgrind = "Mingli Yu <[email protected]>"
meta/recipes-core/uutils-coreutils/uutils-coreutils-crates.inc new file +1250 -0
1,250 lines not on this page · raw message
meta/recipes-core/uutils-coreutils/uutils-coreutils_0.12.0.bb new file +86 -0
86 lines not on this page · raw message
Signature · 1 line
2.43.0
Quoted 4 lines · Etienne Cordonnier
uutils-coreutils, the Rust reimplementation of GNU coreutils, is mature enough for production use, so move the recipe from meta-oe to oe-core. It provides and rprovides coreutils, and is excluded from world unless selected as the coreutils provider.
Total disregard for the requirements imposed by the GPL is helpful to it being "mature enough for production use", of course. See the downplayed mention in the 0.12.0 release notes:
Also, some contributions had carried comments and code over from GNU coreutils, against our own rule that uutils is an original implementation. We reviewed the tree and rewrote what we found, in comments and in the few functions that followed the C too closely; no behaviour change. The GNU coreutils maintainers have been notified.
Quoted 1 line
+SUMMARY = "coreutils ~ GNU coreutils (updated); implemented as universal (cross-platform) utils, written in Rust"
This summary reads as if it is trying to claim that uutils is the official GNU coreutils project.
I'm not familiar enough with openembedded, but a concern raised by other distributions and me is that supporting two packages as "coreutils" increases maintainers' workload because the implementations are bound to diverge, requiring otherwise unnecessary code churn. E.g., for around a year, Ubuntu 26.04 had the following:
$ export LC_ALL=en_US.UTF-8
$ printf 'USA\nU.S.A\nU-S-A\n' | sort -u
USA
$ printf 'USA\nU.S.A\nU-S-A\n' | gnusort -u
U-S-A
U.S.A
USAIt still has the following issue [1]:
$ export LC_ALL=en_US.UTF-8
$ echo a A b B | tr ' ' '\n' | sort > tst1.txt
$ echo a A b C | tr ' ' '\n' | sort > tst2.txt
$ comm -23 tst1.txt tst2.txt
comm: file 1 is not in sorted order
comm: file 2 is not in sorted order
B
comm: input is not in sorted order
$ gnucomm -23 tst1.txt tst2.txt
BCollin
[1] https://bugs.launchpad.net/ubuntu/+source/rust-coreutils/+bug/2169780
Quoted 50 lines · Collin Funk via lists.openembedded.org
Quoted 4 lines · Etienne Cordonnier
uutils-coreutils, the Rust reimplementation of GNU coreutils, is mature enough for production use, so move the recipe from meta-oe to oe-core. It provides and rprovides coreutils, and is excluded from world unless selected as the coreutils provider.
Total disregard for the requirements imposed by the GPL is helpful to it being "mature enough for production use", of course. See the downplayed mention in the 0.12.0 release notes:
Also, some contributions had carried comments and code over from GNU coreutils, against our own rule that uutils is an original implementation. We reviewed the tree and rewrote what we found, in comments and in the few functions that followed the C too closely; no behaviour change. The GNU coreutils maintainers have been notified.
Quoted 1 line
+SUMMARY = "coreutils ~ GNU coreutils (updated); implemented as universal (cross-platform) utils, written in Rust"
This summary reads as if it is trying to claim that uutils is the official GNU coreutils project.
I'm not familiar enough with openembedded, but a concern raised by other distributions and me is that supporting two packages as "coreutils" increases maintainers' workload because the implementations are bound to diverge, requiring otherwise unnecessary code churn. E.g., for around a year, Ubuntu 26.04 had the following:
$ export LC_ALL=en_US.UTF-8 $ printf 'USA\nU.S.A\nU-S-A\n' | sort -u USA $ printf 'USA\nU.S.A\nU-S-A\n' | gnusort -u U-S-A U.S.A USAIt still has the following issue [1]:
$ export LC_ALL=en_US.UTF-8 $ echo a A b B | tr ' ' '\n' | sort > tst1.txt $ echo a A b C | tr ' ' '\n' | sort > tst2.txt $ comm -23 tst1.txt tst2.txt comm: file 1 is not in sorted order comm: file 2 is not in sorted order B comm: input is not in sorted order $ gnucomm -23 tst1.txt tst2.txt B
Quoted 3 lines
Collin
[1] https://bugs.launchpad.net/ubuntu/+source/rust-coreutils/+bug/2169780
For reference we also had issues with uutils-coreutils implementations:
* classes/base: prefer gnu-prefixed HOSTTOOLS - openembedded-core - OpenEmbedded Core layer
https://git.openembedded.org/openembedded-core/commit/?id=16f2684ebeffa72b5d90525cf9102751b68c298e
* 16028 – Investigate pseudo/uutils incompatibilities
https://bugzilla.yoctoproject.org/show_bug.cgi?id=16028
Regards,
Signature · 2 lines
Yoann Congal Smile ECS
Quoted 6 lines · Etienne Cordonnier via lists.openembedded.org
uutils-coreutils, the Rust reimplementation of GNU coreutils, is mature enough for production use, so move the recipe from meta-oe to oe-core. It provides and rprovides coreutils, and is excluded from world unless selected as the coreutils provider.
This needs a bit more justification. What is achieved by moving it to core? What specifically makes this component mature enough? What tests were performed with it? Does it pass the full autobuilder matrix when selected as coreutiuls provider? Should we add that test to the autobuilder? Should we make uutils-coreutils the default choice, or maybe the only choice? Does it work as a provider of coreutils-native, and is it compatible with pseudo?
Adding things to core has a pretty high bar, and the above commit message does not meet it.
Alex
Hi Alex,
- what is achieved by moving it to core is improving no-gplv3.inc for users who are not using gplv3 and still want full coreutils syntax support. It will also make it possible to add it to poky-altcfg.conf, and generally make it easier to consume without needing meta-openembedded.
- what specifically makes this component mature enough is:
- I've added the recipe to meta-openembedded in July 2022 and it has been
continuously tested and improved since (e.g. I fixed the musl support upstream, the cross-compilation of stdbuf upstream, some buildpath issue)
- its adoption in Ubuntu 26.04 as default coreutils implementation and
the resulting large-scale usage and bug-fixing
- its availability in Windows as "Coreutils for Windows" starting in June
2026 as a package maintained by Microsoft, and the resulting large-scale usage and bug-fixing
- its massive adoption in various distros and package-managers (see
https://uutils.org/users/ )
- its >95% pass-rate of the GNU testing suite in release 0.0.12 (some
tests are still skipped)
- anecdotally, my own usage in production on embedded devices for several
years without issues (on arm and qemu-arm64)
- "should we make uutils-coreutils the default choice, or maybe the only choice?"
-> maybe in several years when uutils-coreutils has matured even more and
has 100% test coverage of the GNU test suite? At the moment I think the safest choice on critical devices is still GNU coreutils, so I wouldn't remove or make rust coreutils the default or only choice. Also rust binaries are bigger than C binaries so there is some tradeoff involved. GNU coreutils 9.12 main package is 10.3MB big in my test build, compared to 13.6MB for the uutils-coreutils 0.0.12 main package (uutils-coreutils uses a multi-binary executable to save space, GNU coreutils doesn't).
- "Does it work as a provider of coreutils-native and is it compatible with pseudo?"
- the recipe uutils-coreutils-native compiles, but it's not compatible
with pseudo because of differences in syscalls, and it's not suitable as a replacement for coreutils-native at the moment. I'll try to make it work with pseudo in the future, but I think almost all users consume it as a target recipe and that's where it provides most value.
- "Does it pass the full autobuilder matrix when selected as coreutils provider? Should we add that test to the autobuilder?"
-> it should, but I've not tested the full matrix of the autobuilder. I
would like to add the recipe to poky-altcfg.conf, and as far as I understand this would add it to the autobuilder as side-effect. I am also working on adding ptest support for the recipe and can send this if necessary for the acceptance to oe-core.
Étienne
Quoted 24 lines · Alexander Kanavin
On Sat, 10 Oct 2026 at 13:30, Etienne Cordonnier via lists.openembedded.org
<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openembedded.org&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=Pf9ypvX_MT4dU4cYC3brv9jpglRcRrrz1luHXObxYfW6J0oO8XAW6KnUoiQHzUES&s=QSmPEv1BkubeZVNksJqkHkppRkyy-Bac7QdkZuoYqaA&e=>Quoted 6 lines · <[email protected]> wrote:
uutils-coreutils, the Rust reimplementation of GNU coreutils, is mature enough for production use, so move the recipe from meta-oe to oe-core. It provides and rprovides coreutils, and is excluded from world unless selected as the coreutils provider.
This needs a bit more justification. What is achieved by moving it to core? What specifically makes this component mature enough? What tests were performed with it? Does it pass the full autobuilder matrix when selected as coreutiuls provider? Should we add that test to the autobuilder? Should we make uutils-coreutils the default choice, or maybe the only choice? Does it work as a provider of coreutils-native, and is it compatible with pseudo?
Adding things to core has a pretty high bar, and the above commit message does not meet it.
Alex
Quoted 4 lines · Etienne Cordonnier
- what is achieved by moving it to core is improving no-gplv3.inc for users who are not using gplv3 and still want full coreutils syntax support. It will also make it possible to add it to poky-altcfg.conf, and generally make it easier to consume without needing meta-openembedded.
That, of course, assumes none of the code is tainted after the attempted cleanup in 0.12.0, which only came after multiple warnings from me. It is difficult, speaking personally, for me to believe that the uutils project has any respect at all for licensing.
Quoted 2 lines
- its adoption in Ubuntu 26.04 as default coreutils implementation and
the resulting large-scale usage and bug-fixing
That is the positive side of things. The negative side is that there were many FTBFS and multiple breakages of Ubuntu ISOs. E.g., one missing a setuid bit on sudo. I highly doubt Canonical engineers enjoy dealing with this. It seems to be a decision made by the company leadership. I can't say for certain, though, since Canonical didn't reach out to me or anyone else working on GNU coreutils, who could have warned them about the easily foreseeable issues they faced.
Quoted 2 lines
- its >95% pass-rate of the GNU testing suite in release 0.0.12 (some
tests are still skipped)
Note that the GNU coreutils test suite is good, in my biased opinion, but it is not really meant to create a new, perfectly compatible implementation to replace us. I have always found the use of these percentages dubious, but I understand the need for some comparison.
Quoted 5 lines
Also rust binaries are bigger than C binaries so there is some tradeoff involved. GNU coreutils 9.12 main package is 10.3MB big in my test build, compared to 13.6MB for the uutils-coreutils 0.0.12 main package (uutils-coreutils uses a multi-binary executable to save space, GNU coreutils doesn't).
This is a very unfair comparison. GNU coreutils has been able to be built as a single binary since 2014 [1]. If there were a need for it in meta-openembedded, I am sure it could be packaged relatively easily. Red Hat uses it for their container images, for example:
$ podman run --rm -it redhat/ubi10
$ du --human-readable /bin/coreutils
1.4M /bin/coreutilsCollin
[1] https://github.com/coreutils/coreutils/commit/71e2ea773414b2316bbe6b803b9a52c38d3752e8
Alright. Here's a suggested plan for you:
- add ptest support to the recipe. It must have feature parity with the GNU recipe.
- investigate and propose a plan for pseudo compatibility. It's not easy:
pseudo relies on substituting libc API, and rust binaries issue kernel syscalls directly.
- address concerns of 'license-washing'. A few of us are deeply uncomfortable with this recent wave of component rewrites (especially AI assisted which causes dubious copyrights) where the primary goal is not technical improvement but simply getting rid of the GPL. The purpose of open source is not serving commercial interests, it's making computing better for everyone.
- get the core community distributions (which is, Fedora and Debian) to have a plan and timeline of adoping uutils as the default. This is the ultimate test of maturity.
- propose a test plan for Yocto. Merely sticking it into poky-altcfg is actually a regression in testing, because then sysvinit images will be tested only with GNU, and systemd images will be tested only with uutils.
- last, but not least. Actually this is the most important one: fix your relationship with GNU coreutils. It's painful to watch you two bicker, and I do not want the Yocto project to be dragged into this.
Alex
Quoted 79 lines · Etienne Cordonnier
Hi Alex,
- what is achieved by moving it to core is improving no-gplv3.inc for users who are not using gplv3 and still want full coreutils syntax support. It will also make it possible to add it to poky-altcfg.conf, and generally make it easier to consume without needing meta-openembedded.- what specifically makes this component mature enough is:
- I've added the recipe to meta-openembedded in July 2022 and it has
been continuously tested and improved since (e.g. I fixed the musl support upstream, the cross-compilation of stdbuf upstream, some buildpath issue)
- its adoption in Ubuntu 26.04 as default coreutils implementation and
the resulting large-scale usage and bug-fixing
- its availability in Windows as "Coreutils for Windows" starting in
June 2026 as a package maintained by Microsoft, and the resulting large-scale usage and bug-fixing
- its massive adoption in various distros and package-managers (see
https://uutils.org/users/ )
- its >95% pass-rate of the GNU testing suite in release 0.0.12 (some
tests are still skipped)
- anecdotally, my own usage in production on embedded devices for
several years without issues (on arm and qemu-arm64)- "should we make uutils-coreutils the default choice, or maybe the only choice?"
-> maybe in several years when uutils-coreutils has matured even more
and has 100% test coverage of the GNU test suite? At the moment I think the safest choice on critical devices is still GNU coreutils, so I wouldn't remove or make rust coreutils the default or only choice. Also rust binaries are bigger than C binaries so there is some tradeoff involved. GNU coreutils 9.12 main package is 10.3MB big in my test build, compared to 13.6MB for the uutils-coreutils 0.0.12 main package (uutils-coreutils uses a multi-binary executable to save space, GNU coreutils doesn't).- "Does it work as a provider of coreutils-native and is it compatible with pseudo?"
- the recipe uutils-coreutils-native compiles, but it's not compatible
with pseudo because of differences in syscalls, and it's not suitable as a replacement for coreutils-native at the moment. I'll try to make it work with pseudo in the future, but I think almost all users consume it as a target recipe and that's where it provides most value.- "Does it pass the full autobuilder matrix when selected as coreutils provider? Should we add that test to the autobuilder?"
-> it should, but I've not tested the full matrix of the autobuilder. I
would like to add the recipe to poky-altcfg.conf, and as far as I understand this would add it to the autobuilder as side-effect. I am also working on adding ptest support for the recipe and can send this if necessary for the acceptance to oe-core.Étienne
Quoted 24 lines · Alexander Kanavin
On Sat, 10 Oct 2026 at 13:30, Etienne Cordonnier via lists.openembedded.org
<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openembedded.org&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=Pf9ypvX_MT4dU4cYC3brv9jpglRcRrrz1luHXObxYfW6J0oO8XAW6KnUoiQHzUES&s=QSmPEv1BkubeZVNksJqkHkppRkyy-Bac7QdkZuoYqaA&e=>Quoted 6 lines · <[email protected]> wrote:
uutils-coreutils, the Rust reimplementation of GNU coreutils, is mature enough for production use, so move the recipe from meta-oe to oe-core. It provides and rprovides coreutils, and is excluded from world unless selected as the coreutils provider.
This needs a bit more justification. What is achieved by moving it to core? What specifically makes this component mature enough? What tests were performed with it? Does it pass the full autobuilder matrix when selected as coreutiuls provider? Should we add that test to the autobuilder? Should we make uutils-coreutils the default choice, or maybe the only choice? Does it work as a provider of coreutils-native, and is it compatible with pseudo?
Adding things to core has a pretty high bar, and the above commit message does not meet it.
Alex
Quoted 4 lines · Alexander Kanavin via lists.openembedded.org
Alright. Here's a suggested plan for you:
- add ptest support to the recipe. It must have feature parity with the GNU recipe.
FYI, on adding ptest, we tried last year but got stuck and dropped it.
See:
* tests.rs: statically defined CARGO_BIN_EXE_coreutils causes all integration tests to fail on Yocto ptest-cargo · Issue #9246
https://github.com/uutils/coreutils/issues/9246
* tests.rs: Read executable binary location from environment at runtime by andstepan · Pull Request #9323
https://github.com/uutils/coreutils/pull/9323
Andrea does not work with us anymore (sadly), feel free to take over his work.
Regards,
Signature · 2 lines
Yoann Congal Smile ECS
Quoted 1 line
- add ptest support to the recipe. It must have feature parity with the
GNU recipe.
OK, I'll do that in meta-openembedded, since I don't want to mix this change with a move between two repos
Quoted 1 line
- investigate and propose a plan for pseudo compatibility. It's not easy:
pseudo relies on substituting libc API, and rust binaries issue kernel syscalls directly.
It is doable, since uutils-coreutils already uses the libc crate for many things, and the rustix crate can be configured at build-time to use the libc crate instead of using kernel syscalls directly.
On the other hand, PREFERRED_PROVIDER_coreutils is independent of PREFERRED_PROVIDER_coreutils-native, and very few recipes depend on coreutils-native, so I suggest decoupling a potential merge of uutils-coreutils with pseudo support. The target version delivers almost all the value, and pseudo compatibility can follow separately. Or am I missing something?
Quoted 1 line
- propose a test plan for Yocto. Merely sticking it into poky-altcfg is
actually a regression in testing, because then sysvinit images will be tested only with GNU, and systemd images will be tested only with uutils.
OK, then I'm not sure what's best in terms of testing matrix. I'll need to investigate.
Quoted 1 line
- last, but not least. Actually this is the most important one: fix your
relationship with GNU coreutils. It's painful to watch you two bicker, and I do not want the Yocto project to be dragged into this.
On the last point: I can see Collin feels strongly about this, and I take the licensing concern seriously. But I haven't been in conflict with him or with GNU coreutils. If there's a specific exchange you'd like me to look at, please point me to it.
Étienne
Quoted 115 lines · Alexander Kanavin
Alright. Here's a suggested plan for you:
- add ptest support to the recipe. It must have feature parity with the GNU recipe.
- investigate and propose a plan for pseudo compatibility. It's not easy:
pseudo relies on substituting libc API, and rust binaries issue kernel syscalls directly.- address concerns of 'license-washing'. A few of us are deeply uncomfortable with this recent wave of component rewrites (especially AI assisted which causes dubious copyrights) where the primary goal is not technical improvement but simply getting rid of the GPL. The purpose of open source is not serving commercial interests, it's making computing better for everyone.
- get the core community distributions (which is, Fedora and Debian) to have a plan and timeline of adoping uutils as the default. This is the ultimate test of maturity.
- propose a test plan for Yocto. Merely sticking it into poky-altcfg is actually a regression in testing, because then sysvinit images will be tested only with GNU, and systemd images will be tested only with uutils.
- last, but not least. Actually this is the most important one: fix your relationship with GNU coreutils. It's painful to watch you two bicker, and I do not want the Yocto project to be dragged into this.
Alex
Quoted 81 lines · Etienne Cordonnier
Hi Alex,
- what is achieved by moving it to core is improving no-gplv3.inc for users who are not using gplv3 and still want full coreutils syntax support. It will also make it possible to add it to poky-altcfg.conf, and generally make it easier to consume without needing meta-openembedded.- what specifically makes this component mature enough is:
- I've added the recipe to meta-openembedded in July 2022 and it has
been continuously tested and improved since (e.g. I fixed the musl support upstream, the cross-compilation of stdbuf upstream, some buildpath issue)
- its adoption in Ubuntu 26.04 as default coreutils implementation and
the resulting large-scale usage and bug-fixing
- its availability in Windows as "Coreutils for Windows" starting in
June 2026 as a package maintained by Microsoft, and the resulting large-scale usage and bug-fixing
- its massive adoption in various distros and package-managers (see
https://uutils.org/users/
<https://urldefense.proofpoint.com/v2/url?u=https-3A__uutils.org_users_&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=TQmofvWjML0rStkG9aORknAn231FNShwwQFbVxuoEpFoFov2hNAHI82gZaXe_Ji-&s=CBQ3d8b8uYSdxwNhKRN4m1Bv7jOIX0EwE2ZOXtUfDj0&e=>
)
- its >95% pass-rate of the GNU testing suite in release 0.0.12 (some
tests are still skipped)
- anecdotally, my own usage in production on embedded devices for
several years without issues (on arm and qemu-arm64)- "should we make uutils-coreutils the default choice, or maybe the only choice?"
-> maybe in several years when uutils-coreutils has matured even more
and has 100% test coverage of the GNU test suite? At the moment I think the safest choice on critical devices is still GNU coreutils, so I wouldn't remove or make rust coreutils the default or only choice. Also rust binaries are bigger than C binaries so there is some tradeoff involved. GNU coreutils 9.12 main package is 10.3MB big in my test build, compared to 13.6MB for the uutils-coreutils 0.0.12 main package (uutils-coreutils uses a multi-binary executable to save space, GNU coreutils doesn't).- "Does it work as a provider of coreutils-native and is it compatible with pseudo?"
- the recipe uutils-coreutils-native compiles, but it's not compatible
with pseudo because of differences in syscalls, and it's not suitable as a replacement for coreutils-native at the moment. I'll try to make it work with pseudo in the future, but I think almost all users consume it as a target recipe and that's where it provides most value.- "Does it pass the full autobuilder matrix when selected as coreutils provider? Should we add that test to the autobuilder?"
-> it should, but I've not tested the full matrix of the autobuilder. I
would like to add the recipe to poky-altcfg.conf, and as far as I understand this would add it to the autobuilder as side-effect. I am also working on adding ptest support for the recipe and can send this if necessary for the acceptance to oe-core.Étienne
Quoted 24 lines · Alexander Kanavin
On Sat, 10 Oct 2026 at 13:30, Etienne Cordonnier via lists.openembedded.org
<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openembedded.org&d=DwMFaQ&c=ncDTmphkJTvjIDPh0hpF_4vCHvabgGkICC2epckfdiw&r=AhkbNonVuMIGRfPx_Qj9TsRih1DULJTKUkSGa66m67E&m=Pf9ypvX_MT4dU4cYC3brv9jpglRcRrrz1luHXObxYfW6J0oO8XAW6KnUoiQHzUES&s=QSmPEv1BkubeZVNksJqkHkppRkyy-Bac7QdkZuoYqaA&e=>Quoted 6 lines · <[email protected]> wrote:
uutils-coreutils, the Rust reimplementation of GNU coreutils, is mature enough for production use, so move the recipe from meta-oe to oe-core. It provides and rprovides coreutils, and is excluded from world unless selected as the coreutils provider.
This needs a bit more justification. What is achieved by moving it to core? What specifically makes this component mature enough? What tests were performed with it? Does it pass the full autobuilder matrix when selected as coreutiuls provider? Should we add that test to the autobuilder? Should we make uutils-coreutils the default choice, or maybe the only choice? Does it work as a provider of coreutils-native, and is it compatible with pseudo?
Adding things to core has a pretty high bar, and the above commit message does not meet it.
Alex
Quoted 4 lines · Etienne Cordonnier
Quoted 1 line
- investigate and propose a plan for pseudo compatibility. It's not
easy: pseudo relies on substituting libc API, and rust binaries issue kernel syscalls directly.
Quoted 10 lines
It is doable, since uutils-coreutils already uses the libc crate for many things, and the rustix crate can be configured at build-time to use the libc crate instead of using kernel syscalls directly.
On the other hand,PREFERRED_PROVIDER_coreutilsis independent of PREFERRED_PROVIDER_coreutils-native, and very few recipes depend on coreutils-native, so I suggest decoupling a potential merge of uutils-coreutils with pseudo support. The target version delivers almost all the value, and pseudo compatibility can follow separately. Or am I missing something?
It's not just about coreutils-native, it's about what implementation of coreutils is provided by host distributions. For now, the project has forced GNU coreutils on Ubuntu, but going forward this may not be sustainable:
https://git.openembedded.org/openembedded-core/commit/?id=16f2684ebeffa72b5d90525cf9102751b68c298e
and we'll either have to convince distros to build with libc crates, or do such a build ourselves via -native recipe.
(note also all the fails that were observed when trying to use uutils from the ubuntu host, which means at least a year ago it was nowhere near mature)
Alex
Quoted 9 lines · Etienne Cordonnier
Quoted 1 line
- last, but not least. Actually this is the most important one: fix your
relationship with GNU coreutils. It's painful to watch you two bicker, and I do not want the Yocto project to be dragged into this.
On the last point: I can see Collin feels strongly about this, and I take the licensing concern seriously. But I haven't been in conflict with him or with GNU coreutils. If there's a specific exchange you'd like me to look at, please point me to it.
The risk of AI-assisted contributions that basically rewrite GPLd gnu coreutils C code into rust is very real, and it has happened in other projects, causing their complete derailment. Are you able to spot and reject such contributions?
There's also a distinct lack of collaboration. Both projects obviously must implement the exact same UI, and the only way to do it is to have a shared, independent, rigorous test suite. Collin has explicitly said he is not endorsing the use of GNU tests for uutils.
I'd also note that quietly skipping my points without saying anything about them only means they remain unresolved.
Alex
Quoted 1 line · Alexander Kanavin
Collin has explicitly said he is not endorsing the use of GNU tests for uutils.
Any implementation is free to use the GNU coreutils test suite, as long as its licensing is respected, of course. It mostly just involves fetching it from git, changing the PATH used by the tests to point to a separate directory with the relevant binaries, and maybe applying some patches for different error messages, for example. I don't think you meant to imply otherwise, but I just wanted to make sure it wasn't misinterpreted by other readers.
My comment was just to say that a high passing rate on the GNU coreutils test suite doesn't necessarily mean two implementations are compatible. A made-up example, based on some behaviors I have seen previously with other programs, though I forget which, a test might check if 'shred' checks for invalid file types by redirecting standard error to a file here:
$ mkfifo fifo
$ shred fifo
shred: fifo: invalid file typeIf one implementation uses open ("fifo", O_NONBLOCK | ...), and the other uses statx, which I commonly see uutils do through std::fs functions, then the behavior will be different on readers waiting for a writer. The open will unblock the reader, while statx will not. Behaviors like this aren't always tested, though; the GNU coreutils test suite is always a work in progress. I have tried to improve the checks for cases like this, but there are theoretically more niche cases that could cause problems if behavior changes.
Collin