Skip to contentSkip to conversation

musl-locales: Add Canadian French (fr_CA) locale support

PATCH In review Khem Raj 3 participants 10 messages Apr 13, 2023 – Apr 14, 2023

lore
Share & tools
lore

Khem Raj

author
#0 lore raw

PATCH 1/2 musl-locales: Add Canadian French (fr_CA) locale support

---
 meta/recipes-core/musl/musl-locales_git.bb | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
meta/recipes-core/musl/musl-locales_git.bb +4 -1
index 92532237e0..6ffac4fade 100644
@@ -10,7 +10,7 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=cf5713fba707073020b1db2acaa73e78 \
 SRC_URI = "git://git.adelielinux.org/adelie/musl-locales;protocol=https;branch=main"
 
 PV = "1.0+git${SRCPV}"
-SRCREV = "cac3dcdccd1d0c08d17f4919d862737d64b76f79"
+SRCREV = "5663f5bfd30bf9e1e0ba3fc5fe2da6725969f30e"
 
 S = "${WORKDIR}/git"
 
@@ -35,6 +35,7 @@ LICENSE:locale-base-en-gb = "MIT"
 LICENSE:locale-base-en-us = "MIT"
 LICENSE:locale-base-es-es = "MIT"
 LICENSE:locale-base-fi-fi = "MIT"
+LICENSE:locale-base-fr-ca = "MIT"
 LICENSE:locale-base-fr-fr = "MIT"
 LICENSE:locale-base-it-it = "MIT"
 LICENSE:locale-base-nb-no = "MIT"
@@ -52,6 +53,7 @@ PACKAGES =+ "locale-base-cs-cz \
              locale-base-en-us \
              locale-base-es-es \
              locale-base-fi-fi \
+             locale-base-fr-ca \
              locale-base-fr-fr \
              locale-base-it-it \
              locale-base-nb-no \
@@ -69,6 +71,7 @@ FILES:locale-base-en-gb += "${datadir}/i18n/locales/musl/en_GB.UTF-8"
 FILES:locale-base-en-us += "${datadir}/i18n/locales/musl/en_US.UTF-8"
 FILES:locale-base-es-es += "${datadir}/i18n/locales/musl/es_ES.UTF-8"
 FILES:locale-base-fi-fi += "${datadir}/i18n/locales/musl/fi_FI.UTF-8"
+FILES:locale-base-fr-ca += "${datadir}/i18n/locales/musl/fr_CA.UTF-8"
 FILES:locale-base-fr-fr += "${datadir}/i18n/locales/musl/fr_FR.UTF-8"
 FILES:locale-base-it-it += "${datadir}/i18n/locales/musl/it_IT.UTF-8"
 FILES:locale-base-nb-no += "${datadir}/i18n/locales/musl/nb_NO.UTF-8"
Signature · 1 line
2.40.0
author
#1 lore raw

PATCH 2/2 gawk: Fix ptests on musl

---
 ...0001-Provide-fixes-for-tests-on-musl.patch | 73 +++++++++++++++++++
 meta/recipes-extended/gawk/gawk_5.2.1.bb      |  7 +-
 2 files changed, 77 insertions(+), 3 deletions(-)
 create mode 100644 meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch
meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch new file +73 -0
new file mode 100644
index 0000000000..4ff2466c0e
@@ -0,0 +1,73 @@
+From 95bcf4397b6f727208f3c627b7d00dacec406d02 Mon Sep 17 00:00:00 2001
+From: Khem Raj <[email protected]>
+Date: Wed, 12 Apr 2023 21:57:18 -0700
+Subject: [PATCH] Provide fixes for tests on musl
+
+Fixes rebt8b1 and regx8bit
+Disable clos1way6 [1]
+
+Disable commas and backsmalls1, they do not work
+with musl-locales somehow even though we do isntall
+en_US.UTF-8 on target
+
+[1] https://lists.gnu.org/archive/html/bug-gawk/2021-02/msg00005.html
+
+Upstream-Status: Inappropriate [musl-specific]
+Signed-off-by: Khem Raj <[email protected]>
+---
+ test/Maketests | 11 ++++-------
+ 1 file changed, 4 insertions(+), 7 deletions(-)
+
+--- a/test/Maketests
++++ b/test/Maketests
+@@ -916,7 +916,8 @@ rebrackloc:
+ 
+ rebt8b1:
+ 	@echo $@
+-	@-AWKPATH="$(srcdir)" $(AWK) -f [email protected]  >_$@ 2>&1 || echo EXIT CODE: $$? >>_$@
++	@-[ -z "$$LANG" ] && LANG=C; export LANG; \
++	AWKPATH="$(srcdir)" $(AWK) -f [email protected]  >_$@ 2>&1 || echo EXIT CODE: $$? >>_$@
+ 	@-$(CMP) "$(srcdir)"/[email protected] _$@ && rm -f _$@
+ 
+ rebuild:
+@@ -1391,17 +1392,6 @@ clos1way5:
+ 	@-AWKPATH="$(srcdir)" $(AWK) -f [email protected]  >_$@ 2>&1 || echo EXIT CODE: $$? >>_$@
+ 	@-$(CMP) "$(srcdir)"/[email protected] _$@ && rm -f _$@
+ 
+-clos1way6:
+-	@echo $@ $(ZOS_FAIL)
+-	@-AWKPATH="$(srcdir)" $(AWK) -f [email protected]  >_$@ 2>&1 || echo EXIT CODE: $$? >>_$@
+-	@-$(CMP) "$(srcdir)"/[email protected] _$@ && rm -f _$@
+-
+-commas:
+-	@echo $@ $(ZOS_FAIL)
+-	@-[ -z "$$GAWKLOCALE" ] && GAWKLOCALE=en_US.UTF-8; export GAWKLOCALE; \
+-	AWKPATH="$(srcdir)" $(AWK) -f [email protected]  >_$@ 2>&1 || echo EXIT CODE: $$? >>_$@
+-	@-$(CMP) "$(srcdir)"/[email protected] _$@ && rm -f _$@
+-
+ crlf:
+ 	@echo $@
+ 	@-AWKPATH="$(srcdir)" $(AWK) -f [email protected]  >_$@ 2>&1 || echo EXIT CODE: $$? >>_$@
+@@ -2003,7 +1993,8 @@ regnul2:
+ 
+ regx8bit:
+ 	@echo $@
+-	@-AWKPATH="$(srcdir)" $(AWK) -f [email protected]  >_$@ 2>&1 || echo EXIT CODE: $$? >>_$@
++	@-[ -z "$$LANG" ] && LANG=C; export LANG; \
++	AWKPATH="$(srcdir)" $(AWK) -f [email protected]  >_$@ 2>&1 || echo EXIT CODE: $$? >>_$@
+ 	@-$(CMP) "$(srcdir)"/[email protected] _$@ && rm -f _$@
+ 
+ reint:
+@@ -2256,12 +2247,6 @@ backbigs1:
+ 	@echo $@ $(ZOS_FAIL)
+ 	@-[ -z "$$GAWKLOCALE" ] && GAWKLOCALE=en_US.UTF-8; export GAWKLOCALE; \
+ 	AWKPATH="$(srcdir)" $(AWK) -f [email protected]  < "$(srcdir)"/[email protected] >_$@ 2>&1 || echo EXIT CODE: $$? >>_$@
+-	@-$(CMP) "$(srcdir)"/[email protected] _$@ && rm -f _$@
+-
+-backsmalls1:
+-	@echo $@ $(ZOS_FAIL)
+-	@-[ -z "$$GAWKLOCALE" ] && GAWKLOCALE=en_US.UTF-8; export GAWKLOCALE; \
+-	AWKPATH="$(srcdir)" $(AWK) -f [email protected]  < "$(srcdir)"/[email protected] >_$@ 2>&1 || echo EXIT CODE: $$? >>_$@
+ 	@-$(CMP) "$(srcdir)"/[email protected] _$@ && rm -f _$@
+ 
+ backsmalls2:
meta/recipes-extended/gawk/gawk_5.2.1.bb +4 -3
index e381bad148..b2eb6987ed 100644
@@ -19,7 +19,7 @@ SRC_URI = "${GNU_MIRROR}/gawk/gawk-${PV}.tar.gz \
            file://remove-sensitive-tests.patch \
            file://run-ptest \
            "
-
+SRC_URI:append:libc-musl = " file://0001-Provide-fixes-for-tests-on-musl.patch "
 SRC_URI[sha256sum] = "529e7c8c6acf21ff3a6183f4d763c632810908989c24675c77995d51ac37b79c"
 
 inherit autotools gettext texinfo update-alternatives
@@ -62,8 +62,9 @@ do_install_ptest() {
 	rm -f ${D}${PTEST_PATH}/test/timeout.*
 }
 
-RDEPENDS:${PN}-ptest += "make"
+RDEPENDS:${PN}-ptest += "make locale-base-en-us"
 
-RDEPENDS:${PN}-ptest:append:libc-glibc = " locale-base-en-us locale-base-en-us.iso-8859-1"
+RDEPENDS:${PN}-ptest:append:libc-glibc = " locale-base-en-us.iso-8859-1"
+RDEPENDS:${PN}-ptest:append:libc-musl = " musl-locales"
 
 BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.40.0
#2 lore raw

Re: [OE-core] [PATCH 2/2] gawk: Fix ptests on musl

Quoted 22 lines · Khem Raj
---
 ...0001-Provide-fixes-for-tests-on-musl.patch | 73 +++++++++++++++++++
 meta/recipes-extended/gawk/gawk_5.2.1.bb      |  7 +-
 2 files changed, 77 insertions(+), 3 deletions(-)
 create mode 100644 meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch
meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch new file +9 -0
new file mode 100644
index 0000000000..4ff2466c0e
@@ -0,0 +1,73 @@
+From 95bcf4397b6f727208f3c627b7d00dacec406d02 Mon Sep 17 00:00:00 2001
+From: Khem Raj <[email protected]>
+Date: Wed, 12 Apr 2023 21:57:18 -0700
+Subject: [PATCH] Provide fixes for tests on musl
+
+Fixes rebt8b1 and regx8bit
+Disable clos1way6 [1]
+
+Disable commas and backsmalls1, they do not work

meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch line 19

Quoted 10 lines
meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch new file +10 -0
+with musl-locales somehow even though we do isntall
+en_US.UTF-8 on target
+
+[1] https://lists.gnu.org/archive/html/bug-gawk/2021-02/msg00005.html
+
+Upstream-Status: Inappropriate [musl-specific]
+Signed-off-by: Khem Raj <[email protected]>
+---
+ test/Maketests | 11 ++++-------
+

Carrying a patch like this makes me a bit nervous as:

a) it is only applied for musl so it makes recipe upgrade testing a nightmare

b) there doesn't appear to be any path to upstream it, we'd just have to carry it indefinitely.

c) the reasons for the failure don't seem to be understood.

Is there something we can do to avoid having to carry this indefinitely?

Out of interest did alpine do anything for this?

Cheers,

Richard

author
#3 lore raw

Re: [OE-core] [PATCH 2/2] gawk: Fix ptests on musl

Quoted 37 lines · Richard Purdie
Quoted 22 lines · Khem Raj
---
 ...0001-Provide-fixes-for-tests-on-musl.patch | 73 +++++++++++++++++++
 meta/recipes-extended/gawk/gawk_5.2.1.bb      |  7 +-
 2 files changed, 77 insertions(+), 3 deletions(-)
 create mode 100644 meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch
meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch new file +9 -0
new file mode 100644
index 0000000000..4ff2466c0e
@@ -0,0 +1,73 @@
+From 95bcf4397b6f727208f3c627b7d00dacec406d02 Mon Sep 17 00:00:00 2001
+From: Khem Raj <[email protected]>
+Date: Wed, 12 Apr 2023 21:57:18 -0700
+Subject: [PATCH] Provide fixes for tests on musl
+
+Fixes rebt8b1 and regx8bit
+Disable clos1way6 [1]
+
+Disable commas and backsmalls1, they do not work

meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch line 19

Quoted 10 lines
meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch new file +10 -0
+with musl-locales somehow even though we do isntall
+en_US.UTF-8 on target
+
+[1] https://lists.gnu.org/archive/html/bug-gawk/2021-02/msg00005.html
+
+Upstream-Status: Inappropriate [musl-specific]
+Signed-off-by: Khem Raj <[email protected]>
+---
+ test/Maketests | 11 ++++-------
+

Carrying a patch like this makes me a bit nervous as:

Quoted 7 lines

a) it is only applied for musl so it makes recipe upgrade testing a nightmare

b) there doesn't appear to be any path to upstream it, we'd just have to carry it indefinitely.

c) the reasons for the failure don't seem to be understood.

reasons are clear, these tests need full locale support which we do not have with musl.

Quoted 5 lines

Is there something we can do to avoid having to carry this indefinitely?

Out of interest did alpine do anything for this?

they have patches.

Quoted 5 lines

Cheers,

Richard

#4 lore raw

Re: [OE-core] [PATCH 2/2] gawk: Fix ptests on musl

I tend to agree with RP, this needs to be discussed with upstream first. We should avoid patches which are both conditional and non-upstreamable. If gawk upstream doesn't want their tests to be compatible with musl, then maybe neither should we.

Alex

Quoted 73 lines · Khem Raj
Quoted 37 lines · Richard Purdie
Quoted 22 lines · Khem Raj
---
 ...0001-Provide-fixes-for-tests-on-musl.patch | 73 +++++++++++++++++++
 meta/recipes-extended/gawk/gawk_5.2.1.bb      |  7 +-
 2 files changed, 77 insertions(+), 3 deletions(-)
 create mode 100644 meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch
meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch new file +9 -0
new file mode 100644
index 0000000000..4ff2466c0e
@@ -0,0 +1,73 @@
+From 95bcf4397b6f727208f3c627b7d00dacec406d02 Mon Sep 17 00:00:00 2001
+From: Khem Raj <[email protected]>
+Date: Wed, 12 Apr 2023 21:57:18 -0700
+Subject: [PATCH] Provide fixes for tests on musl
+
+Fixes rebt8b1 and regx8bit
+Disable clos1way6 [1]
+
+Disable commas and backsmalls1, they do not work

meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch line 19

Quoted 10 lines
meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch new file +10 -0
+with musl-locales somehow even though we do isntall
+en_US.UTF-8 on target
+
+[1] https://lists.gnu.org/archive/html/bug-gawk/2021-02/msg00005.html
+
+Upstream-Status: Inappropriate [musl-specific]
+Signed-off-by: Khem Raj <[email protected]>
+---
+ test/Maketests | 11 ++++-------
+

Carrying a patch like this makes me a bit nervous as:

Quoted 7 lines

a) it is only applied for musl so it makes recipe upgrade testing a nightmare

b) there doesn't appear to be any path to upstream it, we'd just have to carry it indefinitely.

c) the reasons for the failure don't seem to be understood.

reasons are clear, these tests need full locale support which we do not have with musl.

Quoted 5 lines

Is there something we can do to avoid having to carry this indefinitely?

Out of interest did alpine do anything for this?

they have patches.

Quoted 5 lines

Cheers,

Richard

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#179976): https://lists.openembedded.org/g/openembedded-core/message/179976
Mute This Topic: https://lists.openembedded.org/mt/98235523/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#5 lore raw

Re: [OE-core] [PATCH 2/2] gawk: Fix ptests on musl

In reply to Khem Raj
Quoted 42 lines · Khem Raj
Quoted 40 lines · Richard Purdie
Quoted 22 lines · Khem Raj
---
 ...0001-Provide-fixes-for-tests-on-musl.patch | 73 +++++++++++++++++++
 meta/recipes-extended/gawk/gawk_5.2.1.bb      |  7 +-
 2 files changed, 77 insertions(+), 3 deletions(-)
 create mode 100644 meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch
meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch new file +9 -0
new file mode 100644
index 0000000000..4ff2466c0e
@@ -0,0 +1,73 @@
+From 95bcf4397b6f727208f3c627b7d00dacec406d02 Mon Sep 17 00:00:00 2001
+From: Khem Raj <[email protected]>
+Date: Wed, 12 Apr 2023 21:57:18 -0700
+Subject: [PATCH] Provide fixes for tests on musl
+
+Fixes rebt8b1 and regx8bit
+Disable clos1way6 [1]
+
+Disable commas and backsmalls1, they do not work

meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch line 19

Quoted 10 lines
meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch new file +10 -0
+with musl-locales somehow even though we do isntall
+en_US.UTF-8 on target
+
+[1] https://lists.gnu.org/archive/html/bug-gawk/2021-02/msg00005.html
+
+Upstream-Status: Inappropriate [musl-specific]
+Signed-off-by: Khem Raj <[email protected]>
+---
+ test/Maketests | 11 ++++-------
+

Carrying a patch like this makes me a bit nervous as:

a) it is only applied for musl so it makes recipe upgrade testing a nightmare

Quoted 7 lines
Quoted 4 lines

b) there doesn't appear to be any path to upstream it, we'd just have to carry it indefinitely.

c) the reasons for the failure don't seem to be understood.

reasons are clear, these tests need full locale support which we do not have with musl.

The above says something quite different to what the patch says:

"they do not work with musl-locales somehow"

which sounds like nobody has looked into it.

Quoted 6 lines
Quoted 4 lines

Is there something we can do to avoid having to carry this indefinitely?

Out of interest did alpine do anything for this?

they have patches.

The same as ours or different? Was there any discussion with upstream on this?

Cheers,

Richard

author
#6 lore raw

Re: [OE-core] [PATCH 2/2] gawk: Fix ptests on musl

Quoted 47 lines · Richard Purdie
Quoted 45 lines · Khem Raj
Quoted 43 lines · Richard Purdie
Quoted 22 lines · Khem Raj
---
 ...0001-Provide-fixes-for-tests-on-musl.patch | 73 +++++++++++++++++++
 meta/recipes-extended/gawk/gawk_5.2.1.bb      |  7 +-
 2 files changed, 77 insertions(+), 3 deletions(-)
 create mode 100644 meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch
meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch new file +9 -0
new file mode 100644
index 0000000000..4ff2466c0e
@@ -0,0 +1,73 @@
+From 95bcf4397b6f727208f3c627b7d00dacec406d02 Mon Sep 17 00:00:00 2001
+From: Khem Raj <[email protected]>
+Date: Wed, 12 Apr 2023 21:57:18 -0700
+Subject: [PATCH] Provide fixes for tests on musl
+
+Fixes rebt8b1 and regx8bit
+Disable clos1way6 [1]
+
+Disable commas and backsmalls1, they do not work

meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch line 19

Quoted 10 lines
meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch new file +10 -0
+with musl-locales somehow even though we do isntall
+en_US.UTF-8 on target
+
+[1] https://lists.gnu.org/archive/html/bug-gawk/2021-02/msg00005.html
+
+Upstream-Status: Inappropriate [musl-specific]
+Signed-off-by: Khem Raj <[email protected]>
+---
+ test/Maketests | 11 ++++-------
+

Carrying a patch like this makes me a bit nervous as:

a) it is only applied for musl so it makes recipe upgrade testing a nightmare

b) there doesn't appear to be any path to upstream it, we'd just have to carry it indefinitely.

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

c) the reasons for the failure don't seem to be understood.

reasons are clear, these tests need full locale support which we do not have with musl.

The above says something quite different to what the patch says:

"they do not work with musl-locales somehow"

yeah the metadata patch subject should be improved to say disable tests

Quoted 2 lines

which sounds like nobody has looked into it.

yes, and my localedata knowledge is limited to root cause this further sadly.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Is there something we can do to avoid having to carry this indefinitely?

You can do sed operations from the recipe. I don't know how that will be better than patch. btw. we are already carrying another patch to disable other tests because I guess they fail on Autobuilder on stress but not always, because they worked
fine on my ptest run.

https://git.openembedded.org/openembedded-core/tree/meta/recipes-extended/gawk/gawk/remove-sensitive-tests.patch

Do we have plans to address this with upstream as well ?

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

Out of interest did alpine do anything for this?

they have patches.

The same as ours or different? Was there any discussion with upstream on this?

There is one failure which is common and I have quoted upstream discussion others are not. I dont have alpine install handy to try it out quickly to confirm.

Quoted 5 lines

Cheers,

Richard

#7 lore raw

Re: [OE-core] [PATCH 2/2] gawk: Fix ptests on musl

Quoted 51 lines · Khem Raj
Quoted 49 lines · Richard Purdie
Quoted 47 lines · Khem Raj
Quoted 44 lines · Richard Purdie
Quoted 22 lines · Khem Raj
---
 ...0001-Provide-fixes-for-tests-on-musl.patch | 73 +++++++++++++++++++
 meta/recipes-extended/gawk/gawk_5.2.1.bb      |  7 +-
 2 files changed, 77 insertions(+), 3 deletions(-)
 create mode 100644 meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch
meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch new file +9 -0
new file mode 100644
index 0000000000..4ff2466c0e
@@ -0,0 +1,73 @@
+From 95bcf4397b6f727208f3c627b7d00dacec406d02 Mon Sep 17 00:00:00 2001
+From: Khem Raj <[email protected]>
+Date: Wed, 12 Apr 2023 21:57:18 -0700
+Subject: [PATCH] Provide fixes for tests on musl
+
+Fixes rebt8b1 and regx8bit
+Disable clos1way6 [1]
+
+Disable commas and backsmalls1, they do not work

meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch line 19

Quoted 10 lines
meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch new file +10 -0
+with musl-locales somehow even though we do isntall
+en_US.UTF-8 on target
+
+[1] https://lists.gnu.org/archive/html/bug-gawk/2021-02/msg00005.html
+
+Upstream-Status: Inappropriate [musl-specific]
+Signed-off-by: Khem Raj <[email protected]>
+---
+ test/Maketests | 11 ++++-------
+

Carrying a patch like this makes me a bit nervous as:

a) it is only applied for musl so it makes recipe upgrade testing a nightmare

b) there doesn't appear to be any path to upstream it, we'd just have to carry it indefinitely.

c) the reasons for the failure don't seem to be understood.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

reasons are clear, these tests need full locale support which we do not have with musl.

The above says something quite different to what the patch says:

"they do not work with musl-locales somehow"

yeah the metadata patch subject should be improved to say disable tests

That would be clearer, yes.

Quoted 4 lines
Quoted 2 lines

which sounds like nobody has looked into it.

yes, and my localedata knowledge is limited to root cause this further sadly.

Fair enough, we just need to be clear about that.

Quoted 9 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Is there something we can do to avoid having to carry this indefinitely?

You can do sed operations from the recipe. I don't know how that will be better than patch. btw. we are already carrying another patch to disable other tests because I guess they fail on Autobuilder on stress but not always, because they worked
fine on my ptest run.

Quoted 3 lines

https://git.openembedded.org/openembedded-core/tree/meta/recipes-extended/gawk/gawk/remove-sensitive-tests.patch

Do we have plans to address this with upstream as well ?

The difference between that patch and the one proposed is that the one above:

a) is unconditional
b) is clear and simple, it just deletes the tests

This means when the recipe is upgraded, the patch is obvious and therefore straight forward to port.

Ideas:

Can we simplify the musl one to delete the tests in question?

Could we get a patch upstream which allowed us to disable tests from the commandline to improve both cases?

I think we should be able to work something out but I just wanted to make it clear the patch as it stands isn't clear enough and will cause maintenance issues so we need to work out something better.

Cheers,

Richard

Quoted 17 lines
Quoted 6 lines
Quoted 3 lines
Quoted 1 line

Out of interest did alpine do anything for this?

they have patches.

The same as ours or different? Was there any discussion with upstream on this?

There is one failure which is common and I have quoted upstream discussion others are not. I dont have alpine install handy to try it out quickly to confirm.

Quoted 5 lines

Cheers,

Richard

author
#8 lore raw

Re: [OE-core] [PATCH 2/2] gawk: Fix ptests on musl

Quoted 95 lines · Richard Purdie
Quoted 51 lines · Khem Raj
Quoted 49 lines · Richard Purdie
Quoted 47 lines · Khem Raj
Quoted 44 lines · Richard Purdie
Quoted 22 lines · Khem Raj
---
 ...0001-Provide-fixes-for-tests-on-musl.patch | 73 +++++++++++++++++++
 meta/recipes-extended/gawk/gawk_5.2.1.bb      |  7 +-
 2 files changed, 77 insertions(+), 3 deletions(-)
 create mode 100644 meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch
meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch new file +9 -0
new file mode 100644
index 0000000000..4ff2466c0e
@@ -0,0 +1,73 @@
+From 95bcf4397b6f727208f3c627b7d00dacec406d02 Mon Sep 17 00:00:00 2001
+From: Khem Raj <[email protected]>
+Date: Wed, 12 Apr 2023 21:57:18 -0700
+Subject: [PATCH] Provide fixes for tests on musl
+
+Fixes rebt8b1 and regx8bit
+Disable clos1way6 [1]
+
+Disable commas and backsmalls1, they do not work

meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch line 19

Quoted 10 lines
meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch new file +10 -0
+with musl-locales somehow even though we do isntall
+en_US.UTF-8 on target
+
+[1] https://lists.gnu.org/archive/html/bug-gawk/2021-02/msg00005.html
+
+Upstream-Status: Inappropriate [musl-specific]
+Signed-off-by: Khem Raj <[email protected]>
+---
+ test/Maketests | 11 ++++-------
+

Carrying a patch like this makes me a bit nervous as:

a) it is only applied for musl so it makes recipe upgrade testing a nightmare

b) there doesn't appear to be any path to upstream it, we'd just have to carry it indefinitely.

c) the reasons for the failure don't seem to be understood.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

reasons are clear, these tests need full locale support which we do not have with musl.

The above says something quite different to what the patch says:

"they do not work with musl-locales somehow"

yeah the metadata patch subject should be improved to say disable tests

That would be clearer, yes.

Quoted 4 lines
Quoted 2 lines

which sounds like nobody has looked into it.

yes, and my localedata knowledge is limited to root cause this further sadly.

Fair enough, we just need to be clear about that.

Quoted 9 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Is there something we can do to avoid having to carry this indefinitely?

You can do sed operations from the recipe. I don't know how that will be better than patch. btw. we are already carrying another patch to disable other tests because I guess they fail on Autobuilder on stress but not always, because they worked
fine on my ptest run.

Quoted 3 lines

https://git.openembedded.org/openembedded-core/tree/meta/recipes-extended/gawk/gawk/remove-sensitive-tests.patch

Do we have plans to address this with upstream as well ?

The difference between that patch and the one proposed is that the one above:

a) is unconditional
b) is clear and simple, it just deletes the tests

This means when the recipe is upgraded, the patch is obvious and therefore straight forward to port.

Ideas:

Quoted 9 lines

Can we simplify the musl one to delete the tests in question?

Could we get a patch upstream which allowed us to disable tests from the commandline to improve both cases?

I think we should be able to work something out but I just wanted to make it clear the patch as it stands isn't clear enough and will cause maintenance issues so we need to work out something better.

if you run make check on musl/emux86-64 then we only get one failure clos1way6 and others pass. Which matches alpine. However, I think the way ptest is copying partially the test environment seems to be the cause. I tried to pass LANG=C etc but it did not help. Secondly, there are some surgeries the run-ptest is also doing
which makes the test environment quite unique to yocto ptests.

Its fine to ignore this patch. I have already drove into weeds trying to fix it, not worth it.

Quoted 23 lines

Cheers,

Richard

Quoted 17 lines
Quoted 6 lines
Quoted 3 lines
Quoted 1 line

Out of interest did alpine do anything for this?

they have patches.

The same as ours or different? Was there any discussion with upstream on this?

There is one failure which is common and I have quoted upstream discussion others are not. I dont have alpine install handy to try it out quickly to confirm.

Quoted 5 lines

Cheers,

Richard

author
#9 lore raw

Re: [OE-core] [PATCH 2/2] gawk: Fix ptests on musl

Quoted 100 lines · Khem Raj
Quoted 97 lines · Richard Purdie
Quoted 51 lines · Khem Raj
Quoted 49 lines · Richard Purdie
Quoted 47 lines · Khem Raj
Quoted 44 lines · Richard Purdie
Quoted 22 lines · Khem Raj
---
 ...0001-Provide-fixes-for-tests-on-musl.patch | 73 +++++++++++++++++++
 meta/recipes-extended/gawk/gawk_5.2.1.bb      |  7 +-
 2 files changed, 77 insertions(+), 3 deletions(-)
 create mode 100644 meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch
meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch new file +9 -0
new file mode 100644
index 0000000000..4ff2466c0e
@@ -0,0 +1,73 @@
+From 95bcf4397b6f727208f3c627b7d00dacec406d02 Mon Sep 17 00:00:00 2001
+From: Khem Raj <[email protected]>
+Date: Wed, 12 Apr 2023 21:57:18 -0700
+Subject: [PATCH] Provide fixes for tests on musl
+
+Fixes rebt8b1 and regx8bit
+Disable clos1way6 [1]
+
+Disable commas and backsmalls1, they do not work

meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch line 19

Quoted 10 lines
meta/recipes-extended/gawk/gawk/0001-Provide-fixes-for-tests-on-musl.patch new file +10 -0
+with musl-locales somehow even though we do isntall
+en_US.UTF-8 on target
+
+[1] https://lists.gnu.org/archive/html/bug-gawk/2021-02/msg00005.html
+
+Upstream-Status: Inappropriate [musl-specific]
+Signed-off-by: Khem Raj <[email protected]>
+---
+ test/Maketests | 11 ++++-------
+

Carrying a patch like this makes me a bit nervous as:

a) it is only applied for musl so it makes recipe upgrade testing a nightmare

b) there doesn't appear to be any path to upstream it, we'd just have to carry it indefinitely.

c) the reasons for the failure don't seem to be understood.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

reasons are clear, these tests need full locale support which we do not have with musl.

The above says something quite different to what the patch says:

"they do not work with musl-locales somehow"

yeah the metadata patch subject should be improved to say disable tests

That would be clearer, yes.

Quoted 4 lines
Quoted 2 lines

which sounds like nobody has looked into it.

yes, and my localedata knowledge is limited to root cause this further sadly.

Fair enough, we just need to be clear about that.

Quoted 9 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Is there something we can do to avoid having to carry this indefinitely?

You can do sed operations from the recipe. I don't know how that will be better than patch. btw. we are already carrying another patch to disable other tests because I guess they fail on Autobuilder on stress but not always, because they worked
fine on my ptest run.

Quoted 3 lines

https://git.openembedded.org/openembedded-core/tree/meta/recipes-extended/gawk/gawk/remove-sensitive-tests.patch

Do we have plans to address this with upstream as well ?

The difference between that patch and the one proposed is that the one above:

a) is unconditional
b) is clear and simple, it just deletes the tests

This means when the recipe is upgraded, the patch is obvious and therefore straight forward to port.

Ideas:

Can we simplify the musl one to delete the tests in question?

Quoted 7 lines
Quoted 7 lines

Could we get a patch upstream which allowed us to disable tests from the commandline to improve both cases?

I think we should be able to work something out but I just wanted to make it clear the patch as it stands isn't clear enough and will cause maintenance issues so we need to work out something better.

I just sent a v3 patchset for this issue which IMO solves the problem and creates less hassle to maintain as well. Let me know.

Quoted 34 lines

if you run make check on musl/emux86-64 then we only get one failure clos1way6 and others pass. Which matches alpine. However, I think the way ptest is copying partially the test environment seems to be the cause. I tried to pass LANG=C etc but it did not help. Secondly, there are some surgeries the run-ptest is also doing
which makes the test environment quite unique to yocto ptests.

Its fine to ignore this patch. I have already drove into weeds trying to fix it, not worth it.

Quoted 23 lines

Cheers,

Richard

Quoted 17 lines
Quoted 6 lines
Quoted 3 lines
Quoted 1 line

Out of interest did alpine do anything for this?

they have patches.

The same as ours or different? Was there any discussion with upstream on this?

There is one failure which is common and I have quoted upstream discussion others are not. I dont have alpine install handy to try it out quickly to confirm.

Quoted 5 lines

Cheers,

Richard