Skip to contentSkip to conversation

systemd: Fix musl fix patch

PATCH In review Khem Raj 2 participants 10 messages Mar 12, 2023 – Mar 13, 2023 1 message not in the archive

lore
Share & tools
lore

Khem Raj

author
#0 lore raw

PATCH 1/1 systemd: Fix musl fix patch

Current patch with fail when stdint.h is in include path before this typedef which can also happen with re-includes. Therefore shunt that by defining __DEFINED_wchar_t which helps musl builds of systemd-boot recipe

Cc Alexander Kanavin <[email protected]>
---
 ...rc-boot-efi-efi-string.c-define-wchar_t-from-__WCHA.patch | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
meta/recipes-core/systemd/systemd/0004-src-boot-efi-efi-string.c-define-wchar_t-from-__WCHA.patch +2 -3
index 927f914848..43173ac32d 100644
@@ -21,17 +21,16 @@ Signed-off-by: Alexander Kanavin <[email protected]>
  src/boot/efi/efi-string.c | 5 +++++
  1 file changed, 5 insertions(+)
 
-diff --git a/src/boot/efi/efi-string.c b/src/boot/efi/efi-string.c
-index 22923d60f6..22a8d1ef71 100644
 --- a/src/boot/efi/efi-string.c
 +++ b/src/boot/efi/efi-string.c
-@@ -2,7 +2,12 @@
+@@ -2,7 +2,13 @@
  
  #include <stdbool.h>
  #include <stdint.h>
 +
 +#if SD_BOOT
 +typedef __WCHAR_TYPE__ wchar_t;
++#define __DEFINED_wchar_t
 +#else
  #include <wchar.h>
 +#endif
Signature · 1 line
2.39.2
#1 lore raw

How can the issue be triggered? I did test this on musl, and so did the autobuilder.

Alex

Quoted 39 lines · Khem Raj

Current patch with fail when stdint.h is in include path before this typedef which can also happen with re-includes. Therefore shunt that by defining __DEFINED_wchar_t which helps musl builds of systemd-boot recipe

Cc Alexander Kanavin <[email protected]>
---
 ...rc-boot-efi-efi-string.c-define-wchar_t-from-__WCHA.patch | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
meta/recipes-core/systemd/systemd/0004-src-boot-efi-efi-string.c-define-wchar_t-from-__WCHA.patch +2 -4
index 927f914848..43173ac32d 100644
@@ -21,17 +21,16 @@ Signed-off-by: Alexander Kanavin <[email protected]>
  src/boot/efi/efi-string.c | 5 +++++
  1 file changed, 5 insertions(+)
 
-diff --git a/src/boot/efi/efi-string.c b/src/boot/efi/efi-string.c
-index 22923d60f6..22a8d1ef71 100644
 --- a/src/boot/efi/efi-string.c
 +++ b/src/boot/efi/efi-string.c
-@@ -2,7 +2,12 @@
+@@ -2,7 +2,13 @@
 
  #include <stdbool.h>
  #include <stdint.h>
 +
 +#if SD_BOOT
 +typedef __WCHAR_TYPE__ wchar_t;
++#define __DEFINED_wchar_t
 +#else
  #include <wchar.h>
 +#endif
--

2.39.2

author
#2 lore raw
Quoted 2 lines · Alexander Kanavin

How can the issue be triggered? I did test this on musl, and so did the autobuilder.

My setup is clang/musl on imx8 Which is similar to qemuarm64 And build works for systemd but Fails for systemd-boot recipe

Quoted 49 lines

Alex

Quoted 6 lines · Khem Raj

Current patch with fail when stdint.h is in include path before this typedef which can also happen with re-includes. Therefore shunt that by defining __DEFINED_wchar_t which helps musl builds of systemd-boot recipe

Quoted 7 lines
Cc Alexander Kanavin <[email protected]>
---
 ...rc-boot-efi-efi-string.c-define-wchar_t-from-__WCHA.patch | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git

a/meta/recipes-core/systemd/systemd/0004-src-boot-efi-efi-string.c-define-wchar_t-from-__WCHA.patch b/meta/recipes-core/systemd/systemd/0004-src-boot-efi-efi-string.c-define-wchar_t-from-__WCHA.patch

Quoted 2 lines

index 927f914848..43173ac32d 100644
---

a/meta/recipes-core/systemd/systemd/0004-src-boot-efi-efi-string.c-define-wchar_t-from-__WCHA.patch

Quoted 1 line

+++

b/meta/recipes-core/systemd/systemd/0004-src-boot-efi-efi-string.c-define-wchar_t-from-__WCHA.patch

Quoted 24 lines
meta/recipes-core/systemd/systemd/0004-src-boot-efi-efi-string.c-define-wchar_t-from-__WCHA.patch +2 -3
@@ -21,17 +21,16 @@ Signed-off-by: Alexander Kanavin <[email protected]
 
  src/boot/efi/efi-string.c | 5 +++++
  1 file changed, 5 insertions(+)
 
-diff --git a/src/boot/efi/efi-string.c b/src/boot/efi/efi-string.c
-index 22923d60f6..22a8d1ef71 100644
 --- a/src/boot/efi/efi-string.c
 +++ b/src/boot/efi/efi-string.c
-@@ -2,7 +2,12 @@
+@@ -2,7 +2,13 @@
 
  #include <stdbool.h>
  #include <stdint.h>
 +
 +#if SD_BOOT
 +typedef __WCHAR_TYPE__ wchar_t;
++#define __DEFINED_wchar_t
 +#else
  #include <wchar.h>

+#endif
--
2.39.2

#3 lore raw

Right, but what is the specific trigger for the error you are seeing that requires this additional definition, compared to passing musl tests with gcc?

I’d like to understand that because your change should not be necessary
and I’ve had a long argument with upstream even for the existing patch.

Alex

Quoted 65 lines · Khem Raj
Quoted 2 lines · Alexander Kanavin

How can the issue be triggered? I did test this on musl, and so did the autobuilder.

My setup is clang/musl on imx8 Which is similar to qemuarm64 And build works for systemd but Fails for systemd-boot recipe

Quoted 49 lines

Alex

Quoted 6 lines · Khem Raj

Current patch with fail when stdint.h is in include path before this typedef which can also happen with re-includes. Therefore shunt that by defining __DEFINED_wchar_t which helps musl builds of systemd-boot recipe

Quoted 7 lines
Cc Alexander Kanavin <[email protected]>
---
 ...rc-boot-efi-efi-string.c-define-wchar_t-from-__WCHA.patch | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git

a/meta/recipes-core/systemd/systemd/0004-src-boot-efi-efi-string.c-define-wchar_t-from-__WCHA.patch b/meta/recipes-core/systemd/systemd/0004-src-boot-efi-efi-string.c-define-wchar_t-from-__WCHA.patch

Quoted 2 lines

index 927f914848..43173ac32d 100644
---

a/meta/recipes-core/systemd/systemd/0004-src-boot-efi-efi-string.c-define-wchar_t-from-__WCHA.patch

Quoted 1 line

+++

b/meta/recipes-core/systemd/systemd/0004-src-boot-efi-efi-string.c-define-wchar_t-from-__WCHA.patch

Review Signed-off-by: Alexander Kanavin <

Quoted 1 line
(quoted diff) +0 -0
@@ -21,17 +21,16 @@ Signed-off-by: Alexander Kanavin <

[email protected]>

Quoted 22 lines
meta/recipes-core/systemd/systemd/0004-src-boot-efi-efi-string.c-define-wchar_t-from-__WCHA.patch +2 -4
  src/boot/efi/efi-string.c | 5 +++++
  1 file changed, 5 insertions(+)
 
-diff --git a/src/boot/efi/efi-string.c b/src/boot/efi/efi-string.c
-index 22923d60f6..22a8d1ef71 100644
 --- a/src/boot/efi/efi-string.c
 +++ b/src/boot/efi/efi-string.c
-@@ -2,7 +2,12 @@
+@@ -2,7 +2,13 @@
 
  #include <stdbool.h>
  #include <stdint.h>
 +
 +#if SD_BOOT
 +typedef __WCHAR_TYPE__ wchar_t;
++#define __DEFINED_wchar_t
 +#else
  #include <wchar.h>
 +#endif
--

2.39.2

author
#4 lore raw
Quoted 5 lines · Alexander Kanavin

Right, but what is the specific trigger for the error you are seeing that requires this additional definition, compared to passing musl tests with gcc?

I’d like to understand that because your change should not be necessary and I’ve had a long argument with upstream even for the existing patch.

alltypes.h is generated by musl during its build which is used by all system headers on musl systems it contains

#if defined(__NEED_wchar_t) && !defined(__DEFINED_wchar_t) typedef unsigned wchar_t;
#define __DEFINED_wchar_t
#endif

stddef.h defines __NEED_wchar_t which is included from efi-string.h and stddef.h immediately includes alltypes.h. so unless somewhere __DEFINED_wchar_t
is pre-defined above will be included and that's what the problem is.

The patch you created makes efi-string.c to type define wchar_t before including efi-string.h without blocking all possible ways for it to be defined by system headers.
Hope that makes it clear.

Quoted 61 lines

Alex

Quoted 58 lines · Khem Raj
Quoted 3 lines · Alexander Kanavin

How can the issue be triggered? I did test this on musl, and so did the autobuilder.

My setup is clang/musl on imx8 Which is similar to qemuarm64 And build works for systemd but Fails for systemd-boot recipe

Quoted 44 lines

Alex

Quoted 39 lines · Khem Raj

Current patch with fail when stdint.h is in include path before this typedef which can also happen with re-includes. Therefore shunt that by defining __DEFINED_wchar_t which helps musl builds of systemd-boot recipe

Cc Alexander Kanavin <[email protected]>
---
 ...rc-boot-efi-efi-string.c-define-wchar_t-from-__WCHA.patch | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
meta/recipes-core/systemd/systemd/0004-src-boot-efi-efi-string.c-define-wchar_t-from-__WCHA.patch +2 -4
index 927f914848..43173ac32d 100644
@@ -21,17 +21,16 @@ Signed-off-by: Alexander Kanavin <[email protected]>
  src/boot/efi/efi-string.c | 5 +++++
  1 file changed, 5 insertions(+)
 
-diff --git a/src/boot/efi/efi-string.c b/src/boot/efi/efi-string.c
-index 22923d60f6..22a8d1ef71 100644
 --- a/src/boot/efi/efi-string.c
 +++ b/src/boot/efi/efi-string.c
-@@ -2,7 +2,12 @@
+@@ -2,7 +2,13 @@
 
  #include <stdbool.h>
  #include <stdint.h>
 +
 +#if SD_BOOT
 +typedef __WCHAR_TYPE__ wchar_t;
++#define __DEFINED_wchar_t
 +#else
  #include <wchar.h>
 +#endif
--

2.39.2

#5 lore raw
Quoted 13 lines · Khem Raj

alltypes.h is generated by musl during its build which is used by all system headers on musl systems it contains

#if defined(__NEED_wchar_t) && !defined(__DEFINED_wchar_t) typedef unsigned wchar_t;
#define __DEFINED_wchar_t
#endif

stddef.h defines __NEED_wchar_t which is included from efi-string.h and stddef.h immediately includes alltypes.h. so unless somewhere __DEFINED_wchar_t
is pre-defined above will be included and that's what the problem is.

Quoted 4 lines

The patch you created makes efi-string.c to type define wchar_t before including efi-string.h without blocking all possible ways for it to be defined by system headers.
Hope that makes it clear.

I understand this. The question was about why this happens with clang, but does not happen with gcc.

I looked into this meanwhile. This seems to be a difference in how compilers look for headers.

There are actually two versions of stddef.h, one provided by the compiler itself (which would be the correct, expected one), and the other coming from musl.

gcc:
alex@Zen2:/srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0$ find . -name stddef.h ./recipe-sysroot-native/usr/lib/aarch64-poky-linux-musl/gcc/aarch64-poky-linux-musl/12.2.0/include/stddef.h ./recipe-sysroot/usr/include/linux/stddef.h
./recipe-sysroot/usr/include/stddef.h
./git/src/basic/linux/stddef.h

clang:
alex@Zen2:/srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0$ find . -name stddef.h
./recipe-sysroot-native/usr/lib/clang/15.0.7/include/stddef.h ./recipe-sysroot/usr/include/linux/stddef.h
./recipe-sysroot/usr/include/stddef.h
./git/src/basic/linux/stddef.h

gcc includes the correct header because its order of searching headers is:
#include <...> search starts here:
/srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/git/src/fundamental /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/git/src/boot/efi /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot/usr/include/efi/aarch64 /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot/usr/include/efi /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux-musl/../../lib/aarch64-poky-linux-musl/gcc/aarch64-poky-linux-musl/12.2.0/include /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot/usr/lib/aarch64-poky-linux-musl/12.2.0/include /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot/usr/include
End of search list.

clang includes the incorrect one:

#include <...> search starts here:
/srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/git/src/fundamental /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/git/src/boot/efi /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot/usr/include/efi/aarch64 /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot/usr/include/efi /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot/usr/include /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot-native/usr/lib/clang/15.0.7/include
End of search list.

So I'd say the search order needs to be aligned with gcc, rather than having to patch around the issue.

Alex

author
#6 lore raw
Quoted 61 lines · Alexander Kanavin
Quoted 13 lines · Khem Raj

alltypes.h is generated by musl during its build which is used by all system headers on musl systems it contains

#if defined(__NEED_wchar_t) && !defined(__DEFINED_wchar_t) typedef unsigned wchar_t;
#define __DEFINED_wchar_t
#endif

stddef.h defines __NEED_wchar_t which is included from efi-string.h and stddef.h immediately includes alltypes.h. so unless somewhere __DEFINED_wchar_t
is pre-defined above will be included and that's what the problem is.

Quoted 4 lines

The patch you created makes efi-string.c to type define wchar_t before including efi-string.h without blocking all possible ways for it to be defined by system headers.
Hope that makes it clear.

I understand this. The question was about why this happens with clang, but does not happen with gcc.

I looked into this meanwhile. This seems to be a difference in how compilers look for headers.

There are actually two versions of stddef.h, one provided by the compiler itself (which would be the correct, expected one), and the other coming from musl.

gcc:
alex@Zen2:/srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0$ find . -name stddef.h ./recipe-sysroot-native/usr/lib/aarch64-poky-linux-musl/gcc/aarch64-poky-linux-musl/12.2.0/include/stddef.h ./recipe-sysroot/usr/include/linux/stddef.h
./recipe-sysroot/usr/include/stddef.h
./git/src/basic/linux/stddef.h

clang:
alex@Zen2:/srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0$ find . -name stddef.h
./recipe-sysroot-native/usr/lib/clang/15.0.7/include/stddef.h ./recipe-sysroot/usr/include/linux/stddef.h
./recipe-sysroot/usr/include/stddef.h
./git/src/basic/linux/stddef.h

gcc includes the correct header because its order of searching headers is:
#include <...> search starts here:
/srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/git/src/fundamental /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/git/src/boot/efi /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot/usr/include/efi/aarch64 /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot/usr/include/efi /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux-musl/../../lib/aarch64-poky-linux-musl/gcc/aarch64-poky-linux-musl/12.2.0/include /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot/usr/lib/aarch64-poky-linux-musl/12.2.0/include /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot/usr/include
End of search list.

clang includes the incorrect one:

Quoted 9 lines

#include <...> search starts here:
/srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/git/src/fundamental /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/git/src/boot/efi /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot/usr/include/efi/aarch64 /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot/usr/include/efi /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot/usr/include /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot-native/usr/lib/clang/15.0.7/include
End of search list.

This is working as intended, unlike glibc, musl provides stddef.h and it wants musl based platforms to use that,
OE clang won't do that when the selected libc is glibc.

Quoted 2 lines

So I'd say the search order needs to be aligned with gcc, rather than having to patch around the issue.

#7 lore raw
Quoted 3 lines · Khem Raj

This is working as intended, unlike glibc, musl provides stddef.h and it wants musl based platforms to use that,
OE clang won't do that when the selected libc is glibc.

But this means gcc is not doing the right thing. It does take its own stddef.h (rather than one from musl), and this leads to undetected breakage like this one and general inconsistency between which set of headers is used.

It's gcc that is the default, AB-tested choice with musl, shouldn't it work correctly?

Alex

author
#8 lore raw
Quoted 11 lines · Alexander Kanavin
Quoted 3 lines · Khem Raj

This is working as intended, unlike glibc, musl provides stddef.h and it wants musl based platforms to use that,
OE clang won't do that when the selected libc is glibc.

But this means gcc is not doing the right thing. It does take its own stddef.h (rather than one from musl), and this leads to undetected breakage like this one and general inconsistency between which set of headers is used.

probably, but there maybe some reasons why its not done which I am not aware of.

Quoted 5 lines

It's gcc that is the default, AB-tested choice with musl, shouldn't it work correctly?

Alex

A message that is not in the archive (sent off-list or not yet received): [email protected]

#10 lore raw
Quoted 11 lines · Alexander Kanavin via lists.openembedded.org

clang includes the incorrect one:

#include <...> search starts here:
/srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/git/src/fundamental /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/git/src/boot/efi /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot/usr/include/efi/aarch64 /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot/usr/include/efi /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot/usr/include /srv/storage/alex/yocto/build-musl-arm-64/tmp/work/cortexa57-poky-linux-musl/systemd-boot/253.1-r0/recipe-sysroot-native/usr/lib/clang/15.0.7/include
End of search list.

Quoted 2 lines

So I'd say the search order needs to be aligned with gcc, rather than having to patch around the issue.

For reference, clang on my host gets it right and matches gcc:

alex@alex-lx-laptop:~$ echo | clang-14 -E -Wp,-v -
clang -cc1 version 14.0.6 based upon LLVM 14.0.6 default target
x86_64-pc-linux-gnu
...
#include <...> search starts here:
 /usr/lib/llvm-14/lib/clang/14.0.6/include
 /usr/local/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.

Alex