Skip to contentSkip to conversation

systemd: Fix build on musl

PATCH In review Khem Raj 5 participants 10 messages Jul 27, 2021 – Aug 6, 2021 1 message not in the archive

lore
Share & tools
lore

Khem Raj

author
#0 lore raw

PATCH 1/1 systemd: Fix build on musl

Latest upgrade to 249 broke a working build on musl, this is incremental fix to exisiting patches adapted to 249 release

Cc Luca Boccassi <[email protected]>
---
 ...002-don-t-use-glibc-specific-qsort_r.patch |  24 +--
 ...missing.h-check-for-missing-strndupa.patch | 173 +++++++++---------
 .../0006-Include-netinet-if_ether.h.patch     | 147 ++++++++-------
 ...OB_BRACE-and-GLOB_ALTDIRFUNC-is-not-.patch |  20 +-
 ...T_SYMLINK_NOFOLLOW-flag-to-faccessat.patch |  14 +-
 ...-not-disable-buffer-in-writing-files.patch | 143 ++++++---------
 .../systemd/0026-Handle-missing-gshadow.patch |  18 +-
 7 files changed, 247 insertions(+), 292 deletions(-)
meta/recipes-core/systemd/systemd/0002-don-t-use-glibc-specific-qsort_r.patch +9 -15
index 4cdf66e767..9db22cd9d2 100644
@@ -19,11 +19,9 @@ Signed-off-by: Luca Boccassi <[email protected]>
  src/shared/format-table.c          | 36 ++++++++++++++++++++----------
  3 files changed, 38 insertions(+), 31 deletions(-)
 
-diff --git a/src/basic/sort-util.h b/src/basic/sort-util.h
-index 49586a4a24..d92a5ab0ed 100644
 --- a/src/basic/sort-util.h
 +++ b/src/basic/sort-util.h
-@@ -55,18 +55,4 @@ static inline void _qsort_safe(void *base, size_t nmemb, size_t size, __compar_f
+@@ -56,18 +56,4 @@ static inline void _qsort_safe(void *bas
                  _qsort_safe((p), (n), sizeof((p)[0]), (__compar_fn_t) _func_); \
          })
  
@@ -42,11 +40,9 @@ index 49586a4a24..d92a5ab0ed 100644
 -        })
 -
  int cmp_int(const int *a, const int *b);
-diff --git a/src/libsystemd/sd-hwdb/hwdb-util.c b/src/libsystemd/sd-hwdb/hwdb-util.c
-index fd45ff0f54..ac4b63c49b 100644
---- a/src/libsystemd/sd-hwdb/hwdb-util.c
-+++ b/src/libsystemd/sd-hwdb/hwdb-util.c
-@@ -126,9 +126,13 @@ static struct trie* trie_free(struct trie *trie) {
+--- a/src/shared/hwdb-util.c
++++ b/src/shared/hwdb-util.c
+@@ -127,9 +127,13 @@ static struct trie* trie_free(struct tri
  
  DEFINE_TRIVIAL_CLEANUP_FUNC(struct trie*, trie_free);
  
@@ -63,7 +59,7 @@ index fd45ff0f54..ac4b63c49b 100644
  }
  
  static int trie_node_add_value(struct trie *trie, struct trie_node *node,
-@@ -156,7 +160,10 @@ static int trie_node_add_value(struct trie *trie, struct trie_node *node,
+@@ -157,7 +161,10 @@ static int trie_node_add_value(struct tr
                          .value_off = v,
                  };
  
@@ -75,7 +71,7 @@ index fd45ff0f54..ac4b63c49b 100644
                  if (val) {
                          /* At this point we have 2 identical properties on the same match-string.
                           * Since we process files in order, we just replace the previous value. */
-@@ -182,7 +189,9 @@ static int trie_node_add_value(struct trie *trie, struct trie_node *node,
+@@ -183,7 +190,9 @@ static int trie_node_add_value(struct tr
                  .line_number = line_number,
          };
          node->values_count++;
@@ -86,11 +82,9 @@ index fd45ff0f54..ac4b63c49b 100644
          return 0;
  }
  
-diff --git a/src/shared/format-table.c b/src/shared/format-table.c
-index dccb796b26..c3ab8ac296 100644
 --- a/src/shared/format-table.c
 +++ b/src/shared/format-table.c
-@@ -1290,30 +1290,32 @@ static int cell_data_compare(TableData *a, size_t index_a, TableData *b, size_t
+@@ -1282,30 +1282,32 @@ static int cell_data_compare(TableData *
          return CMP(index_a, index_b);
  }
  
@@ -133,7 +127,7 @@ index dccb796b26..c3ab8ac296 100644
          }
  
          /* Order identical lines by the order there were originally added in */
-@@ -1952,7 +1954,12 @@ int table_print(Table *t, FILE *f) {
+@@ -1944,7 +1946,12 @@ int table_print(Table *t, FILE *f) {
                  for (size_t i = 0; i < n_rows; i++)
                          sorted[i] = i * t->n_columns;
  
@@ -147,7 +141,7 @@ index dccb796b26..c3ab8ac296 100644
          }
  
          if (t->display_map)
-@@ -2580,7 +2587,12 @@ int table_to_json(Table *t, JsonVariant **ret) {
+@@ -2572,7 +2579,12 @@ int table_to_json(Table *t, JsonVariant
                  for (size_t i = 0; i < n_rows; i++)
                          sorted[i] = i * t->n_columns;
  
meta/recipes-core/systemd/systemd/0005-src-basic-missing.h-check-for-missing-strndupa.patch +82 -91

493 lines not on this page · raw message

meta/recipes-core/systemd/systemd/0006-Include-netinet-if_ether.h.patch +78 -69

274 lines not on this page · raw message

meta/recipes-core/systemd/systemd/0007-don-t-fail-if-GLOB_BRACE-and-GLOB_ALTDIRFUNC-is-not-.patch +7 -13
index 2ff08c7fae..4a60ee32ef 100644
@@ -23,8 +23,6 @@ Signed-off-by: Scott Murray <[email protected]>
  src/tmpfiles/tmpfiles.c   | 10 ++++++++++
  3 files changed, 38 insertions(+)
 
-diff --git a/src/basic/glob-util.c b/src/basic/glob-util.c
-index bc0278e57f..c973f82e54 100644
 --- a/src/basic/glob-util.c
 +++ b/src/basic/glob-util.c
 @@ -12,6 +12,12 @@
@@ -48,7 +46,7 @@ index bc0278e57f..c973f82e54 100644
          /* We want to set GLOB_ALTDIRFUNC ourselves, don't allow it to be set. */
          assert(!(flags & GLOB_ALTDIRFUNC));
  
-@@ -32,9 +39,14 @@ int safe_glob(const char *path, int flags, glob_t *pglob) {
+@@ -32,9 +39,14 @@ int safe_glob(const char *path, int flag
                  pglob->gl_lstat = lstat;
          if (!pglob->gl_stat)
                  pglob->gl_stat = stat;
@@ -63,8 +61,6 @@ index bc0278e57f..c973f82e54 100644
          if (k == GLOB_NOMATCH)
                  return -ENOENT;
          if (k == GLOB_NOSPACE)
-diff --git a/src/test/test-glob-util.c b/src/test/test-glob-util.c
-index df6444c433..79a692046e 100644
 --- a/src/test/test-glob-util.c
 +++ b/src/test/test-glob-util.c
 @@ -12,6 +12,12 @@
@@ -78,9 +74,9 @@ index df6444c433..79a692046e 100644
 +#endif
 +
  static void test_glob_exists(void) {
-         char name[] = "/tmp/test-glob_exists.XXXXXX";
-         int fd = -1;
-@@ -39,11 +45,13 @@ static void test_glob_no_dot(void) {
+         log_info("/* %s */", __func__);
+ 
+@@ -41,11 +47,13 @@ static void test_glob_no_dot(void) {
          const char *fn;
  
          _cleanup_globfree_ glob_t g = {
@@ -94,7 +90,7 @@ index df6444c433..79a692046e 100644
          };
  
          int r;
-@@ -51,11 +59,19 @@ static void test_glob_no_dot(void) {
+@@ -55,11 +63,19 @@ static void test_glob_no_dot(void) {
          assert_se(mkdtemp(template));
  
          fn = strjoina(template, "/*");
@@ -114,8 +110,6 @@ index df6444c433..79a692046e 100644
          assert_se(r == GLOB_NOMATCH);
  
          (void) rm_rf(template, REMOVE_ROOT|REMOVE_PHYSICAL);
-diff --git a/src/tmpfiles/tmpfiles.c b/src/tmpfiles/tmpfiles.c
-index 6b73d1e6d5..26b3b74275 100644
 --- a/src/tmpfiles/tmpfiles.c
 +++ b/src/tmpfiles/tmpfiles.c
 @@ -66,6 +66,12 @@
@@ -131,7 +125,7 @@ index 6b73d1e6d5..26b3b74275 100644
  /* This reads all files listed in /etc/tmpfiles.d/?*.conf and creates
   * them in the file system. This is intended to be used to create
   * properly owned directories beneath /tmp, /var/tmp, /run, which are
-@@ -1947,7 +1953,9 @@ finish:
+@@ -1990,7 +1996,9 @@ finish:
  
  static int glob_item(Item *i, action_t action) {
          _cleanup_globfree_ glob_t g = {
@@ -141,7 +135,7 @@ index 6b73d1e6d5..26b3b74275 100644
          };
          int r = 0, k;
          char **fn;
-@@ -1967,7 +1975,9 @@ static int glob_item(Item *i, action_t action) {
+@@ -2010,7 +2018,9 @@ static int glob_item(Item *i, action_t a
  
  static int glob_item_recursively(Item *i, fdaction_t action) {
          _cleanup_globfree_ glob_t g = {
meta/recipes-core/systemd/systemd/0012-don-t-pass-AT_SYMLINK_NOFOLLOW-flag-to-faccessat.patch +5 -9
index 5f2178734d..cf06572b82 100644
@@ -31,11 +31,9 @@ Signed-off-by: Andre McCurdy <[email protected]>
  src/shared/base-filesystem.c |  6 +++---
  2 files changed, 24 insertions(+), 5 deletions(-)
 
-diff --git a/src/basic/fs-util.h b/src/basic/fs-util.h
-index 027037f7a7..16eb379caf 100644
 --- a/src/basic/fs-util.h
 +++ b/src/basic/fs-util.h
-@@ -44,8 +44,27 @@ int futimens_opath(int fd, const struct timespec ts[2]);
+@@ -47,8 +47,27 @@ int futimens_opath(int fd, const struct
  int fd_warn_permissions(const char *path, int fd);
  int stat_warn_permissions(const char *path, const struct stat *st);
  
@@ -65,20 +63,18 @@ index 027037f7a7..16eb379caf 100644
  
  int touch_file(const char *path, bool parents, usec_t stamp, uid_t uid, gid_t gid, mode_t mode);
  int touch(const char *path);
-diff --git a/src/shared/base-filesystem.c b/src/shared/base-filesystem.c
-index 1d05409086..1ed06c31ab 100644
 --- a/src/shared/base-filesystem.c
 +++ b/src/shared/base-filesystem.c
-@@ -54,7 +54,7 @@ int base_filesystem_create(const char *root, uid_t uid, gid_t gid) {
+@@ -53,7 +53,7 @@ int base_filesystem_create(const char *r
                  return log_error_errno(errno, "Failed to open root file system: %m");
  
-         for (i = 0; i < ELEMENTSOF(table); i ++) {
+         for (size_t i = 0; i < ELEMENTSOF(table); i++) {
 -                if (faccessat(fd, table[i].dir, F_OK, AT_SYMLINK_NOFOLLOW) >= 0)
 +                if (faccessat(fd, table[i].dir, F_OK, 0) >= 0)
                          continue;
  
                  if (table[i].target) {
-@@ -62,7 +62,7 @@ int base_filesystem_create(const char *root, uid_t uid, gid_t gid) {
+@@ -61,7 +61,7 @@ int base_filesystem_create(const char *r
  
                          /* check if one of the targets exists */
                          NULSTR_FOREACH(s, table[i].target) {
@@ -87,7 +83,7 @@ index 1d05409086..1ed06c31ab 100644
                                          continue;
  
                                  /* check if a specific file exists at the target path */
-@@ -73,7 +73,7 @@ int base_filesystem_create(const char *root, uid_t uid, gid_t gid) {
+@@ -72,7 +72,7 @@ int base_filesystem_create(const char *r
                                          if (!p)
                                                  return log_oom();
  
meta/recipes-core/systemd/systemd/0022-do-not-disable-buffer-in-writing-files.patch +60 -83

323 lines not on this page · raw message

meta/recipes-core/systemd/systemd/0026-Handle-missing-gshadow.patch +6 -12
index 068600fa44..9394eeb450 100644
@@ -17,8 +17,6 @@ Signed-off-by: Luca Boccassi <[email protected]>
  src/shared/userdb.c          |  6 ++++++
  3 files changed, 30 insertions(+)
 
-diff --git a/src/shared/user-record-nss.c b/src/shared/user-record-nss.c
-index 88b8fc2f8f..a819d41bac 100644
 --- a/src/shared/user-record-nss.c
 +++ b/src/shared/user-record-nss.c
 @@ -331,8 +331,10 @@ int nss_group_to_group_record(
@@ -48,7 +46,7 @@ index 88b8fc2f8f..a819d41bac 100644
  
          r = json_build(&g->json, JSON_BUILD_OBJECT(
                                         JSON_BUILD_PAIR("groupName", JSON_BUILD_STRING(g->group_name)),
-@@ -388,6 +392,7 @@ int nss_sgrp_for_group(const struct group *grp, struct sgrp *ret_sgrp, char **re
+@@ -388,6 +392,7 @@ int nss_sgrp_for_group(const struct grou
          assert(ret_sgrp);
          assert(ret_buffer);
  
@@ -56,7 +54,7 @@ index 88b8fc2f8f..a819d41bac 100644
          for (;;) {
                  _cleanup_free_ char *buf = NULL;
                  struct sgrp sgrp, *result;
-@@ -416,6 +421,9 @@ int nss_sgrp_for_group(const struct group *grp, struct sgrp *ret_sgrp, char **re
+@@ -416,6 +421,9 @@ int nss_sgrp_for_group(const struct grou
                  buflen *= 2;
                  buf = mfree(buf);
          }
@@ -122,8 +120,6 @@ index 88b8fc2f8f..a819d41bac 100644
          if (r < 0)
                  return r;
  
-diff --git a/src/shared/user-record-nss.h b/src/shared/user-record-nss.h
-index 22ab04d6ee..4e52e7a911 100644
 --- a/src/shared/user-record-nss.h
 +++ b/src/shared/user-record-nss.h
 @@ -2,7 +2,11 @@
@@ -138,28 +134,26 @@ index 22ab04d6ee..4e52e7a911 100644
  #include <pwd.h>
  #include <shadow.h>
  
-diff --git a/src/shared/userdb.c b/src/shared/userdb.c
-index 613350bd46..5c821d28cd 100644
 --- a/src/shared/userdb.c
 +++ b/src/shared/userdb.c
-@@ -938,13 +938,16 @@ int groupdb_iterator_get(UserDBIterator *iterator, GroupRecord **ret) {
+@@ -1047,13 +1047,15 @@ int groupdb_iterator_get(UserDBIterator
                  if (gr) {
                          _cleanup_free_ char *buffer = NULL;
                          bool incomplete = false;
 +#if ENABLE_GSHADOW
                          struct sgrp sgrp;
+-
 +#endif
- 
                          if (streq_ptr(gr->gr_name, "root"))
                                  iterator->synthesize_root = false;
                          if (gr->gr_gid == GID_NOBODY)
                                  iterator->synthesize_nobody = false;
  
 +#if ENABLE_GSHADOW
-                         if (!FLAGS_SET(iterator->flags, USERDB_AVOID_SHADOW)) {
+                         if (!FLAGS_SET(iterator->flags, USERDB_SUPPRESS_SHADOW)) {
                                  r = nss_sgrp_for_group(gr, &sgrp, &buffer);
                                  if (r < 0) {
-@@ -957,6 +960,9 @@ int groupdb_iterator_get(UserDBIterator *iterator, GroupRecord **ret) {
+@@ -1066,6 +1068,9 @@ int groupdb_iterator_get(UserDBIterator
                          }
  
                          r = nss_group_to_group_record(gr, r >= 0 ? &sgrp : NULL, ret);
Signature · 1 line
2.32.0
#1 lore raw

Hi,

Does this mean we have musl and systemd working together?

Regards,

Robert

Quoted 8 lines · Khem Raj

Latest upgrade to 249 broke a working build on musl, this is incremental fix to exisiting patches adapted to 249 release

Cc Luca Boccassi <[email protected]>
author
#2 lore raw
Quoted 5 lines · Robert Berger

Hi,

Does this mean we have musl and systemd working together?

yes it was always there

Quoted 14 lines

Regards,

Robert

Quoted 8 lines · Khem Raj

Latest upgrade to 249 broke a working build on musl, this is incremental fix to exisiting patches adapted to 249 release

Cc Luca Boccassi <[email protected]>

A message that is not in the archive (sent off-list or not yet received): AM7PR83MB0436960F9D4BFDD1F504F0DFF1EB9@AM7PR83MB0436.EURPRD83.prod.outlook.com

#4 lore raw
Quoted 6 lines · Khem Raj

Latest upgrade to 249 broke a working build on musl, this is incremental fix to exisiting patches adapted to 249 release

Cc Luca Boccassi <[email protected]>
Quoted 10 lines
---
 ...002-don-t-use-glibc-specific-qsort_r.patch |  24 +--
 ...missing.h-check-for-missing-strndupa.patch | 173 +++++++++---------
 .../0006-Include-netinet-if_ether.h.patch     | 147 ++++++++-------
 ...OB_BRACE-and-GLOB_ALTDIRFUNC-is-not-.patch |  20 +-
 ...T_SYMLINK_NOFOLLOW-flag-to-faccessat.patch |  14 +-
 ...-not-disable-buffer-in-writing-files.patch | 143 ++++++---------
 .../systemd/0026-Handle-missing-gshadow.patch |  18 +-
 7 files changed, 247 insertions(+), 292 deletions(-)

Having a look at the patches, a few comments:

- 0001-test-parse-argument-Include-signal.h.patch was merged upstream, but never removed, I think because it was refactored so it applies cleanly and doesn't raise errors?

- 0005-src-basic-missing.h-check-for-missing-strndupa.patch this is massive and will keep growing and breaking the build, wouldn't it be better to just implement strndupa in musl? Even out-of-tree it sounds like it would be easier to maintain than this

- 0006-Include-netinet-if_ether.h.patch sounds like it should be doable in a way that is compatible with both glibc and musl, and thus could be upstreamed?

- 0010-Use-uintmax_t-for-handling-rlim_t.patch can be upstreamed but needs to be reworked slightly as explained in
https://github.com/systemd/systemd/pull/7199#issuecomment-358268647
which seems worth doing to me

- 0012-don-t-pass-AT_SYMLINK_NOFOLLOW-flag-to-faccessat.patch I find quite worrying, as it fundamentally changes access patterns, some of which are done for security reasons. At best, this will cause completely different runtime behaviours for the same filesystem depending on the libc implementation, which doesn't sound great?

- 0016-Hide-__start_BUS_ERROR_MAP-and-__stop_BUS_ERROR_MAP.patch sounds like a genuine bug, have you tried upstreaming it?

- 0019-Handle-missing-LOCK_EX.patch I see no problem upstreaming this

- 0028-missing_syscall.h-Define-MIPS-ABI-defines-for-musl.patch should also be fine to upstream I think, in src/basic/missing_syscall_def.h

Signature · 2 lines
Kind regards,
Luca Boccassi
#5 lore raw
Quoted 43 lines · Luca Boccassi
Quoted 5 lines · Khem Raj

Latest upgrade to 249 broke a working build on musl, this is incremental fix to exisiting patches adapted to 249 release

Cc Luca Boccassi <[email protected]>
Quoted 10 lines
---
 ...002-don-t-use-glibc-specific-qsort_r.patch |  24 +--
 ...missing.h-check-for-missing-strndupa.patch | 173 +++++++++---------
 .../0006-Include-netinet-if_ether.h.patch     | 147 ++++++++-------
 ...OB_BRACE-and-GLOB_ALTDIRFUNC-is-not-.patch |  20 +-
 ...T_SYMLINK_NOFOLLOW-flag-to-faccessat.patch |  14 +-
 ...-not-disable-buffer-in-writing-files.patch | 143 ++++++---------
 .../systemd/0026-Handle-missing-gshadow.patch |  18 +-
 7 files changed, 247 insertions(+), 292 deletions(-)

Having a look at the patches, a few comments:

- 0001-test-parse-argument-Include-signal.h.patch was merged upstream, but never removed, I think because it was refactored so it applies cleanly and doesn't raise errors?

- 0005-src-basic-missing.h-check-for-missing-strndupa.patch this is massive and will keep growing and breaking the build, wouldn't it be better to just implement strndupa in musl? Even out-of-tree it sounds like it would be easier to maintain than this

- 0006-Include-netinet-if_ether.h.patch sounds like it should be doable in a way that is compatible with both glibc and musl, and thus could be upstreamed?

- 0010-Use-uintmax_t-for-handling-rlim_t.patch can be upstreamed but needs to be reworked slightly as explained in
https://github.com/systemd/systemd/pull/7199#issuecomment-358268647
which seems worth doing to me

- 0012-don-t-pass-AT_SYMLINK_NOFOLLOW-flag-to-faccessat.patch I find quite worrying, as it fundamentally changes access patterns, some of which are done for security reasons. At best, this will cause completely different runtime behaviours for the same filesystem depending on the libc implementation, which doesn't sound great?

Quoted 7 lines

- 0016-Hide-__start_BUS_ERROR_MAP-and-__stop_BUS_ERROR_MAP.patch sounds like a genuine bug, have you tried upstreaming it?

- 0019-Handle-missing-LOCK_EX.patch I see no problem upstreaming this

- 0028-missing_syscall.h-Define-MIPS-ABI-defines-for-musl.patch should also be fine to upstream I think, in src/basic/missing_syscall_def.h

Also isn't 0001-binfmt-Don-t-install-dependency-links-at-install- tim.patch doable with two drop-ins and a post-install adjustment in the recipe, rather than with a patch?

This also sounds like a good candidate for a drop-in, doesn't seem generally applicable to me:

https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=f65757ad1791a46cc210141c84ccc51552171e98

Signature · 2 lines
Kind regards,
Luca Boccassi
#6 lore raw

I have to wonder again if it would be better to maintain the musl spin of systemd in a separate repo and provided via separate recipe?

Alex

Quoted 70 lines · Luca Bocassi
Quoted 2 lines · Luca Boccassi
Quoted 1 line · Khem Raj

Latest upgrade to 249 broke a working build on musl, this is

incremental

Quoted 41 lines
Quoted 4 lines

fix to exisiting patches adapted to 249 release

Cc Luca Boccassi <[email protected]>
Quoted 10 lines
---
 ...002-don-t-use-glibc-specific-qsort_r.patch |  24 +--
 ...missing.h-check-for-missing-strndupa.patch | 173 +++++++++---------
 .../0006-Include-netinet-if_ether.h.patch     | 147 ++++++++-------
 ...OB_BRACE-and-GLOB_ALTDIRFUNC-is-not-.patch |  20 +-
 ...T_SYMLINK_NOFOLLOW-flag-to-faccessat.patch |  14 +-
 ...-not-disable-buffer-in-writing-files.patch | 143 ++++++---------
 .../systemd/0026-Handle-missing-gshadow.patch |  18 +-
 7 files changed, 247 insertions(+), 292 deletions(-)

Having a look at the patches, a few comments:

- 0001-test-parse-argument-Include-signal.h.patch was merged upstream, but never removed, I think because it was refactored so it applies cleanly and doesn't raise errors?

- 0005-src-basic-missing.h-check-for-missing-strndupa.patch this is massive and will keep growing and breaking the build, wouldn't it be better to just implement strndupa in musl? Even out-of-tree it sounds like it would be easier to maintain than this

- 0006-Include-netinet-if_ether.h.patch sounds like it should be doable in a way that is compatible with both glibc and musl, and thus could be upstreamed?

- 0010-Use-uintmax_t-for-handling-rlim_t.patch can be upstreamed but needs to be reworked slightly as explained in
https://github.com/systemd/systemd/pull/7199#issuecomment-358268647
which seems worth doing to me

- 0012-don-t-pass-AT_SYMLINK_NOFOLLOW-flag-to-faccessat.patch I find quite worrying, as it fundamentally changes access patterns, some of which are done for security reasons. At best, this will cause completely different runtime behaviours for the same filesystem depending on the libc implementation, which doesn't sound great?

Quoted 7 lines

- 0016-Hide-__start_BUS_ERROR_MAP-and-__stop_BUS_ERROR_MAP.patch sounds like a genuine bug, have you tried upstreaming it?

- 0019-Handle-missing-LOCK_EX.patch I see no problem upstreaming this

- 0028-missing_syscall.h-Define-MIPS-ABI-defines-for-musl.patch should also be fine to upstream I think, in src/basic/missing_syscall_def.h

Also isn't 0001-binfmt-Don-t-install-dependency-links-at-install- tim.patch doable with two drop-ins and a post-install adjustment in the recipe, rather than with a patch?

This also sounds like a good candidate for a drop-in, doesn't seem generally applicable to me:

https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=f65757ad1791a46cc210141c84ccc51552171e98

--
Kind regards,
Luca Boccassi

author
#7 lore raw
In reply to Luca Bocassi
Quoted 19 lines · Luca Boccassi
Quoted 17 lines · Luca Boccassi
Quoted 15 lines · Khem Raj

Latest upgrade to 249 broke a working build on musl, this is incremental fix to exisiting patches adapted to 249 release

Cc Luca Boccassi <[email protected]>
---
 ...002-don-t-use-glibc-specific-qsort_r.patch |  24 +--
 ...missing.h-check-for-missing-strndupa.patch | 173 +++++++++---------
 .../0006-Include-netinet-if_ether.h.patch     | 147 ++++++++-------
 ...OB_BRACE-and-GLOB_ALTDIRFUNC-is-not-.patch |  20 +-
 ...T_SYMLINK_NOFOLLOW-flag-to-faccessat.patch |  14 +-
 ...-not-disable-buffer-in-writing-files.patch | 143 ++++++---------
 .../systemd/0026-Handle-missing-gshadow.patch |  18 +-
 7 files changed, 247 insertions(+), 292 deletions(-)
Quoted 5 lines
Quoted 5 lines

Having a look at the patches, a few comments:

- 0001-test-parse-argument-Include-signal.h.patch was merged upstream, but never removed, I think because it was refactored so it applies cleanly and doesn't raise errors?

seems so.

Quoted 9 lines
Quoted 9 lines

- 0005-src-basic-missing.h-check-for-missing-strndupa.patch this is massive and will keep growing and breaking the build, wouldn't it be better to just implement strndupa in musl? Even out-of-tree it sounds like it would be easier to maintain than this

Quoted 3 lines
Quoted 3 lines

- 0006-Include-netinet-if_ether.h.patch sounds like it should be doable in a way that is compatible with both glibc and musl, and thus could be upstreamed?

I think it could be, although systemd has started caching some of these system headers in its own sources
which is also troublesome. So overall it has to be addressed with a bit wider scope.

Quoted 6 lines
Quoted 6 lines

- 0010-Use-uintmax_t-for-handling-rlim_t.patch can be upstreamed but needs to be reworked slightly as explained in
https://github.com/systemd/systemd/pull/7199#issuecomment-358268647
which seems worth doing to me

perhaps yes, although I was not sure what it would be for a patch.

Quoted 9 lines
Quoted 9 lines

- 0012-don-t-pass-AT_SYMLINK_NOFOLLOW-flag-to-faccessat.patch I find quite worrying, as it fundamentally changes access patterns, some of which are done for security reasons. At best, this will cause completely different runtime behaviours for the same filesystem depending on the libc implementation, which doesn't sound great?

- 0016-Hide-__start_BUS_ERROR_MAP-and-__stop_BUS_ERROR_MAP.patch sounds like a genuine bug, have you tried upstreaming it?

It was part of a cumulative patchset to support musl which was not accepted but I can try this patch alone.

Quoted 1 line
Quoted 1 line

- 0019-Handle-missing-LOCK_EX.patch I see no problem upstreaming this

OK

Quoted 3 lines
Quoted 3 lines

- 0028-missing_syscall.h-Define-MIPS-ABI-defines-for-musl.patch should also be fine to upstream I think, in src/basic/missing_syscall_def.h

OK

Quoted 10 lines

Also isn't 0001-binfmt-Don-t-install-dependency-links-at-install- tim.patch doable with two drop-ins and a post-install adjustment in the recipe, rather than with a patch?

This also sounds like a good candidate for a drop-in, doesn't seem generally applicable to me:

https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=f65757ad1791a46cc210141c84ccc51552171e98

surely although its generic and not musl specific perhaps deal with it separately.

Quoted 3 lines

--
Kind regards,
Luca Boccassi

#8 lore raw
In reply to Luca Bocassi
Quoted 8 lines · Luca Bocassi

Having a look at the patches, a few comments:

- 0012-don-t-pass-AT_SYMLINK_NOFOLLOW-flag-to-faccessat.patch I find quite worrying, as it fundamentally changes access patterns, some of which are done for security reasons. At best, this will cause completely different runtime behaviours for the same filesystem depending on the libc implementation, which doesn't sound great?

I wrote a long and verbose comment when I created the patch which tries to document any differences in runtime behaviour.

----
Avoid using AT_SYMLINK_NOFOLLOW flag. It doesn't seem like the right thing to do and it's not portable (not supported by musl). See:

    http://lists.landley.net/pipermail/toybox-landley.net/2014-September/003610.html
    http://www.openwall.com/lists/musl/2015/02/05/2

Note that laccess() is never passing AT_EACCESS so a lot of the discussion in the links above doesn't apply. Note also that (currently) all systemd callers of laccess() pass mode as F_OK, so only check for existence of a file, not access permissions. Therefore, in this case, the only distiction between faccessat() with (flag == 0) and (flag == AT_SYMLINK_NOFOLLOW) is the behaviour for broken symlinks; laccess() on a broken symlink will succeed with (flag == AT_SYMLINK_NOFOLLOW) and fail (flag == 0).

The laccess() macros was added to systemd some time ago and it's not clear if or why it needs to return success for broken symlinks. Maybe just historical and not actually necessary or desired behaviour?
----

If that comment is now out of date or something is missing then please send a patch to update it.

However looking at this patch again now, it appears to have got broken during a past rebase:

https://git.openembedded.org/openembedded-core/commit/?id=e8dd5a36bf2f1e645fb2ff15eb3b5e97c04776e6

The upstream code changed from:

#define laccess(path, mode) faccessat(AT_FDCWD, (path), (mode),
AT_SYMLINK_NOFOLLOW)

to

#define laccess(path, mode) \
(faccessat(AT_FDCWD, (path), (mode), AT_SYMLINK_NOFOLLOW) <
0 ? -errno : 0)

but the replacement version in the patch still returns the raw result from faccessat(). That looks like an issue.

#9 lore raw
Quoted 47 lines · Andre McCurdy
Quoted 8 lines · Luca Bocassi

Having a look at the patches, a few comments:

- 0012-don-t-pass-AT_SYMLINK_NOFOLLOW-flag-to-faccessat.patch I find quite worrying, as it fundamentally changes access patterns, some of which are done for security reasons. At best, this will cause completely different runtime behaviours for the same filesystem depending on the libc implementation, which doesn't sound great?

I wrote a long and verbose comment when I created the patch which tries to document any differences in runtime behaviour.

----
Avoid using AT_SYMLINK_NOFOLLOW flag. It doesn't seem like the right thing to do and it's not portable (not supported by musl). See:

    http://lists.landley.net/pipermail/toybox-landley.net/2014-September/003610.html
    http://www.openwall.com/lists/musl/2015/02/05/2

Note that laccess() is never passing AT_EACCESS so a lot of the discussion in the links above doesn't apply. Note also that (currently) all systemd callers of laccess() pass mode as F_OK, so only check for existence of a file, not access permissions. Therefore, in this case, the only distiction between faccessat() with (flag == 0) and (flag == AT_SYMLINK_NOFOLLOW) is the behaviour for broken symlinks; laccess() on a broken symlink will succeed with (flag == AT_SYMLINK_NOFOLLOW) and fail (flag == 0).

The laccess() macros was added to systemd some time ago and it's not clear if or why it needs to return success for broken symlinks. Maybe just historical and not actually necessary or desired behaviour?
----

If that comment is now out of date or something is missing then please send a patch to update it.

However looking at this patch again now, it appears to have got broken during a past rebase:

https://git.openembedded.org/openembedded-core/commit/?id=e8dd5a36bf2f1e645fb2ff15eb3b5e97c04776e6

The upstream code changed from:

#define laccess(path, mode) faccessat(AT_FDCWD, (path), (mode),
AT_SYMLINK_NOFOLLOW)

Quoted 8 lines

to

#define laccess(path, mode) \
(faccessat(AT_FDCWD, (path), (mode), AT_SYMLINK_NOFOLLOW) <
0 ? -errno : 0)

but the replacement version in the patch still returns the raw result from faccessat(). That looks like an issue.

If you think the flag is unnecessary (I don't, we use these for a reason, but that's not important right now), the correct action is to send a PR upstream to discuss removing it. Patching it out for one build case of many is just going to be a source of incompatibility and surprises for users, as the behaviour on the same filesystem changes depending on the build option. Having said that, I don't use musl so all of this is really not a problem for me, just providing some feedback as upstream maintainer, in case it can be useful.

Kind regards,
Luca Boccassi

#10 lore raw
Quoted 58 lines · Luca Boccassi
Quoted 47 lines · Andre McCurdy
Quoted 8 lines · Luca Bocassi

Having a look at the patches, a few comments:

- 0012-don-t-pass-AT_SYMLINK_NOFOLLOW-flag-to-faccessat.patch I find quite worrying, as it fundamentally changes access patterns, some of which are done for security reasons. At best, this will cause completely different runtime behaviours for the same filesystem depending on the libc implementation, which doesn't sound great?

I wrote a long and verbose comment when I created the patch which tries to document any differences in runtime behaviour.

----
Avoid using AT_SYMLINK_NOFOLLOW flag. It doesn't seem like the right thing to do and it's not portable (not supported by musl). See:

    http://lists.landley.net/pipermail/toybox-landley.net/2014-September/003610.html
    http://www.openwall.com/lists/musl/2015/02/05/2

Note that laccess() is never passing AT_EACCESS so a lot of the discussion in the links above doesn't apply. Note also that (currently) all systemd callers of laccess() pass mode as F_OK, so only check for existence of a file, not access permissions. Therefore, in this case, the only distiction between faccessat() with (flag == 0) and (flag == AT_SYMLINK_NOFOLLOW) is the behaviour for broken symlinks; laccess() on a broken symlink will succeed with (flag == AT_SYMLINK_NOFOLLOW) and fail (flag == 0).

The laccess() macros was added to systemd some time ago and it's not clear if or why it needs to return success for broken symlinks. Maybe just historical and not actually necessary or desired behaviour?
----

If that comment is now out of date or something is missing then please send a patch to update it.

However looking at this patch again now, it appears to have got broken during a past rebase:

https://git.openembedded.org/openembedded-core/commit/?id=e8dd5a36bf2f1e645fb2ff15eb3b5e97c04776e6

The upstream code changed from:

#define laccess(path, mode) faccessat(AT_FDCWD, (path), (mode),
AT_SYMLINK_NOFOLLOW)

Quoted 8 lines

to

#define laccess(path, mode) \
(faccessat(AT_FDCWD, (path), (mode), AT_SYMLINK_NOFOLLOW) <
0 ? -errno : 0)

but the replacement version in the patch still returns the raw result from faccessat(). That looks like an issue.

Quoted 8 lines

If you think the flag is unnecessary (I don't, we use these for a reason, but that's not important right now), the correct action is to send a PR upstream to discuss removing it. Patching it out for one build case of many is just going to be a source of incompatibility and surprises for users, as the behaviour on the same filesystem changes depending on the build option. Having said that, I don't use musl so all of this is really not a problem for me, just providing some feedback as upstream maintainer, in case it can be useful.

I don't have any interest in systemd + musl anymore either. I did an initial port as a proof of concept and sent patches to Khem off list... and was somewhat surprised when they showed up some time later in oe-core.

Note that there have been long discussions here previously about whether OE should claim to support systemd + musl. The effort to support it properly (including clarifying questions like this with upstream as you suggest) doesn't seem huge but so far no one seems to care enough about systemd + musl to do it. We rebase and tweak the patches but guidance to potential users should still be "use at your own risk".