Fix modules.builtin.alias handling
cover Fix modules.builtin.alias handling
This should fix 2 bugs reported last week:
- depmod -n failing. Like for other binary indexes we just skip now
- programs like lspci failing since they don't ignore errors from kmod_load_resources() - the new modules.builtin.alias.bin may not exist if depmod from kmod 27 wasn't executed for that kernel version. Also modules.builtin.modinfo (from where the former is extracted) is only available on recent kernel versions. So make this index optional for kmod_load_resources() - other parts of the code already assume it's optional
Lucas De Marchi (6):
gitignore: ignore .cache.mk when building modules
depmod: do not output .bin to stdout
libkmod: simplify lookup when builtin.modinfo.bin file is missing
libkmod: fix return error when opening index
libkmod: allow modules.alias.builtin to be optional
testsuite: add check for kmod_load_resources
libkmod/libkmod-index.c | 31 ++++++----- libkmod/libkmod-index.h | 4 +- libkmod/libkmod-module.c | 8 +-- libkmod/libkmod.c | 50 ++++++++++-------- testsuite/module-playground/.gitignore | 1 + .../lib/modules/5.6.0/modules.alias | 1 + .../lib/modules/5.6.0/modules.alias.bin | Bin 0 -> 12 bytes .../lib/modules/5.6.0/modules.builtin | 1 + .../lib/modules/5.6.0/modules.builtin.bin | Bin 0 -> 39 bytes .../lib/modules/5.6.0/modules.dep | 0 .../lib/modules/5.6.0/modules.dep.bin | Bin 0 -> 12 bytes .../lib/modules/5.6.0/modules.devname | 1 + .../lib/modules/5.6.0/modules.softdep | 1 + .../lib/modules/5.6.0/modules.symbols | 1 + .../lib/modules/5.6.0/modules.symbols.bin | Bin 0 -> 12 bytes testsuite/test-init.c | 31 +++++++++++ tools/depmod.c | 6 ++- 17 files changed, 95 insertions(+), 41 deletions(-) create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.alias create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.alias.bin create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.builtin create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.builtin.bin create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.dep create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.dep.bin create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.devname create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.softdep create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.symbols create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.symbols.bin
Signature · 1 line
2.25.1
PATCH 1/6 gitignore: ignore .cache.mk when building modules
--- testsuite/module-playground/.gitignore | 1 + 1 file changed, 1 insertion(+)
testsuite/module-playground/.gitignore +1 -0
@@ -2,6 +2,7 @@ *.ko !mod-simple-*.ko !cache/*.ko +.cache.mk *.mod.c .tmp_versions *.mod
Signature · 1 line
2.25.1
PATCH 2/6 depmod: do not output .bin to stdout
In reply to Lucas De Marchiindex_write() relies on fseek/ftell to manage the position to which we are write and thus needs the file stream to support it.
Right now when trying to write the index to stdout we fail with:
depmod: tools/depmod.c:416: index_write: Assertion `initial_offset >= 0' failed. Aborted (core dumped)
We have no interest in outputting our index to stdout, so just skip it like is done with other indexes.
While at it, add/remove some newlines to improve readability.
Fix: b866b2165ae6 ("Lookup aliases in the modules.builtin.modinfo")
--- tools/depmod.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
tools/depmod.c +5 -1
@@ -2408,8 +2408,10 @@ static int output_builtin_alias_bin(struct depmod *depmod, FILE *out)struct index_node *idx;struct kmod_list *l, *builtin =NULL ; - idx =index_create (); +if (out == stdout) +return 0 ; + idx =index_create ();if (idx ==NULL ) { ret = -ENOMEM ; goto fail;
@@ -2456,7 +2458,9 @@ static int output_builtin_alias_bin(struct depmod *depmod, FILE *out)if (count)index_write (idx, out); +index_destroy (idx); + fail:if (builtin)kmod_module_unref_list (builtin);
Signature · 1 line
2.25.1
PATCH 3/6 libkmod: simplify lookup when builtin.modinfo.bin file is missing
In reply to Lucas De MarchiWhen we try to lookup a module and builtin.modinfo.bin is missing, we would do the right thing because the caller was replacing the return code with 0 (and the list was not modified).
Make it simpler by allowing the caller to check and differentiate the errors between module not found and index not found.
--- libkmod/libkmod-module.c | 8 ++++---- libkmod/libkmod.c | 25 +++++++++++-------------- 2 files changed, 15 insertions(+), 18 deletions(-)
libkmod/libkmod-module.c +4 -4
@@ -577,13 +577,13 @@ KMOD_EXPORT int kmod_module_new_from_lookup(struct kmod_ctx *ctx,DBG (ctx,"lookup modules.builtin.modinfo %s\n" , alias); err =kmod_lookup_alias_from_kernel_builtin_file (ctx, alias, list); -CHECK_ERR_AND_FINISH (err, fail, list, finish); - -if (err ==0 ) { +if (err == -ENOSYS ) { +/* Optional index missing, try the old one */ DBG (ctx,"lookup modules.builtin %s\n" , alias); err =kmod_lookup_alias_from_builtin_file (ctx, alias, list); -CHECK_ERR_AND_FINISH (err, fail, list, finish); } +CHECK_ERR_AND_FINISH (err, fail, list, finish); + finish:DBG (ctx,"lookup %s=%d, list=%p\n" , alias, err, *list);
libkmod/libkmod.c +11 -14
@@ -528,20 +528,17 @@ int kmod_lookup_alias_from_kernel_builtin_file(struct kmod_ctx *ctx, struct kmod_list **list) {struct kmod_list *l; -int ret =kmod_lookup_alias_from_alias_bin (ctx, -KMOD_INDEX_MODULES_BUILTIN_ALIAS , - name, list); -if (ret >0 ) { - kmod_list_foreach(l, *list) { -struct kmod_module *mod = l->data; -kmod_module_set_builtin (mod, true); - } - }else if (ret == -ENOSYS ) { -/* -* If the system does not support this yet, then -* there is no need to return an error. -*/ - ret =0 ; +int ret; + +assert (*list ==NULL ); + + ret =kmod_lookup_alias_from_alias_bin (ctx, +KMOD_INDEX_MODULES_BUILTIN_ALIAS , + name, list); + + kmod_list_foreach(l, *list) { +struct kmod_module *mod = l->data; +kmod_module_set_builtin (mod, true); }return ret;
Signature · 1 line
2.25.1
PATCH 4/6 libkmod: fix return error when opening index
In reply to Lucas De MarchiWhen calling kmod_load_resources() we could end up getting a bogus return value -ENOMEM due to several other reasons, like the index not existing. Change index_mm_open() to propagate the failure reason so we can take actions on it or return to the caller.
--- libkmod/libkmod-index.c | 31 +++++++++++++++++++------------ libkmod/libkmod-index.h | 4 ++-- libkmod/libkmod.c | 16 ++++++++-------- 3 files changed, 29 insertions(+), 22 deletions(-)
libkmod/libkmod-index.c +19 -12
@@ -611,7 +611,7 @@ struct index_value *index_searchwild(struct index_file *in, const char *key)static const char _idx_empty_str[] ="" ;struct index_mm { -struct kmod_ctx *ctx; +const struct kmod_ctx *ctx;void *mm;uint32_t root_offset;size_t size;
@@ -739,10 +739,10 @@ static void index_mm_free_node(struct index_mm_node *node)free (node); } -struct index_mm *index_mm_open (struct kmod_ctx *ctx,const char *filename, -unsigned long long *stamp) +int index_mm_open (const struct kmod_ctx *ctx,const char *filename, +unsigned long long *stamp,struct index_mm **pidx) { -int fd; +int fd, err;struct stat st;struct index_mm *idx;struct {
@@ -752,28 +752,32 @@ struct index_mm *index_mm_open(struct kmod_ctx *ctx, const char *filename, } hdr;void *p; +assert (pidx !=NULL ); +DBG (ctx,"file=%s\n" , filename); idx =malloc (sizeof (*idx));if (idx ==NULL ) {ERR (ctx,"malloc: %m\n" ); -return NULL ; +return -ENOMEM ; }if ((fd =open (filename,O_RDONLY |O_CLOEXEC )) <0 ) {DBG (ctx,"open(%s, O_RDONLY|O_CLOEXEC): %m\n" , filename); + err = -errno; goto fail_open; } -if (fstat (fd, &st) <0 ) - goto fail_nommap; -if ((size_t ) st.st_size <sizeof (hdr)) +if (fstat (fd, &st) <0 || (size_t ) st.st_size <sizeof (hdr)) { + err = -EINVAL ; goto fail_nommap; + } -if ((idx->mm =mmap (NULL , st.st_size,PROT_READ ,MAP_PRIVATE , fd,0 )) - ==MAP_FAILED ) { + idx->mm =mmap (NULL , st.st_size,PROT_READ ,MAP_PRIVATE , fd,0 ); +if (idx->mm ==MAP_FAILED ) {ERR (ctx,"mmap(NULL, %" PRIu64", PROT_READ, %d, MAP_PRIVATE, 0): %m\n" , st.st_size, fd); + err = -errno; goto fail_nommap; }
@@ -785,12 +789,14 @@ struct index_mm *index_mm_open(struct kmod_ctx *ctx, const char *filename,if (hdr.magic !=INDEX_MAGIC ) {ERR (ctx,"magic check fail: %x instead of %x\n" , hdr.magic,INDEX_MAGIC ); + err = -EINVAL ; goto fail; }if (hdr.version >>16 !=INDEX_VERSION_MAJOR ) {ERR (ctx,"major version check fail: %u instead of %u\n" , hdr.version >>16 ,INDEX_VERSION_MAJOR ); + err = -EINVAL ; goto fail; }
@@ -800,8 +806,9 @@ struct index_mm *index_mm_open(struct kmod_ctx *ctx, const char *filename,close (fd); *stamp =stat_mstamp (&st); + *pidx = idx; -return idx; +return 0 ; fail:munmap (idx->mm, st.st_size);
@@ -809,7 +816,7 @@ fail_nommap:close (fd); fail_open:free (idx); -return NULL ; +return err; }void index_mm_close (struct index_mm *idx)
libkmod/libkmod-index.h +2 -2
@@ -40,8 +40,8 @@ void index_values_free(struct index_value *values);/* Implementation using mmap */ struct index_mm ; -struct index_mm *index_mm_open (struct kmod_ctx *ctx,const char *filename, -unsigned long long *stamp); +int index_mm_open (const struct kmod_ctx *ctx,const char *filename, +unsigned long long *stamp,struct index_mm **pidx);void index_mm_close (struct index_mm *index);char *index_mm_search (struct index_mm *idx,const char *key);struct index_value *index_mm_searchwild (struct index_mm *idx,const char *key);
libkmod/libkmod.c +8 -8
@@ -856,6 +856,7 @@ KMOD_EXPORT int kmod_validate_resources(struct kmod_ctx *ctx) KMOD_EXPORT int kmod_load_resources(struct kmod_ctx *ctx) {size_t i; +int ret;if (ctx ==NULL )return -ENOENT ;
@@ -871,17 +872,16 @@ KMOD_EXPORT int kmod_load_resources(struct kmod_ctx *ctx)snprintf (path,sizeof (path),"%s/%s.bin" , ctx->dirname, index_files[i].fn); - ctx->indexes[i] =index_mm_open (ctx, path, - &ctx->indexes_stamp[i]); -if (ctx->indexes[i] ==NULL ) - goto fail; + ret =index_mm_open (ctx, path, &ctx->indexes_stamp[i], + &ctx->indexes[i]); +if (ret) +break ; } -return 0 ; +if (ret) +kmod_unload_resources (ctx); -fail: -kmod_unload_resources (ctx); -return -ENOMEM ; +return ret; }/**
Signature · 1 line
2.25.1
PATCH 5/6 libkmod: allow modules.alias.builtin to be optional
In reply to Lucas De Marchi--- libkmod/libkmod.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)
libkmod/libkmod.c +12 -3
@@ -855,8 +855,8 @@ KMOD_EXPORT int kmod_validate_resources(struct kmod_ctx *ctx)*/ KMOD_EXPORT int kmod_load_resources(struct kmod_ctx *ctx) { +int ret =0 ;size_t i; -int ret;if (ctx ==NULL )return -ENOENT ;
@@ -874,8 +874,17 @@ KMOD_EXPORT int kmod_load_resources(struct kmod_ctx *ctx) index_files[i].fn); ret =index_mm_open (ctx, path, &ctx->indexes_stamp[i], &ctx->indexes[i]); -if (ret) -break ; + +/* +* modules.builtin.alias are considered optional since it's +* recently added and older installations may not have it; +* we allow failing for any reason +*/ +if (ret) { +if (i !=KMOD_INDEX_MODULES_BUILTIN_ALIAS ) +break ; + ret =0 ; + } }if (ret)
Signature · 1 line
2.25.1
PATCH 6/6 testsuite: add check for kmod_load_resources
In reply to Lucas De MarchiMake sure we can call kmod_load_resources when we do have all the mandatory indexes in place.
--- .../lib/modules/5.6.0/modules.alias | 1 + .../lib/modules/5.6.0/modules.alias.bin | Bin 0 -> 12 bytes .../lib/modules/5.6.0/modules.builtin | 1 + .../lib/modules/5.6.0/modules.builtin.bin | Bin 0 -> 39 bytes .../lib/modules/5.6.0/modules.dep | 0 .../lib/modules/5.6.0/modules.dep.bin | Bin 0 -> 12 bytes .../lib/modules/5.6.0/modules.devname | 1 + .../lib/modules/5.6.0/modules.softdep | 1 + .../lib/modules/5.6.0/modules.symbols | 1 + .../lib/modules/5.6.0/modules.symbols.bin | Bin 0 -> 12 bytes testsuite/test-init.c | 31 ++++++++++++++++++ 11 files changed, 36 insertions(+) create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.alias create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.alias.bin create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.builtin create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.builtin.bin create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.dep create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.dep.bin create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.devname create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.softdep create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.symbols create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.symbols.bin
testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.alias new file +1 -0
@@ -0,0 +1 @@ +# Aliases extracted from modules themselves.
testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.alias.bin new file binary +0 -0
testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.builtin new file +1 -0
@@ -0,0 +1 @@ +kernel/fake_builtin.ko
testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.builtin.bin new file binary +0 -0
testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.dep new file +0 -0
testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.dep.bin new file binary +0 -0
testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.devname new file +1 -0
@@ -0,0 +1 @@ +# Device nodes to trigger on-demand module loading.
testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.softdep new file +1 -0
@@ -0,0 +1 @@ +# Soft dependencies extracted from modules themselves.
testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.symbols new file +1 -0
@@ -0,0 +1 @@ +# Aliases for symbols, used by symbol_request().
testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.symbols.bin new file binary +0 -0
testsuite/test-init.c +31 -0
@@ -29,6 +29,37 @@#include "testsuite.h" +static noreturnint test_load_resources (const struct test *t) +{ +struct kmod_ctx *ctx; +const char *null_config =NULL ; +int err; + + ctx =kmod_new (NULL , &null_config); +if (ctx ==NULL ) +exit (EXIT_FAILURE ); + +kmod_set_log_priority (ctx,7 ); + + err =kmod_load_resources (ctx); +if (err !=0 ) { +ERR ("could not load libkmod resources: %s\n" ,strerror (-err)); +exit (EXIT_FAILURE ); + } + +kmod_unref (ctx); + +exit (EXIT_SUCCESS ); +} +DEFINE_TEST(test_load_resources, + .description = "test if kmod_load_resources works", + .config = { + [TC_ROOTFS] = TESTSUITE_ROOTFS "test-init-load-resources/", + [TC_UNAME_R] = "5.6.0", + }, + .need_spawn = true); + +static noreturnint test_initlib (const struct test *t) {struct kmod_ctx *ctx;
Signature · 1 line
2.25.1
All patches here applied.
Lucas De Marchi
Quoted 53 lines · Lucas De Marchi
This should fix 2 bugs reported last week:
-
depmod -nfailing. Like for other binary indexes we just skip now- programs like lspci failing since they don't ignore errors from
kmod_load_resources()- the new modules.builtin.alias.bin may not exist if depmod from kmod 27 wasn't executed for that kernel version. Also modules.builtin.modinfo (from where the former is extracted) is only available on recent kernel versions. So make this index optional forkmod_load_resources()- other parts of the code already assume it's optionalLucas De Marchi (6):
gitignore: ignore .cache.mk when building modules
depmod: do not output .bin to stdout
libkmod: simplify lookup when builtin.modinfo.bin file is missing
libkmod: fix return error when opening index
libkmod: allow modules.alias.builtin to be optional
testsuite: add check forkmod_load_resourceslibkmod/libkmod-index.c | 31 ++++++----- libkmod/libkmod-index.h | 4 +- libkmod/libkmod-module.c | 8 +-- libkmod/libkmod.c | 50 ++++++++++-------- testsuite/module-playground/.gitignore | 1 + .../lib/modules/5.6.0/modules.alias | 1 + .../lib/modules/5.6.0/modules.alias.bin | Bin 0 -> 12 bytes .../lib/modules/5.6.0/modules.builtin | 1 + .../lib/modules/5.6.0/modules.builtin.bin | Bin 0 -> 39 bytes .../lib/modules/5.6.0/modules.dep | 0 .../lib/modules/5.6.0/modules.dep.bin | Bin 0 -> 12 bytes .../lib/modules/5.6.0/modules.devname | 1 + .../lib/modules/5.6.0/modules.softdep | 1 + .../lib/modules/5.6.0/modules.symbols | 1 + .../lib/modules/5.6.0/modules.symbols.bin | Bin 0 -> 12 bytes testsuite/test-init.c | 31 +++++++++++ tools/depmod.c | 6 ++- 17 files changed, 95 insertions(+), 41 deletions(-)create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.alias create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.alias.bin create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.builtin create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.builtin.bin create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.dep create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.dep.bin create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.devname create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.softdep create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.symbols create mode 100644 testsuite/rootfs-pristine/test-init-load-resources/lib/modules/5.6.0/modules.symbols.bin
Signature · 1 line
2.25.1