Skip to contentSkip to conversation

kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h)

PATCH Superseded Masahiro Yamada 4 participants 24 messages May 11, 2022 – May 13, 2022

Subsystems (6)

v5 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v5 cover kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h)

This is the third batch of cleanups in this development cycle.

Major changes in v5:
- Fix build errors
- More cleanups

Major changes in v4:
- Move static EXPORT_SYMBOL check to a script
- Some refactoring

Major changes in v3:

- Generate symbol CRCs as C code, and remove CONFIG_MODULE_REL_CRCS.

Major changes in v2:

- V1 did not work with CONFIG_MODULE_REL_CRCS.
I fixed this for v2.

- Reflect some review comments in v1

- Refactor the code more

- Avoid too long argument error

Masahiro Yamada (12):
modpost: split the section mismatch checks into section-check.c
modpost: add sym_find_with_module() helper
modpost: extract symbol versions from *.cmd files
kbuild: link symbol CRCs at final link, removing
CONFIG_MODULE_REL_CRCS
kbuild: stop merging *.symversions
genksyms: adjust the output format to modpost
kbuild: do not create *.prelink.o for Clang LTO or IBT
kbuild: check static EXPORT_SYMBOL* by script instead of modpost
kbuild: make built-in.a rule robust against too long argument error
kbuild: make *.mod rule robust against too long argument error
kbuild: add cmd_and_savecmd macro
kbuild: rebuild multi-object modules when objtool is updated

 arch/m68k/include/asm/Kbuild    |    1 +
 arch/m68k/include/asm/export.h  |    2 -
 arch/powerpc/Kconfig            |    1 -
 arch/s390/Kconfig               |    1 -
 arch/um/Kconfig                 |    1 -
 include/asm-generic/export.h    |   22 +-
 include/linux/export-internal.h |   16 +
 include/linux/export.h          |   30 +-
 init/Kconfig                    |    4 -
 kernel/module.c                 |   10 +-
 scripts/Kbuild.include          |   10 +-
 scripts/Makefile.build          |  134 +--
 scripts/Makefile.lib            |    7 -
 scripts/Makefile.modfinal       |    5 +-
 scripts/Makefile.modpost        |    9 +-
 scripts/check-local-export      |   64 ++
 scripts/genksyms/genksyms.c     |   18 +-
 scripts/link-vmlinux.sh         |   33 +-
 scripts/mod/Makefile            |    4 +-
 scripts/mod/modpost.c           | 1395 +++----------------------------
 scripts/mod/modpost.h           |   34 +-
 scripts/mod/section-check.c     | 1222 +++++++++++++++++++++++++++
 22 files changed, 1541 insertions(+), 1482 deletions(-)
 delete mode 100644 arch/m68k/include/asm/export.h
 create mode 100644 include/linux/export-internal.h
 create mode 100755 scripts/check-local-export
 create mode 100644 scripts/mod/section-check.c
Signature · 1 line
2.32.0
#1 lore raw

PATCH v5 1/12 modpost: split the section mismatch checks into section-check.c

modpost.c is too big, and the half of the code is for section checks. Split it.

Also, move some related typedefs and macros from modpost.h to section-check.c

Copy Sam's Copyright there. Commit b39927cf4cc5 ("kbuild: check for section mismatch during modpost stage") is the initial work for the section checks.

---

Changes in v5:
- Fix the build error (spotted by Nathan)
- Do not do codying style changes (Suggested by Nick)
- Copy Sam's copyright to the new file

Changes in v4:
- New patch

 scripts/mod/Makefile        |    4 +-
 scripts/mod/modpost.c       | 1202 +---------------------------------
 scripts/mod/modpost.h       |   34 +-
 scripts/mod/section-check.c | 1222 +++++++++++++++++++++++++++++++++++
 4 files changed, 1241 insertions(+), 1221 deletions(-)
 create mode 100644 scripts/mod/section-check.c
scripts/mod/Makefile +2 -2
index c9e38ad937fd..6e548eb6317d 100644
@@ -5,7 +5,7 @@ CFLAGS_REMOVE_empty.o += $(CC_FLAGS_LTO)
 hostprogs-always-y	+= modpost mk_elfconfig
 always-y		+= empty.o
 
-modpost-objs	:= modpost.o file2alias.o sumversion.o
+modpost-objs	:= modpost.o section-check.o file2alias.o sumversion.o
 
 devicetable-offsets-file := devicetable-offsets.h
 
@@ -16,7 +16,7 @@ targets += $(devicetable-offsets-file) devicetable-offsets.s
 
 # dependencies on generated files need to be listed explicitly
 
-$(obj)/modpost.o $(obj)/file2alias.o $(obj)/sumversion.o: $(obj)/elfconfig.h
+$(addprefix $(obj)/, $(modpost-objs)): $(obj)/elfconfig.h
 $(obj)/file2alias.o: $(obj)/$(devicetable-offsets-file)
 
 quiet_cmd_elfconfig = MKELF   $@
scripts/mod/modpost.c +5 -1197

1,235 lines not on this page · raw message

scripts/mod/modpost.h +12 -22
index d9daeff07b83..ffa4512e992f 100644
@@ -52,28 +52,6 @@
 #define ELF_R_TYPE  ELF64_R_TYPE
 #endif
 
-/* The 64-bit MIPS ELF ABI uses an unusual reloc format. */
-typedef struct
-{
-	Elf32_Word    r_sym;	/* Symbol index */
-	unsigned char r_ssym;	/* Special symbol for 2nd relocation */
-	unsigned char r_type3;	/* 3rd relocation type */
-	unsigned char r_type2;	/* 2nd relocation type */
-	unsigned char r_type1;	/* 1st relocation type */
-} _Elf64_Mips_R_Info;
-
-typedef union
-{
-	Elf64_Xword		r_info_number;
-	_Elf64_Mips_R_Info	r_info_fields;
-} _Elf64_Mips_R_Info_union;
-
-#define ELF64_MIPS_R_SYM(i) \
-  ((__extension__ (_Elf64_Mips_R_Info_union)(i)).r_info_fields.r_sym)
-
-#define ELF64_MIPS_R_TYPE(i) \
-  ((__extension__ (_Elf64_Mips_R_Info_union)(i)).r_info_fields.r_type1)
-
 #if KERNEL_ELFDATA != HOST_ELFDATA
 
 static inline void __endian(const void *src, void *dest, unsigned int size)
@@ -96,6 +74,7 @@ static inline void __endian(const void *src, void *dest, unsigned int size)
 
 #endif
 
+#define strstarts(str, prefix) (strncmp(str, prefix, strlen(prefix)) == 0)
 #define NOFAIL(ptr)   do_nofail((ptr), #ptr)
 void *do_nofail(void *ptr, const char *expr);
 
@@ -173,6 +152,10 @@ static inline unsigned int get_secindex(const struct elf_info *info,
 	return info->symtab_shndx_start[sym - info->symtab_start];
 }
 
+/* section-check.c */
+void check_sec_ref(struct module *mod, const char *modname,
+		   struct elf_info *elf);
+
 /* file2alias.c */
 void handle_moddevtable(struct module *mod, struct elf_info *info,
 			Elf_Sym *sym, const char *symname);
@@ -182,6 +165,13 @@ void add_moddevtable(struct buffer *buf, struct module *mod);
 void get_src_version(const char *modname, char sum[], unsigned sumlen);
 
 /* from modpost.c */
+extern int sec_mismatch_count;
+
+void *sym_get_data_by_offset(const struct elf_info *info,
+			     unsigned int secindex, unsigned long offset);
+const char *sech_name(const struct elf_info *info, Elf_Shdr *sechdr);
+const char *sec_name(const struct elf_info *info, int secindex);
+
 char *read_text_file(const char *filename);
 char *get_line(char **stringp);
 
scripts/mod/section-check.c new file +1222 -0

1,222 lines not on this page · raw message

Signature · 1 line
2.32.0
Reviewed-by on v5 1/12
#2 lore raw

Re: [PATCH v5 01/12] modpost: split the section mismatch checks into section-check.c

Quoted 4 lines · Masahiro Yamada

modpost.c is too big, and the half of the code is for section checks. Split it.

Quoted 8 lines

Also, move some related typedefs and macros from modpost.h to section-check.c

Copy Sam's Copyright there. Commit b39927cf4cc5 ("kbuild: check for section mismatch during modpost stage") is the initial work for the section checks.

Sorry again for my "brain fart" in reviewing the global var in v4. :( I think my blood sugar was getting low or something...

Quoted 5 lines

---

Changes in v5:
- Fix the build error (spotted by Nathan)
- Do not do codying style changes (Suggested by Nick)

I'm surprised the style changes didn't show up in v5 as a later patch in the series. I wasn't against them in general, just thought it would be easier to review if they were distinct commits.

Signature · 2 lines
Thanks,
~Nick Desaulniers
#3 lore raw

Re: [PATCH v5 01/12] modpost: split the section mismatch checks into section-check.c

Quoted 13 lines · Nick Desaulniers
Quoted 11 lines · Masahiro Yamada

modpost.c is too big, and the half of the code is for section checks. Split it.

Also, move some related typedefs and macros from modpost.h to section-check.c

Copy Sam's Copyright there. Commit b39927cf4cc5 ("kbuild: check for section mismatch during modpost stage") is the initial work for the section checks.

Quoted 6 lines
Quoted 1 line

Sorry again for my "brain fart" in reviewing the global var in v4. :( I think my blood sugar was getting low or something...

Never mind.
Thanks for the close review all the time!

Quoted 10 lines
Quoted 6 lines

---

Changes in v5:
- Fix the build error (spotted by Nathan)
- Do not do codying style changes (Suggested by Nick)

I'm surprised the style changes didn't show up in v5 as a later patch in the series. I wasn't against them in general, just thought it would be easier to review if they were distinct commits.

I just postponed the coding style fixes because they are not essential for my main goal in this patch set.

A lot of old code exists in modpost.
We can improve it later.

Signature · 2 lines
Best Regards
Masahiro Yamada
#4 lore raw

Re: [PATCH v5 01/12] modpost: split the section mismatch checks into section-check.c

In reply to Masahiro Yamada
Quoted 78 lines · Masahiro Yamada
(quoted diff) +0 -75
-/* sections that we do not want to do full section mismatch check on */
-static const char *const section_white_list[] =
-{
-       ".comment*",
-       ".debug*",
-       ".cranges",             /* sh64 */
-       ".zdebug*",             /* Compressed debug sections. */
-       ".GCC.command.line",    /* record-gcc-switches */
-       ".mdebug*",        /* alpha, score, mips etc. */
-       ".pdr",            /* alpha, score, mips etc. */
-       ".stab*",
-       ".note*",
-       ".got*",
-       ".toc*",
-       ".xt.prop",                              /* xtensa */
-       ".xt.lit",         /* xtensa */
-       ".arcextmap*",                  /* arc */
-       ".gnu.linkonce.arcext*",        /* arc : modules */
-       ".cmem*",                       /* EZchip */
-       ".fmt_slot*",                   /* EZchip */
-       ".gnu.lto*",
-       ".discard.*",
-       NULL
-};
 
 
-/**
- * Whitelist to allow certain references to pass with no warning.
- *
- * Pattern 1:
- *   If a module parameter is declared __initdata and permissions=0
- *   then this is legal despite the warning generated.
- *   We cannot see value of permissions here, so just ignore
- *   this pattern.
- *   The pattern is identified by:
- *   tosec   = .init.data
- *   fromsec = .data*
- *   atsym   =__param*
- *
- * Pattern 1a:
- *   module_param_call() ops can refer to __init set function if permissions=0
- *   The pattern is identified by:
- *   tosec   = .init.text
- *   fromsec = .data*
- *   atsym   = __param_ops_*
- *
- * Pattern 2:
- *   Many drivers utilise a *driver container with references to
- *   add, remove, probe functions etc.
- *   the pattern is identified by:
- *   tosec   = init or exit section
- *   fromsec = data section
- *   atsym = *driver, *_template, *_sht, *_ops, *_probe,
- *           *probe_one, *_console, *_timer
- *
- * Pattern 3:
- *   Whitelist all references from .head.text to any init section
- *
- * Pattern 4:
- *   Some symbols belong to init section but still it is ok to reference
- *   these from non-init sections as these symbols don't have any memory
- *   allocated for them and symbol address and value are same. So even
- *   if init section is freed, its ok to reference those symbols.
- *   For ex. symbols marking the init section boundaries.
- *   This pattern is identified by
- *   refsymname = __init_begin, _sinittext, _einittext
- *
- * Pattern 5:
- *   GCC may optimize static inlines when fed constant arg(s) resulting
- *   in functions like cpumask_empty() -- generating an associated symbol
- *   cpumask_empty.constprop.3 that appears in the audit.  If the const that
- *   is passed in comes from __init, like say nmi_ipi_mask, we get a
- *   meaningless section warning.  May need to add isra symbols too...
- *   This pattern is identified by
- *   tosec   = init section
- *   fromsec = text section
- *   refsymname = *.constprop.*

Review

Quoted 10 lines
(quoted diff) +0 -10
- *
- * Pattern 6:
- *   Hide section mismatch warnings for ELF local symbols.  The goal
- *   is to eliminate false positive modpost warnings caused by
- *   compiler-generated ELF local symbol names such as ".LANCHOR1".
- *   Autogenerated symbol names bypass modpost's "Pattern 2"
- *   whitelisting, which relies on pattern-matching against symbol
- *   names to work.  (One situation where gcc can autogenerate ELF
- *   local symbols is when "-fsection-anchors" is used.)
- **/

Losing the ability to git blame (from the top level) the above lines does cause me grief and mental anguish though. It's not gone, just buried a bit deeper.

Signature · 2 lines
Thanks,
~Nick Desaulniers
#5 lore raw

Re: [PATCH v5 01/12] modpost: split the section mismatch checks into section-check.c

Quoted 91 lines · Nick Desaulniers
Quoted 78 lines · Masahiro Yamada
(quoted diff) +0 -75
-/* sections that we do not want to do full section mismatch check on */
-static const char *const section_white_list[] =
-{
-       ".comment*",
-       ".debug*",
-       ".cranges",             /* sh64 */
-       ".zdebug*",             /* Compressed debug sections. */
-       ".GCC.command.line",    /* record-gcc-switches */
-       ".mdebug*",        /* alpha, score, mips etc. */
-       ".pdr",            /* alpha, score, mips etc. */
-       ".stab*",
-       ".note*",
-       ".got*",
-       ".toc*",
-       ".xt.prop",                              /* xtensa */
-       ".xt.lit",         /* xtensa */
-       ".arcextmap*",                  /* arc */
-       ".gnu.linkonce.arcext*",        /* arc : modules */
-       ".cmem*",                       /* EZchip */
-       ".fmt_slot*",                   /* EZchip */
-       ".gnu.lto*",
-       ".discard.*",
-       NULL
-};
 
 
-/**
- * Whitelist to allow certain references to pass with no warning.
- *
- * Pattern 1:
- *   If a module parameter is declared __initdata and permissions=0
- *   then this is legal despite the warning generated.
- *   We cannot see value of permissions here, so just ignore
- *   this pattern.
- *   The pattern is identified by:
- *   tosec   = .init.data
- *   fromsec = .data*
- *   atsym   =__param*
- *
- * Pattern 1a:
- *   module_param_call() ops can refer to __init set function if permissions=0
- *   The pattern is identified by:
- *   tosec   = .init.text
- *   fromsec = .data*
- *   atsym   = __param_ops_*
- *
- * Pattern 2:
- *   Many drivers utilise a *driver container with references to
- *   add, remove, probe functions etc.
- *   the pattern is identified by:
- *   tosec   = init or exit section
- *   fromsec = data section
- *   atsym = *driver, *_template, *_sht, *_ops, *_probe,
- *           *probe_one, *_console, *_timer
- *
- * Pattern 3:
- *   Whitelist all references from .head.text to any init section
- *
- * Pattern 4:
- *   Some symbols belong to init section but still it is ok to reference
- *   these from non-init sections as these symbols don't have any memory
- *   allocated for them and symbol address and value are same. So even
- *   if init section is freed, its ok to reference those symbols.
- *   For ex. symbols marking the init section boundaries.
- *   This pattern is identified by
- *   refsymname = __init_begin, _sinittext, _einittext
- *
- * Pattern 5:
- *   GCC may optimize static inlines when fed constant arg(s) resulting
- *   in functions like cpumask_empty() -- generating an associated symbol
- *   cpumask_empty.constprop.3 that appears in the audit.  If the const that
- *   is passed in comes from __init, like say nmi_ipi_mask, we get a
- *   meaningless section warning.  May need to add isra symbols too...
- *   This pattern is identified by
- *   tosec   = init section
- *   fromsec = text section
- *   refsymname = *.constprop.*

Review

Quoted 10 lines
(quoted diff) +0 -10
- *
- * Pattern 6:
- *   Hide section mismatch warnings for ELF local symbols.  The goal
- *   is to eliminate false positive modpost warnings caused by
- *   compiler-generated ELF local symbol names such as ".LANCHOR1".
- *   Autogenerated symbol names bypass modpost's "Pattern 2"
- *   whitelisting, which relies on pattern-matching against symbol
- *   names to work.  (One situation where gcc can autogenerate ELF
- *   local symbols is when "-fsection-anchors" is used.)
- **/
Quoted 3 lines

Losing the ability to git blame (from the top level) the above lines does cause me grief and mental anguish though. It's not gone, just buried a bit deeper.

According to 'man git-blame', -C option is needed
to follow the code move between files.

git blame -C scripts/mod/section-check.c

showed me the origin commit of each line.

Quoted 3 lines

--
Thanks,
~Nick Desaulniers

--
Best Regards
Masahiro Yamada

#6 lore raw

PATCH v5 2/12 modpost: add sym_find_with_module() helper

In reply to Masahiro Yamada

find_symbol() returns the first symbol found in the hash table. This table is global, so it may return a symbol from an unexpected module.

There is a case where we want to search for a symbol with a given name in a specified module.

Add sym_find_with_module(), which receives the module pointer as the second argument. It is equivalent to find_module() if NULL is passed as the module pointer.

---

(no changes since v4)

Changes in v4:
- Only takes the new helper from

    https://patchwork.kernel.org/project/linux-kbuild/patch/[email protected]/

Changes in v2:
- Rename the new func to sym_find_with_module()

 scripts/mod/modpost.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
scripts/mod/modpost.c +7 -2
index e7e2c70a98f5..fc5db1f73cf1 100644
@@ -266,7 +266,7 @@ static void sym_add_unresolved(const char *name, struct module *mod, bool weak)
 	list_add_tail(&sym->list, &mod->unresolved_symbols);
 }
 
-static struct symbol *find_symbol(const char *name)
+static struct symbol *sym_find_with_module(const char *name, struct module *mod)
 {
 	struct symbol *s;
 
@@ -275,12 +275,17 @@ static struct symbol *find_symbol(const char *name)
 		name++;
 
 	for (s = symbolhash[tdb_hash(name) % SYMBOL_HASH_SIZE]; s; s = s->next) {
-		if (strcmp(s->name, name) == 0)
+		if (strcmp(s->name, name) == 0 && (!mod || s->module == mod))
 			return s;
 	}
 	return NULL;
 }
 
+static struct symbol *find_symbol(const char *name)
+{
+	return sym_find_with_module(name, NULL);
+}
+
 struct namespace_list {
 	struct list_head list;
 	char namespace[];
Signature · 1 line
2.32.0
#7 lore raw

PATCH v5 3/12 modpost: extract symbol versions from *.cmd files

In reply to Masahiro Yamada

Currently, CONFIG_MODVERSIONS needs extra link to embed the symbol versions into ELF objects. Then, modpost extracts the version CRCs from them.

The following figures show how it currently works, and how I am trying to change it.

Current implementation
======================

                                                           |----------|
                 embed CRC      -------------------------->| final    |
       $(CC)       $(LD)       /  |---------|              | link for |
       -----> *.o -------> *.o -->| modpost |              | vmlinux  |
      /              /            |         |-- *.mod.c -->| or       |
     / genksyms     /             |---------|              | module   |
  *.c ------> *.symversions                                |----------|

Genksyms outputs the calculated CRCs in the form of linker script (*.symversions), which is used by $(LD) to update the object.

If CONFIG_LTO_CLANG=y, the build process is much more complex. Embedding the CRCs is postponed until the LLVM bitcode is converted into ELF, creating another intermediate *.prelink.o.

However, this complexity is unneeded. There is no reason why we must embed version CRCs in objects so early.

There is final link stage for vmlinux (scripts/link-vmlinux.sh) and modules (scripts/Makefile.modfinal). We can link CRCs at the very last moment.

New implementation
==================

                                                           |----------|
                   --------------------------------------->| final    |
       $(CC)      /    |---------|                         | link for |
       -----> *.o ---->|         |                         | vmlinux  |
      /                | modpost |--- .vmlinux.export.c -->| or       |
     / genksyms        |         |--- *.mod.c ------------>| module   |
  *.c ------> *.cmd -->|---------|                         |----------|

Pass the symbol versions to modpost as separate text data, which are available in *.cmd files.

This commit changes modpost to extract CRCs from *.cmd files instead of from ELF objects.

---

(no changes since v2)

Changes in v2:
- Simplify the implementation (parse .cmd files after ELF)

 scripts/mod/modpost.c | 177 ++++++++++++++++++++++++++++++------------
 1 file changed, 129 insertions(+), 48 deletions(-)
scripts/mod/modpost.c +129 -48
index fc5db1f73cf1..e0f9c02d9f83 100644
@@ -381,19 +381,10 @@ static struct symbol *sym_add_exported(const char *name, struct module *mod,
 	return s;
 }
 
-static void sym_set_crc(const char *name, unsigned int crc)
+static void sym_set_crc(struct symbol *sym, unsigned int crc)
 {
-	struct symbol *s = find_symbol(name);
-
-	/*
-	 * Ignore stand-alone __crc_*, which might be auto-generated symbols
-	 * such as __*_veneer in ARM ELF.
-	 */
-	if (!s)
-		return;
-
-	s->crc = crc;
-	s->crc_valid = true;
+	sym->crc = crc;
+	sym->crc_valid = true;
 }
 
 static void *grab_file(const char *filename, size_t *size)
@@ -616,33 +607,6 @@ static int ignore_undef_symbol(struct elf_info *info, const char *symname)
 	return 0;
 }
 
-static void handle_modversion(const struct module *mod,
-			      const struct elf_info *info,
-			      const Elf_Sym *sym, const char *symname)
-{
-	unsigned int crc;
-
-	if (sym->st_shndx == SHN_UNDEF) {
-		warn("EXPORT symbol \"%s\" [%s%s] version generation failed, symbol will not be versioned.\n"
-		     "Is \"%s\" prototyped in <asm/asm-prototypes.h>?\n",
-		     symname, mod->name, mod->is_vmlinux ? "" : ".ko",
-		     symname);
-
-		return;
-	}
-
-	if (sym->st_shndx == SHN_ABS) {
-		crc = sym->st_value;
-	} else {
-		unsigned int *crcp;
-
-		/* symbol points to the CRC in the ELF object */
-		crcp = sym_get_data(info, sym);
-		crc = TO_NATIVE(*crcp);
-	}
-	sym_set_crc(symname, crc);
-}
-
 static void handle_symbol(struct module *mod, struct elf_info *info,
 			  const Elf_Sym *sym, const char *symname)
 {
@@ -760,6 +724,102 @@ static char *remove_dot(char *s)
 	return s;
 }
 
+/*
+ * The CRCs are recorded in .*.cmd files in the form of:
+ * #SYMVER <name> <crc>
+ */
+static void extract_crcs_for_object(const char *object, struct module *mod)
+{
+	char cmd_file[PATH_MAX];
+	char *buf, *p;
+	const char *base;
+	int dirlen, ret;
+
+	base = strrchr(object, '/');
+	if (base) {
+		base++;
+		dirlen = base - object;
+	} else {
+		dirlen = 0;
+		base = object;
+	}
+
+	ret = snprintf(cmd_file, sizeof(cmd_file), "%.*s.%s.cmd",
+		       dirlen, object, base);
+	if (ret >= sizeof(cmd_file)) {
+		error("%s: too long path was truncated\n", cmd_file);
+		return;
+	}
+
+	buf = read_text_file(cmd_file);
+	p = buf;
+
+	while ((p = strstr(p, "\n#SYMVER "))) {
+		char *name;
+		size_t namelen;
+		unsigned int crc;
+		struct symbol *sym;
+
+		name = p + strlen("\n#SYMVER ");
+
+		p = strchr(name, ' ');
+		if (!p)
+			break;
+
+		namelen = p - name;
+		p++;
+
+		if (!isdigit(*p))
+			continue;	/* skip this line */
+
+		crc = strtol(p, &p, 0);
+		if (*p != '\n')
+			continue;	/* skip this line */
+
+		name[namelen] = '\0';
+
+		sym = sym_find_with_module(name, mod);
+		if (!sym) {
+			warn("Skip the version for unexported symbol \"%s\" [%s%s]\n",
+			     name, mod->name, mod->is_vmlinux ? "" : ".ko");
+			continue;
+		}
+		sym_set_crc(sym, crc);
+	}
+
+	free(buf);
+}
+
+/*
+ * The symbol versions (CRC) are recorded in the .*.cmd files.
+ * Parse them to retrieve CRCs for the current module.
+ */
+static void mod_set_crcs(struct module *mod)
+{
+	char objlist[PATH_MAX];
+	char *buf, *p, *obj;
+	int ret;
+
+	if (mod->is_vmlinux) {
+		strcpy(objlist, ".vmlinux.objs");
+	} else {
+		/* objects for a module are listed in the *.mod file. */
+		ret = snprintf(objlist, sizeof(objlist), "%s.mod", mod->name);
+		if (ret >= sizeof(objlist)) {
+			error("%s: too long path was truncated\n", objlist);
+			return;
+		}
+	}
+
+	buf = read_text_file(objlist);
+	p = buf;
+
+	while ((obj = strsep(&p, "\n")) && obj[0])
+		extract_crcs_for_object(obj, mod);
+
+	free(buf);
+}
+
 static void read_symbols(const char *modname)
 {
 	const char *symname;
@@ -820,9 +880,6 @@ static void read_symbols(const char *modname)
 		if (strstarts(symname, "__kstrtabns_"))
 			sym_update_namespace(symname + strlen("__kstrtabns_"),
 					     sym_get_data(&info, sym));
-		if (strstarts(symname, "__crc_"))
-			handle_modversion(mod, &info, sym,
-					  symname + strlen("__crc_"));
 	}
 
 	// check for static EXPORT_SYMBOL_* functions && global vars
@@ -850,12 +907,17 @@ static void read_symbols(const char *modname)
 
 	parse_elf_finish(&info);
 
-	/* Our trick to get versioning for module struct etc. - it's
-	 * never passed as an argument to an exported function, so
-	 * the automatic versioning doesn't pick it up, but it's really
-	 * important anyhow */
-	if (modversions)
+	if (modversions) {
+		/*
+		 * Our trick to get versioning for module struct etc. - it's
+		 * never passed as an argument to an exported function, so
+		 * the automatic versioning doesn't pick it up, but it's really
+		 * important anyhow.
+		 */
 		sym_add_unresolved("module_layout", mod, false);
+
+		mod_set_crcs(mod);
+	}
 }
 
 static void read_symbols_from_files(const char *filename)
@@ -1012,6 +1074,23 @@ static void add_header(struct buffer *b, struct module *mod)
 		buf_printf(b, "\nMODULE_INFO(staging, \"Y\");\n");
 }
 
+static void check_symversions(struct module *mod)
+{
+	struct symbol *sym;
+
+	if (!modversions)
+		return;
+
+	list_for_each_entry(sym, &mod->exported_symbols, list) {
+		if (!sym->crc_valid) {
+			warn("EXPORT symbol \"%s\" [%s%s] version generation failed, symbol will not be versioned.\n"
+			     "Is \"%s\" prototyped in <asm/asm-prototypes.h>?\n",
+			     sym->name, mod->name, mod->is_vmlinux ? "" : ".ko",
+			     sym->name);
+		}
+	}
+}
+
 /**
  * Record CRCs for unresolved symbols
  **/
@@ -1227,7 +1306,7 @@ static void read_dump(const char *fname)
 		}
 		s = sym_add_exported(symname, mod, gpl_only);
 		s->is_static = false;
-		sym_set_crc(symname, crc);
+		sym_set_crc(s, crc);
 		sym_update_namespace(symname, namespace);
 	}
 	free(buf);
@@ -1353,6 +1432,8 @@ int main(int argc, char **argv)
 		if (mod->from_dump)
 			continue;
 
+		check_symversions(mod);
+
 		if (!mod->is_vmlinux)
 			write_mod_c_file(mod);
 	}
Signature · 1 line
2.32.0
#8 lore raw

Re: [PATCH v5 03/12] modpost: extract symbol versions from *.cmd files

Quoted 114 lines · Masahiro Yamada

Currently, CONFIG_MODVERSIONS needs extra link to embed the symbol versions into ELF objects. Then, modpost extracts the version CRCs from them.

The following figures show how it currently works, and how I am trying to change it.

Current implementation
======================

                                                           |----------|
                 embed CRC      -------------------------->| final    |
       $(CC)       $(LD)       /  |---------|              | link for |
       -----> *.o -------> *.o -->| modpost |              | vmlinux  |
      /              /            |         |-- *.mod.c -->| or       |
     / genksyms     /             |---------|              | module   |
  *.c ------> *.symversions                                |----------|

Genksyms outputs the calculated CRCs in the form of linker script (*.symversions), which is used by $(LD) to update the object.

If CONFIG_LTO_CLANG=y, the build process is much more complex. Embedding the CRCs is postponed until the LLVM bitcode is converted into ELF, creating another intermediate *.prelink.o.

However, this complexity is unneeded. There is no reason why we must embed version CRCs in objects so early.

There is final link stage for vmlinux (scripts/link-vmlinux.sh) and modules (scripts/Makefile.modfinal). We can link CRCs at the very last moment.

New implementation
==================

                                                           |----------|
                   --------------------------------------->| final    |
       $(CC)      /    |---------|                         | link for |
       -----> *.o ---->|         |                         | vmlinux  |
      /                | modpost |--- .vmlinux.export.c -->| or       |
     / genksyms        |         |--- *.mod.c ------------>| module   |
  *.c ------> *.cmd -->|---------|                         |----------|

Pass the symbol versions to modpost as separate text data, which are available in *.cmd files.

This commit changes modpost to extract CRCs from *.cmd files instead of from ELF objects.

---

(no changes since v2)

Changes in v2:
- Simplify the implementation (parse .cmd files after ELF)

 scripts/mod/modpost.c | 177 ++++++++++++++++++++++++++++++------------
 1 file changed, 129 insertions(+), 48 deletions(-)
scripts/mod/modpost.c +50 -0
+/*
+ * The CRCs are recorded in .*.cmd files in the form of:
+ * #SYMVER <name> <crc>
+ */
+static void extract_crcs_for_object(const char *object, struct module *mod)
+{
+       char cmd_file[PATH_MAX];
+       char *buf, *p;
+       const char *base;
+       int dirlen, ret;
+
+       base = strrchr(object, '/');
+       if (base) {
+               base++;
+               dirlen = base - object;
+       } else {
+               dirlen = 0;
+               base = object;
+       }
+
+       ret = snprintf(cmd_file, sizeof(cmd_file), "%.*s.%s.cmd",
+                      dirlen, object, base);
+       if (ret >= sizeof(cmd_file)) {
+               error("%s: too long path was truncated\n", cmd_file);
+               return;
+       }
+
+       buf = read_text_file(cmd_file);
+       p = buf;
+
+       while ((p = strstr(p, "\n#SYMVER "))) {
+               char *name;
+               size_t namelen;
+               unsigned int crc;
+               struct symbol *sym;
+
+               name = p + strlen("\n#SYMVER ");
+
+               p = strchr(name, ' ');
+               if (!p)
+                       break;
+
+               namelen = p - name;
+               p++;
+
+               if (!isdigit(*p))
+                       continue;       /* skip this line */
+
+               crc = strtol(p, &p, 0);
+               if (*p != '\n')

scripts/mod/modpost.c static char *remove_dot(char *s) · line 786

Quoted 10 lines
scripts/mod/modpost.c +10 -0
+                       continue;       /* skip this line */
+
+               name[namelen] = '\0';
+
+               sym = sym_find_with_module(name, mod);
+               if (!sym) {
+                       warn("Skip the version for unexported symbol \"%s\" [%s%s]\n",
+                            name, mod->name, mod->is_vmlinux ? "" : ".ko");
+                       continue;
+               }

A lot of warnings are displayed when CONFIG_TRIM_UNUSED_KSYMS=y.

I will fix this in v6.

Signature · 2 lines
Best Regards
Masahiro Yamada
#9 lore raw

PATCH v5 4/12 kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS

In reply to Masahiro Yamada

include/{linux,asm-generic}/export.h defines a weak symbol, __crc_* as a placeholder.

Genksyms writes the version CRCs into the linker script, which will be used for filling the __crc_* symbols. The linker script format depends on CONFIG_MODULE_REL_CRCS. If it is enabled, __crc_* holds the offset to the reference of CRC.

It is time to get rid of this complexity.

Now that modpost parses text files (.*.cmd) to collect all the CRCs, it can generate C code that will be linked to the vmlinux or modules.

Generate a new C file, .vmlinux.export.c, which contains the CRCs of symbols exported by vmlinux. It is compiled and linked to vmlinux in scripts/link-vmlinux.sh.

Put the CRCs of symbols exported by modules into the existing *.mod.c files. No additional build step is needed for modules. As before, *.mod.c are compiled and linked to *.ko in scripts/Makefile.modfinal.

No linker magic is used here. The new C implementation works in the same way, whether CONFIG_RELOCATABLE is enabled or not. CONFIG_MODULE_REL_CRCS is no longer needed.

Previously, Kbuild invoked additional $(LD) to update the CRCs in objects, but this step is unneeded too.

---

Changes in v5:
- Fix the build error when CONFIG_DEBUG_INFO_BTF=y (reported by Nathan)
- Clean up arch/m68k/include/asm/export.h (Nick)
- Keep gen_symversions (and will be removed by a later patch)

Changes in v4:
- Rename .vmlinux-symver.c to .vmlinux.export.c because I notice this approach is useful for further cleanups, not only for modversioning but also for overall EXPORT_SYMBOL.

Changes in v3:
- New patch

 arch/m68k/include/asm/Kbuild    |  1 +
 arch/m68k/include/asm/export.h  |  2 --
 arch/powerpc/Kconfig            |  1 -
 arch/s390/Kconfig               |  1 -
 arch/um/Kconfig                 |  1 -
 include/asm-generic/export.h    | 22 ++++++++--------------
 include/linux/export-internal.h | 16 ++++++++++++++++
 include/linux/export.h          | 30 ++++++++----------------------
 init/Kconfig                    |  4 ----
 kernel/module.c                 | 10 +---------
 scripts/Makefile.build          | 27 ++++-----------------------
 scripts/genksyms/genksyms.c     | 18 ++++--------------
 scripts/link-vmlinux.sh         | 14 +++++++++++++-
 scripts/mod/modpost.c           | 28 ++++++++++++++++++++++++----
 14 files changed, 79 insertions(+), 96 deletions(-)
 delete mode 100644 arch/m68k/include/asm/export.h
 create mode 100644 include/linux/export-internal.h
arch/m68k/include/asm/Kbuild +1 -0
index 0dbf9c5c6fae..1b720299deb1 100644
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
 generated-y += syscall_table.h
+generic-y += export.h
 generic-y += extable.h
 generic-y += kvm_para.h
 generic-y += mcs_spinlock.h
arch/m68k/include/asm/export.h deleted +0 -2
deleted file mode 100644
index b53008b67ce1..000000000000
@@ -1,2 +0,0 @@
-#define KCRC_ALIGN 2
-#include <asm-generic/export.h>
arch/powerpc/Kconfig +0 -1
index 174edabb74fa..a4e8dd889e29 100644
@@ -566,7 +566,6 @@ config RELOCATABLE
 	bool "Build a relocatable kernel"
 	depends on PPC64 || (FLATMEM && (44x || FSL_BOOKE))
 	select NONSTATIC_KERNEL
-	select MODULE_REL_CRCS if MODVERSIONS
 	help
 	  This builds a kernel image that is capable of running at the
 	  location the kernel is loaded at. For ppc32, there is no any
arch/s390/Kconfig +0 -1
index 77b5a03de13a..aa5848004c76 100644
@@ -567,7 +567,6 @@ endchoice
 
 config RELOCATABLE
 	bool "Build a relocatable kernel"
-	select MODULE_REL_CRCS if MODVERSIONS
 	default y
 	help
 	  This builds a kernel image that retains relocation information
arch/um/Kconfig +0 -1
index 4d398b80aea8..e8983d098e73 100644
@@ -106,7 +106,6 @@ config LD_SCRIPT_DYN
 	bool
 	default y
 	depends on !LD_SCRIPT_STATIC
-	select MODULE_REL_CRCS if MODVERSIONS
 
 config LD_SCRIPT_DYN_RPATH
 	bool "set rpath in the binary" if EXPERT
include/asm-generic/export.h +8 -14
index 07a36a874dca..5e4b1f2369d2 100644
@@ -2,6 +2,14 @@
 #ifndef __ASM_GENERIC_EXPORT_H
 #define __ASM_GENERIC_EXPORT_H
 
+/*
+ * This comment block is used by fixdep. Please do not remove.
+ *
+ * When CONFIG_MODVERSIONS is changed from n to y, all source files having
+ * EXPORT_SYMBOL variants must be re-compiled because genksyms is run as a
+ * side effect of the *.o build rule.
+ */
+
 #ifndef KSYM_FUNC
 #define KSYM_FUNC(x) x
 #endif
@@ -12,9 +20,6 @@
 #else
 #define KSYM_ALIGN 4
 #endif
-#ifndef KCRC_ALIGN
-#define KCRC_ALIGN 4
-#endif
 
 .macro __put, val, name
 #ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
@@ -43,17 +48,6 @@ __ksymtab_\name:
 __kstrtab_\name:
 	.asciz "\name"
 	.previous
-#ifdef CONFIG_MODVERSIONS
-	.section ___kcrctab\sec+\name,"a"
-	.balign KCRC_ALIGN
-#if defined(CONFIG_MODULE_REL_CRCS)
-	.long __crc_\name - .
-#else
-	.long __crc_\name
-#endif
-	.weak __crc_\name
-	.previous
-#endif
 #endif
 .endm
 
include/linux/export-internal.h new file +16 -0
new file mode 100644
index 000000000000..77175d561058
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Please do not include this explicitly.
+ * This is used by C files generated by modpost.
+ */
+
+#ifndef __LINUX_EXPORT_INTERNAL_H__
+#define __LINUX_EXPORT_INTERNAL_H__
+
+#include <linux/compiler.h>
+#include <linux/types.h>
+
+#define SYMBOL_CRC(sym, crc, sec)   \
+	u32 __section("___kcrctab" sec "+" #sym) __crc_##sym = crc
+
+#endif /* __LINUX_EXPORT_INTERNAL_H__ */
include/linux/export.h +8 -22
index 27d848712b90..565c5ffcb26f 100644
@@ -11,6 +11,14 @@
  * hackers place grumpy comments in header files.
  */
 
+/*
+ * This comment block is used by fixdep. Please do not remove.
+ *
+ * When CONFIG_MODVERSIONS is changed from n to y, all source files having
+ * EXPORT_SYMBOL variants must be re-compiled because genksyms is run as a
+ * side effect of the *.o build rule.
+ */
+
 #ifndef __ASSEMBLY__
 #ifdef MODULE
 extern struct module __this_module;
@@ -19,26 +27,6 @@ extern struct module __this_module;
 #define THIS_MODULE ((struct module *)0)
 #endif
 
-#ifdef CONFIG_MODVERSIONS
-/* Mark the CRC weak since genksyms apparently decides not to
- * generate a checksums for some symbols */
-#if defined(CONFIG_MODULE_REL_CRCS)
-#define __CRC_SYMBOL(sym, sec)						\
-	asm("	.section \"___kcrctab" sec "+" #sym "\", \"a\"	\n"	\
-	    "	.weak	__crc_" #sym "				\n"	\
-	    "	.long	__crc_" #sym " - .			\n"	\
-	    "	.previous					\n")
-#else
-#define __CRC_SYMBOL(sym, sec)						\
-	asm("	.section \"___kcrctab" sec "+" #sym "\", \"a\"	\n"	\
-	    "	.weak	__crc_" #sym "				\n"	\
-	    "	.long	__crc_" #sym "				\n"	\
-	    "	.previous					\n")
-#endif
-#else
-#define __CRC_SYMBOL(sym, sec)
-#endif
-
 #ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
 #include <linux/compiler.h>
 /*
@@ -85,7 +73,6 @@ struct kernel_symbol {
 /*
  * For every exported symbol, do the following:
  *
- * - If applicable, place a CRC entry in the __kcrctab section.
  * - Put the name of the symbol and namespace (empty string "" for none) in
  *   __ksymtab_strings.
  * - Place a struct kernel_symbol entry in the __ksymtab section.
@@ -98,7 +85,6 @@ struct kernel_symbol {
 	extern typeof(sym) sym;							\
 	extern const char __kstrtab_##sym[];					\
 	extern const char __kstrtabns_##sym[];					\
-	__CRC_SYMBOL(sym, sec);							\
 	asm("	.section \"__ksymtab_strings\",\"aMS\",%progbits,1	\n"	\
 	    "__kstrtab_" #sym ":					\n"	\
 	    "	.asciz 	\"" #sym "\"					\n"	\
init/Kconfig +0 -4
index ddcbefe535e9..f5b14318dfcb 100644
@@ -2136,10 +2136,6 @@ config ASM_MODVERSIONS
 	  assembly. This can be enabled only when the target architecture
 	  supports it.
 
-config MODULE_REL_CRCS
-	bool
-	depends on MODVERSIONS
-
 config MODULE_SRCVERSION_ALL
 	bool "Source checksum for all modules"
 	help
kernel/module.c +1 -9
index 6cea788fd965..c9e2342da28e 100644
@@ -1231,11 +1231,6 @@ static int try_to_force_load(struct module *mod, const char *reason)
 
 #ifdef CONFIG_MODVERSIONS
 
-static u32 resolve_rel_crc(const s32 *crc)
-{
-	return *(u32 *)((void *)crc + *crc);
-}
-
 static int check_version(const struct load_info *info,
 			 const char *symname,
 			 struct module *mod,
@@ -1264,10 +1259,7 @@ static int check_version(const struct load_info *info,
 		if (strcmp(versions[i].name, symname) != 0)
 			continue;
 
-		if (IS_ENABLED(CONFIG_MODULE_REL_CRCS))
-			crcval = resolve_rel_crc(crc);
-		else
-			crcval = *crc;
+		crcval = *crc;
 		if (versions[i].crc == crcval)
 			return 1;
 		pr_debug("Found checksum %X vs module %lX\n",
scripts/Makefile.build +4 -23
index a1023868775f..ddd9080fc028 100644
@@ -128,7 +128,6 @@ $(obj)/%.i: $(src)/%.c FORCE
 
 genksyms = scripts/genksyms/genksyms		\
 	$(if $(1), -T $(2))			\
-	$(if $(CONFIG_MODULE_REL_CRCS), -R)	\
 	$(if $(KBUILD_PRESERVE), -p)		\
 	-r $(or $(wildcard $(2:.symtypes=.symref)), /dev/null)
 
@@ -162,19 +161,11 @@ ifdef CONFIG_MODVERSIONS
 # o if <file>.o doesn't contain a __ksymtab version, i.e. does
 #   not export symbols, it's done.
 # o otherwise, we calculate symbol versions using the good old
-#   genksyms on the preprocessed source and postprocess them in a way
-#   that they are usable as a linker script
-# o generate .tmp_<file>.o from <file>.o using the linker to
-#   replace the unresolved symbols __crc_exported_symbol with
-#   the actual value of the checksum generated by genksyms
-# o remove .tmp_<file>.o to <file>.o
+#   genksyms on the preprocessed source and dump them into the .cmd file.
+# o modpost will extract versions from that file and create *.c files that will
+#   be compiled and linked to the kernel and/or modules.
 
-# Generate .o.symversions files for each .o with exported symbols, and link these
-# to the kernel and/or modules at the end.
-
-genksyms_format_rel_crc := [^_]*__crc_\([^ ]*\) = \.; LONG(\([^)]*\)).*
-genksyms_format_normal := __crc_\(.*\) = \(.*\);
-genksyms_format := $(if $(CONFIG_MODULE_REL_CRCS),$(genksyms_format_rel_crc),$(genksyms_format_normal))
+genksyms_format := __crc_\(.*\) = \(.*\);
 
 gen_symversions =								\
 	if $(NM) $@ 2>/dev/null | grep -q __ksymtab; then			\
@@ -188,12 +179,6 @@ gen_symversions =								\
 
 cmd_gen_symversions_c =	$(call gen_symversions,c)
 
-cmd_modversions =								\
-	if [ -r [email protected] ]; then						\
-		$(LD) $(KBUILD_LDFLAGS) -r -o $(@D)/.tmp_$(@F) $@ 		\
-			-T [email protected];					\
-		mv -f $(@D)/.tmp_$(@F) $@;					\
-	fi
 endif
 
 ifdef CONFIG_FTRACE_MCOUNT_USE_RECORDMCOUNT
@@ -273,7 +258,6 @@ define rule_cc_o_c
 	$(call cmd,checkdoc)
 	$(call cmd,gen_objtooldep)
 	$(call cmd,gen_symversions_c)
-	$(if $(CONFIG_LTO_CLANG),,$(call cmd,modversions))
 	$(call cmd,record_mcount)
 endef
 
@@ -282,7 +266,6 @@ define rule_as_o_S
 	$(call cmd,gen_ksymdeps)
 	$(call cmd,gen_objtooldep)
 	$(call cmd,gen_symversions_S)
-	$(call cmd,modversions)
 endef
 
 # Built-in and composite module parts
@@ -296,8 +279,6 @@ ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),)
 quiet_cmd_cc_prelink_modules = LD [M]  $@
       cmd_cc_prelink_modules =						\
 	$(LD) $(ld_flags) -r -o $@					\
-               $(shell [ -s $(@:.prelink.o=.o.symversions) ] &&		\
-                       echo -T $(@:.prelink.o=.o.symversions))		\
 		--whole-archive $(filter-out FORCE,$^)			\
 		$(cmd_objtool)
 
scripts/genksyms/genksyms.c +4 -14
index 4827c5abe5b7..67b23cc0df0f 100644
@@ -33,7 +33,7 @@ char *cur_filename;
 int in_source_file;
 
 static int flag_debug, flag_dump_defs, flag_reference, flag_dump_types,
-	   flag_preserve, flag_warnings, flag_rel_crcs;
+	   flag_preserve, flag_warnings;
 
 static int errors;
 static int nsyms;
@@ -680,11 +680,7 @@ void export_symbol(const char *name)
 		if (flag_dump_defs)
 			fputs(">\n", debugfile);
 
-		/* Used as a linker script. */
-		printf(!flag_rel_crcs ? "__crc_%s = 0x%08lx;\n" :
-		       "SECTIONS { .rodata : ALIGN(4) { "
-		       "__crc_%s = .; LONG(0x%08lx); } }\n",
-		       name, crc);
+		printf("__crc_%s = 0x%08lx;\n", name, crc);
 	}
 }
 
@@ -733,7 +729,6 @@ static void genksyms_usage(void)
 	      "  -q, --quiet           Disable warnings (default)\n"
 	      "  -h, --help            Print this message\n"
 	      "  -V, --version         Print the release version\n"
-	      "  -R, --relative-crc    Emit section relative symbol CRCs\n"
 #else				/* __GNU_LIBRARY__ */
 	      "  -s                    Select symbol prefix\n"
 	      "  -d                    Increment the debug level (repeatable)\n"
@@ -745,7 +740,6 @@ static void genksyms_usage(void)
 	      "  -q                    Disable warnings (default)\n"
 	      "  -h                    Print this message\n"
 	      "  -V                    Print the release version\n"
-	      "  -R                    Emit section relative symbol CRCs\n"
 #endif				/* __GNU_LIBRARY__ */
 	      , stderr);
 }
@@ -766,14 +760,13 @@ int main(int argc, char **argv)
 		{"preserve", 0, 0, 'p'},
 		{"version", 0, 0, 'V'},
 		{"help", 0, 0, 'h'},
-		{"relative-crc", 0, 0, 'R'},
 		{0, 0, 0, 0}
 	};
 
-	while ((o = getopt_long(argc, argv, "s:dwqVDr:T:phR",
+	while ((o = getopt_long(argc, argv, "s:dwqVDr:T:ph",
 				&long_opts[0], NULL)) != EOF)
 #else				/* __GNU_LIBRARY__ */
-	while ((o = getopt(argc, argv, "s:dwqVDr:T:phR")) != EOF)
+	while ((o = getopt(argc, argv, "s:dwqVDr:T:ph")) != EOF)
 #endif				/* __GNU_LIBRARY__ */
 		switch (o) {
 		case 'd':
@@ -813,9 +806,6 @@ int main(int argc, char **argv)
 		case 'h':
 			genksyms_usage();
 			return 0;
-		case 'R':
-			flag_rel_crcs = 1;
-			break;
 		default:
 			genksyms_usage();
 			return 1;
scripts/link-vmlinux.sh +13 -1
index eceb3ee7ec06..7313cbd755df 100755
@@ -90,7 +90,6 @@ modpost_link()
 
 		if is_enabled CONFIG_MODVERSIONS; then
 			gen_symversions
-			lds="${lds} -T .tmp_symversions.lds"
 		fi
 
 		# This might take a while, so indicate that we're doing
@@ -183,6 +182,10 @@ vmlinux_link()
 		libs="${KBUILD_VMLINUX_LIBS}"
 	fi
 
+	if is_enabled CONFIG_MODULES; then
+		objs="${objs} .vmlinux.export.o"
+	fi
+
 	if [ "${SRCARCH}" = "um" ]; then
 		wl=-Wl,
 		ld="${CC}"
@@ -312,6 +315,7 @@ cleanup()
 	rm -f vmlinux.o
 	rm -f .vmlinux.d
 	rm -f .vmlinux.objs
+	rm -f .vmlinux.export.c
 }
 
 # Use "make V=1" to debug this script
@@ -363,6 +367,14 @@ info GEN modules.builtin
 tr '\0' '\n' < modules.builtin.modinfo | sed -n 's/^[[:alnum:]:_]*\.file=//p' |
 	tr ' ' '\n' | uniq | sed -e 's:^:kernel/:' -e 's/$/.ko/' > modules.builtin
 
+if is_enabled CONFIG_MODULES; then
+	info CC .vmlinux.export.c
+	${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} \
+	      ${KBUILD_CPPFLAGS} ${KBUILD_CFLAGS} \
+	      ${KBUILD_CFLAGS_KERNEL} ${CFLAGS_KERNEL} \
+	      -c -o .vmlinux.export.o .vmlinux.export.c
+fi
+
 btf_vmlinux_bin_o=""
 if is_enabled CONFIG_DEBUG_INFO_BTF; then
 	btf_vmlinux_bin_o=.btf.vmlinux.bin.o
scripts/mod/modpost.c +24 -4
index e0f9c02d9f83..6b47a8723389 100644
@@ -1040,6 +1040,7 @@ static void add_header(struct buffer *b, struct module *mod)
 	buf_printf(b, "#define INCLUDE_VERMAGIC\n");
 	buf_printf(b, "#include <linux/build-salt.h>\n");
 	buf_printf(b, "#include <linux/elfnote-lto.h>\n");
+	buf_printf(b, "#include <linux/export-internal.h>\n");
 	buf_printf(b, "#include <linux/vermagic.h>\n");
 	buf_printf(b, "#include <linux/compiler.h>\n");
 	buf_printf(b, "\n");
@@ -1074,20 +1075,26 @@ static void add_header(struct buffer *b, struct module *mod)
 		buf_printf(b, "\nMODULE_INFO(staging, \"Y\");\n");
 }
 
-static void check_symversions(struct module *mod)
+static void add_exported_symbols(struct buffer *buf, struct module *mod)
 {
 	struct symbol *sym;
 
 	if (!modversions)
 		return;
 
+	/* record CRCs for exported symbols */
+	buf_printf(buf, "\n");
 	list_for_each_entry(sym, &mod->exported_symbols, list) {
 		if (!sym->crc_valid) {
 			warn("EXPORT symbol \"%s\" [%s%s] version generation failed, symbol will not be versioned.\n"
 			     "Is \"%s\" prototyped in <asm/asm-prototypes.h>?\n",
 			     sym->name, mod->name, mod->is_vmlinux ? "" : ".ko",
 			     sym->name);
+			continue;
 		}
+
+		buf_printf(buf, "SYMBOL_CRC(%s, 0x%08x, \"%s\");\n",
+			   sym->name, sym->crc, sym->is_gpl_only ? "_gpl" : "");
 	}
 }
 
@@ -1224,6 +1231,18 @@ static void write_if_changed(struct buffer *b, const char *fname)
 	write_buf(b, fname);
 }
 
+static void write_vmlinux_export_c_file(struct module *mod)
+{
+	struct buffer buf = { };
+
+	buf_printf(&buf,
+		   "#include <linux/export-internal.h>\n");
+
+	add_exported_symbols(&buf, mod);
+	write_if_changed(&buf, ".vmlinux.export.c");
+	free(buf.p);
+}
+
 /* do sanity checks, and generate *.mod.c file */
 static void write_mod_c_file(struct module *mod)
 {
@@ -1235,6 +1254,7 @@ static void write_mod_c_file(struct module *mod)
 	check_exports(mod);
 
 	add_header(&buf, mod);
+	add_exported_symbols(&buf, mod);
 	add_versions(&buf, mod);
 	add_depends(&buf, mod);
 	add_moddevtable(&buf, mod);
@@ -1432,9 +1452,9 @@ int main(int argc, char **argv)
 		if (mod->from_dump)
 			continue;
 
-		check_symversions(mod);
-
-		if (!mod->is_vmlinux)
+		if (mod->is_vmlinux)
+			write_vmlinux_export_c_file(mod);
+		else
 			write_mod_c_file(mod);
 	}
 
Signature · 1 line
2.32.0
Tested-by Reviewed-by on v5 4/12
#10 lore raw

Re: [PATCH v5 04/12] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS

Quoted 563 lines · Masahiro Yamada

include/{linux,asm-generic}/export.h defines a weak symbol, __crc_* as a placeholder.

Genksyms writes the version CRCs into the linker script, which will be used for filling the __crc_* symbols. The linker script format depends on CONFIG_MODULE_REL_CRCS. If it is enabled, __crc_* holds the offset to the reference of CRC.

It is time to get rid of this complexity.

Now that modpost parses text files (.*.cmd) to collect all the CRCs, it can generate C code that will be linked to the vmlinux or modules.

Generate a new C file, .vmlinux.export.c, which contains the CRCs of symbols exported by vmlinux. It is compiled and linked to vmlinux in scripts/link-vmlinux.sh.

Put the CRCs of symbols exported by modules into the existing *.mod.c files. No additional build step is needed for modules. As before, *.mod.c are compiled and linked to *.ko in scripts/Makefile.modfinal.

No linker magic is used here. The new C implementation works in the same way, whether CONFIG_RELOCATABLE is enabled or not. CONFIG_MODULE_REL_CRCS is no longer needed.

Previously, Kbuild invoked additional $(LD) to update the CRCs in objects, but this step is unneeded too.

---

Changes in v5:
- Fix the build error when CONFIG_DEBUG_INFO_BTF=y (reported by Nathan)
- Clean up arch/m68k/include/asm/export.h (Nick)
- Keep gen_symversions (and will be removed by a later patch)

Changes in v4:
- Rename .vmlinux-symver.c to .vmlinux.export.c because I notice this approach is useful for further cleanups, not only for modversioning but also for overall EXPORT_SYMBOL.

Changes in v3:
- New patch

 arch/m68k/include/asm/Kbuild    |  1 +
 arch/m68k/include/asm/export.h  |  2 --
 arch/powerpc/Kconfig            |  1 -
 arch/s390/Kconfig               |  1 -
 arch/um/Kconfig                 |  1 -
 include/asm-generic/export.h    | 22 ++++++++--------------
 include/linux/export-internal.h | 16 ++++++++++++++++
 include/linux/export.h          | 30 ++++++++----------------------
 init/Kconfig                    |  4 ----
 kernel/module.c                 | 10 +---------
 scripts/Makefile.build          | 27 ++++-----------------------
 scripts/genksyms/genksyms.c     | 18 ++++--------------
 scripts/link-vmlinux.sh         | 14 +++++++++++++-
 scripts/mod/modpost.c           | 28 ++++++++++++++++++++++++----
 14 files changed, 79 insertions(+), 96 deletions(-)
 delete mode 100644 arch/m68k/include/asm/export.h
 create mode 100644 include/linux/export-internal.h
arch/m68k/include/asm/Kbuild +1 -0
index 0dbf9c5c6fae..1b720299deb1 100644
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
 generated-y += syscall_table.h
+generic-y += export.h
 generic-y += extable.h
 generic-y += kvm_para.h
 generic-y += mcs_spinlock.h
arch/m68k/include/asm/export.h deleted +0 -2
deleted file mode 100644
index b53008b67ce1..000000000000
@@ -1,2 +0,0 @@
-#define KCRC_ALIGN 2
-#include <asm-generic/export.h>
arch/powerpc/Kconfig +0 -1
index 174edabb74fa..a4e8dd889e29 100644
@@ -566,7 +566,6 @@ config RELOCATABLE
 	bool "Build a relocatable kernel"
 	depends on PPC64 || (FLATMEM && (44x || FSL_BOOKE))
 	select NONSTATIC_KERNEL
-	select MODULE_REL_CRCS if MODVERSIONS
 	help
 	  This builds a kernel image that is capable of running at the
 	  location the kernel is loaded at. For ppc32, there is no any
arch/s390/Kconfig +0 -1
index 77b5a03de13a..aa5848004c76 100644
@@ -567,7 +567,6 @@ endchoice
 
 config RELOCATABLE
 	bool "Build a relocatable kernel"
-	select MODULE_REL_CRCS if MODVERSIONS
 	default y
 	help
 	  This builds a kernel image that retains relocation information
arch/um/Kconfig +0 -1
index 4d398b80aea8..e8983d098e73 100644
@@ -106,7 +106,6 @@ config LD_SCRIPT_DYN
 	bool
 	default y
 	depends on !LD_SCRIPT_STATIC
-	select MODULE_REL_CRCS if MODVERSIONS
 
 config LD_SCRIPT_DYN_RPATH
 	bool "set rpath in the binary" if EXPERT
include/asm-generic/export.h +8 -14
index 07a36a874dca..5e4b1f2369d2 100644
@@ -2,6 +2,14 @@
 #ifndef __ASM_GENERIC_EXPORT_H
 #define __ASM_GENERIC_EXPORT_H
 
+/*
+ * This comment block is used by fixdep. Please do not remove.
+ *
+ * When CONFIG_MODVERSIONS is changed from n to y, all source files having
+ * EXPORT_SYMBOL variants must be re-compiled because genksyms is run as a
+ * side effect of the *.o build rule.
+ */
+
 #ifndef KSYM_FUNC
 #define KSYM_FUNC(x) x
 #endif
@@ -12,9 +20,6 @@
 #else
 #define KSYM_ALIGN 4
 #endif
-#ifndef KCRC_ALIGN
-#define KCRC_ALIGN 4
-#endif
 
 .macro __put, val, name
 #ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
@@ -43,17 +48,6 @@ __ksymtab_\name:
 __kstrtab_\name:
 	.asciz "\name"
 	.previous
-#ifdef CONFIG_MODVERSIONS
-	.section ___kcrctab\sec+\name,"a"
-	.balign KCRC_ALIGN
-#if defined(CONFIG_MODULE_REL_CRCS)
-	.long __crc_\name - .
-#else
-	.long __crc_\name
-#endif
-	.weak __crc_\name
-	.previous
-#endif
 #endif
 .endm
 
include/linux/export-internal.h new file +16 -0
new file mode 100644
index 000000000000..77175d561058
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Please do not include this explicitly.
+ * This is used by C files generated by modpost.
+ */
+
+#ifndef __LINUX_EXPORT_INTERNAL_H__
+#define __LINUX_EXPORT_INTERNAL_H__
+
+#include <linux/compiler.h>
+#include <linux/types.h>
+
+#define SYMBOL_CRC(sym, crc, sec)   \
+	u32 __section("___kcrctab" sec "+" #sym) __crc_##sym = crc
+
+#endif /* __LINUX_EXPORT_INTERNAL_H__ */
include/linux/export.h +8 -22
index 27d848712b90..565c5ffcb26f 100644
@@ -11,6 +11,14 @@
  * hackers place grumpy comments in header files.
  */
 
+/*
+ * This comment block is used by fixdep. Please do not remove.
+ *
+ * When CONFIG_MODVERSIONS is changed from n to y, all source files having
+ * EXPORT_SYMBOL variants must be re-compiled because genksyms is run as a
+ * side effect of the *.o build rule.
+ */
+
 #ifndef __ASSEMBLY__
 #ifdef MODULE
 extern struct module __this_module;
@@ -19,26 +27,6 @@ extern struct module __this_module;
 #define THIS_MODULE ((struct module *)0)
 #endif
 
-#ifdef CONFIG_MODVERSIONS
-/* Mark the CRC weak since genksyms apparently decides not to
- * generate a checksums for some symbols */
-#if defined(CONFIG_MODULE_REL_CRCS)
-#define __CRC_SYMBOL(sym, sec)						\
-	asm("	.section \"___kcrctab" sec "+" #sym "\", \"a\"	\n"	\
-	    "	.weak	__crc_" #sym "				\n"	\
-	    "	.long	__crc_" #sym " - .			\n"	\
-	    "	.previous					\n")
-#else
-#define __CRC_SYMBOL(sym, sec)						\
-	asm("	.section \"___kcrctab" sec "+" #sym "\", \"a\"	\n"	\
-	    "	.weak	__crc_" #sym "				\n"	\
-	    "	.long	__crc_" #sym "				\n"	\
-	    "	.previous					\n")
-#endif
-#else
-#define __CRC_SYMBOL(sym, sec)
-#endif
-
 #ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
 #include <linux/compiler.h>
 /*
@@ -85,7 +73,6 @@ struct kernel_symbol {
 /*
  * For every exported symbol, do the following:
  *
- * - If applicable, place a CRC entry in the __kcrctab section.
  * - Put the name of the symbol and namespace (empty string "" for none) in
  *   __ksymtab_strings.
  * - Place a struct kernel_symbol entry in the __ksymtab section.
@@ -98,7 +85,6 @@ struct kernel_symbol {
 	extern typeof(sym) sym;							\
 	extern const char __kstrtab_##sym[];					\
 	extern const char __kstrtabns_##sym[];					\
-	__CRC_SYMBOL(sym, sec);							\
 	asm("	.section \"__ksymtab_strings\",\"aMS\",%progbits,1	\n"	\
 	    "__kstrtab_" #sym ":					\n"	\
 	    "	.asciz 	\"" #sym "\"					\n"	\
init/Kconfig +0 -4
index ddcbefe535e9..f5b14318dfcb 100644
@@ -2136,10 +2136,6 @@ config ASM_MODVERSIONS
 	  assembly. This can be enabled only when the target architecture
 	  supports it.
 
-config MODULE_REL_CRCS
-	bool
-	depends on MODVERSIONS
-
 config MODULE_SRCVERSION_ALL
 	bool "Source checksum for all modules"
 	help
kernel/module.c +1 -9
index 6cea788fd965..c9e2342da28e 100644
@@ -1231,11 +1231,6 @@ static int try_to_force_load(struct module *mod, const char *reason)
 
 #ifdef CONFIG_MODVERSIONS
 
-static u32 resolve_rel_crc(const s32 *crc)
-{
-	return *(u32 *)((void *)crc + *crc);
-}
-
 static int check_version(const struct load_info *info,
 			 const char *symname,
 			 struct module *mod,
@@ -1264,10 +1259,7 @@ static int check_version(const struct load_info *info,
 		if (strcmp(versions[i].name, symname) != 0)
 			continue;
 
-		if (IS_ENABLED(CONFIG_MODULE_REL_CRCS))
-			crcval = resolve_rel_crc(crc);
-		else
-			crcval = *crc;
+		crcval = *crc;
 		if (versions[i].crc == crcval)
 			return 1;
 		pr_debug("Found checksum %X vs module %lX\n",
scripts/Makefile.build +4 -23
index a1023868775f..ddd9080fc028 100644
@@ -128,7 +128,6 @@ $(obj)/%.i: $(src)/%.c FORCE
 
 genksyms = scripts/genksyms/genksyms		\
 	$(if $(1), -T $(2))			\
-	$(if $(CONFIG_MODULE_REL_CRCS), -R)	\
 	$(if $(KBUILD_PRESERVE), -p)		\
 	-r $(or $(wildcard $(2:.symtypes=.symref)), /dev/null)
 
@@ -162,19 +161,11 @@ ifdef CONFIG_MODVERSIONS
 # o if <file>.o doesn't contain a __ksymtab version, i.e. does
 #   not export symbols, it's done.
 # o otherwise, we calculate symbol versions using the good old
-#   genksyms on the preprocessed source and postprocess them in a way
-#   that they are usable as a linker script
-# o generate .tmp_<file>.o from <file>.o using the linker to
-#   replace the unresolved symbols __crc_exported_symbol with
-#   the actual value of the checksum generated by genksyms
-# o remove .tmp_<file>.o to <file>.o
+#   genksyms on the preprocessed source and dump them into the .cmd file.
+# o modpost will extract versions from that file and create *.c files that will
+#   be compiled and linked to the kernel and/or modules.
 
-# Generate .o.symversions files for each .o with exported symbols, and link these
-# to the kernel and/or modules at the end.
-
-genksyms_format_rel_crc := [^_]*__crc_\([^ ]*\) = \.; LONG(\([^)]*\)).*
-genksyms_format_normal := __crc_\(.*\) = \(.*\);
-genksyms_format := $(if $(CONFIG_MODULE_REL_CRCS),$(genksyms_format_rel_crc),$(genksyms_format_normal))
+genksyms_format := __crc_\(.*\) = \(.*\);
 
 gen_symversions =								\
 	if $(NM) $@ 2>/dev/null | grep -q __ksymtab; then			\
@@ -188,12 +179,6 @@ gen_symversions =								\
 
 cmd_gen_symversions_c =	$(call gen_symversions,c)
 
-cmd_modversions =								\
-	if [ -r [email protected] ]; then						\
-		$(LD) $(KBUILD_LDFLAGS) -r -o $(@D)/.tmp_$(@F) $@ 		\
-			-T [email protected];					\
-		mv -f $(@D)/.tmp_$(@F) $@;					\
-	fi
 endif
 
 ifdef CONFIG_FTRACE_MCOUNT_USE_RECORDMCOUNT
@@ -273,7 +258,6 @@ define rule_cc_o_c
 	$(call cmd,checkdoc)
 	$(call cmd,gen_objtooldep)
 	$(call cmd,gen_symversions_c)
-	$(if $(CONFIG_LTO_CLANG),,$(call cmd,modversions))
 	$(call cmd,record_mcount)
 endef
 
@@ -282,7 +266,6 @@ define rule_as_o_S
 	$(call cmd,gen_ksymdeps)
 	$(call cmd,gen_objtooldep)
 	$(call cmd,gen_symversions_S)
-	$(call cmd,modversions)
 endef
 
 # Built-in and composite module parts
@@ -296,8 +279,6 @@ ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),)
 quiet_cmd_cc_prelink_modules = LD [M]  $@
       cmd_cc_prelink_modules =						\
 	$(LD) $(ld_flags) -r -o $@					\
-               $(shell [ -s $(@:.prelink.o=.o.symversions) ] &&		\
-                       echo -T $(@:.prelink.o=.o.symversions))		\
 		--whole-archive $(filter-out FORCE,$^)			\
 		$(cmd_objtool)
 
scripts/genksyms/genksyms.c +4 -14
index 4827c5abe5b7..67b23cc0df0f 100644
@@ -33,7 +33,7 @@ char *cur_filename;
 int in_source_file;
 
 static int flag_debug, flag_dump_defs, flag_reference, flag_dump_types,
-	   flag_preserve, flag_warnings, flag_rel_crcs;
+	   flag_preserve, flag_warnings;
 
 static int errors;
 static int nsyms;
@@ -680,11 +680,7 @@ void export_symbol(const char *name)
 		if (flag_dump_defs)
 			fputs(">\n", debugfile);
 
-		/* Used as a linker script. */
-		printf(!flag_rel_crcs ? "__crc_%s = 0x%08lx;\n" :
-		       "SECTIONS { .rodata : ALIGN(4) { "
-		       "__crc_%s = .; LONG(0x%08lx); } }\n",
-		       name, crc);
+		printf("__crc_%s = 0x%08lx;\n", name, crc);
 	}
 }
 
@@ -733,7 +729,6 @@ static void genksyms_usage(void)
 	      "  -q, --quiet           Disable warnings (default)\n"
 	      "  -h, --help            Print this message\n"
 	      "  -V, --version         Print the release version\n"
-	      "  -R, --relative-crc    Emit section relative symbol CRCs\n"
 #else				/* __GNU_LIBRARY__ */
 	      "  -s                    Select symbol prefix\n"
 	      "  -d                    Increment the debug level (repeatable)\n"
@@ -745,7 +740,6 @@ static void genksyms_usage(void)
 	      "  -q                    Disable warnings (default)\n"
 	      "  -h                    Print this message\n"
 	      "  -V                    Print the release version\n"
-	      "  -R                    Emit section relative symbol CRCs\n"
 #endif				/* __GNU_LIBRARY__ */
 	      , stderr);
 }
@@ -766,14 +760,13 @@ int main(int argc, char **argv)
 		{"preserve", 0, 0, 'p'},
 		{"version", 0, 0, 'V'},
 		{"help", 0, 0, 'h'},
-		{"relative-crc", 0, 0, 'R'},
 		{0, 0, 0, 0}
 	};
 
-	while ((o = getopt_long(argc, argv, "s:dwqVDr:T:phR",
+	while ((o = getopt_long(argc, argv, "s:dwqVDr:T:ph",
 				&long_opts[0], NULL)) != EOF)
 #else				/* __GNU_LIBRARY__ */
-	while ((o = getopt(argc, argv, "s:dwqVDr:T:phR")) != EOF)
+	while ((o = getopt(argc, argv, "s:dwqVDr:T:ph")) != EOF)
 #endif				/* __GNU_LIBRARY__ */
 		switch (o) {
 		case 'd':
@@ -813,9 +806,6 @@ int main(int argc, char **argv)
 		case 'h':
 			genksyms_usage();
 			return 0;
-		case 'R':
-			flag_rel_crcs = 1;
-			break;
 		default:
 			genksyms_usage();
 			return 1;
scripts/link-vmlinux.sh +13 -1
index eceb3ee7ec06..7313cbd755df 100755
@@ -90,7 +90,6 @@ modpost_link()
 
 		if is_enabled CONFIG_MODVERSIONS; then
 			gen_symversions
-			lds="${lds} -T .tmp_symversions.lds"
 		fi
 
 		# This might take a while, so indicate that we're doing
@@ -183,6 +182,10 @@ vmlinux_link()
 		libs="${KBUILD_VMLINUX_LIBS}"
 	fi
 
+	if is_enabled CONFIG_MODULES; then
+		objs="${objs} .vmlinux.export.o"
+	fi
+
 	if [ "${SRCARCH}" = "um" ]; then
 		wl=-Wl,
 		ld="${CC}"
@@ -312,6 +315,7 @@ cleanup()
 	rm -f vmlinux.o
 	rm -f .vmlinux.d
 	rm -f .vmlinux.objs
+	rm -f .vmlinux.export.c
 }
 
 # Use "make V=1" to debug this script
@@ -363,6 +367,14 @@ info GEN modules.builtin
 tr '\0' '\n' < modules.builtin.modinfo | sed -n 's/^[[:alnum:]:_]*\.file=//p' |
 	tr ' ' '\n' | uniq | sed -e 's:^:kernel/:' -e 's/$/.ko/' > modules.builtin
 
+if is_enabled CONFIG_MODULES; then
+	info CC .vmlinux.export.c
+	${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} \
+	      ${KBUILD_CPPFLAGS} ${KBUILD_CFLAGS} \
+	      ${KBUILD_CFLAGS_KERNEL} ${CFLAGS_KERNEL} \
+	      -c -o .vmlinux.export.o .vmlinux.export.c
+fi
+
 btf_vmlinux_bin_o=""
 if is_enabled CONFIG_DEBUG_INFO_BTF; then
 	btf_vmlinux_bin_o=.btf.vmlinux.bin.o
scripts/mod/modpost.c +24 -4
index e0f9c02d9f83..6b47a8723389 100644
@@ -1040,6 +1040,7 @@ static void add_header(struct buffer *b, struct module *mod)
 	buf_printf(b, "#define INCLUDE_VERMAGIC\n");
 	buf_printf(b, "#include <linux/build-salt.h>\n");
 	buf_printf(b, "#include <linux/elfnote-lto.h>\n");
+	buf_printf(b, "#include <linux/export-internal.h>\n");
 	buf_printf(b, "#include <linux/vermagic.h>\n");
 	buf_printf(b, "#include <linux/compiler.h>\n");
 	buf_printf(b, "\n");
@@ -1074,20 +1075,26 @@ static void add_header(struct buffer *b, struct module *mod)
 		buf_printf(b, "\nMODULE_INFO(staging, \"Y\");\n");
 }
 
-static void check_symversions(struct module *mod)
+static void add_exported_symbols(struct buffer *buf, struct module *mod)
 {
 	struct symbol *sym;
 
 	if (!modversions)
 		return;
 
+	/* record CRCs for exported symbols */
+	buf_printf(buf, "\n");
 	list_for_each_entry(sym, &mod->exported_symbols, list) {
 		if (!sym->crc_valid) {
 			warn("EXPORT symbol \"%s\" [%s%s] version generation failed, symbol will not be versioned.\n"
 			     "Is \"%s\" prototyped in <asm/asm-prototypes.h>?\n",
 			     sym->name, mod->name, mod->is_vmlinux ? "" : ".ko",
 			     sym->name);
+			continue;
 		}
+
+		buf_printf(buf, "SYMBOL_CRC(%s, 0x%08x, \"%s\");\n",
+			   sym->name, sym->crc, sym->is_gpl_only ? "_gpl" : "");
 	}
 }
 
@@ -1224,6 +1231,18 @@ static void write_if_changed(struct buffer *b, const char *fname)
 	write_buf(b, fname);
 }
 
+static void write_vmlinux_export_c_file(struct module *mod)
+{
+	struct buffer buf = { };
+
+	buf_printf(&buf,
+		   "#include <linux/export-internal.h>\n");
+
+	add_exported_symbols(&buf, mod);
+	write_if_changed(&buf, ".vmlinux.export.c");
+	free(buf.p);
+}
+
 /* do sanity checks, and generate *.mod.c file */
 static void write_mod_c_file(struct module *mod)
 {
@@ -1235,6 +1254,7 @@ static void write_mod_c_file(struct module *mod)
 	check_exports(mod);
 
 	add_header(&buf, mod);
+	add_exported_symbols(&buf, mod);
 	add_versions(&buf, mod);
 	add_depends(&buf, mod);
 	add_moddevtable(&buf, mod);
@@ -1432,9 +1452,9 @@ int main(int argc, char **argv)
 		if (mod->from_dump)
 			continue;
 
-		check_symversions(mod);
-
-		if (!mod->is_vmlinux)
+		if (mod->is_vmlinux)
+			write_vmlinux_export_c_file(mod);
+		else
 			write_mod_c_file(mod);
 	}
 
Signature · 1 line
2.32.0
#11 lore raw

PATCH v5 5/12 kbuild: stop merging *.symversions

In reply to Masahiro Yamada

Now modpost reads symbol versions from .*.cmd files.

The merged *.symversions are no longer needed.

---

(no changes since v1)

 scripts/Makefile.build  | 21 ++-------------------
 scripts/link-vmlinux.sh | 19 -------------------
 2 files changed, 2 insertions(+), 38 deletions(-)
scripts/Makefile.build +2 -19
index ddd9080fc028..dff9220135c4 100644
@@ -390,17 +390,6 @@ $(obj)/%.asn1.c $(obj)/%.asn1.h: $(src)/%.asn1 $(objtree)/scripts/asn1_compiler
 $(subdir-builtin): $(obj)/%/built-in.a: $(obj)/% ;
 $(subdir-modorder): $(obj)/%/modules.order: $(obj)/% ;
 
-# combine symversions for later processing
-ifeq ($(CONFIG_LTO_CLANG) $(CONFIG_MODVERSIONS),y y)
-      cmd_update_lto_symversions =					\
-	rm -f [email protected]						\
-	$(foreach n, $(filter-out FORCE,$^),				\
-		$(if $(shell test -s $(n).symversions && echo y),	\
-			; cat $(n).symversions >> [email protected]))
-else
-      cmd_update_lto_symversions = echo >/dev/null
-endif
-
 #
 # Rule to compile a set of .o files into one .a file (without symbol table)
 #
@@ -408,11 +397,8 @@ endif
 quiet_cmd_ar_builtin = AR      $@
       cmd_ar_builtin = rm -f $@; $(AR) cDPrST $@ $(real-prereqs)
 
-quiet_cmd_ar_and_symver = AR      $@
-      cmd_ar_and_symver = $(cmd_update_lto_symversions); $(cmd_ar_builtin)
-
 $(obj)/built-in.a: $(real-obj-y) FORCE
-	$(call if_changed,ar_and_symver)
+	$(call if_changed,ar_builtin)
 
 #
 # Rule to create modules.order file
@@ -432,16 +418,13 @@ $(obj)/modules.order: $(obj-m) FORCE
 #
 # Rule to compile a set of .o files into one .a file (with symbol table)
 #
-quiet_cmd_ar_lib = AR      $@
-      cmd_ar_lib = $(cmd_update_lto_symversions); $(cmd_ar)
 
 $(obj)/lib.a: $(lib-y) FORCE
-	$(call if_changed,ar_lib)
+	$(call if_changed,ar)
 
 ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),)
 quiet_cmd_link_multi-m = AR [M]  $@
 cmd_link_multi-m =						\
-	$(cmd_update_lto_symversions);				\
 	rm -f $@; 						\
 	$(AR) cDPrsT $@ @$(patsubst %.o,%.mod,$@)
 else
scripts/link-vmlinux.sh +0 -19
index 7313cbd755df..47e5336d0c75 100755
@@ -56,20 +56,6 @@ gen_initcalls()
 		> .tmp_initcalls.lds
 }
 
-# If CONFIG_LTO_CLANG is selected, collect generated symbol versions into
-# .tmp_symversions.lds
-gen_symversions()
-{
-	info GEN .tmp_symversions.lds
-	rm -f .tmp_symversions.lds
-
-	for o in ${KBUILD_VMLINUX_OBJS} ${KBUILD_VMLINUX_LIBS}; do
-		if [ -f ${o}.symversions ]; then
-			cat ${o}.symversions >> .tmp_symversions.lds
-		fi
-	done
-}
-
 # Link of vmlinux.o used for section mismatch analysis
 # ${1} output file
 modpost_link()
@@ -88,10 +74,6 @@ modpost_link()
 		gen_initcalls
 		lds="-T .tmp_initcalls.lds"
 
-		if is_enabled CONFIG_MODVERSIONS; then
-			gen_symversions
-		fi
-
 		# This might take a while, so indicate that we're doing
 		# an LTO link
 		info LTO ${1}
@@ -307,7 +289,6 @@ cleanup()
 	rm -f .btf.*
 	rm -f .tmp_System.map
 	rm -f .tmp_initcalls.lds
-	rm -f .tmp_symversions.lds
 	rm -f .tmp_vmlinux*
 	rm -f System.map
 	rm -f vmlinux
Signature · 1 line
2.32.0
#12 lore raw

PATCH v5 6/12 genksyms: adjust the output format to modpost

In reply to Masahiro Yamada

Make genksyms output symbol versions in the format modpost expects, so the 'sed' is unneeded.

This commit makes *.symversions completely unneeded.

I will keep *.symversions in .gitignore and 'make clean' for a while. Otherwise, 'git status' might be surprising.

---

(no changes since v2)

Changes in v2:
- New patch

 scripts/Makefile.build      | 6 ------
 scripts/genksyms/genksyms.c | 2 +-
 2 files changed, 1 insertion(+), 7 deletions(-)
scripts/Makefile.build +0 -6
index dff9220135c4..461998a2ad2b 100644
@@ -165,16 +165,10 @@ ifdef CONFIG_MODVERSIONS
 # o modpost will extract versions from that file and create *.c files that will
 #   be compiled and linked to the kernel and/or modules.
 
-genksyms_format := __crc_\(.*\) = \(.*\);
-
 gen_symversions =								\
 	if $(NM) $@ 2>/dev/null | grep -q __ksymtab; then			\
 		$(call cmd_gensymtypes_$(1),$(KBUILD_SYMTYPES),$(@:.o=.symtypes)) \
-		    > [email protected];						\
-		sed -n 's/$(genksyms_format)/$(pound)SYMVER \1 \2/p' [email protected] \
 			>> $(dot-target).cmd;					\
-	else									\
-		rm -f [email protected];						\
 	fi
 
 cmd_gen_symversions_c =	$(call gen_symversions,c)
scripts/genksyms/genksyms.c +1 -1
index 67b23cc0df0f..f5dfdb9d80e9 100644
@@ -680,7 +680,7 @@ void export_symbol(const char *name)
 		if (flag_dump_defs)
 			fputs(">\n", debugfile);
 
-		printf("__crc_%s = 0x%08lx;\n", name, crc);
+		printf("#SYMVER %s 0x%08lx\n", name, crc);
 	}
 }
 
Signature · 1 line
2.32.0
#13 lore raw

PATCH v5 7/12 kbuild: do not create *.prelink.o for Clang LTO or IBT

In reply to Masahiro Yamada

When CONFIG_LTO_CLANG=y, additional intermediate *.prelink.o is created for each module. Also, objtool is postponed until LLVM IR is converted to ELF.

CONFIG_X86_KERNEL_IBT works in a similar way to postpone objtool until objects are merged together.

This commit stops generating *.prelink.o, so the build flow will look similar with/without LTO.

The following figures show how the LTO build currently works, and how this commit is changing it.

Current build flow
==================

[1] single-object module

                                      $(LD)
           $(CC)                     +objtool              $(LD)
    foo.c --------------------> foo.o -----> foo.prelink.o -----> foo.ko
                              (LLVM IR)          (ELF)       |
                                                             |
                                                 foo.mod.o --/

[2] multi-object module

                                      $(LD)
           $(CC)         $(AR)       +objtool               $(LD)
    foo1.c -----> foo1.o -----> foo.o -----> foo.prelink.o -----> foo.ko
                           |  (archive)          (ELF)       |
    foo2.c -----> foo2.o --/                                 |
                 (LLVM IR)                       foo.mod.o --/

One confusion is foo.o in multi-object module is an archive despite of its suffix.

New build flow
==============

[1] single-object module

  Since there is only one object, there is no need to keep the LLVM IR.
  Use $(CC)+$(LD) to generate an ELF object in one build rule. When LTO
  is disabled, $(LD) is unneeded because $(CC) produces an ELF object.
               $(CC)+$(LD)+objtool              $(LD)
    foo.c ----------------------------> foo.o ---------> foo.ko
                                        (ELF)     |
                                                  |
                                      foo.mod.o --/

[2] multi-object module

  Previously, $(AR) was used to combine LLVM bitcode into an archive,
  but there was no technical reason to do so. Use $(LD) to merge them
  into a single ELF object.
                               $(LD)
             $(CC)            +objtool          $(LD)
    foo1.c ---------> foo1.o ---------> foo.o ---------> foo.ko
                                 |      (ELF)     |
    foo2.c ---------> foo2.o ----/                |
                     (LLVM IR)        foo.mod.o --/

---

(no changes since v2)

Changes in v2:
- replace the chain of $(if ...) with $(and )

 scripts/Kbuild.include    |  4 +++
 scripts/Makefile.build    | 58 ++++++++++++---------------------------
 scripts/Makefile.lib      |  7 -----
 scripts/Makefile.modfinal |  5 ++--
 scripts/Makefile.modpost  |  9 ++----
 scripts/mod/modpost.c     |  7 -----
 6 files changed, 25 insertions(+), 65 deletions(-)
scripts/Kbuild.include +4 -0
index 3514c2149e9d..455a0a6ce12d 100644
@@ -15,6 +15,10 @@ pound := \#
 # Name of target with a '.' as filename prefix. foo/bar.o => foo/.bar.o
 dot-target = $(dir $@).$(notdir $@)
 
+###
+# Name of target with a '.tmp_' as filename prefix. foo/bar.o => foo/.tmp_bar.o
+tmp-target = $(dir $@).tmp_$(notdir $@)
+
 ###
 # The temporary file to save gcc -MMD generated dependencies must not
 # contain a comma
scripts/Makefile.build +17 -41

103 lines not on this page · raw message

scripts/Makefile.lib +0 -7
index 0453a1904646..f75138385449 100644
@@ -225,13 +225,6 @@ dtc_cpp_flags  = -Wp,-MMD,$(depfile).pre.tmp -nostdinc                    \
 		 $(addprefix -I,$(DTC_INCLUDE))                          \
 		 -undef -D__DTS__
 
-ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),)
-# With CONFIG_LTO_CLANG, .o files in modules might be LLVM bitcode, so we
-# need to run LTO to compile them into native code (.lto.o) before further
-# processing.
-mod-prelink-ext := .prelink
-endif
-
 # Useful for describing the dependency of composite objects
 # Usage:
 #   $(call multi_depend, multi_used_targets, suffix_to_remove, suffix_to_add)
scripts/Makefile.modfinal +2 -3
index 7f39599e9fae..35100e981f4a 100644
@@ -9,7 +9,7 @@ __modfinal:
 include include/config/auto.conf
 include $(srctree)/scripts/Kbuild.include
 
-# for c_flags and mod-prelink-ext
+# for c_flags
 include $(srctree)/scripts/Makefile.lib
 
 # find all modules listed in modules.order
@@ -54,9 +54,8 @@ if_changed_except = $(if $(call newer_prereqs_except,$(2))$(cmd-check),      \
 	$(cmd);                                                              \
 	printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd, @:)
 
-
 # Re-generate module BTFs if either module's .ko or vmlinux changed
-$(modules): %.ko: %$(mod-prelink-ext).o %.mod.o scripts/module.lds $(if $(KBUILD_BUILTIN),vmlinux) FORCE
+$(modules): %.ko: %.o %.mod.o scripts/module.lds $(if $(KBUILD_BUILTIN),vmlinux) FORCE
 	+$(call if_changed_except,ld_ko_o,vmlinux)
 ifdef CONFIG_DEBUG_INFO_BTF_MODULES
 	+$(if $(newer-prereqs),$(call cmd,btf_ko))
scripts/Makefile.modpost +2 -7
index 48585c4d04ad..f2ce411acd59 100644
@@ -41,9 +41,6 @@ __modpost:
 include include/config/auto.conf
 include $(srctree)/scripts/Kbuild.include
 
-# for mod-prelink-ext
-include $(srctree)/scripts/Makefile.lib
-
 MODPOST = scripts/mod/modpost								\
 	$(if $(CONFIG_MODVERSIONS),-m)							\
 	$(if $(CONFIG_MODULE_SRCVERSION_ALL),-a)					\
@@ -118,8 +115,6 @@ $(input-symdump):
 	@echo >&2 '         Modules may not have dependencies or modversions.'
 	@echo >&2 '         You may get many unresolved symbol warnings.'
 
-modules := $(sort $(shell cat $(MODORDER)))
-
 # KBUILD_MODPOST_WARN can be set to avoid error out in case of undefined symbols
 ifneq ($(KBUILD_MODPOST_WARN)$(filter-out $(existing-input-symdump), $(input-symdump)),)
 MODPOST += -w
@@ -128,9 +123,9 @@ endif
 # Read out modules.order to pass in modpost.
 # Otherwise, allmodconfig would fail with "Argument list too long".
 quiet_cmd_modpost = MODPOST $@
-      cmd_modpost = sed 's/\.ko$$/$(mod-prelink-ext)\.o/' $< | $(MODPOST) -T -
+      cmd_modpost = sed 's/ko$$/o/' $< | $(MODPOST) -T -
 
-$(output-symdump): $(MODORDER) $(input-symdump) $(modules:.ko=$(mod-prelink-ext).o) FORCE
+$(output-symdump): $(MODORDER) $(input-symdump) FORCE
 	$(call if_changed,modpost)
 
 targets += $(output-symdump)
scripts/mod/modpost.c +0 -7
index 6b47a8723389..18f4d4a5a0ee 100644
@@ -716,10 +716,6 @@ static char *remove_dot(char *s)
 		size_t m = strspn(s + n + 1, "0123456789");
 		if (m && (s[n + m] == '.' || s[n + m] == 0))
 			s[n] = 0;
-
-		/* strip trailing .prelink */
-		if (strends(s, ".prelink"))
-			s[strlen(s) - 8] = '\0';
 	}
 	return s;
 }
@@ -839,9 +835,6 @@ static void read_symbols(const char *modname)
 		/* strip trailing .o */
 		tmp = NOFAIL(strdup(modname));
 		tmp[strlen(tmp) - 2] = '\0';
-		/* strip trailing .prelink */
-		if (strends(tmp, ".prelink"))
-			tmp[strlen(tmp) - 8] = '\0';
 		mod = new_module(tmp);
 		free(tmp);
 	}
Signature · 1 line
2.32.0
#14 lore raw

PATCH v5 8/12 kbuild: check static EXPORT_SYMBOL* by script instead of modpost

In reply to Masahiro Yamada

The 'static' specifier and EXPORT_SYMBOL() are an odd combination.

Commit 15bfc2348d54 ("modpost: check for static EXPORT_SYMBOL* functions") tried to detect it, but this check has false negatives.

Here is the sample code.

Makefile:

    obj-y += foo1.o foo2.o

foo1.c:

    #include <linux/export.h>
    static void foo(void) {}
    EXPORT_SYMBOL(foo);

foo2.c:

    void foo(void) {}

foo1.c exports the static symbol 'foo', but modpost cannot catch it because it is fooled by foo2.c, which has a global symbol with the same name.

s->is_static is cleared if a global symbol with the same name is found somewhere, but EXPORT_SYMBOL() and the global symbol do not necessarily belong to the same compilation unit.

This check should be done per compilation unit, but I do not know how to do it in modpost. modpost runs against vmlinux.o or modules, which merges multiple objects, then forgets their origin.

It is true modpost gets access to the lists of all the member objects (.vmlinux.objs and *.mod), but modpost cannot parse individual objects because they may not be ELF but LLVM IR when CONFIG_LTO_CLANG=y.

Add a simple bash script to parse the output from ${NM}. This works for CONFIG_LTO_CLANG=y because llvm-nm can dump symbols of LLVM IR files.

Revert 15bfc2348d54.

---

Changes in v5:
- Add more comments in the script

Changes in v4:
- New patch

 scripts/Makefile.build     |  4 +++
 scripts/check-local-export | 64 ++++++++++++++++++++++++++++++++++++++
 scripts/mod/modpost.c      | 28 +----------------
 3 files changed, 69 insertions(+), 27 deletions(-)
 create mode 100755 scripts/check-local-export
scripts/Makefile.build +4 -0
index 838ea5e83174..c2a173b3fd60 100644
@@ -244,9 +244,12 @@ cmd_gen_ksymdeps = \
 	$(CONFIG_SHELL) $(srctree)/scripts/gen_ksymdeps.sh $@ >> $(dot-target).cmd
 endif
 
+cmd_check_local_export = $(srctree)/scripts/check-local-export $@
+
 define rule_cc_o_c
 	$(call cmd_and_fixdep,cc_o_c)
 	$(call cmd,gen_ksymdeps)
+	$(call cmd,check_local_export)
 	$(call cmd,checksrc)
 	$(call cmd,checkdoc)
 	$(call cmd,gen_objtooldep)
@@ -257,6 +260,7 @@ endef
 define rule_as_o_S
 	$(call cmd_and_fixdep,as_o_S)
 	$(call cmd,gen_ksymdeps)
+	$(call cmd,check_local_export)
 	$(call cmd,gen_objtooldep)
 	$(call cmd,gen_symversions_S)
 endef
scripts/check-local-export new file +64 -0

64 lines not on this page · raw message

scripts/mod/modpost.c +1 -27
index 18f4d4a5a0ee..50dbd2e9939c 100644
@@ -212,7 +212,6 @@ struct symbol {
 	unsigned int crc;
 	bool crc_valid;
 	bool weak;
-	bool is_static;		/* true if symbol is not global */
 	bool is_gpl_only;	/* exported by EXPORT_SYMBOL_GPL */
 	char name[];
 };
@@ -242,7 +241,7 @@ static struct symbol *alloc_symbol(const char *name)
 
 	memset(s, 0, sizeof(*s));
 	strcpy(s->name, name);
-	s->is_static = true;
+
 	return s;
 }
 
@@ -875,20 +874,6 @@ static void read_symbols(const char *modname)
 					     sym_get_data(&info, sym));
 	}
 
-	// check for static EXPORT_SYMBOL_* functions && global vars
-	for (sym = info.symtab_start; sym < info.symtab_stop; sym++) {
-		unsigned char bind = ELF_ST_BIND(sym->st_info);
-
-		if (bind == STB_GLOBAL || bind == STB_WEAK) {
-			struct symbol *s =
-				find_symbol(remove_dot(info.strtab +
-						       sym->st_name));
-
-			if (s)
-				s->is_static = false;
-		}
-	}
-
 	check_sec_ref(mod, modname, &info);
 
 	if (!mod->is_vmlinux) {
@@ -1318,7 +1303,6 @@ static void read_dump(const char *fname)
 			mod->from_dump = true;
 		}
 		s = sym_add_exported(symname, mod, gpl_only);
-		s->is_static = false;
 		sym_set_crc(s, crc);
 		sym_update_namespace(symname, namespace);
 	}
@@ -1383,7 +1367,6 @@ int main(int argc, char **argv)
 	char *missing_namespace_deps = NULL;
 	char *dump_write = NULL, *files_source = NULL;
 	int opt;
-	int n;
 	LIST_HEAD(dump_lists);
 	struct dump_list *dl, *dl2;
 
@@ -1459,15 +1442,6 @@ int main(int argc, char **argv)
 	if (sec_mismatch_count && !sec_mismatch_warn_only)
 		error("Section mismatches detected.\n"
 		      "Set CONFIG_SECTION_MISMATCH_WARN_ONLY=y to allow them.\n");
-	for (n = 0; n < SYMBOL_HASH_SIZE; n++) {
-		struct symbol *s;
-
-		for (s = symbolhash[n]; s; s = s->next) {
-			if (s->is_static)
-				error("\"%s\" [%s] is a static EXPORT_SYMBOL\n",
-				      s->name, s->module->name);
-		}
-	}
 
 	if (nr_unresolved > MAX_UNRESOLVED_REPORTS)
 		warn("suppressed %u unresolved symbol warnings because there were too many)\n",
Signature · 1 line
2.32.0
#15 lore raw

PATCH v5 9/12 kbuild: make built-in.a rule robust against too long argument error

In reply to Masahiro Yamada

Kbuild runs at the top of objtree instead of changing the working directory to subdirectories. I think this design is nice overall but some commands have a scalability issue.

The build command of built-in.a is one of them whose length scales with:

    O(D * N)

Here, D is the length of the directory path (i.e. $(obj)/ prefix), N is the number of objects in the Makefile, O() is the big O notation.

The deeper directory the Makefile directory is located, the more easily it will hit the too long argument error.

We can make it better. Trim the $(obj)/ by Make's builtin function, and restore it by a shell command (sed).

With this, the command length scales with:

    O(D + N)

In-tree modules still have some room to the limit (ARG_MAX=2097152), but this is more future-proof for big modules in a deep directory.

For example, you can build i915 as builtin (CONFIG_DRM_I915=y) and compare drivers/gpu/drm/i915/.built-in.a.cmd with/without this commit.

---

(no changes since v2)

Changes in v2:
- New patch

 scripts/Makefile.build | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
scripts/Makefile.build +4 -1
index c2a173b3fd60..8f1a355df7aa 100644
@@ -374,7 +374,10 @@ $(subdir-modorder): $(obj)/%/modules.order: $(obj)/% ;
 #
 
 quiet_cmd_ar_builtin = AR      $@
-      cmd_ar_builtin = rm -f $@; $(AR) cDPrST $@ $(real-prereqs)
+      cmd_ar_builtin = rm -f $@; \
+		echo $(patsubst $(obj)/%,%,$(real-prereqs)) | \
+		sed -E 's:([^ ]+):$(obj)/\1:g' | \
+		xargs $(AR) cDPrST $@
 
 $(obj)/built-in.a: $(real-obj-y) FORCE
 	$(call if_changed,ar_builtin)
Signature · 1 line
2.32.0
#16 lore raw

PATCH v5 10/12 kbuild: make *.mod rule robust against too long argument error

In reply to Masahiro Yamada

Like built-in.a, the command length of the *.mod rule scales with the depth of the directory times the number of objects in the Makefile.

Add $(obj)/ by the shell command (awk) instead of by Make's builtin function.

In-tree modules still have some room to the limit (ARG_MAX=2097152), but this is more future-proof for big modules in a deep directory.

For example, you can build i915 as a module (CONFIG_DRM_I915=m) and compare drivers/gpu/drm/i915/.i915.mod.cmd with/without this commit.

The issue is more critical for external modules because the M= path can be very long as Jeff Johnson reported before [1].

[1] https://lore.kernel.org/linux-kbuild/[email protected]/

---

(no changes since v2)

Changes in v2:
- New patch

 scripts/Makefile.build | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
scripts/Makefile.build +2 -2
index 8f1a355df7aa..f546b5f1f33f 100644
@@ -270,8 +270,8 @@ $(obj)/%.o: $(src)/%.c $(recordmcount_source) FORCE
 	$(call if_changed_rule,cc_o_c)
 	$(call cmd,force_checksrc)
 
-cmd_mod = echo $(addprefix $(obj)/, $(call real-search, $*.o, .o, -objs -y -m)) | \
-	$(AWK) -v RS='( |\n)' '!x[$$0]++' > $@
+cmd_mod = echo $(call real-search, $*.o, .o, -objs -y -m) | \
+	$(AWK) -v RS='( |\n)' '!x[$$0]++ { print("$(obj)/"$$0) }' > $@
 
 $(obj)/%.mod: FORCE
 	$(call if_changed,mod)
Signature · 1 line
2.32.0
#17 lore raw

PATCH v5 11/12 kbuild: add cmd_and_savecmd macro

In reply to Masahiro Yamada

Separate out the command execution part of if_changed, as we did for if_changed_dep.

This allows us to reuse it in if_changed_rule.

  define rule_foo
          $(call cmd_and_savecmd,foo)
          $(call cmd,bar)
  endef

---

(no changes since v4)

Changes in v4:
- New.
Resent of my previous submission.
https://lore.kernel.org/all/[email protected]/

 scripts/Kbuild.include | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
scripts/Kbuild.include +4 -2
index 455a0a6ce12d..ece44b735061 100644
@@ -142,9 +142,11 @@ check-FORCE = $(if $(filter FORCE, $^),,$(warning FORCE prerequisite is missing)
 if-changed-cond = $(newer-prereqs)$(cmd-check)$(check-FORCE)
 
 # Execute command if command has changed or prerequisite(s) are updated.
-if_changed = $(if $(if-changed-cond),                                        \
+if_changed = $(if $(if-changed-cond),$(cmd_and_savecmd),@:)
+
+cmd_and_savecmd =                                                            \
 	$(cmd);                                                              \
-	printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd, @:)
+	printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd
 
 # Execute the command and also postprocess generated .d dependencies file.
 if_changed_dep = $(if $(if-changed-cond),$(cmd_and_fixdep),@:)
Signature · 1 line
2.32.0
Reviewed-by on v5 11/12
#18 lore raw

Re: [PATCH v5 11/12] kbuild: add cmd_and_savecmd macro

Quoted 45 lines · Masahiro Yamada

Separate out the command execution part of if_changed, as we did for if_changed_dep.

This allows us to reuse it in if_changed_rule.

  define rule_foo
          $(call cmd_and_savecmd,foo)
          $(call cmd,bar)
  endef

---

(no changes since v4)

Changes in v4:
- New.
Resent of my previous submission.
https://lore.kernel.org/all/[email protected]/

 scripts/Kbuild.include | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
scripts/Kbuild.include +4 -2
index 455a0a6ce12d..ece44b735061 100644
@@ -142,9 +142,11 @@ check-FORCE = $(if $(filter FORCE, $^),,$(warning FORCE prerequisite is missing)
 if-changed-cond = $(newer-prereqs)$(cmd-check)$(check-FORCE)
 
 # Execute command if command has changed or prerequisite(s) are updated.
-if_changed = $(if $(if-changed-cond),                                        \
+if_changed = $(if $(if-changed-cond),$(cmd_and_savecmd),@:)
+
+cmd_and_savecmd =                                                            \
 	$(cmd);                                                              \
-	printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd, @:)
+	printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd
 
 # Execute the command and also postprocess generated .d dependencies file.
 if_changed_dep = $(if $(if-changed-cond),$(cmd_and_fixdep),@:)
Signature · 1 line
2.32.0
#19 lore raw

PATCH v5 12/12 kbuild: rebuild multi-object modules when objtool is updated

In reply to Masahiro Yamada

When CONFIG_LTO_CLANG or CONFIG_X86_KERNEL_IBT is enabled, objtool for multi-object modules is postponed until the objects are linked together.

Make sure to re-run objtool and re-link multi-object modules when objtool is updated.

---

(no changes since v4)

Changes in v4:
- New
Resent of my previous submission
https://lore.kernel.org/linux-kbuild/[email protected]/

 scripts/Makefile.build | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
scripts/Makefile.build +8 -3
index f546b5f1f33f..4e6902e099e8 100644
@@ -404,13 +404,18 @@ $(obj)/modules.order: $(obj-m) FORCE
 $(obj)/lib.a: $(lib-y) FORCE
 	$(call if_changed,ar)
 
-quiet_cmd_link_multi-m = LD [M]  $@
-      cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ @$(patsubst %.o,%.mod,$@) $(cmd_objtool)
+quiet_cmd_ld_multi_m = LD [M]  $@
+      cmd_ld_multi_m = $(LD) $(ld_flags) -r -o $@ @$(patsubst %.o,%.mod,$@) $(cmd_objtool)
+
+define rule_ld_multi_m
+	$(call cmd_and_savecmd,ld_multi_m)
+	$(call cmd,gen_objtooldep)
+endef
 
 $(multi-obj-m): objtool-enabled := $(delay-objtool)
 $(multi-obj-m): part-of-module := y
 $(multi-obj-m): %.o: %.mod FORCE
-	$(call if_changed,link_multi-m)
+	$(call if_changed_rule,ld_multi_m)
 $(call multi_depend, $(multi-obj-m), .o, -objs -y -m)
 
 targets := $(filter-out $(PHONY), $(targets))
Signature · 1 line
2.32.0
Reviewed-by on v5 12/12
#20 lore raw

Re: [PATCH v5 12/12] kbuild: rebuild multi-object modules when objtool is updated

Quoted 50 lines · Masahiro Yamada

When CONFIG_LTO_CLANG or CONFIG_X86_KERNEL_IBT is enabled, objtool for multi-object modules is postponed until the objects are linked together.

Make sure to re-run objtool and re-link multi-object modules when objtool is updated.

---

(no changes since v4)

Changes in v4:
- New
Resent of my previous submission
https://lore.kernel.org/linux-kbuild/[email protected]/

 scripts/Makefile.build | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
scripts/Makefile.build +8 -3
index f546b5f1f33f..4e6902e099e8 100644
@@ -404,13 +404,18 @@ $(obj)/modules.order: $(obj-m) FORCE
 $(obj)/lib.a: $(lib-y) FORCE
 	$(call if_changed,ar)
 
-quiet_cmd_link_multi-m = LD [M]  $@
-      cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ @$(patsubst %.o,%.mod,$@) $(cmd_objtool)
+quiet_cmd_ld_multi_m = LD [M]  $@
+      cmd_ld_multi_m = $(LD) $(ld_flags) -r -o $@ @$(patsubst %.o,%.mod,$@) $(cmd_objtool)
+
+define rule_ld_multi_m
+	$(call cmd_and_savecmd,ld_multi_m)
+	$(call cmd,gen_objtooldep)
+endef
 
 $(multi-obj-m): objtool-enabled := $(delay-objtool)
 $(multi-obj-m): part-of-module := y
 $(multi-obj-m): %.o: %.mod FORCE
-	$(call if_changed,link_multi-m)
+	$(call if_changed_rule,ld_multi_m)
 $(call multi_depend, $(multi-obj-m), .o, -objs -y -m)
 
 targets := $(filter-out $(PHONY), $(targets))
Signature · 1 line
2.32.0
maintainer Tested-by on the v5 series
#21 lore raw
In reply to Masahiro Yamada

Hi Masahiro,

Quoted 28 lines · Masahiro Yamada

This is the third batch of cleanups in this development cycle.

Major changes in v5:
- Fix build errors
- More cleanups

Major changes in v4:
- Move static EXPORT_SYMBOL check to a script
- Some refactoring

Major changes in v3:

- Generate symbol CRCs as C code, and remove CONFIG_MODULE_REL_CRCS.

Major changes in v2:

- V1 did not work with CONFIG_MODULE_REL_CRCS.
I fixed this for v2.

- Reflect some review comments in v1

- Refactor the code more

- Avoid too long argument error

Quoted 14 lines

Masahiro Yamada (12):
modpost: split the section mismatch checks into section-check.c
modpost: add sym_find_with_module() helper
modpost: extract symbol versions from *.cmd files
kbuild: link symbol CRCs at final link, removing
CONFIG_MODULE_REL_CRCS
kbuild: stop merging *.symversions
genksyms: adjust the output format to modpost
kbuild: do not create *.prelink.o for Clang LTO or IBT
kbuild: check static EXPORT_SYMBOL* by script instead of modpost
kbuild: make built-in.a rule robust against too long argument error
kbuild: make *.mod rule robust against too long argument error
kbuild: add cmd_and_savecmd macro
kbuild: rebuild multi-object modules when objtool is updated

I applied this series to your latest kbuild branch, built an Arch Linux x86_64 config and Fedora aarch64 config with CONFIG_LTO_CLANG_FULL=y then CONFIG_LTO_CLANG_THIN=y, and boot tested each kernel in QEMU with a full userspace. I did not notice any new warnings/errors and everything appears to work fine.

Cheers,
Nathan

#22 lore raw
Quoted 53 lines · Nathan Chancellor

Hi Masahiro,

Quoted 28 lines · Masahiro Yamada

This is the third batch of cleanups in this development cycle.

Major changes in v5:
- Fix build errors
- More cleanups

Major changes in v4:
- Move static EXPORT_SYMBOL check to a script
- Some refactoring

Major changes in v3:

- Generate symbol CRCs as C code, and remove CONFIG_MODULE_REL_CRCS.

Major changes in v2:

- V1 did not work with CONFIG_MODULE_REL_CRCS.
I fixed this for v2.

- Reflect some review comments in v1

- Refactor the code more

- Avoid too long argument error

Quoted 14 lines

Masahiro Yamada (12):
modpost: split the section mismatch checks into section-check.c
modpost: add sym_find_with_module() helper
modpost: extract symbol versions from *.cmd files
kbuild: link symbol CRCs at final link, removing
CONFIG_MODULE_REL_CRCS
kbuild: stop merging *.symversions
genksyms: adjust the output format to modpost
kbuild: do not create *.prelink.o for Clang LTO or IBT
kbuild: check static EXPORT_SYMBOL* by script instead of modpost
kbuild: make built-in.a rule robust against too long argument error
kbuild: make *.mod rule robust against too long argument error
kbuild: add cmd_and_savecmd macro
kbuild: rebuild multi-object modules when objtool is updated

I applied this series to your latest kbuild branch, built an Arch Linux x86_64 config and Fedora aarch64 config with CONFIG_LTO_CLANG_FULL=y then CONFIG_LTO_CLANG_THIN=y, and boot tested each kernel in QEMU with a full userspace. I did not notice any new warnings/errors and everything appears to work fine.

Quoted 4 lines

Cheers,
Nathan

Thanks for your testing.

I ended up with v6 due to a build issue
of CONFIG_TRIM_UNUSED_KSYMS,
but this patch set is getting in good shape.

Thanks.

Signature · 2 lines
Best Regards
Masahiro Yamada
#23 lore raw
Quoted 34 lines · Masahiro Yamada
Quoted 32 lines · Nathan Chancellor

Hi Masahiro,

Quoted 28 lines · Masahiro Yamada

This is the third batch of cleanups in this development cycle.

Major changes in v5:
- Fix build errors
- More cleanups

Major changes in v4:
- Move static EXPORT_SYMBOL check to a script
- Some refactoring

Major changes in v3:

- Generate symbol CRCs as C code, and remove CONFIG_MODULE_REL_CRCS.

Major changes in v2:

- V1 did not work with CONFIG_MODULE_REL_CRCS.
I fixed this for v2.

- Reflect some review comments in v1

- Refactor the code more

- Avoid too long argument error

Quoted 15 lines
Quoted 6 lines
Quoted 6 lines

Masahiro Yamada (12):
modpost: split the section mismatch checks into section-check.c
modpost: add sym_find_with_module() helper
modpost: extract symbol versions from *.cmd files
kbuild: link symbol CRCs at final link, removing
CONFIG_MODULE_REL_CRCS

Quoted 9 lines
Quoted 8 lines

kbuild: stop merging *.symversions
genksyms: adjust the output format to modpost
kbuild: do not create *.prelink.o for Clang LTO or IBT
kbuild: check static EXPORT_SYMBOL* by script instead of modpost
kbuild: make built-in.a rule robust against too long argument error
kbuild: make *.mod rule robust against too long argument error
kbuild: add cmd_and_savecmd macro
kbuild: rebuild multi-object modules when objtool is updated

The first two applied.

I will send v6 for the rest.

Signature · 2 lines
Best Regards
Masahiro Yamada