Skip to contentSkip to conversation

modules: introduce the MODULE_SCMVERSION config

PATCH Superseded Will McVicker 5 participants 9 messages Jan 8, 2021 – Jan 20, 2021

Subsystems (2)

v5 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

PATCH v5 1/1 modules: introduce the MODULE_SCMVERSION config

Config MODULE_SCMVERSION introduces a new module attribute -- scmversion -- which can be used to identify a given module's SCM version. This is very useful for developers that update their kernel independently from their kernel modules or vice-versa since the SCM version provided by UTS_RELEASE (uname -r) will now differ from the module's vermagic attribute.

For example, we have a CI setup that tests new kernel changes on the hikey960 and db845c devices without updating their kernel modules. When these tests fail, we need to be able to identify the exact device configuration the test was using. By including MODULE_SCMVERSION, we can identify the exact kernel and modules' SCM versions for debugging the failures.

Additionally, by exposing the SCM version via the sysfs node /sys/module/MODULENAME/scmversion, one can also verify the SCM versions of the modules loaded from the initramfs. Currently, modinfo can only retrieve module attributes from the module's ko on disk and not from the actual module that is loaded in RAM.

You can retrieve the SCM version in two ways,

1) By using modinfo:

Quoted 1 line

modinfo -F scmversion MODULENAME

2) By module sysfs node:

Quoted 1 line

cat /sys/module/MODULENAME/scmversion

---
 Documentation/ABI/stable/sysfs-module | 18 ++++++++++++++++++
 include/linux/module.h                |  1 +
 init/Kconfig                          | 14 ++++++++++++++
 kernel/module.c                       |  2 ++
 scripts/Makefile.modpost              | 22 ++++++++++++++++++++++
 scripts/mod/modpost.c                 | 24 +++++++++++++++++++++++-
 6 files changed, 80 insertions(+), 1 deletion(-)
Documentation/ABI/stable/sysfs-module +18 -0
index 6272ae5fb366..a75d137e79f4 100644
@@ -32,3 +32,21 @@ Description:
 		Note: If the module is built into the kernel, or if the
 		CONFIG_MODULE_UNLOAD kernel configuration value is not enabled,
 		this file will not be present.
+
+What:		/sys/module/MODULENAME/scmversion
+Date:		November 2020
+KernelVersion:	5.12
+Contact:	Will McVicker <[email protected]>
+Description:	This read-only file will appear if modpost was supplied with an
+		SCM version for the module. It can be enabled with the config
+		MODULE_SCMVERSION. The SCM version is retrieved by
+		scripts/setlocalversion, which means that the presence of this
+		file depends on CONFIG_LOCALVERSION_AUTO=y. When read, the SCM
+		version that the module was compiled with is returned. The SCM
+		version is returned in the following format::
+
+		===
+		Git:		g[a-f0-9]\+(-dirty)\?
+		Mercurial:	hg[a-f0-9]\+(-dirty)\?
+		Subversion:	svn[0-9]\+
+		===
include/linux/module.h +1 -0
index 7a0bcb5b1ffc..3b1612193cf9 100644
@@ -372,6 +372,7 @@ struct module {
 	struct module_attribute *modinfo_attrs;
 	const char *version;
 	const char *srcversion;
+	const char *scmversion;
 	struct kobject *holders_dir;
 
 	/* Exported symbols */
init/Kconfig +14 -0
index b77c60f8b963..3d9dac3c4e8f 100644
@@ -2131,6 +2131,20 @@ config MODULE_SRCVERSION_ALL
 	  the version).  With this option, such a "srcversion" field
 	  will be created for all modules.  If unsure, say N.
 
+config MODULE_SCMVERSION
+	bool "SCM version for modules"
+	depends on LOCALVERSION_AUTO
+	help
+	  This enables the module attribute "scmversion" which can be used
+	  by developers to identify the SCM version of a given module, e.g.
+	  git sha1 or hg sha1. The SCM version can be queried by modinfo or
+	  via the sysfs node: /sys/modules/MODULENAME/scmversion. This is
+	  useful when the kernel or kernel modules are updated separately
+	  since that causes the vermagic of the kernel and the module to
+	  differ.
+
+	  If unsure, say N.
+
 config MODULE_SIG
 	bool "Module signature verification"
 	select MODULE_SIG_FORMAT
kernel/module.c +2 -0
index 4bf30e4b3eaa..d0b359c7e9c9 100644
@@ -837,6 +837,7 @@ static struct module_attribute modinfo_##field = {                    \
 
 MODINFO_ATTR(version);
 MODINFO_ATTR(srcversion);
+MODINFO_ATTR(scmversion);
 
 static char last_unloaded_module[MODULE_NAME_LEN+1];
 
@@ -1298,6 +1299,7 @@ static struct module_attribute *modinfo_attrs[] = {
 	&module_uevent,
 	&modinfo_version,
 	&modinfo_srcversion,
+	&modinfo_scmversion,
 	&modinfo_initstate,
 	&modinfo_coresize,
 	&modinfo_initsize,
scripts/Makefile.modpost +22 -0
index f54b6ac37ac2..f1126b60adb7 100644
@@ -66,6 +66,7 @@ ifeq ($(KBUILD_EXTMOD),)
 
 input-symdump := vmlinux.symvers
 output-symdump := Module.symvers
+module_srcpath := $(srctree)
 
 else
 
@@ -77,6 +78,17 @@ src := $(obj)
 include $(if $(wildcard $(KBUILD_EXTMOD)/Kbuild), \
              $(KBUILD_EXTMOD)/Kbuild, $(KBUILD_EXTMOD)/Makefile)
 
+# Get the external module's source path. KBUILD_EXTMOD could either be an
+# absolute path or relative path from $(srctree). This makes sure that we
+# aren't using a relative path from a separate working directory (O= or
+# KBUILD_OUTPUT) since that may not be the actual module's SCM project path.
+# So check the path relative to $(srctree) first.
+ifneq ($(realpath $(srctree)/$(KBUILD_EXTMOD) 2>/dev/null),)
+	module_srcpath := $(srctree)/$(KBUILD_EXTMOD)
+else
+	module_srcpath := $(KBUILD_EXTMOD)
+endif
+
 # modpost option for external modules
 MODPOST += -e
 
@@ -85,6 +97,16 @@ output-symdump := $(KBUILD_EXTMOD)/Module.symvers
 
 endif
 
+ifeq ($(CONFIG_MODULE_SCMVERSION),y)
+# Get the SCM version of the module. Sed verifies setlocalversion returns
+# a proper revision based on the SCM type, e.g. git, mercurial, or svn.
+module_scmversion := $(shell $(srctree)/scripts/setlocalversion $(module_srcpath) | \
+	sed -n 's/.*-\(\(g\|hg\)[a-fA-F0-9]\+\(-dirty\)\?\|svn[0-9]\+\).*/\1/p')
+ifneq ($(module_scmversion),)
+MODPOST += -v$(module_scmversion)
+endif
+endif
+
 # modpost options for modules (both in-kernel and external)
 MODPOST += \
 	$(addprefix -i ,$(wildcard $(input-symdump))) \
scripts/mod/modpost.c +23 -1
index d6c81657d695..489b65bc37de 100644
@@ -30,6 +30,8 @@ static int have_vmlinux = 0;
 static int all_versions = 0;
 /* If we are modposting external module set to 1 */
 static int external_module = 0;
+#define MODULE_SCMVERSION_SIZE 64
+static char module_scmversion[MODULE_SCMVERSION_SIZE];
 /* Only warn about unresolved symbols */
 static int warn_unresolved = 0;
 /* How a symbol is exported */
@@ -2264,6 +2266,20 @@ static void add_intree_flag(struct buffer *b, int is_intree)
 		buf_printf(b, "\nMODULE_INFO(intree, \"Y\");\n");
 }
 
+/**
+ * add_scmversion() - Adds the MODULE_INFO macro for the scmversion.
+ * @b: Buffer to append to.
+ *
+ * This function fills in the module attribute `scmversion` for the kernel
+ * module. This is useful for determining a given module's SCM version on
+ * device via /sys/modules/<module>/scmversion and/or using the modinfo tool.
+ */
+static void add_scmversion(struct buffer *b)
+{
+	if (module_scmversion[0] != '\0')
+		buf_printf(b, "\nMODULE_INFO(scmversion, \"%s\");\n", module_scmversion);
+}
+
 /* Cannot check for assembler */
 static void add_retpoline(struct buffer *b)
 {
@@ -2546,7 +2562,7 @@ int main(int argc, char **argv)
 	struct dump_list *dump_read_start = NULL;
 	struct dump_list **dump_read_iter = &dump_read_start;
 
-	while ((opt = getopt(argc, argv, "ei:mnT:o:awENd:")) != -1) {
+	while ((opt = getopt(argc, argv, "ei:mnT:o:awENd:v:")) != -1) {
 		switch (opt) {
 		case 'e':
 			external_module = 1;
@@ -2584,6 +2600,11 @@ int main(int argc, char **argv)
 		case 'd':
 			missing_namespace_deps = optarg;
 			break;
+		case 'v':
+			if (!optarg)
+				fatal("'-v' requires an argument defining the SCM version.");
+			strncpy(module_scmversion, optarg, sizeof(module_scmversion) - 1);
+			break;
 		default:
 			exit(1);
 		}
@@ -2630,6 +2651,7 @@ int main(int argc, char **argv)
 		add_depends(&buf, mod);
 		add_moddevtable(&buf, mod);
 		add_srcversion(&buf, mod);
+		add_scmversion(&buf);
 
 		sprintf(fname, "%s.mod.c", mod->name);
 		write_if_changed(&buf, fname);
Signature · 1 line
2.30.0.284.gd98b1dd5eaa7-goog
#1 lore raw

+++ Will McVicker [08/01/21 00:30 +0000]:

Quoted 20 lines

Config MODULE_SCMVERSION introduces a new module attribute -- scmversion -- which can be used to identify a given module's SCM version. This is very useful for developers that update their kernel independently from their kernel modules or vice-versa since the SCM version provided by UTS_RELEASE (uname -r) will now differ from the module's vermagic attribute.

For example, we have a CI setup that tests new kernel changes on the hikey960 and db845c devices without updating their kernel modules. When these tests fail, we need to be able to identify the exact device configuration the test was using. By including MODULE_SCMVERSION, we can identify the exact kernel and modules' SCM versions for debugging the failures.

Additionally, by exposing the SCM version via the sysfs node /sys/module/MODULENAME/scmversion, one can also verify the SCM versions of the modules loaded from the initramfs. Currently, modinfo can only retrieve module attributes from the module's ko on disk and not from the actual module that is loaded in RAM.

Quoted 8 lines

You can retrieve the SCM version in two ways,

1) By using modinfo:

Quoted 1 line

modinfo -F scmversion MODULENAME

2) By module sysfs node:

Quoted 1 line

cat /sys/module/MODULENAME/scmversion

Hi Will,

Thanks for v5, I'm fine with this patch now that we've made it a configurable developer/debug option to supply an scmversion field for in-tree modules (although, this currently works for out-of-tree modules too since we based module_srcpath on KBUILD_EXTMOD in the external module case). I basically see this option as an alternative to CONFIG_MODULE_SRCVERSION_ALL to aid distro developers debug issues when the kernel and in-tree modules are updated separately.

In any case, since there was pushback in our earlier discussions, I'd like to ask if there are any remaining objections to this patch.

Masahiro, are you fine with the Makefile and modpost changes?

Thanks,

Jessica

Quoted 205 lines

---

Documentation/ABI/stable/sysfs-module | 18 ++++++++++++++++++
include/linux/module.h                |  1 +
init/Kconfig                          | 14 ++++++++++++++
kernel/module.c                       |  2 ++
scripts/Makefile.modpost              | 22 ++++++++++++++++++++++
scripts/mod/modpost.c                 | 24 +++++++++++++++++++++++-
6 files changed, 80 insertions(+), 1 deletion(-)
Documentation/ABI/stable/sysfs-module +18 -0
index 6272ae5fb366..a75d137e79f4 100644
@@ -32,3 +32,21 @@ Description:
 		Note: If the module is built into the kernel, or if the
 		CONFIG_MODULE_UNLOAD kernel configuration value is not enabled,
 		this file will not be present.
+
+What:		/sys/module/MODULENAME/scmversion
+Date:		November 2020
+KernelVersion:	5.12
+Contact:	Will McVicker <[email protected]>
+Description:	This read-only file will appear if modpost was supplied with an
+		SCM version for the module. It can be enabled with the config
+		MODULE_SCMVERSION. The SCM version is retrieved by
+		scripts/setlocalversion, which means that the presence of this
+		file depends on CONFIG_LOCALVERSION_AUTO=y. When read, the SCM
+		version that the module was compiled with is returned. The SCM
+		version is returned in the following format::
+
+		===
+		Git:		g[a-f0-9]\+(-dirty)\?
+		Mercurial:	hg[a-f0-9]\+(-dirty)\?
+		Subversion:	svn[0-9]\+
+		===
include/linux/module.h +1 -0
index 7a0bcb5b1ffc..3b1612193cf9 100644
@@ -372,6 +372,7 @@ struct module {
 	struct module_attribute *modinfo_attrs;
 	const char *version;
 	const char *srcversion;
+	const char *scmversion;
 	struct kobject *holders_dir;
 
 	/* Exported symbols */
init/Kconfig +14 -0
index b77c60f8b963..3d9dac3c4e8f 100644
@@ -2131,6 +2131,20 @@ config MODULE_SRCVERSION_ALL
 	  the version).  With this option, such a "srcversion" field
 	  will be created for all modules.  If unsure, say N.
 
+config MODULE_SCMVERSION
+	bool "SCM version for modules"
+	depends on LOCALVERSION_AUTO
+	help
+	  This enables the module attribute "scmversion" which can be used
+	  by developers to identify the SCM version of a given module, e.g.
+	  git sha1 or hg sha1. The SCM version can be queried by modinfo or
+	  via the sysfs node: /sys/modules/MODULENAME/scmversion. This is
+	  useful when the kernel or kernel modules are updated separately
+	  since that causes the vermagic of the kernel and the module to
+	  differ.
+
+	  If unsure, say N.
+

config MODULE_SIG
bool "Module signature verification"
select MODULE_SIG_FORMAT

kernel/module.c +2 -0
index 4bf30e4b3eaa..d0b359c7e9c9 100644
@@ -837,6 +837,7 @@ static struct module_attribute modinfo_##field = {                    \
 
 MODINFO_ATTR(version);
 MODINFO_ATTR(srcversion);
+MODINFO_ATTR(scmversion);
 
 static char last_unloaded_module[MODULE_NAME_LEN+1];
 
@@ -1298,6 +1299,7 @@ static struct module_attribute *modinfo_attrs[] = {
 	&module_uevent,
 	&modinfo_version,
 	&modinfo_srcversion,
+	&modinfo_scmversion,
 	&modinfo_initstate,
 	&modinfo_coresize,
 	&modinfo_initsize,
scripts/Makefile.modpost +0 -0
index f54b6ac37ac2..f1126b60adb7 100644
@@ -66,6 +66,7 @@ ifeq ($(KBUILD_EXTMOD),)
 

input-symdump := vmlinux.symvers
output-symdump := Module.symvers
+module_srcpath := $(srctree)

else

scripts/Makefile.modpost +11 -0
@@ -77,6 +78,17 @@ src := $(obj)
 include $(if $(wildcard $(KBUILD_EXTMOD)/Kbuild), \
             $(KBUILD_EXTMOD)/Kbuild, $(KBUILD_EXTMOD)/Makefile)
 
+# Get the external module's source path. KBUILD_EXTMOD could either be an
+# absolute path or relative path from $(srctree). This makes sure that we
+# aren't using a relative path from a separate working directory (O= or
+# KBUILD_OUTPUT) since that may not be the actual module's SCM project path.
+# So check the path relative to $(srctree) first.
+ifneq ($(realpath $(srctree)/$(KBUILD_EXTMOD) 2>/dev/null),)
+	module_srcpath := $(srctree)/$(KBUILD_EXTMOD)
+else
+	module_srcpath := $(KBUILD_EXTMOD)
+endif
+
 # modpost option for external modules

MODPOST += -e

scripts/Makefile.modpost +0 -0
@@ -85,6 +97,16 @@ output-symdump := $(KBUILD_EXTMOD)/Module.symvers
 

endif

scripts/Makefile.modpost +10 -0
+ifeq ($(CONFIG_MODULE_SCMVERSION),y)
+# Get the SCM version of the module. Sed verifies setlocalversion returns
+# a proper revision based on the SCM type, e.g. git, mercurial, or svn.
+module_scmversion := $(shell $(srctree)/scripts/setlocalversion $(module_srcpath) | \
+	sed -n 's/.*-\(\(g\|hg\)[a-fA-F0-9]\+\(-dirty\)\?\|svn[0-9]\+\).*/\1/p')
+ifneq ($(module_scmversion),)
+MODPOST += -v$(module_scmversion)
+endif
+endif
+
# modpost options for modules (both in-kernel and external)
MODPOST += \
	$(addprefix -i ,$(wildcard $(input-symdump))) \
scripts/mod/modpost.c +23 -1
index d6c81657d695..489b65bc37de 100644
@@ -30,6 +30,8 @@ static int have_vmlinux = 0;
 static int all_versions = 0;
 /* If we are modposting external module set to 1 */
 static int external_module = 0;
+#define MODULE_SCMVERSION_SIZE 64
+static char module_scmversion[MODULE_SCMVERSION_SIZE];
 /* Only warn about unresolved symbols */
 static int warn_unresolved = 0;
 /* How a symbol is exported */
@@ -2264,6 +2266,20 @@ static void add_intree_flag(struct buffer *b, int is_intree)
 		buf_printf(b, "\nMODULE_INFO(intree, \"Y\");\n");
 }
 
+/**
+ * add_scmversion() - Adds the MODULE_INFO macro for the scmversion.
+ * @b: Buffer to append to.
+ *
+ * This function fills in the module attribute `scmversion` for the kernel
+ * module. This is useful for determining a given module's SCM version on
+ * device via /sys/modules/<module>/scmversion and/or using the modinfo tool.
+ */
+static void add_scmversion(struct buffer *b)
+{
+	if (module_scmversion[0] != '\0')
+		buf_printf(b, "\nMODULE_INFO(scmversion, \"%s\");\n", module_scmversion);
+}
+
 /* Cannot check for assembler */
 static void add_retpoline(struct buffer *b)
 {
@@ -2546,7 +2562,7 @@ int main(int argc, char **argv)
 	struct dump_list *dump_read_start = NULL;
 	struct dump_list **dump_read_iter = &dump_read_start;
 
-	while ((opt = getopt(argc, argv, "ei:mnT:o:awENd:")) != -1) {
+	while ((opt = getopt(argc, argv, "ei:mnT:o:awENd:v:")) != -1) {
 		switch (opt) {
 		case 'e':
 			external_module = 1;
@@ -2584,6 +2600,11 @@ int main(int argc, char **argv)
 		case 'd':
 			missing_namespace_deps = optarg;
 			break;
+		case 'v':
+			if (!optarg)
+				fatal("'-v' requires an argument defining the SCM version.");
+			strncpy(module_scmversion, optarg, sizeof(module_scmversion) - 1);
+			break;
 		default:
 			exit(1);
 		}
@@ -2630,6 +2651,7 @@ int main(int argc, char **argv)
 		add_depends(&buf, mod);
 		add_moddevtable(&buf, mod);
 		add_srcversion(&buf, mod);
+		add_scmversion(&buf);
 
 		sprintf(fname, "%s.mod.c", mod->name);
 		write_if_changed(&buf, fname);
Signature · 1 line
2.30.0.284.gd98b1dd5eaa7-goog
#2 lore raw

Thanks Jessica for the reviews!

--Will

Quoted 255 lines · Jessica Yu

+++ Will McVicker [08/01/21 00:30 +0000]:

Quoted 20 lines

Config MODULE_SCMVERSION introduces a new module attribute -- scmversion -- which can be used to identify a given module's SCM version. This is very useful for developers that update their kernel independently from their kernel modules or vice-versa since the SCM version provided by UTS_RELEASE (uname -r) will now differ from the module's vermagic attribute.

For example, we have a CI setup that tests new kernel changes on the hikey960 and db845c devices without updating their kernel modules. When these tests fail, we need to be able to identify the exact device configuration the test was using. By including MODULE_SCMVERSION, we can identify the exact kernel and modules' SCM versions for debugging the failures.

Additionally, by exposing the SCM version via the sysfs node /sys/module/MODULENAME/scmversion, one can also verify the SCM versions of the modules loaded from the initramfs. Currently, modinfo can only retrieve module attributes from the module's ko on disk and not from the actual module that is loaded in RAM.

Quoted 8 lines

You can retrieve the SCM version in two ways,

1) By using modinfo:

Quoted 1 line

modinfo -F scmversion MODULENAME

2) By module sysfs node:

Quoted 1 line

cat /sys/module/MODULENAME/scmversion

Hi Will,

Thanks for v5, I'm fine with this patch now that we've made it a configurable developer/debug option to supply an scmversion field for in-tree modules (although, this currently works for out-of-tree modules too since we based module_srcpath on KBUILD_EXTMOD in the external module case). I basically see this option as an alternative to CONFIG_MODULE_SRCVERSION_ALL to aid distro developers debug issues when the kernel and in-tree modules are updated separately.

In any case, since there was pushback in our earlier discussions, I'd like to ask if there are any remaining objections to this patch.

Masahiro, are you fine with the Makefile and modpost changes?

Thanks,

Jessica

Quoted 205 lines

---

Documentation/ABI/stable/sysfs-module | 18 ++++++++++++++++++
include/linux/module.h                |  1 +
init/Kconfig                          | 14 ++++++++++++++
kernel/module.c                       |  2 ++
scripts/Makefile.modpost              | 22 ++++++++++++++++++++++
scripts/mod/modpost.c                 | 24 +++++++++++++++++++++++-
6 files changed, 80 insertions(+), 1 deletion(-)
Documentation/ABI/stable/sysfs-module +18 -0
index 6272ae5fb366..a75d137e79f4 100644
@@ -32,3 +32,21 @@ Description:
              Note: If the module is built into the kernel, or if the
              CONFIG_MODULE_UNLOAD kernel configuration value is not enabled,
              this file will not be present.
+
+What:         /sys/module/MODULENAME/scmversion
+Date:         November 2020
+KernelVersion:        5.12
+Contact:      Will McVicker <[email protected]>
+Description:  This read-only file will appear if modpost was supplied with an
+              SCM version for the module. It can be enabled with the config
+              MODULE_SCMVERSION. The SCM version is retrieved by
+              scripts/setlocalversion, which means that the presence of this
+              file depends on CONFIG_LOCALVERSION_AUTO=y. When read, the SCM
+              version that the module was compiled with is returned. The SCM
+              version is returned in the following format::
+
+              ===
+              Git:            g[a-f0-9]\+(-dirty)\?
+              Mercurial:      hg[a-f0-9]\+(-dirty)\?
+              Subversion:     svn[0-9]\+
+              ===
include/linux/module.h +1 -0
index 7a0bcb5b1ffc..3b1612193cf9 100644
@@ -372,6 +372,7 @@ struct module {
      struct module_attribute *modinfo_attrs;
      const char *version;
      const char *srcversion;
+      const char *scmversion;
      struct kobject *holders_dir;
 
      /* Exported symbols */
init/Kconfig +14 -0
index b77c60f8b963..3d9dac3c4e8f 100644
@@ -2131,6 +2131,20 @@ config MODULE_SRCVERSION_ALL
        the version).  With this option, such a "srcversion" field
        will be created for all modules.  If unsure, say N.
 
+config MODULE_SCMVERSION
+      bool "SCM version for modules"
+      depends on LOCALVERSION_AUTO
+      help
+        This enables the module attribute "scmversion" which can be used
+        by developers to identify the SCM version of a given module, e.g.
+        git sha1 or hg sha1. The SCM version can be queried by modinfo or
+        via the sysfs node: /sys/modules/MODULENAME/scmversion. This is
+        useful when the kernel or kernel modules are updated separately
+        since that causes the vermagic of the kernel and the module to
+        differ.
+
+        If unsure, say N.
+

config MODULE_SIG
bool "Module signature verification"
select MODULE_SIG_FORMAT

kernel/module.c +2 -0
index 4bf30e4b3eaa..d0b359c7e9c9 100644
@@ -837,6 +837,7 @@ static struct module_attribute modinfo_##field = {                    \
 
 MODINFO_ATTR(version);
 MODINFO_ATTR(srcversion);
+MODINFO_ATTR(scmversion);
 
 static char last_unloaded_module[MODULE_NAME_LEN+1];
 
@@ -1298,6 +1299,7 @@ static struct module_attribute *modinfo_attrs[] = {
      &module_uevent,
      &modinfo_version,
      &modinfo_srcversion,
+      &modinfo_scmversion,
      &modinfo_initstate,
      &modinfo_coresize,
      &modinfo_initsize,
scripts/Makefile.modpost +0 -0
index f54b6ac37ac2..f1126b60adb7 100644
@@ -66,6 +66,7 @@ ifeq ($(KBUILD_EXTMOD),)
 

input-symdump := vmlinux.symvers
output-symdump := Module.symvers
+module_srcpath := $(srctree)

else

scripts/Makefile.modpost +11 -0
@@ -77,6 +78,17 @@ src := $(obj)
 include $(if $(wildcard $(KBUILD_EXTMOD)/Kbuild), \
             $(KBUILD_EXTMOD)/Kbuild, $(KBUILD_EXTMOD)/Makefile)
 
+# Get the external module's source path. KBUILD_EXTMOD could either be an
+# absolute path or relative path from $(srctree). This makes sure that we
+# aren't using a relative path from a separate working directory (O= or
+# KBUILD_OUTPUT) since that may not be the actual module's SCM project path.
+# So check the path relative to $(srctree) first.
+ifneq ($(realpath $(srctree)/$(KBUILD_EXTMOD) 2>/dev/null),)
+      module_srcpath := $(srctree)/$(KBUILD_EXTMOD)
+else
+      module_srcpath := $(KBUILD_EXTMOD)
+endif
+
 # modpost option for external modules

MODPOST += -e

scripts/Makefile.modpost +0 -0
@@ -85,6 +97,16 @@ output-symdump := $(KBUILD_EXTMOD)/Module.symvers
 

endif

scripts/Makefile.modpost +10 -0
+ifeq ($(CONFIG_MODULE_SCMVERSION),y)
+# Get the SCM version of the module. Sed verifies setlocalversion returns
+# a proper revision based on the SCM type, e.g. git, mercurial, or svn.
+module_scmversion := $(shell $(srctree)/scripts/setlocalversion $(module_srcpath) | \
+      sed -n 's/.*-\(\(g\|hg\)[a-fA-F0-9]\+\(-dirty\)\?\|svn[0-9]\+\).*/\1/p')
+ifneq ($(module_scmversion),)
+MODPOST += -v$(module_scmversion)
+endif
+endif
+
# modpost options for modules (both in-kernel and external)
MODPOST += \
      $(addprefix -i ,$(wildcard $(input-symdump))) \
scripts/mod/modpost.c +23 -2
index d6c81657d695..489b65bc37de 100644
@@ -30,6 +30,8 @@ static int have_vmlinux = 0;
 static int all_versions = 0;
 /* If we are modposting external module set to 1 */
 static int external_module = 0;
+#define MODULE_SCMVERSION_SIZE 64
+static char module_scmversion[MODULE_SCMVERSION_SIZE];
 /* Only warn about unresolved symbols */
 static int warn_unresolved = 0;
 /* How a symbol is exported */
@@ -2264,6 +2266,20 @@ static void add_intree_flag(struct buffer *b, int is_intree)
              buf_printf(b, "\nMODULE_INFO(intree, \"Y\");\n");
 }
 
+/**
+ * add_scmversion() - Adds the MODULE_INFO macro for the scmversion.
+ * @b: Buffer to append to.
+ *
+ * This function fills in the module attribute `scmversion` for the kernel
+ * module. This is useful for determining a given module's SCM version on
+ * device via /sys/modules/<module>/scmversion and/or using the modinfo tool.
+ */
+static void add_scmversion(struct buffer *b)
+{
+      if (module_scmversion[0] != '\0')
+              buf_printf(b, "\nMODULE_INFO(scmversion, \"%s\");\n", module_scmversion);
+}
+
 /* Cannot check for assembler */
 static void add_retpoline(struct buffer *b)
 {
@@ -2546,7 +2562,7 @@ int main(int argc, char **argv)
      struct dump_list *dump_read_start = NULL;
      struct dump_list **dump_read_iter = &dump_read_start;
 
-      while ((opt = getopt(argc, argv, "ei:mnT:o:awENd:")) != -1) {
+      while ((opt = getopt(argc, argv, "ei:mnT:o:awENd:v:")) != -1) {
              switch (opt) {
              case 'e':
                      external_module = 1;
@@ -2584,6 +2600,11 @@ int main(int argc, char **argv)
              case 'd':
                      missing_namespace_deps = optarg;
                      break;
+              case 'v':
+                      if (!optarg)
+                              fatal("'-v' requires an argument defining the SCM version.");
+                      strncpy(module_scmversion, optarg, sizeof(module_scmversion) - 1);
+                      break;
              default:
                      exit(1);
              }
@@ -2630,6 +2651,7 @@ int main(int argc, char **argv)
              add_depends(&buf, mod);
              add_moddevtable(&buf, mod);
              add_srcversion(&buf, mod);
+              add_scmversion(&buf);
 
              sprintf(fname, "%s.mod.c", mod->name);
              write_if_changed(&buf, fname);
--

2.30.0.284.gd98b1dd5eaa7-goog

#3 lore raw
In reply to Will McVicker
Quoted 134 lines · Will McVicker

Config MODULE_SCMVERSION introduces a new module attribute -- scmversion -- which can be used to identify a given module's SCM version. This is very useful for developers that update their kernel independently from their kernel modules or vice-versa since the SCM version provided by UTS_RELEASE (uname -r) will now differ from the module's vermagic attribute.

For example, we have a CI setup that tests new kernel changes on the hikey960 and db845c devices without updating their kernel modules. When these tests fail, we need to be able to identify the exact device configuration the test was using. By including MODULE_SCMVERSION, we can identify the exact kernel and modules' SCM versions for debugging the failures.

Additionally, by exposing the SCM version via the sysfs node /sys/module/MODULENAME/scmversion, one can also verify the SCM versions of the modules loaded from the initramfs. Currently, modinfo can only retrieve module attributes from the module's ko on disk and not from the actual module that is loaded in RAM.

You can retrieve the SCM version in two ways,

1) By using modinfo:

Quoted 1 line

modinfo -F scmversion MODULENAME

2) By module sysfs node:

Quoted 1 line

cat /sys/module/MODULENAME/scmversion

---
 Documentation/ABI/stable/sysfs-module | 18 ++++++++++++++++++
 include/linux/module.h                |  1 +
 init/Kconfig                          | 14 ++++++++++++++
 kernel/module.c                       |  2 ++
 scripts/Makefile.modpost              | 22 ++++++++++++++++++++++
 scripts/mod/modpost.c                 | 24 +++++++++++++++++++++++-
 6 files changed, 80 insertions(+), 1 deletion(-)
Documentation/ABI/stable/sysfs-module +18 -0
index 6272ae5fb366..a75d137e79f4 100644
@@ -32,3 +32,21 @@ Description:
                Note: If the module is built into the kernel, or if the
                CONFIG_MODULE_UNLOAD kernel configuration value is not enabled,
                this file will not be present.
+
+What:          /sys/module/MODULENAME/scmversion
+Date:          November 2020
+KernelVersion: 5.12
+Contact:       Will McVicker <[email protected]>
+Description:   This read-only file will appear if modpost was supplied with an
+               SCM version for the module. It can be enabled with the config
+               MODULE_SCMVERSION. The SCM version is retrieved by
+               scripts/setlocalversion, which means that the presence of this
+               file depends on CONFIG_LOCALVERSION_AUTO=y. When read, the SCM
+               version that the module was compiled with is returned. The SCM
+               version is returned in the following format::
+
+               ===
+               Git:            g[a-f0-9]\+(-dirty)\?
+               Mercurial:      hg[a-f0-9]\+(-dirty)\?
+               Subversion:     svn[0-9]\+
+               ===
include/linux/module.h +1 -0
index 7a0bcb5b1ffc..3b1612193cf9 100644
@@ -372,6 +372,7 @@ struct module {
        struct module_attribute *modinfo_attrs;
        const char *version;
        const char *srcversion;
+       const char *scmversion;
        struct kobject *holders_dir;
 
        /* Exported symbols */
init/Kconfig +14 -0
index b77c60f8b963..3d9dac3c4e8f 100644
@@ -2131,6 +2131,20 @@ config MODULE_SRCVERSION_ALL
          the version).  With this option, such a "srcversion" field
          will be created for all modules.  If unsure, say N.
 
+config MODULE_SCMVERSION
+       bool "SCM version for modules"
+       depends on LOCALVERSION_AUTO
+       help
+         This enables the module attribute "scmversion" which can be used
+         by developers to identify the SCM version of a given module, e.g.
+         git sha1 or hg sha1. The SCM version can be queried by modinfo or
+         via the sysfs node: /sys/modules/MODULENAME/scmversion. This is
+         useful when the kernel or kernel modules are updated separately
+         since that causes the vermagic of the kernel and the module to
+         differ.
+
+         If unsure, say N.
+
 config MODULE_SIG
        bool "Module signature verification"
        select MODULE_SIG_FORMAT
kernel/module.c +2 -0
index 4bf30e4b3eaa..d0b359c7e9c9 100644
@@ -837,6 +837,7 @@ static struct module_attribute modinfo_##field = {                    \
 
 MODINFO_ATTR(version);
 MODINFO_ATTR(srcversion);
+MODINFO_ATTR(scmversion);
 
 static char last_unloaded_module[MODULE_NAME_LEN+1];
 
@@ -1298,6 +1299,7 @@ static struct module_attribute *modinfo_attrs[] = {
        &module_uevent,
        &modinfo_version,
        &modinfo_srcversion,
+       &modinfo_scmversion,
        &modinfo_initstate,
        &modinfo_coresize,
        &modinfo_initsize,
scripts/Makefile.modpost +1 -0
index f54b6ac37ac2..f1126b60adb7 100644
@@ -66,6 +66,7 @@ ifeq ($(KBUILD_EXTMOD),)
 
 input-symdump := vmlinux.symvers
 output-symdump := Module.symvers
+module_srcpath := $(srctree)
 
 else
 

scripts/Makefile.modpost src := $(obj) · line 82

Quoted 5 lines
scripts/Makefile.modpost +2 -0
@@ -77,6 +78,17 @@ src := $(obj)
 include $(if $(wildcard $(KBUILD_EXTMOD)/Kbuild), \
              $(KBUILD_EXTMOD)/Kbuild, $(KBUILD_EXTMOD)/Makefile)
 
+# Get the external module's source path. KBUILD_EXTMOD could either be an
+# absolute path or relative path from $(srctree).

No.
KBUILD_EXTMOD could either be an absolute or relative path from $(objtree).

scripts/Makefile.modpost

Quoted 10 lines

This makes sure that we

scripts/Makefile.modpost +9 -0
+# aren't using a relative path from a separate working directory (O= or
+# KBUILD_OUTPUT) since that may not be the actual module's SCM project path.
+# So check the path relative to $(srctree) first.
+ifneq ($(realpath $(srctree)/$(KBUILD_EXTMOD) 2>/dev/null),)
+       module_srcpath := $(srctree)/$(KBUILD_EXTMOD)
+else
+       module_srcpath := $(KBUILD_EXTMOD)
+endif
+

This hunk is pointless in my view.

The code should be much simpler, as follows:

scripts/Makefile.modpost +2 -0
@@ -66,6 +66,7 @@ ifeq ($(KBUILD_EXTMOD),)
 
 input-symdump := vmlinux.symvers
 output-symdump := Module.symvers
+module_srcpath := $(srctree)
 
 else
 
@@ -82,9 +83,20 @@ MODPOST += -e
 
 input-symdump := Module.symvers $(KBUILD_EXTRA_SYMBOLS)
 output-symdump := $(KBUILD_EXTMOD)/Module.symvers
+module_srcpath := $(KBUILD_EXTMOD)
 
 endif
 
 
 
 
Quoted 4 lines
 # modpost option for external modules
 MODPOST += -e

scripts/Makefile.modpost output-symdump := $(KBUILD_EXTMOD)/Module.symvers · line 106

Quoted 10 lines
scripts/Makefile.modpost +7 -0
@@ -85,6 +97,16 @@ output-symdump := $(KBUILD_EXTMOD)/Module.symvers
 
 endif
 
+ifeq ($(CONFIG_MODULE_SCMVERSION),y)
+# Get the SCM version of the module. Sed verifies setlocalversion returns
+# a proper revision based on the SCM type, e.g. git, mercurial, or svn.
+module_scmversion := $(shell $(srctree)/scripts/setlocalversion $(module_srcpath) | \
+       sed -n 's/.*-\(\(g\|hg\)[a-fA-F0-9]\+\(-dirty\)\?\|svn[0-9]\+\).*/\1/p')
+ifneq ($(module_scmversion),)
+MODPOST += -v$(module_scmversion)

A space after -v for consistency.

Quoted 50 lines
scripts/Makefile.modpost +3 -0
+endif
+endif
+
 # modpost options for modules (both in-kernel and external)
 MODPOST += \
        $(addprefix -i ,$(wildcard $(input-symdump))) \
scripts/mod/modpost.c +17 -1
index d6c81657d695..489b65bc37de 100644
@@ -30,6 +30,8 @@ static int have_vmlinux = 0;
 static int all_versions = 0;
 /* If we are modposting external module set to 1 */
 static int external_module = 0;
+#define MODULE_SCMVERSION_SIZE 64
+static char module_scmversion[MODULE_SCMVERSION_SIZE];
 /* Only warn about unresolved symbols */
 static int warn_unresolved = 0;
 /* How a symbol is exported */
@@ -2264,6 +2266,20 @@ static void add_intree_flag(struct buffer *b, int is_intree)
                buf_printf(b, "\nMODULE_INFO(intree, \"Y\");\n");
 }
 
+/**
+ * add_scmversion() - Adds the MODULE_INFO macro for the scmversion.
+ * @b: Buffer to append to.
+ *
+ * This function fills in the module attribute `scmversion` for the kernel
+ * module. This is useful for determining a given module's SCM version on
+ * device via /sys/modules/<module>/scmversion and/or using the modinfo tool.
+ */
+static void add_scmversion(struct buffer *b)
+{
+       if (module_scmversion[0] != '\0')
+               buf_printf(b, "\nMODULE_INFO(scmversion, \"%s\");\n", module_scmversion);
+}
+
 /* Cannot check for assembler */
 static void add_retpoline(struct buffer *b)
 {
@@ -2546,7 +2562,7 @@ int main(int argc, char **argv)
        struct dump_list *dump_read_start = NULL;
        struct dump_list **dump_read_iter = &dump_read_start;
 
-       while ((opt = getopt(argc, argv, "ei:mnT:o:awENd:")) != -1) {
+       while ((opt = getopt(argc, argv, "ei:mnT:o:awENd:v:")) != -1) {
                switch (opt) {
                case 'e':
                        external_module = 1;

scripts/mod/modpost.c int main(int argc, char **argv) · line 2605

Quoted 6 lines
scripts/mod/modpost.c +3 -0
@@ -2584,6 +2600,11 @@ int main(int argc, char **argv)
                case 'd':
                        missing_namespace_deps = optarg;
                        break;
+               case 'v':
+                       if (!optarg)
+                               fatal("'-v' requires an argument defining the SCM version.");

The existing -i, -T, -o, -d options take an argument as well. Why don't they have such a check as you added?
The answer is because getopt() checks it.

I applied your patch, and ran modpost directly
with -v at the end of the command line.

masahiro@oscar:~/workspace/linux$ ./scripts/mod/modpost  -v
./scripts/mod/modpost: option requires an argument -- 'v'

I see the error message from getopt() instead of
your hand-made error check.

Quoted 16 lines
scripts/mod/modpost.c +3 -1
+                       strncpy(module_scmversion, optarg, sizeof(module_scmversion) - 1);
+                       break;
                default:
                        exit(1);
                }
@@ -2630,6 +2651,7 @@ int main(int argc, char **argv)
                add_depends(&buf, mod);
                add_moddevtable(&buf, mod);
                add_srcversion(&buf, mod);
+               add_scmversion(&buf);
 
                sprintf(fname, "%s.mod.c", mod->name);
                write_if_changed(&buf, fname);
--

2.30.0.284.gd98b1dd5eaa7-goog

Signature · 2 lines
Best Regards
Masahiro Yamada
#4 lore raw

Thanks for the review Masahiro! I'll upload v6 shortly.

--Will

Quoted 312 lines · Masahiro Yamada
Quoted 134 lines · Will McVicker

Config MODULE_SCMVERSION introduces a new module attribute -- scmversion -- which can be used to identify a given module's SCM version. This is very useful for developers that update their kernel independently from their kernel modules or vice-versa since the SCM version provided by UTS_RELEASE (uname -r) will now differ from the module's vermagic attribute.

For example, we have a CI setup that tests new kernel changes on the hikey960 and db845c devices without updating their kernel modules. When these tests fail, we need to be able to identify the exact device configuration the test was using. By including MODULE_SCMVERSION, we can identify the exact kernel and modules' SCM versions for debugging the failures.

Additionally, by exposing the SCM version via the sysfs node /sys/module/MODULENAME/scmversion, one can also verify the SCM versions of the modules loaded from the initramfs. Currently, modinfo can only retrieve module attributes from the module's ko on disk and not from the actual module that is loaded in RAM.

You can retrieve the SCM version in two ways,

1) By using modinfo:

Quoted 1 line

modinfo -F scmversion MODULENAME

2) By module sysfs node:

Quoted 1 line

cat /sys/module/MODULENAME/scmversion

---
 Documentation/ABI/stable/sysfs-module | 18 ++++++++++++++++++
 include/linux/module.h                |  1 +
 init/Kconfig                          | 14 ++++++++++++++
 kernel/module.c                       |  2 ++
 scripts/Makefile.modpost              | 22 ++++++++++++++++++++++
 scripts/mod/modpost.c                 | 24 +++++++++++++++++++++++-
 6 files changed, 80 insertions(+), 1 deletion(-)
Documentation/ABI/stable/sysfs-module +18 -0
index 6272ae5fb366..a75d137e79f4 100644
@@ -32,3 +32,21 @@ Description:
                Note: If the module is built into the kernel, or if the
                CONFIG_MODULE_UNLOAD kernel configuration value is not enabled,
                this file will not be present.
+
+What:          /sys/module/MODULENAME/scmversion
+Date:          November 2020
+KernelVersion: 5.12
+Contact:       Will McVicker <[email protected]>
+Description:   This read-only file will appear if modpost was supplied with an
+               SCM version for the module. It can be enabled with the config
+               MODULE_SCMVERSION. The SCM version is retrieved by
+               scripts/setlocalversion, which means that the presence of this
+               file depends on CONFIG_LOCALVERSION_AUTO=y. When read, the SCM
+               version that the module was compiled with is returned. The SCM
+               version is returned in the following format::
+
+               ===
+               Git:            g[a-f0-9]\+(-dirty)\?
+               Mercurial:      hg[a-f0-9]\+(-dirty)\?
+               Subversion:     svn[0-9]\+
+               ===
include/linux/module.h +1 -0
index 7a0bcb5b1ffc..3b1612193cf9 100644
@@ -372,6 +372,7 @@ struct module {
        struct module_attribute *modinfo_attrs;
        const char *version;
        const char *srcversion;
+       const char *scmversion;
        struct kobject *holders_dir;
 
        /* Exported symbols */
init/Kconfig +14 -0
index b77c60f8b963..3d9dac3c4e8f 100644
@@ -2131,6 +2131,20 @@ config MODULE_SRCVERSION_ALL
          the version).  With this option, such a "srcversion" field
          will be created for all modules.  If unsure, say N.
 
+config MODULE_SCMVERSION
+       bool "SCM version for modules"
+       depends on LOCALVERSION_AUTO
+       help
+         This enables the module attribute "scmversion" which can be used
+         by developers to identify the SCM version of a given module, e.g.
+         git sha1 or hg sha1. The SCM version can be queried by modinfo or
+         via the sysfs node: /sys/modules/MODULENAME/scmversion. This is
+         useful when the kernel or kernel modules are updated separately
+         since that causes the vermagic of the kernel and the module to
+         differ.
+
+         If unsure, say N.
+
 config MODULE_SIG
        bool "Module signature verification"
        select MODULE_SIG_FORMAT
kernel/module.c +2 -0
index 4bf30e4b3eaa..d0b359c7e9c9 100644
@@ -837,6 +837,7 @@ static struct module_attribute modinfo_##field = {                    \
 
 MODINFO_ATTR(version);
 MODINFO_ATTR(srcversion);
+MODINFO_ATTR(scmversion);
 
 static char last_unloaded_module[MODULE_NAME_LEN+1];
 
@@ -1298,6 +1299,7 @@ static struct module_attribute *modinfo_attrs[] = {
        &module_uevent,
        &modinfo_version,
        &modinfo_srcversion,
+       &modinfo_scmversion,
        &modinfo_initstate,
        &modinfo_coresize,
        &modinfo_initsize,
scripts/Makefile.modpost +1 -0
index f54b6ac37ac2..f1126b60adb7 100644
@@ -66,6 +66,7 @@ ifeq ($(KBUILD_EXTMOD),)
 
 input-symdump := vmlinux.symvers
 output-symdump := Module.symvers
+module_srcpath := $(srctree)
 
 else
 

scripts/Makefile.modpost src := $(obj) · line 82

Quoted 5 lines
scripts/Makefile.modpost +2 -0
@@ -77,6 +78,17 @@ src := $(obj)
 include $(if $(wildcard $(KBUILD_EXTMOD)/Kbuild), \
              $(KBUILD_EXTMOD)/Kbuild, $(KBUILD_EXTMOD)/Makefile)
 
+# Get the external module's source path. KBUILD_EXTMOD could either be an
+# absolute path or relative path from $(srctree).

No.
KBUILD_EXTMOD could either be an absolute or relative path from $(objtree).

scripts/Makefile.modpost

Quoted 10 lines

This makes sure that we

scripts/Makefile.modpost +9 -0
+# aren't using a relative path from a separate working directory (O= or
+# KBUILD_OUTPUT) since that may not be the actual module's SCM project path.
+# So check the path relative to $(srctree) first.
+ifneq ($(realpath $(srctree)/$(KBUILD_EXTMOD) 2>/dev/null),)
+       module_srcpath := $(srctree)/$(KBUILD_EXTMOD)
+else
+       module_srcpath := $(KBUILD_EXTMOD)
+endif
+

This hunk is pointless in my view.

The code should be much simpler, as follows:

scripts/Makefile.modpost +2 -0
@@ -66,6 +66,7 @@ ifeq ($(KBUILD_EXTMOD),)
 
 input-symdump := vmlinux.symvers
 output-symdump := Module.symvers
+module_srcpath := $(srctree)
 
 else
 
@@ -82,9 +83,20 @@ MODPOST += -e
 
 input-symdump := Module.symvers $(KBUILD_EXTRA_SYMBOLS)
 output-symdump := $(KBUILD_EXTMOD)/Module.symvers
+module_srcpath := $(KBUILD_EXTMOD)
 
 endif
 
 
 
 
Quoted 8 lines



 # modpost option for external modules
 MODPOST += -e

scripts/Makefile.modpost output-symdump := $(KBUILD_EXTMOD)/Module.symvers · line 106

Quoted 10 lines
scripts/Makefile.modpost +7 -0
@@ -85,6 +97,16 @@ output-symdump := $(KBUILD_EXTMOD)/Module.symvers
 
 endif
 
+ifeq ($(CONFIG_MODULE_SCMVERSION),y)
+# Get the SCM version of the module. Sed verifies setlocalversion returns
+# a proper revision based on the SCM type, e.g. git, mercurial, or svn.
+module_scmversion := $(shell $(srctree)/scripts/setlocalversion $(module_srcpath) | \
+       sed -n 's/.*-\(\(g\|hg\)[a-fA-F0-9]\+\(-dirty\)\?\|svn[0-9]\+\).*/\1/p')
+ifneq ($(module_scmversion),)
+MODPOST += -v$(module_scmversion)

A space after -v for consistency.

Quoted 50 lines
scripts/Makefile.modpost +3 -0
+endif
+endif
+
 # modpost options for modules (both in-kernel and external)
 MODPOST += \
        $(addprefix -i ,$(wildcard $(input-symdump))) \
scripts/mod/modpost.c +17 -1
index d6c81657d695..489b65bc37de 100644
@@ -30,6 +30,8 @@ static int have_vmlinux = 0;
 static int all_versions = 0;
 /* If we are modposting external module set to 1 */
 static int external_module = 0;
+#define MODULE_SCMVERSION_SIZE 64
+static char module_scmversion[MODULE_SCMVERSION_SIZE];
 /* Only warn about unresolved symbols */
 static int warn_unresolved = 0;
 /* How a symbol is exported */
@@ -2264,6 +2266,20 @@ static void add_intree_flag(struct buffer *b, int is_intree)
                buf_printf(b, "\nMODULE_INFO(intree, \"Y\");\n");
 }
 
+/**
+ * add_scmversion() - Adds the MODULE_INFO macro for the scmversion.
+ * @b: Buffer to append to.
+ *
+ * This function fills in the module attribute `scmversion` for the kernel
+ * module. This is useful for determining a given module's SCM version on
+ * device via /sys/modules/<module>/scmversion and/or using the modinfo tool.
+ */
+static void add_scmversion(struct buffer *b)
+{
+       if (module_scmversion[0] != '\0')
+               buf_printf(b, "\nMODULE_INFO(scmversion, \"%s\");\n", module_scmversion);
+}
+
 /* Cannot check for assembler */
 static void add_retpoline(struct buffer *b)
 {
@@ -2546,7 +2562,7 @@ int main(int argc, char **argv)
        struct dump_list *dump_read_start = NULL;
        struct dump_list **dump_read_iter = &dump_read_start;
 
-       while ((opt = getopt(argc, argv, "ei:mnT:o:awENd:")) != -1) {
+       while ((opt = getopt(argc, argv, "ei:mnT:o:awENd:v:")) != -1) {
                switch (opt) {
                case 'e':
                        external_module = 1;

scripts/mod/modpost.c int main(int argc, char **argv) · line 2605

Quoted 6 lines
scripts/mod/modpost.c +3 -0
@@ -2584,6 +2600,11 @@ int main(int argc, char **argv)
                case 'd':
                        missing_namespace_deps = optarg;
                        break;
+               case 'v':
+                       if (!optarg)
+                               fatal("'-v' requires an argument defining the SCM version.");

The existing -i, -T, -o, -d options take an argument as well. Why don't they have such a check as you added?
The answer is because getopt() checks it.

I applied your patch, and ran modpost directly
with -v at the end of the command line.

masahiro@oscar:~/workspace/linux$ ./scripts/mod/modpost  -v
./scripts/mod/modpost: option requires an argument -- 'v'

I see the error message from getopt() instead of
your hand-made error check.

Quoted 14 lines
scripts/mod/modpost.c +3 -1
+                       strncpy(module_scmversion, optarg, sizeof(module_scmversion) - 1);
+                       break;
                default:
                        exit(1);
                }
@@ -2630,6 +2651,7 @@ int main(int argc, char **argv)
                add_depends(&buf, mod);
                add_moddevtable(&buf, mod);
                add_srcversion(&buf, mod);
+               add_scmversion(&buf);
 
                sprintf(fname, "%s.mod.c", mod->name);
                write_if_changed(&buf, fname);
--
Quoted 2 lines

2.30.0.284.gd98b1dd5eaa7-goog

--
Best Regards
Masahiro Yamada

#5 lore raw

Hi Masahiro,

After testing your suggestions for module_srcpath, I remembered why I needed this:

+ifneq ($(realpath $(srctree)/$(KBUILD_EXTMOD) 2>/dev/null),)
+ module_srcpath := $(srctree)/$(KBUILD_EXTMOD)
+else
+ module_srcpath := $(KBUILD_EXTMOD)
+endif

Basically KBUILD_EXTMOD actually has two uses when defined as a relative path:

(1) defines the path to the external module's source relative to $(srctree)
(2) defines the path to the external module's build artifacts relative to $(objtree)

Since setlocalversion is run from $(objtree), we need to use $(srctree)/$(KBUILD_EXTMOD) when it's a relative path. Just to note, if I do just use KBUILD_EXTMOD, then the script setlocalversion fails to find the srctree and returns an empty string.

Please correct me if I'm wrong. I messed around with this by adding many prints to Makefile.modpost and setlocalversion to make sure everything was defined as expected.

Thanks,
Will

Quoted 318 lines · Will McVicker

Thanks for the review Masahiro! I'll upload v6 shortly.

--Will

Quoted 312 lines · Masahiro Yamada
Quoted 134 lines · Will McVicker

Config MODULE_SCMVERSION introduces a new module attribute -- scmversion -- which can be used to identify a given module's SCM version. This is very useful for developers that update their kernel independently from their kernel modules or vice-versa since the SCM version provided by UTS_RELEASE (uname -r) will now differ from the module's vermagic attribute.

For example, we have a CI setup that tests new kernel changes on the hikey960 and db845c devices without updating their kernel modules. When these tests fail, we need to be able to identify the exact device configuration the test was using. By including MODULE_SCMVERSION, we can identify the exact kernel and modules' SCM versions for debugging the failures.

Additionally, by exposing the SCM version via the sysfs node /sys/module/MODULENAME/scmversion, one can also verify the SCM versions of the modules loaded from the initramfs. Currently, modinfo can only retrieve module attributes from the module's ko on disk and not from the actual module that is loaded in RAM.

You can retrieve the SCM version in two ways,

1) By using modinfo:

Quoted 1 line

modinfo -F scmversion MODULENAME

2) By module sysfs node:

Quoted 1 line

cat /sys/module/MODULENAME/scmversion

---
 Documentation/ABI/stable/sysfs-module | 18 ++++++++++++++++++
 include/linux/module.h                |  1 +
 init/Kconfig                          | 14 ++++++++++++++
 kernel/module.c                       |  2 ++
 scripts/Makefile.modpost              | 22 ++++++++++++++++++++++
 scripts/mod/modpost.c                 | 24 +++++++++++++++++++++++-
 6 files changed, 80 insertions(+), 1 deletion(-)
Documentation/ABI/stable/sysfs-module +18 -0
index 6272ae5fb366..a75d137e79f4 100644
@@ -32,3 +32,21 @@ Description:
                Note: If the module is built into the kernel, or if the
                CONFIG_MODULE_UNLOAD kernel configuration value is not enabled,
                this file will not be present.
+
+What:          /sys/module/MODULENAME/scmversion
+Date:          November 2020
+KernelVersion: 5.12
+Contact:       Will McVicker <[email protected]>
+Description:   This read-only file will appear if modpost was supplied with an
+               SCM version for the module. It can be enabled with the config
+               MODULE_SCMVERSION. The SCM version is retrieved by
+               scripts/setlocalversion, which means that the presence of this
+               file depends on CONFIG_LOCALVERSION_AUTO=y. When read, the SCM
+               version that the module was compiled with is returned. The SCM
+               version is returned in the following format::
+
+               ===
+               Git:            g[a-f0-9]\+(-dirty)\?
+               Mercurial:      hg[a-f0-9]\+(-dirty)\?
+               Subversion:     svn[0-9]\+
+               ===
include/linux/module.h +1 -0
index 7a0bcb5b1ffc..3b1612193cf9 100644
@@ -372,6 +372,7 @@ struct module {
        struct module_attribute *modinfo_attrs;
        const char *version;
        const char *srcversion;
+       const char *scmversion;
        struct kobject *holders_dir;
 
        /* Exported symbols */
init/Kconfig +14 -0
index b77c60f8b963..3d9dac3c4e8f 100644
@@ -2131,6 +2131,20 @@ config MODULE_SRCVERSION_ALL
          the version).  With this option, such a "srcversion" field
          will be created for all modules.  If unsure, say N.
 
+config MODULE_SCMVERSION
+       bool "SCM version for modules"
+       depends on LOCALVERSION_AUTO
+       help
+         This enables the module attribute "scmversion" which can be used
+         by developers to identify the SCM version of a given module, e.g.
+         git sha1 or hg sha1. The SCM version can be queried by modinfo or
+         via the sysfs node: /sys/modules/MODULENAME/scmversion. This is
+         useful when the kernel or kernel modules are updated separately
+         since that causes the vermagic of the kernel and the module to
+         differ.
+
+         If unsure, say N.
+
 config MODULE_SIG
        bool "Module signature verification"
        select MODULE_SIG_FORMAT
kernel/module.c +2 -0
index 4bf30e4b3eaa..d0b359c7e9c9 100644
@@ -837,6 +837,7 @@ static struct module_attribute modinfo_##field = {                    \
 
 MODINFO_ATTR(version);
 MODINFO_ATTR(srcversion);
+MODINFO_ATTR(scmversion);
 
 static char last_unloaded_module[MODULE_NAME_LEN+1];
 
@@ -1298,6 +1299,7 @@ static struct module_attribute *modinfo_attrs[] = {
        &module_uevent,
        &modinfo_version,
        &modinfo_srcversion,
+       &modinfo_scmversion,
        &modinfo_initstate,
        &modinfo_coresize,
        &modinfo_initsize,
scripts/Makefile.modpost +1 -0
index f54b6ac37ac2..f1126b60adb7 100644
@@ -66,6 +66,7 @@ ifeq ($(KBUILD_EXTMOD),)
 
 input-symdump := vmlinux.symvers
 output-symdump := Module.symvers
+module_srcpath := $(srctree)
 
 else
 

scripts/Makefile.modpost src := $(obj) · line 82

Quoted 5 lines
scripts/Makefile.modpost +2 -0
@@ -77,6 +78,17 @@ src := $(obj)
 include $(if $(wildcard $(KBUILD_EXTMOD)/Kbuild), \
              $(KBUILD_EXTMOD)/Kbuild, $(KBUILD_EXTMOD)/Makefile)
 
+# Get the external module's source path. KBUILD_EXTMOD could either be an
+# absolute path or relative path from $(srctree).

No.
KBUILD_EXTMOD could either be an absolute or relative path from $(objtree).

scripts/Makefile.modpost

Quoted 10 lines

This makes sure that we

scripts/Makefile.modpost +9 -0
+# aren't using a relative path from a separate working directory (O= or
+# KBUILD_OUTPUT) since that may not be the actual module's SCM project path.
+# So check the path relative to $(srctree) first.
+ifneq ($(realpath $(srctree)/$(KBUILD_EXTMOD) 2>/dev/null),)
+       module_srcpath := $(srctree)/$(KBUILD_EXTMOD)
+else
+       module_srcpath := $(KBUILD_EXTMOD)
+endif
+

This hunk is pointless in my view.

The code should be much simpler, as follows:

scripts/Makefile.modpost +2 -0
@@ -66,6 +66,7 @@ ifeq ($(KBUILD_EXTMOD),)
 
 input-symdump := vmlinux.symvers
 output-symdump := Module.symvers
+module_srcpath := $(srctree)
 
 else
 
@@ -82,9 +83,20 @@ MODPOST += -e
 
 input-symdump := Module.symvers $(KBUILD_EXTRA_SYMBOLS)
 output-symdump := $(KBUILD_EXTMOD)/Module.symvers
+module_srcpath := $(KBUILD_EXTMOD)
 
 endif
 
 
 
 
Quoted 8 lines



 # modpost option for external modules
 MODPOST += -e

scripts/Makefile.modpost output-symdump := $(KBUILD_EXTMOD)/Module.symvers · line 106

Quoted 10 lines
scripts/Makefile.modpost +7 -0
@@ -85,6 +97,16 @@ output-symdump := $(KBUILD_EXTMOD)/Module.symvers
 
 endif
 
+ifeq ($(CONFIG_MODULE_SCMVERSION),y)
+# Get the SCM version of the module. Sed verifies setlocalversion returns
+# a proper revision based on the SCM type, e.g. git, mercurial, or svn.
+module_scmversion := $(shell $(srctree)/scripts/setlocalversion $(module_srcpath) | \
+       sed -n 's/.*-\(\(g\|hg\)[a-fA-F0-9]\+\(-dirty\)\?\|svn[0-9]\+\).*/\1/p')
+ifneq ($(module_scmversion),)
+MODPOST += -v$(module_scmversion)

A space after -v for consistency.

Quoted 50 lines
scripts/Makefile.modpost +3 -0
+endif
+endif
+
 # modpost options for modules (both in-kernel and external)
 MODPOST += \
        $(addprefix -i ,$(wildcard $(input-symdump))) \
scripts/mod/modpost.c +17 -1
index d6c81657d695..489b65bc37de 100644
@@ -30,6 +30,8 @@ static int have_vmlinux = 0;
 static int all_versions = 0;
 /* If we are modposting external module set to 1 */
 static int external_module = 0;
+#define MODULE_SCMVERSION_SIZE 64
+static char module_scmversion[MODULE_SCMVERSION_SIZE];
 /* Only warn about unresolved symbols */
 static int warn_unresolved = 0;
 /* How a symbol is exported */
@@ -2264,6 +2266,20 @@ static void add_intree_flag(struct buffer *b, int is_intree)
                buf_printf(b, "\nMODULE_INFO(intree, \"Y\");\n");
 }
 
+/**
+ * add_scmversion() - Adds the MODULE_INFO macro for the scmversion.
+ * @b: Buffer to append to.
+ *
+ * This function fills in the module attribute `scmversion` for the kernel
+ * module. This is useful for determining a given module's SCM version on
+ * device via /sys/modules/<module>/scmversion and/or using the modinfo tool.
+ */
+static void add_scmversion(struct buffer *b)
+{
+       if (module_scmversion[0] != '\0')
+               buf_printf(b, "\nMODULE_INFO(scmversion, \"%s\");\n", module_scmversion);
+}
+
 /* Cannot check for assembler */
 static void add_retpoline(struct buffer *b)
 {
@@ -2546,7 +2562,7 @@ int main(int argc, char **argv)
        struct dump_list *dump_read_start = NULL;
        struct dump_list **dump_read_iter = &dump_read_start;
 
-       while ((opt = getopt(argc, argv, "ei:mnT:o:awENd:")) != -1) {
+       while ((opt = getopt(argc, argv, "ei:mnT:o:awENd:v:")) != -1) {
                switch (opt) {
                case 'e':
                        external_module = 1;

scripts/mod/modpost.c int main(int argc, char **argv) · line 2605

Quoted 6 lines
scripts/mod/modpost.c +3 -0
@@ -2584,6 +2600,11 @@ int main(int argc, char **argv)
                case 'd':
                        missing_namespace_deps = optarg;
                        break;
+               case 'v':
+                       if (!optarg)
+                               fatal("'-v' requires an argument defining the SCM version.");

The existing -i, -T, -o, -d options take an argument as well. Why don't they have such a check as you added?
The answer is because getopt() checks it.

I applied your patch, and ran modpost directly
with -v at the end of the command line.

masahiro@oscar:~/workspace/linux$ ./scripts/mod/modpost  -v
./scripts/mod/modpost: option requires an argument -- 'v'

I see the error message from getopt() instead of
your hand-made error check.

Quoted 14 lines
scripts/mod/modpost.c +3 -1
+                       strncpy(module_scmversion, optarg, sizeof(module_scmversion) - 1);
+                       break;
                default:
                        exit(1);
                }
@@ -2630,6 +2651,7 @@ int main(int argc, char **argv)
                add_depends(&buf, mod);
                add_moddevtable(&buf, mod);
                add_srcversion(&buf, mod);
+               add_scmversion(&buf);
 
                sprintf(fname, "%s.mod.c", mod->name);
                write_if_changed(&buf, fname);
--
Quoted 2 lines

2.30.0.284.gd98b1dd5eaa7-goog

--
Best Regards
Masahiro Yamada

#7 lore raw
In reply to Will McVicker
Quoted 6 lines · Will McVicker

For example, we have a CI setup that tests new kernel changes on the hikey960 and db845c devices without updating their kernel modules. When these tests fail, we need to be able to identify the exact device configuration the test was using. By including MODULE_SCMVERSION, we can identify the exact kernel and modules' SCM versions for debugging the failures.

Sorry, but this still has no business in the upstream kernel as every change to the kernel is free to just change any API. That is whatever you test there is a completely unsupported setup.

More importantly the "scmversion" of a module simply does not matter, as we only support modules from the kernel tree and it thus must be the kernel version.
be supported.

You are still trying to sneak out of tree module infrastructure in here with a bad cover story. Please stop doing that.

#8 lore raw
Quoted 11 lines · Christoph Hellwig
Quoted 6 lines · Will McVicker

For example, we have a CI setup that tests new kernel changes on the hikey960 and db845c devices without updating their kernel modules. When these tests fail, we need to be able to identify the exact device configuration the test was using. By including MODULE_SCMVERSION, we can identify the exact kernel and modules' SCM versions for debugging the failures.

Sorry, but this still has no business in the upstream kernel as every change to the kernel is free to just change any API.

Sure, and this patch is making no claims one way or the other on that topic.

Quoted 2 lines

That is whatever
you test there is a completely unsupported setup.

Plenty of distributions maintain stable kernels based on LTS. We've done that too and we are able to do LTS kernel binary updates (so better security) without waiting around for the modules to get updated. Keep in mind, that not all modules might be updated at the same time either. That's something that's definitely feasible and works.

And if the API changes, MODVERSIONS catches + CI helps catch them. And if something slips in and things fail, we want to find out what kernel source was used vs what module source was used to debug the problem. This is all relevant even for in-tree modules.

Quoted 4 lines

More importantly the "scmversion" of a module simply does not matter, as we only support modules from the kernel tree and it thus must be the kernel version.
be supported.

This is all talking about only in-tree modules. If you update the kernel vs the modules separately, the scmversion does matter. Also, by your argument, the vermagic or srcversion properties in a module shouldn't be there either.

Quoted 2 lines

You are still trying to sneak out of tree module infrastructure in here with a bad cover story. Please stop doing that.

If Will needs to maintain a downstream patch for adding out-of-tree module support, maintaining this additional patch isn't going to significantly increase his work. But he's trying to upstream at least the part that's useful for upstream. This is still a real problem for a device/board that's fully upstream. Please stop ignoring real upstream problems just because it can also be a problem for out of tree modules. This is not how we encourage folks to upstream their changes.

-Saravana

#6 lore raw
In reply to Will McVicker
Quoted 18 lines · Will McVicker

Hi Masahiro,

After testing your suggestions for module_srcpath, I remembered why I needed this:

(quoted diff) +5 -0
+ifneq ($(realpath $(srctree)/$(KBUILD_EXTMOD) 2>/dev/null),)
+       module_srcpath := $(srctree)/$(KBUILD_EXTMOD)
+else
+       module_srcpath := $(KBUILD_EXTMOD)
+endif

Basically KBUILD_EXTMOD actually has two uses when defined as a relative path:

(1) defines the path to the external module's source relative to $(srctree)
(2) defines the path to the external module's build artifacts relative to $(objtree)

Quoted 8 lines

Since setlocalversion is run from $(objtree), we need to use $(srctree)/$(KBUILD_EXTMOD) when it's a relative path. Just to note, if I do just use KBUILD_EXTMOD, then the script setlocalversion fails to find the srctree and returns an empty string.

Please correct me if I'm wrong. I messed around with this by adding many prints to Makefile.modpost and setlocalversion to make sure everything was defined as expected.

Hmm, you are right.

I did not know the current Kbuild works like that.

But, I do not want to uglify the code
just for supporting this rare use-case.

Rather, I want to apply this
https://lore.kernel.org/patchwork/patch/1368277/

and, still want to see as simple code as the following:

scripts/Makefile.modpost +2 -0
@@ -66,6 +66,7 @@ ifeq ($(KBUILD_EXTMOD),)
 
 input-symdump := vmlinux.symvers
 output-symdump := Module.symvers
+module-srcpath := $(srctree)
 
 else
 
@@ -82,9 +83,20 @@ MODPOST += -e
 
 input-symdump := Module.symvers $(KBUILD_EXTRA_SYMBOLS)
 output-symdump := $(KBUILD_EXTMOD)/Module.symvers
+module-srcpath := $(KBUILD_EXTMOD)
 
 endif
 
 
 
 
Signature · 2 lines
Best Regards
Masahiro Yamada