Skip to contentSkip to conversation

modules: introduce the MODULE_SCMVERSION config

PATCH Superseded Will McVicker 2 participants 5 messages Dec 16, 2020 – Jan 8, 2021

Subsystems (2)

v4 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

PATCH v4 1/1 modules: introduce the MODULE_SCMVERSION config

Config MODULE_SCMVERSION introduces a new module attribute -- scmversion -- which can be used to identify a given module's SCM version. This is very useful for developers that update their kernel independently from their kernel modules or vice-versa since the SCM version provided by UTS_RELEASE (uname -r) will now differ from the module's vermagic attribute.

For example, we have a CI setup that tests new kernel changes on the hikey960 and db845c devices without updating their kernel modules. When these tests fail, we need to be able to identify the exact device configuration the test was using. By including MODULE_SCMVERSION, we can identify the exact kernel and modules' SCM versions for debugging the failures.

Additionally, by exposing the SCM version via the sysfs node /sys/module/MODULENAME/scmversion, one can also verify the SCM versions of the modules loaded from the initramfs. Currently, modinfo can only retrieve module attributes from the module's ko on disk and not from the actual module that is loaded in RAM.

You can retrieve the SCM version in two ways,

1) By using modinfo:

Quoted 1 line

modinfo -F scmversion MODULENAME

2) By module sysfs node:

Quoted 1 line

cat /sys/module/MODULENAME/scmversion

---
Changelog since v3:
- Dropped [PATCH v2 1/2] scripts/setlocalversion: allow running in a subdir

 Documentation/ABI/stable/sysfs-module | 18 ++++++++++++++++++
 include/linux/module.h                |  1 +
 init/Kconfig                          | 12 ++++++++++++
 kernel/module.c                       |  2 ++
 scripts/Makefile.modpost              | 22 ++++++++++++++++++++++
 scripts/mod/modpost.c                 | 24 +++++++++++++++++++++++-
 6 files changed, 78 insertions(+), 1 deletion(-)
Documentation/ABI/stable/sysfs-module +18 -0
index 6272ae5fb366..2ba731767737 100644
@@ -32,3 +32,21 @@ Description:
 		Note: If the module is built into the kernel, or if the
 		CONFIG_MODULE_UNLOAD kernel configuration value is not enabled,
 		this file will not be present.
+
+What:		/sys/module/MODULENAME/scmversion
+Date:		November 2020
+KernelVersion:	5.11
+Contact:	Will McVicker <[email protected]>
+Description:	This read-only file will appear if modpost was supplied with an
+		SCM version for the module. It can be enabled with the config
+		MODULE_SCMVERSION. The SCM version is retrieved by
+		scripts/setlocalversion, which means that the presence of this
+		file depends on CONFIG_LOCALVERSION_AUTO=y or LOCALVERSION=.
+		When read, the SCM version that the module was compiled with is
+		returned. The SCM version is returned in the following format::
+
+		===
+		Git:		g[a-f0-9]\+(-dirty)\?
+		Mercurial:	hg[a-f0-9]\+(-dirty)\?
+		Subversion:	svn[0-9]\+
+		===
include/linux/module.h +1 -0
index c4e7a887f469..6bd710308863 100644
@@ -372,6 +372,7 @@ struct module {
 	struct module_attribute *modinfo_attrs;
 	const char *version;
 	const char *srcversion;
+	const char *scmversion;
 	struct kobject *holders_dir;
 
 	/* Exported symbols */
init/Kconfig +12 -0
index b77c60f8b963..d9ae12f16ba2 100644
@@ -2131,6 +2131,18 @@ config MODULE_SRCVERSION_ALL
 	  the version).  With this option, such a "srcversion" field
 	  will be created for all modules.  If unsure, say N.
 
+config MODULE_SCMVERSION
+	bool "SCM version for modules"
+	depends on LOCALVERSION_AUTO
+	help
+	  This enables the module attribute "scmversion" which can be used
+	  by developers to identify the SCM version of a given module, e.g.
+	  git sha1 or hg sha1. The SCM version can be queried by modinfo or
+	  via the sysfs node: /sys/modules/MODULENAME/scmversion. This is
+	  useful when the kernel or kernel modules are updated separately
+	  since that causes the vermagic of the kernel and the module to
+	  differ.
+
 config MODULE_SIG
 	bool "Module signature verification"
 	select MODULE_SIG_FORMAT
kernel/module.c +2 -0
index c3a9e972d3b2..6967cd3abf9b 100644
@@ -836,6 +836,7 @@ static struct module_attribute modinfo_##field = {                    \
 
 MODINFO_ATTR(version);
 MODINFO_ATTR(srcversion);
+MODINFO_ATTR(scmversion);
 
 static char last_unloaded_module[MODULE_NAME_LEN+1];
 
@@ -1298,6 +1299,7 @@ static struct module_attribute *modinfo_attrs[] = {
 	&module_uevent,
 	&modinfo_version,
 	&modinfo_srcversion,
+	&modinfo_scmversion,
 	&modinfo_initstate,
 	&modinfo_coresize,
 	&modinfo_initsize,
scripts/Makefile.modpost +22 -0
index f54b6ac37ac2..f1126b60adb7 100644
@@ -66,6 +66,7 @@ ifeq ($(KBUILD_EXTMOD),)
 
 input-symdump := vmlinux.symvers
 output-symdump := Module.symvers
+module_srcpath := $(srctree)
 
 else
 
@@ -77,6 +78,17 @@ src := $(obj)
 include $(if $(wildcard $(KBUILD_EXTMOD)/Kbuild), \
              $(KBUILD_EXTMOD)/Kbuild, $(KBUILD_EXTMOD)/Makefile)
 
+# Get the external module's source path. KBUILD_EXTMOD could either be an
+# absolute path or relative path from $(srctree). This makes sure that we
+# aren't using a relative path from a separate working directory (O= or
+# KBUILD_OUTPUT) since that may not be the actual module's SCM project path.
+# So check the path relative to $(srctree) first.
+ifneq ($(realpath $(srctree)/$(KBUILD_EXTMOD) 2>/dev/null),)
+	module_srcpath := $(srctree)/$(KBUILD_EXTMOD)
+else
+	module_srcpath := $(KBUILD_EXTMOD)
+endif
+
 # modpost option for external modules
 MODPOST += -e
 
@@ -85,6 +97,16 @@ output-symdump := $(KBUILD_EXTMOD)/Module.symvers
 
 endif
 
+ifeq ($(CONFIG_MODULE_SCMVERSION),y)
+# Get the SCM version of the module. Sed verifies setlocalversion returns
+# a proper revision based on the SCM type, e.g. git, mercurial, or svn.
+module_scmversion := $(shell $(srctree)/scripts/setlocalversion $(module_srcpath) | \
+	sed -n 's/.*-\(\(g\|hg\)[a-fA-F0-9]\+\(-dirty\)\?\|svn[0-9]\+\).*/\1/p')
+ifneq ($(module_scmversion),)
+MODPOST += -v$(module_scmversion)
+endif
+endif
+
 # modpost options for modules (both in-kernel and external)
 MODPOST += \
 	$(addprefix -i ,$(wildcard $(input-symdump))) \
scripts/mod/modpost.c +23 -1
index f882ce0d9327..db71e0c9ab20 100644
@@ -30,6 +30,8 @@ static int have_vmlinux = 0;
 static int all_versions = 0;
 /* If we are modposting external module set to 1 */
 static int external_module = 0;
+#define MODULE_SCMVERSION_SIZE 64
+static char module_scmversion[MODULE_SCMVERSION_SIZE];
 /* Only warn about unresolved symbols */
 static int warn_unresolved = 0;
 /* How a symbol is exported */
@@ -2272,6 +2274,20 @@ static void add_intree_flag(struct buffer *b, int is_intree)
 		buf_printf(b, "\nMODULE_INFO(intree, \"Y\");\n");
 }
 
+/**
+ * add_scmversion() - Adds the MODULE_INFO macro for the scmversion.
+ * @b: Buffer to append to.
+ *
+ * This function fills in the module attribute `scmversion` for the kernel
+ * module. This is useful for determining a given module's SCM version on
+ * device via /sys/modules/<module>/scmversion and/or using the modinfo tool.
+ */
+static void add_scmversion(struct buffer *b)
+{
+	if (module_scmversion[0] != '\0')
+		buf_printf(b, "\nMODULE_INFO(scmversion, \"%s\");\n", module_scmversion);
+}
+
 /* Cannot check for assembler */
 static void add_retpoline(struct buffer *b)
 {
@@ -2559,7 +2575,7 @@ int main(int argc, char **argv)
 	struct dump_list *dump_read_start = NULL;
 	struct dump_list **dump_read_iter = &dump_read_start;
 
-	while ((opt = getopt(argc, argv, "ei:mnT:o:awENd:")) != -1) {
+	while ((opt = getopt(argc, argv, "ei:mnT:o:awENd:v:")) != -1) {
 		switch (opt) {
 		case 'e':
 			external_module = 1;
@@ -2597,6 +2613,11 @@ int main(int argc, char **argv)
 		case 'd':
 			missing_namespace_deps = optarg;
 			break;
+		case 'v':
+			if (!optarg)
+				fatal("'-v' requires an argument defining the SCM version.");
+			strncpy(module_scmversion, optarg, sizeof(module_scmversion) - 1);
+			break;
 		default:
 			exit(1);
 		}
@@ -2645,6 +2666,7 @@ int main(int argc, char **argv)
 		add_depends(&buf, mod);
 		add_moddevtable(&buf, mod);
 		add_srcversion(&buf, mod);
+		add_scmversion(&buf);
 
 		sprintf(fname, "%s.mod.c", mod->name);
 		write_if_changed(&buf, fname);
Signature · 1 line
2.29.2.684.gfbc64c5ab5-goog
#1 lore raw

+++ Will McVicker [16/12/20 22:08 +0000]:

Quoted 20 lines

Config MODULE_SCMVERSION introduces a new module attribute -- scmversion -- which can be used to identify a given module's SCM version. This is very useful for developers that update their kernel independently from their kernel modules or vice-versa since the SCM version provided by UTS_RELEASE (uname -r) will now differ from the module's vermagic attribute.

For example, we have a CI setup that tests new kernel changes on the hikey960 and db845c devices without updating their kernel modules. When these tests fail, we need to be able to identify the exact device configuration the test was using. By including MODULE_SCMVERSION, we can identify the exact kernel and modules' SCM versions for debugging the failures.

Additionally, by exposing the SCM version via the sysfs node /sys/module/MODULENAME/scmversion, one can also verify the SCM versions of the modules loaded from the initramfs. Currently, modinfo can only retrieve module attributes from the module's ko on disk and not from the actual module that is loaded in RAM.

Quoted 8 lines

You can retrieve the SCM version in two ways,

1) By using modinfo:

Quoted 1 line

modinfo -F scmversion MODULENAME

2) By module sysfs node:

Quoted 1 line

cat /sys/module/MODULENAME/scmversion

Hi Will, thanks for v4. I'm on vacation for the next two weeks and with the current merge window almost over, I hope it'd be OK with you if we revisit this early January. Just wanted to give you a heads up.

Thanks and happy holidays!

Jessica

#2 lore raw

Thanks for the vacation notice Jessica! I'm just letting you know I'm back as well and am happy to respond to any concerns regarding v4 when you get all caught up.

I hope you had a relaxing holiday :)

Thanks,
Will

Quoted 39 lines · Jessica Yu

+++ Will McVicker [16/12/20 22:08 +0000]:

Quoted 20 lines

Config MODULE_SCMVERSION introduces a new module attribute -- scmversion -- which can be used to identify a given module's SCM version. This is very useful for developers that update their kernel independently from their kernel modules or vice-versa since the SCM version provided by UTS_RELEASE (uname -r) will now differ from the module's vermagic attribute.

For example, we have a CI setup that tests new kernel changes on the hikey960 and db845c devices without updating their kernel modules. When these tests fail, we need to be able to identify the exact device configuration the test was using. By including MODULE_SCMVERSION, we can identify the exact kernel and modules' SCM versions for debugging the failures.

Additionally, by exposing the SCM version via the sysfs node /sys/module/MODULENAME/scmversion, one can also verify the SCM versions of the modules loaded from the initramfs. Currently, modinfo can only retrieve module attributes from the module's ko on disk and not from the actual module that is loaded in RAM.

Quoted 8 lines

You can retrieve the SCM version in two ways,

1) By using modinfo:

Quoted 1 line

modinfo -F scmversion MODULENAME

2) By module sysfs node:

Quoted 1 line

cat /sys/module/MODULENAME/scmversion

Hi Will, thanks for v4. I'm on vacation for the next two weeks and with the current merge window almost over, I hope it'd be OK with you if we revisit this early January. Just wanted to give you a heads up.

Thanks and happy holidays!

Jessica

#3 lore raw

+++ Will McVicker [06/01/21 10:44 -0800]:

Quoted 5 lines

Thanks for the vacation notice Jessica! I'm just letting you know I'm back as well and am happy to respond to any concerns regarding v4 when you get all caught up.

I hope you had a relaxing holiday :)

Hi Will - thanks, same to you!

Quoted 25 lines · Jessica Yu
Quoted 24 lines

+++ Will McVicker [16/12/20 22:08 +0000]:

Quoted 22 lines

Config MODULE_SCMVERSION introduces a new module attribute -- scmversion -- which can be used to identify a given module's SCM version. This is very useful for developers that update their kernel independently from their kernel modules or vice-versa since the SCM version provided by UTS_RELEASE (uname -r) will now differ from the module's vermagic attribute.

For example, we have a CI setup that tests new kernel changes on the hikey960 and db845c devices without updating their kernel modules. When these tests fail, we need to be able to identify the exact device configuration the test was using. By including MODULE_SCMVERSION, we can identify the exact kernel and modules' SCM versions for debugging the failures.

Additionally, by exposing the SCM version via the sysfs node /sys/module/MODULENAME/scmversion, one can also verify the SCM versions of the modules loaded from the initramfs. Currently, modinfo can only retrieve module attributes from the module's ko on disk and not from the actual module that is loaded in RAM.

You can retrieve the SCM version in two ways,

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

1) By using modinfo:

Quoted 1 line

modinfo -F scmversion MODULENAME

2) By module sysfs node:

Quoted 1 line

cat /sys/module/MODULENAME/scmversion

---

[ added back diff ]

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

Changelog since v3:
- Dropped [PATCH v2 1/2] scripts/setlocalversion: allow running in a subdir

Documentation/ABI/stable/sysfs-module | 18 ++++++++++++++++++
include/linux/module.h                |  1 +
init/Kconfig                          | 12 ++++++++++++
kernel/module.c                       |  2 ++
scripts/Makefile.modpost              | 22 ++++++++++++++++++++++
scripts/mod/modpost.c                 | 24 +++++++++++++++++++++++-
6 files changed, 78 insertions(+), 1 deletion(-)
Quoted 12 lines
Quoted 4 lines
Quoted 4 lines
Documentation/ABI/stable/sysfs-module +0 -0
index 6272ae5fb366..2ba731767737 100644
Quoted 8 lines

Documentation/ABI/stable/sysfs-module Description: · line 38

Quoted 8 lines
Documentation/ABI/stable/sysfs-module +4 -0
@@ -32,3 +32,21 @@ Description:
 		Note: If the module is built into the kernel, or if the
 		CONFIG_MODULE_UNLOAD kernel configuration value is not enabled,
 		this file will not be present.
+
+What:		/sys/module/MODULENAME/scmversion
+Date:		November 2020
+KernelVersion:	5.11

I guess we'll have to bump KernelVersion now (sorry about the timing!)

Quoted 6 lines
Quoted 6 lines

Documentation/ABI/stable/sysfs-module Description: · line 44

Quoted 6 lines
Documentation/ABI/stable/sysfs-module +6 -0
+Contact:	Will McVicker <[email protected]>
+Description:	This read-only file will appear if modpost was supplied with an
+		SCM version for the module. It can be enabled with the config
+		MODULE_SCMVERSION. The SCM version is retrieved by
+		scripts/setlocalversion, which means that the presence of this
+		file depends on CONFIG_LOCALVERSION_AUTO=y or LOCALVERSION=.

I think the "or LOCALVERSION=" part is inaccurate, right? We need just LOCALVERSION_AUTO for the full scm string for this to work.

Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Documentation/ABI/stable/sysfs-module +8 -0
+		When read, the SCM version that the module was compiled with is
+		returned. The SCM version is returned in the following format::
+
+		===
+		Git:		g[a-f0-9]\+(-dirty)\?
+		Mercurial:	hg[a-f0-9]\+(-dirty)\?
+		Subversion:	svn[0-9]\+
+		===
include/linux/module.h +1 -0
index c4e7a887f469..6bd710308863 100644
@@ -372,6 +372,7 @@ struct module {
 	struct module_attribute *modinfo_attrs;
 	const char *version;
 	const char *srcversion;
+	const char *scmversion;
 	struct kobject *holders_dir;
 
 	/* Exported symbols */
init/Kconfig +0 -0
index b77c60f8b963..d9ae12f16ba2 100644
@@ -2131,6 +2131,18 @@ config MODULE_SRCVERSION_ALL
 	  the version).  With this option, such a "srcversion" field
 	  will be created for all modules.  If unsure, say N.
 
Quoted 11 lines
Quoted 11 lines

init/Kconfig config MODULE_SRCVERSION_ALL · line 2144

Quoted 11 lines
init/Kconfig +11 -0
+config MODULE_SCMVERSION
+	bool "SCM version for modules"
+	depends on LOCALVERSION_AUTO
+	help
+	  This enables the module attribute "scmversion" which can be used
+	  by developers to identify the SCM version of a given module, e.g.
+	  git sha1 or hg sha1. The SCM version can be queried by modinfo or
+	  via the sysfs node: /sys/modules/MODULENAME/scmversion. This is
+	  useful when the kernel or kernel modules are updated separately
+	  since that causes the vermagic of the kernel and the module to
+	  differ.

Since I consider this a debug/developer option, let's add a "If unsure, say N." at the end of this, similar to the other module options.

Thanks!

Jessica

#4 lore raw

Hi Jessica,

I looked into the LOCALVERSION= part more and finally figured out how that part of the script works. I wasn't familiar with how the parameter substitution worked -- ${parameter:+alt_value}. I verified that we will only get an SCM version returned when LOCALVERSION_AUTO=y and updated the Documentation in v5.

Thanks,
Will

Quoted 126 lines · Jessica Yu

+++ Will McVicker [06/01/21 10:44 -0800]:

Quoted 5 lines

Thanks for the vacation notice Jessica! I'm just letting you know I'm back as well and am happy to respond to any concerns regarding v4 when you get all caught up.

I hope you had a relaxing holiday :)

Hi Will - thanks, same to you!

Quoted 25 lines · Jessica Yu
Quoted 24 lines

+++ Will McVicker [16/12/20 22:08 +0000]:

Quoted 22 lines

Config MODULE_SCMVERSION introduces a new module attribute -- scmversion -- which can be used to identify a given module's SCM version. This is very useful for developers that update their kernel independently from their kernel modules or vice-versa since the SCM version provided by UTS_RELEASE (uname -r) will now differ from the module's vermagic attribute.

For example, we have a CI setup that tests new kernel changes on the hikey960 and db845c devices without updating their kernel modules. When these tests fail, we need to be able to identify the exact device configuration the test was using. By including MODULE_SCMVERSION, we can identify the exact kernel and modules' SCM versions for debugging the failures.

Additionally, by exposing the SCM version via the sysfs node /sys/module/MODULENAME/scmversion, one can also verify the SCM versions of the modules loaded from the initramfs. Currently, modinfo can only retrieve module attributes from the module's ko on disk and not from the actual module that is loaded in RAM.

You can retrieve the SCM version in two ways,

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

1) By using modinfo:

Quoted 1 line

modinfo -F scmversion MODULENAME

2) By module sysfs node:

Quoted 1 line

cat /sys/module/MODULENAME/scmversion

---

[ added back diff ]

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

Changelog since v3:
- Dropped [PATCH v2 1/2] scripts/setlocalversion: allow running in a subdir

Documentation/ABI/stable/sysfs-module | 18 ++++++++++++++++++
include/linux/module.h                |  1 +
init/Kconfig                          | 12 ++++++++++++
kernel/module.c                       |  2 ++
scripts/Makefile.modpost              | 22 ++++++++++++++++++++++
scripts/mod/modpost.c                 | 24 +++++++++++++++++++++++-
6 files changed, 78 insertions(+), 1 deletion(-)
Quoted 12 lines
Quoted 4 lines
Quoted 4 lines
Documentation/ABI/stable/sysfs-module +0 -0
index 6272ae5fb366..2ba731767737 100644
Quoted 8 lines

Documentation/ABI/stable/sysfs-module Description: · line 38

Quoted 8 lines
Documentation/ABI/stable/sysfs-module +4 -0
@@ -32,3 +32,21 @@ Description:
           Note: If the module is built into the kernel, or if the
           CONFIG_MODULE_UNLOAD kernel configuration value is not enabled,
           this file will not be present.
+
+What:              /sys/module/MODULENAME/scmversion
+Date:              November 2020
+KernelVersion:     5.11

I guess we'll have to bump KernelVersion now (sorry about the timing!)

Quoted 6 lines
Quoted 6 lines

Documentation/ABI/stable/sysfs-module Description: · line 44

Quoted 6 lines
Documentation/ABI/stable/sysfs-module +6 -0
+Contact:   Will McVicker <[email protected]>
+Description:       This read-only file will appear if modpost was supplied with an
+           SCM version for the module. It can be enabled with the config
+           MODULE_SCMVERSION. The SCM version is retrieved by
+           scripts/setlocalversion, which means that the presence of this
+           file depends on CONFIG_LOCALVERSION_AUTO=y or LOCALVERSION=.

I think the "or LOCALVERSION=" part is inaccurate, right? We need just LOCALVERSION_AUTO for the full scm string for this to work.

Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Documentation/ABI/stable/sysfs-module +8 -0
+           When read, the SCM version that the module was compiled with is
+           returned. The SCM version is returned in the following format::
+
+           ===
+           Git:            g[a-f0-9]\+(-dirty)\?
+           Mercurial:      hg[a-f0-9]\+(-dirty)\?
+           Subversion:     svn[0-9]\+
+           ===
include/linux/module.h +1 -0
index c4e7a887f469..6bd710308863 100644
@@ -372,6 +372,7 @@ struct module {
   struct module_attribute *modinfo_attrs;
   const char *version;
   const char *srcversion;
+   const char *scmversion;
   struct kobject *holders_dir;
 
   /* Exported symbols */
init/Kconfig +0 -0
index b77c60f8b963..d9ae12f16ba2 100644
@@ -2131,6 +2131,18 @@ config MODULE_SRCVERSION_ALL
     the version).  With this option, such a "srcversion" field
     will be created for all modules.  If unsure, say N.
 
Quoted 11 lines
Quoted 11 lines

init/Kconfig config MODULE_SRCVERSION_ALL · line 2144

Quoted 11 lines
init/Kconfig +11 -0
+config MODULE_SCMVERSION
+   bool "SCM version for modules"
+   depends on LOCALVERSION_AUTO
+   help
+     This enables the module attribute "scmversion" which can be used
+     by developers to identify the SCM version of a given module, e.g.
+     git sha1 or hg sha1. The SCM version can be queried by modinfo or
+     via the sysfs node: /sys/modules/MODULENAME/scmversion. This is
+     useful when the kernel or kernel modules are updated separately
+     since that causes the vermagic of the kernel and the module to
+     differ.

Since I consider this a debug/developer option, let's add a "If unsure, say N." at the end of this, similar to the other module options.

Thanks!

Jessica