Skip to contentSkip to conversation

daxctl, util/sysfs: fix builtin-driver false failure on enable

PATCH Applied Chen Pei 3 participants 5 messages Jun 18 – Jul 22 libnvdimm: non-volatile memory device

lore
Share & tools
lore

Chen Pei

author
#0 lore raw

v3 cover daxctl, util/sysfs: fix builtin-driver false failure on enable

When a DAX / ndctl driver is builtin (not a loadable module), daxctl_insert_kmod_for_mode() and __util_bind() still call kmod_module_probe_insert_module() unconditionally. libkmod only short-circuits builtin modules when it can find the modules.builtin index; otherwise it falls through to init_module() and returns -ENOENT, surfacing as a spurious "insert failure".

Pre-check kmod_module_get_initstate() and skip probe-insert when the module is already BUILTIN or LIVE, matching the pattern used by ndctl's own test/core.c.

Changes since v2 [3]:
- Patch 2/2: Add a Reviewed-by tag.

Changes since v1 [1]:
- Patch 1/2: unchanged; collected Reviewed-by from Dave and Alison.
- Patch 2/2: factored the state check into a new helper util_kmod_skip_probe_insert() in util/sysfs.{c,h} so both daxctl_insert_kmod_for_mode() and __util_bind() share it. The helper also returns the observed libkmod state via an out parameter so the caller does not re-read /sys/module/<name>/ initstate to distinguish LIVE from BUILTIN.
- Patch 2/2: additionally treat KMOD_MODULE_COMING as builtin when /sys/module/<name>/ exists but the initstate file does not. This is the pattern libkmod's sysfs fallback emits for builtin drivers when the modules.builtin index is missing (e.g. a kernel installed without running modules_install). This was the case Jonathan hit on a builtin DAX VM setup; rather than rely on a libkmod fix, ndctl handles the corner case directly. Suggested by Alison [2].

[1]: https://lore.kernel.org/nvdimm/[email protected]/
[2]: https://lore.kernel.org/nvdimm/[email protected]/
[3]: https://lore.kernel.org/nvdimm/[email protected]/

Chen Pei (2):
daxctl: fix kmod reference leak on probe-insert failure daxctl, util/sysfs: skip module probe-insert when driver is builtin or
live

 daxctl/lib/libdaxctl.c | 23 ++++++++++++++++++++--
 util/sysfs.c           | 44 +++++++++++++++++++++++++++++++++++++++++-
 util/sysfs.h           | 16 +++++++++++++++
 3 files changed, 80 insertions(+), 3 deletions(-)
Signature · 1 line
2.43.0
author
#1 lore raw

PATCH v3 1/2 daxctl: fix kmod reference leak on probe-insert failure

daxctl_insert_kmod_for_mode() obtains a kmod reference via kmod_module_new_from_name() and only stores it in dev->module after a successful kmod_module_probe_insert_module() call. On the failure path the local reference was returned without being released, leaking one reference per failed enable attempt.

Drop the reference before returning the error code.

---
 daxctl/lib/libdaxctl.c | 1 +
 1 file changed, 1 insertion(+)
daxctl/lib/libdaxctl.c +1 -0
index 01b1915..8c3ac47 100644
@@ -975,6 +975,7 @@ static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev,
 			NULL, NULL, NULL, NULL);
 	if (rc < 0) {
 		err(ctx, "%s: insert failure: %d\n", devname, rc);
+		kmod_module_unref(kmod);
 		return rc;
 	}
 	dev->module = kmod;
Signature · 1 line
2.43.0
author
#2 lore raw

PATCH v3 2/2 daxctl, util/sysfs: skip module probe-insert when driver is builtin or live

In reply to Chen Pei

kmod_module_probe_insert_module() is supposed to return 0 for builtin modules, but only when libkmod can locate the modules.builtin index. If the index is missing (e.g. a kernel built with the driver as builtin but installed without running modules_install), libkmod falls through to the real init_module() syscall and returns an error such as -ENOENT, producing a spurious "insert failure" even though the driver is already part of the running kernel.

Add a helper util_kmod_skip_probe_insert() that returns true when the module state is KMOD_MODULE_BUILTIN or KMOD_MODULE_LIVE. As an additional heuristic, treat KMOD_MODULE_COMING as builtin when /sys/module/<name>/ exists but the initstate file does not - this is the exact pattern libkmod's sysfs fallback emits for builtin drivers when the modules.builtin index is unavailable. The pattern mirrors the KMOD_MODULE_LIVE / KMOD_MODULE_BUILTIN check already used by ndctl's own test/core.c (see test/core.c:218-236).

The helper also returns the observed libkmod state via an out parameter so daxctl_insert_kmod_for_mode() can distinguish LIVE (retain the kmod reference in dev->module) from BUILTIN (drop it, since builtin drivers cannot be unloaded) without re-reading /sys/module/<name>/initstate. __util_bind() passes NULL since it does not need the state.

---
 daxctl/lib/libdaxctl.c | 22 +++++++++++++++++++--
 util/sysfs.c           | 44 +++++++++++++++++++++++++++++++++++++++++-
 util/sysfs.h           | 16 +++++++++++++++
 3 files changed, 79 insertions(+), 3 deletions(-)
daxctl/lib/libdaxctl.c +20 -2
index 8c3ac47..5b47c77 100644
@@ -958,7 +958,7 @@ static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev,
 	const char *devname = daxctl_dev_get_devname(dev);
 	struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev);
 	struct kmod_module *kmod;
-	int rc;
+	int state, rc;
 
 	rc = kmod_module_new_from_name(ctx->kmod_ctx, mod_name, &kmod);
 	if (rc < 0) {
@@ -967,7 +967,25 @@ static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev,
 		return rc;
 	}
 
-	/* if the driver is builtin, this Just Works */
+	/*
+	 * If the driver is builtin or already live, skip probe-insert.
+	 * For live modules retain the local reference in dev->module so
+	 * the module can be unreffed alongside the device; for builtin
+	 * drivers drop it because builtin modules cannot be unloaded.
+	 */
+	if (util_kmod_skip_probe_insert(kmod, ctx, &state)) {
+		if (state == KMOD_MODULE_LIVE) {
+			dbg(ctx, "%s: module %s already loaded\n", devname,
+				kmod_module_get_name(kmod));
+			dev->module = kmod;
+		} else {
+			dbg(ctx, "%s: module %s is builtin\n", devname,
+				kmod_module_get_name(kmod));
+			kmod_module_unref(kmod);
+		}
+		return 0;
+	}
+
 	dbg(ctx, "%s inserting module: %s\n", devname,
 		kmod_module_get_name(kmod));
 	rc = kmod_module_probe_insert_module(kmod,
util/sysfs.c +43 -1

62 lines not on this page · raw message

util/sysfs.h +16 -0
index 4c95c70..e4f6115 100644
@@ -3,6 +3,7 @@
 #ifndef __UTIL_SYSFS_H__
 #define __UTIL_SYSFS_H__
 
+#include <stdbool.h>
 #include <string.h>
 
 typedef void *(*add_dev_fn)(void *parent, int id, const char *dev_path);
@@ -36,6 +37,21 @@ struct kmod_module *__util_modalias_to_module(struct kmod_ctx *kmod_ctx,
 #define util_modalias_to_module(ctx, buf)                                      \
 	__util_modalias_to_module((ctx)->kmod_ctx, buf, &(ctx)->ctx)
 
+/*
+ * __util_kmod_skip_probe_insert - true when kmod_module_probe_insert_module()
+ * should be skipped because @module is already part of the running kernel:
+ * KMOD_MODULE_BUILTIN, KMOD_MODULE_LIVE, or KMOD_MODULE_COMING with
+ * /sys/module/<name>/ existing but no initstate file (the fingerprint
+ * libkmod's sysfs fallback emits for builtin drivers when the
+ * modules.builtin index is missing). If @state_out is non-NULL, the
+ * libkmod state actually observed is stored there so callers can avoid
+ * an extra kmod_module_get_initstate() call.
+ */
+bool __util_kmod_skip_probe_insert(struct kmod_module *module,
+				   struct log_ctx *ctx, int *state_out);
+#define util_kmod_skip_probe_insert(m, c, s)                                   \
+	__util_kmod_skip_probe_insert((m), &(c)->ctx, (s))
+
 int __util_bind(const char *devname, struct kmod_module *module, const char *bus,
 	      struct log_ctx *ctx);
 #define util_bind(n, m, b, c) __util_bind(n, m, b, &(c)->ctx)
Signature · 1 line
2.43.0
Reviewed-by on v3 2/2
#3 lore raw

Re: [ndctl PATCH v3 2/2] daxctl, util/sysfs: skip module probe-insert when driver is builtin or live

Quoted 23 lines · On Thu, 18 Jun 2026 17:06:53 +0800 Chen Pei <[email protected]> wrote:

kmod_module_probe_insert_module() is supposed to return 0 for builtin modules, but only when libkmod can locate the modules.builtin index. If the index is missing (e.g. a kernel built with the driver as builtin but installed without running modules_install), libkmod falls through to the real init_module() syscall and returns an error such as -ENOENT, producing a spurious "insert failure" even though the driver is already part of the running kernel.

Add a helper util_kmod_skip_probe_insert() that returns true when the module state is KMOD_MODULE_BUILTIN or KMOD_MODULE_LIVE. As an additional heuristic, treat KMOD_MODULE_COMING as builtin when /sys/module/<name>/ exists but the initstate file does not - this is the exact pattern libkmod's sysfs fallback emits for builtin drivers when the modules.builtin index is unavailable. The pattern mirrors the KMOD_MODULE_LIVE / KMOD_MODULE_BUILTIN check already used by ndctl's own test/core.c (see test/core.c:218-236).

The helper also returns the observed libkmod state via an out parameter so daxctl_insert_kmod_for_mode() can distinguish LIVE (retain the kmod reference in dev->module) from BUILTIN (drop it, since builtin drivers cannot be unloaded) without re-reading /sys/module/<name>/initstate. __util_bind() passes NULL since it does not need the state.

Quoted 4 lines

I'm not set up to test this right now but fix looks good to me. So a tentative

maintainer Applied
#4 lore raw
In reply to Chen Pei
Quoted 30 lines · Chen Pei

When a DAX / ndctl driver is builtin (not a loadable module), daxctl_insert_kmod_for_mode() and __util_bind() still call kmod_module_probe_insert_module() unconditionally. libkmod only short-circuits builtin modules when it can find the modules.builtin index; otherwise it falls through to init_module() and returns -ENOENT, surfacing as a spurious "insert failure".

Pre-check kmod_module_get_initstate() and skip probe-insert when the module is already BUILTIN or LIVE, matching the pattern used by ndctl's own test/core.c.

Changes since v2 [3]:
- Patch 2/2: Add a Reviewed-by tag.

Changes since v1 [1]:
- Patch 1/2: unchanged; collected Reviewed-by from Dave and Alison.
- Patch 2/2: factored the state check into a new helper util_kmod_skip_probe_insert() in util/sysfs.{c,h} so both daxctl_insert_kmod_for_mode() and __util_bind() share it. The helper also returns the observed libkmod state via an out parameter so the caller does not re-read /sys/module/<name>/ initstate to distinguish LIVE from BUILTIN.
- Patch 2/2: additionally treat KMOD_MODULE_COMING as builtin when /sys/module/<name>/ exists but the initstate file does not. This is the pattern libkmod's sysfs fallback emits for builtin drivers when the modules.builtin index is missing (e.g. a kernel installed without running modules_install). This was the case Jonathan hit on a builtin DAX VM setup; rather than rely on a libkmod fix, ndctl handles the corner case directly. Suggested by Alison [2].

Quoted 8 lines

[1]: https://lore.kernel.org/nvdimm/[email protected]/
[2]: https://lore.kernel.org/nvdimm/[email protected]/
[3]: https://lore.kernel.org/nvdimm/[email protected]/

Chen Pei (2):
daxctl: fix kmod reference leak on probe-insert failure daxctl, util/sysfs: skip module probe-insert when driver is builtin or
live

Thanks!

Applied to pending for NDCTL v86
https://github.com/pmem/ndctl/commits/pending/