Skip to contentSkip to conversation

daxctl, util/sysfs: fix builtin-driver false failure on enable

PATCH Superseded Chen Pei 5 participants 14 messages May 14 – May 26 libnvdimm: non-volatile memory device

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Chen Pei

author
#0 lore raw

cover daxctl, util/sysfs: fix builtin-driver false failure on enable

When a DAX / ndctl driver is builtin (not a loadable module), daxctl_insert_kmod_for_mode() and __util_bind() still call kmod_module_probe_insert_module() unconditionally. libkmod only short-circuits builtin modules when it can find the modules.builtin index; otherwise it falls through to init_module() and returns -ENOENT, surfacing as a spurious "insert failure".

Pre-check kmod_module_get_initstate() and skip probe-insert when the module is already BUILTIN or LIVE, matching the pattern used by ndctl's own test/core.c.

Chen Pei (2):
daxctl: fix kmod reference leak on probe-insert failure daxctl, util/sysfs: skip module probe-insert when driver is builtin or
live

 daxctl/lib/libdaxctl.c | 19 +++++++++++++++++--
 util/sysfs.c           | 17 +++++++++++------
 2 files changed, 28 insertions(+), 8 deletions(-)
Signature · 1 line
2.43.0
author
#1 lore raw

PATCH 1/2 daxctl: fix kmod reference leak on probe-insert failure

daxctl_insert_kmod_for_mode() obtains a kmod reference via kmod_module_new_from_name() and only stores it in dev->module after a successful kmod_module_probe_insert_module() call. On the failure path the local reference was returned without being released, leaking one reference per failed enable attempt.

Drop the reference before returning the error code.

---
 daxctl/lib/libdaxctl.c | 1 +
 1 file changed, 1 insertion(+)
daxctl/lib/libdaxctl.c +1 -0
index 02ae7e5..ffc81eb 100644
@@ -927,6 +927,7 @@ static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev,
 			NULL, NULL, NULL, NULL);
 	if (rc < 0) {
 		err(ctx, "%s: insert failure: %d\n", devname, rc);
+		kmod_module_unref(kmod);
 		return rc;
 	}
 	dev->module = kmod;
Signature · 1 line
2.43.0
maintainer Reviewed-by on v1 1/2
#2 lore raw

Re: [ndctl PATCH 1/2] daxctl: fix kmod reference leak on probe-insert failure

Quoted 9 lines · Chen Pei

daxctl_insert_kmod_for_mode() obtains a kmod reference via kmod_module_new_from_name() and only stores it in dev->module after a successful kmod_module_probe_insert_module() call. On the failure path the local reference was returned without being released, leaking one reference per failed enable attempt.

Drop the reference before returning the error code.

Quoted 16 lines
---
 daxctl/lib/libdaxctl.c | 1 +
 1 file changed, 1 insertion(+)
daxctl/lib/libdaxctl.c +1 -0
index 02ae7e5..ffc81eb 100644
@@ -927,6 +927,7 @@ static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev,
 			NULL, NULL, NULL, NULL);
 	if (rc < 0) {
 		err(ctx, "%s: insert failure: %d\n", devname, rc);
+		kmod_module_unref(kmod);
 		return rc;
 	}
 	dev->module = kmod;
maintainer Reviewed-by on v1 1/2
#3 lore raw

Re: [ndctl PATCH 1/2] daxctl: fix kmod reference leak on probe-insert failure

In reply to Chen Pei
Quoted 7 lines · Chen Pei

daxctl_insert_kmod_for_mode() obtains a kmod reference via kmod_module_new_from_name() and only stores it in dev->module after a successful kmod_module_probe_insert_module() call. On the failure path the local reference was returned without being released, leaking one reference per failed enable attempt.

Drop the reference before returning the error code.

Thanks!

Reviewed-by on v1 1/2
#4 lore raw

Re: [ndctl PATCH 1/2] daxctl: fix kmod reference leak on probe-insert failure

In reply to Chen Pei
Quoted 27 lines · Chen Pei

daxctl_insert_kmod_for_mode() obtains a kmod reference via kmod_module_new_from_name() and only stores it in dev->module after a successful kmod_module_probe_insert_module() call. On the failure path the local reference was returned without being released, leaking one reference per failed enable attempt.

Drop the reference before returning the error code.

---
 daxctl/lib/libdaxctl.c | 1 +
 1 file changed, 1 insertion(+)
daxctl/lib/libdaxctl.c +1 -1
index 02ae7e5..ffc81eb 100644
@@ -927,6 +927,7 @@ static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev,
                        NULL, NULL, NULL, NULL);
        if (rc < 0) {
                err(ctx, "%s: insert failure: %d\n", devname, rc);
+               kmod_module_unref(kmod);
                return rc;
        }
        dev->module = kmod;
--
Quoted 2 lines

2.43.0

Signature · 2 lines
Best Regards
 Guo Ren
author
#5 lore raw

PATCH 2/2 daxctl, util/sysfs: skip module probe-insert when driver is builtin or live

In reply to Chen Pei

kmod_module_probe_insert_module() is supposed to return 0 for builtin modules, but only when libkmod can locate the modules.builtin index. If the index is missing or out of sync, libkmod falls through to the real init_module() syscall and returns an error such as -ENOENT, producing a spurious "insert failure" even though the driver is already part of the running kernel.

Pre-check kmod_module_get_initstate() and short-circuit when the module is KMOD_MODULE_BUILTIN or KMOD_MODULE_LIVE, matching the pattern used by ndctl's own test/core.c.

For builtin modules the local kmod reference is dropped because builtin drivers cannot be unloaded; for live modules the reference is retained in dev->module, matching the post-probe-success behavior.

---
 daxctl/lib/libdaxctl.c | 18 ++++++++++++++++--
 util/sysfs.c           | 17 +++++++++++------
 2 files changed, 27 insertions(+), 8 deletions(-)
daxctl/lib/libdaxctl.c +16 -2
index ffc81eb..42bfc39 100644
@@ -910,7 +910,7 @@ static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev,
 	const char *devname = daxctl_dev_get_devname(dev);
 	struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev);
 	struct kmod_module *kmod;
-	int rc;
+	int state, rc;
 
 	rc = kmod_module_new_from_name(ctx->kmod_ctx, mod_name, &kmod);
 	if (rc < 0) {
@@ -919,7 +919,21 @@ static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev,
 		return rc;
 	}
 
-	/* if the driver is builtin, this Just Works */
+	/* If the driver is builtin or already live, skip probe-insert. */
+	state = kmod_module_get_initstate(kmod);
+	if (state == KMOD_MODULE_BUILTIN) {
+		dbg(ctx, "%s: module %s is builtin\n", devname,
+			kmod_module_get_name(kmod));
+		kmod_module_unref(kmod);
+		return 0;
+	}
+	if (state == KMOD_MODULE_LIVE) {
+		dbg(ctx, "%s: module %s already loaded\n", devname,
+			kmod_module_get_name(kmod));
+		dev->module = kmod;
+		return 0;
+	}
+
 	dbg(ctx, "%s inserting module: %s\n", devname,
 		kmod_module_get_name(kmod));
 	rc = kmod_module_probe_insert_module(kmod,
util/sysfs.c +11 -6
index e027e38..641b86d 100644
@@ -183,12 +183,17 @@ int __util_bind(const char *devname, struct kmod_module *module,
 	}
 
 	if (module) {
-		rc = kmod_module_probe_insert_module(module,
-						     KMOD_PROBE_APPLY_BLACKLIST,
-						     NULL, NULL, NULL, NULL);
-		if (rc < 0) {
-			log_err(ctx, "%s: insert failure: %d\n", __func__, rc);
-			return rc;
+		/* Skip probe-insert when the module is already builtin or live. */
+		int state = kmod_module_get_initstate(module);
+
+		if (state != KMOD_MODULE_BUILTIN && state != KMOD_MODULE_LIVE) {
+			rc = kmod_module_probe_insert_module(module,
+							     KMOD_PROBE_APPLY_BLACKLIST,
+							     NULL, NULL, NULL, NULL);
+			if (rc < 0) {
+				log_err(ctx, "%s: insert failure: %d\n", __func__, rc);
+				return rc;
+			}
 		}
 	}
 
Signature · 1 line
2.43.0
maintainer Reviewed-by on v1 2/2
#6 lore raw

Re: [ndctl PATCH 2/2] daxctl, util/sysfs: skip module probe-insert when driver is builtin or live

Quoted 11 lines · Chen Pei

kmod_module_probe_insert_module() is supposed to return 0 for builtin modules, but only when libkmod can locate the modules.builtin index. If the index is missing or out of sync, libkmod falls through to the real init_module() syscall and returns an error such as -ENOENT, producing a spurious "insert failure" even though the driver is already part of the running kernel.

Pre-check kmod_module_get_initstate() and short-circuit when the module is KMOD_MODULE_BUILTIN or KMOD_MODULE_LIVE, matching the pattern used by ndctl's own test/core.c.

Quoted 5 lines

For builtin modules the local kmod reference is dropped because builtin drivers cannot be unloaded; for live modules the reference is retained in dev->module, matching the post-probe-success behavior.

Quoted 69 lines
---
 daxctl/lib/libdaxctl.c | 18 ++++++++++++++++--
 util/sysfs.c           | 17 +++++++++++------
 2 files changed, 27 insertions(+), 8 deletions(-)
daxctl/lib/libdaxctl.c +16 -2
index ffc81eb..42bfc39 100644
@@ -910,7 +910,7 @@ static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev,
 	const char *devname = daxctl_dev_get_devname(dev);
 	struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev);
 	struct kmod_module *kmod;
-	int rc;
+	int state, rc;
 
 	rc = kmod_module_new_from_name(ctx->kmod_ctx, mod_name, &kmod);
 	if (rc < 0) {
@@ -919,7 +919,21 @@ static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev,
 		return rc;
 	}
 
-	/* if the driver is builtin, this Just Works */
+	/* If the driver is builtin or already live, skip probe-insert. */
+	state = kmod_module_get_initstate(kmod);
+	if (state == KMOD_MODULE_BUILTIN) {
+		dbg(ctx, "%s: module %s is builtin\n", devname,
+			kmod_module_get_name(kmod));
+		kmod_module_unref(kmod);
+		return 0;
+	}
+	if (state == KMOD_MODULE_LIVE) {
+		dbg(ctx, "%s: module %s already loaded\n", devname,
+			kmod_module_get_name(kmod));
+		dev->module = kmod;
+		return 0;
+	}
+
 	dbg(ctx, "%s inserting module: %s\n", devname,
 		kmod_module_get_name(kmod));
 	rc = kmod_module_probe_insert_module(kmod,
util/sysfs.c +11 -6
index e027e38..641b86d 100644
@@ -183,12 +183,17 @@ int __util_bind(const char *devname, struct kmod_module *module,
 	}
 
 	if (module) {
-		rc = kmod_module_probe_insert_module(module,
-						     KMOD_PROBE_APPLY_BLACKLIST,
-						     NULL, NULL, NULL, NULL);
-		if (rc < 0) {
-			log_err(ctx, "%s: insert failure: %d\n", __func__, rc);
-			return rc;
+		/* Skip probe-insert when the module is already builtin or live. */
+		int state = kmod_module_get_initstate(module);
+
+		if (state != KMOD_MODULE_BUILTIN && state != KMOD_MODULE_LIVE) {
+			rc = kmod_module_probe_insert_module(module,
+							     KMOD_PROBE_APPLY_BLACKLIST,
+							     NULL, NULL, NULL, NULL);
+			if (rc < 0) {
+				log_err(ctx, "%s: insert failure: %d\n", __func__, rc);
+				return rc;
+			}
 		}
 	}
 
#7 lore raw

Re: [ndctl PATCH 2/2] daxctl, util/sysfs: skip module probe-insert when driver is builtin or live

In reply to Chen Pei
Quoted 10 lines · On Thu, 14 May 2026 14:32:34 +0800 Chen Pei <[email protected]> wrote:

kmod_module_probe_insert_module() is supposed to return 0 for builtin modules, but only when libkmod can locate the modules.builtin index. If the index is missing or out of sync, libkmod falls through to the real init_module() syscall and returns an error such as -ENOENT, producing a spurious "insert failure" even though the driver is already part of the running kernel.

Pre-check kmod_module_get_initstate() and short-circuit when the module is KMOD_MODULE_BUILTIN or KMOD_MODULE_LIVE, matching the pattern used by ndctl's own test/core.c.

So I happened to run into exactly this print earlier today and was very happy to see this resolving it! I'm lazy so when developing in a VM tend to do everything I care about built in and not bother with installing the modules.

However - despite having CONFIG_DEV_DAX = y in the kernel, I'm getting a state of KMOD_MODULE_COMING which is curious as there is no initstate file to read that from.

Looking at the code in libkmod it seems to first check if it can open /sys/modules/device_dax/initstate and if it can't checks if
the directory /sys/modules/device_dax/ exists. If it finds that it returns KMOD_MODULE_COMING which seems odd given in a fully initialized built in driver that particular set of circumstances is normal.

Any ideas?

To me the description above is misleading if we need to have something else for the builtin case to work.

I'm out of time to today but may get time to look at this tomorrow and chase down if there is a way to get it to work.

Jonathan

Quoted 75 lines

For builtin modules the local kmod reference is dropped because builtin drivers cannot be unloaded; for live modules the reference is retained in dev->module, matching the post-probe-success behavior.

---
 daxctl/lib/libdaxctl.c | 18 ++++++++++++++++--
 util/sysfs.c           | 17 +++++++++++------
 2 files changed, 27 insertions(+), 8 deletions(-)
daxctl/lib/libdaxctl.c +16 -2
index ffc81eb..42bfc39 100644
@@ -910,7 +910,7 @@ static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev,
 	const char *devname = daxctl_dev_get_devname(dev);
 	struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev);
 	struct kmod_module *kmod;
-	int rc;
+	int state, rc;
 
 	rc = kmod_module_new_from_name(ctx->kmod_ctx, mod_name, &kmod);
 	if (rc < 0) {
@@ -919,7 +919,21 @@ static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev,
 		return rc;
 	}
 
-	/* if the driver is builtin, this Just Works */
+	/* If the driver is builtin or already live, skip probe-insert. */
+	state = kmod_module_get_initstate(kmod);
+	if (state == KMOD_MODULE_BUILTIN) {
+		dbg(ctx, "%s: module %s is builtin\n", devname,
+			kmod_module_get_name(kmod));
+		kmod_module_unref(kmod);
+		return 0;
+	}
+	if (state == KMOD_MODULE_LIVE) {
+		dbg(ctx, "%s: module %s already loaded\n", devname,
+			kmod_module_get_name(kmod));
+		dev->module = kmod;
+		return 0;
+	}
+
 	dbg(ctx, "%s inserting module: %s\n", devname,
 		kmod_module_get_name(kmod));
 	rc = kmod_module_probe_insert_module(kmod,
util/sysfs.c +11 -6
index e027e38..641b86d 100644
@@ -183,12 +183,17 @@ int __util_bind(const char *devname, struct kmod_module *module,
 	}
 
 	if (module) {
-		rc = kmod_module_probe_insert_module(module,
-						     KMOD_PROBE_APPLY_BLACKLIST,
-						     NULL, NULL, NULL, NULL);
-		if (rc < 0) {
-			log_err(ctx, "%s: insert failure: %d\n", __func__, rc);
-			return rc;
+		/* Skip probe-insert when the module is already builtin or live. */
+		int state = kmod_module_get_initstate(module);
+
+		if (state != KMOD_MODULE_BUILTIN && state != KMOD_MODULE_LIVE) {
+			rc = kmod_module_probe_insert_module(module,
+							     KMOD_PROBE_APPLY_BLACKLIST,
+							     NULL, NULL, NULL, NULL);
+			if (rc < 0) {
+				log_err(ctx, "%s: insert failure: %d\n", __func__, rc);
+				return rc;
+			}
 		}
 	}
 
maintainer
#8 lore raw

Re: [ndctl PATCH 2/2] daxctl, util/sysfs: skip module probe-insert when driver is builtin or live

Quoted 14 lines · Jonathan Cameron
Quoted 10 lines · On Thu, 14 May 2026 14:32:34 +0800 Chen Pei <[email protected]> wrote:

kmod_module_probe_insert_module() is supposed to return 0 for builtin modules, but only when libkmod can locate the modules.builtin index. If the index is missing or out of sync, libkmod falls through to the real init_module() syscall and returns an error such as -ENOENT, producing a spurious "insert failure" even though the driver is already part of the running kernel.

Pre-check kmod_module_get_initstate() and short-circuit when the module is KMOD_MODULE_BUILTIN or KMOD_MODULE_LIVE, matching the pattern used by ndctl's own test/core.c.

Quoted 8 lines

So I happened to run into exactly this print earlier today and was very happy to see this resolving it! I'm lazy so when developing in a VM tend to do everything I care about built in and not bother with installing the modules.

However - despite having CONFIG_DEV_DAX = y in the kernel, I'm getting a state of KMOD_MODULE_COMING which is curious as there is no initstate file to read that from.

I think this patch is worth you trying. In libmkmod code I'm looking at:

https://github.com/lucasdemarchi/kmod/blob/master/libkmod/libkmod-module.c

the "module directory exists but initstate cannot be opened" case returns KMOD_MODULE_BUILTIN, not KMOD_MODULE_COMING.

So if device_dax is builtin and /sys/module/device_dax exists without initstate, this patch should short-circuit before attempting insert. If you still see COMING with this patch applied, then we need to figure out where that state is coming from (before thinking about special casing it in ndctl).

Quoted 94 lines

Looking at the code in libkmod it seems to first check if it can open /sys/modules/device_dax/initstate and if it can't checks if
the directory /sys/modules/device_dax/ exists. If it finds that it returns KMOD_MODULE_COMING which seems odd given in a fully initialized built in driver that particular set of circumstances is normal.

Any ideas?

To me the description above is misleading if we need to have something else for the builtin case to work.

I'm out of time to today but may get time to look at this tomorrow and chase down if there is a way to get it to work.

Jonathan

Quoted 75 lines

For builtin modules the local kmod reference is dropped because builtin drivers cannot be unloaded; for live modules the reference is retained in dev->module, matching the post-probe-success behavior.

---
 daxctl/lib/libdaxctl.c | 18 ++++++++++++++++--
 util/sysfs.c           | 17 +++++++++++------
 2 files changed, 27 insertions(+), 8 deletions(-)
daxctl/lib/libdaxctl.c +16 -2
index ffc81eb..42bfc39 100644
@@ -910,7 +910,7 @@ static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev,
 	const char *devname = daxctl_dev_get_devname(dev);
 	struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev);
 	struct kmod_module *kmod;
-	int rc;
+	int state, rc;
 
 	rc = kmod_module_new_from_name(ctx->kmod_ctx, mod_name, &kmod);
 	if (rc < 0) {
@@ -919,7 +919,21 @@ static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev,
 		return rc;
 	}
 
-	/* if the driver is builtin, this Just Works */
+	/* If the driver is builtin or already live, skip probe-insert. */
+	state = kmod_module_get_initstate(kmod);
+	if (state == KMOD_MODULE_BUILTIN) {
+		dbg(ctx, "%s: module %s is builtin\n", devname,
+			kmod_module_get_name(kmod));
+		kmod_module_unref(kmod);
+		return 0;
+	}
+	if (state == KMOD_MODULE_LIVE) {
+		dbg(ctx, "%s: module %s already loaded\n", devname,
+			kmod_module_get_name(kmod));
+		dev->module = kmod;
+		return 0;
+	}
+
 	dbg(ctx, "%s inserting module: %s\n", devname,
 		kmod_module_get_name(kmod));
 	rc = kmod_module_probe_insert_module(kmod,
util/sysfs.c +11 -6
index e027e38..641b86d 100644
@@ -183,12 +183,17 @@ int __util_bind(const char *devname, struct kmod_module *module,
 	}
 
 	if (module) {
-		rc = kmod_module_probe_insert_module(module,
-						     KMOD_PROBE_APPLY_BLACKLIST,
-						     NULL, NULL, NULL, NULL);
-		if (rc < 0) {
-			log_err(ctx, "%s: insert failure: %d\n", __func__, rc);
-			return rc;
+		/* Skip probe-insert when the module is already builtin or live. */
+		int state = kmod_module_get_initstate(module);
+
+		if (state != KMOD_MODULE_BUILTIN && state != KMOD_MODULE_LIVE) {
+			rc = kmod_module_probe_insert_module(module,
+							     KMOD_PROBE_APPLY_BLACKLIST,
+							     NULL, NULL, NULL, NULL);
+			if (rc < 0) {
+				log_err(ctx, "%s: insert failure: %d\n", __func__, rc);
+				return rc;
+			}
 		}
 	}
 
#9 lore raw

Re: [ndctl PATCH 2/2] daxctl, util/sysfs: skip module probe-insert when driver is builtin or live

Quoted 20 lines · On Thu, 14 May 2026 15:13:20 -0700 Alison Schofield…
Quoted 19 lines · Jonathan Cameron
Quoted 10 lines · On Thu, 14 May 2026 14:32:34 +0800 Chen Pei <[email protected]> wrote:

kmod_module_probe_insert_module() is supposed to return 0 for builtin modules, but only when libkmod can locate the modules.builtin index. If the index is missing or out of sync, libkmod falls through to the real init_module() syscall and returns an error such as -ENOENT, producing a spurious "insert failure" even though the driver is already part of the running kernel.

Pre-check kmod_module_get_initstate() and short-circuit when the module is KMOD_MODULE_BUILTIN or KMOD_MODULE_LIVE, matching the pattern used by ndctl's own test/core.c.

So I happened to run into exactly this print earlier today and was very happy to see this resolving it! I'm lazy so when developing in a VM tend to do everything I care about built in and not bother with installing the modules.

Quoted 5 lines
Quoted 3 lines

However - despite having CONFIG_DEV_DAX = y in the kernel, I'm getting a state of KMOD_MODULE_COMING which is curious as there is no initstate file to read that from.

I think this patch is worth you trying. In libmkmod code I'm looking at:

It doesn't work - hence the reply!

Quoted 5 lines

https://github.com/lucasdemarchi/kmod/blob/master/libkmod/libkmod-module.c

the "module directory exists but initstate cannot be opened" case returns KMOD_MODULE_BUILTIN, not KMOD_MODULE_COMING.

I'm not following... Also...
https://github.com/kmod-project/kmod/tree/master/libkmod
Is a lot more recent than that tree of Lucas and I'm guessing the current home given it has 2 week old commits from Lucas.

Can you give me a line number for the path you are talking about because even in that code of Lucas I'm failing to see it. Note the kmod_module_is_buitin() fails for the reason this patch is trying to fix. The file to check that isn't there - hence we hit the path that tries to figure it out from sysfs. I can't see any other path to a KMOD_MODULE_BUILTIN.

Jonathan

Quoted 101 lines

So if device_dax is builtin and /sys/module/device_dax exists without initstate, this patch should short-circuit before attempting insert. If you still see COMING with this patch applied, then we need to figure out where that state is coming from (before thinking about special casing it in ndctl).

Quoted 94 lines

Looking at the code in libkmod it seems to first check if it can open /sys/modules/device_dax/initstate and if it can't checks if
the directory /sys/modules/device_dax/ exists. If it finds that it returns KMOD_MODULE_COMING which seems odd given in a fully initialized built in driver that particular set of circumstances is normal.

Any ideas?

To me the description above is misleading if we need to have something else for the builtin case to work.

I'm out of time to today but may get time to look at this tomorrow and chase down if there is a way to get it to work.

Jonathan

Quoted 75 lines

For builtin modules the local kmod reference is dropped because builtin drivers cannot be unloaded; for live modules the reference is retained in dev->module, matching the post-probe-success behavior.

---
 daxctl/lib/libdaxctl.c | 18 ++++++++++++++++--
 util/sysfs.c           | 17 +++++++++++------
 2 files changed, 27 insertions(+), 8 deletions(-)
daxctl/lib/libdaxctl.c +16 -2
index ffc81eb..42bfc39 100644
@@ -910,7 +910,7 @@ static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev,
 	const char *devname = daxctl_dev_get_devname(dev);
 	struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev);
 	struct kmod_module *kmod;
-	int rc;
+	int state, rc;
 
 	rc = kmod_module_new_from_name(ctx->kmod_ctx, mod_name, &kmod);
 	if (rc < 0) {
@@ -919,7 +919,21 @@ static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev,
 		return rc;
 	}
 
-	/* if the driver is builtin, this Just Works */
+	/* If the driver is builtin or already live, skip probe-insert. */
+	state = kmod_module_get_initstate(kmod);
+	if (state == KMOD_MODULE_BUILTIN) {
+		dbg(ctx, "%s: module %s is builtin\n", devname,
+			kmod_module_get_name(kmod));
+		kmod_module_unref(kmod);
+		return 0;
+	}
+	if (state == KMOD_MODULE_LIVE) {
+		dbg(ctx, "%s: module %s already loaded\n", devname,
+			kmod_module_get_name(kmod));
+		dev->module = kmod;
+		return 0;
+	}
+
 	dbg(ctx, "%s inserting module: %s\n", devname,
 		kmod_module_get_name(kmod));
 	rc = kmod_module_probe_insert_module(kmod,
util/sysfs.c +11 -6
index e027e38..641b86d 100644
@@ -183,12 +183,17 @@ int __util_bind(const char *devname, struct kmod_module *module,
 	}
 
 	if (module) {
-		rc = kmod_module_probe_insert_module(module,
-						     KMOD_PROBE_APPLY_BLACKLIST,
-						     NULL, NULL, NULL, NULL);
-		if (rc < 0) {
-			log_err(ctx, "%s: insert failure: %d\n", __func__, rc);
-			return rc;
+		/* Skip probe-insert when the module is already builtin or live. */
+		int state = kmod_module_get_initstate(module);
+
+		if (state != KMOD_MODULE_BUILTIN && state != KMOD_MODULE_LIVE) {
+			rc = kmod_module_probe_insert_module(module,
+							     KMOD_PROBE_APPLY_BLACKLIST,
+							     NULL, NULL, NULL, NULL);
+			if (rc < 0) {
+				log_err(ctx, "%s: insert failure: %d\n", __func__, rc);
+				return rc;
+			}
 		}
 	}
 
maintainer
#10 lore raw

Re: [ndctl PATCH 2/2] daxctl, util/sysfs: skip module probe-insert when driver is builtin or live

Quoted 42 lines · Jonathan Cameron
Quoted 20 lines · On Thu, 14 May 2026 15:13:20 -0700 Alison Schofield…
Quoted 19 lines · Jonathan Cameron
Quoted 10 lines · On Thu, 14 May 2026 14:32:34 +0800 Chen Pei <[email protected]> wrote:

kmod_module_probe_insert_module() is supposed to return 0 for builtin modules, but only when libkmod can locate the modules.builtin index. If the index is missing or out of sync, libkmod falls through to the real init_module() syscall and returns an error such as -ENOENT, producing a spurious "insert failure" even though the driver is already part of the running kernel.

Pre-check kmod_module_get_initstate() and short-circuit when the module is KMOD_MODULE_BUILTIN or KMOD_MODULE_LIVE, matching the pattern used by ndctl's own test/core.c.

So I happened to run into exactly this print earlier today and was very happy to see this resolving it! I'm lazy so when developing in a VM tend to do everything I care about built in and not bother with installing the modules.

Quoted 5 lines
Quoted 3 lines

However - despite having CONFIG_DEV_DAX = y in the kernel, I'm getting a state of KMOD_MODULE_COMING which is curious as there is no initstate file to read that from.

I think this patch is worth you trying. In libmkmod code I'm looking at:

It doesn't work - hence the reply!

Quoted 5 lines

https://github.com/lucasdemarchi/kmod/blob/master/libkmod/libkmod-module.c

the "module directory exists but initstate cannot be opened" case returns KMOD_MODULE_BUILTIN, not KMOD_MODULE_COMING.

I'm not following... Also...
https://github.com/kmod-project/kmod/tree/master/libkmod
Is a lot more recent than that tree of Lucas and I'm guessing the current home given it has 2 week old commits from Lucas.

Quoted 7 lines

Can you give me a line number for the path you are talking about because even in that code of Lucas I'm failing to see it. Note the kmod_module_is_buitin() fails for the reason this patch is trying to fix. The file to check that isn't there - hence we hit the path that tries to figure it out from sysfs. I can't see any other path to a KMOD_MODULE_BUILTIN.

Jonathan

Ah, thanks, I see where I went off track. Aside from looking at old tree, I conflated kmod_module_is_builtin() w kmod_module_get_initstate() path that this patch is actually using.

So, I think I get it now and agree w you that once the builtin lookup path fails, we end up in sysfs probe path instead, which returns KMOD_MODULE_COMING when both these are true:

/sys/module/name/ exists
/sys/module/name/initstate does not exist

Seems like patch needs one more case. In addition to LIVE || BUILTIN, like:

	|| (COMING && sysfs-dir-exists-without-initstate)

rather than reling on kmod_module_getinitstate() alone.

-- Alison

Quoted 104 lines
Quoted 103 lines

So if device_dax is builtin and /sys/module/device_dax exists without initstate, this patch should short-circuit before attempting insert. If you still see COMING with this patch applied, then we need to figure out where that state is coming from (before thinking about special casing it in ndctl).

Quoted 94 lines

Looking at the code in libkmod it seems to first check if it can open /sys/modules/device_dax/initstate and if it can't checks if
the directory /sys/modules/device_dax/ exists. If it finds that it returns KMOD_MODULE_COMING which seems odd given in a fully initialized built in driver that particular set of circumstances is normal.

Any ideas?

To me the description above is misleading if we need to have something else for the builtin case to work.

I'm out of time to today but may get time to look at this tomorrow and chase down if there is a way to get it to work.

Jonathan

Quoted 75 lines

For builtin modules the local kmod reference is dropped because builtin drivers cannot be unloaded; for live modules the reference is retained in dev->module, matching the post-probe-success behavior.

---
 daxctl/lib/libdaxctl.c | 18 ++++++++++++++++--
 util/sysfs.c           | 17 +++++++++++------
 2 files changed, 27 insertions(+), 8 deletions(-)
daxctl/lib/libdaxctl.c +16 -2
index ffc81eb..42bfc39 100644
@@ -910,7 +910,7 @@ static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev,
 	const char *devname = daxctl_dev_get_devname(dev);
 	struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev);
 	struct kmod_module *kmod;
-	int rc;
+	int state, rc;
 
 	rc = kmod_module_new_from_name(ctx->kmod_ctx, mod_name, &kmod);
 	if (rc < 0) {
@@ -919,7 +919,21 @@ static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev,
 		return rc;
 	}
 
-	/* if the driver is builtin, this Just Works */
+	/* If the driver is builtin or already live, skip probe-insert. */
+	state = kmod_module_get_initstate(kmod);
+	if (state == KMOD_MODULE_BUILTIN) {
+		dbg(ctx, "%s: module %s is builtin\n", devname,
+			kmod_module_get_name(kmod));
+		kmod_module_unref(kmod);
+		return 0;
+	}
+	if (state == KMOD_MODULE_LIVE) {
+		dbg(ctx, "%s: module %s already loaded\n", devname,
+			kmod_module_get_name(kmod));
+		dev->module = kmod;
+		return 0;
+	}
+
 	dbg(ctx, "%s inserting module: %s\n", devname,
 		kmod_module_get_name(kmod));
 	rc = kmod_module_probe_insert_module(kmod,
util/sysfs.c +11 -6
index e027e38..641b86d 100644
@@ -183,12 +183,17 @@ int __util_bind(const char *devname, struct kmod_module *module,
 	}
 
 	if (module) {
-		rc = kmod_module_probe_insert_module(module,
-						     KMOD_PROBE_APPLY_BLACKLIST,
-						     NULL, NULL, NULL, NULL);
-		if (rc < 0) {
-			log_err(ctx, "%s: insert failure: %d\n", __func__, rc);
-			return rc;
+		/* Skip probe-insert when the module is already builtin or live. */
+		int state = kmod_module_get_initstate(module);
+
+		if (state != KMOD_MODULE_BUILTIN && state != KMOD_MODULE_LIVE) {
+			rc = kmod_module_probe_insert_module(module,
+							     KMOD_PROBE_APPLY_BLACKLIST,
+							     NULL, NULL, NULL, NULL);
+			if (rc < 0) {
+				log_err(ctx, "%s: insert failure: %d\n", __func__, rc);
+				return rc;
+			}
 		}
 	}
 
author
#11 lore raw

Re: [ndctl PATCH 2/2] daxctl, util/sysfs: skip module probe-insert when driver is builtin or live

In reply to Jonathan Cameron
Quoted 9 lines · Jonathan Cameron
Quoted 1 line

I think this patch is worth you trying. In libmkmod code I'm looking at:

It doesn't work - hence the reply!

Quoted 5 lines

https://github.com/lucasdemarchi/kmod/blob/master/libkmod/libkmod-module.c

the "module directory exists but initstate cannot be opened" case returns KMOD_MODULE_BUILTIN, not KMOD_MODULE_COMING.

Hi Jonathan,

Thanks for testing and the detailed analysis.

I wasn't able to reproduce this on my setup because the modules.builtin index is correctly installed, so kmod_module_get_initstate() returns KMOD_MODULE_BUILTIN at the first step (via kmod_module_is_builtin()) without ever falling through to the sysfs path.

That said, I understand the scenario — developing in a VM with CONFIG_DEV_DAX=y but without running modules_install is a perfectly reasonable workflow, and I agree it's worth addressing.

However, I feel that the suggested workaround (adding a COMING + sysfs-dir-exists-without-initstate check in ndctl) is papering over what is fundamentally a libkmod issue. When a module is builtin and /sys/module/<name>/ exists without an initstate file, libkmod should not return KMOD_MODULE_COMING — that state implies a module is actively being loaded, which is not the case for a fully initialized builtin driver. The root cause is that kmod_module_is_builtin() fails when modules.builtin is missing, and the sysfs fallback doesn't distinguish "builtin without initstate" from "module in transition".

I think the proper fix belongs in libkmod: the sysfs fallback in kmod_module_get_initstate() should return KMOD_MODULE_BUILTIN (or at least not COMING) when the directory exists but initstate does not.

For now, would it be acceptable to keep this patch as-is (covering the BUILTIN and LIVE cases) and note in the commit message that the modules.builtin index must be properly installed for the builtin detection to work? If there is consensus that ndctl should carry the workaround regardless, I'm happy to send a v2 with the additional COMING handling. CC Alison.

Thanks,
Pei

#12 lore raw

Re: [ndctl PATCH 2/2] daxctl, util/sysfs: skip module probe-insert when driver is builtin or live

Quoted 25 lines · On Tue, 19 May 2026 11:48:47 +0800 Chen Pei <[email protected]> wrote:
Quoted 9 lines · Jonathan Cameron
Quoted 1 line

I think this patch is worth you trying. In libmkmod code I'm looking at:

It doesn't work - hence the reply!

Quoted 5 lines

https://github.com/lucasdemarchi/kmod/blob/master/libkmod/libkmod-module.c

the "module directory exists but initstate cannot be opened" case returns KMOD_MODULE_BUILTIN, not KMOD_MODULE_COMING.

Hi Jonathan,

Thanks for testing and the detailed analysis.

I wasn't able to reproduce this on my setup because the modules.builtin index is correctly installed, so kmod_module_get_initstate() returns KMOD_MODULE_BUILTIN at the first step (via kmod_module_is_builtin()) without ever falling through to the sysfs path.

That said, I understand the scenario — developing in a VM with CONFIG_DEV_DAX=y but without running modules_install is a perfectly reasonable workflow, and I agree it's worth addressing.

Quoted 9 lines

However, I feel that the suggested workaround (adding a COMING + sysfs-dir-exists-without-initstate check in ndctl) is papering over what is fundamentally a libkmod issue. When a module is builtin and /sys/module/<name>/ exists without an initstate file, libkmod should not return KMOD_MODULE_COMING — that state implies a module is actively being loaded, which is not the case for a fully initialized builtin driver. The root cause is that kmod_module_is_builtin() fails when modules.builtin is missing, and the sysfs fallback doesn't distinguish "builtin without initstate" from "module in transition".

Agreed. I'm curious why it was built that way. I scraped an email address that is hopefully the right Lucas from another thread. +CC

Brief summary is that this series was trying to avoid a message when module load fails due to it already being loaded, or built in but some
of the files are missing (as say no modules installed). If modules.builtin
isn't there we get return value of KMOD_MODULE_COMING as the extra is it loaded heuristic doesn't work because the initstate file isn't there at all.

My suspicion is this is papering over a race condition that occurs in a normal module load - it may not be possible to tell the difference between COMMING and BUILTIN reliably.

Quoted 11 lines

I think the proper fix belongs in libkmod: the sysfs fallback in kmod_module_get_initstate() should return KMOD_MODULE_BUILTIN (or at least not COMING) when the directory exists but initstate does not.

For now, would it be acceptable to keep this patch as-is (covering the BUILTIN and LIVE cases) and note in the commit message that the modules.builtin index must be properly installed for the builtin detection to work? If there is consensus that ndctl should carry the workaround regardless, I'm happy to send a v2 with the additional COMING handling. CC Alison.

This is indeed better than current situation and given it's ultimately a case of message suppression that seems harmless. So I don't mind this if handing the corner case I hit is too complex.

Jonathan

Quoted 3 lines

Thanks,
Pei

author
#13 lore raw

Re: [ndctl PATCH 2/2] daxctl, util/sysfs: skip module probe-insert when driver is builtin or live

Quoted 8 lines · Jonathan Cameron

Agreed. I'm curious why it was built that way. I scraped an email address that is hopefully the right Lucas from another thread. +CC

[...]

My suspicion is this is papering over a race condition that occurs in a normal module load - it may not be possible to tell the difference between COMMING and BUILTIN reliably.

I went and read through libkmod's code and history carefully on this, and your suspicion is correct -- the COMING return is deliberate, not incidental.

The behavior was introduced in libkmod commit fd44a98 ("Fix race while loading modules", 2015), and Lucas spelled out the rationale in the commit message:

The problem is that the creation of /sys/module/<name> and /sys/module/<name>/initstate are not atomic. There's a small window in which the directory exists but the initstate file was still not created.
[...]
We enforce mod->builtin to always be up-to-date when kmod_module_get_initstate() is called. This way if the directory exists but the initstate doesn't, we can be sure this is because the module is in the "coming" state, i.e. kernel didn't create the file yet, but since builtin modules were already handled by checking our index the only reason for that to happen is that we hit the race condition.

So COMING on the sysfs fallback path is intentional: libkmod treats kmod_module_is_builtin() (i.e. the modules.builtin lookup) as the authoritative answer to "is this builtin", and assumes anything else hitting the dir-without-initstate window has to be a concurrent load race. When modules.builtin is missing, that assumption silently flips and a builtin driver looks exactly like a load-in-progress -- which is the COMING you saw.

Given that, I don't think the right fix is in libkmod. Returning BUILTIN unconditionally on the dir-without-initstate fallback would re-open the very race fd44a98 set out to close: a parallel modprobe of a real loadable module that wins the directory creation but not yet the initstate write would be misreported as builtin, and callers waiting on it (the nls_cp437 / sd-card scenario in fd44a98) would skip the wait. That's a behavioral regression on a configuration libkmod has supported for ten years, in exchange for tidying up a no-modules_install setup that libkmod arguably never promised to handle. Lucas (CC'd) -- please correct me if I'm misreading the original intent.

So I'll take Alison's suggestion and handle it on the ndctl side. v2 will keep the BUILTIN/LIVE short-circuit and additionally treat KMOD_MODULE_COMING as builtin when /sys/module/<name>/ exists but the initstate file does not -- exactly the fingerprint your VM hits.

I'll post v2 shortly.

Thanks,
Pei