fs: support $ORIGIN in ELF interpreter paths
Subsystems (8)
cover fs: support $ORIGIN in ELF interpreter paths
Currently, standard ELF and ELF FDPIC loaders require a fixed, absolute path to the dynamic linker/interpreter (specified via PT_INTERP). This creates significant inflexibility for relocatable dynamic interpreters, where binaries are packaged independent of global system paths.
The primary goal of this patch series is to support relocatable binaries in Nix, where packages are stored in a read-only store (typically /nix/store). Allowing the ELF interpreter path to be resolved dynamically relative to the binary's location via $ORIGIN enables Nix packages to be relocated without needing patchelf or wrapper scripts.
For details on the design and motivation for this in Nix, see:
https://fzakaria.com/2026/06/21/nix-needs-relocatable-binaries
This series introduces support for resolving the $ORIGIN placeholder in the ELF interpreter path, bringing the kernel's binary loading behavior in line with user-space dynamic linker origin resolution.
To achieve this cleanly:
- We introduce a shared 'resolve_elf_interpreter()' helper in the VFS exec subsystem to avoid code duplication across loader implementations.
- For security, we restrict detection strictly to the prefix string "$ORIGIN" to prevent complex parsing exploits in kernel space.
Testing & Verification:
- Added a KUnit test case verifying path resolution logic.
- Added a kselftests integration test checking that a dynamically linked binary with its interpreter set to '$ORIGIN/mock_interp' successfully loads the mock interpreter (built statically using nolibc to avoid glibc TLS setup constraints during interpreter load-time).
- Verified end-to-end correct execution (PASS) using a minimal initramfs under QEMU.
Farid Zakaria (2):
fs: support $ORIGIN in ELF interpreter paths selftests/exec: add test suites for $ORIGIN interpreter resolution
fs/binfmt_elf.c | 11 ++++- fs/binfmt_elf_fdpic.c | 15 ++++++- fs/exec.c | 42 +++++++++++++++++++ fs/tests/exec_kunit.c | 26 ++++++++++++ include/linux/binfmts.h | 2 + tools/testing/selftests/exec/Makefile | 12 ++++-- tools/testing/selftests/exec/mock_interp.c | 6 +++ tools/testing/selftests/exec/origin_interp.sh | 16 +++++++ tools/testing/selftests/exec/test_prog.c | 5 +++ 9 files changed, 128 insertions(+), 7 deletions(-) create mode 100644 tools/testing/selftests/exec/mock_interp.c create mode 100755 tools/testing/selftests/exec/origin_interp.sh create mode 100644 tools/testing/selftests/exec/test_prog.c
Signature · 1 line
2.51.2
PATCH 1/2 fs: support $ORIGIN in ELF interpreter paths
Currently, standard ELF and ELF FDPIC loaders expect a fixed path to the dynamic linker/interpreter (PT_INTERP). However, for systems utilizing relocatable dynamic interpreters (such as Nix/store-based environments), hardcoding this path is inflexible and breaks binary portability.
Introduce support for resolving the $ORIGIN placeholder in the ELF interpreter path. This maps the dynamic linker relative to the path of the binary being executed, matching user-space origin resolution.
To avoid code duplication, implement a shared 'resolve_elf_interpreter()' helper in the VFS exec layer. For safety, limit detection strictly to the prefix string "$ORIGIN" to prevent complex parsing exploits.
--- fs/binfmt_elf.c | 11 +++++++++-- fs/binfmt_elf_fdpic.c | 15 +++++++++++++-- fs/exec.c | 42 +++++++++++++++++++++++++++++++++++++++++ include/linux/binfmts.h | 2 ++ 4 files changed, 66 insertions(+), 4 deletions(-)
fs/binfmt_elf.c +9 -2
@@ -872,7 +872,7 @@ static int load_elf_binary(struct linux_binprm *bprm) elf_ppnt = elf_phdata;for (i =0 ; i < elf_ex->e_phnum; i++, elf_ppnt++) { -char *elf_interpreter; +char *elf_interpreter, *resolved_interp;if (elf_ppnt->p_type ==PT_GNU_PROPERTY ) { elf_property_phdata = elf_ppnt;
@@ -904,8 +904,15 @@ static int load_elf_binary(struct linux_binprm *bprm)if (elf_interpreter[elf_ppnt->p_filesz -1 ] !='\0' ) goto out_free_interp; - interpreter =open_exec (elf_interpreter); + resolved_interp =resolve_elf_interpreter (bprm, elf_interpreter);kfree (elf_interpreter); +if (IS_ERR (resolved_interp)) { + retval =PTR_ERR (resolved_interp); + goto out_free_ph; + } + + interpreter =open_exec (resolved_interp); +kfree (resolved_interp); retval =PTR_ERR (interpreter);if (IS_ERR (interpreter)) goto out_free_ph;
fs/binfmt_elf_fdpic.c +13 -2
@@ -230,7 +230,9 @@ static int load_elf_fdpic_binary(struct linux_binprm *bprm)for (i =0 ; i < exec_params.hdr.e_phnum; i++, phdr++) {switch (phdr->p_type) { -case PT_INTERP : +case PT_INTERP : { +char *resolved_interp; + retval = -ENOMEM ;if (phdr->p_filesz >PATH_MAX ) goto error;
@@ -259,7 +261,15 @@ static int load_elf_fdpic_binary(struct linux_binprm *bprm)kdebug ("Using ELF interpreter %s" , interpreter_name);/* replace the program with the interpreter */ - interpreter =open_exec (interpreter_name); + resolved_interp =resolve_elf_interpreter (bprm, interpreter_name); +kfree (interpreter_name); +if (IS_ERR (resolved_interp)) { + retval =PTR_ERR (resolved_interp); + goto error; + } + + interpreter =open_exec (resolved_interp); +kfree (resolved_interp); retval =PTR_ERR (interpreter);if (IS_ERR (interpreter)) { interpreter =NULL ;
@@ -284,6 +294,7 @@ static int load_elf_fdpic_binary(struct linux_binprm *bprm) interp_params.hdr = *((struct elfhdr *) bprm->buf);break ; + }case PT_LOAD :#ifdef CONFIG_MMU
fs/exec.c +42 -0
48 lines not on this page · raw message
include/linux/binfmts.h +2 -0
@@ -150,4 +150,6 @@ extern ssize_t read_code(struct file *, unsigned long, loff_t, size_t);int kernel_execve (const char *filename,const char *const *argv,const char *const *envp); +char *resolve_elf_interpreter (struct linux_binprm *bprm,const char *elf_interpreter); +#endif /* _LINUX_BINFMTS_H */
Signature · 1 line
2.51.2
Hi Farid,
Quoted 9 lines · Farid Zakaria
Currently, standard ELF and ELF FDPIC loaders expect a fixed path to the dynamic linker/interpreter (
PT_INTERP). However, for systems utilizing relocatable dynamic interpreters (such as Nix/store-based environments), hardcoding this path is inflexible and breaks binary portability.Introduce support for resolving the $ORIGIN placeholder in the ELF interpreter path. This maps the dynamic linker relative to the path of the binary being executed, matching user-space origin resolution.
Quoted 5 lines
To avoid code duplication, implement a shared '
resolve_elf_interpreter()' helper in the VFS exec layer. For safety, limit detection strictly to the prefix string "$ORIGIN" to prevent complex parsing exploits.Assisted-by Antigravity:Gemini-Pro
This isn't a requirement from the community or anything, but I always find it useful if I see an Assisted-by tag to know what assistence was actually delivered by an LLM. Otherwise we might as well add assisted-by tags for any editor.
Talking about LLMs, your patch has some issues flagged by Sashiko[1]. Please take a look.
Quoted 145 lines
Signed-off-by Farid Zakaria <[email protected]>--- fs/binfmt_elf.c | 11 +++++++++-- fs/binfmt_elf_fdpic.c | 15 +++++++++++++-- fs/exec.c | 42 +++++++++++++++++++++++++++++++++++++++++ include/linux/binfmts.h | 2 ++ 4 files changed, 66 insertions(+), 4 deletions(-)fs/binfmt_elf.c +9
-2@@ -872,7 +872,7 @@ static int load_elf_binary(struct linux_binprm *bprm) elf_ppnt = elf_phdata;for (i =0 ; i < elf_ex->e_phnum; i++, elf_ppnt++) { -char *elf_interpreter; +char *elf_interpreter, *resolved_interp;if (elf_ppnt->p_type ==PT_GNU_PROPERTY ) { elf_property_phdata = elf_ppnt;@@ -904,8 +904,15 @@ static int load_elf_binary(struct linux_binprm *bprm)if (elf_interpreter[elf_ppnt->p_filesz -1 ] !='\0' ) goto out_free_interp; - interpreter =open_exec (elf_interpreter); + resolved_interp =resolve_elf_interpreter (bprm, elf_interpreter);kfree (elf_interpreter); +if (IS_ERR (resolved_interp)) { + retval =PTR_ERR (resolved_interp); + goto out_free_ph; + } + + interpreter =open_exec (resolved_interp); +kfree (resolved_interp); retval =PTR_ERR (interpreter);if (IS_ERR (interpreter)) goto out_free_ph;fs/binfmt_elf_fdpic.c +13
-2@@ -230,7 +230,9 @@ static int load_elf_fdpic_binary(struct linux_binprm *bprm)for (i =0 ; i < exec_params.hdr.e_phnum; i++, phdr++) {switch (phdr->p_type) { -case PT_INTERP : +case PT_INTERP : { +char *resolved_interp; + retval = -ENOMEM ;if (phdr->p_filesz >PATH_MAX ) goto error;@@ -259,7 +261,15 @@ static int load_elf_fdpic_binary(struct linux_binprm *bprm)kdebug ("Using ELF interpreter %s" , interpreter_name);/* replace the program with the interpreter */ - interpreter =open_exec (interpreter_name); + resolved_interp =resolve_elf_interpreter (bprm, interpreter_name); +kfree (interpreter_name); +if (IS_ERR (resolved_interp)) { + retval =PTR_ERR (resolved_interp); + goto error; + } + + interpreter =open_exec (resolved_interp); +kfree (resolved_interp); retval =PTR_ERR (interpreter);if (IS_ERR (interpreter)) { interpreter =NULL ;@@ -284,6 +294,7 @@ static int load_elf_fdpic_binary(struct linux_binprm *bprm) interp_params.hdr = *((struct elfhdr *) bprm->buf);break ; + }case PT_LOAD :#ifdef CONFIG_MMU fs/exec.c +42
-0@@ -2024,6 +2024,48 @@ static int __init init_fs_exec_sysctls(void)fs_initcall (init_fs_exec_sysctls);#endif /* CONFIG_SYSCTL */ +char *resolve_elf_interpreter (struct linux_binprm *bprm,const char *elf_interpreter) +{ +char *pathbuf, *path, *slash, *resolved; + +if (strncmp (elf_interpreter,"$ORIGIN" ,7 ) !=0 ) { +char *ret =kstrdup (elf_interpreter,GFP_KERNEL ); + +return ret ? ret :ERR_PTR (-ENOMEM ); + } + + pathbuf =kmalloc (PATH_MAX ,GFP_KERNEL ); +if (!pathbuf) +return ERR_PTR (-ENOMEM ); + + path =file_path (bprm->file, pathbuf,PATH_MAX ); +if (IS_ERR (path)) { +kfree (pathbuf); +return (char *)path; + } + + slash =strrchr (path,'/' ); +if (slash) { +if (slash == path) + *(slash +1 ) ='\0' ; +else + *slash ='\0' ; + }else { +kfree (pathbuf); +char *ret =kstrdup (elf_interpreter,GFP_KERNEL ); + +return ret ? ret :ERR_PTR (-ENOMEM ); + } + + resolved =kasprintf (GFP_KERNEL ,"%s%s" , path, elf_interpreter +7 ); +kfree (pathbuf); +if (!resolved) +return ERR_PTR (-ENOMEM ); + +return resolved; +} +EXPORT_SYMBOL (resolve_elf_interpreter); +#ifdef CONFIG_EXEC_KUNIT_TEST #include "tests/exec_kunit.c" #endif include/linux/binfmts.h +2
-0@@ -150,4 +150,6 @@ extern ssize_t read_code(struct file *, unsigned long, loff_t, size_t);int kernel_execve (const char *filename,const char *const *argv,const char *const *envp); +char *resolve_elf_interpreter (struct linux_binprm *bprm,const char *elf_interpreter); +#endif /* _LINUX_BINFMTS_H */ Signature · 1 line
2.51.2
Thanks,
Jori.
[1]: https://sashiko.dev/#/patchset/20260622043934.179879-1-farid.m.zakaria%40gmail.com
Kees Cook
maintainerQuoted 5 lines · Farid Zakaria
Currently, standard ELF and ELF FDPIC loaders expect a fixed path to the dynamic linker/interpreter (
PT_INTERP). However, for systems utilizing relocatable dynamic interpreters (such as Nix/store-based environments), hardcoding this path is inflexible and breaks binary portability.
Quoted 7 lines
Introduce support for resolving the $ORIGIN placeholder in the ELF interpreter path. This maps the dynamic linker relative to the path of the binary being executed, matching user-space origin resolution.
To avoid code duplication, implement a shared '
resolve_elf_interpreter()' helper in the VFS exec layer. For safety, limit detection strictly to the prefix string "$ORIGIN" to prevent complex parsing exploits.
Does any other OS that implements ELF support also support $ORIGIN in the loader? $ORIGIN isn't even part of the ELF spec at all and is a glibc extension, IIUC.
I'm not excited about path-based string manipulations as they end up being racy, and mucking with loader path seems like we're inviting trouble (since the _binary_ specifies setuid-ness), and we've seen issues with $ORIGIN before, even strictly outside of the kernel:
https://seclists.org/fulldisclosure/2010/Oct/257
Quoted 8 lines
[...]
fs/exec.c +0
-0@@ -2024,6 +2024,48 @@ static int __init init_fs_exec_sysctls(void)fs_initcall (init_fs_exec_sysctls);#endif /* CONFIG_SYSCTL */
fs/exec.c static int __init init_fs_exec_sysctls(void) · line 2035
Quoted 10 lines
fs/exec.c +9
-0+char *resolve_elf_interpreter (struct linux_binprm *bprm,const char *elf_interpreter) +{ +char *pathbuf, *path, *slash, *resolved; + +if (strncmp (elf_interpreter,"$ORIGIN" ,7 ) !=0 ) { +char *ret =kstrdup (elf_interpreter,GFP_KERNEL ); + +return ret ? ret :ERR_PTR (-ENOMEM ); + }
But even if we did take this, I really don't want to incur a universal penalty on exec for it. This is doing a kmalloc+dup (and later kfree) for all non-$ORIGIN execs. So even if this gets added, it needs to be handled differently.
I would probably say this helper should return a struct file * instead and have a fast-path for the common case. I think a check for leading '$' (if strncmp doesn't get inlined) would be okay here as far as "incurring common performance cost"; that string is almost certainly cache-hot.
fs/exec.c static int __init init_fs_exec_sysctls(void) · line 2045
Quoted 9 lines
fs/exec.c +9
-0+ pathbuf =kmalloc (PATH_MAX ,GFP_KERNEL ); +if (!pathbuf) +return ERR_PTR (-ENOMEM ); + + path =file_path (bprm->file, pathbuf,PATH_MAX ); +if (IS_ERR (path)) { +kfree (pathbuf); +return (char *)path; + }
This still just _feels_ like an info leak or a race condition to me. I can't give a credible example, though. But it creeps me out. :) (I note the blog post also says "and the shabang" and I get even more creeped out about seeing that patch.)
fs/exec.c static int __init init_fs_exec_sysctls(void) · line 2050
Quoted 5 lines
fs/exec.c +5
-0+ + slash =strrchr (path,'/' ); +if (slash) { +if (slash == path) + *(slash +1 ) ='\0' ;
This is not idiomatic string manipulation.
fs/exec.c static int __init init_fs_exec_sysctls(void) · line 2052
Quoted 2 lines
fs/exec.c +2
-0+else + *slash ='\0' ;
More readable, IMO, as:
if (slash) slash[1] = '\0'; else path = "";
But does this match the glibc resolution logic? i.e. should it be:
if (strncmp(elf_interpreter, "$ORIGIN/", 8) != 0) ... if (!slash) slash = path; *slash = '\0'; ... resolved = kasprintf(GFP_KERNEL, "%s/%s", path, elf_interpreter + 8);
(requires the trailing /)
fs/exec.c static int __init init_fs_exec_sysctls(void) · line 2057
Quoted 5 lines
fs/exec.c +5
-0+ }else { +kfree (pathbuf); +char *ret =kstrdup (elf_interpreter,GFP_KERNEL ); + +return ret ? ret :ERR_PTR (-ENOMEM );
This is the same as the logic top of the function. This repetition smells of the LLM. :)
Quoted 28 lines
fs/exec.c +11
-0+ } + + resolved =kasprintf (GFP_KERNEL ,"%s%s" , path, elf_interpreter +7 ); +kfree (pathbuf); +if (!resolved) +return ERR_PTR (-ENOMEM ); + +return resolved; +} +EXPORT_SYMBOL (resolve_elf_interpreter); +#ifdef CONFIG_EXEC_KUNIT_TEST #include "tests/exec_kunit.c" #endif include/linux/binfmts.h +2
-0@@ -150,4 +150,6 @@ extern ssize_t read_code(struct file *, unsigned long, loff_t, size_t);int kernel_execve (const char *filename,const char *const *argv,const char *const *envp); +char *resolve_elf_interpreter (struct linux_binprm *bprm,const char *elf_interpreter); +#endif /* _LINUX_BINFMTS_H */ Signature · 1 line
2.51.2
So, I guess, I'd like more convincing, but I'm very happy to see a KUnit test included!
-Kees
Signature · 1 line
Kees Cook
Thanks for all the improvement suggestions.
Yes, I leveraged an LLM to generate the initial code (open & honest) but I'm willing to keep refining it to whatever state upstream requires.
Thank you for not dismissing it outright right away. (FWIW, the initial code was even worse so this is the product of me intervening and editing it, despite my lack of C expertise).
A few more attempts at convincing :)
* musl also supports $ORIGIN
[https://elixir.bootlin.com/musl/v1.2.5/source/ldso/dynlink.c#L911] so there seems to be strong convergence on the concept beyond glibc
* ideologically, everything about a program should be portable easily from one environment to another (the goal of systems such as Nix). Userland has allowed this support in nearly ever-space and the DT_INTERP / shebang path from the kernel seems to be a missing gap.
We also have the shebang patch ready (CC @[email protected] ) but we wanted to see the reception to this first.
I will wait to update the patch based on your feedback and Sashiko's, if you are convinced just to avoid spamming us with more patch files :)
Farid Zakaria
Quoted 145 lines · Kees Cook
Quoted 5 lines · Farid Zakaria
Currently, standard ELF and ELF FDPIC loaders expect a fixed path to the dynamic linker/interpreter (
PT_INTERP). However, for systems utilizing relocatable dynamic interpreters (such as Nix/store-based environments), hardcoding this path is inflexible and breaks binary portability.Quoted 7 lines
Introduce support for resolving the $ORIGIN placeholder in the ELF interpreter path. This maps the dynamic linker relative to the path of the binary being executed, matching user-space origin resolution.
To avoid code duplication, implement a shared '
resolve_elf_interpreter()' helper in the VFS exec layer. For safety, limit detection strictly to the prefix string "$ORIGIN" to prevent complex parsing exploits.Does any other OS that implements ELF support also support $ORIGIN in the loader? $ORIGIN isn't even part of the ELF spec at all and is a glibc extension, IIUC.
I'm not excited about path-based string manipulations as they end up being racy, and mucking with loader path seems like we're inviting trouble (since the _binary_ specifies setuid-ness), and we've seen issues with $ORIGIN before, even strictly outside of the kernel:
https://seclists.org/fulldisclosure/2010/Oct/257Quoted 8 lines
[...]
fs/exec.c +0
-0@@ -2024,6 +2024,48 @@ static int __init init_fs_exec_sysctls(void)fs_initcall (init_fs_exec_sysctls);#endif /* CONFIG_SYSCTL */ fs/exec.c static int __init init_fs_exec_sysctls(void) · line 2035
Quoted 10 lines
fs/exec.c +9
-0+char *resolve_elf_interpreter (struct linux_binprm *bprm,const char *elf_interpreter) +{ +char *pathbuf, *path, *slash, *resolved; + +if (strncmp (elf_interpreter,"$ORIGIN" ,7 ) !=0 ) { +char *ret =kstrdup (elf_interpreter,GFP_KERNEL ); + +return ret ? ret :ERR_PTR (-ENOMEM ); + }But even if we did take this, I really don't want to incur a universal penalty on exec for it. This is doing a kmalloc+dup (and later kfree) for all non-$ORIGIN execs. So even if this gets added, it needs to be handled differently.
I would probably say this helper should return a struct file * instead and have a fast-path for the common case. I think a check for leading '$' (if strncmp doesn't get inlined) would be okay here as far as "incurring common performance cost"; that string is almost certainly cache-hot.
fs/exec.c static int __init init_fs_exec_sysctls(void) · line 2045
Quoted 9 lines
fs/exec.c +9
-0+ pathbuf =kmalloc (PATH_MAX ,GFP_KERNEL ); +if (!pathbuf) +return ERR_PTR (-ENOMEM ); + + path =file_path (bprm->file, pathbuf,PATH_MAX ); +if (IS_ERR (path)) { +kfree (pathbuf); +return (char *)path; + }This still just _feels_ like an info leak or a race condition to me. I can't give a credible example, though. But it creeps me out. :) (I note the blog post also says "and the shabang" and I get even more creeped out about seeing that patch.)
fs/exec.c static int __init init_fs_exec_sysctls(void) · line 2050
Quoted 5 lines
fs/exec.c +5
-0+ + slash =strrchr (path,'/' ); +if (slash) { +if (slash == path) + *(slash +1 ) ='\0' ;This is not idiomatic string manipulation.
fs/exec.c static int __init init_fs_exec_sysctls(void) · line 2052
Quoted 2 lines
fs/exec.c +2
-0+else + *slash ='\0' ;More readable, IMO, as:
if (slash) slash[1] = '\0'; else path = "";But does this match the glibc resolution logic? i.e. should it be:
if (strncmp(elf_interpreter, "$ORIGIN/", 8) != 0) ... if (!slash) slash = path; *slash = '\0'; ... resolved = kasprintf(GFP_KERNEL, "%s/%s", path, elf_interpreter + 8);(requires the trailing /)
fs/exec.c static int __init init_fs_exec_sysctls(void) · line 2057
Quoted 5 lines
fs/exec.c +5
-0+ }else { +kfree (pathbuf); +char *ret =kstrdup (elf_interpreter,GFP_KERNEL ); + +return ret ? ret :ERR_PTR (-ENOMEM );This is the same as the logic top of the function. This repetition smells of the LLM. :)
Quoted 26 lines
fs/exec.c +11
-0+ } + + resolved =kasprintf (GFP_KERNEL ,"%s%s" , path, elf_interpreter +7 ); +kfree (pathbuf); +if (!resolved) +return ERR_PTR (-ENOMEM ); + +return resolved; +} +EXPORT_SYMBOL (resolve_elf_interpreter); +#ifdef CONFIG_EXEC_KUNIT_TEST #include "tests/exec_kunit.c" #endif include/linux/binfmts.h +2
-1@@ -150,4 +150,6 @@ extern ssize_t read_code(struct file *, unsigned long, loff_t, size_t);int kernel_execve (const char *filename,const char *const *argv,const char *const *envp); +char *resolve_elf_interpreter (struct linux_binprm *bprm,const char *elf_interpreter); +#endif /* _LINUX_BINFMTS_H */ --Quoted 2 lines
2.51.2
So, I guess, I'd like more convincing, but I'm very happy to see a KUnit test included!
-Kees
--
Kees Cook
PATCH 2/2 selftests/exec: add test suites for $ORIGIN interpreter resolution
In reply to Farid ZakariaAdd verification suites to test the kernel VFS and ELF loader $ORIGIN interpreter resolution.
1. Add a KUnit unit test 'exec_test_resolve_elf_interpreter()' verifying path resolution format logic.
2. Add a kselftests integration test containing:
- A nolibc-based statically linked mock interpreter that prints a success message and returns 42. nolibc is used to bypass glibc's static startup code which segfaults when loaded as an interpreter due to AT_PHDR mismatches.
- A dynamic test program configured to look for the interpreter at $ORIGIN/mock_interp.
- A shell script harness checking for a PASS result.
--- fs/tests/exec_kunit.c | 26 +++++++++++++++++++ tools/testing/selftests/exec/Makefile | 12 ++++++--- tools/testing/selftests/exec/mock_interp.c | 6 +++++ tools/testing/selftests/exec/origin_interp.sh | 16 ++++++++++++ tools/testing/selftests/exec/test_prog.c | 5 ++++ 5 files changed, 62 insertions(+), 3 deletions(-) create mode 100644 tools/testing/selftests/exec/mock_interp.c create mode 100755 tools/testing/selftests/exec/origin_interp.sh create mode 100644 tools/testing/selftests/exec/test_prog.c
fs/tests/exec_kunit.c +26 -0
@@ -119,8 +119,34 @@ static void exec_test_bprm_stack_limits(struct kunit *test) } } +static void exec_test_resolve_elf_interpreter (struct kunit *test) +{ +struct linux_binprm bprm = { .file =NULL }; +struct file *f; +char *resolved; + +// Test 1: Non-$ORIGIN interpreter path should just be duplicated + resolved =resolve_elf_interpreter (&bprm,"/lib64/ld-linux-x86-64.so.2" ); +KUNIT_ASSERT_NOT_ERR_OR_NULL (test, resolved); +KUNIT_EXPECT_STREQ (test, resolved,"/lib64/ld-linux-x86-64.so.2" ); +kfree (resolved); + +// Test 2: $ORIGIN interpreter path + f =filp_open ("/" ,O_RDONLY ,0 ); +KUNIT_ASSERT_NOT_ERR_OR_NULL (test, f); + bprm.file = f; + + resolved =resolve_elf_interpreter (&bprm,"$ORIGIN/../lib/ld.so" ); +KUNIT_ASSERT_NOT_ERR_OR_NULL (test, resolved); +KUNIT_EXPECT_STREQ (test, resolved,"//../lib/ld.so" ); +kfree (resolved); + +filp_close (f,NULL ); +} +static struct kunit_case exec_test_cases[] = {KUNIT_CASE (exec_test_bprm_stack_limits), +KUNIT_CASE (exec_test_resolve_elf_interpreter), {}, };
tools/testing/selftests/exec/Makefile +9 -3
@@ -10,9 +10,9 @@ ALIGN_PIES := $(patsubst %,load_address.%,$(ALIGNS)) ALIGN_STATIC_PIES := $(patsubst %,load_address.static.%,$(ALIGNS)) ALIGNMENT_TESTS := $(ALIGN_PIES) $(ALIGN_STATIC_PIES) -TEST_PROGS := binfmt_script.py check-exec-tests.sh -TEST_GEN_PROGS := execveat non-regular $(ALIGNMENT_TESTS) -TEST_GEN_PROGS_EXTENDED := false inc set-exec script-exec.inc script-noexec.inc +TEST_PROGS := binfmt_script.py check-exec-tests.sh origin_interp.sh +TEST_GEN_PROGS := execveat non-regular $(ALIGNMENT_TESTS) test_prog +TEST_GEN_PROGS_EXTENDED := false inc set-exec script-exec.inc script-noexec.inc mock_interp TEST_GEN_FILES := execveat.symlink execveat.denatured script subdir # Makefile is a run-time dependency, since it's accessed by the execveat test TEST_FILES := Makefile
@@ -55,3 +55,9 @@ $(OUTPUT)/script-exec.inc: $(CHECK_EXEC_SAMPLES)/script-exec.inc cp $< $@ $(OUTPUT)/script-noexec.inc: $(CHECK_EXEC_SAMPLES)/script-noexec.inc cp $< $@ + +$(OUTPUT)/mock_interp: mock_interp.c + $(CC) $(CFLAGS) $(LDFLAGS) -static -nostdlib -include ../../../include/nolibc/nolibc.h $< -o $@ + +$(OUTPUT)/test_prog: test_prog.c $(OUTPUT)/mock_interp + $(CC) $(CFLAGS) $(LDFLAGS) -Wl,-dynamic-linker,'$$ORIGIN/mock_interp' $< -o $@
tools/testing/selftests/exec/mock_interp.c new file +6 -0
@@ -0,0 +1,6 @@ +// SPDX-License-Identifier: GPL-2.0 +int main (void ) +{ +write (1 ,"Hello from mock interpreter!\n" ,29 ); +return 42 ; +}
tools/testing/selftests/exec/origin_interp.sh new file +16 -0
@@ -0,0 +1,16 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 + +# Execute the test program which has its interpreter set to $ORIGIN/mock_interp +# Note that mock_interp must be in the same directory. +dir=$(dirname "$0") +out=$("$dir"/test_prog 2>&1) +exit_code=$? + +if [ $exit_code -eq 42 ] && [ "$out" = "Hello from mock interpreter!" ]; then + echo "origin_interp: PASS" + exit 0 +else + echo "origin_interp: FAIL (exit_code=$exit_code, output='$out')" + exit 1 +fi
tools/testing/selftests/exec/test_prog.c new file +5 -0
@@ -0,0 +1,5 @@ +// SPDX-License-Identifier: GPL-2.0 +int main (void ) +{ +return 0 ;/* Should never be reached if interpreter is loaded instead */ +}
Signature · 1 line
2.51.2
Jan Kara
reviewerQuoted 11 lines · Farid Zakaria
Currently, standard ELF and ELF FDPIC loaders require a fixed, absolute path to the dynamic linker/interpreter (specified via
PT_INTERP). This creates significant inflexibility for relocatable dynamic interpreters, where binaries are packaged independent of global system paths.The primary goal of this patch series is to support relocatable binaries in Nix, where packages are stored in a read-only store (typically /nix/store). Allowing the ELF interpreter path to be resolved dynamically relative to the binary's location via $ORIGIN enables Nix packages to be relocated without needing patchelf or wrapper scripts.
Quoted 6 lines
For details on the design and motivation for this in Nix, see:
https://fzakaria.com/2026/06/21/nix-needs-relocatable-binariesThis series introduces support for resolving the $ORIGIN placeholder in the ELF interpreter path, bringing the kernel's binary loading behavior in line with user-space dynamic linker origin resolution.
Thanks for the patches! Before dwelving into implementation details we need to discuss whether something like this even belongs to the kernel. Frankly to me this looks rather arbitrary and tied to the particular way you've decided to setup your package management system. In particular the usual answer for situations like these is to use namespaces which you discount in your blog with: "If you are using tools like Bazel or Buck2 they likely already employ their own sandboxing via namespacing for builds. Integrating Nix into these ecosystems becomes incredibly impractical because we run into nested user namespace and mount restrictions."
I don't know enough details to really judge but it seems to me like you are trying to workaround a mess in userspace with kernel changes.
Anyway I'm pretty sure Christian will have more educated answer than me but I just wanted to express my skepticism about this approach and that perhaps you wait for feedback from him before spending more time on these patches.
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Quoted 33 lines · Jan Kara
Quoted 11 lines · Farid Zakaria
Currently, standard ELF and ELF FDPIC loaders require a fixed, absolute path to the dynamic linker/interpreter (specified via
PT_INTERP). This creates significant inflexibility for relocatable dynamic interpreters, where binaries are packaged independent of global system paths.The primary goal of this patch series is to support relocatable binaries in Nix, where packages are stored in a read-only store (typically /nix/store). Allowing the ELF interpreter path to be resolved dynamically relative to the binary's location via $ORIGIN enables Nix packages to be relocated without needing patchelf or wrapper scripts.
Quoted 6 lines
For details on the design and motivation for this in Nix, see:
https://fzakaria.com/2026/06/21/nix-needs-relocatable-binariesThis series introduces support for resolving the $ORIGIN placeholder in the ELF interpreter path, bringing the kernel's binary loading behavior in line with user-space dynamic linker origin resolution.
Thanks for the patches! Before dwelving into implementation details we need to discuss whether something like this even belongs to the kernel. Frankly to me this looks rather arbitrary and tied to the particular way you've decided to setup your package management system. In particular the usual answer for situations like these is to use namespaces which you discount in your blog with: "If you are using tools like Bazel or Buck2 they likely already employ their own sandboxing via namespacing for builds. Integrating Nix into these ecosystems becomes incredibly impractical because we run into nested user namespace and mount restrictions."
I don't know enough details to really judge but it seems to me like you are trying to workaround a mess in userspace with kernel changes.
Quoted 8 lines
Anyway I'm pretty sure Christian will have more educated answer than me but I just wanted to express my skepticism about this approach and that perhaps you wait for feedback from him before spending more time on these patches.
Honza
--
Jan Kara <[email protected]>
SUSE Labs, CR
Thank you for taking the time to look at my patch.
I'm new to contributing to Linux mailing list so receiving feedback and responses is welcoming :)
Having put forward the patch, I'm clearly biased toward thinking this support should exist in the kernel.
If I had to think to strengthen my argument would be that the kernel should not be imposing how the interpreter is found on userland. Finding the interpreter relative to the binary would be useful for package deployment scenarios similar to app-bundles beyond systems like Nix -- which is the originating reason why $ORIGIN exists in the dynamic linker.
To me, the gap is that prior to systems like Nix, the idea of wanting your dynamic linker to be part of your app bundle was not necessary but Nix models the dependency chain down to the loader. Such functionality would be even more correct for these other bundled solutions as well, making them portable across userspace glibc versions for instance.
I see Jori Koolstra mentioned that Sashiko found feedback on the implementation.
Is it worthwhile to amend the patch now or should I wait to hear back on whether such a contribution would be accepted?
Jori:
I'm not 100% clear on your question but the LLM assisted with some code generation and brainstorming.
Hi, I am another Nix developer, and have participated in some LKML discussions in the (recent and distant) past, and thought I should weigh in here too.
Quoted 9 lines · Farid Zakaria
Quoted 7 lines · Jan Kara
Thanks for the patches! Before dwelving into implementation details we need to discuss whether something like this even belongs to the kernel. Frankly to me this looks rather arbitrary and tied to the particular way you've decided to setup your package management system.
I don't know enough details to really judge but it seems to me like you are trying to workaround a mess in userspace with kernel changes.
Quoted 8 lines
Having put forward the patch, I'm clearly biased toward thinking this support should exist in the kernel.
If I had to think to strengthen my argument would be that the kernel should not be imposing how the interpreter is found on userland. Finding the interpreter relative to the binary would be useful for package deployment scenarios similar to app-bundles beyond systems like Nix -- which is the originating reason why $ORIGIN exists in the dynamic linker.
Yes, the idea of making "relocatable software" is not a new one, and indeed it is why $ORIGIN is supported in the RPATH etc. in the first place.
Most of the programming model for writing relocatable software is fixed at this point. For example, /proc/self/exe made it much easier to look up arbitrary stuff relevant to the current executable. It is just some initial entry point stuff (the ELF interpreter, and shebangs) which is a glaring exception. Those should support $ORIGIN too. There is no good technical justification (that I can think of) for some but not all of these supporting $ORIGIN --- either it makes sense everywhere, or it makes sense nowhere.
(I suspect the only reason it didn't happen was pure inertia/Conway's law --- easier for whoever was excited about $ORIGIN to change the glibc loader than the kernel.)
Quoted 6 lines
To me, the gap is that prior to systems like Nix, the idea of wanting your dynamic linker to be part of your app bundle was not necessary but Nix models the dependency chain down to the loader. Such functionality would be even more correct for these other bundled solutions as well, making them portable across userspace glibc versions for instance.
Yes, exactly. Traditionally people thought "eh /lib/ld-linux.so.* doesn't change too much", and decided relocatable software that nonetheless hard-coded that absolute path to an unknown system-provided ELF interpreter was good enough. (Or if they weren't good enough, they went with static linking, but that imposes other costs.)
Now there do exist purely-user-space work-arounds, like
https://github.com/Mic92/wrap-buddy, but they are quite complex, and involve various patching trickery that is likely to scare a lot of security analysis tools. A kernel-based solution that allows clean declarative expression of intent with $ORIGIN is much more elegant.
Quoted 6 lines
Quoted 6 lines
In particular the
usual answer for situations like these is to use namespaces which you discount in your blog with: "If you are using tools like Bazel or Buck2 they likely already employ their own sandboxing via namespacing for builds. Integrating Nix into these ecosystems becomes incredibly impractical because we run into nested user namespace and mount restrictions."
I think it is good to see what Conda does as documented in
<https://docs.conda.io/projects/conda-build/en/stable/resources/make-relocatable.html>
and consider why relying on namespaces vs good old-fashioned relocatable isn't good enough for them either.
(I don't doubt that Conda would find this approach more robust than their sedding tricks, and prefer to use it where possible.)
The short answer is while all of us in the build system space love sandboxing during the build, we don't want that to lead to *requiring* run time sandboxing of the built artifacts. For example, we can certainly arrange sandboxing so /lib/ld-linux.so.* is the one that some executable expects now, but every time that executable is run, it *must* be run in a root filesystem where /lib/ld-linux.so.* is the loader it expects.
If you have multiple programs that (for whatever reasons) expect multiple different loaders, all spawning one another, it would potentially incur quite the development cost to ensure that they all do the proper unsharing to make everything work.
Relocatability recognizes that whether or not namespaces exist, in an "open world" scenario where we don't know how the software we are writing will be combined with other software for deployment downstream in different ways, it is easiest to adopt an idiom where different things can be placed at different absolute paths, at the user's discretion, and so conflicts are always avoidable.
Quoted 8 lines
Quoted 8 lines
Anyway I'm pretty sure Christian will have more educated answer than me but I just wanted to express my skepticism about this approach and that perhaps you wait for feedback from him before spending more time on these patches.
Honza
--
Jan Kara <[email protected]>
SUSE Labs, CR
Waiting makes sense, I am curious too what he will have to say.
Christian Brauner
maintainerQuoted 83 lines · John Ericson
Hi, I am another Nix developer, and have participated in some LKML discussions in the (recent and distant) past, and thought I should weigh in here too.
Quoted 10 lines · Farid Zakaria
On Mon, Jun 22, 2026 at 3:40 AM Jan Kara <[email protected]> wrote:
Having put forward the patch, I'm clearly biased toward thinking this support should exist in the kernel.
If I had to think to strengthen my argument would be that the kernel should not be imposing how the interpreter is found on userland. Finding the interpreter relative to the binary would be useful for package deployment scenarios similar to app-bundles beyond systems like Nix -- which is the originating reason why $ORIGIN exists in the dynamic linker.Yes, the idea of making "relocatable software" is not a new one, and indeed it is why
$ORIGINis supported in the RPATH etc. in the first place.Most of the programming model for writing relocatable software is fixed at this point. For example,
/proc/self/exemade it much easier to look up arbitrary stuff relevant to the current executable. It is just some initial entry point stuff (the ELF interpreter, and shebangs) which is a glaring exception. Those should support$ORIGINtoo. There is no good technical justification (that I can think of) for some but not all of these supporting$ORIGIN--- either it makes sense everywhere, or it makes sense nowhere.(I suspect the only reason it didn't happen was pure inertia/Conway's law --- easier for whoever was excited about
$ORIGINto change the glibc loader than the kernel.)Quoted 6 lines
To me, the gap is that prior to systems like Nix, the idea of wanting your dynamic linker to be part of your app bundle was not necessary but Nix models the dependency chain down to the loader. Such functionality would be even more correct for these other bundled solutions as well, making them portable across userspace glibc versions for instance.
Yes, exactly. Traditionally people thought "eh
/lib/ld-linux.so.*doesn't change too much", and decided relocatable software that nonetheless hard-coded that absolute path to an unknown system-provided ELF interpreter was good enough. (Or if they weren't good enough, they went with static linking, but that imposes other costs.)Now there do exist purely-user-space work-arounds, like
https://github.com/Mic92/wrap-buddy, but they are quite complex, and involve various patching trickery that is likely to scare a lot of security analysis tools. A kernel-based solution that allows clean declarative expression of intent with$ORIGINis much more elegant.Quoted 1 line
Quoted 1 line
In particular the
I think it is good to see what Conda does as documented in
<https://docs.conda.io/projects/conda-build/en/stable/resources/make-relocatable.html>
and consider why relying on namespaces vs good old-fashioned relocatable isn't good enough for them either.(I don't doubt that Conda would find this approach more robust than their sedding tricks, and prefer to use it where possible.)
The short answer is while all of us in the build system space love sandboxing during the build, we don't want that to lead to *requiring* run time sandboxing of the built artifacts. For example, we can certainly arrange sandboxing so
/lib/ld-linux.so.*is the one that some executable expects now, but every time that executable is run, it *must* be run in a root filesystem where/lib/ld-linux.so.*is the loader it expects.If you have multiple programs that (for whatever reasons) expect multiple different loaders, all spawning one another, it would potentially incur quite the development cost to ensure that they all do the proper unsharing to make everything work.
Relocatability recognizes that whether or not namespaces exist, in an "open world" scenario where we don't know how the software we are writing will be combined with other software for deployment downstream in different ways, it is easiest to adopt an idiom where different things can be placed at different absolute paths, at the user's discretion, and so conflicts are always avoidable.
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Anyway I'm pretty sure Christian will have more educated answer than me but
Waiting makes sense, I am curious too what he will have to say.
The arguments I have heard from various people so far are:
(1) Userspace would be able to clone a random chroot to /woot and run a binary from it without having to set up a complicated sandbox effectively making dynamically linked binaries more like static binaries in a sense.
(2) Quote:
"If you debootstrap/dnf a chroot to some location in your home dir and try to run a binary from it, that it tries to load the libraries from your /usr is a pretty unintuitive and not at all useful behavior."
(3) Quote:
"[Various remote execution things run in locked down containers that disable userns, which makes the sandbox impossible and hence our builds wouldn't work there."
I'm discounting "Oh, userspace already allows this so why not the kernel.". I think that's generally a bad argument. Kernel and userspace aren't really alike in that regard.
The userspace ORIGIN concept is guarded behind AT_SECURE. The kernel has to enforce the same rule. That means the loader now depends on the type of binary. I think this is a rather serious issue.
First, it creates confusion in userspace what loader is used. Second, it means anything that any build/chroot that uses AT_SECURE binaries now has to use the sandboxing solution anyway or risk that some binaries use the system loader and others the chroot loader.
Ignoring AT_SECURE, LSMs likely will need a say in whether that ORIGIN thing gets honored or not introducing yet another vector where this can be overriden or ignored.
Also, we change long-standing kernel behavior which will be very surprising for any userspace that might implicitly rely on the fact that the system loader is used. So even if we were to do something like this it would very likely have to be configurable in some way.
This makes this all ripe for malicious loader injection attacks. And we need to consider this possibility.
So I'm not enthusiastic about this. I want this to be consistent.
Quoted 4 lines · Christian Brauner
Also, we change long-standing kernel behavior which will be very surprising for any userspace that might implicitly rely on the fact that the system loader is used. So even if we were to do something like this it would very likely have to be configurable in some way.
Yes that's definitely something that worried me.
Quoted 1 line
So I'm not enthusiastic about this. I want this to be consistent.
For what it's worth, I somewhat wish the PT_INTERP field didn't exist at all. IMO the ELF spec is very weird for some parts of the same headers being for the kernel, and the other parts being for userspace. If there were separate data structures (think encapsulated packets) so it was crystal clear who was responsible for what, we wouldn't be in this situation.
While I earlier complained that our
<https://github.com/Mic92/wrap-buddy> work-around was likely to make some security analyses/policies go haywire, arguably it implements this
behavior: from the kernel's perspective, everything is statically linked, and the program, entirely in userspace, takes responsibility for whatever loading it needs.
Put that way, this is less of an ugly work-around than the way I wish things worked period in a world without PT_INTERP. Dynamic linking would be entirely a userspace concern and the kernel would not know or care.
----
What about shebangs? It's the same problem, with the same solutions and pitfalls. We can also make wrapper executables around them that do the right thing, but turning mere scripts into statically linked binaries is a bit heavier. I assume the answer is the same, but just want to double check before we close this off.
Cheers,
John
Quoted 13 lines · Christian Brauner
The arguments I have heard from various people so far are:
(1) Userspace would be able to clone a random chroot to /woot and run a binary from it without having to set up a complicated sandbox effectively making dynamically linked binaries more like static binaries in a sense.
(2) Quote:
"If you debootstrap/dnf a chroot to some location in your home dir and try to run a binary from it, that it tries to load the libraries from your /usr is a pretty unintuitive and not at all useful behavior."
Quoted 4 lines
(3) Quote:
"[Various remote execution things run in locked down containers that disable userns, which makes the sandbox impossible and hence our builds wouldn't work there."
FWIW I think someone also mentioned to me that it would make things easier for them if they could build a piece of software in one environment and then bundle it up with all required libraries and such and run it in a very different environment, without container/sandboxing stuff and without static linking. But I guess that's kinda niche.
Quoted 5 lines
I'm discounting "Oh, userspace already allows this so why not the kernel.". I think that's generally a bad argument. Kernel and userspace aren't really alike in that regard.
The userspace ORIGIN concept is guarded behind
AT_SECURE. The kernel has
(To be pedantic: The userspace $ORIGIN concept is only partially gated on AT_SECURE - glibc has an allowlist of acceptable library directories, listed in "/lib64/ld-linux-x86-64.so.2 --list-diagnostics | grep ^path.system_dirs". But clearly we wouldn't want to mirror that in the kernel.)
Quoted 2 lines
to enforce the same rule. That means the loader now depends on the type of binary. I think this is a rather serious issue.
And annoyingly, the bprm->secureexec flag can change in security_bprm_creds_from_file(), which is currently reached from begin_new_exec(), which is called after we've already opened the interpreter, so accessing ->secureexec state during the interpreter lookup would require some refactoring. So I think this is a doable change, but would require more work.
Or we could take the easy way out and say "the kernel always rejects this unless LSM_UNSAFE_NO_NEW_PRIVS is set", which would make it clear that this can't lead to privilege escalation and would serve as an incentive for people to stop doing stuff that relies on setuid binaries or privileged apparmor/selinux transitions. :P
Quoted 4 lines
First, it creates confusion in userspace what loader is used. Second, it means anything that any build/chroot that uses
AT_SECUREbinaries now has to use the sandboxing solution anyway or risk that some binaries use the system loader and others the chroot loader.
I think we would probably just fail the execve() attempt if we see $ORIGIN in the interpreter in an AT_SECURE execution? Since the interpreter field does not allow listing multiple alternatives.
Quoted 8 lines
Ignoring
AT_SECURE, LSMs likely will need a say in whether that ORIGIN thing gets honored or not introducing yet another vector where this can be overriden or ignored.Also, we change long-standing kernel behavior which will be very surprising for any userspace that might implicitly rely on the fact that the system loader is used. So even if we were to do something like this it would very likely have to be configurable in some way.
I think the proposed patch will only change behavior if the interpreter path starts with "$ORIGIN"? That wouldn't work on existing kernels unless you have a directory literally named "$ORIGIN" in the cwd, because "$ORIGIN/..." would be interpreted as a normal relative path.
Quoted 4 lines
This makes this all ripe for malicious loader injection attacks. And we need to consider this possibility.
So I'm not enthusiastic about this. I want this to be consistent.
Quoted 19 lines · On Fri, 26 Jun 2026 14:39:22 +0200 Jann Horn <[email protected]> wrote:
Quoted 13 lines · Christian Brauner
The arguments I have heard from various people so far are:
(1) Userspace would be able to clone a random chroot to /woot and run a binary from it without having to set up a complicated sandbox effectively making dynamically linked binaries more like static binaries in a sense.
(2) Quote:
"If you debootstrap/dnf a chroot to some location in your home dir and try to run a binary from it, that it tries to load the libraries from your /usr is a pretty unintuitive and not at all useful behavior."Quoted 4 lines
(3) Quote:
"[Various remote execution things run in locked down containers that disable userns, which makes the sandbox impossible and hence our builds wouldn't work there."
Quoted 6 lines
FWIW I think someone also mentioned to me that it would make things easier for them if they could build a piece of software in one environment and then bundle it up with all required libraries and such and run it in a very different environment, without container/sandboxing stuff and without static linking. But I guess that's kinda niche.
The problem with 'ship the shared libraries with the application' is that you get all the problems of static linking.
If there is a bug in the library code you can't fix it without getting the 3rd party to rebuild their application package.
If the bug is in a system shared library updating the system libraries fixes the bug.
Now this does require that the writers of shared libraries maintain backwards compatibility and that the 'system' provides the required updates.
I remember a long time ago the company I worked for shipped a system where the libc.so the linker found was actually an archive library one of whose members was a shared library.
So some functions were dynamically loaded and others static. There was a bug in one of the static functions (IIRC it corrupted the utmp file), once located and fixed the 3rd party had to be persuaded to rebuild and re-release their product.
(It has to be said that anyone with half a brain would have realised that because libc was split for compatibility reasons, statically linking this particular function was actually stupid.)
David
Quoted 29 lines · David Laight
Quoted 19 lines · On Fri, 26 Jun 2026 14:39:22 +0200 Jann Horn <[email protected]> wrote:
Quoted 13 lines · Christian Brauner
The arguments I have heard from various people so far are:
(1) Userspace would be able to clone a random chroot to /woot and run a binary from it without having to set up a complicated sandbox effectively making dynamically linked binaries more like static binaries in a sense.
(2) Quote:
"If you debootstrap/dnf a chroot to some location in your home dir and try to run a binary from it, that it tries to load the libraries from your /usr is a pretty unintuitive and not at all useful behavior."Quoted 4 lines
(3) Quote:
"[Various remote execution things run in locked down containers that disable userns, which makes the sandbox impossible and hence our builds wouldn't work there."Quoted 6 lines
FWIW I think someone also mentioned to me that it would make things easier for them if they could build a piece of software in one environment and then bundle it up with all required libraries and such and run it in a very different environment, without container/sandboxing stuff and without static linking. But I guess that's kinda niche.
Quoted 4 lines
The problem with 'ship the shared libraries with the application' is that you get all the problems of static linking.
If there is a bug in the library code you can't fix it without getting the 3rd party to rebuild their application package.
Yes, it's appropriate for weird use cases like "I want to run this historical version of the software and its dependencies", it's not necessarily a good idea for normal application use.
It's very tough not to get too idealogical but please forgive me a little as a Nix user.
What you are describing is a feature and not a bug. In Nix and similar store-based systems the goal is purity and reproduction. Security vulnerabilities are solved by potentially rebuilding the world if they are sufficiently deep in the dependency graph.
The flip side to this deployment strategy is that it's very easy to understand and audit one's system. By not linking to global library directories, you cut down on a complete vector of abuse or system wide updates that maybe make one vulnerable.
Back to the discussion at hand :)
Glad to see some discussion. This is something I continue to think fits well in the kernel and this strategy of deployment is becoming increasingly popular with tools like Bazel and Buck as well.
I continue to wait and am ready to make any necessary changes.
Quoted 45 lines · Jann Horn
Quoted 4 lines · David Laight
Quoted 1 line · On Fri, 26 Jun 2026 14:39:22 +0200 Jann Horn <[email protected]> wrote:
On Thu, Jun 25, 2026 at 10:50 AM Christian Brauner <[email protected]>
Quoted 3 lines · wrote:
Quoted 3 lines
Quoted 3 lines
The arguments I have heard from various people so far are:
(1) Userspace would be able to clone a random chroot to /woot and
run a
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
binary from it without having to set up a complicated sandbox effectively making dynamically linked binaries more like static binaries in a sense.
(2) Quote:
"If you debootstrap/dnf a chroot to some location in your home dir and try to run a binary from it, that it tries to loadthe
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
libraries from your /usr is a pretty unintuitive and not at all useful behavior."
(3) Quote:
"[Various remote execution things run in locked down containersthat
Quoted 10 lines
Quoted 9 lines
Quoted 2 lines
disable userns, which makes the sandbox impossible and hence our builds wouldn't work there."
FWIW I think someone also mentioned to me that it would make things easier for them if they could build a piece of software in one environment and then bundle it up with all required libraries and such and run it in a very different environment, without container/sandboxing stuff and without static linking. But I guess that's kinda niche.
Quoted 3 lines
The problem with 'ship the shared libraries with the application' is that you get all the problems of static linking.
If there is a bug in the library code you can't fix it without gettingthe
Quoted 1 line
3rd party to rebuild their application package.
Yes, it's appropriate for weird use cases like "I want to run this historical version of the software and its dependencies", it's not necessarily a good idea for normal application use.
Quoted 31 lines · On Fri, 26 Jun 2026 15:34:12 +0200 Jann Horn <[email protected]> wrote:
Quoted 29 lines · David Laight
Quoted 25 lines · On Fri, 26 Jun 2026 14:39:22 +0200 Jann Horn <[email protected]> wrote:
Quoted 13 lines · Christian Brauner
The arguments I have heard from various people so far are:
(1) Userspace would be able to clone a random chroot to /woot and run a binary from it without having to set up a complicated sandbox effectively making dynamically linked binaries more like static binaries in a sense.
(2) Quote:
"If you debootstrap/dnf a chroot to some location in your home dir and try to run a binary from it, that it tries to load the libraries from your /usr is a pretty unintuitive and not at all useful behavior."Quoted 4 lines
(3) Quote:
"[Various remote execution things run in locked down containers that disable userns, which makes the sandbox impossible and hence our builds wouldn't work there."FWIW I think someone also mentioned to me that it would make things easier for them if they could build a piece of software in one environment and then bundle it up with all required libraries and such and run it in a very different environment, without container/sandboxing stuff and without static linking. But I guess that's kinda niche.
Quoted 8 lines
Quoted 4 lines
The problem with 'ship the shared libraries with the application' is that you get all the problems of static linking.
If there is a bug in the library code you can't fix it without getting the 3rd party to rebuild their application package.Yes, it's appropriate for weird use cases like "I want to run this historical version of the software and its dependencies", it's not necessarily a good idea for normal application use.
That's what LD_LIBRARY_PATH is for ...
And if you want to use a different elf interpreter just run it and pass the program name and arguments to it. eg:
/lib64/ld-linux-x64-64.so.2 /bin/echo fubar
Last time I did that I was trying to run non-linux ppc elf program. I got part way there, but needed to build a lot more of libc.
David
Christian Brauner
maintainerQuoted 9 lines
Quoted 8 lines
Ignoring
AT_SECURE, LSMs likely will need a say in whether that ORIGIN thing gets honored or not introducing yet another vector where this can be overriden or ignored.Also, we change long-standing kernel behavior which will be very surprising for any userspace that might implicitly rely on the fact that the system loader is used. So even if we were to do something like this it would very likely have to be configurable in some way.
Quoted 5 lines
I think the proposed patch will only change behavior if the interpreter path starts with "$ORIGIN"? That wouldn't work on existing kernels unless you have a directory literally named "$ORIGIN" in the cwd, because "$ORIGIN/..." would be interpreted as a normal relative path.
I was thinking:
You download a bunch of $ORIGIN binaries into /woot on a kernel that doesn't support $ORIGIN. You forget about /woot and update your kernel. You have no idea that kernel behavior has changed to allow $ORIGIN to work for the loader. You remember /woot exists and are starting to execute binaries under the assumption that $ORIGIN is purely a userspace concept applicable to shared libraries.
/woot came with a malicious loader and it suddenly gets used - if that happens in a sandbox noboday cares. Now that $ORIGIN works and no sandbox might be needed I think enabling $ORIGIN might still end up with funny surprises.
Christian Brauner
maintainerQuoted 7 lines · Christian Brauner
Quoted 5 lines
I think the proposed patch will only change behavior if the interpreter path starts with "$ORIGIN"? That wouldn't work on existing kernels unless you have a directory literally named "$ORIGIN" in the cwd, because "$ORIGIN/..." would be interpreted as a normal relative path.
I was thinking:
A few other things came to my mind:
(1) memfds: /memfd:woot-woot
A memfd doesn't have a path so any $ORIGIN type behavior - script or not - will be at least useless if not very confusing outside of a sandbox.
Btw, used by runC to guard against binary overwrite attacks.
(1.1) deleted memfds: /memfd:woot-woot (deleted)
Same as (1).
(2) deleted executables:
/woot/woot-woot (deleted)
(3) unrecoverable executable paths
No path can be resolved at all. What to do?
All of that needs consistent, easy to reason about treatment. Reading through the glibc implementation of $ORIGIN for shared libraries in rpath - even with an eye on cutting through most of the complexity - doesn't give me very warm feelings. It feels very hackish and full of edge cases...
Imho, tying the lookup of the interpreter to the binary itself and making the kernel responsible for figuring out the relationship by resolving bprm->file and splicing it with PT_INTERP is terrible.
So, I kinda like the concept of having relocatable dynamic executables but then let's cut through all the userspace red tape and figure out something that's super simple and outsources the problem to userspace.
I think the wrap-buddy approach here: https://github.com/Mic92/wrap-buddy is going in the right direction in that it gets the kernel completely out of the way. I like that a lot more than moving more nasty bits into the kernel itself.
In a way we have been doing something in systemd that goes in a similar direction. As of systemd 261 systemd _only_ links against libc and nothing else.
Any other shared library is dlopen()ened as needed (discoverable via elf-notes). Lennart wrote about this just a few days ago:
https://mastodon.social/@pid_eins/116781776665322560
This effectively minimizes the work the loader has to do at startup. Imho, your effort with wrap-buddy is related. To me moving the loader invocation out of the kernel and into userspace makes a lot of sense to me.
On Sun, 28 Jun 2026 15:20:11 +0200
Christian Brauner <[email protected]> wrote:
...
Quoted 4 lines
All of that needs consistent, easy to reason about treatment. Reading through the glibc implementation of $ORIGIN for shared libraries in rpath - even with an eye on cutting through most of the complexity - doesn't give me very warm feelings. It feels very hackish and full of edge cases...
I'm sure that NetBSD refused to implement $ORIGIN because it was full of loopholes.
You'd need the kernel to hold a reference to the directory inode and have a 'magic' argument to openat() to be relative to that directory rather than cwd.
David
Quoted 52 lines · Christian Brauner
Quoted 7 lines · Christian Brauner
Quoted 5 lines
I think the proposed patch will only change behavior if the interpreter path starts with "$ORIGIN"? That wouldn't work on existing kernels unless you have a directory literally named "$ORIGIN" in the cwd, because "$ORIGIN/..." would be interpreted as a normal relative path.
I was thinking:
A few other things came to my mind:
(1) memfds: /memfd:woot-woot
A memfd doesn't have a path so any $ORIGIN type behavior - script or not - will be at least useless if not very confusing outside of a sandbox.
Btw, used by runC to guard against binary overwrite attacks.
(1.1) deleted memfds: /memfd:woot-woot (deleted)
Same as (1).(2) deleted executables:
/woot/woot-woot (deleted)
(3) unrecoverable executable paths
No path can be resolved at all. What to do?
All of that needs consistent, easy to reason about treatment. Reading through the glibc implementation of $ORIGIN for shared libraries in rpath - even with an eye on cutting through most of the complexity - doesn't give me very warm feelings. It feels very hackish and full of edge cases...
Imho, tying the lookup of the interpreter to the binary itself and making the kernel responsible for figuring out the relationship by resolving
bprm->fileand splicing it withPT_INTERPis terrible.So, I kinda like the concept of having relocatable dynamic executables but then let's cut through all the userspace red tape and figure out something that's super simple and outsources the problem to userspace.
I think the wrap-buddy approach here: https://github.com/Mic92/wrap-buddy is going in the right direction in that it gets the kernel completely out of the way. I like that a lot more than moving more nasty bits into the kernel itself.
In a way we have been doing something in systemd that goes in a similar direction. As of systemd 261 systemd _only_ links against libc and nothing else.
Quoted 8 lines
Any other shared library is dlopen()ened as needed (discoverable via elf-notes). Lennart wrote about this just a few days ago:
https://mastodon.social/@pid_eins/116781776665322560This effectively minimizes the work the loader has to do at startup. Imho, your effort with wrap-buddy is related. To me moving the loader invocation out of the kernel and into userspace makes a lot of sense to me.
Wow the systemd approach sounds pretty surprising. As a Nix/NixOS user having software enable/disable features depending on changes to the system ad-hoc seems like a footgun. Despite the allure of shared objects, the are less re-used in practice and rebuilding them when a CVE appears is just as easy.
A good read to me:
https://web.archive.org/web/20260320072121/https://drewdevault.com/dynlib.html
https://lore.kernel.org/lkml/CAHk-=whs8QZf3YnifdLv57+FhBi5_WeNTG1B-suOES=RcUSmQg@mail.gmail.com/
(yes true, NixOS also uses "shared libraries" but they are effectively static in how they are setup with fixed paths).
Anyway, I digress.
What other options are there to go after for a relocatable binary? The steps wrap-buddy has to take... are a bit horrendous in practicality (although amazing from an engineering-sense).
binfmt itself is configurable and modules can register additional exec types. What about something similar for ELF & INTERP ? Maybe we can meet-half way and support it via a plugin architecture that NixOS can leverage.
Christian Brauner
maintainerQuoted 7 lines
Anyway, I digress.
What other options are there to go after for a relocatable binary? The steps wrap-buddy has to take... are a bit horrendous in practicality (although amazing from an engineering-sense).binfmt itself is configurable and modules can register additional exec types. What about something similar for ELF & INTERP ?
Not sure if my message made it yesterday. It somehow seems it didn't so I'm repeating it: let's look at the code for what you had in mind.
On Sun, 28 Jun 2026 15:20:11 +0200
Christian Brauner <[email protected]> wrote:
...
Quoted 11 lines
In a way we have been doing something in systemd that goes in a similar direction. As of systemd 261 systemd _only_ links against libc and nothing else.
Any other shared library is dlopen()ened as needed (discoverable via elf-notes). Lennart wrote about this just a few days ago:
https://mastodon.social/@pid_eins/116781776665322560This effectively minimizes the work the loader has to do at startup. Imho, your effort with wrap-buddy is related. To me moving the loader invocation out of the kernel and into userspace makes a lot of sense to me.
I've done that for libraries that a program doesn't normally need. But it doesn't make sense for a long-lived daemon.
The work almost certainly needs doing at some point.
The other problem is that it is hard to handle symbol versioning. So the function you get may not match the one the header file defined and you suffer the consequences.
David
RFC PATCH 1/1 fs: introduce pluggable ELF interpreter loader registry
Introduce a pluggable framework for ELF binary loading to allow dynamic resolution and redirection of program interpreters (PT_INTERP). This is primarily designed to support hermetic path resolution like NixOS $ORIGIN relative dynamic linkers without bloating the core ELF loader or compromising system execution security.
Introduce a new registration interface for kernel modules to register open_interpreter callbacks. Standard ELF loading queries this registry; if a plugin resolves a custom segment type (like PT_INTERP_NIX), it returns a file descriptor for the resolved interpreter. Secure execution environments (bprm->secureexec) bypass relative resolution for safety.
---
Hey Christian,
Here is a rough draft of what I thought something could look like for a pluggable ELF interpreter loader. I've included the relocatable loader in the patch but this can be out-of-tree if needed; it's also named "Nix" to distinguish the need for it.
I chose to read from a distinct segment type (PT_INTERP_NIX) to avoid backwards incompatibility.
The default loader itself could be a plugin similar to binfmt_elf, but for now I wanted the patch to be small to demonstrate the concept.
fs/Kconfig.binfmt | 15 +++++ fs/Makefile | 1 + fs/binfmt_elf.c | 24 ++++++++ fs/binfmt_elf_nix.c | 108 ++++++++++++++++++++++++++++++++++++ fs/exec.c | 47 ++++++++++++++++ include/linux/elf_plugins.h | 39 +++++++++++++ 6 files changed, 234 insertions(+) create mode 100644 fs/binfmt_elf_nix.c create mode 100644 include/linux/elf_plugins.h
fs/Kconfig.binfmt +15 -0
@@ -38,6 +38,21 @@ config BINFMT_ELF_KUNIT_TEST only needed for debugging. Note that with CONFIG_COMPAT=y, the compat_binfmt_elf KUnit test is also created. +config BINFMT_ELF_PLUGINS + bool "Enable plugin support for ELF interpreter loading" + depends on BINFMT_ELF + help + This option allows kernel modules to register handlers to dynamically + resolve and override the ELF program interpreter (e.g. supporting relative + interpreter paths with $ORIGIN). + +config BINFMT_ELF_NIX + tristate "ELF interpreter plugin for NixOS ($ORIGIN support)" + depends on BINFMT_ELF_PLUGINS + help + This builds the NixOS ELF interpreter plugin. It intercepts PT_INTERP_NIX + headers to resolve relative and $ORIGIN interpreter paths. + config COMPAT_BINFMT_ELF def_bool y depends on COMPAT && BINFMT_ELF
fs/Makefile +1 -0
@@ -35,6 +35,7 @@ obj-$(CONFIG_FILE_LOCKING) += locks.o obj-$(CONFIG_BINFMT_MISC) += binfmt_misc.o obj-$(CONFIG_BINFMT_SCRIPT) += binfmt_script.o obj-$(CONFIG_BINFMT_ELF) += binfmt_elf.o +obj-$(CONFIG_BINFMT_ELF_NIX) += binfmt_elf_nix.o obj-$(CONFIG_COMPAT_BINFMT_ELF) += compat_binfmt_elf.o obj-$(CONFIG_BINFMT_ELF_FDPIC) += binfmt_elf_fdpic.o obj-$(CONFIG_BINFMT_FLAT) += binfmt_flat.o
fs/binfmt_elf.c +24 -0
@@ -35,6 +35,7 @@#include <linux/random.h> #include <linux/elf.h> #include <linux/elf-randomize.h> +#include <linux/elf_plugins.h> #include <linux/utsname.h> #include <linux/coredump.h> #include <linux/sched.h>
@@ -870,6 +871,12 @@ static int load_elf_binary(struct linux_binprm *bprm)if (!elf_phdata) goto out; + interpreter =elf_plugin_open_interpreter (bprm, elf_ex, elf_phdata); +if (IS_ERR (interpreter)) { + retval =PTR_ERR (interpreter); + goto out_free_ph; + } + elf_ppnt = elf_phdata;for (i =0 ; i < elf_ex->e_phnum; i++, elf_ppnt++) {char *elf_interpreter;
@@ -882,6 +889,9 @@ static int load_elf_binary(struct linux_binprm *bprm)if (elf_ppnt->p_type !=PT_INTERP )continue ; +if (interpreter) +continue ; +/* * This is the program interpreter used for shared libraries - * for now assume that this is an a.out format binary.
@@ -935,6 +945,20 @@ static int load_elf_binary(struct linux_binprm *bprm) goto out_free_ph; } +if (interpreter && !interp_elf_ex) { + interp_elf_ex =kmalloc_obj (*interp_elf_ex); +if (!interp_elf_ex) { + retval = -ENOMEM ; + goto out_free_file; + } + +/* Get the exec headers */ + retval =elf_read (interpreter, interp_elf_ex, +sizeof (*interp_elf_ex),0 ); +if (retval <0 ) + goto out_free_dentry; + } + elf_ppnt = elf_phdata;for (i =0 ; i < elf_ex->e_phnum; i++, elf_ppnt++)switch (elf_ppnt->p_type) {
fs/binfmt_elf_nix.c new file +108 -0
108 lines not on this page · raw message
fs/exec.c +47 -0
59 lines not on this page · raw message
include/linux/elf_plugins.h new file +39 -0
@@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef _LINUX_ELF_PLUGINS_H +#define _LINUX_ELF_PLUGINS_H + +#include <linux/binfmts.h> +#include <linux/elf.h> +#include <linux/list.h> + +struct elf_plugin { +struct list_head list; +struct module *owner; +struct file *(*open_interpreter)(struct linux_binprm *bprm, +struct elfhdr *elf_ex, +struct elf_phdr *elf_phdata); +}; + +#if IS_ENABLED (CONFIG_BINFMT_ELF_PLUGINS ) +int register_elf_plugin (struct elf_plugin *plugin); +void unregister_elf_plugin (struct elf_plugin *plugin); +struct file *elf_plugin_open_interpreter (struct linux_binprm *bprm, +struct elfhdr *elf_ex, +struct elf_phdr *elf_phdata); +#else +static inline int register_elf_plugin (struct elf_plugin *plugin) +{ +return 0 ; +} +static inline void unregister_elf_plugin (struct elf_plugin *plugin) +{ +} +static inline struct file *elf_plugin_open_interpreter (struct linux_binprm *bprm, +struct elfhdr *elf_ex, +struct elf_phdr *elf_phdata) +{ +return NULL ; +} +#endif + +#endif /* _LINUX_ELF_PLUGINS_H */
Signature · 1 line
2.51.2
Christian Brauner
maintainerRe: [RFC PATCH] fs: introduce pluggable ELF interpreter loader registry
Quoted 273 lines
Introduce a pluggable framework for ELF binary loading to allow dynamic resolution and redirection of program interpreters (
PT_INTERP). This is primarily designed to support hermetic path resolution like NixOS $ORIGIN relative dynamic linkers without bloating the core ELF loader or compromising system execution security.Introduce a new registration interface for kernel modules to register
open_interpretercallbacks. Standard ELF loading queries this registry; if a plugin resolves a custom segment type (likePT_INTERP_NIX), it returns a file descriptor for the resolved interpreter. Secure execution environments (bprm->secureexec) bypass relative resolution for safety.Signed-off-by Farid Zakaria <[email protected]>fs/Kconfig.binfmt +15
-0@@ -38,6 +38,21 @@ config BINFMT_ELF_KUNIT_TEST only needed for debugging. Note that with CONFIG_COMPAT=y, the compat_binfmt_elf KUnit test is also created. +config BINFMT_ELF_PLUGINS + bool "Enable plugin support for ELF interpreter loading" + depends on BINFMT_ELF + help + This option allows kernel modules to register handlers to dynamically + resolve and override the ELF program interpreter (e.g. supporting relative + interpreter paths with $ORIGIN). + +config BINFMT_ELF_NIX + tristate "ELF interpreter plugin for NixOS ($ORIGIN support)" + depends on BINFMT_ELF_PLUGINS + help + This builds the NixOS ELF interpreter plugin. It intercepts PT_INTERP_NIX + headers to resolve relative and $ORIGIN interpreter paths. + config COMPAT_BINFMT_ELF def_bool y depends on COMPAT && BINFMT_ELFfs/Makefile +1
-0@@ -35,6 +35,7 @@ obj-$(CONFIG_FILE_LOCKING) += locks.o obj-$(CONFIG_BINFMT_MISC) += binfmt_misc.o obj-$(CONFIG_BINFMT_SCRIPT) += binfmt_script.o obj-$(CONFIG_BINFMT_ELF) += binfmt_elf.o +obj-$(CONFIG_BINFMT_ELF_NIX) += binfmt_elf_nix.o obj-$(CONFIG_COMPAT_BINFMT_ELF) += compat_binfmt_elf.o obj-$(CONFIG_BINFMT_ELF_FDPIC) += binfmt_elf_fdpic.o obj-$(CONFIG_BINFMT_FLAT) += binfmt_flat.ofs/binfmt_elf.c +10
-0@@ -35,6 +35,7 @@#include <linux/random.h> #include <linux/elf.h> #include <linux/elf-randomize.h> +#include <linux/elf_plugins.h> #include <linux/utsname.h> #include <linux/coredump.h> #include <linux/sched.h> @@ -870,6 +871,12 @@ static int load_elf_binary(struct linux_binprm *bprm)if (!elf_phdata) goto out; + interpreter =elf_plugin_open_interpreter (bprm, elf_ex, elf_phdata); +if (IS_ERR (interpreter)) { + retval =PTR_ERR (interpreter); + goto out_free_ph; + } + elf_ppnt = elf_phdata;for (i =0 ; i < elf_ex->e_phnum; i++, elf_ppnt++) {char *elf_interpreter;@@ -882,6 +889,9 @@ static int load_elf_binary(struct linux_binprm *bprm)if (elf_ppnt->p_type !=PT_INTERP )continue ; +if (interpreter) +continue ; +/* fs/binfmt_elf.c +14
-0* This is the program interpreter used for shared libraries - * for now assume that this is an a.out format binary. @@ -935,6 +945,20 @@ static int load_elf_binary(struct linux_binprm *bprm) goto out_free_ph; } +if (interpreter && !interp_elf_ex) { + interp_elf_ex =kmalloc_obj (*interp_elf_ex); +if (!interp_elf_ex) { + retval = -ENOMEM ; + goto out_free_file; + } + +/* Get the exec headers */ + retval =elf_read (interpreter, interp_elf_ex, +sizeof (*interp_elf_ex),0 ); +if (retval <0 ) + goto out_free_dentry; + } + elf_ppnt = elf_phdata;for (i =0 ; i < elf_ex->e_phnum; i++, elf_ppnt++)switch (elf_ppnt->p_type) {fs/binfmt_elf_nix.c new file +108
-0@@ -0,0 +1,108 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include <linux/module.h> +#include <linux/kernel.h> +#include <linux/init.h> +#include <linux/fs.h> +#include <linux/path.h> +#include <linux/namei.h> +#include <linux/elf.h> +#include <linux/elf_plugins.h> +#include <linux/slab.h> + +MODULE_DESCRIPTION ("ELF Interpreter plugin for NixOS / $ORIGIN" ); +MODULE_AUTHOR ("Farid Zakaria" ); +MODULE_LICENSE ("GPL" ); + +/* Mnemonic value for NixOS-specific program interpreter: 'N', 'I', 'X', 3 */ +#define PT_INTERP_NIX (PT_LOOS + 0x4e49583) + +static struct file *nix_open_interpreter (struct linux_binprm *bprm, +struct elfhdr *elf_ex, +struct elf_phdr *elf_phdata) +{ +struct elf_phdr *elf_ppnt; +struct file *interpreter =NULL ; +char *elf_interpreter =NULL ; +int i, retval; + +/* Find the custom Nix interpreter header */ + elf_ppnt = elf_phdata; +for (i =0 ; i < elf_ex->e_phnum; i++, elf_ppnt++) { +if (elf_ppnt->p_type ==PT_INTERP_NIX ) +break ; + } + +if (i == elf_ex->e_phnum) +return NULL ;/* Segment not present; fall back to others */ + +/* Security check: refuse relative interp resolution on secure execution */ +if (bprm->secureexec) { +pr_warn_once ("binfmt_elf_nix: secureexec active, refusing custom interpreter lookup\n" ); +return NULL ;/* Fallback to standard PT_INTERP */ + } + +if (elf_ppnt->p_filesz >PATH_MAX || elf_ppnt->p_filesz <2 ) +return ERR_PTR (-ENOEXEC ); + + elf_interpreter =kmalloc (elf_ppnt->p_filesz,GFP_KERNEL ); +if (!elf_interpreter) +return ERR_PTR (-ENOMEM ); + +/* Read the interpreter path from the executable file */ + retval =kernel_read (bprm->file, elf_interpreter, elf_ppnt->p_filesz, &elf_ppnt->p_offset); +if (retval != elf_ppnt->p_filesz) { + retval = (retval <0 ) ? retval : -EIO ; + goto out_free; + } + +if (elf_interpreter[elf_ppnt->p_filesz -1 ] !='\0' ) { + retval = -ENOEXEC ; + goto out_free; + } + +/* Path Resolution: Absolute vs. $ORIGIN */ +if (elf_interpreter[0 ] =='/' ) { + interpreter =open_exec (elf_interpreter); + }else if (strncmp (elf_interpreter,"$ORIGIN/" ,8 ) ==0 ||strncmp (elf_interpreter,"${ORIGIN}/" ,10 ) ==0 ) { +const char *rel_path = (elf_interpreter[0 ] =='$' ) ? (elf_interpreter +8 ) : (elf_interpreter +10 ); +struct path parent_path; + +/* Reference parent directory of the executed file safely */ + parent_path.mnt =mntget (bprm->file->f_path.mnt); + parent_path.dentry =dget_parent (bprm->file->f_path.dentry); + +/* Open relative to parent directory */ + interpreter =file_open_root (&parent_path, rel_path,O_RDONLY ,0 ); + +path_put (&parent_path); + }else { +/* Naked relative paths are rejected for safety */ + retval = -ENOEXEC ; + goto out_free; + } + +kfree (elf_interpreter); +return interpreter; + +out_free: +kfree (elf_interpreter); +return ERR_PTR (retval); +} + +static struct elf_plugin nix_elf_plugin = { + .owner =THIS_MODULE , + .open_interpreter = nix_open_interpreter, +}; + +static int __init binfmt_elf_nix_init (void ) +{ +return register_elf_plugin (&nix_elf_plugin); +} + +static void __exit binfmt_elf_nix_exit(void) +{ +unregister_elf_plugin (&nix_elf_plugin); +} + +module_init (binfmt_elf_nix_init); +module_exit (binfmt_elf_nix_exit);fs/exec.c +26
-0@@ -46,6 +46,7 @@#include <linux/key.h> #include <linux/personality.h> #include <linux/binfmts.h> +#include <linux/elf_plugins.h> #include <linux/utsname.h> #include <linux/pid_namespace.h> #include <linux/module.h> @@ -108,6 +109,52 @@ void unregister_binfmt(struct linux_binfmt * fmt)EXPORT_SYMBOL (unregister_binfmt); +#if IS_ENABLED (CONFIG_BINFMT_ELF_PLUGINS ) +static DEFINE_MUTEX (elf_plugins_lock); +static LIST_HEAD (elf_plugins); + +int register_elf_plugin (struct elf_plugin *plugin) +{ +mutex_lock (&elf_plugins_lock); +list_add_tail (&plugin->list, &elf_plugins); +mutex_unlock (&elf_plugins_lock); +return 0 ; +} +EXPORT_SYMBOL_GPL (register_elf_plugin); + +void unregister_elf_plugin (struct elf_plugin *plugin) +{ +mutex_lock (&elf_plugins_lock); +list_del (&plugin->list); +mutex_unlock (&elf_plugins_lock); +} +EXPORT_SYMBOL_GPL (unregister_elf_plugin); + +struct file *elf_plugin_open_interpreter (struct linux_binprm *bprm, +struct elfhdr *elf_ex, +struct elf_phdr *elf_phdata) +{
fs/exec.c void unregister_binfmt(struct linux_binfmt * fmt) · line 146
Quoted 10 lines
fs/exec.c +10
-0+struct elf_plugin *plugin; +struct file *file =NULL ; + +mutex_lock (&elf_plugins_lock); +list_for_each_entry (plugin, &elf_plugins, list) { +if (!try_module_get (plugin->owner)) +continue ; +mutex_unlock (&elf_plugins_lock); + + file = plugin->open_interpreter (bprm, elf_ex, elf_phdata);
I have to say I do not like this at all because it also means you need actual kernel modules for custom binaries. Yeech.
I think you should extend binfmt_misc for that, combining it with bpf. When binfmt_misc selects the interpreter a bpf program is run. That bpf program is passed the necessary information to make a decision. It can check whether it's in the right sandbox. Then checks if the interpreter starts with the magic ORIGIN string or whatever. Once it has determined that a binfmt_misc entry applies things progress as usual. Then you can point binfmt_misc at a static binary that finds the right loader to use.
Signature · 1 line
Christian Brauner <[email protected]>
Re: [RFC PATCH] fs: introduce pluggable ELF interpreter loader registry
Quoted 279 lines · Christian Brauner
Quoted 279 lines
Introduce a pluggable framework for ELF binary loading to allow dynamic resolution and redirection of program interpreters (
PT_INTERP). This is primarily designed to support hermetic path resolution like NixOS $ORIGIN relative dynamic linkers without bloating the core ELF loader or compromising system execution security.Introduce a new registration interface for kernel modules to register
open_interpretercallbacks. Standard ELF loading queries this registry; if a plugin resolves a custom segment type (likePT_INTERP_NIX), it returns a file descriptor for the resolved interpreter. Secure execution environments (bprm->secureexec) bypass relative resolution for safety.Signed-off-by Farid Zakaria <[email protected]>fs/Kconfig.binfmt +15
-0@@ -38,6 +38,21 @@ config BINFMT_ELF_KUNIT_TEST only needed for debugging. Note that with CONFIG_COMPAT=y, the compat_binfmt_elf KUnit test is also created. +config BINFMT_ELF_PLUGINS + bool "Enable plugin support for ELF interpreter loading" + depends on BINFMT_ELF + help + This option allows kernel modules to register handlers to dynamically + resolve and override the ELF program interpreter (e.g. supporting relative + interpreter paths with $ORIGIN). + +config BINFMT_ELF_NIX + tristate "ELF interpreter plugin for NixOS ($ORIGIN support)" + depends on BINFMT_ELF_PLUGINS + help + This builds the NixOS ELF interpreter plugin. It intercepts PT_INTERP_NIX + headers to resolve relative and $ORIGIN interpreter paths. + config COMPAT_BINFMT_ELF def_bool y depends on COMPAT && BINFMT_ELFfs/Makefile +1
-0@@ -35,6 +35,7 @@ obj-$(CONFIG_FILE_LOCKING) += locks.o obj-$(CONFIG_BINFMT_MISC) += binfmt_misc.o obj-$(CONFIG_BINFMT_SCRIPT) += binfmt_script.o obj-$(CONFIG_BINFMT_ELF) += binfmt_elf.o +obj-$(CONFIG_BINFMT_ELF_NIX) += binfmt_elf_nix.o obj-$(CONFIG_COMPAT_BINFMT_ELF) += compat_binfmt_elf.o obj-$(CONFIG_BINFMT_ELF_FDPIC) += binfmt_elf_fdpic.o obj-$(CONFIG_BINFMT_FLAT) += binfmt_flat.ofs/binfmt_elf.c +10
-0@@ -35,6 +35,7 @@#include <linux/random.h> #include <linux/elf.h> #include <linux/elf-randomize.h> +#include <linux/elf_plugins.h> #include <linux/utsname.h> #include <linux/coredump.h> #include <linux/sched.h> @@ -870,6 +871,12 @@ static int load_elf_binary(struct linux_binprm *bprm)if (!elf_phdata) goto out; + interpreter =elf_plugin_open_interpreter (bprm, elf_ex, elf_phdata); +if (IS_ERR (interpreter)) { + retval =PTR_ERR (interpreter); + goto out_free_ph; + } + elf_ppnt = elf_phdata;for (i =0 ; i < elf_ex->e_phnum; i++, elf_ppnt++) {char *elf_interpreter;@@ -882,6 +889,9 @@ static int load_elf_binary(struct linux_binprm *bprm)if (elf_ppnt->p_type !=PT_INTERP )continue ; +if (interpreter) +continue ; +/* * This is the program interpreter used for shared libraries -
* for now assume that this is an a.out format binary.fs/binfmt_elf.c +14
-0@@ -935,6 +945,20 @@ static int load_elf_binary(struct linux_binprm *bprm) goto out_free_ph; } +if (interpreter && !interp_elf_ex) { + interp_elf_ex =kmalloc_obj (*interp_elf_ex); +if (!interp_elf_ex) { + retval = -ENOMEM ; + goto out_free_file; + } + +/* Get the exec headers */ + retval =elf_read (interpreter, interp_elf_ex, +sizeof (*interp_elf_ex),0 ); +if (retval <0 ) + goto out_free_dentry; + } + elf_ppnt = elf_phdata;for (i =0 ; i < elf_ex->e_phnum; i++, elf_ppnt++)switch (elf_ppnt->p_type) {fs/binfmt_elf_nix.c new file +108
-0@@ -0,0 +1,108 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include <linux/module.h> +#include <linux/kernel.h> +#include <linux/init.h> +#include <linux/fs.h> +#include <linux/path.h> +#include <linux/namei.h> +#include <linux/elf.h> +#include <linux/elf_plugins.h> +#include <linux/slab.h> + +MODULE_DESCRIPTION ("ELF Interpreter plugin for NixOS / $ORIGIN" ); +MODULE_AUTHOR ("Farid Zakaria" ); +MODULE_LICENSE ("GPL" ); + +/* Mnemonic value for NixOS-specific program interpreter: 'N', 'I', 'X', 3 */ +#define PT_INTERP_NIX (PT_LOOS + 0x4e49583) + +static struct file *nix_open_interpreter (struct linux_binprm *bprm, +struct elfhdr *elf_ex, +struct elf_phdr *elf_phdata) +{ +struct elf_phdr *elf_ppnt; +struct file *interpreter =NULL ; +char *elf_interpreter =NULL ; +int i, retval; + +/* Find the custom Nix interpreter header */ + elf_ppnt = elf_phdata; +for (i =0 ; i < elf_ex->e_phnum; i++, elf_ppnt++) { +if (elf_ppnt->p_type ==PT_INTERP_NIX ) +break ; + } + +if (i == elf_ex->e_phnum) +return NULL ;/* Segment not present; fall back to others */ + +/* Security check: refuse relative interp resolution on secure execution */ +if (bprm->secureexec) { +pr_warn_once ("binfmt_elf_nix: secureexec active, refusing custom interpreter lookup\n" ); +return NULL ;/* Fallback to standard PT_INTERP */ + } + +if (elf_ppnt->p_filesz >PATH_MAX || elf_ppnt->p_filesz <2 ) +return ERR_PTR (-ENOEXEC ); + + elf_interpreter =kmalloc (elf_ppnt->p_filesz,GFP_KERNEL ); +if (!elf_interpreter) +return ERR_PTR (-ENOMEM ); + +/* Read the interpreter path from the executable file */ + retval =kernel_read (bprm->file, elf_interpreter, elf_ppnt->p_filesz, &elf_ppnt->p_offset); +if (retval != elf_ppnt->p_filesz) { + retval = (retval <0 ) ? retval : -EIO ; + goto out_free; + } + +if (elf_interpreter[elf_ppnt->p_filesz -1 ] !='\0' ) { + retval = -ENOEXEC ; + goto out_free; + } + +/* Path Resolution: Absolute vs. $ORIGIN */ +if (elf_interpreter[0 ] =='/' ) { + interpreter =open_exec (elf_interpreter); + }else if (strncmp (elf_interpreter,"$ORIGIN/" ,8 ) ==0 ||strncmp (elf_interpreter,"${ORIGIN}/" ,10 ) ==0 ) { +const char *rel_path = (elf_interpreter[0 ] =='$' ) ? (elf_interpreter +8 ) : (elf_interpreter +10 ); +struct path parent_path; + +/* Reference parent directory of the executed file safely */ + parent_path.mnt =mntget (bprm->file->f_path.mnt); + parent_path.dentry =dget_parent (bprm->file->f_path.dentry); + +/* Open relative to parent directory */ + interpreter =file_open_root (&parent_path, rel_path,O_RDONLY ,0 ); + +path_put (&parent_path); + }else { +/* Naked relative paths are rejected for safety */ + retval = -ENOEXEC ; + goto out_free; + } + +kfree (elf_interpreter); +return interpreter; + +out_free: +kfree (elf_interpreter); +return ERR_PTR (retval); +} + +static struct elf_plugin nix_elf_plugin = { + .owner =THIS_MODULE , + .open_interpreter = nix_open_interpreter, +}; + +static int __init binfmt_elf_nix_init (void ) +{ +return register_elf_plugin (&nix_elf_plugin); +} + +static void __exit binfmt_elf_nix_exit(void) +{ +unregister_elf_plugin (&nix_elf_plugin); +} + +module_init (binfmt_elf_nix_init); +module_exit (binfmt_elf_nix_exit);fs/exec.c +31
-0@@ -46,6 +46,7 @@#include <linux/key.h> #include <linux/personality.h> #include <linux/binfmts.h> +#include <linux/elf_plugins.h> #include <linux/utsname.h> #include <linux/pid_namespace.h> #include <linux/module.h> @@ -108,6 +109,52 @@ void unregister_binfmt(struct linux_binfmt * fmt)EXPORT_SYMBOL (unregister_binfmt); +#if IS_ENABLED (CONFIG_BINFMT_ELF_PLUGINS ) +static DEFINE_MUTEX (elf_plugins_lock); +static LIST_HEAD (elf_plugins); + +int register_elf_plugin (struct elf_plugin *plugin) +{ +mutex_lock (&elf_plugins_lock); +list_add_tail (&plugin->list, &elf_plugins); +mutex_unlock (&elf_plugins_lock); +return 0 ; +} +EXPORT_SYMBOL_GPL (register_elf_plugin); + +void unregister_elf_plugin (struct elf_plugin *plugin) +{ +mutex_lock (&elf_plugins_lock); +list_del (&plugin->list); +mutex_unlock (&elf_plugins_lock); +} +EXPORT_SYMBOL_GPL (unregister_elf_plugin); + +struct file *elf_plugin_open_interpreter (struct linux_binprm *bprm, +struct elfhdr *elf_ex, +struct elf_phdr *elf_phdata) +{ +struct elf_plugin *plugin; +struct file *file =NULL ; + +mutex_lock (&elf_plugins_lock); +list_for_each_entry (plugin, &elf_plugins, list) {
Quoted 9 lines
fs/exec.c void unregister_binfmt(struct linux_binfmt * fmt) · line 146
Quoted 5 lines
fs/exec.c +5
-0+if (!try_module_get (plugin->owner)) +continue ; +mutex_unlock (&elf_plugins_lock); + + file = plugin->open_interpreter (bprm, elf_ex, elf_phdata);I have to say I do not like this at all because it also means you need actual kernel modules for custom binaries. Yeech.
Isn't this already true via binfmt for new binary formats? ELF support in binfmt itself is also built-in from what I understand.
Anyway, I'm not too attached to the idea, I'm just trying to brainstorm on how to get it done.
Quoted 8 lines
I think you should extend
binfmt_miscfor that, combining it with bpf. Whenbinfmt_miscselects the interpreter a bpf program is run. That bpf program is passed the necessary information to make a decision. It can check whether it's in the right sandbox. Then checks if the interpreter starts with the magic ORIGIN string or whatever. Once it has determined that abinfmt_miscentry applies things progress as usual. Then you can pointbinfmt_miscat a static binary that finds the right loader to use.
My audit of this shows it might be doable, but it's definitely confusing, and I'm still not clear it solves the problem -- I will work through an example on my NixOS machine. I think I can patch
https://github.com/nix-community/nix-ld to maybe act as this trampoline for binfmt_misc to read PT_INTERP_NIX. I will try that and respond back if it works conceptually at least to maybe keep the discussion moving forward and to explore the space more.
If binfmt_misc has to point to a wrapper binary, that wrapper must live at a fixed, hardcoded absolute path.
I guess we could bootstrap it for NixOS only but one other problem I
forsee: /proc/self/exe is now something else than expected
The entire goal of relocatability is to allow binaries to run on any host machine without relying on global, hardcoded system paths. Just to demo how Nix is designed to let you copy code from a machine to another with *ideally* no assumptions.
Here is a simple hello world:
#hello.nix
with import <nixpkgs> {};
runCommandCC "hello" {} ''
mkdir -p $out/bin
gcc -x c -o $out/bin/hello - <<\INNER_EOF
#include <stdio.h>
int main() {
printf("Hello, World!\n");
return 0;
}INNER_EOF
''
We can then build it:
Quoted 1 line
nix build -f hello.nix
My machine uses the default /nix/store prefix, so the interpreter is set to there.
Quoted 1 line
patchelf --print-interpreter ./result/bin/hello
/nix/store/7nbi22pcc92y2fqbkyp7h3srvvklmckb-glibc-2.40-224/lib/ld-linux-x86-64.so.2
If I were to use a machine with a different prefix /home/fmzakari/.local/nix then I couldn't "substitute" (download from the cache), this binary.
I would have to build it and the whole graph down to our bootstrap seeds with the new store prefix.
Nix then let's us then query the graph and copy it, including ld.so and glibc to any other machine to ideally run.
Quoted 1 line
nix-store --query --tree $(realpath result)
/nix/store/adbwpmzwgdcmk5qcskyqwygmxbkffb32-hello
├───/nix/store/7nbi22pcc92y2fqbkyp7h3srvvklmckb-glibc-2.40-224 │ ├───/nix/store/fv5lgysa3hmf3l3dkkpwvndcg6xwhy8m-xgcc-14.3.0-libgcc │ ├───/nix/store/qywg7bxskvihq62ms2g51fkzkrdnyfkh-libidn2-2.3.8 │ │ ├───/nix/store/hjwppd89fk8781xl4r35xqlddwqi5f66-libunistring-1.4.1 │ │ │ └───/nix/store/hjwppd89fk8781xl4r35xqlddwqi5f66-libunistring-1.4.1 [...] │ │ └───/nix/store/qywg7bxskvihq62ms2g51fkzkrdnyfkh-libidn2-2.3.8 [...] │ └───/nix/store/7nbi22pcc92y2fqbkyp7h3srvvklmckb-glibc-2.40-224 [...] ├───/nix/store/ybp235ps7m4yd85v0pgvqkhd4xmxf6jq-gcc-14.3.0-lib │ ├───/nix/store/7nbi22pcc92y2fqbkyp7h3srvvklmckb-glibc-2.40-224 [...] │ ├───/nix/store/phpq5h7cp6y1w84aysy29548irqy5dd9-gcc-14.3.0-libgcc │ └───/nix/store/ybp235ps7m4yd85v0pgvqkhd4xmxf6jq-gcc-14.3.0-lib [...] └───/nix/store/adbwpmzwgdcmk5qcskyqwygmxbkffb32-hello [...]
I appreciate you engaging me on this topic -- thank you.
Quoted 2 lines
--
Christian Brauner <[email protected]>
Christian Brauner
maintainerRe: [RFC PATCH] fs: introduce pluggable ELF interpreter loader registry
Quoted 340 lines · Farid Zakaria
Quoted 279 lines · Christian Brauner
Quoted 279 lines
Introduce a pluggable framework for ELF binary loading to allow dynamic resolution and redirection of program interpreters (
PT_INTERP). This is primarily designed to support hermetic path resolution like NixOS $ORIGIN relative dynamic linkers without bloating the core ELF loader or compromising system execution security.Introduce a new registration interface for kernel modules to register
open_interpretercallbacks. Standard ELF loading queries this registry; if a plugin resolves a custom segment type (likePT_INTERP_NIX), it returns a file descriptor for the resolved interpreter. Secure execution environments (bprm->secureexec) bypass relative resolution for safety.Signed-off-by Farid Zakaria <[email protected]>fs/Kconfig.binfmt +15
-0@@ -38,6 +38,21 @@ config BINFMT_ELF_KUNIT_TEST only needed for debugging. Note that with CONFIG_COMPAT=y, the compat_binfmt_elf KUnit test is also created. +config BINFMT_ELF_PLUGINS + bool "Enable plugin support for ELF interpreter loading" + depends on BINFMT_ELF + help + This option allows kernel modules to register handlers to dynamically + resolve and override the ELF program interpreter (e.g. supporting relative + interpreter paths with $ORIGIN). + +config BINFMT_ELF_NIX + tristate "ELF interpreter plugin for NixOS ($ORIGIN support)" + depends on BINFMT_ELF_PLUGINS + help + This builds the NixOS ELF interpreter plugin. It intercepts PT_INTERP_NIX + headers to resolve relative and $ORIGIN interpreter paths. + config COMPAT_BINFMT_ELF def_bool y depends on COMPAT && BINFMT_ELFfs/Makefile +1
-0@@ -35,6 +35,7 @@ obj-$(CONFIG_FILE_LOCKING) += locks.o obj-$(CONFIG_BINFMT_MISC) += binfmt_misc.o obj-$(CONFIG_BINFMT_SCRIPT) += binfmt_script.o obj-$(CONFIG_BINFMT_ELF) += binfmt_elf.o +obj-$(CONFIG_BINFMT_ELF_NIX) += binfmt_elf_nix.o obj-$(CONFIG_COMPAT_BINFMT_ELF) += compat_binfmt_elf.o obj-$(CONFIG_BINFMT_ELF_FDPIC) += binfmt_elf_fdpic.o obj-$(CONFIG_BINFMT_FLAT) += binfmt_flat.ofs/binfmt_elf.c +10
-0@@ -35,6 +35,7 @@#include <linux/random.h> #include <linux/elf.h> #include <linux/elf-randomize.h> +#include <linux/elf_plugins.h> #include <linux/utsname.h> #include <linux/coredump.h> #include <linux/sched.h> @@ -870,6 +871,12 @@ static int load_elf_binary(struct linux_binprm *bprm)if (!elf_phdata) goto out; + interpreter =elf_plugin_open_interpreter (bprm, elf_ex, elf_phdata); +if (IS_ERR (interpreter)) { + retval =PTR_ERR (interpreter); + goto out_free_ph; + } + elf_ppnt = elf_phdata;for (i =0 ; i < elf_ex->e_phnum; i++, elf_ppnt++) {char *elf_interpreter;@@ -882,6 +889,9 @@ static int load_elf_binary(struct linux_binprm *bprm)if (elf_ppnt->p_type !=PT_INTERP )continue ; +if (interpreter) +continue ; +/* * This is the program interpreter used for shared libraries -
* for now assume that this is an a.out format binary.fs/binfmt_elf.c +14
-0@@ -935,6 +945,20 @@ static int load_elf_binary(struct linux_binprm *bprm) goto out_free_ph; } +if (interpreter && !interp_elf_ex) { + interp_elf_ex =kmalloc_obj (*interp_elf_ex); +if (!interp_elf_ex) { + retval = -ENOMEM ; + goto out_free_file; + } + +/* Get the exec headers */ + retval =elf_read (interpreter, interp_elf_ex, +sizeof (*interp_elf_ex),0 ); +if (retval <0 ) + goto out_free_dentry; + } + elf_ppnt = elf_phdata;for (i =0 ; i < elf_ex->e_phnum; i++, elf_ppnt++)switch (elf_ppnt->p_type) {fs/binfmt_elf_nix.c new file +108
-0@@ -0,0 +1,108 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include <linux/module.h> +#include <linux/kernel.h> +#include <linux/init.h> +#include <linux/fs.h> +#include <linux/path.h> +#include <linux/namei.h> +#include <linux/elf.h> +#include <linux/elf_plugins.h> +#include <linux/slab.h> + +MODULE_DESCRIPTION ("ELF Interpreter plugin for NixOS / $ORIGIN" ); +MODULE_AUTHOR ("Farid Zakaria" ); +MODULE_LICENSE ("GPL" ); + +/* Mnemonic value for NixOS-specific program interpreter: 'N', 'I', 'X', 3 */ +#define PT_INTERP_NIX (PT_LOOS + 0x4e49583) + +static struct file *nix_open_interpreter (struct linux_binprm *bprm, +struct elfhdr *elf_ex, +struct elf_phdr *elf_phdata) +{ +struct elf_phdr *elf_ppnt; +struct file *interpreter =NULL ; +char *elf_interpreter =NULL ; +int i, retval; + +/* Find the custom Nix interpreter header */ + elf_ppnt = elf_phdata; +for (i =0 ; i < elf_ex->e_phnum; i++, elf_ppnt++) { +if (elf_ppnt->p_type ==PT_INTERP_NIX ) +break ; + } + +if (i == elf_ex->e_phnum) +return NULL ;/* Segment not present; fall back to others */ + +/* Security check: refuse relative interp resolution on secure execution */ +if (bprm->secureexec) { +pr_warn_once ("binfmt_elf_nix: secureexec active, refusing custom interpreter lookup\n" ); +return NULL ;/* Fallback to standard PT_INTERP */ + } + +if (elf_ppnt->p_filesz >PATH_MAX || elf_ppnt->p_filesz <2 ) +return ERR_PTR (-ENOEXEC ); + + elf_interpreter =kmalloc (elf_ppnt->p_filesz,GFP_KERNEL ); +if (!elf_interpreter) +return ERR_PTR (-ENOMEM ); + +/* Read the interpreter path from the executable file */ + retval =kernel_read (bprm->file, elf_interpreter, elf_ppnt->p_filesz, &elf_ppnt->p_offset); +if (retval != elf_ppnt->p_filesz) { + retval = (retval <0 ) ? retval : -EIO ; + goto out_free; + } + +if (elf_interpreter[elf_ppnt->p_filesz -1 ] !='\0' ) { + retval = -ENOEXEC ; + goto out_free; + } + +/* Path Resolution: Absolute vs. $ORIGIN */ +if (elf_interpreter[0 ] =='/' ) { + interpreter =open_exec (elf_interpreter); + }else if (strncmp (elf_interpreter,"$ORIGIN/" ,8 ) ==0 ||strncmp (elf_interpreter,"${ORIGIN}/" ,10 ) ==0 ) { +const char *rel_path = (elf_interpreter[0 ] =='$' ) ? (elf_interpreter +8 ) : (elf_interpreter +10 ); +struct path parent_path; + +/* Reference parent directory of the executed file safely */ + parent_path.mnt =mntget (bprm->file->f_path.mnt); + parent_path.dentry =dget_parent (bprm->file->f_path.dentry); + +/* Open relative to parent directory */ + interpreter =file_open_root (&parent_path, rel_path,O_RDONLY ,0 ); + +path_put (&parent_path); + }else { +/* Naked relative paths are rejected for safety */ + retval = -ENOEXEC ; + goto out_free; + } + +kfree (elf_interpreter); +return interpreter; + +out_free: +kfree (elf_interpreter); +return ERR_PTR (retval); +} + +static struct elf_plugin nix_elf_plugin = { + .owner =THIS_MODULE , + .open_interpreter = nix_open_interpreter, +}; + +static int __init binfmt_elf_nix_init (void ) +{ +return register_elf_plugin (&nix_elf_plugin); +} + +static void __exit binfmt_elf_nix_exit(void) +{ +unregister_elf_plugin (&nix_elf_plugin); +} + +module_init (binfmt_elf_nix_init); +module_exit (binfmt_elf_nix_exit);fs/exec.c +31
-0@@ -46,6 +46,7 @@#include <linux/key.h> #include <linux/personality.h> #include <linux/binfmts.h> +#include <linux/elf_plugins.h> #include <linux/utsname.h> #include <linux/pid_namespace.h> #include <linux/module.h> @@ -108,6 +109,52 @@ void unregister_binfmt(struct linux_binfmt * fmt)EXPORT_SYMBOL (unregister_binfmt); +#if IS_ENABLED (CONFIG_BINFMT_ELF_PLUGINS ) +static DEFINE_MUTEX (elf_plugins_lock); +static LIST_HEAD (elf_plugins); + +int register_elf_plugin (struct elf_plugin *plugin) +{ +mutex_lock (&elf_plugins_lock); +list_add_tail (&plugin->list, &elf_plugins); +mutex_unlock (&elf_plugins_lock); +return 0 ; +} +EXPORT_SYMBOL_GPL (register_elf_plugin); + +void unregister_elf_plugin (struct elf_plugin *plugin) +{ +mutex_lock (&elf_plugins_lock); +list_del (&plugin->list); +mutex_unlock (&elf_plugins_lock); +} +EXPORT_SYMBOL_GPL (unregister_elf_plugin); + +struct file *elf_plugin_open_interpreter (struct linux_binprm *bprm, +struct elfhdr *elf_ex, +struct elf_phdr *elf_phdata) +{ +struct elf_plugin *plugin; +struct file *file =NULL ; + +mutex_lock (&elf_plugins_lock); +list_for_each_entry (plugin, &elf_plugins, list) {Quoted 9 lines
fs/exec.c void unregister_binfmt(struct linux_binfmt * fmt) · line 146
Quoted 5 lines
fs/exec.c +5
-0+if (!try_module_get (plugin->owner)) +continue ; +mutex_unlock (&elf_plugins_lock); + + file = plugin->open_interpreter (bprm, elf_ex, elf_phdata);I have to say I do not like this at all because it also means you need actual kernel modules for custom binaries. Yeech.
Isn't this already true via binfmt for new binary formats? ELF support in binfmt itself is also built-in from what I understand.
Anyway, I'm not too attached to the idea, I'm just trying to brainstorm on how to get it done.
Quoted 8 lines
I think you should extend
binfmt_miscfor that, combining it with bpf. Whenbinfmt_miscselects the interpreter a bpf program is run. That bpf program is passed the necessary information to make a decision. It can check whether it's in the right sandbox. Then checks if the interpreter starts with the magic ORIGIN string or whatever. Once it has determined that abinfmt_miscentry applies things progress as usual. Then you can pointbinfmt_miscat a static binary that finds the right loader to use.My audit of this shows it might be doable, but it's definitely confusing, and I'm still not clear it solves the problem -- I will work through an example on my NixOS machine. I think I can patch
https://github.com/nix-community/nix-ld to maybe act as this trampoline forbinfmt_miscto readPT_INTERP_NIX. I will try that and respond back if it works conceptually at least to maybe keep the discussion moving forward and to explore the space more.If
binfmt_mischas to point to a wrapper binary, that wrapper must live at a fixed, hardcoded absolute path.
I guess we could bootstrap it for NixOS only but one other problem I
forsee:/proc/self/exeis now something else than expectedThe entire goal of relocatability is to allow binaries to run on any host machine without relying on global, hardcoded system paths. Just to demo how Nix is designed to let you copy code from a machine to another with *ideally* no assumptions.
Here is a simple hello world:#hello.nix
with import <nixpkgs> {};
runCommandCC "hello" {} ''
mkdir -p $out/bin
gcc -x c -o $out/bin/hello - <<\INNER_EOF
#include <stdio.h>int main() { printf("Hello, World!\n"); return 0; }
INNER_EOF
''We can then build it:
Quoted 1 line
nix build -f hello.nix
Quoted 8 lines
My machine uses the default /nix/store prefix, so the interpreter is set to there.
Quoted 1 line
patchelf --print-interpreter ./result/bin/hello
/nix/store/7nbi22pcc92y2fqbkyp7h3srvvklmckb-glibc-2.40-224/lib/ld-linux-x86-64.so.2
If I were to use a machine with a different prefix
/home/fmzakari/.local/nixthen I couldn't "substitute" (download from the cache), this binary.
This was just an example. The bpf program can calculate a path however it sees fit including making use of ORIGIN or whatever.
RFC PATCH 1/1 fs: binfmt_misc: introduce eBPF-based matching and interpreter selection
Historically binfmt_misc has only matched binaries by magic bytes or file extension, and always redirects a match to a fixed interpreter recorded in the registration. This is insufficient when the match requires parsing the binary header (e.g. inspecting ELF program headers) or when the interpreter must be computed per-binary rather than hard-coded.
Introduce a new 'B' (BPF) handler type. A pinned SOCKET_FILTER program is registered in place of the magic/mask, and no interpreter is recorded:
echo ':name:B:<bpf_pinned_path>::::<flags>' \
> /proc/sys/fs/binfmt_misc/registerWhen a binary is executed, binfmt_misc runs the program with the BINPRM_BUF_SIZE file-header buffer as context. Returning 1 selects the handler; returning 0 falls through to the remaining handlers.
Unlike magic/extension handlers, a 'B' handler carries no interpreter of its
own: the program chooses it via a new helper, bpf_binprm_set_interp(). This lets the program compute the interpreter path however it sees fit (for example relative to the binary). A 'B' handler is therefore a strict superset of the existing magic handlers -- any of them can be expressed as a program that matches on the header and sets a fixed interpreter.
bpf_binprm_set_interp() is exposed to SOCKET_FILTER programs and stashes the chosen path on a per-CPU area that binfmt_misc reads back immediately after the run under migrate_disable(); only a match that set an interpreter allocates.
---
Hey Christian,
Thank you for the suggestion and discussion on the mailing list. I took a stab at your idea of driving binfmt_misc interpreter selection from a BPF program (hopefully this is what you had in mind). The other approach was also to use a static interpreter during registration but this current approach feels right.
This prototype demonstrates binfmt_misc interpreter selection from a BPF program so the interpreter can be *computed* (e.g. $ORIGIN-relative) rather than recorded statically. It replaces my earlier "pluggable ELF interpreter loader registry" RFC.
As this is a first draft, a few things you probably will have notes on. I tried my best but I am a novice here so I welcome your feedback.
* The prototype uses socket filter type. This seemed the easiest to wire up to start. My guess is it should be a dedicated type (eg. BPF_PROG_TYPE_BINFMT).
* bpf_binprm_set_interp() is a "classic helper", but that list
is frozen in favour of kfuncs. A kfunc needs BTF and can't be called from the raw-bytecode selftest I wrote.
* The program only gets bprm->buf today.
That is enough to match on the ELF header but not to compute an $ORIGIN-relative path from the binary's location, which needs bprm->filename
* I have some selftests that exercises match + program-chosen interpreter. I don't include them here yet since it seems unnecessary when discussing the idea.
To demo the functionality, I thought it would be neat to see how this is a superset and can replace all the qemu binfmt usage [1].
$ ./hello-aarch64; -bash: /etc/binfmt-demo/hello-aarch64: cannot execute binary file: Exec format error
$ bpftool prog load filter.bpf.o /sys/fs/bpf/qemu type socket $ echo ':qemu-bpf:B:/sys/fs/bpf/qemu::::P' > /proc/sys/fs/binfmt_misc/register
$ ./hello-aarch64 AARCH64_RAN_VIA_QEMU
All the per-CPU guards were recommended by AI.
[1] https://gist.github.com/fzakaria/bef27d2e21b0e36ffccda1cbf417b636
fs/binfmt_misc.c | 181 ++++++++++++++++++++++++++++++++++++--- include/linux/bpf.h | 1 + include/uapi/linux/bpf.h | 1 + net/core/filter.c | 8 ++ 4 files changed, 180 insertions(+), 11 deletions(-)
fs/binfmt_misc.c +170 -11
@@ -29,6 +29,11 @@#include <linux/fs.h> #include <linux/uaccess.h> +#ifdef CONFIG_BPF_SYSCALL +#include <linux/bpf.h> +#include <linux/filter.h> +#endif +#include "internal.h" #ifdef DEBUG
@@ -41,12 +46,14 @@ enum {VERBOSE_STATUS =1 /* make it zero to save 400 bytes kernel memory */ }; -enum {Enabled, Magic}; +enum {Enabled, Magic, Bpf};#define MISC_FMT_PRESERVE_ARGV0 (1UL << 31)#define MISC_FMT_OPEN_BINARY (1UL << 30)#define MISC_FMT_CREDENTIALS (1UL << 29)#define MISC_FMT_OPEN_FILE (1UL << 28) +struct bpf_prog ; + typedef struct { struct list_head list; unsigned long flags; /* type, status, etc. */
@@ -59,6 +66,9 @@ typedef struct {struct dentry *dentry;struct file *interp_file;refcount_t users;/* sync removal with load_misc_binary() */ +#ifdef CONFIG_BPF_SYSCALL +struct bpf_prog *bpf_prog; +#endif } Node;static struct file_system_type bm_fs_type;
@@ -78,10 +88,51 @@ static struct file_system_type bm_fs_type;*/ #define MAX_REGISTER_LENGTH 1920 +#ifdef CONFIG_BPF_SYSCALL +struct binfmt_bpf_interp { +char path[PATH_MAX ]; +int len;/* < 0 if the current program set no interpreter */ +}; +static DEFINE_PER_CPU(struct binfmt_bpf_interp, binfmt_bpf_interp); + +/* +* bpf_binprm_set_interp - let a binfmt_misc 'B' program pick the interpreter. +* @path: interpreter path, in BPF-accessible memory +* @len: number of bytes in @path +* +* The program computes the interpreter path however it sees fit (e.g. relative +* to the binary). The path is stashed on a per-CPU area that binfmt_misc reads +* back immediately after running the program under migrate_disable(), so it +* cannot race with another CPU. +*/ +BPF_CALL_2(bpf_binprm_set_interp, const char *, path, u32, len) +{ +struct binfmt_bpf_interp *sc =this_cpu_ptr (&binfmt_bpf_interp); + +if (len ==0 || len >=PATH_MAX ) +return -EINVAL ; +/* @path is @len bytes of BPF memory and is not NUL-terminated. */ +memcpy (sc->path, path, len); + sc->path[len] ='\0' ; + sc->len = len; +return 0 ; +} + +const struct bpf_func_proto bpf_binprm_set_interp_proto = { + .func = bpf_binprm_set_interp, + .gpl_only = false, + .ret_type =RET_INTEGER , + .arg1_type =ARG_PTR_TO_MEM |MEM_RDONLY , + .arg2_type =ARG_CONST_SIZE , +}; +#endif /* CONFIG_BPF_SYSCALL */ +/** * search_binfmt_handler - search for a binary handler for @bprm * @misc: handle to binfmt_misc instance * @bprm: binary for which we are looking for a handler +* @bpf_interp: receives a kmalloc'd interpreter path if a 'B' program chooses +* one via bpf_binprm_set_interp(); the caller must kfree() it * * Search for a binary type handler for @bprm in the list of registered binary * type handlers.
@@ -89,7 +140,8 @@ static struct file_system_type bm_fs_type;* Return: binary type list entry on success, NULL on failure */ static Node *search_binfmt_handler (struct binfmt_misc *misc, -struct linux_binprm *bprm) +struct linux_binprm *bprm, +char **bpf_interp) {char *p =strrchr (bprm->interp,'.' );Node *e;
@@ -103,6 +155,37 @@ static Node *search_binfmt_handler(struct binfmt_misc *misc,if (!test_bit (Enabled, &e->flags))continue ; +/* Do matching based on BPF if applicable. */ +if (test_bit (Bpf, &e->flags)) { +#ifdef CONFIG_BPF_SYSCALL +if (e->bpf_prog) { +struct binfmt_bpf_interp *sc; +u32 ret; + +migrate_disable (); + sc =this_cpu_ptr (&binfmt_bpf_interp); + sc->len =-1 ; + +rcu_read_lock (); + ret =bpf_prog_run (e->bpf_prog, bprm->buf); +rcu_read_unlock (); + +if (ret ==1 && sc->len >0 ) + *bpf_interp =kmemdup_nul (sc->path, + sc->len, +GFP_ATOMIC ); +migrate_enable (); + +pr_debug ("binfmt_misc: ran BPF program for %s, ret = %u\n" , + bprm->filename, ret); + +if (ret ==1 ) +return e; + } +#endif +continue ; + } +/* Do matching based on extension if applicable. */ if (!test_bit (Magic, &e->flags)) {if (p && !strcmp (e->magic, p +1 ))
@@ -139,12 +222,13 @@ static Node *search_binfmt_handler(struct binfmt_misc *misc,* Return: binary type list entry on success, NULL on failure */ static Node *get_binfmt_handler (struct binfmt_misc *misc, -struct linux_binprm *bprm) +struct linux_binprm *bprm, +char **bpf_interp) {Node *e;read_lock (&misc->entries_lock); - e =search_binfmt_handler (misc, bprm); + e =search_binfmt_handler (misc, bprm, bpf_interp);if (e)refcount_inc (&e->users);read_unlock (&misc->entries_lock);
@@ -164,6 +248,10 @@ static void put_binfmt_handler(Node *e)if (refcount_dec_and_test (&e->users)) {if (e->flags &MISC_FMT_OPEN_FILE )filp_close (e->interp_file,NULL ); +#ifdef CONFIG_BPF_SYSCALL +if (test_bit (Bpf, &e->flags) && e->bpf_prog) +bpf_prog_put (e->bpf_prog); +#endif kfree (e); } }
@@ -206,15 +294,27 @@ static int load_misc_binary(struct linux_binprm *bprm)struct file *interp_file =NULL ;int retval = -ENOEXEC ;struct binfmt_misc *misc; + char *bpf_interp __free(kfree) = NULL; +const char *interpreter; misc =load_binfmt_misc ();if (!misc->enabled)return retval; - fmt =get_binfmt_handler (misc, bprm); + fmt =get_binfmt_handler (misc, bprm, &bpf_interp);if (!fmt)return retval; +/* +* A 'B' (BPF) handler carries no interpreter of its own; the program +* chooses it via bpf_binprm_set_interp(). Other handlers use the +* interpreter recorded at registration. +*/ + interpreter = bpf_interp ? bpf_interp : fmt->interpreter; + retval = -ENOEXEC ; +if (!interpreter[0 ]) + goto ret; +/* Need to be able to load the file after exec */ retval = -ENOENT ;if (bprm->interp_flags &BINPRM_FLAGS_PATH_INACCESSIBLE )
@@ -238,22 +338,27 @@ static int load_misc_binary(struct linux_binprm *bprm) bprm->argc++;/* add the interp as argv[0] */ - retval =copy_string_kernel (fmt->interpreter, bprm); + retval =copy_string_kernel (interpreter, bprm);if (retval <0 ) goto ret; bprm->argc++;/* Update interp in case binfmt_script needs it. */ - retval =bprm_change_interp (fmt->interpreter, bprm); + retval =bprm_change_interp (interpreter, bprm);if (retval <0 ) goto ret; -if (fmt->flags &MISC_FMT_OPEN_FILE ) { +/* +* The pre-opened interp_file (MISC_FMT_OPEN_FILE / 'F' flag) only +* applies to the statically registered interpreter; a program-supplied +* path is opened here. +*/ +if ((fmt->flags &MISC_FMT_OPEN_FILE ) && !bpf_interp) { interp_file =file_clone_open (fmt->interp_file);if (!IS_ERR (interp_file))deny_write_access (interp_file); }else { - interp_file =open_exec (fmt->interpreter); + interp_file =open_exec (interpreter); } retval =PTR_ERR (interp_file);if (IS_ERR (interp_file))
@@ -404,6 +509,10 @@ static Node *create_entry(const char __user *buffer, size_t count)pr_debug ("register: type: M (magic)\n" ); e->flags = (1 << Enabled) | (1 << Magic);break ; +case 'B' : +pr_debug ("register: type: B (bpf)\n" ); + e->flags = (1 << Enabled) | (1 << Bpf); +break ;default : goto einval; }
@@ -492,6 +601,45 @@ static Node *create_entry(const char __user *buffer, size_t count) } } } + }else if (test_bit (Bpf, &e->flags)) { +/* Handle the 'B' (BPF) format. */ +char *s; + +/* The offset field actually holds the pinned BPF program path */ + s =strchr (p, del); +if (!s) + goto einval; + *s++ ='\0' ; + e->magic = p;/* Keep path in e->magic */ +pr_debug ("register: bpf program path: %s\n" , e->magic); + +#ifdef CONFIG_BPF_SYSCALL + e->bpf_prog =bpf_prog_get_type_path (e->magic,BPF_PROG_TYPE_SOCKET_FILTER ); +if (IS_ERR (e->bpf_prog)) { + err =PTR_ERR (e->bpf_prog); + e->bpf_prog =NULL ; +kfree (e); +return ERR_PTR (err); + } +#else + goto einval; +#endif + + p = s; + +/* The magic field is unused, must be empty */ + s =strchr (p, del); +if (!s || p != s) + goto einval; + *s++ ='\0' ; + p = s; + +/* The mask field is unused, must be empty */ + s =strchr (p, del); +if (!s || p != s) + goto einval; + *s++ ='\0' ; + p = s; }else {/* Handle the 'E' (extension) format. */
@@ -524,8 +672,17 @@ static Node *create_entry(const char __user *buffer, size_t count)if (!p) goto einval; *p++ ='\0' ; -if (!e->interpreter[0 ]) +if (test_bit (Bpf, &e->flags)) { +/* +* A 'B' (BPF) handler carries no interpreter of its own; the +* program picks it via bpf_binprm_set_interp(). Reject a +* statically registered one. +*/ +if (e->interpreter[0 ]) + goto einval; + }else if (!e->interpreter[0 ]) { goto einval; + }pr_debug ("register: interpreter: {%s}\n" , e->interpreter);/* Parse the 'flags' field. */
@@ -602,7 +759,9 @@ static void entry_status(Node *e, char *page) *dp++ ='F' ; *dp++ ='\n' ; -if (!test_bit (Magic, &e->flags)) { +if (test_bit (Bpf, &e->flags)) { +sprintf (dp,"bpf %s\n" , e->magic); + }else if (!test_bit (Magic, &e->flags)) {sprintf (dp,"extension .%s\n" , e->magic); }else { dp +=sprintf (dp,"offset %i\nmagic " , e->offset);
include/linux/bpf.h +1 -0
@@ -3874,6 +3874,7 @@ extern const struct bpf_func_proto bpf_ktime_get_ns_proto;extern const struct bpf_func_proto bpf_ktime_get_boot_ns_proto;extern const struct bpf_func_proto bpf_ktime_get_tai_ns_proto;extern const struct bpf_func_proto bpf_get_current_pid_tgid_proto; +extern const struct bpf_func_proto bpf_binprm_set_interp_proto;extern const struct bpf_func_proto bpf_get_current_uid_gid_proto;extern const struct bpf_func_proto bpf_get_current_comm_proto;extern const struct bpf_func_proto bpf_get_stackid_proto;
include/uapi/linux/bpf.h +1 -0
@@ -6142,6 +6142,7 @@ union bpf_attr { FN(user_ringbuf_drain, 209, ##ctx) \ FN(cgrp_storage_get, 210, ##ctx) \ FN(cgrp_storage_delete, 211, ##ctx) \ + FN(binprm_set_interp, 212, ##ctx) \/* This helper list is effectively frozen. If you are trying to \ * add a new helper, you should add a kfunc instead which has \ * less stability guarantees. See Documentation/bpf/kfuncs.rst \
net/core/filter.c +8 -0
@@ -8397,6 +8397,14 @@ sk_filter_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)return &bpf_get_socket_uid_proto;case BPF_FUNC_perf_event_output:return &bpf_skb_event_output_proto; +#if IS_BUILTIN (CONFIG_BINFMT_MISC ) +case BPF_FUNC_binprm_set_interp: +/* +* binfmt_misc reuses SOCKET_FILTER programs to select an +* interpreter; expose the helper that lets them set it. +*/ +return &bpf_binprm_set_interp_proto; +#endif default :return bpf_sk_base_func_proto (func_id, prog); }
Signature · 1 line
2.51.2
Christian Brauner
maintainerRe: [RFC PATCH] fs: binfmt_misc: introduce eBPF-based matching and interpreter selection
Quoted 23 lines
Historically
binfmt_mischas only matched binaries by magic bytes or file extension, and always redirects a match to a fixed interpreter recorded in the registration. This is insufficient when the match requires parsing the binary header (e.g. inspecting ELF program headers) or when the interpreter must be computed per-binary rather than hard-coded.Introduce a new 'B' (BPF) handler type. A pinned
SOCKET_FILTERprogram is registered in place of the magic/mask, and no interpreter is recorded:echo ':name:B:<bpf_pinned_path>::::<flags>' \ > /proc/sys/fs/binfmt_misc/registerWhen a binary is executed,
binfmt_miscruns the program with theBINPRM_BUF_SIZEfile-header buffer as context. Returning 1 selects the handler; returning 0 falls through to the remaining handlers.Unlike magic/extension handlers, a 'B' handler carries no interpreter of its
own: the program chooses it via a new helper,bpf_binprm_set_interp(). This lets the program compute the interpreter path however it sees fit (for example relative to the binary). A 'B' handler is therefore a strict superset of the existing magic handlers -- any of them can be expressed as a program that matches on the header and sets a fixed interpreter.
Quoted 6 lines
bpf_binprm_set_interp()is exposed toSOCKET_FILTERprograms and stashes the chosen path on a per-CPU area thatbinfmt_miscreads back immediately after the run undermigrate_disable(); only a match that set an interpreter allocates.Signed-off-by Farid Zakaria <[email protected]>
Ignoring the blatant bpf abuse here I think this is quite workable. So this can be turned into an actual design and patch imho...
Note that binfmt_misc is namespaced and can be mounted inside of user namespace + mount namespaces fwiw. So a container mounting binfmt_misc (a fresh instance - few do) would escape that bpf program. On the other hand it would allow to register a custom bpf program per container if needed...
Signature · 1 line
Christian Brauner <[email protected]>
Re: [RFC PATCH] fs: binfmt_misc: introduce eBPF-based matching and interpreter selection
Quoted 29 lines · Christian Brauner
Quoted 29 lines
Historically
binfmt_mischas only matched binaries by magic bytes or file extension, and always redirects a match to a fixed interpreter recorded in the registration. This is insufficient when the match requires parsing the binary header (e.g. inspecting ELF program headers) or when the interpreter must be computed per-binary rather than hard-coded.Introduce a new 'B' (BPF) handler type. A pinned
SOCKET_FILTERprogram is registered in place of the magic/mask, and no interpreter is recorded:echo ':name:B:<bpf_pinned_path>::::<flags>' \ > /proc/sys/fs/binfmt_misc/registerWhen a binary is executed,
binfmt_miscruns the program with theBINPRM_BUF_SIZEfile-header buffer as context. Returning 1 selects the handler; returning 0 falls through to the remaining handlers.Unlike magic/extension handlers, a 'B' handler carries no interpreter of its
own: the program chooses it via a new helper,bpf_binprm_set_interp(). This lets the program compute the interpreter path however it sees fit (for example relative to the binary). A 'B' handler is therefore a strict superset of the existing magic handlers -- any of them can be expressed as a program that matches on the header and sets a fixed interpreter.
bpf_binprm_set_interp()is exposed toSOCKET_FILTERprograms and stashes the chosen path on a per-CPU area thatbinfmt_miscreads back immediately after the run undermigrate_disable(); only a match that set an interpreter allocates.
Quoted 4 lines
Quoted 1 line
Signed-off-by Farid Zakaria <[email protected]>Ignoring the blatant bpf abuse here I think this is quite workable. So this can be turned into an actual design and patch imho...
Glad to see you think this proposal is in the right direction. Just for my edification, when you say "balatant bpf abuse", you mean how it abuses the socket bpf type?
If so, do you have any guidance on a new type or should I just take a stab at it.
For the proper RFC patch unbastardized with tests, should I start a new thread or is
the convention to keep attaching it to this same thread?
Quoted 6 lines
Note that
binfmt_miscis namespaced and can be mounted inside of user namespace + mount namespaces fwiw. So a container mountingbinfmt_misc(a fresh instance - few do) would escape that bpf program. On the other hand it would allow to register a custom bpf program per container if needed...
That makes sense -- but you will need CAP_BPF etc.. to load the new programs though
if inside a userns.
Quoted 3 lines
--
Christian Brauner <[email protected]>
Christian Brauner
maintainerRe: [RFC PATCH] fs: binfmt_misc: introduce eBPF-based matching and interpreter selection
Quoted 11 lines · Farid Zakaria
Quoted 2 lines · Christian Brauner
Ignoring the blatant bpf abuse here I think this is quite workable. So this can be turned into an actual design and patch imho...
Glad to see you think this proposal is in the right direction. Just for my edification, when you say "balatant bpf abuse", you mean how it abuses the socket bpf type?
If so, do you have any guidance on a new type or should I just take a stab at it.
Quoted 3 lines
For the proper RFC patch unbastardized with tests, should I start a new thread or is
the convention to keep attaching it to this same thread?
Always start a new thread and bump the version number. So we're at v3 now, if I'm counting correctly.
Fwiw, if you have access to an LLM you can try and setup b4 to manage patch series. It's quite easy once you get the hang of it.
Quoted 9 lines
Quoted 5 lines
Note that
binfmt_miscis namespaced and can be mounted inside of user namespace + mount namespaces fwiw. So a container mountingbinfmt_misc(a fresh instance - few do) would escape that bpf program. On the other hand it would allow to register a custom bpf program per container if needed...That makes sense -- but you will need
CAP_BPFetc.. to load the new programs though
if inside a userns.
Right, bpf tokens exist to delegate bpf access to containers. Not always ideal but it works for such cases. My main point was to communicate that containers are able to disconnect themselves from being able to execute binaries by mounting an empty binfmt_misc instance, iirc. I should remember that as I did that work but I kinda don't exactly. :)
In any case, I think it wouldn't be crazy to allow registering "default" handlers that appear in all binfmt_misc instances if needed so containers with separate binfmt_misc always have the bpf program available. Just thinking out loud...
Christian Brauner
maintainerRe: [RFC PATCH] fs: binfmt_misc: introduce eBPF-based matching and interpreter selection
Quoted 11 lines · Christian Brauner
Quoted 10 lines · Farid Zakaria
Quoted 2 lines · Christian Brauner
Ignoring the blatant bpf abuse here I think this is quite workable. So this can be turned into an actual design and patch imho...
Glad to see you think this proposal is in the right direction. Just for my edification, when you say "balatant bpf abuse", you mean how it abuses the socket bpf type?
If so, do you have any guidance on a new type or should I just take a stab at it.
I drafted a POC for you that should be closer to how this should look like with the bpf people in Cc. I'll let you lead from now on. I just wanted to help you out and illustrate a more viable direction. Might be that the bpf people want it done differentely. Let's see.
Re: [RFC PATCH] fs: binfmt_misc: introduce eBPF-based matching and interpreter selection
Quoted 13 lines · Christian Brauner
Quoted 11 lines · Christian Brauner
Quoted 10 lines · Farid Zakaria
Quoted 2 lines · Christian Brauner
Ignoring the blatant bpf abuse here I think this is quite workable. So this can be turned into an actual design and patch imho...
Glad to see you think this proposal is in the right direction. Just for my edification, when you say "balatant bpf abuse", you mean how it abuses the socket bpf type?
If so, do you have any guidance on a new type or should I just take a stab at it.
Quoted 4 lines
I drafted a POC for you that should be closer to how this should look like with the bpf people in Cc. I'll let you lead from now on. I just wanted to help you out and illustrate a more viable direction. Might be that the bpf people want it done differentely. Let's see.
Cool. I just saw the POC.
(I'm trying out aerc to view the patch series. I will also try b4, I've never set it up before)
I will rework my qemu demo to use the struct_ops and go through my own testing to see it work.
That looks very similar to what I was researching with an LLM. I was just going over the idea of struct_ops bpf (a new concept to me) and using traditional bpftool load, a new bpf program type and kfuncs.
You put out a surprising amount of polished code -- thank you. I would have had to lean on an LLM (heavily) and from the patches I've put up so far they still have quite the gap to cover in terms of polish.
What does "take it from here" mean in practicality? I will test & validate the changes and just wait for comments. If there are none and it works....then I guess it's done ;P
Christian Brauner
maintainerRe: [RFC PATCH] fs: binfmt_misc: introduce eBPF-based matching and interpreter selection
Quoted 25 lines · Farid Zakaria
Quoted 13 lines · Christian Brauner
Quoted 11 lines · Christian Brauner
Quoted 10 lines · Farid Zakaria
Quoted 2 lines · Christian Brauner
Ignoring the blatant bpf abuse here I think this is quite workable. So this can be turned into an actual design and patch imho...
Glad to see you think this proposal is in the right direction. Just for my edification, when you say "balatant bpf abuse", you mean how it abuses the socket bpf type?
If so, do you have any guidance on a new type or should I just take a stab at it.Quoted 4 lines
I drafted a POC for you that should be closer to how this should look like with the bpf people in Cc. I'll let you lead from now on. I just wanted to help you out and illustrate a more viable direction. Might be that the bpf people want it done differentely. Let's see.
Cool. I just saw the POC.
(I'm trying out aerc to view the patch series. I will also try b4, I've never set it up before)
I will rework my qemu demo to use thestruct_opsand go through my own testing to see it work.
Quoted 5 lines
That looks very similar to what I was researching with an LLM. I was just going over the idea of
struct_opsbpf (a new concept to me) and using traditional bpftool load, a new bpf program type and kfuncs.You put out a surprising amount of polished code -- thank you.
I would not call this polished and it also lacks any tests... I've not given the locking much thought and maybe there's easy wins on the table I haven't been considering.
Quoted 2 lines
I would have had to lean on an LLM (heavily) and from the patches I've put up so far they still have quite the gap to cover in terms of polish.
This isn't my first bpf ride in both kernel and userspace.
Quoted 1 line
What does "take it from here" mean in practicality?
Bring it over the finish line so I can merge it into my tree. :) If the bpf people have no comments or worries than we're done. If they want changes I'd appreciate if you could make them and slap you Co-developed-by onto the whole thing as necessary.
Also, please ensure that this works for all the cases you're considering.
Christian Brauner
maintainerRe: [RFC PATCH] fs: binfmt_misc: introduce eBPF-based matching and interpreter selection
Quoted 43 lines · Christian Brauner
Quoted 25 lines · Farid Zakaria
Quoted 13 lines · Christian Brauner
Quoted 11 lines · Christian Brauner
Quoted 10 lines · Farid Zakaria
Quoted 2 lines · Christian Brauner
Ignoring the blatant bpf abuse here I think this is quite workable. So this can be turned into an actual design and patch imho...
Glad to see you think this proposal is in the right direction. Just for my edification, when you say "balatant bpf abuse", you mean how it abuses the socket bpf type?
If so, do you have any guidance on a new type or should I just take a stab at it.Quoted 4 lines
I drafted a POC for you that should be closer to how this should look like with the bpf people in Cc. I'll let you lead from now on. I just wanted to help you out and illustrate a more viable direction. Might be that the bpf people want it done differentely. Let's see.
Cool. I just saw the POC.
(I'm trying out aerc to view the patch series. I will also try b4, I've never set it up before)
I will rework my qemu demo to use thestruct_opsand go through my own testing to see it work.Quoted 5 lines
That looks very similar to what I was researching with an LLM. I was just going over the idea of
struct_opsbpf (a new concept to me) and using traditional bpftool load, a new bpf program type and kfuncs.You put out a surprising amount of polished code -- thank you.
I would not call this polished and it also lacks any tests... I've not given the locking much thought and maybe there's easy wins on the table I haven't been considering.
Quoted 2 lines
I would have had to lean on an LLM (heavily) and from the patches I've put up so far they still have quite the gap to cover in terms of polish.
This isn't my first bpf ride in both kernel and userspace.
Quoted 1 line
What does "take it from here" mean in practicality?
Quoted 7 lines
Bring it over the finish line so I can merge it into my tree. :) If the bpf people have no comments or worries than we're done. If they want changes I'd appreciate if you could make them and slap you Co-developed-by onto the whole thing as necessary.
Also, please ensure that this works for all the cases you're considering.
I have another cleanup series for binfmt_misc btw. That code is quite old. There might be a few additional series coming out of this work that we base the bpf bits on.
Re: [RFC PATCH] fs: binfmt_misc: introduce eBPF-based matching and interpreter selection
Quoted 49 lines · Christian Brauner
Quoted 48 lines · Christian Brauner
Quoted 25 lines · Farid Zakaria
Quoted 13 lines · Christian Brauner
Quoted 11 lines · Christian Brauner
Quoted 10 lines · Farid Zakaria
Quoted 2 lines · Christian Brauner
Ignoring the blatant bpf abuse here I think this is quite workable. So this can be turned into an actual design and patch imho...
Glad to see you think this proposal is in the right direction. Just for my edification, when you say "balatant bpf abuse", you mean how it abuses the socket bpf type?
If so, do you have any guidance on a new type or should I just take a stab at it.Quoted 4 lines
I drafted a POC for you that should be closer to how this should look like with the bpf people in Cc. I'll let you lead from now on. I just wanted to help you out and illustrate a more viable direction. Might be that the bpf people want it done differentely. Let's see.
Cool. I just saw the POC.
(I'm trying out aerc to view the patch series. I will also try b4, I've never set it up before)
I will rework my qemu demo to use thestruct_opsand go through my own testing to see it work.Quoted 5 lines
That looks very similar to what I was researching with an LLM. I was just going over the idea of
struct_opsbpf (a new concept to me) and using traditional bpftool load, a new bpf program type and kfuncs.You put out a surprising amount of polished code -- thank you.
I would not call this polished and it also lacks any tests... I've not given the locking much thought and maybe there's easy wins on the table I haven't been considering.
Quoted 2 lines
I would have had to lean on an LLM (heavily) and from the patches I've put up so far they still have quite the gap to cover in terms of polish.
This isn't my first bpf ride in both kernel and userspace.
Quoted 1 line
What does "take it from here" mean in practicality?
Bring it over the finish line so I can merge it into my tree. :) If the bpf people have no comments or worries than we're done. If they want changes I'd appreciate if you could make them and slap you Co-developed-by onto the whole thing as necessary.
Quoted 6 lines
Quoted 2 lines
Also, please ensure that this works for all the cases you're considering.
I have another cleanup series for
binfmt_miscbtw. That code is quite old. There might be a few additional series coming out of this work that we base the bpf bits on.
Sounds good. I just saw the patch series.
My plan at the least by the weekend is to:
* apply your binfmt_misc cleanup series
* I will try b4 with aerc. Here is my config if anyone wants to crib it [1].
* apply your POC series for the bpf to validate
* make any changes if necessary or just add selftests if it works as is
Christian Brauner
maintainerRe: [RFC PATCH] fs: binfmt_misc: introduce eBPF-based matching and interpreter selection
Quoted 57 lines · Farid Zakaria
Quoted 49 lines · Christian Brauner
Quoted 48 lines · Christian Brauner
Quoted 25 lines · Farid Zakaria
Quoted 13 lines · Christian Brauner
Quoted 11 lines · Christian Brauner
Quoted 10 lines · Farid Zakaria
Quoted 2 lines · Christian Brauner
Ignoring the blatant bpf abuse here I think this is quite workable. So this can be turned into an actual design and patch imho...
Glad to see you think this proposal is in the right direction. Just for my edification, when you say "balatant bpf abuse", you mean how it abuses the socket bpf type?
If so, do you have any guidance on a new type or should I just take a stab at it.Quoted 4 lines
I drafted a POC for you that should be closer to how this should look like with the bpf people in Cc. I'll let you lead from now on. I just wanted to help you out and illustrate a more viable direction. Might be that the bpf people want it done differentely. Let's see.
Cool. I just saw the POC.
(I'm trying out aerc to view the patch series. I will also try b4, I've never set it up before)
I will rework my qemu demo to use thestruct_opsand go through my own testing to see it work.Quoted 5 lines
That looks very similar to what I was researching with an LLM. I was just going over the idea of
struct_opsbpf (a new concept to me) and using traditional bpftool load, a new bpf program type and kfuncs.You put out a surprising amount of polished code -- thank you.
I would not call this polished and it also lacks any tests... I've not given the locking much thought and maybe there's easy wins on the table I haven't been considering.
Quoted 2 lines
I would have had to lean on an LLM (heavily) and from the patches I've put up so far they still have quite the gap to cover in terms of polish.
This isn't my first bpf ride in both kernel and userspace.
Quoted 1 line
What does "take it from here" mean in practicality?
Bring it over the finish line so I can merge it into my tree. :) If the bpf people have no comments or worries than we're done. If they want changes I'd appreciate if you could make them and slap you Co-developed-by onto the whole thing as necessary.
Quoted 6 lines
Quoted 2 lines
Also, please ensure that this works for all the cases you're considering.
I have another cleanup series for
binfmt_miscbtw. That code is quite old. There might be a few additional series coming out of this work that we base the bpf bits on.
Quoted 8 lines
Sounds good. I just saw the patch series.
My plan at the least by the weekend is to:
* apply yourbinfmt_misccleanup series
* I will try b4 with aerc. Here is my config if anyone wants to crib it [1].
* apply your POC series for the bpf to validate
* make any changes if necessary or just add selftests if it works as is
I hade some vibe-coded tests somewhere. Let me see if I can find them... Ok, appending them. There's an elf-section parsing test and a $ORIGIN test.
// SPDX-License-Identifier: GPL-2.0
/*
* elf_sections.bpf.c - walking ELF section headers from a binfmt_misc_ops
* program.
*
* Demonstrates full section-table parsing via the file dynptr: locate a
* section by name (through e_shstrndx/shstrtab) and read its contents.
* The demo looks up ".comment" and dumps it, then declines so the binary
* is loaded normally.
*/
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
char _license[] SEC("license") = "GPL";
#define EI_CLASS 4
#define ELFCLASS64 2
#define SHT_NOBITS 8
#define SHN_UNDEF 0
#define SHN_LORESERVE 0xff00
#define MAX_SHDRS 64
#define SEC_NAME_MAX 64
#define DUMP_MAX 64
extern int bpf_dynptr_from_file(struct file *file, __u32 flags,
struct bpf_dynptr *ptr__uninit) __ksym;
extern int bpf_dynptr_file_discard(struct bpf_dynptr *dynptr) __ksym;
/* Look up a section by name; @name_len includes the NUL. */ static __always_inline int find_section(struct bpf_dynptr *dp,
const struct elf64_hdr *ehdr, const char *name, __u32 name_len, struct elf64_shdr *out)
{
char nbuf[SEC_NAME_MAX];
struct elf64_shdr sh;
__u64 shoff = ehdr->e_shoff;
__u32 shnum = ehdr->e_shnum;
__u64 stroff, strsz;
int i, j;
if (!shoff || !shnum || shnum >= SHN_LORESERVE)
return -1;
if (ehdr->e_shentsize != sizeof(sh))
return -1;
if (ehdr->e_shstrndx == SHN_UNDEF || ehdr->e_shstrndx >= shnum)
return -1;
if (!name_len || name_len > sizeof(nbuf))
return -1;
/* The section-name string table. */
if (bpf_dynptr_read(&sh, sizeof(sh), dp,
shoff + ehdr->e_shstrndx * sizeof(sh), 0))
return -1;
stroff = sh.sh_offset;
strsz = sh.sh_size;
bpf_for(i, 0, shnum) {
bool match = true;
if (i >= MAX_SHDRS)
break;
if (bpf_dynptr_read(&sh, sizeof(sh), dp,
shoff + i * sizeof(sh), 0))
return -1;
/* Name (incl. NUL) must fit inside shstrtab. */
if (sh.sh_name >= strsz || strsz - sh.sh_name < name_len)
continue;
if (bpf_dynptr_read(nbuf, name_len, dp, stroff + sh.sh_name, 0))
continue;
bpf_for(j, 0, name_len) {
if (j >= SEC_NAME_MAX || nbuf[j] != name[j]) {
match = false;
break;
}
}
if (!match)
continue;
*out = sh;
return 0;
}
return -1;
}#define FIND_SECTION(dp, ehdr, lit, out) \
find_section(dp, ehdr, lit, sizeof(lit), out)
SEC("struct_ops.s/load")
int BPF_PROG(elf_sections_load, struct linux_binprm *bprm)
{
char data[DUMP_MAX] = {};
struct elf64_hdr ehdr;
struct elf64_shdr sh;
struct bpf_dynptr dp;
__u32 n;
if (bprm->buf[0] != 0x7f || bprm->buf[1] != 'E' ||
bprm->buf[2] != 'L' || bprm->buf[3] != 'F' ||
bprm->buf[EI_CLASS] != ELFCLASS64)
return 0;
if (bpf_dynptr_from_file(bprm->file, 0, &dp))
goto out;
if (bpf_dynptr_read(&ehdr, sizeof(ehdr), &dp, 0, 0))
goto out;
if (FIND_SECTION(&dp, &ehdr, ".comment", &sh))
goto out;
/* SHT_NOBITS (.bss-like) sections have no file content. */
if (sh.sh_type == SHT_NOBITS || !sh.sh_size)
goto out;
n = sh.sh_size;
if (n > sizeof(data) - 1)
n = sizeof(data) - 1;
if (bpf_dynptr_read(data, n, &dp, sh.sh_offset, 0))
goto out;
bpf_printk("elf_sections: .comment @%llu +%llu: %s",
(__u64)sh.sh_offset, (__u64)sh.sh_size, data);
out:
bpf_dynptr_file_discard(&dp);
return 0; /* observer only: always decline */
}SEC(".struct_ops.link")
struct binfmt_misc_ops elf_sections = {
.load = (void *)elf_sections_load,
.name = "elf_sections",
};// SPDX-License-Identifier: GPL-2.0
/*
* nix_origin.bpf.c - $ORIGIN-relative PT_INTERP resolution *
* A binfmt_misc_ops handler that makes relocatable (Nix-style) ELF
* binaries work: if PT_INTERP starts with "$ORIGIN/", the loader is
* resolved relative to the directory of the binary being executed and
* selected via bpf_binprm_set_interp(). Anything else is declined so
* regular binaries fall through untouched.
*
* Activate with:
* bpftool struct_ops register nix_origin.bpf.o /sys/fs/bpf
* echo ':nix-origin:B:nix_origin::::' > /proc/sys/fs/binfmt_misc/register
*/
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
char _license[] SEC("license") = "GPL";
#define PATH_MAX 4096
#define EI_CLASS 4
#define ELFCLASSXX 2 /* ELFCLASS64; flip to 1 for 32-bit */
#define PT_INTERP 3
#define MAX_PHDRS 64
#define ORIGIN "$ORIGIN"
#define ORIGIN_LEN (sizeof(ORIGIN) - 1)
#define ENOENT 2
#define ENAMETOOLONG 36
extern int bpf_dynptr_from_file(struct file *file, __u32 flags,
struct bpf_dynptr *ptr__uninit) __ksym; extern int bpf_dynptr_file_discard(struct bpf_dynptr *dynptr) __ksym; extern int bpf_path_d_path(const struct path *path, char *buf, size_t buf__sz) __ksym; extern int bpf_binprm_set_interp(struct linux_binprm *bprm, const char *path, size_t path__sz) __ksym;
struct scratch {
char interp[PATH_MAX]; /* PT_INTERP as embedded in the binary */
char path[PATH_MAX]; /* d_path of the binary, becomes the result */
};
/* Keyed by pid: execs run concurrently and the program can sleep. */
struct {
__uint(type, BPF_MAP_TYPE_HASH);
__uint(max_entries, 512);
__type(key, __u64);
__type(value, struct scratch);
} scratch_map SEC(".maps");static const struct scratch zero_scratch;
SEC("struct_ops.s/load")
int BPF_PROG(nix_origin_load, struct linux_binprm *bprm)
{
__u32 isz, sfx, rsz, slash;
struct elf64_phdr phdr;
struct elf64_hdr ehdr;
struct bpf_dynptr dp;
struct scratch *sc;
bool found = false;
__u64 id;
int ret = 0, len, i;
/* Cheap reject from the prefetched header. */
if (bprm->buf[0] != 0x7f || bprm->buf[1] != 'E' ||
bprm->buf[2] != 'L' || bprm->buf[3] != 'F' ||
bprm->buf[EI_CLASS] != ELFCLASSXX)
return 0;
if (bpf_dynptr_from_file(bprm->file, 0, &dp))
goto out;
if (bpf_dynptr_read(&ehdr, sizeof(ehdr), &dp, 0, 0))
goto out;
if (ehdr.e_phentsize != sizeof(struct elf64_phdr))
goto out;
bpf_for(i, 0, ehdr.e_phnum) {
if (i >= MAX_PHDRS)
break;
if (bpf_dynptr_read(&phdr, sizeof(phdr), &dp,
ehdr.e_phoff + i * sizeof(phdr), 0))
goto out;
if (phdr.p_type == PT_INTERP) {
found = true;
break;
}
}
if (!found)
goto out;
isz = phdr.p_filesz;
if (isz <= ORIGIN_LEN + 1 || isz > sizeof(sc->interp))
goto out;
id = bpf_get_current_pid_tgid();
if (bpf_map_update_elem(&scratch_map, &id, &zero_scratch, BPF_ANY))
goto out;
sc = bpf_map_lookup_elem(&scratch_map, &id);
if (!sc)
goto out_del;
if (bpf_dynptr_read(sc->interp, isz, &dp, phdr.p_offset, 0))
goto out_del;
if (sc->interp[isz - 1] != '\0')
goto out_del;
/* Not "$ORIGIN/..."? Decline, the elf loader owns it. */
if (sc->interp[0] != '$' || sc->interp[1] != 'O' ||
sc->interp[2] != 'R' || sc->interp[3] != 'I' ||
sc->interp[4] != 'G' || sc->interp[5] != 'I' ||
sc->interp[6] != 'N' || sc->interp[7] != '/')
goto out_del;
/*
* From here on the binary is ours: resolution failures fail the
* exec instead of falling back to binfmt_elf, which would resolve
* the literal "$ORIGIN/..." relative to the caller's cwd.
*/
ret = -ENOENT;
len = bpf_path_d_path(&bprm->file->f_path, sc->path, sizeof(sc->path));
if (len <= 0 || len > sizeof(sc->path))
goto out_del;
/* Unreachable or unlinked ("... (deleted)") binaries can't resolve. */
if (sc->path[0] != '/')
goto out_del;
/* $ORIGIN = dirname of the binary. */
slash = 0;
bpf_for(i, 1, len - 1) {
if (i >= sizeof(sc->path))
break;
if (sc->path[i] == '/')
slash = i;
}
/* Splice the suffix (leading '/' and NUL included) onto the dir. */
sfx = isz - ORIGIN_LEN;
rsz = slash + sfx;
if (rsz > sizeof(sc->path)) {
ret = -ENAMETOOLONG;
goto out_del;
}
bpf_for(i, 0, sfx) {
__u32 s = ORIGIN_LEN + i, d = slash + i;
if (s >= sizeof(sc->interp) || d >= sizeof(sc->path))
break;
sc->path[d] = sc->interp[s];
}
ret = bpf_binprm_set_interp(bprm, sc->path, rsz);
if (!ret)
ret = 1;
out_del:
bpf_map_delete_elem(&scratch_map, &id);
out:
bpf_dynptr_file_discard(&dp);
return ret;
}SEC(".struct_ops.link")
struct binfmt_misc_ops nix_origin = {
.load = (void *)nix_origin_load,
.name = "nix_origin",
};Re: [RFC PATCH] fs: binfmt_misc: introduce eBPF-based matching and interpreter selection
In reply to Christian BraunerQuoted 5 lines · On Tue, 07 Jul 2026 13:34:46 +0200 Christian Brauner <[email protected]> wrote:
Quoted 2 lines
That makes sense -- but you will need
CAP_BPFetc.. to load the new programs though if inside a userns.Right, bpf tokens exist to delegate bpf access to containers. Not always ideal but it works for such cases.
Hi everyone, first post here.
tl;dr I think an $ORIGIN+PT_INTERP-like feature should be available by default (no root) since portable software is most important when you're an unprivileged user.
I've recently read about this patch and I'm super excited. An ORIGIN-like feature for loaders is something I wanted for years. Let me share our use case + some additional nix use cases.
We have a package/build system that makes heavy use of RPATH+ORIGIN to make portable installation directories of our software, as in: extract a plain .tar in ~/mysoftware, ./myexecutable and it works [1]. No root, no run-time patching, no namespaces.
Now, this means we need to ship all of our dependencies properly RPATH'd (using $ORIGIN). The only exception is the libc, which we can't ship, since we can't put $ORIGIN in PT_INTERP. This means we need to link against an ancient libc in order to be compatible with a wide range of host libcs. Needless to say, is a *huge* pain.
This patch could in theory solve our problems, except for the "no root" part, which I think it's the most important one. After all, if you assume your user is root (at least at install time), you don't have big problems, just drop your software in /opt/mysoftware and ship your loader in there.
But I think being able to use $ORIGIN from non-root users would be useful also for nix, not when you are on NixOS (where I'd assume they'd enable PT_INTERP_NIX by default), but when you're on a non-NixOS system.
There are several projects that aim at running nix without root:
https://github.com/davhau/nix-portable
https://github.com/nixie-dev/nixie
Currently they either use user namespaces or ptrace. However, the former is restricted under Ubuntu (apparmor_restrict_unprivileged_unconfined) and the latter is not ideal.
AFAIU it'd be possible to avoid all of this if $ORIGIN was available by default in PT_INTERP, or at least the biggest roadblocker would go away.
I understand that enabling $ORIGIN in PT_INTERP can be problematic (I should dig into that actually).
However, in whatever way this gets supported, I think that in order to be actually useful in the most important use cases where portability is important (i.e., no root, no user namespaces), having this feature available *by default* is important.
Quoted 4 lines
In any case, I think it wouldn't be crazy to allow registering "default" handlers that appear in all
binfmt_miscinstances if needed so containers with separatebinfmt_miscalways have the bpf program available. [...]
Maybe we can have a "default eBPF program" loaded for PT_INTERP_WITH_ORIGIN?
We toolchain people are happy to tamper with our binaries, but if these features are not available by default, their use is going to be limited.
In any case, thanks for this work, I've been wanting this for a long time.
--
Alessandro Di Federico
rev.ng Labs
[1] https://github.com/revng/orchestra
Note that this is soon going to be dismissed in favor of nix itself.
Example artifact:
wget 'https://rev.ng/downloads/revng-distributable/develop.tar.xz' tar xaf develop.tar.xaf
$ ./revng-public-demo/root/lib64/llvm/llvm/bin/clang --version clang version 16.0.1
Christian Brauner
maintainerRe: [RFC PATCH] fs: binfmt_misc: introduce eBPF-based matching and interpreter selection
Quoted 1 line
This patch could in theory solve our problems, except for the "no root"
We're not really enabling $ORIGIN in core elf unconditionally with weird escape hatches to make it fit for a subset of binaries.
Quoted 7 lines
Quoted 4 lines
In any case, I think it wouldn't be crazy to allow registering "default" handlers that appear in all
binfmt_miscinstances if needed so containers with separatebinfmt_miscalways have the bpf program available. [...]Maybe we can have a "default eBPF program" loaded for
PT_INTERP_WITH_ORIGIN?
I think this could even just be part of systemd. You ship the distro with a file for that in /etc/binfmt.d/ and systemd loads that bpf program for PT_INTERP_WITH_ORIGIN. Then you're good to go for all your binaries with the 'L' option my other patch series adds.
This is policy that the admin should decide, not an arbitrary extension into elf core.
Re: [RFC PATCH] fs: binfmt_misc: introduce eBPF-based matching and interpreter selection
Quoted 3 lines · On Wed, 22 Jul 2026 15:33:40 +0200 Christian Brauner <[email protected]> wrote:
I think this could even just be part of systemd. You ship the distro with a file for that in
/etc/binfmt.d/and systemd loads that bpf program forPT_INTERP_WITH_ORIGIN.
One could delegate the feature to user space, but that makes it much harder to get adopted in a uniform way.
Will systemd, Android [1] and OpenWrt implement this? In the same way? If so, at that point it could have just been a feature of the Linux ELF extension.
Of course the Android people could disable the feature even if it's in the kernel, but if the convention is established by the kernel, there's really no reason to deviate from it in the way it's implemented.
The whole point of the portability thing is to reduce the assumptions you make about the target system. Ideally, you just want to use features widely available in kernels in the wild.
But from what I understand from another thread, your intention is to use userspace as testbench, correct?
Quoted 1 line
This is policy that the admin should decide
Agreed, but the admin is probably satisfied with enabling/disabling, not deciding whether the program header type is PT_INTERP_WITH_ORIGIN or PT_ORIGIN_ENABLED_INTERP. Same for the distro maintainers.
Quoted 1 line
not an arbitrary extension into elf core.
I think that, in order for this to be useful, it needs to be a ELF feature, not something the admin/distro maintainer puts together in eBPF.
Then, from a security perspective, AFAIU, implementing this in eBPF does not solve the problems that have been discussed.
For instance, selftests/exec: add binfmt_misc bpf-backed handler test does string manipulation (no openat).
I understand that's just an example and it's not the same as committing to a new ABI feature, but still.
Overall, I feel like the current proposal pushes back on having a new ELF feature for security reasons, but then delegates to distro maintainers who want to use the feature to take those risks. But also, distro maintainers don't really need this feature because they can just write /nix.
I love binfmt_misc enabling you to do ./foreign-architecture-binary or ./notepad.exe and spawn qemu/wine (usually with your friendly sysadmin approval) and extra flexibility in that area with eBPF is super cool. However if we are thinking about portability, this make sense only if it's a default-enabled ELF feature (implemented in eBPF or not).
I'd be happy to try to work on the example eBPF program using openat, address the various security concerns that have been mentioned until it looks right and then it could be loaded by default, but with an option to disable it. But I'm not sure whether there's such a thing as "default loaded eBPF program".
Alternatively, I could extend the ELF core as a parallel feature of the eBPF-based mechanism, reviving some patches from previous iterations.
What do you think?
Signature · 7 lines
Alessandro Di Federico
rev.ng Labs
[1] There's a flavor of nix for Android (using proot) which would
benefit from this:
https://github.com/nix-community/nix-on-droid-appChristian Brauner
maintainerRe: [RFC PATCH] fs: binfmt_misc: introduce eBPF-based matching and interpreter selection
Quoted 6 lines · Alessandro Di Federico
One could delegate the feature to user space, but that makes it much harder to get adopted in a uniform way.
Will systemd, Android [1] and OpenWrt implement this? In the same way? If so, at that point it could have just been a feature of the Linux ELF extension.
Whether systemd, Android or OpenWrt adopt something isn't a decision the kernel gets to make and it isn't one it has to make. Merging code doesn't create adoption. It creates an obligation to keep the behavior working forever. That includes the machines whose owners never asked for it.
It's also not how ELF conventions get established. PT_GNU_STACK and PT_GNU_PROPERTY are the two OS-range headers our loader acts on. And they came out of the toolchain and we implemented them afterwards.
The one time we picked a program header type ourselves ahead of the ABI PT_ARM_MEMTAG_MTE in 761b9b366cec we collided with PT_AARCH64_UNWIND and had to change the name and the value three months later. We got away with it because it's a coredump segment that gdb reads, not the exec behavior of every binary on the system.
If you want systemd, Android and OpenWrt to agree on one marker, the thing that produces that agreement is generic-abi with binutils and glibc/musl on board, and then we can implement it.
So we don't actually disagree about there being an ELF feature. We disagree about what the header means. My position is that the header names the intent and userspace implements the resolution. Yours is that the kernel does both.
Quoted 3 lines
Of course the Android people could disable the feature even if it's in the kernel, but if the convention is established by the kernel, there's really no reason to deviate from it in the way it's implemented.
The kernel can't establish this convention. It can only implement one. If binutils won't emit the header and no loader cooperates, we've shipped uapi that nothing uses and that we still can't remove.
Quoted 3 lines
The whole point of the portability thing is to reduce the assumptions you make about the target system. Ideally, you just want to use features widely available in kernels in the wild.
Sure, but nothing merged today is widely available tomorrow. Both need a new kernel. Uniform-by-default doesn't arrive sooner because it went into core elf.
And a new phdr doesn't degrade nicely. On any kernel that doesn't know it the binary has no PT_INTERP it loads as static. Dies horribly at an unrelocated _start.
And keeping a literal $ORIGIN fallback in PT_INTERP instead and old kernels or gVisor open it cwd-relative, so an attacker who controls cwd plants ./$ORIGIN/ld.so.
Quoted 2 lines
But from what I understand from another thread, your intention is to use userspace as testbench, correct?
Not testbench as in "go away and come back later". The difference is where the semantics live once they turn out to be wrong. Get a corner case wrong in a bpf handler and the distro ships a fix in the next update. Get it wrong in load_elf_binary() and it's the exec behavior of arbitrary binaries... which we can't change afterwards.
Quoted 5 lines
Quoted 1 line
This is policy that the admin should decide
Agreed, but the admin is probably satisfied with enabling/disabling, not deciding whether the program header type is
PT_INTERP_WITH_ORIGINorPT_ORIGIN_ENABLED_INTERP. Same for the distro maintainers.
Nobody is asking an admin to name a program header type. The distro ships the handler and a file in /etc/binfmt.d and the admin never sees either.
But note what you argued here... If it needs an enable/disable switch then it is policy. And the only question left is what the switch looks like. A global boot-time on/off is the worst available version of it. It's one bit, machine-wide and can't be scoped to a subtree, can't be delegated.
Quoted 5 lines
Then, from a security perspective, AFAIU, implementing this in eBPF does not solve the problems that have been discussed.
For instance,
selftests/exec: add binfmt_misc bpf-backed handler testdoes string manipulation (noopenat).
The handler doing nothing but string manipulation isn't a simplification in the example. It's the design. The program never opens anything. It computes a path and hands it back, and the kernel opens it with open_exec() under the credentials of whoever called execve(). There is no openat() to add. And fwiw, it also supports pre-opened binaries for really crazy use-cases where you want to pin a set of loaders once.
The module RFC resolved $ORIGIN in kernel code opened the interpreter itself with file_open_root(..., O_RDONLY, 0).
That skips everything open_exec() does and it achors path resolution at the binary's directory. .. clamps there and absolute symlinks resolve inside it. $ORIGIN/../lib/ld.so, which is the RPATH idiom everyone actually writes, silently means <bindir>/lib/ld.so.
I'm not holding an RFC to final-code standards. The point is that this is fiddly.
Quoted 3 lines
Overall, I feel like the current proposal pushes back on having a new ELF feature for security reasons, but then delegates to distro maintainers who want to use the feature to take those risks.
It doesn't delegate the risk. It binds it to whoever wants the feature.
Whether resolving the loader relative to the binary is safe depends on invariants of the deployment. Is the tree immutable, who may write to it, which prefixes are trusted. The kernel doesn't know any of that. /nix/store is root-owned and immutable, so the policy is safe there. The same semantics pointed at a directory the user can write to is a hole. Only the deployment can tell those two apart.
Userspace arrived at the same answer, incidentally. glibc expands $ORIGIN under __libc_enable_secure only when it is the first path component and the result is rooted in a trusted directory. That trusted-directory notion is exactly what an in-kernel implementation would have to invent. And it is deployment-specific.
Quoted 2 lines
But also, distro maintainers don't really need this feature because they can just write /nix.
Right, and that's the honest core of it. The people who need this are users on machines that someone else administers.
But "default-enabled ELF feature" is a strange shape of fix for it. Because the machines you'd be enabling it on belong to the people who never asked. This proposals is to change what exec does for everyone else in order to serve some use-cases.
Quoted 5 lines
I'd be happy to try to work on the example eBPF program using openat, address the various security concerns that have been mentioned until it looks right and then it could be loaded by default, but with an option to disable it. But I'm not sure whether there's such a thing as "default loaded eBPF program".
And there is such a thing. The distro ships the object plus a file in /etc/binfmt.d and systemd loads it at boot. That is a default-loaded program with an option to disable, it just lives in the distro image rather than in vmlinux. If one handler ends up being what everybody ships, that's the uniformity you're after, and nobody had to freeze resolution semantics to get it. Could even be upstream in systemd.
Quoted 3 lines
Alternatively, I could extend the ELF core as a parallel feature of the eBPF-based mechanism, reviving some patches from previous iterations.
I'd rather not, for the reasons above.
Re: [RFC PATCH] fs: binfmt_misc: introduce eBPF-based matching and interpreter selection
Quoted 146 lines · Christian Brauner
Quoted 6 lines · Alessandro Di Federico
One could delegate the feature to user space, but that makes it much harder to get adopted in a uniform way.
Will systemd, Android [1] and OpenWrt implement this? In the same way? If so, at that point it could have just been a feature of the Linux ELF extension.
Whether systemd, Android or OpenWrt adopt something isn't a decision the kernel gets to make and it isn't one it has to make. Merging code doesn't create adoption. It creates an obligation to keep the behavior working forever. That includes the machines whose owners never asked for it.
It's also not how ELF conventions get established.
PT_GNU_STACKandPT_GNU_PROPERTYare the two OS-range headers our loader acts on. And they came out of the toolchain and we implemented them afterwards.The one time we picked a program header type ourselves ahead of the ABI
PT_ARM_MEMTAG_MTEin761b9b366cecwe collided withPT_AARCH64_UNWINDand had to change the name and the value three months later. We got away with it because it's a coredump segment that gdb reads, not the exec behavior of every binary on the system.If you want systemd, Android and OpenWrt to agree on one marker, the thing that produces that agreement is generic-abi with binutils and glibc/musl on board, and then we can implement it.
So we don't actually disagree about there being an ELF feature. We disagree about what the header means. My position is that the header names the intent and userspace implements the resolution. Yours is that the kernel does both.
Quoted 3 lines
Of course the Android people could disable the feature even if it's in the kernel, but if the convention is established by the kernel, there's really no reason to deviate from it in the way it's implemented.
The kernel can't establish this convention. It can only implement one. If binutils won't emit the header and no loader cooperates, we've shipped uapi that nothing uses and that we still can't remove.
Quoted 3 lines
The whole point of the portability thing is to reduce the assumptions you make about the target system. Ideally, you just want to use features widely available in kernels in the wild.
Sure, but nothing merged today is widely available tomorrow. Both need a new kernel. Uniform-by-default doesn't arrive sooner because it went into core elf.
And a new phdr doesn't degrade nicely. On any kernel that doesn't know it the binary has no
PT_INTERPit loads as static. Dies horribly at an unrelocated_start.And keeping a literal $ORIGIN fallback in
PT_INTERPinstead and old kernels or gVisor open it cwd-relative, so an attacker who controls cwd plants ./$ORIGIN/ld.so.Quoted 2 lines
But from what I understand from another thread, your intention is to use userspace as testbench, correct?
Not testbench as in "go away and come back later". The difference is where the semantics live once they turn out to be wrong. Get a corner case wrong in a bpf handler and the distro ships a fix in the next update. Get it wrong in
load_elf_binary()and it's the exec behavior of arbitrary binaries... which we can't change afterwards.Quoted 5 lines
Quoted 1 line
This is policy that the admin should decide
Agreed, but the admin is probably satisfied with enabling/disabling, not deciding whether the program header type is
PT_INTERP_WITH_ORIGINorPT_ORIGIN_ENABLED_INTERP. Same for the distro maintainers.Nobody is asking an admin to name a program header type. The distro ships the handler and a file in
/etc/binfmt.dand the admin never sees either.But note what you argued here... If it needs an enable/disable switch then it is policy. And the only question left is what the switch looks like. A global boot-time on/off is the worst available version of it. It's one bit, machine-wide and can't be scoped to a subtree, can't be delegated.
Quoted 5 lines
Then, from a security perspective, AFAIU, implementing this in eBPF does not solve the problems that have been discussed.
For instance,
selftests/exec: add binfmt_misc bpf-backed handler testdoes string manipulation (noopenat).The handler doing nothing but string manipulation isn't a simplification in the example. It's the design. The program never opens anything. It computes a path and hands it back, and the kernel opens it with
open_exec()under the credentials of whoever calledexecve(). There is noopenat()to add. And fwiw, it also supports pre-opened binaries for really crazy use-cases where you want to pin a set of loaders once.The module RFC resolved $ORIGIN in kernel code opened the interpreter itself with
file_open_root(..., O_RDONLY, 0).
That skips everythingopen_exec()does and it achors path resolution at the binary's directory. .. clamps there and absolute symlinks resolve inside it. $ORIGIN/../lib/ld.so, which is the RPATH idiom everyone actually writes, silently means <bindir>/lib/ld.so.I'm not holding an RFC to final-code standards. The point is that this is fiddly.
Quoted 3 lines
Overall, I feel like the current proposal pushes back on having a new ELF feature for security reasons, but then delegates to distro maintainers who want to use the feature to take those risks.
It doesn't delegate the risk. It binds it to whoever wants the feature.
Whether resolving the loader relative to the binary is safe depends on invariants of the deployment. Is the tree immutable, who may write to it, which prefixes are trusted. The kernel doesn't know any of that. /nix/store is root-owned and immutable, so the policy is safe there. The same semantics pointed at a directory the user can write to is a hole. Only the deployment can tell those two apart.
Userspace arrived at the same answer, incidentally. glibc expands $ORIGIN under
__libc_enable_secureonly when it is the first path component and the result is rooted in a trusted directory. That trusted-directory notion is exactly what an in-kernel implementation would have to invent. And it is deployment-specific.Quoted 2 lines
But also, distro maintainers don't really need this feature because they can just write /nix.
Right, and that's the honest core of it. The people who need this are users on machines that someone else administers.
But "default-enabled ELF feature" is a strange shape of fix for it. Because the machines you'd be enabling it on belong to the people who never asked. This proposals is to change what exec does for everyone else in order to serve some use-cases.
Quoted 5 lines
I'd be happy to try to work on the example eBPF program using openat, address the various security concerns that have been mentioned until it looks right and then it could be loaded by default, but with an option to disable it. But I'm not sure whether there's such a thing as "default loaded eBPF program".
And there is such a thing. The distro ships the object plus a file in
/etc/binfmt.dand systemd loads it at boot. That is a default-loaded program with an option to disable, it just lives in the distro image rather than in vmlinux. If one handler ends up being what everybody ships, that's the uniformity you're after, and nobody had to freeze resolution semantics to get it. Could even be upstream in systemd.
Quoted 5 lines
Quoted 3 lines
Alternatively, I could extend the ELF core as a parallel feature of the eBPF-based mechanism, reviving some patches from previous iterations.
I'd rather not, for the reasons above.
Christian,
Could you help me please understand how the next branch makes its way to the kernel?
I see NixOS Offers the latest as of 7.1.
The topic for binfmt is named on 7.3.binfmt in vfs so my assumption is that it's slated for 7.3.
So that means when Linux pulls for 7.3.rc1 I could start to test it on NixOS? Maybe 3 more weeks?
Christian Brauner
maintainerRe: [RFC PATCH] fs: binfmt_misc: introduce eBPF-based matching and interpreter selection
Quoted 4 lines
Christian,
Could you help me please understand how the next branch makes its way to the kernel?
-next is just an integration tree but is intended to only contain work that is going to be merged into the upcoming merge window.
Quoted 4 lines
I see NixOS Offers the latest as of 7.1.
The topic for binfmt is named on 7.3.binfmt in vfs so my assumption is that it's slated for 7.3.
Yes.
Quoted 3 lines
So that means when Linux pulls for 7.3.rc1 I could start to test it on NixOS? Maybe 3 more weeks?
Two more weeks - unless Linus decided to do -rc8. That happens rarely but it does sometimes happen.
Note, I'm off this week and will be back on Monday.