rust: build: remap path to avoid absolute path
PATCH 1/1 rust: build: remap path to avoid absolute path
From: Gary Guo <[email protected]>
When building with a out directory (O=), absolute paths can end up in the file name in #[track_caller] or the panic message. This is not desirable as this leaks the exact path being used to build the kernel into the binary which is not very helpful to achieve reproducibility. It also means that the same location can appear in two forms (relative or absolute).
This is reported by Asahi [1] and is being workaround in [2] previously to force everything to be absolute path. Using absolute path for everything sovles the inconsistency, however it does not address the reproducibility issue. So, fix this by remap all absolute paths to srctree to relative path instead.
This change can be validated by building a kernel with O=, strip debug info on vmlinux, and then check if the absolute path exists in strings vmlinux, e.g. strings vmlinux |grep \/home.
--- rust/Makefile | 1 + scripts/Makefile.build | 1 + 2 files changed, 2 insertions(+)
rust/Makefile +1 -0
@@ -582,6 +582,7 @@ quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L --crate-type rlib -L$(objtree)/$(obj) \ --crate-name $(patsubst %.o,%,$(notdir $@)) $< \ --sysroot=/dev/null \ + --remap-path-prefix=$(abspath $(srctree))= \ -Zunstable-options \ $(if $(rustc_objcopy),;$(OBJCOPY) $(rustc_objcopy) $@) \ $(cmd_objtool)
scripts/Makefile.build +1 -0
@@ -333,6 +333,7 @@ rust_common_cmd = \ --crate-type rlib -L $(objtree)/rust/ \ --crate-name $(basename $(notdir $@)) \ --sysroot=/dev/null \ + --remap-path-prefix=$(abspath $(srctree))= \ --out-dir $(dir $@) --emit=dep-info=$(depfile) # `--emit=obj`, `--emit=asm` and `--emit=llvm-ir` imply a single codegen unit
base-commit: 36896083ef4dbc302e406f01975a227784160cf8
Signature · 1 line
2.51.2
Quoted 3 lines · Gary Guo
So, fix this by remap all absolute paths to srctree to relative path instead.
In case it matters to Kbuild, we had a relatively recent revert related to this flag:
dbdffaf50ff9 ("kbuild, rust: use -fremap-path-prefix to make paths relative") 8cf5b3f83614 ("Revert "kbuild, rust: use -fremap-path-prefix to make
paths relative"")
From what I see in the thread (but I didn't get a confirmation back then), the issue was some developers relying on invoking the debugger in a different working directory than the srctree.
Thanks Gary!
Cheers,
Miguel
Gary Guo
reviewerQuoted 5 lines · Miguel Ojeda
Quoted 3 lines · Gary Guo
So, fix this by remap all absolute paths to srctree to relative path instead.
Quoted 8 lines
In case it matters to
Kbuild, we had a relatively recent revert related to this flag:
dbdffaf50ff9("kbuild, rust: use -fremap-path-prefix to make paths relative")8cf5b3f83614("Revert "kbuild, rust: use -fremap-path-prefix to make
paths relative"")
Ah, that explains why I recall we had this flag in Kbuild but can only find the filter-out directives now. I missed the revert email.
I am not convinced that the ability to launch debugger outside source directory overweights the benefit of not leaking absolute paths and making builds reproducible.
The reverting cover letter says "As there is no simple or uniform way to specify the source directory explicitly" which is clearly not the case as you can just invoke the debugger in a different working directory... GDB also provides a way to provide source directory search path:
https://sourceware.org/gdb/current/onlinedocs/gdb.html/Source-Path.html.
Similarly, LLDB provides settings set target.source-map:
https://lldb.llvm.org/use/map.html#remap-source-file-pathnames-for-the-debug-session
I think we should revert the revert, then.
Best,
Gary
Quoted 9 lines
From what I see in the thread (but I didn't get a confirmation back then), the issue was some developers relying on invoking the debugger in a different working directory than the
srctree.Thanks Gary!
Cheers,
Miguel
Nathan Chancellor
maintainerQuoted 30 lines · Gary Guo
Quoted 5 lines · Miguel Ojeda
Quoted 3 lines · Gary Guo
So, fix this by remap all absolute paths to srctree to relative path instead.
Quoted 8 lines
In case it matters to
Kbuild, we had a relatively recent revert related to this flag:
dbdffaf50ff9("kbuild, rust: use -fremap-path-prefix to make paths relative")8cf5b3f83614("Revert "kbuild, rust: use -fremap-path-prefix to make
paths relative"")Ah, that explains why I recall we had this flag in
Kbuildbut can only find the filter-out directives now. I missed the revert email.I am not convinced that the ability to launch debugger outside source directory overweights the benefit of not leaking absolute paths and making builds reproducible.
The reverting cover letter says "As there is no simple or uniform way to specify the source directory explicitly" which is clearly not the case as you can just invoke the debugger in a different working directory... GDB also provides a way to provide source directory search path:
https://sourceware.org/gdb/current/onlinedocs/gdb.html/Source-Path.html.
Similarly, LLDB providessettings set target.source-map:
https://lldb.llvm.org/use/map.html#remap-source-file-pathnames-for-the-debug-session
Quoted 1 line
I think we should revert the revert, then.
For what it's worth, the C side revert also mentions that it broke 'objdump -S' [1] and working with GCOV data [2], so it seems like it is not just debuggers that have a hard time with that loss of information. Maybe those do not matter for Rust code or developers though.
[1] https://lore.kernel.org/[email protected]/
[2] https://lore.kernel.org/[email protected]/
Cheers,
Nathan
Gary Guo
reviewerQuoted 33 lines · Nathan Chancellor
Quoted 30 lines · Gary Guo
Quoted 5 lines · Miguel Ojeda
Quoted 3 lines · Gary Guo
So, fix this by remap all absolute paths to srctree to relative path instead.
Quoted 8 lines
In case it matters to
Kbuild, we had a relatively recent revert related to this flag:
dbdffaf50ff9("kbuild, rust: use -fremap-path-prefix to make paths relative")8cf5b3f83614("Revert "kbuild, rust: use -fremap-path-prefix to make
paths relative"")Ah, that explains why I recall we had this flag in
Kbuildbut can only find the filter-out directives now. I missed the revert email.I am not convinced that the ability to launch debugger outside source directory overweights the benefit of not leaking absolute paths and making builds reproducible.
The reverting cover letter says "As there is no simple or uniform way to specify the source directory explicitly" which is clearly not the case as you can just invoke the debugger in a different working directory... GDB also provides a way to provide source directory search path:
https://sourceware.org/gdb/current/onlinedocs/gdb.html/Source-Path.html.
Similarly, LLDB providessettings set target.source-map:
https://lldb.llvm.org/use/map.html#remap-source-file-pathnames-for-the-debug-sessionQuoted 1 line
I think we should revert the revert, then.
Quoted 7 lines
For what it's worth, the C side revert also mentions that it broke 'objdump -S' [1] and working with GCOV data [2], so it seems like it is not just debuggers that have a hard time with that loss of information. Maybe those do not matter for Rust code or developers though.
[1] https://lore.kernel.org/[email protected]/
[2] https://lore.kernel.org/[email protected]/
Hi Nathan, thanks for the background. I tested that objdump will indeed fail to find files with Rust --remap-path-prefix (even inside correct directory) and that's indeed problematic (objdump has an -I to add search path but it will only use the file stem and not the full path).
The best path forward is then perhaps to use -Zremap-path-scope=macro to limit the remap to Rust macros (this include #[track_caller]). The debug info will still have non-reproducible parts in it, but at least the ones ended up in .rodata will be remapped. This flag is going to be stable in Rust 1.95, but is available unstably for a while, so we can start using it in the kernel already. I can confirm that objdump is working fine when the remap-path-scope is limited to macro (default is macro + debuginfo + coverage).
Unfortunately, using remap-path-scope does not give the expected result in older Rust compiler versions, so the issue reports that I linked will not be resolved with it. It looks like remap-path-scope is working as intended only since
https://github.com/rust-lang/rust/pull/149709, so Rust 1.94+ (I haven't double-checked the exact Rust version, but in latest nightly Rust, the remap is working as intended).
Best,
Gary
Quoted 7 lines · Gary Guo
Unfortunately, using remap-path-scope does not give the expected result in older Rust compiler versions, so the issue reports that I linked will not be resolved with it. It looks like remap-path-scope is working as intended only since
https://github.com/rust-lang/rust/pull/149709, so Rust 1.94+ (I haven't double-checked the exact Rust version, but in latest nightly Rust, the remap is working as intended).
Linked the stabilization and noted the fix 1.94 PR that we likely need and this thread in the -Zremap-path-scope entry in
https://github.com/Rust-for-Linux/linux/issues/2.
And since it is now stabilized, moved to
https://github.com/Rust-for-Linux/linux/issues/1223, which I just had to create due to GitHub's 64k limit on issue text...
Cheers,
Miguel