Skip to contentSkip to conversation

bpf: preload: Fix build error when O= is set

RFC PATCH In review David Gow 4 participants 6 messages Nov 19, 2020 – Dec 17, 2020

Subsystems (4)
lore
Share & tools
lore

David Gow

author
#0 lore raw

RFC PATCH 1/1 bpf: preload: Fix build error when O= is set

If BPF_PRELOAD is enabled, and an out-of-tree build is requested with make O=<path>, compilation seems to fail with:

tools/scripts/Makefile.include:4: *** O=.kunit does not exist. Stop. make[4]: *** [../kernel/bpf/preload/Makefile:8: kernel/bpf/preload/libbpf.a] Error 2 make[3]: *** [../scripts/Makefile.build:500: kernel/bpf/preload] Error 2 make[2]: *** [../scripts/Makefile.build:500: kernel/bpf] Error 2 make[2]: *** Waiting for unfinished jobs....
make[1]: *** [.../Makefile:1799: kernel] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:185: __sub-make] Error 2

By the looks of things, this is because the (relative path) O= passed on the command line is being passed to the libbpf Makefile, which then can't find the directory. Given OUTPUT= is being passed anyway, we can work around this by explicitly setting an empty O=, which will be ignored in favour of OUTPUT= in tools/scripts/Makefile.include.

---

Hi all,

I'm not 100% sure this is the correct fix here -- it seems to work for me, and makes some sense, but let me know if there's a better way.

One other thing worth noting is that I've been hitting this with make allyesconfig on ARCH=um, but there's a comment in the Kconfig suggesting that, because BPF_PRELOAD depends on !COMPILE_TEST, that maybe it shouldn't be being built at all. I figured that it was worth trying to fix this anyway.

Cheers,
-- David

 kernel/bpf/preload/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/bpf/preload/Makefile +1 -1
index 23ee310b6eb4..39848d296097 100644
@@ -5,7 +5,7 @@ LIBBPF_A = $(obj)/libbpf.a
 LIBBPF_OUT = $(abspath $(obj))
 
 $(LIBBPF_A):
-	$(Q)$(MAKE) -C $(LIBBPF_SRCS) OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
+	$(Q)$(MAKE) -C $(LIBBPF_SRCS) O= OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
 
 userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi \
 	-I $(srctree)/tools/lib/ -Wno-unused-result
Signature · 1 line
2.29.2.454.gaff20da3a2-goog
Reviewed-by on v1 1/1
#1 lore raw
Quoted 13 lines · David Gow

If BPF_PRELOAD is enabled, and an out-of-tree build is requested with make O=<path>, compilation seems to fail with:

tools/scripts/Makefile.include:4: *** O=.kunit does not exist. Stop. make[4]: *** [../kernel/bpf/preload/Makefile:8: kernel/bpf/preload/libbpf.a] Error 2 make[3]: *** [../scripts/Makefile.build:500: kernel/bpf/preload] Error 2 make[2]: *** [../scripts/Makefile.build:500: kernel/bpf] Error 2 make[2]: *** Waiting for unfinished jobs....
make[1]: *** [.../Makefile:1799: kernel] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:185: __sub-make] Error 2

Quoted 7 lines

By the looks of things, this is because the (relative path) O= passed on the command line is being passed to the libbpf Makefile, which then can't find the directory. Given OUTPUT= is being passed anyway, we can work around this by explicitly setting an empty O=, which will be ignored in favour of OUTPUT= in tools/scripts/Makefile.include.

Seems sensible to me. I have no strong feeling as to whether we just turn this off on UML or whether we do the fix you proposed here though. Nevertheless, I would like to see *some* fix go in before v5.10 is released.

#2 lore raw
In reply to David Gow
Quoted 13 lines · David Gow

If BPF_PRELOAD is enabled, and an out-of-tree build is requested with make O=<path>, compilation seems to fail with:

tools/scripts/Makefile.include:4: *** O=.kunit does not exist. Stop. make[4]: *** [../kernel/bpf/preload/Makefile:8: kernel/bpf/preload/libbpf.a] Error 2 make[3]: *** [../scripts/Makefile.build:500: kernel/bpf/preload] Error 2 make[2]: *** [../scripts/Makefile.build:500: kernel/bpf] Error 2 make[2]: *** Waiting for unfinished jobs....
make[1]: *** [.../Makefile:1799: kernel] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:185: __sub-make] Error 2

Quoted 5 lines

By the looks of things, this is because the (relative path) O= passed on the command line is being passed to the libbpf Makefile, which then can't find the directory. Given OUTPUT= is being passed anyway, we can work around this by explicitly setting an empty O=, which will be ignored in favour of OUTPUT= in tools/scripts/Makefile.include.

Strange, but I can't repro it. I use make O=<absolute path> all the time with no issues. I just tried specifically with a make O=.build, where .build is inside Linux repo, and it still worked fine. See also be40920fbf10 ("tools: Let O= makes handle a relative path with -C option") which was supposed to address such an issue. So I'm wondering what exactly is causing this problem.

Quoted 38 lines

---

Hi all,

I'm not 100% sure this is the correct fix here -- it seems to work for me, and makes some sense, but let me know if there's a better way.

One other thing worth noting is that I've been hitting this with make allyesconfig on ARCH=um, but there's a comment in the Kconfig suggesting that, because BPF_PRELOAD depends on !COMPILE_TEST, that maybe it shouldn't be being built at all. I figured that it was worth trying to fix this anyway.

Cheers,
-- David

 kernel/bpf/preload/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/bpf/preload/Makefile +1 -2
index 23ee310b6eb4..39848d296097 100644
@@ -5,7 +5,7 @@ LIBBPF_A = $(obj)/libbpf.a
 LIBBPF_OUT = $(abspath $(obj))
 
 $(LIBBPF_A):
-       $(Q)$(MAKE) -C $(LIBBPF_SRCS) OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
+       $(Q)$(MAKE) -C $(LIBBPF_SRCS) O= OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
 
 userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi \
        -I $(srctree)/tools/lib/ -Wno-unused-result
--

2.29.2.454.gaff20da3a2-goog

author
#3 lore raw
Quoted 21 lines · Andrii Nakryiko
Quoted 13 lines · David Gow

If BPF_PRELOAD is enabled, and an out-of-tree build is requested with make O=<path>, compilation seems to fail with:

tools/scripts/Makefile.include:4: *** O=.kunit does not exist. Stop. make[4]: *** [../kernel/bpf/preload/Makefile:8: kernel/bpf/preload/libbpf.a] Error 2 make[3]: *** [../scripts/Makefile.build:500: kernel/bpf/preload] Error 2 make[2]: *** [../scripts/Makefile.build:500: kernel/bpf] Error 2 make[2]: *** Waiting for unfinished jobs....
make[1]: *** [.../Makefile:1799: kernel] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:185: __sub-make] Error 2

Quoted 5 lines

By the looks of things, this is because the (relative path) O= passed on the command line is being passed to the libbpf Makefile, which then can't find the directory. Given OUTPUT= is being passed anyway, we can work around this by explicitly setting an empty O=, which will be ignored in favour of OUTPUT= in tools/scripts/Makefile.include.

Quoted 7 lines

Strange, but I can't repro it. I use make O=<absolute path> all the time with no issues. I just tried specifically with a make O=.build, where .build is inside Linux repo, and it still worked fine. See also be40920fbf10 ("tools: Let O= makes handle a relative path with -C option") which was supposed to address such an issue. So I'm wondering what exactly is causing this problem.

[+ linux-um list]

Hmm... From a quick check, I can't reproduce this on x86, so it's possibly a UML-specific issue.

The problem here seems to be that $PWD is, for whatever reason, equal to the srcdir on x86, but not on UML. In general, $PWD behaves pretty weirdly -- I don't fully understand it -- but if I add a tactical "PWD := $(shell pwd)" or use $(CURDIR) instead, the issue shows up on x86 as well. I guess this is because PWD only gets updated when set by a shell or something, and UML does this somewhere?

Thoughts?

Cheers,
-- David

Quoted 38 lines
Quoted 38 lines

---

Hi all,

I'm not 100% sure this is the correct fix here -- it seems to work for me, and makes some sense, but let me know if there's a better way.

One other thing worth noting is that I've been hitting this with make allyesconfig on ARCH=um, but there's a comment in the Kconfig suggesting that, because BPF_PRELOAD depends on !COMPILE_TEST, that maybe it shouldn't be being built at all. I figured that it was worth trying to fix this anyway.

Cheers,
-- David

 kernel/bpf/preload/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/bpf/preload/Makefile +1 -2
index 23ee310b6eb4..39848d296097 100644
@@ -5,7 +5,7 @@ LIBBPF_A = $(obj)/libbpf.a
 LIBBPF_OUT = $(abspath $(obj))
 
 $(LIBBPF_A):
-       $(Q)$(MAKE) -C $(LIBBPF_SRCS) OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
+       $(Q)$(MAKE) -C $(LIBBPF_SRCS) O= OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
 
 userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi \
        -I $(srctree)/tools/lib/ -Wno-unused-result
--

2.29.2.454.gaff20da3a2-goog

#4 lore raw

2020-11-21 17:48 UTC+0800 ~ David Gow <[email protected]>

Quoted 42 lines
Quoted 21 lines · Andrii Nakryiko
Quoted 13 lines · David Gow

If BPF_PRELOAD is enabled, and an out-of-tree build is requested with make O=<path>, compilation seems to fail with:

tools/scripts/Makefile.include:4: *** O=.kunit does not exist. Stop. make[4]: *** [../kernel/bpf/preload/Makefile:8: kernel/bpf/preload/libbpf.a] Error 2 make[3]: *** [../scripts/Makefile.build:500: kernel/bpf/preload] Error 2 make[2]: *** [../scripts/Makefile.build:500: kernel/bpf] Error 2 make[2]: *** Waiting for unfinished jobs....
make[1]: *** [.../Makefile:1799: kernel] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:185: __sub-make] Error 2

Quoted 5 lines

By the looks of things, this is because the (relative path) O= passed on the command line is being passed to the libbpf Makefile, which then can't find the directory. Given OUTPUT= is being passed anyway, we can work around this by explicitly setting an empty O=, which will be ignored in favour of OUTPUT= in tools/scripts/Makefile.include.

Quoted 7 lines

Strange, but I can't repro it. I use make O=<absolute path> all the time with no issues. I just tried specifically with a make O=.build, where .build is inside Linux repo, and it still worked fine. See also be40920fbf10 ("tools: Let O= makes handle a relative path with -C option") which was supposed to address such an issue. So I'm wondering what exactly is causing this problem.

[+ linux-um list]

Hmm... From a quick check, I can't reproduce this on x86, so it's possibly a UML-specific issue.

The problem here seems to be that $PWD is, for whatever reason, equal to the srcdir on x86, but not on UML. In general, $PWD behaves pretty weirdly -- I don't fully understand it -- but if I add a tactical "PWD := $(shell pwd)" or use $(CURDIR) instead, the issue shows up on x86 as well. I guess this is because PWD only gets updated when set by a shell or something, and UML does this somewhere?

Quoted 4 lines

Thoughts?

Cheers,
-- David

Hi David, Andrii,

David, did you use a different command for building for UML and x86? I'm asking because I reproduce on x86, but only for some targets, in particular when I tried bindeb-pkg.

With "make O=.build vmlinux", I have:
- $(O) for "dummy" check in tools/scripts/Makefile.include set to /linux/.build
- $(PWD) for same check set to /linux/tools
- Since $(O) is an absolute path, the "dummy" check passes

With "make O=.build bindeb-pkg", I have instead:
- $(O) set to .build (relative path)
- $(PWD) set to /linux/.build
- "dummy" check changes to /linux/.build and searches for .build in it, which fails and aborts the build

(tools/scripts/Makefile.include is included from libbpf's Makefile, called from kernel/bpf/preload/Makefile.)

I'm not sure how exactly the bindeb-pkg target ends up passing these values.

For what it's worth, I have been solving this (before finding this thread) with a fix close to yours, I pass "O=$(abspath .)" on the command line for building libbpf in kernel/bpf/preload/Makefile. It looked consistent to me with the "tools/:" target from the main Makefile, where "O=$(abspath $(objtree))" is passed (and $(objtree) is ".").

I hope this helps,
Quentin

_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um

author
#5 lore raw
Quoted 46 lines · Quentin Monnet

2020-11-21 17:48 UTC+0800 ~ David Gow <[email protected]>

Quoted 44 lines
Quoted 21 lines · Andrii Nakryiko
Quoted 13 lines · David Gow

If BPF_PRELOAD is enabled, and an out-of-tree build is requested with make O=<path>, compilation seems to fail with:

tools/scripts/Makefile.include:4: *** O=.kunit does not exist. Stop. make[4]: *** [../kernel/bpf/preload/Makefile:8: kernel/bpf/preload/libbpf.a] Error 2 make[3]: *** [../scripts/Makefile.build:500: kernel/bpf/preload] Error 2 make[2]: *** [../scripts/Makefile.build:500: kernel/bpf] Error 2 make[2]: *** Waiting for unfinished jobs....
make[1]: *** [.../Makefile:1799: kernel] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:185: __sub-make] Error 2

Quoted 5 lines

By the looks of things, this is because the (relative path) O= passed on the command line is being passed to the libbpf Makefile, which then can't find the directory. Given OUTPUT= is being passed anyway, we can work around this by explicitly setting an empty O=, which will be ignored in favour of OUTPUT= in tools/scripts/Makefile.include.

Quoted 7 lines

Strange, but I can't repro it. I use make O=<absolute path> all the time with no issues. I just tried specifically with a make O=.build, where .build is inside Linux repo, and it still worked fine. See also be40920fbf10 ("tools: Let O= makes handle a relative path with -C option") which was supposed to address such an issue. So I'm wondering what exactly is causing this problem.

[+ linux-um list]

Hmm... From a quick check, I can't reproduce this on x86, so it's possibly a UML-specific issue.

The problem here seems to be that $PWD is, for whatever reason, equal to the srcdir on x86, but not on UML. In general, $PWD behaves pretty weirdly -- I don't fully understand it -- but if I add a tactical "PWD := $(shell pwd)" or use $(CURDIR) instead, the issue shows up on x86 as well. I guess this is because PWD only gets updated when set by a shell or something, and UML does this somewhere?

Thoughts?

Quoted 8 lines
Quoted 2 lines

Cheers,
-- David

Hi David, Andrii,

David, did you use a different command for building for UML and x86? I'm asking because I reproduce on x86, but only for some targets, in particular when I tried bindeb-pkg.

I just ran "make ARCH={x86,um} O=.bpftest", with defconfig + enabling BPF_PRELOAD and its dependencies. UML fails, x86 works. (Though I can reproduce the failure if I make bindeb-pkg on x86).

(It also shows up when building UML with the allyesconfig-based KUnit alltests option by running "./tools/testing/kunit/kunit.py run --alltests", though this understandably takes a long time and is less obvious)

Quoted 13 lines

With "make O=.build vmlinux", I have:
- $(O) for "dummy" check in tools/scripts/Makefile.include set to /linux/.build
- $(PWD) for same check set to /linux/tools
- Since $(O) is an absolute path, the "dummy" check passes

With "make O=.build bindeb-pkg", I have instead:
- $(O) set to .build (relative path)
- $(PWD) set to /linux/.build
- "dummy" check changes to /linux/.build and searches for .build in it, which fails and aborts the build

Quoted 4 lines

(tools/scripts/Makefile.include is included from libbpf's Makefile, called from kernel/bpf/preload/Makefile.)

I'm not sure how exactly the bindeb-pkg target ends up passing these values.

Yeah: I haven't been able to find where uml is changing them either:
I'm assuming there's something which changes directory and/or spawns a shell/recursive make to change $(PWD) or something.

Quoted 5 lines

For what it's worth, I have been solving this (before finding this thread) with a fix close to yours, I pass "O=$(abspath .)" on the command line for building libbpf in kernel/bpf/preload/Makefile. It looked consistent to me with the "tools/:" target from the main Makefile, where "O=$(abspath $(objtree))" is passed (and $(objtree) is ".").

Given that there are several targets being broken here, it's probably worth having a fix like this which overrides O= rather than trying to hunt down every target which could change $(PWD). I don't particularly mind whether we use O= or O=$(abspath .), both are working in the UML usecase as well.

Does anyone object to basically accepting either this patch as-is, or using O=$(abspath .)?

Cheers,
-- David

_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um