Skip to contentSkip to conversation

possible dependency error?

DISCUSSION Bjorn Helgaas 3 participants 7 messages May 17, 2023 – Jul 2, 2024

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

This is on v6.4-rc1. I fat-fingered the make target (I intended "pciehp.o", not "pciehp.c"), then interrupted the build when I noticed my mistake:

06:04:15 ~/linux (hotplug)$ make drivers/pci/hotplug/pciehp.c
SYNC include/config/auto.conf.cmd
^Cmake: *** [include/config/auto.conf.cmd] Deleting file 'include/generated/rustc_cfg'
make: *** [include/config/auto.conf.cmd] Deleting file 'include/generated/autoconf.h' make[2]: *** [scripts/kconfig/Makefile:77: syncconfig] Interrupt make[1]: *** [Makefile:692: syncconfig] Interrupt
make: *** [Makefile:793: include/config/auto.conf.cmd] Interrupt

Subsequent builds now fail ("pciehp.o" is *also* an incorrect target, but doesn't seem related to the error):

06:04:22 ~/linux (hotplug)$ make drivers/pci/hotplug/pciehp.o

    SYNC    include/config/auto.conf.cmd
    UPD     include/config/kernel.release
    UPD     include/generated/utsrelease.h
    UPD     include/generated/compile.h
    CC      scripts/mod/empty.o
    MKELF   scripts/mod/elfconfig.h
    HOSTCC  scripts/mod/modpost.o
    CC      scripts/mod/devicetable-offsets.s
    HOSTCC  scripts/mod/file2alias.o
    HOSTCC  scripts/mod/sumversion.o
    HOSTLD  scripts/mod/modpost
    CC      kernel/bounds.s
    CC      arch/x86/kernel/asm-offsets.s
    CALL    scripts/checksyscalls.sh

DESCEND objtool
HOSTCC /home/bjorn/linux/tools/objtool/fixdep.o
HOSTLD /home/bjorn/linux/tools/objtool/fixdep-in.o
LINK /home/bjorn/linux/tools/objtool/fixdep
make[4]: *** No rule to make target '/usr/include/x86_64-linux-gnu/bits/sys_errlist.h', needed by '/home/bjorn/linux/tools/objtool/libsubcmd/exec-cmd.o'. Stop. make[3]: *** [Makefile:80: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd-in.o] Error 2
make[2]: *** [Makefile:78: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd.a] Error 2
make[1]: *** [Makefile:73: objtool] Error 2
make: *** [Makefile:1440: tools/objtool] Error 2

I finally got the right target, but the build still fails:

06:04:39 ~/linux (hotplug)$ make drivers/pci/hotplug/
CALL scripts/checksyscalls.sh
DESCEND objtool
make[4]: *** No rule to make target '/usr/include/x86_64-linux-gnu/bits/sys_errlist.h', needed by '/home/bjorn/linux/tools/objtool/libsubcmd/exec-cmd.o'. Stop. make[3]: *** [Makefile:80: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd-in.o] Error 2
make[2]: *** [Makefile:78: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd.a] Error 2
make[1]: *** [Makefile:73: objtool] Error 2
make: *** [Makefile:1440: tools/objtool] Error 2

After "make distclean", everything works as expected, so maybe this is just the expected behavior after my initial user error? I dunno; it seemed surprising. Just FYI.

Bjorn

#1 lore raw

(+CC: Josh Poimboeuf,Peter Zijlstra, objtool maintainer)

Quoted 52 lines · Bjorn Helgaas

This is on v6.4-rc1. I fat-fingered the make target (I intended "pciehp.o", not "pciehp.c"), then interrupted the build when I noticed my mistake:

06:04:15 ~/linux (hotplug)$ make drivers/pci/hotplug/pciehp.c
SYNC include/config/auto.conf.cmd
^Cmake: *** [include/config/auto.conf.cmd] Deleting file 'include/generated/rustc_cfg'
make: *** [include/config/auto.conf.cmd] Deleting file 'include/generated/autoconf.h' make[2]: *** [scripts/kconfig/Makefile:77: syncconfig] Interrupt make[1]: *** [Makefile:692: syncconfig] Interrupt
make: *** [Makefile:793: include/config/auto.conf.cmd] Interrupt

Subsequent builds now fail ("pciehp.o" is *also* an incorrect target, but doesn't seem related to the error):

06:04:22 ~/linux (hotplug)$ make drivers/pci/hotplug/pciehp.o

    SYNC    include/config/auto.conf.cmd
    UPD     include/config/kernel.release
    UPD     include/generated/utsrelease.h
    UPD     include/generated/compile.h
    CC      scripts/mod/empty.o
    MKELF   scripts/mod/elfconfig.h
    HOSTCC  scripts/mod/modpost.o
    CC      scripts/mod/devicetable-offsets.s
    HOSTCC  scripts/mod/file2alias.o
    HOSTCC  scripts/mod/sumversion.o
    HOSTLD  scripts/mod/modpost
    CC      kernel/bounds.s
    CC      arch/x86/kernel/asm-offsets.s
    CALL    scripts/checksyscalls.sh

DESCEND objtool
HOSTCC /home/bjorn/linux/tools/objtool/fixdep.o
HOSTLD /home/bjorn/linux/tools/objtool/fixdep-in.o
LINK /home/bjorn/linux/tools/objtool/fixdep
make[4]: *** No rule to make target '/usr/include/x86_64-linux-gnu/bits/sys_errlist.h', needed by '/home/bjorn/linux/tools/objtool/libsubcmd/exec-cmd.o'. Stop. make[3]: *** [Makefile:80: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd-in.o] Error 2
make[2]: *** [Makefile:78: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd.a] Error 2
make[1]: *** [Makefile:73: objtool] Error 2
make: *** [Makefile:1440: tools/objtool] Error 2

I finally got the right target, but the build still fails:

06:04:39 ~/linux (hotplug)$ make drivers/pci/hotplug/
CALL scripts/checksyscalls.sh
DESCEND objtool
make[4]: *** No rule to make target '/usr/include/x86_64-linux-gnu/bits/sys_errlist.h', needed by '/home/bjorn/linux/tools/objtool/libsubcmd/exec-cmd.o'. Stop. make[3]: *** [Makefile:80: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd-in.o] Error 2
make[2]: *** [Makefile:78: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd.a] Error 2
make[1]: *** [Makefile:73: objtool] Error 2
make: *** [Makefile:1440: tools/objtool] Error 2

Quoted 5 lines

After "make distclean", everything works as expected, so maybe this is just the expected behavior after my initial user error? I dunno; it seemed surprising. Just FYI.

Bjorn

I do not know what is happening on your build machine, but judging from the error log, something went wrong while building objtool.

objtool Makefile is not a part of Kbuild.
The maintainers of objtool may have some insight.

BTW, I could not reproduce the issue.
I could not find anything weird from the Kbuild perspective so far. The errors in the single target builds are expected.

This is what I did.

$ git clean -fdx
$ make defconfig
$ make drivers/pci/hotplug/pciehp.c
    -> Fail. This is expected behavior
$ make drivers/pci/hotplug/pciehp.o
    -> Fail. This is also expected behavior
       because CONFIG_HOTPLUG_PCI_PCIE is unset
$ make drivers/pci/hotplug/
    -> success for me

Is this the same command sequence as you did?

Signature · 2 lines
Best Regards
Masahiro Yamada
#2 lore raw

Necromancing an old thread, since it identifies the same problem I've been poking at recently:

Quoted 58 lines · Masahiro Yamada

(+CC: Josh Poimboeuf,Peter Zijlstra, objtool maintainer)

Quoted 55 lines · Bjorn Helgaas

This is on v6.4-rc1. I fat-fingered the make target (I intended "pciehp.o", not "pciehp.c"), then interrupted the build when I noticed my mistake:

06:04:15 ~/linux (hotplug)$ make drivers/pci/hotplug/pciehp.c
SYNC include/config/auto.conf.cmd
^Cmake: *** [include/config/auto.conf.cmd] Deleting file 'include/generated/rustc_cfg'
make: *** [include/config/auto.conf.cmd] Deleting file 'include/generated/autoconf.h' make[2]: *** [scripts/kconfig/Makefile:77: syncconfig] Interrupt make[1]: *** [Makefile:692: syncconfig] Interrupt
make: *** [Makefile:793: include/config/auto.conf.cmd] Interrupt

Subsequent builds now fail ("pciehp.o" is *also* an incorrect target, but doesn't seem related to the error):

06:04:22 ~/linux (hotplug)$ make drivers/pci/hotplug/pciehp.o

    SYNC    include/config/auto.conf.cmd
    UPD     include/config/kernel.release
    UPD     include/generated/utsrelease.h
    UPD     include/generated/compile.h
    CC      scripts/mod/empty.o
    MKELF   scripts/mod/elfconfig.h
    HOSTCC  scripts/mod/modpost.o
    CC      scripts/mod/devicetable-offsets.s
    HOSTCC  scripts/mod/file2alias.o
    HOSTCC  scripts/mod/sumversion.o
    HOSTLD  scripts/mod/modpost
    CC      kernel/bounds.s
    CC      arch/x86/kernel/asm-offsets.s
    CALL    scripts/checksyscalls.sh

DESCEND objtool
HOSTCC /home/bjorn/linux/tools/objtool/fixdep.o
HOSTLD /home/bjorn/linux/tools/objtool/fixdep-in.o
LINK /home/bjorn/linux/tools/objtool/fixdep
make[4]: *** No rule to make target '/usr/include/x86_64-linux-gnu/bits/sys_errlist.h', needed by '/home/bjorn/linux/tools/objtool/libsubcmd/exec-cmd.o'. Stop. make[3]: *** [Makefile:80: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd-in.o] Error 2
make[2]: *** [Makefile:78: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd.a] Error 2
make[1]: *** [Makefile:73: objtool] Error 2
make: *** [Makefile:1440: tools/objtool] Error 2

I finally got the right target, but the build still fails:

06:04:39 ~/linux (hotplug)$ make drivers/pci/hotplug/
CALL scripts/checksyscalls.sh
DESCEND objtool
make[4]: *** No rule to make target '/usr/include/x86_64-linux-gnu/bits/sys_errlist.h', needed by '/home/bjorn/linux/tools/objtool/libsubcmd/exec-cmd.o'. Stop. make[3]: *** [Makefile:80: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd-in.o] Error 2
make[2]: *** [Makefile:78: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd.a] Error 2
make[1]: *** [Makefile:73: objtool] Error 2
make: *** [Makefile:1440: tools/objtool] Error 2

After "make distclean", everything works as expected, so maybe this is just the expected behavior after my initial user error? I dunno; it seemed surprising. Just FYI.

I believe we've been hitting a similar issue at $JOB, which looks like the following (this is on a 5.15-ish kernel, but AFAICT everything is still relevant):

make[5]: *** No rule to make target '[...]/tools/include/linux/compiler.h', needed by '[...]/tools/bpf/resolve_btfids/libsubcmd/exec-cmd.o'. Stop. make[4]: *** [Makefile:59: [...]/tools/bpf/resolve_btfids/libsubcmd/libsubcmd-in.o] Error 2
make[3]: *** [Makefile:45: [...]/tools/bpf/resolve_btfids//libsubcmd/libsubcmd.a] Error 2
make[2]: *** [Makefile:72: bpf/resolve_btfids] Error 2
make[1]: *** [[...]/Makefile:1401: tools/bpf/resolve_btfids] Error 2

This particular case happens for us when the source tree is moving, but we're sharing an O= cache. This may or may not be a good idea, but AFAICT there still is a real bug underneath, which I explore below.

Quoted 6 lines

I do not know what is happening on your build machine, but judging from the error log, something went wrong while building objtool.

objtool Makefile is not a part of Kbuild.
The maintainers of objtool may have some insight.

I'm no maintainer, but I found that the .exec-cmd.o.cmd dep file is generated incorrectly due to improper fixdep dependencies:

$ head -2 [...]/tools/bpf/resolve_btfids/libsubcmd/.exec-cmd.o.cmd
# cannot find fixdep ([...]/tools/bpf/resolve_btfids/libsubcmd//fixdep)
# using basic dep data

Now, this soft error is normally OK, as long as you don't have any missing or moved headers. But if these moved around, then normally the fixdep'd dependencies would help rebuild (and regenerate .cmd files with new paths) silently.

The bad .cmd file is reliably reproduced by:

# for an easier-to-build target that also builds libsubcmd:
cd tools/objtool
# for maximum cleanliness:
git clean -xfd ..
make
head -2 libsubcmd/.exec-cmd.o.cmd

(NB: if you look hard enough, you'll notice that we have a similar "cannot find fixdep" error for tools/.../.fixdep.o.cmd too. I have some analysis at https://issuetracker.google.com/313508829#comment32 -- this one is publicly accessible -- but its solution would be more complex. I may raise a separate thread.)

The following patch fixes libsubcmd stuff for me. I can resubmit in a proper patch form if desired, or feel free to scrape it as-is.

---

tools/lib/subcmd/Makefile +1 -1
index b87213263a5e..59b09f280e49 100644
@@ -76,7 +76,7 @@ include $(srctree)/tools/build/Makefile.include
 
 all: fixdep $(LIBFILE)
 
-$(SUBCMD_IN): FORCE
+$(SUBCMD_IN): fixdep FORCE
 	@$(MAKE) $(build)=libsubcmd
 
 $(LIBFILE): $(SUBCMD_IN)
#3 lore raw
Quoted 123 lines · Brian Norris

Necromancing an old thread, since it identifies the same problem I've been poking at recently:

Quoted 58 lines · Masahiro Yamada

(+CC: Josh Poimboeuf,Peter Zijlstra, objtool maintainer)

Quoted 55 lines · Bjorn Helgaas

This is on v6.4-rc1. I fat-fingered the make target (I intended "pciehp.o", not "pciehp.c"), then interrupted the build when I noticed my mistake:

06:04:15 ~/linux (hotplug)$ make drivers/pci/hotplug/pciehp.c
SYNC include/config/auto.conf.cmd
^Cmake: *** [include/config/auto.conf.cmd] Deleting file 'include/generated/rustc_cfg'
make: *** [include/config/auto.conf.cmd] Deleting file 'include/generated/autoconf.h' make[2]: *** [scripts/kconfig/Makefile:77: syncconfig] Interrupt make[1]: *** [Makefile:692: syncconfig] Interrupt
make: *** [Makefile:793: include/config/auto.conf.cmd] Interrupt

Subsequent builds now fail ("pciehp.o" is *also* an incorrect target, but doesn't seem related to the error):

06:04:22 ~/linux (hotplug)$ make drivers/pci/hotplug/pciehp.o

    SYNC    include/config/auto.conf.cmd
    UPD     include/config/kernel.release
    UPD     include/generated/utsrelease.h
    UPD     include/generated/compile.h
    CC      scripts/mod/empty.o
    MKELF   scripts/mod/elfconfig.h
    HOSTCC  scripts/mod/modpost.o
    CC      scripts/mod/devicetable-offsets.s
    HOSTCC  scripts/mod/file2alias.o
    HOSTCC  scripts/mod/sumversion.o
    HOSTLD  scripts/mod/modpost
    CC      kernel/bounds.s
    CC      arch/x86/kernel/asm-offsets.s
    CALL    scripts/checksyscalls.sh

DESCEND objtool
HOSTCC /home/bjorn/linux/tools/objtool/fixdep.o
HOSTLD /home/bjorn/linux/tools/objtool/fixdep-in.o
LINK /home/bjorn/linux/tools/objtool/fixdep
make[4]: *** No rule to make target '/usr/include/x86_64-linux-gnu/bits/sys_errlist.h', needed by '/home/bjorn/linux/tools/objtool/libsubcmd/exec-cmd.o'. Stop. make[3]: *** [Makefile:80: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd-in.o] Error 2
make[2]: *** [Makefile:78: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd.a] Error 2
make[1]: *** [Makefile:73: objtool] Error 2
make: *** [Makefile:1440: tools/objtool] Error 2

I finally got the right target, but the build still fails:

06:04:39 ~/linux (hotplug)$ make drivers/pci/hotplug/
CALL scripts/checksyscalls.sh
DESCEND objtool
make[4]: *** No rule to make target '/usr/include/x86_64-linux-gnu/bits/sys_errlist.h', needed by '/home/bjorn/linux/tools/objtool/libsubcmd/exec-cmd.o'. Stop. make[3]: *** [Makefile:80: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd-in.o] Error 2
make[2]: *** [Makefile:78: /home/bjorn/linux/tools/objtool/libsubcmd/libsubcmd.a] Error 2
make[1]: *** [Makefile:73: objtool] Error 2
make: *** [Makefile:1440: tools/objtool] Error 2

After "make distclean", everything works as expected, so maybe this is just the expected behavior after my initial user error? I dunno; it seemed surprising. Just FYI.

I believe we've been hitting a similar issue at $JOB, which looks like the following (this is on a 5.15-ish kernel, but AFAICT everything is still relevant):

make[5]: *** No rule to make target '[...]/tools/include/linux/compiler.h', needed by '[...]/tools/bpf/resolve_btfids/libsubcmd/exec-cmd.o'. Stop. make[4]: *** [Makefile:59: [...]/tools/bpf/resolve_btfids/libsubcmd/libsubcmd-in.o] Error 2
make[3]: *** [Makefile:45: [...]/tools/bpf/resolve_btfids//libsubcmd/libsubcmd.a] Error 2
make[2]: *** [Makefile:72: bpf/resolve_btfids] Error 2
make[1]: *** [[...]/Makefile:1401: tools/bpf/resolve_btfids] Error 2

This particular case happens for us when the source tree is moving, but we're sharing an O= cache. This may or may not be a good idea, but AFAICT there still is a real bug underneath, which I explore below.

Quoted 6 lines

I do not know what is happening on your build machine, but judging from the error log, something went wrong while building objtool.

objtool Makefile is not a part of Kbuild.
The maintainers of objtool may have some insight.

I'm no maintainer, but I found that the .exec-cmd.o.cmd dep file is generated incorrectly due to improper fixdep dependencies:

$ head -2 [...]/tools/bpf/resolve_btfids/libsubcmd/.exec-cmd.o.cmd
# cannot find fixdep ([...]/tools/bpf/resolve_btfids/libsubcmd//fixdep)
# using basic dep data

Now, this soft error is normally OK, as long as you don't have any missing or moved headers. But if these moved around, then normally the fixdep'd dependencies would help rebuild (and regenerate .cmd files with new paths) silently.

The bad .cmd file is reliably reproduced by:

# for an easier-to-build target that also builds libsubcmd:
cd tools/objtool
# for maximum cleanliness:
git clean -xfd ..
make
head -2 libsubcmd/.exec-cmd.o.cmd

(NB: if you look hard enough, you'll notice that we have a similar "cannot find fixdep" error for tools/.../.fixdep.o.cmd too. I have some analysis at https://issuetracker.google.com/313508829#comment32 -- this one is publicly accessible -- but its solution would be more complex. I may raise a separate thread.)

The following patch fixes libsubcmd stuff for me. I can resubmit in a proper patch form if desired, or feel free to scrape it as-is.

---

tools/lib/subcmd/Makefile +0 -0
index b87213263a5e..59b09f280e49 100644

tools/lib/subcmd/Makefile include $(srctree)/tools/build/Makefile.include · line 82

Quoted 8 lines
tools/lib/subcmd/Makefile +1 -1
@@ -76,7 +76,7 @@ include $(srctree)/tools/build/Makefile.include
 
 all: fixdep $(LIBFILE)
 
-$(SUBCMD_IN): FORCE
+$(SUBCMD_IN): fixdep FORCE
        @$(MAKE) $(build)=libsubcmd
 
 $(LIBFILE): $(SUBCMD_IN)

I may not fully understand the design policy of the tools/ build system, but this fix is presumably correct because the 'fixdep' binary is needed in each sub-directory for it to work correctly.

tools/bpf/resolve_btfids/libsubcmd/.exec-cmd.o.cmd must be generated by tools/bpf/resolve_btfids/libsubcmd/fixdep instead of by tools/bpf/resolve_btfids/fixdep.

But, fixing tools/lib/subcmd/Makefile is not enough.

*.cmd files under tools/bpf/resolve_btfids/libbpf/staticobjs/ are broken for the same reason.
So, this is fundamentally broken in many places.

And, as you noted, there is no easy way to fix .fixdep.o.cmd

Your description in
https://issuetracker.google.com/issues/313508829#comment32
is all correct.

"can we just use Kbuild?" is a good question. I do not understand why they use fragile build systems, where obviously they cannot do it correctly.

In fact, I submitted a patch to migrate objtool to Kbuild because that fixes all the issues cleanly.

The objtool maintainers rejected it.
https://lore.kernel.org/linux-kbuild/[email protected]/

Not only the build system.
He also refused to participate in the standard Documentation directory.
tools/objtool/Documentation/objtool.txt still resides in its own directory.

Signature · 2 lines
Best Regards
Masahiro Yamada
#4 lore raw

Hi Masahiro,

Thanks for the reply. I've been away for a bit, but I've poked at a few more things now.

Quoted 23 lines · Masahiro Yamada

tools/lib/subcmd/Makefile include $(srctree)/tools/build/Makefile.include · line 82

Quoted 11 lines · Brian Norris
tools/lib/subcmd/Makefile +1 -1
@@ -76,7 +76,7 @@ include $(srctree)/tools/build/Makefile.include
 
 all: fixdep $(LIBFILE)
 
-$(SUBCMD_IN): FORCE
+$(SUBCMD_IN): fixdep FORCE
        @$(MAKE) $(build)=libsubcmd
 
 $(LIBFILE): $(SUBCMD_IN)

I may not fully understand the design policy of the tools/ build system, but this fix is presumably correct because the 'fixdep' binary is needed in each sub-directory for it to work correctly.

tools/bpf/resolve_btfids/libsubcmd/.exec-cmd.o.cmd must be generated by tools/bpf/resolve_btfids/libsubcmd/fixdep instead of by tools/bpf/resolve_btfids/fixdep.

Quoted 5 lines

But, fixing tools/lib/subcmd/Makefile is not enough.

*.cmd files under tools/bpf/resolve_btfids/libbpf/staticobjs/ are broken for the same reason.
So, this is fundamentally broken in many places.

I think I hacked something that works there too. It gets a bit uglier, but not too bad IMO.

Quoted 1 line

And, as you noted, there is no easy way to fix .fixdep.o.cmd

I haven't come up with a *great* solution, but I came up with something that works for the most part, by circumventing the normal Build / Makefile.build split. It's getting pretty ugly too though...

Quoted 13 lines

Your description in
https://issuetracker.google.com/issues/313508829#comment32
is all correct.

"can we just use Kbuild?" is a good question. I do not understand why they use fragile build systems, where obviously they cannot do it correctly.

In fact, I submitted a patch to migrate objtool to Kbuild because that fixes all the issues cleanly.

Quoted 8 lines

The objtool maintainers rejected it.
https://lore.kernel.org/linux-kbuild/[email protected]/

Not only the build system.
He also refused to participate in the standard Documentation directory.
tools/objtool/Documentation/objtool.txt still resides in its own directory.

While I don't love having to solve the same problems repeatedly (once in Kbuild; potentially-many in tools/), I'm also OK with trying to hack fixes into the current duplicate build system if it's not prohibitively complex to do so.

Here's what I have for now -- I might submit some or all of this as a proper patchset if I can fix a few rough edges.

tools/build/Makefile +2 -5
index 17cdf01e29a0..fad93f64035d 100644
@@ -43,11 +43,8 @@ ifneq ($(wildcard $(TMP_O)),)
 	$(Q)$(MAKE) -C feature OUTPUT=$(TMP_O) clean >/dev/null
 endif
 
-$(OUTPUT)fixdep-in.o: FORCE
-	$(Q)$(MAKE) $(build)=fixdep
-
-$(OUTPUT)fixdep: $(OUTPUT)fixdep-in.o
-	$(QUIET_LINK)$(HOSTCC) $(KBUILD_HOSTLDFLAGS) -o $@ $<
+$(OUTPUT)fixdep: $(srctree)/tools/build/fixdep.c
+	$(QUIET_CC)$(HOSTCC) $(KBUILD_HOSTLDFLAGS) -o $@ $<
 
 FORCE:
 
tools/build/Makefile.include +9 -1
index 8dadaa0fbb43..27b2090cb293 100644
@@ -1,8 +1,16 @@
 # SPDX-License-Identifier: GPL-2.0-only
 build := -f $(srctree)/tools/build/Makefile.build dir=. obj
 
+# More than just $(Q), we sometimes want to suppress all command output from a
+# recursive make -- even the 'up to date' printout.
+ifeq ($(V),1)
+  SILENT_MAKE = $(Q)$(MAKE)
+else
+  SILENT_MAKE = $(Q)$(MAKE) --silent
+endif
+
 fixdep:
-	$(Q)$(MAKE) -C $(srctree)/tools/build CFLAGS= LDFLAGS= $(OUTPUT)fixdep
+	$(SILENT_MAKE) -C $(srctree)/tools/build CFLAGS= LDFLAGS= $(OUTPUT)fixdep
 
 fixdep-clean:
 	$(Q)$(MAKE) -C $(srctree)/tools/build clean
tools/lib/bpf/Makefile +2 -0
index 2cf892774346..a8f34de1ca25 100644
@@ -154,6 +154,8 @@ $(BPF_IN_SHARED): force $(BPF_GENERATED)
 	$(Q)$(MAKE) $(build)=libbpf OUTPUT=$(SHARED_OBJDIR) CFLAGS="$(CFLAGS) $(SHLIB_FLAGS)"
 
 $(BPF_IN_STATIC): force $(BPF_GENERATED)
+	$(call rule_mkdir)
+	$(SILENT_MAKE) -C $(srctree)/tools/build CFLAGS= LDFLAGS= OUTPUT=$(STATIC_OBJDIR) $(STATIC_OBJDIR)fixdep
 	$(Q)$(MAKE) $(build)=libbpf OUTPUT=$(STATIC_OBJDIR)
 
 $(BPF_HELPER_DEFS): $(srctree)/tools/include/uapi/linux/bpf.h
tools/lib/subcmd/Makefile +1 -1
index b87213263a5e..59b09f280e49 100644
@@ -76,7 +76,7 @@ include $(srctree)/tools/build/Makefile.include
 
 all: fixdep $(LIBFILE)
 
-$(SUBCMD_IN): FORCE
+$(SUBCMD_IN): fixdep FORCE
 	@$(MAKE) $(build)=libsubcmd
 
 $(LIBFILE): $(SUBCMD_IN)
#5 lore raw
Quoted 67 lines · Brian Norris

Hi Masahiro,

Thanks for the reply. I've been away for a bit, but I've poked at a few more things now.

Quoted 23 lines · Masahiro Yamada

tools/lib/subcmd/Makefile include $(srctree)/tools/build/Makefile.include · line 82

Quoted 11 lines · Brian Norris
tools/lib/subcmd/Makefile +1 -1
@@ -76,7 +76,7 @@ include $(srctree)/tools/build/Makefile.include
 
 all: fixdep $(LIBFILE)
 
-$(SUBCMD_IN): FORCE
+$(SUBCMD_IN): fixdep FORCE
        @$(MAKE) $(build)=libsubcmd
 
 $(LIBFILE): $(SUBCMD_IN)

I may not fully understand the design policy of the tools/ build system, but this fix is presumably correct because the 'fixdep' binary is needed in each sub-directory for it to work correctly.

tools/bpf/resolve_btfids/libsubcmd/.exec-cmd.o.cmd must be generated by tools/bpf/resolve_btfids/libsubcmd/fixdep instead of by tools/bpf/resolve_btfids/fixdep.

Quoted 5 lines

But, fixing tools/lib/subcmd/Makefile is not enough.

*.cmd files under tools/bpf/resolve_btfids/libbpf/staticobjs/ are broken for the same reason.
So, this is fundamentally broken in many places.

I think I hacked something that works there too. It gets a bit uglier, but not too bad IMO.

Quoted 1 line

And, as you noted, there is no easy way to fix .fixdep.o.cmd

I haven't come up with a *great* solution, but I came up with something that works for the most part, by circumventing the normal Build / Makefile.build split. It's getting pretty ugly too though...

Quoted 13 lines

Your description in
https://issuetracker.google.com/issues/313508829#comment32
is all correct.

"can we just use Kbuild?" is a good question. I do not understand why they use fragile build systems, where obviously they cannot do it correctly.

In fact, I submitted a patch to migrate objtool to Kbuild because that fixes all the issues cleanly.

Quoted 8 lines

The objtool maintainers rejected it.
https://lore.kernel.org/linux-kbuild/[email protected]/

Not only the build system.
He also refused to participate in the standard Documentation directory.
tools/objtool/Documentation/objtool.txt still resides in its own directory.

Quoted 7 lines

While I don't love having to solve the same problems repeatedly (once in Kbuild; potentially-many in tools/), I'm also OK with trying to hack fixes into the current duplicate build system if it's not prohibitively complex to do so.

Here's what I have for now -- I might submit some or all of this as a proper patchset if I can fix a few rough edges.

I do not see any maintainer for tools/build/, but at least you can find who is picking up the patches.

masahiro@zoe:~/ref/linux(master)$ ./scripts/get_maintainer.pl -f tools/build/
Arnaldo Carvalho de Melo <[email protected]>
(commit_signer:7/10=70%,authored:1/10=10%)
Namhyung Kim <[email protected]> (commit_signer:5/10=50%,authored:2/10=20%)
Ian Rogers <[email protected]> (commit_signer:3/10=30%,authored:1/10=10%)
Thomas Richter <[email protected]>
(commit_signer:2/10=20%,authored:2/10=20%)
Quentin Monnet <[email protected]> (commit_signer:2/10=20%)
Jiri Olsa <[email protected]> (authored:1/10=10%)
[email protected] (open list)
Quoted 5 lines
tools/build/Makefile +0 -0
index 17cdf01e29a0..fad93f64035d 100644

tools/build/Makefile ifneq ($(wildcard $(TMP_O)),) · line 47

Quoted 10 lines
tools/build/Makefile +2 -5
@@ -43,11 +43,8 @@ ifneq ($(wildcard $(TMP_O)),)
        $(Q)$(MAKE) -C feature OUTPUT=$(TMP_O) clean >/dev/null
 endif
 
-$(OUTPUT)fixdep-in.o: FORCE
-       $(Q)$(MAKE) $(build)=fixdep
-
-$(OUTPUT)fixdep: $(OUTPUT)fixdep-in.o
-       $(QUIET_LINK)$(HOSTCC) $(KBUILD_HOSTLDFLAGS) -o $@ $<
+$(OUTPUT)fixdep: $(srctree)/tools/build/fixdep.c
+       $(QUIET_CC)$(HOSTCC) $(KBUILD_HOSTLDFLAGS) -o $@ $<

OK, you bypassed fixdep for fixdep itself.

fixdep will not be rebuilt when the command line changes, but it may not be a big deal.
It is not working that way already.

BTW, did you have a chance to test your code
with the -j<N> option?

I quickly tested your change, and I observed new
"jobserver unavailable" warnings.

masahiro@zoe:~/ref/linux(master)$ make -j24
mkdir -p /home/masahiro/ref/linux/tools/objtool && make
O=/home/masahiro/ref/linux subdir=tools/objtool --no-print-directory
-C objtool
make[4]: warning: jobserver unavailable: using -j1.  Add '+' to parent
make rule.
make[5]: warning: jobserver unavailable: using -j1.  Add '+' to parent
make rule.

The first line:

mkdir -p /home/masahiro/ref/linux/tools/objtool && make
O=/home/masahiro/ref/linux subdir=tools/objtool --no-print-directory -C objtool

is an existing noise regardless of your change. (I do not know if anybody cares about this either)

Quoted 43 lines
tools/build/Makefile +0 -0
 FORCE:
 
tools/build/Makefile.include +9 -1
index 8dadaa0fbb43..27b2090cb293 100644
@@ -1,8 +1,16 @@
 # SPDX-License-Identifier: GPL-2.0-only
 build := -f $(srctree)/tools/build/Makefile.build dir=. obj
 
+# More than just $(Q), we sometimes want to suppress all command output from a
+# recursive make -- even the 'up to date' printout.
+ifeq ($(V),1)
+  SILENT_MAKE = $(Q)$(MAKE)
+else
+  SILENT_MAKE = $(Q)$(MAKE) --silent
+endif
+
 fixdep:
-       $(Q)$(MAKE) -C $(srctree)/tools/build CFLAGS= LDFLAGS= $(OUTPUT)fixdep
+       $(SILENT_MAKE) -C $(srctree)/tools/build CFLAGS= LDFLAGS= $(OUTPUT)fixdep
 
 fixdep-clean:
        $(Q)$(MAKE) -C $(srctree)/tools/build clean
tools/lib/bpf/Makefile +2 -0
index 2cf892774346..a8f34de1ca25 100644
@@ -154,6 +154,8 @@ $(BPF_IN_SHARED): force $(BPF_GENERATED)
        $(Q)$(MAKE) $(build)=libbpf OUTPUT=$(SHARED_OBJDIR) CFLAGS="$(CFLAGS) $(SHLIB_FLAGS)"
 
 $(BPF_IN_STATIC): force $(BPF_GENERATED)
+       $(call rule_mkdir)
+       $(SILENT_MAKE) -C $(srctree)/tools/build CFLAGS= LDFLAGS= OUTPUT=$(STATIC_OBJDIR) $(STATIC_OBJDIR)fixdep
        $(Q)$(MAKE) $(build)=libbpf OUTPUT=$(STATIC_OBJDIR)
 
 $(BPF_HELPER_DEFS): $(srctree)/tools/include/uapi/linux/bpf.h
tools/lib/subcmd/Makefile +0 -0
index b87213263a5e..59b09f280e49 100644

tools/lib/subcmd/Makefile include $(srctree)/tools/build/Makefile.include · line 82

Quoted 8 lines
tools/lib/subcmd/Makefile +1 -1
@@ -76,7 +76,7 @@ include $(srctree)/tools/build/Makefile.include
 
 all: fixdep $(LIBFILE)
 
-$(SUBCMD_IN): FORCE
+$(SUBCMD_IN): fixdep FORCE
        @$(MAKE) $(build)=libsubcmd
 
 $(LIBFILE): $(SUBCMD_IN)

--
Best Regards
Masahiro Yamada

#6 lore raw

Hi,

Quoted 5 lines · Masahiro Yamada

I do not see any maintainer for tools/build/, but at least you can find who is picking up the patches.

masahiro@zoe:~/ref/linux(master)$ ./scripts/get_maintainer.pl -f tools/build/

Thanks. I've factored some of this into my addressee list.

Quoted 5 lines

OK, you bypassed fixdep for fixdep itself.

fixdep will not be rebuilt when the command line changes, but it may not be a big deal.
It is not working that way already.

Yeah, that was one rough edge. I didn't manage to smooth that one out, but like you say, it's not really much of a regression. (It *can* work today sometimes for just the right kind of incremental build, because you may have a pre-existing fixdep laying around that will be used if available. But it's not really *correct* still.)

Quoted 19 lines

BTW, did you have a chance to test your code
with the -j<N> option?

I quickly tested your change, and I observed new "jobserver unavailable" warnings.

masahiro@zoe:~/ref/linux(master)$ make -j24
mkdir -p /home/masahiro/ref/linux/tools/objtool && make
O=/home/masahiro/ref/linux subdir=tools/objtool --no-print-directory
-C objtool
make[4]: warning: jobserver unavailable: using -j1.  Add '+' to parent
make rule.
make[5]: warning: jobserver unavailable: using -j1.  Add '+' to parent
make rule.
Quoted 8 lines

The first line:

mkdir -p /home/masahiro/ref/linux/tools/objtool && make
O=/home/masahiro/ref/linux subdir=tools/objtool --no-print-directory -C objtool

is an existing noise regardless of your change. (I do not know if anybody cares about this either)

Thanks. I tried to tackle the existing and new noise in my posting here:

Subject: [PATCH 0/3] tools build: Incorrect fixdep dependencies
https://lore.kernel.org/lkml/[email protected]/

Hopefully that's not horribly off the mark, although I'll admit tools/ is a bit of unfamiliar territory.

Brian