Skip to contentSkip to conversation

kbuild: Run syncconfig with -s

PATCH In review Ingo Molnar 2 participants 5 messages Aug 20, 2020 – Sep 14, 2020 Kbuild

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kbuild: Run syncconfig with -s

On every kernel build that runs --syncconfig, there's an output of the following line:

  scripts/kconfig/conf  --syncconfig Kconfig

This is the only non-platform build message the kbuild system emits that isn't prefixed by at least a space, or is a build warning.

Run it under -s - if there's any problem it will emit messages anyway.

With this change the following simple grep filter will show all build warnings and errors of a kernel build:

make | grep -v '^ '

---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Makefile +1 -1
index 9cac6fde3479..2057c92a6205 100644
@@ -709,7 +709,7 @@ $(KCONFIG_CONFIG):
 # The syncconfig should be executed only once to make all the targets.
 # (Note: use the grouped target '&:' when we bump to GNU Make 4.3)
 %/config/auto.conf %/config/auto.conf.cmd %/generated/autoconf.h: $(KCONFIG_CONFIG)
-	$(Q)$(MAKE) -f $(srctree)/Makefile syncconfig
+	$(Q)$(MAKE) -sf $(srctree)/Makefile syncconfig
 else # !may-sync-config
 # External modules and some install targets need include/generated/autoconf.h
 # and include/config/auto.conf but do not care if they are up-to-date.
#1 lore raw
Quoted 8 lines · Ingo Molnar

On every kernel build that runs --syncconfig, there's an output of the following line:

  scripts/kconfig/conf  --syncconfig Kconfig

This is the only non-platform build message the kbuild system emits that isn't prefixed by at least a space, or is a build warning.

Quoted 6 lines

Run it under -s - if there's any problem it will emit messages anyway.

With this change the following simple grep filter will show all build warnings and errors of a kernel build:

   make | grep -v '^ '

I do want to see something when syncconfig is invoked.

I will apply this instead:
https://patchwork.kernel.org/patch/11727445/

Quoted 18 lines
---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Makefile +1 -1
index 9cac6fde3479..2057c92a6205 100644
@@ -709,7 +709,7 @@ $(KCONFIG_CONFIG):
 # The syncconfig should be executed only once to make all the targets.
 # (Note: use the grouped target '&:' when we bump to GNU Make 4.3)
 %/config/auto.conf %/config/auto.conf.cmd %/generated/autoconf.h: $(KCONFIG_CONFIG)
-       $(Q)$(MAKE) -f $(srctree)/Makefile syncconfig
+       $(Q)$(MAKE) -sf $(srctree)/Makefile syncconfig
 else # !may-sync-config
 # External modules and some install targets need include/generated/autoconf.h
 # and include/config/auto.conf but do not care if they are up-to-date.
Signature · 2 lines
Best Regards
Masahiro Yamada
#2 lore raw
Quoted 14 lines · Masahiro Yamada
Quoted 13 lines · Ingo Molnar

On every kernel build that runs --syncconfig, there's an output of the following line:

  scripts/kconfig/conf  --syncconfig Kconfig

This is the only non-platform build message the kbuild system emits that isn't prefixed by at least a space, or is a build warning.

Run it under -s - if there's any problem it will emit messages anyway.

With this change the following simple grep filter will show all build warnings and errors of a kernel build:

Quoted 8 lines
Quoted 1 line
   make | grep -v '^ '

I do want to see something when syncconfig is invoked.

I will apply this instead:
https://patchwork.kernel.org/patch/11727445/

That's perfect, thank you very much!

Ingo

#3 lore raw
In reply to Masahiro Yamada
Quoted 14 lines · Masahiro Yamada
Quoted 13 lines · Ingo Molnar

On every kernel build that runs --syncconfig, there's an output of the following line:

  scripts/kconfig/conf  --syncconfig Kconfig

This is the only non-platform build message the kbuild system emits that isn't prefixed by at least a space, or is a build warning.

Run it under -s - if there's any problem it will emit messages anyway.

With this change the following simple grep filter will show all build warnings and errors of a kernel build:

Quoted 8 lines
Quoted 1 line
   make | grep -v '^ '

I do want to see something when syncconfig is invoked.

I will apply this instead:
https://patchwork.kernel.org/patch/11727445/

BTW., there's another, rather spurious bug I recently triggered in kbuild.

Occasionally when I Ctrl-C a kernel build on a system with a lot of CPUs, the .o.cmd file gets corrupted:

mm/.pgtable-generic.o.cmd:5: *** unterminated call to function 'wildcard': missing ')'. Stop.
make: *** [Makefile:1788: mm] Error 2
make: *** Waiting for unfinished jobs....

The .o.cmd file is half-finished:

    $(wildcard include/config/shmem.h) \
    $(wildcard include/config/hugetlb/page.h) \
    $(wildcard include/config/zone/device.h) \
    $(wildcard include/config/dev/pagemap/ops.h) \
    $(wildcard include/config/device/private.h) \
    $(wildcard include/config/pci/p2pdma.h) \
    $(wildcard include/config/sparsemem.h) \
    $(wildcard include/config/sparsemem/vmemmap.h) \
    $(wildcard include/config/numa/balancing.h) \
    $(wildcard i
    [premature EOF]

Instead of the regular rules that end in:

    $(wildcard include/config/memory/hotplug/sparse.h) \

    mm/pgtable-generic.o: $(deps_mm/pgtable-generic.o)

    $(deps_mm/pgtable-generic.o):
    [regular EOF]

Manually removing the corrupted .o.cmd dot file solves the bug.

There's no reproducer other than Ctrl-C-ing large build jobs a couple of times.

Thanks,

Ingo

#4 lore raw
Quoted 38 lines · Ingo Molnar
Quoted 14 lines · Masahiro Yamada
Quoted 13 lines · Ingo Molnar

On every kernel build that runs --syncconfig, there's an output of the following line:

  scripts/kconfig/conf  --syncconfig Kconfig

This is the only non-platform build message the kbuild system emits that isn't prefixed by at least a space, or is a build warning.

Run it under -s - if there's any problem it will emit messages anyway.

With this change the following simple grep filter will show all build warnings and errors of a kernel build:

Quoted 8 lines
Quoted 1 line
   make | grep -v '^ '

I do want to see something when syncconfig is invoked.

I will apply this instead:
https://patchwork.kernel.org/patch/11727445/

BTW., there's another, rather spurious bug I recently triggered in kbuild.

Occasionally when I Ctrl-C a kernel build on a system with a lot of CPUs, the .o.cmd file gets corrupted:

mm/.pgtable-generic.o.cmd:5: *** unterminated call to function 'wildcard': missing ')'. Stop.
make: *** [Makefile:1788: mm] Error 2
make: *** Waiting for unfinished jobs....

The .o.cmd file is half-finished:

Quoted 11 lines
    $(wildcard include/config/shmem.h) \
    $(wildcard include/config/hugetlb/page.h) \
    $(wildcard include/config/zone/device.h) \
    $(wildcard include/config/dev/pagemap/ops.h) \
    $(wildcard include/config/device/private.h) \
    $(wildcard include/config/pci/p2pdma.h) \
    $(wildcard include/config/sparsemem.h) \
    $(wildcard include/config/sparsemem/vmemmap.h) \
    $(wildcard include/config/numa/balancing.h) \
    $(wildcard i
    [premature EOF]

Hmm, previously fixdep was not robust against errors or interruption.
So, I occasionally observed the same problem as you saw.

I fixed it in
6f9ac9f4427ec0470ccffbf852cfaf326677cc21

Since then, I have not seen
the broken .cmd file issue.

I repeated 'make -j256' and Ctrl-C,
but I could not reproduce it...

If GNU Make was interrupted during the build rule of mm/pgtable-generic.o, GNU Make should automatically have deleted mm/pgtable-generic.o because partially updated (i.e. corrupted) files are the source of all the troubles.

Isn't this the case for you?

If I interrupt Kbuild, I see several
Deleting file '.....'
messages, like follows:

  CC      fs/seq_file.o
  CC      drivers/acpi/dock.o
  CC      drivers/scsi/scsi_devinfo.o
  CC      fs/xattr.o

^Cmake[1]: *** [scripts/Makefile.build:283: fs/seq_file.o] Interrupt make[2]: *** [scripts/Makefile.build:283: net/ipv6/af_inet6.o] Interrupt make[3]: *** [scripts/Makefile.build:283:
drivers/net/phy/mdio-boardinfo.o] Interrupt
make[4]: *** [scripts/Makefile.build:283:
drivers/tty/serial/8250/8250_pnp.o] Interrupt
make[2]: *** [scripts/Makefile.build:283: security/selinux/status.o] Interrupt make[2]: *** [scripts/Makefile.build:283: arch/x86/kernel/dumpstack.o] Interrupt make[2]: *** Deleting file 'drivers/acpi/resource.o'
make[2]: *** [scripts/Makefile.build:500: drivers/net/phy] Interrupt make[2]: *** [scripts/Makefile.build:283: net/sched/act_api.o] Interrupt make[2]: *** [scripts/Makefile.build:283: drivers/scsi/scsi_devinfo.o] Interrupt make[1]: *** Deleting file 'arch/x86/pci/mmconfig_64.o'
make[1]: *** [scripts/Makefile.build:500: net/ipv6] Interrupt make[2]: *** [scripts/Makefile.build:283: net/ethtool/wol.o] Interrupt make[1]: *** Deleting file 'arch/x86/pci/init.o'
make[1]: *** [scripts/Makefile.build:283: arch/x86/pci/mmconfig_64.o] Interrupt make[2]: *** [scripts/Makefile.build:283: net/sunrpc/socklib.o] Interrupt make[1]: *** [scripts/Makefile.build:283: fs/xattr.o] Interrupt make[2]: *** [scripts/Makefile.build:283: sound/hda/hdac_sysfs.o] Interrupt make[2]: *** [scripts/Makefile.build:283: net/mac80211/main.o] Interrupt make[2]: *** [scripts/Makefile.build:283: drivers/input/input-mt.o] Interrupt make[2]: *** [scripts/Makefile.build:283: net/rfkill/core.o] Interrupt make[1]: *** [scripts/Makefile.build:283: lib/bug.o] Interrupt
make: *** [Makefile:1792: lib] Interrupt
make[2]: *** Deleting file 'drivers/rtc/systohc.o'

Quoted 9 lines

Instead of the regular rules that end in:

    $(wildcard include/config/memory/hotplug/sparse.h) \

    mm/pgtable-generic.o: $(deps_mm/pgtable-generic.o)

    $(deps_mm/pgtable-generic.o):
    [regular EOF]
Quoted 7 lines

Manually removing the corrupted .o.cmd dot file solves the bug.

There's no reproducer other than Ctrl-C-ing large build jobs a couple of times.

Thanks,

Ingo

--
Best Regards
Masahiro Yamada