Skip to contentSkip to conversation

kbuild: copy scripts/atomic/atomic-*.h to include/generated/atomic-*.h

PATCH New Masahiro Yamada 2 participants 3 messages May 13, 2022 – May 14, 2022

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kbuild: copy scripts/atomic/atomic-*.h to include/generated/atomic-*.h

include/linux/atomic/*.h are generated by scripts/atomic/gen-atomics.sh.

To ensure they are not edited, scripts/atomic/check-atomics.sh checks sha1sum every time.

This commit moves include/linux/atomic/*.h to scripts/atomic/*.h_shipped, which are copied to include/generated/ at build time:

  COPY    include/generated/atomic-instrumented.h
  COPY    include/generated/atomic-long.h
  COPY    include/generated/atomic-arch-fallback.h

I dropped the sha1sum checks. I hope nobody would try to directly modify *_shipped files.

Kbuild runs more and more tools at build time these days because they are fast enough on modern systems.

For example,

- 29c833061c1d8c2d1d23a62e7061561eadd76cdb
("kconfig: generate lexer and parser during build instead of shipping")

- 7c0303ff7e67b637c47d8afee533ca9e2a02359b
("crypto: arm - generate *.S by Perl at build time instead of shipping them")

Yet, gen-atomics.sh is too slow.

I guess it can be improved because it is just simple text processing. Then, Kbuild can execute it at build time.

---
 Kbuild                                        |   26 +-
 include/linux/atomic/atomic-arch-fallback.h   | 2391 +----------------
 include/linux/atomic/atomic-instrumented.h    | 2050 +-------------
 include/linux/atomic/atomic-long.h            | 1016 +------
 .../atomic/atomic-arch-fallback.h_shipped     |    1 -
 .../atomic/atomic-instrumented.h_shipped      |    1 -
 .../atomic/atomic-long.h_shipped              |    1 -
 scripts/atomic/check-atomics.sh               |   33 -
 scripts/atomic/gen-atomics.sh                 |   14 +-
 9 files changed, 24 insertions(+), 5509 deletions(-)
 copy include/linux/atomic/atomic-arch-fallback.h => scripts/atomic/atomic-arch-fallback.h_shipped (99%)
 copy include/linux/atomic/atomic-instrumented.h => scripts/atomic/atomic-instrumented.h_shipped (99%)
 copy include/linux/atomic/atomic-long.h => scripts/atomic/atomic-long.h_shipped (99%)
 delete mode 100755 scripts/atomic/check-atomics.sh
Kbuild +13 -13
index fa441b98c9f6..768921bca727 100644
@@ -2,13 +2,24 @@
 #
 # Kbuild for top-level directory of the kernel
 
+####
+# Copy atomic headers
+
+atomic-files := $(addprefix include/generated/, \
+		atomic-instrumented.h atomic-long.h atomic-arch-fallback.h)
+
+targets := $(atomic-files)
+
+$(atomic-files): include/generated/%.h: $(srctree)/scripts/atomic/%.h_shipped
+	$(call cmd,copy)
+
 #####
 # Generate bounds.h
 
 bounds-file := include/generated/bounds.h
 
 always-y := $(bounds-file)
-targets := kernel/bounds.s
+targets += kernel/bounds.s
 
 $(bounds-file): kernel/bounds.s FORCE
 	$(call filechk,offsets,__LINUX_BOUNDS_H__)
@@ -31,7 +42,7 @@ offsets-file := include/generated/asm-offsets.h
 always-y += $(offsets-file)
 targets += arch/$(SRCARCH)/kernel/asm-offsets.s
 
-arch/$(SRCARCH)/kernel/asm-offsets.s: $(timeconst-file) $(bounds-file)
+arch/$(SRCARCH)/kernel/asm-offsets.s: $(timeconst-file) $(bounds-file) $(atomic-files)
 
 $(offsets-file): arch/$(SRCARCH)/kernel/asm-offsets.s FORCE
 	$(call filechk,offsets,__ASM_OFFSETS_H__)
@@ -46,14 +57,3 @@ quiet_cmd_syscalls = CALL    $<
 
 missing-syscalls: scripts/checksyscalls.sh $(offsets-file) FORCE
 	$(call cmd,syscalls)
-
-#####
-# Check atomic headers are up-to-date
-
-always-y += old-atomics
-
-quiet_cmd_atomics = CALL    $<
-      cmd_atomics = $(CONFIG_SHELL) $<
-
-old-atomics: scripts/atomic/check-atomics.sh FORCE
-	$(call cmd,atomics)
include/linux/atomic/atomic-arch-fallback.h +2 -2389

2,391 lines not on this page · raw message

include/linux/atomic/atomic-instrumented.h +2 -2048

562 lines not on this page · raw message

include/linux/atomic/atomic-long.h +2 -1014

1,016 lines not on this page · raw message

scripts/atomic/atomic-arch-fallback.h_shipped renamed from include/linux/atomic/atomic-arch-fallback.h +0 -1
similarity index 99%
copy from include/linux/atomic/atomic-arch-fallback.h
copy to scripts/atomic/atomic-arch-fallback.h_shipped
index 6db58d180866..357627e3e304 100644

Lines of this file are not shown. View the raw message.

scripts/atomic/atomic-instrumented.h_shipped renamed from include/linux/atomic/atomic-instrumented.h +0 -1
similarity index 99%
copy from include/linux/atomic/atomic-instrumented.h
copy to scripts/atomic/atomic-instrumented.h_shipped
index 5d69b143c28e..89cde1f1eb67 100644

Lines of this file are not shown. View the raw message.

scripts/atomic/atomic-long.h_shipped renamed from include/linux/atomic/atomic-long.h +0 -1
similarity index 99%
copy from include/linux/atomic/atomic-long.h
copy to scripts/atomic/atomic-long.h_shipped
index 800b8c35992d..411d16a7f4eb 100644

Lines of this file are not shown. View the raw message.

scripts/atomic/check-atomics.sh deleted +0 -33
deleted file mode 100755
index 0e7bab3eb0d1..000000000000

Lines of this file are not shown. View the raw message.

scripts/atomic/gen-atomics.sh +5 -9
index 5b98a8307693..ea90b7ac8f2c 100755

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.32.0

Part of the diff is not shown here. View the raw message.

#1 lore raw
I love your patch! Yet something to improve:

Hi Masahiro,

I love your patch! Yet something to improve:

[auto build test ERROR on masahiroy-kbuild/for-next]
[also build test ERROR on linux/master linus/master v5.18-rc6 next-20220513] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/intel-lab-lkp/linux/commits/Masahiro-Yamada/kbuild-copy-scripts-atomic-atomic-h-to-include-generated-atomic-h/20220514-025623
base:   https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git for-next
config: um-x86_64_defconfig (https://download.01.org/0day-ci/archive/20220514/[email protected]/config)
compiler: gcc-11 (Debian 11.2.0-20) 11.2.0
reproduce (this is a W=1 build):
        # https://github.com/intel-lab-lkp/linux/commit/3f6dd60121830fdd3733dd804d9ae163d88acc98
        git remote add linux-review https://github.com/intel-lab-lkp/linux
        git fetch --no-tags linux-review Masahiro-Yamada/kbuild-copy-scripts-atomic-atomic-h-to-include-generated-atomic-h/20220514-025623
        git checkout 3f6dd60121830fdd3733dd804d9ae163d88acc98
        # save the config file
        mkdir build_dir && cp config build_dir/.config
        make W=1 O=build_dir ARCH=um SUBARCH=x86_64 prepare

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>):

arch/x86/um/user-offsets.c:17:6: warning: no previous prototype for 'foo' [-Wmissing-prototypes]

      17 | void foo(void)
         |      ^~~
   In file included from include/linux/atomic.h:80,
                    from include/linux/jump_label.h:256,
                    from arch/x86/include/asm/string_64.h:6,
                    from arch/x86/include/asm/string.h:5,
                    from include/linux/string.h:20,
                    from include/linux/uuid.h:12,
                    from include/linux/mod_devicetable.h:13,
                    from scripts/mod/devicetable-offsets.c:3:
>> include/linux/atomic/atomic-arch-fallback.h:2:10: fatal error: generated/atomic-arch-fallback.h: No such file or directory

2 | #include <generated/atomic-arch-fallback.h>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [scripts/Makefile.build:121: scripts/mod/devicetable-offsets.s] Error 1 make[2]: Target '__build' not remade because of errors.
make[1]: *** [Makefile:1194: prepare0] Error 2
make[1]: Target 'prepare' not remade because of errors.
make: *** [Makefile:219: __sub-make] Error 2
make: Target 'prepare' not remade because of errors.

vim +2 include/linux/atomic/atomic-arch-fallback.h

   > 2	#include <generated/atomic-arch-fallback.h>
Signature · 2 lines
0-DAY CI Kernel Test Service
https://01.org/lkp
#2 lore raw
In reply to Masahiro Yamada
I love your patch! Yet something to improve:

Hi Masahiro,

I love your patch! Yet something to improve:

[auto build test ERROR on masahiroy-kbuild/for-next]
[also build test ERROR on linux/master linus/master v5.18-rc6 next-20220513] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/intel-lab-lkp/linux/commits/Masahiro-Yamada/kbuild-copy-scripts-atomic-atomic-h-to-include-generated-atomic-h/20220514-025623
base:   https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git for-next
config: x86_64-randconfig-a005 (https://download.01.org/0day-ci/archive/20220514/[email protected]/config)
compiler: clang version 15.0.0 (https://github.com/llvm/llvm-project 38189438b69ca27b4c6ce707c52dbd217583d046)

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/3f6dd60121830fdd3733dd804d9ae163d88acc98
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Masahiro-Yamada/kbuild-copy-scripts-atomic-atomic-h-to-include-generated-atomic-h/20220514-025623 git checkout 3f6dd60121830fdd3733dd804d9ae163d88acc98
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=x86_64 prepare

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>):

In file included from scripts/mod/devicetable-offsets.c:3:
In file included from include/linux/mod_devicetable.h:13:
In file included from include/linux/uuid.h:12:
In file included from include/linux/string.h:20:
In file included from arch/x86/include/asm/string.h:5:
In file included from arch/x86/include/asm/string_64.h:6:
In file included from include/linux/jump_label.h:256:
In file included from include/linux/atomic.h:80:

>> include/linux/atomic/atomic-arch-fallback.h:2:10: fatal error: 'generated/atomic-arch-fallback.h' file not found
   #include <generated/atomic-arch-fallback.h>
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

1 error generated.
make[2]: *** [scripts/Makefile.build:121: scripts/mod/devicetable-offsets.s] Error 1 make[2]: Target '__build' not remade because of errors.
make[1]: *** [Makefile:1194: prepare0] Error 2
make[1]: Target 'prepare' not remade because of errors.
make: *** [Makefile:219: __sub-make] Error 2
make: Target 'prepare' not remade because of errors.

vim +2 include/linux/atomic/atomic-arch-fallback.h

   > 2	#include <generated/atomic-arch-fallback.h>
Signature · 2 lines
0-DAY CI Kernel Test Service
https://01.org/lkp