Skip to contentSkip to conversation

kbuild: add a tool to list files ignored by git

PATCH Superseded Masahiro Yamada 1 participant 7 messages Feb 10, 2023 Kbuild

v5 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

PATCH v5 1/7 kbuild: add a tool to list files ignored by git

In short, the motivation of this commit is to build a source package without cleaning the source tree.

The deb-pkg and (src)rpm-pkg targets first run 'make clean' before creating a source tarball. Otherwise build artifacts such as *.o, *.a, etc. would be included in the tarball. Yet, the tarball ends up containing several garbage files since 'make clean' does not clean everything.

Cleaning the tree every time is annoying since it makes the incremental build impossible. It is desirable to create a source tarball without cleaning the tree.

In fact, there are some ways to achieve this.

The easiest solution is 'git archive'. 'make perf-tar*-src-pkg' uses it, but I do not like it because it works only when the source tree is managed by git, and all files you want in the tarball must be committed in advance.

I want to make it work without relying on git. We can do this.

Files that are ignored by git are generated files, so should be excluded from the source tarball. We can list them out by parsing the .gitignore files. Of course, .gitignore does not cover all the cases, but it works well enough.

tar(1) claims to support it:

--exclude-vcs-ignores

Exclude files that match patterns read from VCS-specific ignore files. Supported files are: .cvsignore, .gitignore, .bzrignore, and .hgignore.

The best scenario would be to use 'tar --exclude-vcs-ignores', but this option does not work. --exclude-vcs-ignore does not understand any of the negation (!), preceding slash, following slash, etc.. So, this option is just useless.

Hence, I wrote this gitignore parser. The previous version [1], written in Python, was so slow. This version is implemented in C, so it works much faster. I imported the code from git, so we get the same result.

This tool traverses the source tree, parsing all .gitignore files, and prints file paths that are ignored by git.

The output is similar to 'git ls-files --ignored --directory --others --exclude-per-directory=.gitignore', except

[1] Not sorted
[2] No trailing slash for directories

[2] is intentional because tar's --exclude-from option cannot handle trailing slashes.

[How to test this tool]

  $ git clean -dfx
  $ make -s -j$(nproc) defconfig all                       # or allmodconifg or whatever
  $ git archive -o ../linux1.tar --prefix=./ HEAD
  $ tar tf ../linux1.tar | LANG=C sort > ../file-list1     # files emitted by 'git archive'
  $ make scripts_package
    HOSTCC  scripts/list-gitignored
  $ scripts/list-gitignored  --prefix=./ -o ../exclude-list
  $ tar cf ../linux2.tar --exclude-from=../exclude-list .
  $ tar tf ../linux2.tar | LANG=C sort > ../file-list2     # files emitted by 'tar'
  $ diff  ../file-list1 ../file-list2 | grep -E '^(<|>)'
  < ./Documentation/devicetree/bindings/.yamllint
  < ./drivers/clk/.kunitconfig
  < ./drivers/gpu/drm/tests/.kunitconfig
  < ./drivers/hid/.kunitconfig
  < ./fs/ext4/.kunitconfig
  < ./fs/fat/.kunitconfig
  < ./kernel/kcsan/.kunitconfig
  < ./lib/kunit/.kunitconfig
  < ./mm/kfence/.kunitconfig
  < ./tools/testing/selftests/arm64/tags/
  < ./tools/testing/selftests/arm64/tags/.gitignore
  < ./tools/testing/selftests/arm64/tags/Makefile
  < ./tools/testing/selftests/arm64/tags/run_tags_test.sh
  < ./tools/testing/selftests/arm64/tags/tags_test.c
  < ./tools/testing/selftests/kvm/.gitignore
  < ./tools/testing/selftests/kvm/Makefile
  < ./tools/testing/selftests/kvm/config
  < ./tools/testing/selftests/kvm/settings

The source tarball contains most of files that are tracked by git. You see some diffs, but it is just because some .gitignore files are wrong.

  $ git ls-files -i -c --exclude-per-directory=.gitignore
  Documentation/devicetree/bindings/.yamllint
  drivers/clk/.kunitconfig
  drivers/gpu/drm/tests/.kunitconfig
  drivers/hid/.kunitconfig
  fs/ext4/.kunitconfig
  fs/fat/.kunitconfig
  kernel/kcsan/.kunitconfig
  lib/kunit/.kunitconfig
  mm/kfence/.kunitconfig
  tools/testing/selftests/arm64/tags/.gitignore
  tools/testing/selftests/arm64/tags/Makefile
  tools/testing/selftests/arm64/tags/run_tags_test.sh
  tools/testing/selftests/arm64/tags/tags_test.c
  tools/testing/selftests/kvm/.gitignore
  tools/testing/selftests/kvm/Makefile
  tools/testing/selftests/kvm/config
  tools/testing/selftests/kvm/settings

[1]: https://lore.kernel.org/all/[email protected]/

---

Changes in v5:
- Import a lot of code from GIT
- Fix a bug that the output file is relative to --rootdir
- Check ENOENT when open() fails to open .gitignore
- add more options, --ignore-case, --stat

Changes in v3:
- Various code refactoring: remove struct gitignore, remove next: label etc.
- Support --extra-pattern option

Changes in v2:
- Reimplement in C

 Makefile                  |    7 +-
 scripts/.gitignore        |    1 +
 scripts/Makefile          |    2 +-
 scripts/list-gitignored.c | 1074 +++++++++++++++++++++++++++++++++++++
 4 files changed, 1082 insertions(+), 2 deletions(-)
 create mode 100644 scripts/list-gitignored.c
Makefile +6 -1
index 2faf872b6808..7c726fd26540 100644
@@ -274,7 +274,8 @@ no-dot-config-targets := $(clean-targets) \
 			 cscope gtags TAGS tags help% %docs check% coccicheck \
 			 $(version_h) headers headers_% archheaders archscripts \
 			 %asm-generic kernelversion %src-pkg dt_binding_check \
-			 outputmakefile rustavailable rustfmt rustfmtcheck
+			 outputmakefile rustavailable rustfmt rustfmtcheck \
+			 scripts_package
 # Installation targets should not require compiler. Unfortunately, vdso_install
 # is an exception where build artifacts may be updated. This must be fixed.
 no-compiler-targets := $(no-dot-config-targets) install dtbs_install \
@@ -1652,6 +1653,10 @@ distclean: mrproper
 %pkg: include/config/kernel.release FORCE
 	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.package $@
 
+PHONY += scripts_package
+scripts_package: scripts_basic
+	$(Q)$(MAKE) $(build)=scripts scripts/list-gitignored
+
 # Brief documentation of the typical targets used
 # ---------------------------------------------------------------------------
 
scripts/.gitignore +1 -0
index 6e9ce6720a05..feb43045d1b1 100644
@@ -3,6 +3,7 @@
 /generate_rust_target
 /insert-sys-cert
 /kallsyms
+/list-gitignored
 /module.lds
 /recordmcount
 /sign-file
scripts/Makefile +1 -1
index 32b6ba722728..e8917975905c 100644
@@ -38,7 +38,7 @@ HOSTCFLAGS_sorttable.o += -DMCOUNT_SORT_ENABLED
 endif
 
 # The following programs are only built on demand
-hostprogs += unifdef
+hostprogs += list-gitignored unifdef
 
 # The module linker script is preprocessed on demand
 targets += module.lds
scripts/list-gitignored.c new file +1074 -0

1,074 lines not on this page · raw message

Signature · 1 line
2.34.1
#1 lore raw

PATCH v5 2/7 kbuild: deb-pkg: create source package without cleaning

If you run 'make deb-pkg', all objects are lost due to 'make clean', which makes the incremental builds impossible.

Instead of cleaning, pass the exclude list to tar's --exclude-from option.

Previously, *.diff.gz contained some check-in files such as .clang-format, .cocciconfig.

With this commit, *.diff.gz will only contain the .config and debian/. The other source files will go into the .orig tarball.

linux.tar.gz is rebuilt only when the source files that would go into the tarball are changed.

---

Changes in v5:
- Avoid unneeded rebuild of the tarball when nothing in the source tree is changed.

Changes in v4:
- Fix a typo in comment

Changes in v3:
- Add --extra-pattern='*.rej'
- Exclude symlinks at the toplevel
- Add --sort=name tar option

 scripts/Makefile.package | 40 ++++++++++++++++++++++++++++++++++------
 scripts/package/mkdebian | 24 ++++++++++++++++++++++++
 2 files changed, 58 insertions(+), 6 deletions(-)
scripts/Makefile.package +34 -6
index dfbf40454a99..0dda909f87a9 100644
@@ -80,13 +80,12 @@ binrpm-pkg:
 		$(UTS_MACHINE)-linux -bb $(objtree)/binkernel.spec
 
 PHONY += deb-pkg
-deb-pkg:
-	$(MAKE) clean
+deb-pkg: linux.tar.gz
 	$(CONFIG_SHELL) $(srctree)/scripts/package/mkdebian
-	$(call cmd,src_tar,$(KDEB_SOURCENAME))
-	origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\
-		mv $(KDEB_SOURCENAME).tar.gz ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz
-	+dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) --source-option=-sP -i.git -us -uc
+	$(Q)origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\
+		cp linux.tar.gz ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz
+	+dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) \
+		--build=source,binary --source-option=-sP -nc -us -uc
 
 PHONY += bindeb-pkg
 bindeb-pkg:
@@ -120,6 +119,32 @@ $(tar-pkgs):
 	$(MAKE) -f $(srctree)/Makefile
 	+$(CONFIG_SHELL) $(srctree)/scripts/package/buildtar $@
 
+# Source Tarball
+# ---------------------------------------------------------------------------
+
+scripts/list-gitignored: FORCE
+	$(Q)$(MAKE) -f $(srctree)/Makefile scripts_package
+
+# 1f5d3a6b6532e25a5cdf1f311956b2b03d343a48 removed '*.rej' from .gitignore,
+# but it is definitely a generated file.
+filechk_filelist = \
+	$< --exclude='*.rej' --output=$@_exclude --prefix=./ --rootdir=$(srctree) --stat=-
+
+.tmp_filelist: scripts/list-gitignored FORCE
+	$(call filechk,filelist)
+
+quiet_cmd_tar = TAR     $@
+      cmd_tar = tar $(compress-opt) -c -f $@ \
+                --exclude=./$@ --exclude-from=$<_exclude \
+                --owner=0 --group=0 --sort=name \
+                --transform 's:^\.:$*:S' -C $(tar-rootdir) .
+
+tar-rootdir := $(srctree)
+
+%.tar.gz: compress-opt := -I $(KGZIP)
+%.tar.gz: .tmp_filelist
+	$(call cmd,tar)
+
 # perf-pkg - generate a source tarball with perf source
 # ---------------------------------------------------------------------------
 
@@ -174,4 +199,7 @@ help:
 	@echo '  perf-tarxz-src-pkg  - Build $(perf-tar).tar.xz source tarball'
 	@echo '  perf-tarzst-src-pkg - Build $(perf-tar).tar.zst source tarball'
 
+PHONY += FORCE
+FORCE:
+
 .PHONY: $(PHONY)
scripts/package/mkdebian +24 -0
index c3bbef7a6754..68b13ef590ba 100755
@@ -84,6 +84,8 @@ set_debarch() {
 	fi
 }
 
+rm -rf debian
+
 # Some variables and settings used throughout the script
 version=$KERNELRELEASE
 if [ -n "$KDEB_PKGVERSION" ]; then
@@ -135,6 +137,28 @@ fi
 mkdir -p debian/source/
 echo "1.0" > debian/source/format
 
+# Ugly: ignore anything except .config or debian/
+cat<<'EOF' > debian/source/local-options
+diff-ignore
+
+extend-diff-ignore = ^[^.d]
+
+extend-diff-ignore = ^\.[^c]
+extend-diff-ignore = ^\.c($|[^o])
+extend-diff-ignore = ^\.co($|[^n])
+extend-diff-ignore = ^\.con($|[^f])
+extend-diff-ignore = ^\.conf($|[^i])
+extend-diff-ignore = ^\.confi($|[^g])
+extend-diff-ignore = ^\.config.
+
+extend-diff-ignore = ^d($|[^e])
+extend-diff-ignore = ^de($|[^b])
+extend-diff-ignore = ^deb($|[^i])
+extend-diff-ignore = ^debi($|[^a])
+extend-diff-ignore = ^debia($|[^n])
+extend-diff-ignore = ^debian[^/]
+EOF
+
 echo $debarch > debian/arch
 extra_build_depends=", $(if_enabled_echo CONFIG_UNWINDER_ORC libelf-dev:native)"
 extra_build_depends="$extra_build_depends, $(if_enabled_echo CONFIG_SYSTEM_TRUSTED_KEYRING libssl-dev:native)"
Signature · 1 line
2.34.1
#2 lore raw

PATCH v5 3/7 kbuild: rpm-pkg: build binary packages from source rpm

In reply to Masahiro Yamada

The build rules of rpm-pkg and srcrpm-pkg are almost the same. Remove the code duplication.

Change rpm-pkg to build binary packages from the source package generated by srcrpm-pkg.

This changes the output directory of the srpm generated by 'make rpm-pkg' because srcrpm-pkg overrides _srcrpmdir.

---

(no changes since v3)

Changes in v3:
- Explain that the source package location will be changed.

 scripts/Makefile.package | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)
scripts/Makefile.package +3 -5
index 0dda909f87a9..0b66bd667212 100644
@@ -53,11 +53,9 @@ tar -I $(KGZIP) -c $(RCS_TAR_IGNORE) -f $(2).tar.gz \
 # rpm-pkg
 # ---------------------------------------------------------------------------
 PHONY += rpm-pkg
-rpm-pkg:
-	$(MAKE) clean
-	$(CONFIG_SHELL) $(MKSPEC) >$(objtree)/kernel.spec
-	$(call cmd,src_tar,$(KERNELPATH),kernel.spec)
-	+rpmbuild $(RPMOPTS) --target $(UTS_MACHINE)-linux -ta $(KERNELPATH).tar.gz \
+rpm-pkg: srpm = $(shell rpmspec --srpm --query --queryformat='%{name}-%{VERSION}-%{RELEASE}.src.rpm' kernel.spec)
+rpm-pkg: srcrpm-pkg
+	+rpmbuild $(RPMOPTS) --target $(UTS_MACHINE)-linux -rb $(srpm) \
 	--define='_smp_mflags %{nil}'
 
 # srcrpm-pkg
Signature · 1 line
2.34.1
#3 lore raw

PATCH v5 4/7 kbuild: srcrpm-pkg: create source package without cleaning

In reply to Masahiro Yamada

If you run 'make (src)rpm-pkg', all objects are lost due to 'make clean', which makes the incremental builds impossible.

Instead of cleaning, pass the exclude list to tar's --exclude-from option.

Previously, the .config was contained in the source tarball.

With this commit, the source rpm consists of separate linux.tar.gz and .config.

Remove stale comments. Now, 'make (src)rpm-pkg' works with O= option.

---

(no changes since v4)

Changes in v4:
- Do not delete the old tar command because it is still used by snap-pkg although snap-pkg is broken, and it does not work at all.

 scripts/Makefile.package | 29 +++--------------------------
 scripts/package/mkspec   |  8 ++++----
 2 files changed, 7 insertions(+), 30 deletions(-)
scripts/Makefile.package +3 -26
index 0b66bd667212..39f67f876092 100644
@@ -3,27 +3,6 @@
 
 include $(srctree)/scripts/Kbuild.include
 
-# RPM target
-# ---------------------------------------------------------------------------
-# The rpm target generates two rpm files:
-# /usr/src/packages/SRPMS/kernel-2.6.7rc2-1.src.rpm
-# /usr/src/packages/RPMS/i386/kernel-2.6.7rc2-1.<arch>.rpm
-# The src.rpm files includes all source for the kernel being built
-# The <arch>.rpm includes kernel configuration, modules etc.
-#
-# Process to create the rpm files
-# a) clean the kernel
-# b) Generate .spec file
-# c) Build a tar ball, using symlink to make kernel version
-#    first entry in the path
-# d) and pack the result to a tar.gz file
-# e) generate the rpm files, based on kernel.spec
-# - Use /. to avoid tar packing just the symlink
-
-# Note that the rpm-pkg target cannot be used with KBUILD_OUTPUT,
-# but the binrpm-pkg target can; for some reason O= gets ignored.
-
-# Remove hyphens since they have special meaning in RPM filenames
 KERNELPATH := kernel-$(subst -,_,$(KERNELRELEASE))
 KDEB_SOURCENAME ?= linux-upstream
 KBUILD_PKG_ROOTCMD ?="fakeroot -u"
@@ -61,12 +40,10 @@ rpm-pkg: srcrpm-pkg
 # srcrpm-pkg
 # ---------------------------------------------------------------------------
 PHONY += srcrpm-pkg
-srcrpm-pkg:
-	$(MAKE) clean
+srcrpm-pkg: linux.tar.gz
 	$(CONFIG_SHELL) $(MKSPEC) >$(objtree)/kernel.spec
-	$(call cmd,src_tar,$(KERNELPATH),kernel.spec)
-	+rpmbuild $(RPMOPTS) --target $(UTS_MACHINE)-linux -ts $(KERNELPATH).tar.gz \
-	--define='_smp_mflags %{nil}' --define='_srcrpmdir $(srctree)'
+	+rpmbuild $(RPMOPTS) --target $(UTS_MACHINE)-linux -bs kernel.spec \
+	--define='_smp_mflags %{nil}' --define='_sourcedir .' --define='_srcrpmdir .'
 
 # binrpm-pkg
 # ---------------------------------------------------------------------------
scripts/package/mkspec +4 -4
index 108c0cb95436..83a64d9d7372 100755
@@ -47,7 +47,8 @@ sed -e '/^DEL/d' -e 's/^\t*//' <<EOF
 	Group: System Environment/Kernel
 	Vendor: The Linux Community
 	URL: https://www.kernel.org
-$S	Source: kernel-$__KERNELRELEASE.tar.gz
+$S	Source0: linux.tar.gz
+$S	Source1: .config
 	Provides: $PROVIDES
 $S	BuildRequires: bc binutils bison dwarves
 $S	BuildRequires: (elfutils-libelf-devel or libelf-devel) flex
@@ -83,9 +84,8 @@ $S$M	This package provides kernel headers and makefiles sufficient to build modu
 $S$M	against the $__KERNELRELEASE kernel package.
 $S$M
 $S	%prep
-$S	%setup -q
-$S	rm -f scripts/basic/fixdep scripts/kconfig/conf
-$S	rm -f tools/objtool/{fixdep,objtool}
+$S	%setup -q -n linux
+$S	cp %{SOURCE1} .
 $S
 $S	%build
 $S	$MAKE %{?_smp_mflags} KERNELRELEASE=$KERNELRELEASE KBUILD_BUILD_VERSION=%{release}
Signature · 1 line
2.34.1
#4 lore raw

PATCH v5 5/7 kbuild: deb-pkg: hide KDEB_SOURCENAME from Makefile

In reply to Masahiro Yamada

scripts/Makefile.package does not need to know the value of KDEB_SOURCENAME because the source name can be taken from debian/changelog by using dpkg-parsechangelog.

Move the default of KDEB_SOURCENAME (i.e. linux-upstream) to scripts/package/mkdebian.

---

Changes in v5:
- Copy linux.tar.gz to .orig tarball

Changes in v3:
- Move cmd_debianize

Changes in v2:
- New patch

 scripts/Makefile.package | 22 ++++++++++++++--------
 scripts/package/mkdebian |  2 +-
 2 files changed, 15 insertions(+), 9 deletions(-)
scripts/Makefile.package +14 -8
index 39f67f876092..b591f036e63f 100644
@@ -4,9 +4,7 @@
 include $(srctree)/scripts/Kbuild.include
 
 KERNELPATH := kernel-$(subst -,_,$(KERNELRELEASE))
-KDEB_SOURCENAME ?= linux-upstream
 KBUILD_PKG_ROOTCMD ?="fakeroot -u"
-export KDEB_SOURCENAME
 # Include only those top-level files that are needed by make, plus the GPL copy
 TAR_CONTENT := Documentation LICENSES arch block certs crypto drivers fs \
                include init io_uring ipc kernel lib mm net rust \
@@ -54,17 +52,25 @@ binrpm-pkg:
 	+rpmbuild $(RPMOPTS) --define "_builddir $(objtree)" --target \
 		$(UTS_MACHINE)-linux -bb $(objtree)/binkernel.spec
 
+quiet_cmd_debianize = GEN     $@
+      cmd_debianize = $(srctree)/scripts/package/mkdebian
+
+debian: FORCE
+	$(call cmd,debianize)
+
+PHONY += debian-orig
+debian-orig: private source = $(shell dpkg-parsechangelog -S Source)
+debian-orig: private version = $(shell dpkg-parsechangelog -S Version | sed 's/-[^-]*$$//')
+debian-orig: linux.tar.gz debian
+	$(Q)cp $< ../$(source)_$(version).orig.tar.gz
+
 PHONY += deb-pkg
-deb-pkg: linux.tar.gz
-	$(CONFIG_SHELL) $(srctree)/scripts/package/mkdebian
-	$(Q)origversion=$$(dpkg-parsechangelog -SVersion |sed 's/-[^-]*$$//');\
-		cp linux.tar.gz ../$(KDEB_SOURCENAME)_$${origversion}.orig.tar.gz
+deb-pkg: debian-orig
 	+dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) \
 		--build=source,binary --source-option=-sP -nc -us -uc
 
 PHONY += bindeb-pkg
-bindeb-pkg:
-	$(CONFIG_SHELL) $(srctree)/scripts/package/mkdebian
+bindeb-pkg: debian
 	+dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) -b -nc -uc
 
 PHONY += intdeb-pkg
scripts/package/mkdebian +1 -1
index 68b13ef590ba..534e9713761a 100755
@@ -95,7 +95,7 @@ else
 	revision=$($srctree/init/build-version)
 	packageversion=$version-$revision
 fi
-sourcename=$KDEB_SOURCENAME
+sourcename=${KDEB_SOURCENAME:-linux-upstream}
 
 if [ "$ARCH" = "um" ] ; then
 	packagename=user-mode-linux
Signature · 1 line
2.34.1
#5 lore raw

PATCH v5 6/7 kbuild: deb-pkg: switch over to source format 3.0 (quilt)

In reply to Masahiro Yamada

Change the source format from "1.0" to "3.0 (quilt)" because it works more cleanly.

All files except .config and debian/ go into the .orig tarball. You can add a single patch, debian/patches/config, and delete the ugly extend-diff-ignore patterns.

The debian tarball will be compressed into *.debian.tar.xz by default. If you like to use a different compression mode, you can pass the command line option, DPKG_FLAGS=-Zgzip, for example.

The .orig tarball only supports gzip for now. The combination of gzip and xz is somewhat clumsy, but it is not a practical problem.

---

(no changes since v4)

Changes in v4:
- New patch

 scripts/Makefile.package |  2 +-
 scripts/package/mkdebian | 41 ++++++++++++++++++----------------------
 2 files changed, 19 insertions(+), 24 deletions(-)
scripts/Makefile.package +1 -1
index b591f036e63f..55be31d0e040 100644
@@ -67,7 +67,7 @@ debian-orig: linux.tar.gz debian
 PHONY += deb-pkg
 deb-pkg: debian-orig
 	+dpkg-buildpackage -r$(KBUILD_PKG_ROOTCMD) -a$$(cat debian/arch) $(DPKG_FLAGS) \
-		--build=source,binary --source-option=-sP -nc -us -uc
+		--build=source,binary -nc -us -uc
 
 PHONY += bindeb-pkg
 bindeb-pkg: debian
scripts/package/mkdebian +18 -23

47 lines not on this page · raw message

Signature · 1 line
2.34.1
#6 lore raw

PATCH v5 7/7 kbuild: make perf-tar*-src-pkg work without relying on git

In reply to Masahiro Yamada

Currently, perf-tar*-src-pkg only uses 'git archive', but it is better to make it work without relying on git.

The file, HEAD, which saves the commit hash, will be included in the tarball only when the source tree is managed by git. The git tree is more precisely checked; it has been copied from scripts/setlocalversion.

---

Changes in v5:
- New patch

 scripts/Makefile.package | 67 ++++++++++++++++++++++++----------------
 1 file changed, 40 insertions(+), 27 deletions(-)
scripts/Makefile.package +40 -27
index 55be31d0e040..e676d14fef23 100644
@@ -122,40 +122,53 @@ quiet_cmd_tar = TAR     $@
 
 tar-rootdir := $(srctree)
 
+%.tar: compress-opt :=
+%.tar: .tmp_filelist
+	$(call cmd,tar)
+
 %.tar.gz: compress-opt := -I $(KGZIP)
 %.tar.gz: .tmp_filelist
 	$(call cmd,tar)
 
+%.tar.bz2: compress-opt := -I $(KBZIP2)
+%.tar.bz2: .tmp_filelist
+	$(call cmd,tar)
+
+%.tar.xz: compress-opt := -I $(XZ)
+%.tar.xz: .tmp_filelist
+	$(call cmd,tar)
+
+%.tar.zst: compress-opt := -I $(ZSTD)
+%.tar.zst: .tmp_filelist
+	$(call cmd,tar)
+
 # perf-pkg - generate a source tarball with perf source
 # ---------------------------------------------------------------------------
 
-perf-tar=perf-$(KERNELVERSION)
-
-quiet_cmd_perf_tar = TAR
-      cmd_perf_tar = \
-git --git-dir=$(srctree)/.git archive --prefix=$(perf-tar)/         \
-	HEAD^{tree} $$(cd $(srctree);                               \
-		       echo $$(cat tools/perf/MANIFEST)) \
-	-o $(perf-tar).tar;                                         \
-mkdir -p $(perf-tar);                                               \
-git --git-dir=$(srctree)/.git rev-parse HEAD > $(perf-tar)/HEAD;    \
-(cd $(srctree)/tools/perf;                                          \
-util/PERF-VERSION-GEN $(CURDIR)/$(perf-tar)/);              \
-tar rf $(perf-tar).tar $(perf-tar)/HEAD $(perf-tar)/PERF-VERSION-FILE; \
-rm -r $(perf-tar);                                                  \
-$(if $(findstring tar-src,$@),,                                     \
-$(if $(findstring bz2,$@),$(KBZIP2),                                 \
-$(if $(findstring gz,$@),$(KGZIP),                                  \
-$(if $(findstring xz,$@),$(XZ),                                     \
-$(if $(findstring zst,$@),$(ZSTD),                                  \
-$(error unknown target $@)))))                                      \
-	-f -9 $(perf-tar).tar)
-
-perf-tar-pkgs := perf-tar-src-pkg perf-targz-src-pkg perf-tarbz2-src-pkg \
-		 perf-tarxz-src-pkg perf-tarzst-src-pkg
-PHONY += $(perf-tar-pkgs)
-$(perf-tar-pkgs):
-	$(call cmd,perf_tar)
+perf-tar = perf-$(KERNELVERSION)
+
+perf-tar-phony = perf-$(subst .,,$(1))-src-pkg
+
+.tmp_perf: FORCE
+	$(Q)rm -rf $@
+	$(Q)mkdir $@
+	$(Q)tar -c -f - -C $(srctree) --files-from=$(srctree)/tools/perf/MANIFEST | tar -x -f - -C $@
+	$(Q)if test -z "$(git -C $(srctree) rev-parse --show-cdup 2>/dev/null)" && \
+	       head=$$(git -C $(srctree) rev-parse --verify HEAD 2>/dev/null); then \
+		echo $$head > $@/HEAD; \
+	fi
+	$(Q)cd $(srctree)/tools/perf; util/PERF-VERSION-GEN $(abspath $@)/
+
+define perf-pkg-rule
+PHONY += $(perf-tar-phony)
+$(perf-tar-phony): $(perf-tar).$(1)
+	@:
+
+$(perf-tar).$(1): private tar-rootdir := .tmp_perf
+$(perf-tar).$(1): | .tmp_perf
+endef
+
+$(foreach x, tar tar.gz tar.bz2 tar.xz tar.zst, $(eval $(call perf-pkg-rule,$(x))))
 
 # Help text displayed when executing 'make help'
 # ---------------------------------------------------------------------------
Signature · 1 line
2.34.1