kbuild: fix package build error due to broken symlinks
PATCH 1/1 kbuild: fix package build error due to broken symlinks
'make deb-pkg' and 'make rpm-pkg' fail if a broken symlink exists in a dirty source tree. Handle symlinks properly, and also, keep the executable permission.
---
scripts/package/gen-diff-patch | 36 +++++++++++++++++++++++++--------- 1 file changed, 27 insertions(+), 9 deletions(-)
scripts/package/gen-diff-patch +27 -9
@@ -23,16 +23,34 @@ fi git -C ${srctree} status --porcelain --untracked-files=all | while read stat path do - if [ "${stat}" = '??' ]; then - - if ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff && - ! head -n1 .tmp_diff | grep -q "Binary files"; then - { - echo "--- /dev/null" - echo "+++ linux/$path" - cat .tmp_diff | tail -n +3 - } >> ${untracked_patch} + if [ "${stat}" != '??' ]; then + continue + fi + + if [ -L "${path}" ]; then + { + echo "diff --git a/${path} b/${path}" + echo "new file mode 120000" + echo "--- /dev/null" + echo "+++ b/$path" + echo "@@ -0,0 +1 @@" + printf "+"; readlink ${path} + echo '\ No newline at end of file' + } >> ${untracked_patch} + elif ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff && + ! head -n1 .tmp_diff | grep -q "Binary files"; then + if [ -x ${path} ]; then + mode=100755 + else + mode=100644 fi + { + echo "diff --git a/${path} b/${path}" + echo "new file mode ${mode}" + echo "--- /dev/null" + echo "+++ b/$path" + cat .tmp_diff | tail -n +3 + } >> ${untracked_patch} fi done
Signature · 1 line
2.34.1
Quoted 30 lines · Masahiro Yamada
'make deb-pkg' and 'make rpm-pkg' fail if a broken symlink exists in a dirty source tree. Handle symlinks properly, and also, keep the executable permission.
Fixes 05e96e96a315 ("kbuild: use git-archive for source package creation")Signed-off-by Masahiro Yamada <[email protected]>---
scripts/package/gen-diff-patch | 36 +++++++++++++++++++++++++--------- 1 file changed, 27 insertions(+), 9 deletions(-)scripts/package/gen-diff-patch +2
-9@@ -23,16 +23,34 @@ fi git -C ${srctree} status --porcelain --untracked-files=all | while read stat path do - if [ "${stat}" = '??' ]; then - - if ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff && - ! head -n1 .tmp_diff | grep -q "Binary files"; then - { - echo "--- /dev/null" - echo "+++ linux/$path" - cat .tmp_diff | tail -n +3 - } >> ${untracked_patch} + if [ "${stat}" != '??' ]; then + continue
scripts/package/gen-diff-patch fi · line 37
Quoted 10 lines
scripts/package/gen-diff-patch +10
-0+ fi + + if [ -L "${path}" ]; then + { + echo "diff --git a/${path} b/${path}" + echo "new file mode 120000" + echo "--- /dev/null" + echo "+++ b/$path" + echo "@@ -0,0 +1 @@" + printf "+"; readlink ${path}
Better quote "${path}"?
scripts/package/gen-diff-patch fi · line 39
Quoted 2 lines
scripts/package/gen-diff-patch +2
-0+ echo '\ No newline at end of file' + } >> ${untracked_patch}
Here quoting should not be necessary as mkdebian and mkspec give only save filenames, but for consistency I'd quote ${untracked_patch} as well.
scripts/package/gen-diff-patch fi · line 42
Quoted 3 lines
scripts/package/gen-diff-patch +3
-0+ elif ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff && + ! head -n1 .tmp_diff | grep -q "Binary files"; then + if [ -x ${path} ]; then
Same here.
scripts/package/gen-diff-patch fi · line 53
Quoted 11 lines
scripts/package/gen-diff-patch +10
-0+ mode=100755 + else + mode=100644 fi + { + echo "diff --git a/${path} b/${path}" + echo "new file mode ${mode}" + echo "--- /dev/null" + echo "+++ b/$path" + cat .tmp_diff | tail -n +3 + } >> ${untracked_patch}
And possibly here?
Quoted 5 lines
scripts/package/gen-diff-patch +0
-0fi doneSignature · 1 line
2.34.1
Quoted 36 lines · Nicolas Schier
Quoted 34 lines · Masahiro Yamada
'make deb-pkg' and 'make rpm-pkg' fail if a broken symlink exists in a dirty source tree. Handle symlinks properly, and also, keep the executable permission.
Fixes 05e96e96a315 ("kbuild: use git-archive for source package creation")Signed-off-by Masahiro Yamada <[email protected]>---
scripts/package/gen-diff-patch | 36 +++++++++++++++++++++++++--------- 1 file changed, 27 insertions(+), 9 deletions(-)scripts/package/gen-diff-patch +6
-9@@ -23,16 +23,34 @@ fi git -C ${srctree} status --porcelain --untracked-files=all | while read stat path do - if [ "${stat}" = '??' ]; then - - if ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff && - ! head -n1 .tmp_diff | grep -q "Binary files"; then - { - echo "--- /dev/null" - echo "+++ linux/$path" - cat .tmp_diff | tail -n +3 - } >> ${untracked_patch} + if [ "${stat}" != '??' ]; then + continue + fi + + if [ -L "${path}" ]; then + {
Quoted 8 lines
scripts/package/gen-diff-patch fi · line 37
Quoted 6 lines
scripts/package/gen-diff-patch +6
-0+ echo "diff --git a/${path} b/${path}" + echo "new file mode 120000" + echo "--- /dev/null" + echo "+++ b/$path" + echo "@@ -0,0 +1 @@" + printf "+"; readlink ${path}Better quote "${path}"?
Thanks for the suggestion.
Quoting variables are correct in most cases. But, that is not enough to generate a valid patch when a file path contains spaces.
'git format-patch' produces a patch that
is accepted by GNU patch and also by dpkg-source.
I learned a trick from GIT source code.
If you are interested, what GIT does [1].
[1] https://github.com/git/git/commit/1a9eb3b9d50367bee8fe85022684d812816fe531
I will send v2 later, where I made some more efforts to fix several corner cases even if that is not perfect.
Signature · 2 lines
Best Regards Masahiro Yamada
Quoted 58 lines · On Sat, Apr 01, 2023 at 11:55:44PM +0900 Masahiro Yamada wrote:
Quoted 36 lines · Nicolas Schier
Quoted 34 lines · Masahiro Yamada
'make deb-pkg' and 'make rpm-pkg' fail if a broken symlink exists in a dirty source tree. Handle symlinks properly, and also, keep the executable permission.
Fixes 05e96e96a315 ("kbuild: use git-archive for source package creation")Signed-off-by Masahiro Yamada <[email protected]>---
scripts/package/gen-diff-patch | 36 +++++++++++++++++++++++++--------- 1 file changed, 27 insertions(+), 9 deletions(-)scripts/package/gen-diff-patch +6
-9@@ -23,16 +23,34 @@ fi git -C ${srctree} status --porcelain --untracked-files=all | while read stat path do - if [ "${stat}" = '??' ]; then - - if ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff && - ! head -n1 .tmp_diff | grep -q "Binary files"; then - { - echo "--- /dev/null" - echo "+++ linux/$path" - cat .tmp_diff | tail -n +3 - } >> ${untracked_patch} + if [ "${stat}" != '??' ]; then + continue + fi + + if [ -L "${path}" ]; then + {Quoted 8 lines
scripts/package/gen-diff-patch fi · line 37
Quoted 6 lines
scripts/package/gen-diff-patch +6
-0+ echo "diff --git a/${path} b/${path}" + echo "new file mode 120000" + echo "--- /dev/null" + echo "+++ b/$path" + echo "@@ -0,0 +1 @@" + printf "+"; readlink ${path}Better quote "${path}"?
Thanks for the suggestion.
Quoting variables are correct in most cases. But, that is not enough to generate a valid patch when a file path contains spaces.
'git format-patch' produces a patch that
is accepted by GNU patch and also by dpkg-source.
Quoted 7 lines
I learned a trick from GIT source code.
If you are interested, what GIT does [1].
[1] https://github.com/git/git/commit/1a9eb3b9d50367bee8fe85022684d812816fe531
thanks for the pointer, that is really interesting!
Kind regards,
Nicolas
Quoted 13 lines
I will send v2 later, where I made some more efforts to fix several corner cases even if that is not perfect.
Signature · 2 lines
Best Regards Masahiro Yamada
Quoted 66 lines · Nicolas Schier
Quoted 64 lines · On Sat, Apr 01, 2023 at 11:55:44PM +0900 Masahiro Yamada wrote:
Quoted 36 lines · Nicolas Schier
Quoted 34 lines · Masahiro Yamada
'make deb-pkg' and 'make rpm-pkg' fail if a broken symlink exists in a dirty source tree. Handle symlinks properly, and also, keep the executable permission.
Fixes 05e96e96a315 ("kbuild: use git-archive for source package creation")Signed-off-by Masahiro Yamada <[email protected]>---
scripts/package/gen-diff-patch | 36 +++++++++++++++++++++++++--------- 1 file changed, 27 insertions(+), 9 deletions(-)scripts/package/gen-diff-patch +6
-9@@ -23,16 +23,34 @@ fi git -C ${srctree} status --porcelain --untracked-files=all | while read stat path do - if [ "${stat}" = '??' ]; then - - if ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff && - ! head -n1 .tmp_diff | grep -q "Binary files"; then - { - echo "--- /dev/null" - echo "+++ linux/$path" - cat .tmp_diff | tail -n +3 - } >> ${untracked_patch} + if [ "${stat}" != '??' ]; then + continue + fi + + if [ -L "${path}" ]; then + {Quoted 8 lines
scripts/package/gen-diff-patch fi · line 37
Quoted 6 lines
scripts/package/gen-diff-patch +6
-0+ echo "diff --git a/${path} b/${path}" + echo "new file mode 120000" + echo "--- /dev/null" + echo "+++ b/$path" + echo "@@ -0,0 +1 @@" + printf "+"; readlink ${path}Better quote "${path}"?
Thanks for the suggestion.
Quoting variables are correct in most cases. But, that is not enough to generate a valid patch when a file path contains spaces.
'git format-patch' produces a patch that
is accepted by GNU patch and also by dpkg-source.I learned a trick from GIT source code.
If you are interested, what GIT does [1].
Quoted 10 lines
thanks for the pointer, that is really interesting!
Kind regards,
NicolasQuoted 2 lines
I will send v2 later, where I made some more efforts to fix several corner cases even if that is not perfect.
I wrote a patch (attached to this email) but I am still thinking if this is worth the complicated scripting...
Another approach would be to stop caring untracked files.
Signature · 2 lines
Best Regards Masahiro Yamada
From 4b91ae3f482950209c356a98b79604394f16765c Mon Sep 17 00:00:00 2001
From: Masahiro Yamada <[email protected]>
Date: Sat, 1 Apr 2023 03:57:31 +0900
Subject: [PATCH] kbuild: make package builds more robust
Running 'make deb-pkg' in a dirty repository (that is, 'git diff' outputs something) has some problems.
[1] If a file path contains whitespaces, 'make deb-pkg' fails with an error message like follows:
diff: ./scripts/package/../../"a b": No such file or directory
[snip]
dpkg-source: error: expected ^@@ at line 2 of diff ...
'git status --porcelain' double-quotes a file path that contains whitespaces. Remove the surrounding double-quotes, and add TAB to the '+++' line to make it a valid patch. This is the same as what GIT does.
See GIT commit 1a9eb3b9d503 ("git-diff/git-apply: make diff output a bit friendlier to GNU patch (part 2)") for reference.
I gave up other special characters such as TAB, new line. It would be possible to create a patch that GNU patch accepts by quoting the file paths (and 'git diff' does this), but dpkg-source fails anyway.
[2] The executable bit is lost.
[3] Symbolic links are not handled.
Rewrite the code to make it work more correctly.
--- scripts/package/gen-diff-patch | 79 ++++++++++++++++++++++++++++------ 1 file changed, 66 insertions(+), 13 deletions(-)
scripts/package/gen-diff-patch +66 -13
@@ -21,24 +21,77 @@ if [ ! -s "${diff_patch}" ]; then fi git -C ${srctree} status --porcelain --untracked-files=all | -while read stat path +while read -r stat path do - if [ "${stat}" = '??' ]; then - - if ! diff -u /dev/null "${srctree}/${path}" > .tmp_diff && - ! head -n1 .tmp_diff | grep -q "Binary files"; then - { - echo "--- /dev/null" - echo "+++ linux/$path" - cat .tmp_diff | tail -n +3 - } >> ${untracked_patch} - fi + if [ "${stat}" != '??' ]; then + continue fi -done + + # path may be quoted if it contains whitespaces. Unquote it + path="${path#\"}" + path="${path%\"}" + + case "${path}" in + *\\*) + # Give up escape sequences such as \n, \t. + # It is possible to create a patch by quoting file paths + # (and it is what GIT does) but dpkg-source fails with + # "... patches file with C-style encoded filename" error. + continue + ;; + *\ *) + # GNU patch expects TAB on ---/+++ lines for paths with spaces. + tab="$(printf '\t')" + ;; + *) + tab= + ;; + esac + + if [ ! -f "${path}" ]; then + continue + fi + + if [ -L "${path}" ]; then + echo "diff --git a/${path} b/${path}" + echo "new file mode 120000" + echo "--- /dev/null" + echo "+++ b/${path}${tab}" + echo "@@ -0,0 +1 @@" + printf "+"; readlink "${path}" + echo '\ No newline at end of file' + continue + fi + + if [ -x "${path}" ]; then + mode=100755 + else + mode=100644 + fi + + if [ ! -s "${path}" ]; then + echo "diff --git a/${path} b/${path}" + echo "new file mode ${mode}" + continue + fi + + if diff -u /dev/null "${srctree}/${path}" > .tmp_diff || [ "$?" != 1 ]; then + continue + fi + + if [ "$(head -n6 .tmp_diff)" = Binary ] ; then + continue + fi + + echo "diff --git a/${path} b/${path}" + echo "new file mode ${mode}" + echo "--- /dev/null" + echo "+++ b/${path}${tab}" + cat .tmp_diff | tail -n +3 +done > "${untracked_patch}" rm -f .tmp_diff if [ ! -s "${diff_patch}" ]; then rm -f "${diff_patch}" - exit fi
Signature · 1 line
2.37.2