support/scripts/apply-patches: use "git apply" as a fallback when applying patches
PATCH 1/1 support/scripts/apply-patches: use "git apply" as a fallback when applying patches
We currently use plain old "patch" to apply patches. While this works fine in most situations, it doesn't work well for patches generated with git format-patch that:
- Contain changes to binary files
- Or contain changes to files in directories that were symlinks to other directories, with the symlink being removed as part of the same patch.
We encountered such issues with a large stack of patches to be applied on top of arm-trusted-firmware. Such patches apply perfectly fine with "git apply".
Switching everybody to unconditionally use "git apply" seems a bit risky, so instead we take a different route: if applying the patch with "patch" fails, then we try with "git apply".
A few implementation notes:
- The script has "set -e" so for the "patch --dry-run" command, we have to take special precautions to not bail out of the script on error.
- Also due to "set -e", the check on the return value of "patch" is no longer needed, and we also don't need to check the return value of "git apply".
- We need to pass "--git-dir=/dev/null", otherwise "git apply" travels up the directory hierarchy until it finds a .git folder. If it finds one, but the files being patched are not under version control, it skips patching those files. Obviously, the files in output/build/foo/ are not under version control, so this behavior is not desirable. Simply making "git apply" believe it is not running from inside a Git repository disables this check.
--- support/scripts/apply-patches.sh | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)
support/scripts/apply-patches.sh +11 -4
@@ -119,11 +119,18 @@ function apply_patch { exit 1 fi echo "${path}/${patch}" >> ${builddir}/.applied_patches_list - ${uncomp} "${path}/$patch" | patch -g0 -p1 -E -d "${builddir}" -t -N $silent - if [ $? != 0 ] ; then - echo "Patch failed! Please fix ${patch}!" - exit 1 + + # We don't want this to abort the script on failure (script is run + # with set -e) + ${uncomp} "${path}/$patch" | patch -g0 -p1 -E -d "${builddir}" --dry-run -t -N -s && retval=0 || retval=$? + if [ $retval -eq 0 ] ; then + ${uncomp} "${path}/$patch" | patch -g0 -p1 -E -d "${builddir}" -t -N $silent + # Due to set -e, if we reach here, applying the patch was successful + return fi + + [ -z "${silent}" ] && gitopts=-v + (cd ${builddir}; ${uncomp} "${path}/$patch" | git --git-dir=/dev/null apply -p1 ${gitopts}) } function scan_patchdir {
Signature · 1 line
2.20.1
PATCH 1/1 support/scripts/apply-patches: use "git apply" as a fallback when applying patches
Hi Thomas,
Quoted 10 lines · Thomas Petazzoni
We currently use plain old "patch" to apply patches. While this works fine in most situations, it doesn't work well for patches generated with git format-patch that:
- Contain changes to binary files
- Or contain changes to files in directories that were symlinks to other directories, with the symlink being removed as part of the same patch.
Quoted 7 lines
We encountered such issues with a large stack of patches to be applied on top of arm-trusted-firmware. Such patches apply perfectly fine with "git apply".
Switching everybody to unconditionally use "git apply" seems a bit risky, so instead we take a different route: if applying the patch with "patch" fails, then we try with "git apply".
This makes git a host dependency. Should we add host-git, or list git with host requirements?
baruch
Quoted 41 lines
A few implementation notes:
- The script has "set -e" so for the "patch --dry-run" command, we have to take special precautions to not bail out of the script on error.
- Also due to "set -e", the check on the return value of "patch" is no longer needed, and we also don't need to check the return value of "git apply".
- We need to pass "--git-dir=/dev/null", otherwise "git apply" travels up the directory hierarchy until it finds a .git folder. If it finds one, but the files being patched are not under version control, it skips patching those files. Obviously, the files in output/build/foo/ are not under version control, so this behavior is not desirable. Simply making "git apply" believe it is not running from inside a Git repository disables this check.
Cc Kostya Porotchkin <[email protected]>Signed-off-by Thomas Petazzoni <[email protected]>--- support/scripts/apply-patches.sh | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)support/scripts/apply-patches.sh +5
-4@@ -119,11 +119,18 @@ function apply_patch { exit 1 fi echo "${path}/${patch}" >> ${builddir}/.applied_patches_list - ${uncomp} "${path}/$patch" | patch -g0 -p1 -E -d "${builddir}" -t -N $silent - if [ $? != 0 ] ; then - echo "Patch failed! Please fix ${patch}!" - exit 1 + + # We don't want this to abort the script on failure (script is run + # with set -e) + ${uncomp} "${path}/$patch" | patch -g0 -p1 -E -d "${builddir}" --dry-run -t -N -s && retval=0 || retval=$? + if [ $retval -eq 0 ] ; then
support/scripts/apply-patches.sh function apply_patch { · line 136
Quoted 10 lines
support/scripts/apply-patches.sh +6
-0+ ${uncomp} "${path}/$patch" | patch -g0 -p1 -E -d "${builddir}" -t -N $silent + # Due to set -e, if we reach here, applying the patch was successful + return fi + + [ -z "${silent}" ] && gitopts=-v + (cd ${builddir}; ${uncomp} "${path}/$patch" | git --git-dir=/dev/null apply -p1 ${gitopts}) } function scan_patchdir {
--
http://baruch.siach.name/blog/ ~. .~ Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
- baruch at tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il -
PATCH 1/1 support/scripts/apply-patches: use "git apply" as a fallback when applying patches
Hello,
Quoted 6 lines · Baruch Siach
Quoted 3 lines
Switching everybody to unconditionally use "git apply" seems a bit risky, so instead we take a different route: if applying the patch with "patch" fails, then we try with "git apply".
This makes git a host dependency. Should we add host-git, or list git with host requirements?
It does not really make git a mandatory dependency. Indeed, "git apply" is only tried if "patch" fails.
So, before my patch, if a patch failed to apply because "patch" failed, then it aborted the build.
With my patch, if a patch fails to apply because "patch" failed, then we will try "git apply". If "git apply" is not available, it will fail, just like it used to be.
So my proposal doesn't *require* git, it only tries harder to apply patches by using "git apply" if available.
Setups that used to work today without "git" installed will continue to work with no change.
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 1/1 support/scripts/apply-patches: use "git apply" as a fallback when applying patches
Quoted 19 lines · Thomas Petazzoni
Quoted 6 lines · Baruch Siach
Quoted 3 lines
Switching everybody to unconditionally use "git apply" seems a bit risky, so instead we take a different route: if applying the patch with "patch" fails, then we try with "git apply".
This makes git a host dependency. Should we add host-git, or list git with host requirements?
It does not really make git a mandatory dependency. Indeed, "git apply" is only tried if "patch" fails.
So, before my patch, if a patch failed to apply because "patch" failed, then it aborted the build.
With my patch, if a patch fails to apply because "patch" failed, then we will try "git apply". If "git apply" is not available, it will fail, just like it used to be.
Quoted 5 lines
So my proposal doesn't *require* git, it only tries harder to apply patches by using "git apply" if available.
Setups that used to work today without "git" installed will continue to work with no change.
With this patch a package selection change that works on the developer's machine where git is installed, would fail on the production build machine that is missing git. Is there a clear error message in case of patch failure?
baruch
Signature · 3 lines
http://baruch.siach.name/blog/ ~. .~ Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
- baruch at tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il -PATCH 1/1 support/scripts/apply-patches: use "git apply" as a fallback when applying patches
Quoted 24 lines · Baruch Siach
Quoted 22 lines · Thomas Petazzoni
Quoted 6 lines · Baruch Siach
Quoted 3 lines
Switching everybody to unconditionally use "git apply" seems a bit risky, so instead we take a different route: if applying the patch with "patch" fails, then we try with "git apply".
This makes git a host dependency. Should we add host-git, or list git with host requirements?
It does not really make git a mandatory dependency. Indeed, "git apply" is only tried if "patch" fails.
So, before my patch, if a patch failed to apply because "patch" failed, then it aborted the build.
With my patch, if a patch fails to apply because "patch" failed, then we will try "git apply". If "git apply" is not available, it will fail, just like it used to be.
So my proposal doesn't *require* git, it only tries harder to apply patches by using "git apply" if available.
Quoted 7 lines
Quoted 2 lines
Setups that used to work today without "git" installed will continue to work with no change.
With this patch a package selection change that works on the developer's machine where git is installed, would fail on the production build machine that is missing git. Is there a clear error message in case of patch failure?
I'm with Baruch here. The reason to have this feature is exactly to have the
possibility to have patches that modify binaries or that modify symlinks. Thus, it becomes likely that someone will submit such a patch. It will work for the developer, and it will also work in the autobuilders since all of them have git installed (otherwise they wouldn't be able to clone the Buildroot repo). Thus, any such patch will go unnoticed.
I don't think it hurts to force git as a dependency.
Regards,
Arnout
PATCH 1/1 support/scripts/apply-patches: use "git apply" as a fallback when applying patches
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Arnout" == Arnout Vandecappelle <[email protected]> writes:
Hi,
Quoted 11 lines
Quoted 4 lines
With this patch a package selection change that works on the developer's machine where git is installed, would fail on the production build machine that is missing git. Is there a clear error message in case of patch failure?
I'm with Baruch here. The reason to have this feature is exactly to have the
possibility to have patches that modify binaries or that modify symlinks. Thus, it becomes likely that someone will submit such a patch. It will work for the developer, and it will also work in the autobuilders since all of them have git installed (otherwise they wouldn't be able to clone the Buildroot repo). Thus, any such patch will go unnoticed.
I agree, that is not good.
Quoted 1 line
I don't think it hurts to force git as a dependency.
If we do that, should we then always use git apply instead of patch? (and drop the patch dependency)
Signature · 1 line
Bye, Peter Korsgaard