xfsprogs: Refactor release scripts to conform to using git archive
Ben Myers
Reviewed-byAndrew,
Quoted 9 lines · Andrew Dahl
Refactored release scripts to conform to using git archive
When generating a release, there is a risk of missing necessary source files. This is fixed by using git archive, which also fixes the lack of conformity between the xfs utilities. As well, some files may be stale during packaging. This is fixed with a clean at the beginning of release generation.
Signed-off-by Andrew Dahl <[email protected]>
Compared the old 'make dist' with the new one:
# diff old.list new.list 129a130
Quoted 1 line
xfsprogs-3.1.10/.gitignore
The only difference in the file list is the addition of .gitignore.
Quoted 19 lines
--- Makefile | 11 ++++++++--- release.sh | 3 +++ 2 files changed, 11 insertions(+), 3 deletions(-)Makefile +2
-0@@ -29,6 +29,8 @@ SRCTAR = $(PKG_NAME)-$(PKG_VERSION).tar.gz CONFIGURE = aclocal.m4 configure config.guess config.sub install-sh ltmain.sh LSRCFILES = configure.ac release.sh README VERSION $(CONFIGURE) +SRCTARINC = m4/libtool.m4 m4/lt~obsolete.m4 m4/ltoptions.m4 m4/ltsugar.m4 \ + m4/ltversion.m4 po/xfsprogs.pot $(CONFIGURE) LDIRT = config.log .ltdep .dep config.status config.cache confdefs.h \ conftest* built .census install.* install-dev.* *.gz \
Makefile $(SRCDIR) : $(_FORCE) · line 151
Quoted 9 lines
Makefile +3
-3@@ -144,7 +146,10 @@ $(SRCDIR) : $(_FORCE) rm -fr $@ mkdir -p $@ -$(SRCTAR) : default $(SRCDIR) - $(Q)$(MAKE) $(MAKEOPTS) source-link - unset TAPE; $(TAR) -cf - $(SRCDIR) | $(ZIP) --best > $@ && \ +$(SRCTAR) : default + $(Q)git archive --prefix=$(SRCDIR)/ --format=tar v$(PKG_VERSION) \ + -o $(SRCDIR).tar
Older versions of git do not support the -o option. I changed this to > $(SRCDIR).tar and it works fine.
Makefile $(SRCDIR) : $(_FORCE) · line 152
Quoted 1 line
Makefile +1
-0+ $(Q)$(TAR) --transform "s,^,$(SRCDIR)/," -rf $(SRCARC).tar \
SRCDIR
With those changes this looks and works fine.
-Ben
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Ben Myers
Reviewed-byRe: [PATCH 2/4] xfsdump: Refactor release scripts to conform to using git archive
In reply to a message not in the archiveAndrew,
Quoted 10 lines · Andrew Dahl
Refactored release scripts to conform to using git archive
When generating a release, there is a risk of some files being stale, such as configure and the m4/autotools temp files. This is fixed with a clean at the beginning of the release generation. In addition, there is no uniformity in the current method of source tar generation between xfs utilities. Using git archive solves this issue accross all utilities.
Signed-off-by Andrew Dahl <[email protected]>
Here is the difference between the old 'make dist' and the new one:
# diff old.list new.list 99a100
Quoted 1 line
xfsdump-3.1.2/.gitignore
190d190
< xfsdump-3.1.2/po/xfsdump.pot
Adding .gitignore is fine, and xfsdump.pot needn't be in the archive, it is an output file from gettext.
Quoted 14 lines
Makefile +2
-0@@ -29,6 +29,8 @@ SRCTAR = $(PKG_NAME)-$(PKG_VERSION).tar.gz CONFIGURE = aclocal.m4 configure config.guess config.sub install-sh ltmain.sh LSRCFILES = configure.ac release.sh README VERSION $(CONFIGURE) +SRCTARINC = m4/libtool.m4 m4/lt~obsolete.m4 m4/ltoptions.m4 m4/ltsugar.m4 \ + m4/ltversion.m4 $(CONFIGURE) LDIRT = config.log .ltdep .dep config.status config.cache confdefs.h \ conftest* built .census install.* install-dev.* *.gz \
Makefile $(SRCDIR) : $(_FORCE) · line 131
Quoted 9 lines
Makefile +3
-3@@ -124,7 +126,10 @@ $(SRCDIR) : $(_FORCE) rm -fr $@ mkdir -p $@ -$(SRCTAR) : default $(SRCDIR) - $(Q)$(MAKE) $(MAKEOPTS) source-link - unset TAPE; $(TAR) -cf - $(SRCDIR) | $(ZIP) --best > $@ && \ +$(SRCTAR) : default + $(Q)git archive --prefix=$(SRCDIR)/ --format=tar v$(PKG_VERSION) \ + -o $(SRCDIR).tar
Changed this to
Quoted 1 line
$(SRCDIR).tar
Regards,
Ben
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Ben Myers
Reviewed-byRe: [PATCH 3/4] xfstests: Refactor release scripts to conform to using git archive
In reply to a message not in the archiveAndrew,
Quoted 43 lines · Andrew Dahl
Refactored release scripts to conform to using git archive
When generating a release, there is a risk of some files being stale, such as configure and the m4/autotools temp files. This is fixed with a clean at the beginning of release generation. In addition, there is no uniformity in the current method of source tar generation between xfs utilities. Using git archive solves this issue across all utilities.
Signed-off-by Andrew Dahl <[email protected]>--- Makefile | 20 ++++++++++++++++++++ release.sh | 11 +++++++++-- 2 files changed, 29 insertions(+), 2 deletions(-) mode change 100644 => 100755 release.shMakefile +6
-0@@ -39,12 +39,17 @@ ifeq ($(HAVE_BUILDDEFS), yes) include $(TOPDIR)/include/builddefs endif +SRCTAR = $(PKG_NAME)-$(PKG_VERSION).tar.gz + TESTS = $(shell sed -n -e '/^[0-9][0-9][0-9]*/s/ .*//p' group) CONFIGURE = configure include/builddefs include/config.h LSRCFILES = configure configure.ac aclocal.m4 README VERSION LDIRT = config.log .ltdep .dep config.status config.cache confdefs.h \ conftest* check.log check.time +ifeq ($(HAVE_BUILDDEFS), yes) +LDIRT += $(SRCTAR) +endif LIB_SUBDIRS = include lib TOOL_SUBDIRS = ltp src m4@@ -102,3 +107,18 @@ install-dev install-lib: realclean distclean: clean $(Q)rm -f $(LDIRT) $(CONFIGURE) $(Q)rm -rf autom4te.cache Logs +
Makefile install-dev install-lib: · line 120
Quoted 10 lines
Makefile +10
-0+dist: include/builddefs include/config.h default +ifeq ($(HAVE_BUILDDEFS), no) + $(Q)$(MAKE) $(MAKEOPTS) -C . $@ +else + $(Q)$(MAKE) $(MAKEOPTS) $(SRCTAR) +endif + +$(SRCTAR) : default + $(Q)git archive --prefix=$(PKG_NAME)-$(PKG_VERSION)/ --format=tar \ + v$(PKG_VERSION) -o $(PKG_NAME)-$(PKG_VERSION).tar
Again, changed this to
Quoted 1 line
$(PKG_NAME)-$(PKG_VERSION).tar
And it worked fine.
Regards,
Ben
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Ben Myers
Reviewed-byRe: [PATCH 4/4] dmapi: Refactor release scripts to conform to using git archive
In reply to a message not in the archiveAndrew,
Quoted 36 lines · Andrew Dahl
Refactored release scripts to conform to using git archive
When generating a release, there is a risk of some files being stale, such as configure and the m4/autotools temp files. This is fixed with a clean at the beginning of release generation. In addition, there is no uniformity in the current method of source tar generation between xfs utilities. Using git archive solves this issue across all utilities.
Signed-off-by Andrew Dahl <[email protected]>--- Makefile | 19 +++++++++++++++++++ Makepkgs | 4 ++-- build/Makefile | 22 +--------------------- release.sh | 5 ++++- 4 files changed, 26 insertions(+), 24 deletions(-)Makefile +4
-0@@ -9,6 +9,8 @@ ifeq ($(HAVE_BUILDDEFS), yes) include $(TOPDIR)/include/builddefs endif +SRCTAR=$(PKG_NAME)-$(PKG_VERSION).tar.gz + CONFIGURE = aclocal.m4 configure config.guess config.sub configure install-sh \ ltmain.sh m4/libtool.m4 m4/ltoptions.m4 m4/ltsugar.m4 \ m4/ltversion.m4 m4/lt~obsolete.m4@@ -84,3 +86,19 @@ distclean: clean realclean: distclean rm -f $(CONFIGURE) + +dist: default
Makefile distclean: clean · line 100
Quoted 10 lines
Makefile +10
-0+ifeq ($(HAVE_BUILDDEFS), no) + $(MAKE) $(MAKEOPTS) -C . $@ +else + $(MAKE) $(MAKEOPTS) $(SRCTAR) + $(MAKE) $(MAKEOPTS) -C build dist +endif + +$(SRCTAR) : default + git archive --prefix=$(PKG_NAME)-$(PKG_VERSION)/ --format=tar \ + v$(PKG_VERSION) -o $(PKG_NAME)-$(PKG_VERSION).tar
>
Other than that it looks fine.
# diff old.list new.list 0a1
Quoted 1 line
dmapi-2.2.12/
1a3
Quoted 1 line
dmapi-2.2.12/build/
2a5
Quoted 1 line
dmapi-2.2.12/build/rpm/
6a10
Quoted 1 line
dmapi-2.2.12/build/tar/
12a17
Quoted 1 line
dmapi-2.2.12/debian/
17a23
Quoted 1 line
dmapi-2.2.12/doc/
22a29,30
Quoted 2 lines
dmapi-2.2.12/.gitignore
dmapi-2.2.12/include/
30a39
Quoted 1 line
dmapi-2.2.12/libdm/
49a59
Quoted 1 line
dmapi-2.2.12/m4/
62a73
Quoted 1 line
dmapi-2.2.12/man/
63a75
Quoted 1 line
dmapi-2.2.12/man/man3/
The old source tarball didn't have some directories in it. That's ok.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 0/4] Refactor release scripts to conform to using git archive
In reply to a message not in the archiveQuoted 5 lines · Andrew Dahl
The following four patches fix an issue seen during the last user space release where not all source files were included in xfsprogs. They each change xfsprogs, xfstests, xfsdump, and dmapi to all use git archive for source tarball generation and add 'make realclean' in the release scripts to ensure the configure and m4/autotools temp files are refreshed.
Applied this series.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 0/4] Refactor release scripts to conform to using git archive
Quoted 8 lines · Ben Myers
Quoted 5 lines · Andrew Dahl
The following four patches fix an issue seen during the last user space release where not all source files were included in xfsprogs. They each change xfsprogs, xfstests, xfsdump, and dmapi to all use git archive for source tarball generation and add 'make realclean' in the release scripts to ensure the configure and m4/autotools temp files are refreshed.
Applied this series.
It breaks the 'make deb' command for all the trees.
The debian package build creates a link tree so that it builds outside the main source tree (e.g. in ./xfsprogs-3.1.10) and as part of the build it runs 'make dist' to build a tarball from that link tree. That now fails with:
fatal: current working directory is untracked
coming from the git archive command that is now used to build the tarball.
Nathan, can the debian package build use git-archive? I'm under the impression that you have to be able to build debian packages from a source tarball or a source package, and so the build can't rely on being inside a VCS controlled tree. That rules out using git archive for tarball creation...
Also, given that the previous tarball creation used the same link tree build as the debian package build (i.e. make source-link), that implies the debian pacakges are going to be missing files using this technique. Doesn't that mean we have to fix the file list being fed into the source-link target regardless of whether we use git archive to build the tarball?
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 0/4] Refactor release scripts to conform to using git archive
Quoted 3 lines · Dave Chinner <david <at> fromorbit.com> writes:
It breaks the 'make deb' command for all the trees.
(thanks for checking!)
Quoted 7 lines
The debian package build creates a link tree so that it builds outside the main source tree (e.g. in ./xfsprogs-3.1.10) and as part of the build it runs 'make dist' to build a tarball from that link tree. That now fails with:
fatal: current working directory is untracked
Quoted 9 lines
coming from the git archive command that is now used to build the tarball.
Nathan, can the debian package build use git-archive? I'm under the impression that you have to be able to build debian packages from a source tarball or a source package, and so the build can't rely on being inside a VCS controlled tree. That rules out using git archive for tarball creation...
IIRC, the requirement is to be building in a tree that had a basename of for <package>-<version> - hence the link'd tree build previously. If that can be maintained somehow, the build should be fine. Symlinks did not work very well (IIRC) - the build tools might've used basename or realpath or something like that, but I do remember trying that.
Quoted 6 lines
Also, given that the previous tarball creation used the same link tree build as the debian package build (i.e. make source-link), that implies the debian pacakges are going to be missing files using this technique. Doesn't that mean we have to fix the file list being fed into the source-link target regardless of whether we use git archive to build the tarball?
The source tarball definitely needs to have the package-version prefix for all the files it contains - has a mechanism to acheive that been found?
cheers.
--
Nathan
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 0/4] Refactor release scripts to conform to using git?archive
Quoted 38 lines · Nathan Scott
Quoted 3 lines · Dave Chinner <david <at> fromorbit.com> writes:
It breaks the 'make deb' command for all the trees.
(thanks for checking!)
Quoted 7 lines
The debian package build creates a link tree so that it builds outside the main source tree (e.g. in ./xfsprogs-3.1.10) and as part of the build it runs 'make dist' to build a tarball from that link tree. That now fails with:
fatal: current working directory is untracked
Quoted 9 lines
coming from the git archive command that is now used to build the tarball.
Nathan, can the debian package build use git-archive? I'm under the impression that you have to be able to build debian packages from a source tarball or a source package, and so the build can't rely on being inside a VCS controlled tree. That rules out using git archive for tarball creation...
IIRC, the requirement is to be building in a tree that had a basename of for <package>-<version> - hence the link'd tree build previously. If that can be maintained somehow, the build should be fine. Symlinks did not work very well (IIRC) - the build tools might've used basename or realpath or something like that, but I do remember trying that.
Quoted 6 lines
Also, given that the previous tarball creation used the same link tree build as the debian package build (i.e. make source-link), that implies the debian pacakges are going to be missing files using this technique. Doesn't that mean we have to fix the file list being fed into the source-link target regardless of whether we use git archive to build the tarball?
Quoted 3 lines
The source tarball definitely needs to have the package-version prefix for all the files it contains - has a mechanism to acheive that been found?
It has... with a combo of git archive --prefix and tar --transform. I'll see what I can do to get 'make deb' going again.
Thanks,
Ben
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 0/4] Refactor release scripts to conform to using git?archive
In reply to Nathan ScottQuoted 7 lines · Nathan Scott
Quoted 3 lines · Dave Chinner <david <at> fromorbit.com> writes:
It breaks the 'make deb' command for all the trees.
(thanks for checking!)
I didn't - my CI system caught it immediately once I switched everything back on after being away for a week at LCA. It builds debs and ships them to VMs, installs them and then runs various tests after every upstream repository check in that occurs.
It was when I checked the RSS feeds coming back from the CI system that I noticed that it was busted and started looking into the problem....
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 0/4] Refactor release scripts to conform to using git archive
In reply to Dave ChinnerQuoted 11 lines · Dave Chinner
Quoted 8 lines · Ben Myers
Quoted 5 lines · Andrew Dahl
The following four patches fix an issue seen during the last user space release where not all source files were included in xfsprogs. They each change xfsprogs, xfstests, xfsdump, and dmapi to all use git archive for source tarball generation and add 'make realclean' in the release scripts to ensure the configure and m4/autotools temp files are refreshed.
Applied this series.
It breaks the 'make deb' command for all the trees.
xfsprogs: update 'make deb' to use tarball
This patch changes the build process so that 'make deb' uses the same process of creating a source tree as the release script.
* Add a list of files which go in the release tarball in .git_census This is needed so that you can create a tarball in a bare release tree, when .git is not available.
* Modify the SRCTAR target to include files from .git_census and use tar instead of git archive.
* Modify the SRCTARINC files to include .git_census, and include .git_census in the 'make realclean' target.
* 'make deb' now creates unsigned packages by default, 'make debsign' creates signed packages.
* remove the 'make source-link' target.
--- Makefile | 29 ++++++++++++++++------------- include/buildrules | 11 ----------- 2 files changed, 16 insertions(+), 24 deletions(-)
Makefile +16 -13
@@ -30,8 +30,7 @@ SRCTAR = $(PKG_NAME)-$(PKG_VERSION).tar. CONFIGURE = aclocal.m4 configure config.guess config.sub install-sh ltmain.sh LSRCFILES = configure.ac release.sh README VERSION $(CONFIGURE) SRCTARINC = m4/libtool.m4 m4/lt~obsolete.m4 m4/ltoptions.m4 m4/ltsugar.m4 \ - m4/ltversion.m4 po/xfsprogs.pot $(CONFIGURE) - + m4/ltversion.m4 po/xfsprogs.pot .git_census $(CONFIGURE) LDIRT = config.log .ltdep .dep config.status config.cache confdefs.h \ conftest* built .census install.* install-dev.* *.gz \ autom4te.cache/* libtool include/builddefs include/platform_defs.h
@@ -120,7 +119,7 @@ distclean: clean $(Q)rm -f $(LDIRT) realclean: distclean - $(Q)rm -f $(CONFIGURE) + $(Q)rm -f $(CONFIGURE) .git_census # # All this gunk is to allow for a make dist on an unconfigured tree
@@ -137,18 +136,22 @@ ifeq ($(HAVE_BUILDDEFS), no) $(Q)$(MAKE) $(MAKEOPTS) -C . $@ else $(Q)$(MAKE) $(MAKEOPTS) $(SRCDIR) - $(Q)$(MAKE) $(MAKEOPTS) -C po - $(Q)$(MAKE) $(MAKEOPTS) source-link - $(Q)cd $(SRCDIR) && dpkg-buildpackage + $(Q)cd $(SRCDIR) && dpkg-buildpackage -us -uc endif -$(SRCDIR) : $(_FORCE) +debsign: deb + debsign + +$(SRCDIR) : $(_FORCE) $(SRCTAR) rm -fr $@ - mkdir -p $@ + $(Q)$(TAR) -zxvf $(SRCTAR) -$(SRCTAR) : default - $(Q)git archive --prefix=$(SRCDIR)/ --format=tar v$(PKG_VERSION) > $(SRCDIR).tar - $(Q)$(TAR) --transform "s,^,$(SRCDIR)/," -rf $(SRCDIR).tar \ - $(SRCTARINC) - $(Q)$(ZIP) $(SRCDIR).tar +$(SRCTAR) : default $(SRCTARINC) .git_census + $(Q)$(TAR) --transform "s,^,$(SRCDIR)/," -zcf $(SRCDIR).tar.gz \ + `cat .git_census` $(SRCTARINC) echo Wrote: $@ + +.git_census: $(_FORCE) + $(Q)if test -d .git; then \ + git ls-files > .git_census && echo "new .git_census"; \ + fi
include/buildrules +0 -11
@@ -23,17 +23,6 @@ $(SUBDIRS): $(Q)$(MAKE) $(MAKEOPTS) -q -C $@ || $(MAKE) $(MAKEOPTS) -C $@ endif -source-link: - @test -z "$$DIR" && DIR="."; \ - for f in `echo $(SRCFILES) $(SUBDIRS) $(POTHEAD)`; do \ - if test -d $$f ; then \ - mkdir $(TOPDIR)/$(PKG_NAME)-$(PKG_VERSION)/$$DIR/$$f || exit $$?; \ - $(MAKEF) DIR=$$DIR/$$f -C $$f $@ || exit $$?; \ - else \ - ln $$f $(TOPDIR)/$(PKG_NAME)-$(PKG_VERSION)/$$DIR/$$f || exit $$?; \ - fi; \ - done - # # Standard targets #
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 0/4] Refactor release scripts to conform to using git archive
Hi Ben,
----- Original Message -----
Quoted 13 lines · Dave Chinner
Quoted 2 lines
...
It breaks the 'make deb' command for all the trees.xfsprogs: update 'make deb' to use tarball
This patch changes the build process so that 'make deb' uses the same process of creating a source tree as the release script.
* Add a list of files which go in the release tarball in .git_census This is needed so that you can create a tarball in a bare release tree, when .git is not available.
I think you'll need .git_census in .gitignore (ala configure). Possibly remove the underscore for naming consistency - *shrug*.
Quoted 4 lines
* 'make deb' now creates unsigned packages by default, 'make debsign' creates signed packages.
Ehrm - why? Everything else in your patch worked, but this part broke (which suggests a larger problem, in that this build path is no longer checked on every build) ...
debsign: Can't find or can't read changes file !
Makefile ifeq ($(HAVE_BUILDDEFS), no) · line 144
Quoted 3 lines
Makefile +3
-0+debsign: deb + debsign +
(That's not valid usage, FWIW).
I would recommend just removing that change in behaviour, "make deb" was fine as it was (for me anyway, and evidently for Dave too). Best to go secure-by-default and not change this.
thanks!
--
Nathan
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 0/4] Refactor release scripts to conform to using git archive
Hey Nathan,
Quoted 10 lines · Nathan Scott
----- Original Message -----
Quoted 9 lines · Dave Chinner
Quoted 2 lines
...
It breaks the 'make deb' command for all the trees.xfsprogs: update 'make deb' to use tarball
This patch changes the build process so that 'make deb' uses the same process of creating a source tree as the release script.
Quoted 7 lines
Quoted 4 lines
* Add a list of files which go in the release tarball in .git_census This is needed so that you can create a tarball in a bare release tree, when .git is not available.
I think you'll need .git_census in .gitignore (ala configure). Possibly remove the underscore for naming consistency - *shrug*.
Will do. Thanks.
Quoted 18 lines
Quoted 5 lines
* 'make deb' now creates unsigned packages by default, 'make debsign' creates signed packages.
Ehrm - why? Everything else in your patch worked, but this part broke (which suggests a larger problem, in that this build path is no longer checked on every build) ...
debsign: Can't find or can't read changes file !
Makefile ifeq ($(HAVE_BUILDDEFS), no) · line 144
Quoted 3 lines
Makefile +3
-0+debsign: deb + debsign +(That's not valid usage, FWIW).
Quoted 3 lines
I would recommend just removing that change in behaviour, "make deb" was fine as it was (for me anyway, and evidently for Dave too). Best to go secure-by-default and not change this.
'make deb' failed for me like this:
signfile xfsprogs_3.1.9.dsc
gpg: skipped "Nathan Scott <[email protected]>": secret key not available
gpg: [stdin]: clearsign failed: secret key not available
dpkg-genchanges >../xfsprogs_3.1.9_amd64.changes
dpkg-genchanges: including full source code in upload
dpkg-buildpackage: full upload; Debian-native package (full source is included)
dpkg-buildpackage: warning: Failed to sign .dsc and .changes file
make: *** [deb] Error 1
Secure-by-default is a great idea but it seems that in this case it also requires that anyone who wants to do a build for debian has to do it on a machine with a keyring containing your secret key. ;)
I think it is also reasonable for devs to be able to build test debs on machines that have empty keyrings. An unsigned 'make deb' does return 0. I'd like this to suit everyone's needs if it can. I'll split this part out into a separate patch and try to figure something out later.
Thanks,
Ben
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
xfsprogs: update 'make deb' to use tarball
This patch changes the build process so that 'make deb' uses the same process of creating a source tree as the release script.
* Add a list of files which go in the release tarball in .gitcensus This is needed so that you can create a tarball in a bare release tree, when .git is not available.
* Modify the SRCTAR target to include files from .git_census and use tar instead of git archive.
* Modify the SRCTARINC files to include .git_census, and include .git_census in the 'make realclean' target.
* remove the 'make source-link' target.
---
v2: don't muck with signing the deb in this patch, rename .git_census to .gitcensus, add .gitcensus to .gitignore.
.gitignore | 1 + Makefile | 24 ++++++++++++------------ include/buildrules | 11 ----------- 3 files changed, 13 insertions(+), 23 deletions(-)
.gitignore +1 -0
@@ -5,6 +5,7 @@ # build system .census +.gitcensus /include/platform_defs.h /include/builddefs /install-sh
Makefile +12 -12
@@ -30,8 +30,7 @@ SRCTAR = $(PKG_NAME)-$(PKG_VERSION).tar. CONFIGURE = aclocal.m4 configure config.guess config.sub install-sh ltmain.sh LSRCFILES = configure.ac release.sh README VERSION $(CONFIGURE) SRCTARINC = m4/libtool.m4 m4/lt~obsolete.m4 m4/ltoptions.m4 m4/ltsugar.m4 \ - m4/ltversion.m4 po/xfsprogs.pot $(CONFIGURE) - + m4/ltversion.m4 po/xfsprogs.pot .gitcensus $(CONFIGURE) LDIRT = config.log .ltdep .dep config.status config.cache confdefs.h \ conftest* built .census install.* install-dev.* *.gz \ autom4te.cache/* libtool include/builddefs include/platform_defs.h
@@ -120,7 +119,7 @@ distclean: clean $(Q)rm -f $(LDIRT) realclean: distclean - $(Q)rm -f $(CONFIGURE) + $(Q)rm -f $(CONFIGURE) .gitcensus # # All this gunk is to allow for a make dist on an unconfigured tree
@@ -137,18 +136,19 @@ ifeq ($(HAVE_BUILDDEFS), no) $(Q)$(MAKE) $(MAKEOPTS) -C . $@ else $(Q)$(MAKE) $(MAKEOPTS) $(SRCDIR) - $(Q)$(MAKE) $(MAKEOPTS) -C po - $(Q)$(MAKE) $(MAKEOPTS) source-link $(Q)cd $(SRCDIR) && dpkg-buildpackage endif -$(SRCDIR) : $(_FORCE) +$(SRCDIR) : $(_FORCE) $(SRCTAR) rm -fr $@ - mkdir -p $@ + $(Q)$(TAR) -zxvf $(SRCTAR) -$(SRCTAR) : default - $(Q)git archive --prefix=$(SRCDIR)/ --format=tar v$(PKG_VERSION) > $(SRCDIR).tar - $(Q)$(TAR) --transform "s,^,$(SRCDIR)/," -rf $(SRCDIR).tar \ - $(SRCTARINC) - $(Q)$(ZIP) $(SRCDIR).tar +$(SRCTAR) : default $(SRCTARINC) .gitcensus + $(Q)$(TAR) --transform "s,^,$(SRCDIR)/," -zcf $(SRCDIR).tar.gz \ + `cat .gitcensus` $(SRCTARINC) echo Wrote: $@ + +.gitcensus: $(_FORCE) + $(Q)if test -d .git; then \ + git ls-files > .gitcensus && echo "new .gitcensus"; \ + fi
include/buildrules +0 -11
@@ -23,17 +23,6 @@ $(SUBDIRS): $(Q)$(MAKE) $(MAKEOPTS) -q -C $@ || $(MAKE) $(MAKEOPTS) -C $@ endif -source-link: - @test -z "$$DIR" && DIR="."; \ - for f in `echo $(SRCFILES) $(SUBDIRS) $(POTHEAD)`; do \ - if test -d $$f ; then \ - mkdir $(TOPDIR)/$(PKG_NAME)-$(PKG_VERSION)/$$DIR/$$f || exit $$?; \ - $(MAKEF) DIR=$$DIR/$$f -C $$f $@ || exit $$?; \ - else \ - ln $$f $(TOPDIR)/$(PKG_NAME)-$(PKG_VERSION)/$$DIR/$$f || exit $$?; \ - fi; \ - done - # # Standard targets #
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
xfsdump: update 'make deb' to use tarball archive
In reply to Ben MyersThis patch changes the build process so that 'make deb' uses the same process of creating a source tree as the release script.
* Add a list of files which go in the release tarball in .gitcensus This is needed so that you can create a tarball in a bare release tree, when .git is not available.
* Modify the SRCTAR target to include files from .gitcensus and use tar instead of git archive.
* Modify the SRCTARINC files to include .gitcensus, and include .gitcensus in the 'make realclean' target.
* remove the 'make source-link' target.
--- .gitignore | 1 + Makefile | 23 ++++++++++++----------- include/buildrules | 11 ----------- 3 files changed, 13 insertions(+), 22 deletions(-)
.gitignore +1 -0
@@ -4,6 +4,7 @@ # build system .census +.gitcensus /include/config.h /include/builddefs /install-sh
Makefile +12 -11
@@ -30,7 +30,7 @@ SRCTAR = $(PKG_NAME)-$(PKG_VERSION).tar. CONFIGURE = aclocal.m4 configure config.guess config.sub install-sh ltmain.sh LSRCFILES = configure.ac release.sh README VERSION $(CONFIGURE) SRCTARINC = m4/libtool.m4 m4/lt~obsolete.m4 m4/ltoptions.m4 m4/ltsugar.m4 \ - m4/ltversion.m4 $(CONFIGURE) + m4/ltversion.m4 .gitcensus $(CONFIGURE) LDIRT = config.log .ltdep .dep config.status config.cache confdefs.h \ conftest* built .census install.* install-dev.* *.gz \
@@ -100,7 +100,7 @@ distclean: clean rm -f $(LDIRT) realclean: distclean - rm -f $(CONFIGURE) + rm -f $(CONFIGURE) .gitcensus # # All this gunk is to allow for a make dist on an unconfigured tree
@@ -117,18 +117,19 @@ ifeq ($(HAVE_BUILDDEFS), no) $(MAKE) $(MAKEOPTS) -C . $@ else $(Q)$(MAKE) $(MAKEOPTS) $(SRCDIR) - $(Q)$(MAKE) $(MAKEOPTS) -C po - $(Q)$(MAKE) $(MAKEOPTS) source-link $(Q)cd $(SRCDIR) && dpkg-buildpackage endif -$(SRCDIR) : $(_FORCE) +$(SRCDIR) : $(_FORCE) $(SRCTAR) rm -fr $@ - mkdir -p $@ + $(Q)$(TAR) -zxvf $(SRCTAR) -$(SRCTAR) : default - $(Q)git archive --prefix=$(SRCDIR)/ --format=tar v$(PKG_VERSION) > $(SRCDIR).tar - $(Q)$(TAR) --transform "s,^,$(SRCDIR)/," -rf $(SRCDIR).tar \ - $(SRCTARINC) - $(Q)$(ZIP) $(SRCDIR).tar +$(SRCTAR) : default $(SRCTARINC) .gitcensus + $(Q)$(TAR) --transform "s,^,$(SRCDIR)/," -zcf $(SRCDIR).tar.gz \ + `cat .gitcensus` $(SRCTARINC) echo Wrote: $@ + +.gitcensus: $(_FORCE) + $(Q) if test -d .git; then \ + git ls-files > .gitcensus && echo "new .gitcensus"; \ + fi
include/buildrules +0 -11
@@ -23,17 +23,6 @@ $(SUBDIRS): $(Q)$(MAKE) $(MAKEOPTS) -q -C $@ || $(MAKE) $(MAKEOPTS) -C $@ endif -source-link: - @test -z "$$DIR" && DIR="."; \ - for f in `echo $(SRCFILES) $(SUBDIRS) $(POTHEAD)`; do \ - if test -d $$f ; then \ - mkdir $(TOPDIR)/$(PKG_NAME)-$(PKG_VERSION)/$$DIR/$$f || exit $$?; \ - $(MAKEF) DIR=$$DIR/$$f -C $$f $@ || exit $$?; \ - else \ - ln $$f $(TOPDIR)/$(PKG_NAME)-$(PKG_VERSION)/$$DIR/$$f || exit $$?; \ - fi; \ - done - # # Standard targets #
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 0/4] Refactor release scripts to conform to using git archive
In reply to Ben Myers----- Original Message -----
Quoted 15 lines
...
'make deb' failed for me like this:signfile xfsprogs_3.1.9.dsc
gpg: skipped "Nathan Scott <[email protected]>": secret key not available
gpg: [stdin]: clearsign failed: secret key not availabledpkg-genchanges >../xfsprogs_3.1.9_amd64.changes
dpkg-genchanges: including full source code in upload
dpkg-buildpackage: full upload; Debian-native package (full source is included)
dpkg-buildpackage: warning: Failed to sign .dsc and .changes file
make: *** [deb] Error 1
Quoted 6 lines
Secure-by-default is a great idea but it seems that in this case it also
requires that anyone who wants to do a build for debian has to do it on a
machine with a keyring containing your secret key. ;)
Awkward. :)
Quoted 1 line
I think it is also reasonable for devs to be able to build test debs
*nod* ... not sure what trickery Dave had used. But, perhaps a better way to tackle this would be similar to the way the build allows extra configure options to be passed in.
include/builddefs: configure
./configure $$LOCAL_CONFIGURE_OPTIONS
so, perhaps something like this...
# LOCAL_BUILDPACKAGE_OPTIONS="-us -uc" for unsigned debs
deb: include/builddefs include/platform_defs.h
...
$(Q)cd $(SRCDIR) && dpkg-buildpackage $$LOCAL_BUILDPACKAGE_OPTIONS
cheers.
--
Nathan
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Nathan Scott
Reviewed-byRe: xfsprogs: update 'make deb' to use tarball
In reply to Ben Myers----- Original Message -----
Quoted 11 lines
This patch changes the build process so that 'make deb' uses the same process of creating a source tree as the release script.
* Add a list of files which go in the release tarball in .gitcensus This is needed so that you can create a tarball in a bare release tree, when .git is not available.
* Modify the SRCTAR target to include files from .git_census and use tar
instead of git archive.
Quoted 7 lines
* Modify the SRCTARINC files to include .git_census, and include .git_census in the 'make realclean' target.
* remove the 'make source-link' target.
Signed-off-by Ben Myers <[email protected]>
thanks!
--
Nathan
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Nathan Scott
Reviewed-byRe: xfsdump: update 'make deb' to use tarball archive
In reply to Ben Myers----- Original Message -----
Quoted 11 lines
This patch changes the build process so that 'make deb' uses the same process of creating a source tree as the release script.
* Add a list of files which go in the release tarball in .gitcensus This is needed so that you can create a tarball in a bare release tree, when .git is not available.
* Modify the SRCTAR target to include files from .gitcensus and use tar
instead of git archive.
Quoted 7 lines
* Modify the SRCTARINC files to include .gitcensus, and include .gitcensus in the 'make realclean' target.
* remove the 'make source-link' target.
Signed-off-by Ben Myers <[email protected]>
thanks!
--
Nathan
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 0/4] Refactor release scripts to conform to using git archive
In reply to Ben MyersQuoted 51 lines · Ben Myers
Hey Nathan,
Quoted 10 lines · Nathan Scott
----- Original Message -----
Quoted 9 lines · Dave Chinner
Quoted 2 lines
...
It breaks the 'make deb' command for all the trees.xfsprogs: update 'make deb' to use tarball
This patch changes the build process so that 'make deb' uses the same process of creating a source tree as the release script.
Quoted 7 lines
Quoted 4 lines
* Add a list of files which go in the release tarball in .git_census This is needed so that you can create a tarball in a bare release tree, when .git is not available.
I think you'll need .git_census in .gitignore (ala configure). Possibly remove the underscore for naming consistency - *shrug*.
Will do. Thanks.
Quoted 18 lines
Quoted 5 lines
* 'make deb' now creates unsigned packages by default, 'make debsign' creates signed packages.
Ehrm - why? Everything else in your patch worked, but this part broke (which suggests a larger problem, in that this build path is no longer checked on every build) ...
debsign: Can't find or can't read changes file !
Makefile ifeq ($(HAVE_BUILDDEFS), no) · line 144
Quoted 3 lines
Makefile +3
-0+debsign: deb + debsign +(That's not valid usage, FWIW).
Quoted 3 lines
I would recommend just removing that change in behaviour, "make deb" was fine as it was (for me anyway, and evidently for Dave too). Best to go secure-by-default and not change this.
'make deb' failed for me like this:
signfile xfsprogs_3.1.9.dsc
gpg: skipped "Nathan Scott <[email protected]>": secret key not available
gpg: [stdin]: clearsign failed: secret key not available
Quoted 5 lines
dpkg-genchanges >../xfsprogs_3.1.9_amd64.changes
dpkg-genchanges: including full source code in upload
dpkg-buildpackage: full upload; Debian-native package (full source is included)
dpkg-buildpackage: warning: Failed to sign .dsc and .changes file
make: *** [deb] Error 1
Don't know what you are doing differently to me, but make deb is not even trying to build signed packages on my build machine. It never has, and I don't ever recall seeing make deb fail because it was trying to build signed packages in the last 5 years on any of my machines...
I'm not passing -uc -us to dpkg-buildpackage, so somewhere aling the line it is working out that it shouldn't be signing the packages itself...
Quoted 2 lines
I think it is also reasonable for devs to be able to build test debs on machines that have empty keyrings.
That's exactly what I have been doing for a long time.
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 0/4] Refactor release scripts to conform to using git archive
In reply to Nathan ScottQuoted 16 lines · Nathan Scott
----- Original Message -----
Quoted 15 lines
...
'make deb' failed for me like this:signfile xfsprogs_3.1.9.dsc
gpg: skipped "Nathan Scott <[email protected]>": secret key not available
gpg: [stdin]: clearsign failed: secret key not availabledpkg-genchanges >../xfsprogs_3.1.9_amd64.changes
dpkg-genchanges: including full source code in upload
dpkg-buildpackage: full upload; Debian-native package (full source is included)
dpkg-buildpackage: warning: Failed to sign .dsc and .changes file
make: *** [deb] Error 1
Quoted 8 lines
Quoted 6 lines
Secure-by-default is a great idea but it seems that in this case it also
requires that anyone who wants to do a build for debian has to do it on a
machine with a keyring containing your secret key. ;)Awkward. :)
Really I don't mind. What was your passphrase again?
Quoted 3 lines
Quoted 1 line
I think it is also reasonable for devs to be able to build test debs
*nod* ... not sure what trickery Dave had used.
I tried setting DEB_BUILD_OPTIONS="-us -uc" but no luck.
Quoted 8 lines
But, perhaps a better
way to tackle this would be similar to the way the build allows extra configure options to be passed in.
include/builddefs: configure
./configure $$LOCAL_CONFIGURE_OPTIONS
Quoted 6 lines
so, perhaps something like this...
# LOCAL_BUILDPACKAGE_OPTIONS="-us -uc" for unsigned debs
deb:include/builddefsinclude/platform_defs.h
...$(Q)cd $(SRCDIR) && dpkg-buildpackage $$LOCAL_BUILDPACKAGE_OPTIONS
Gotcha. I'll take a look. Thanks for the reviews!
-Ben
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Dave Chinner
Reviewed-byRe: xfsprogs: update 'make deb' to use tarball
In reply to Ben MyersQuoted 10 lines · Ben Myers
This patch changes the build process so that 'make deb' uses the same process of creating a source tree as the release script.
* Add a list of files which go in the release tarball in .gitcensus This is needed so that you can create a tarball in a bare release tree, when .git is not available.
* Modify the SRCTAR target to include files from .git_census and use tar instead of git archive.
Quoted 6 lines
* Modify the SRCTARINC files to include .git_census, and include .git_census in the 'make realclean' target.
* remove the 'make source-link' target.
Signed-off-by Ben Myers <[email protected]>
Works fine here.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Dave Chinner
Reviewed-byRe: xfsdump: update 'make deb' to use tarball archive
In reply to Ben MyersQuoted 10 lines · Ben Myers
This patch changes the build process so that 'make deb' uses the same process of creating a source tree as the release script.
* Add a list of files which go in the release tarball in .gitcensus This is needed so that you can create a tarball in a bare release tree, when .git is not available.
* Modify the SRCTAR target to include files from .gitcensus and use tar instead of git archive.
Quoted 6 lines
* Modify the SRCTARINC files to include .gitcensus, and include .gitcensus in the 'make realclean' target.
* remove the 'make source-link' target.
Signed-off-by Ben Myers <[email protected]>
Works fine.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs