package/pkg-generic.mk: add --delete to rsync

When a source overriden package is built and then its source code is modified and rebuilt with make package-rebuild, it is needed for the build directory to contain exactly the same source that is in the original directory, so rsync must not only copy missing files, but also remove those that existed previously but now don't.

Signed-off-by Alvaro G. M <[email protected]>

---
 package/pkg-generic.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

package/pkg-generic.mk +0 -0

index f09f83e..440a750 100644

Dear Alvaro G. M,

Quoted 16 lines · Alvaro G. M

When a source overriden package is built and then its source code is modified and rebuilt with make package-rebuild, it is needed for the build directory to contain exactly the same source that is in the original directory, so rsync must not only copy missing files, but also remove those that existed previously but now don't.

Signed-off-by Alvaro G. M <[email protected]>

---
 package/pkg-generic.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

package/pkg-generic.mk +0 -0

index f09f83e..440a750 100644

package/pkg-generic.mk $(BUILD_DIR)/%/.stamp_rsynced: · line 122

Quoted 7 lines

package/pkg-generic.mk +1 -1

@@ -117,7 +117,7 @@ $(BUILD_DIR)/%/.stamp_rsynced:
      @$(call MESSAGE,"Syncing from source dir $(SRCDIR)")
      @test -d $(SRCDIR) || (echo "ERROR: $(SRCDIR) does not exist" ; exit 1)
      $(foreach hook,$($(PKG)_PRE_RSYNC_HOOKS),$(call $(hook))$(sep))
-     rsync -au $(RSYNC_VCS_EXCLUSIONS) $(SRCDIR)/ $(@D)
+     rsync --delete -au $(RSYNC_VCS_EXCLUSIONS) $(SRCDIR)/ $(@D)
      $(foreach hook,$($(PKG)_POST_RSYNC_HOOKS),$(call $(hook))$(sep))
      $(Q)touch $@

I don't remember how rsync behave exactly, but wouldn't this also remove object files, shared libraries and executables that will have been built in $(@D) ?

Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

You would like for the object files to remain there in case nothing or little has changed and its compilation can be omitted by the build tool (make or whatever).

But it is my understanding that the main idea is to make $(@D) a copy of the source to do the build there.
I've faced a problem in the case of a build that basically does gcc *.c -o out. In this situation, if I remove one .c file on the source directory, this doesn't get removed from $(@D) and the next time I invoke package-rebuild this removed file is still compiled, even though it doesn't exist now on the source.

When a source overriden package is built and then its source code is modified and rebuilt with make package-rebuild, it is needed for the build directory to contain exactly the same source that is in the original directory, so rsync must not only copy missing files, but also remove those that existed previously but now don't.

Signed-off-by Alvaro G. M <[email protected]>

---
 package/pkg-generic.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Dear Alvaro G. M,

Quoted 10 lines · Alvaro G. M

When a source overriden package is built and then its source code is modified and rebuilt with make package-rebuild, it is needed for the build directory to contain exactly the same source that is in the original directory, so rsync must not only copy missing files, but also remove those that existed previously but now don't.

Signed-off-by Alvaro G. M <[email protected]>

---
 package/pkg-generic.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

The fact that adding --delete would delete all object/generated files makes this patch really not usable, as it basically kills completely the entire purpose of OVERRIDE_SRCDIR + make <foo>-rebuild. We think that the small advantage brought by this patch is largely outweighed by the major inconvenience it causes. The build system of most packages is reasonable, and will not take into account random source files hanging around. If the build system of your package is so broken that it builds random files by just building *.c or *.cpp, then it's really the build system that should be fixed.

Since there is no way to distinguish the files to remove from the files to keep, we prefer to keep them all, in order to keep its usefulness to 'make <foo>-rebuild'.

As a consequence, we've marked the patch as Rejected in our patch tracking system.

Best regards,

Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Although it's true that something like gcc *.c isn't very elegant, I can't agree on being the build system broken.

After all, if you start
adding and removing files from the source code, the expectation should be that the next build is going to be different from the previous one. Nevertheless, if this hasn't come to attention before it's probably a non problem (I saw this happen on a custom package that isn't even a real package, just fiddling around and making tests), so I understand the reject.

The only solution I can think of that satisfies both conditions (fast builds and identical tree) is something like saving (where?) the output of find $(X_OVERRIDE_SRCDIR) before rsyncing for the first time, and after rsyncing for the second time remove files listed on the saved list from before previous to doing the next rsync.

Do you have any opinion on this idea? Admittedly, it seems ugly.