Makefile: add target to clean targetfs

Hi Emeric,

Makefile graph-depends: graph-depends-requirements · line 685

Quoted 4 lines · Emeric Vigier

Makefile +4 -0

+clean-target:
+	rm -rf $(TARGET_DIR) $(STAGING_DIR) $(BUILD_DIR)/.root \
+		$(BUILD_DIR)/*/.stamp_target_installed $(BUILD_DIR)/*/.stamp_staging_installed \
+		$(BUILD_DIR)/linux-*/.stamp_installed $(BUILD_DIR)/*/.built

.stamp_built ?

baruch

Quoted 7 lines · Baruch Siach

Makefile graph-depends: graph-depends-requirements · line 685

Quoted 4 lines · Emeric Vigier

Makefile +4 -0

+clean-target:
+	rm -rf $(TARGET_DIR) $(STAGING_DIR) $(BUILD_DIR)/.root \
+		$(BUILD_DIR)/*/.stamp_target_installed $(BUILD_DIR)/*/.stamp_staging_installed \
+		$(BUILD_DIR)/linux-*/.stamp_installed $(BUILD_DIR)/*/.built

.stamp_built ?

Yes, you are right.

Also it seems that linux-*/.stamp_installed is deprecated. I only see .stamp_target_installed files.

If you had some files added to the targetfs (fs-overlay, new packages, ...) and you no longer need them, buildroot does not offer a simple way to remove these items from the output/target directory. The rule added by this commit allows you to clean the targetfs. Issuing 'make' afterward will generate a new and clean targetfs.

A section in the documentation is also added. It describes few examples when developers could need it. 'make help' is also updated accordingly.

Signed-off-by Emeric Vigier <[email protected]>

---
Changes v1 -> v2
- remove non-existent .built and deprecated .stamp_installed references (suggested by Baruch Siach)
- mention the new target in 'make help'

---
 Makefile                                |  5 +++++
 docs/manual/advanced.txt                |  2 ++
 docs/manual/clean-target-filesystem.txt | 37 +++++++++++++++++++++++++++++++++
 3 files changed, 44 insertions(+)
 create mode 100644 docs/manual/clean-target-filesystem.txt

docs/manual/clean-target-filesystem.txt new file +6 -0

@@ -0,0 +1,37 @@
+// -*- mode:doc; -*-
+// vim: set syntax=asciidoc:
+
+==== Clean target filesystem
+
+There are various cases where you would like to clean the target

docs/manual/clean-target-filesystem.txt line 16

Quoted 6 lines

+1. You enabled +PACKAGE_OPENSSH+ to have a SSH server running on your

docs/manual/clean-target-filesystem.txt new file +5 -0

+   board. But you figure out that +sshd+ does not run well on your
+   target's architecture. Thus you decide to disable +PACKAGE_OPENSSH+
+   and enable +PACKAGE_DROPBEAR+ instead. You run +make+, +openssh+
+   gets built and installed. Problem: +openssh+ files are still present
+   in +output/target+!

One case that comes to mind here is that will this ensure that packages are installed in the correct order with dependencies? I am not exactly sure how the dependencies for packages works, but if you remove all the '.stamp_target_install' will that ensure that all the packages get installed in the correct order?

Busybox is the package that comes to my mind first that we need to ensure the installation order. From testing your scenario outlined above it appears that busybox gets installed before even dropbear would get built which would ensure that any full version of busybox would get install correctly over the symlinks to busybox.

Hi Ryan,

Quoted 13 lines · Ryan Barnett

docs/manual/clean-target-filesystem.txt line 16

Quoted 6 lines

+1. You enabled +PACKAGE_OPENSSH+ to have a SSH server running on your

docs/manual/clean-target-filesystem.txt new file +5 -0

+   board. But you figure out that +sshd+ does not run well on your
+   target's architecture. Thus you decide to disable +PACKAGE_OPENSSH+
+   and enable +PACKAGE_DROPBEAR+ instead. You run +make+, +openssh+
+   gets built and installed. Problem: +openssh+ files are still present
+   in +output/target+!

One case that comes to mind here is that will this ensure that packages are installed in the correct order with dependencies? I am not exactly sure how the dependencies for packages works, but if you remove all the '.stamp_target_install' will that ensure that all the packages get installed in the correct order?

Quoted 5 lines

Busybox is the package that comes to my mind first that we need to ensure the installation order. From testing your scenario outlined above it appears that busybox gets installed before even dropbear would get built which would ensure that any full version of busybox would get install correctly over the symlinks to busybox.

The package dependencies system should ensure the right installation order. The following excerpts from package/pkg-generic.mk demonstrate that:

$(2)_TARGET_INSTALL_TARGET = $$($(2)_DIR)/.stamp_target_installed

...

$(1): $(1)-install

...

$(1)-install: $(1)-install-staging $(1)-install-target $(1)-install-images

...

$(1)-install-target: $$($(2)_TARGET_INSTALL_TARGET)

baruch

--
http://baruch.siach.name/blog/ ~. .~ Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
- baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il -
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

I submitted a couple of patches some times ago, but they was rejected because potentially dangerous for users[1][2].
So probably you will not have that much luck with this patch.

[1] http://lists.busybox.net/pipermail/buildroot/2014-July/101963.html
[2] http://lists.busybox.net/pipermail/buildroot/2014-July/103143.html

Hi Angelo,

Quoted 6 lines · Angelo Compagnucci

I submitted a couple of patches some times ago, but they was rejected because potentially dangerous for users[1][2].
So probably you will not have that much luck with this patch.

[1] http://lists.busybox.net/pipermail/buildroot/2014-July/101963.html
[2] http://lists.busybox.net/pipermail/buildroot/2014-July/103143.html

Well, it is not me suggesting this patch, it is Emeric Vigier. But yes, one of our clients asked for this functionality, so I really want this patch applied.

As for previous rejections, the response of ThomasP[1] and Arnout[2] seems to indicate that they would be willing to accept this functional addition provided that it is (1) properly documented[1], and (2) prints a "big fat warning"[2]. Something like Thomas' scenario description[3] should be added to the documentation part of this patch.

Thanks for reminding us this previous discussion.

[1] http://lists.busybox.net/pipermail/buildroot/2014-July/101966.html

[2] http://lists.busybox.net/pipermail/buildroot/2014-July/102064.html

[3] http://lists.busybox.net/pipermail/buildroot/2014-July/101941.html

baruch

--
http://baruch.siach.name/blog/ ~. .~ Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
- baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il -

Hi Baruch,

2015-03-11 9:58 GMT+01:00 Baruch Siach <[email protected]>:

Quoted 23 lines

Hi Angelo,

Quoted 6 lines · Angelo Compagnucci

I submitted a couple of patches some times ago, but they was rejected because potentially dangerous for users[1][2].
So probably you will not have that much luck with this patch.

[1] http://lists.busybox.net/pipermail/buildroot/2014-July/101963.html
[2] http://lists.busybox.net/pipermail/buildroot/2014-July/103143.html

Well, it is not me suggesting this patch, it is Emeric Vigier. But yes, one of our clients asked for this functionality, so I really want this patch applied.

As for previous rejections, the response of ThomasP[1] and Arnout[2] seems to indicate that they would be willing to accept this functional addition provided that it is (1) properly documented[1], and (2) prints a "big fat warning"[2]. Something like Thomas' scenario description[3] should be added to the documentation part of this patch.

Thanks for reminding us this previous discussion.

[1] http://lists.busybox.net/pipermail/buildroot/2014-July/101966.html

Quoted 3 lines

[2] http://lists.busybox.net/pipermail/buildroot/2014-July/102064.html

[3] http://lists.busybox.net/pipermail/buildroot/2014-July/101941.html

I have my patch on top of my git rebase everytime I pull from master, so I will be veeeeery glad if you can have it upstreamed! I love so much the option to clean the target that I use it everyday!

Sincerely, Angelo

Quoted 7 lines

baruch

--
http://baruch.siach.name/blog/ ~. .~ Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
- baruch at tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il -

Emeric,

In the future - could you put a version - such as 'v2' - into the title utilizing the -'-subject-prefix' parameter of git-send-email?

For details and an example see the buildroot manual and section 21.5.2:

http://nightly.buildroot.org/manual.html#submitting-patches

Would it make sense to also update the section of the manual that talks about understand when a full-rebuild is necessary? You could also update that section to reference this advance usage section.

http://nightly.buildroot.org/manual.html#full-rebuild

Quoted 6 lines

docs/manual/clean-target-filesystem.txt new file +4 -0

@@ -0,0 +1,37 @@
+// -*- mode:doc; -*-
+// vim: set syntax=asciidoc:
+
+==== Clean target filesystem

docs/manual/clean-target-filesystem.txt line 14

Quoted 10 lines

docs/manual/clean-target-filesystem.txt new file +10 -0

+
+There are various cases where you would like to clean the target
+filesystem. Especially once you understood that buildroot build system
+is kept simple, and does not allow to uninstall files. Let's take some
+examples:
+
+1. You enabled +PACKAGE_OPENSSH+ to have a SSH server running on your
+   board. But you figure out that +sshd+ does not run well on your
+   target's architecture. Thus you decide to disable +PACKAGE_OPENSSH+
+   and enable +PACKAGE_DROPBEAR+ instead. You run +make+, +openssh+

docs/manual/clean-target-filesystem.txt line 16

Quoted 2 lines

docs/manual/clean-target-filesystem.txt new file +2 -0

+   gets built and installed. Problem: +openssh+ files are still present
+   in +output/target+!

One case that comes to mind here is that will this ensure that packages are installed in the correct order with dependencies? I am not exactly sure how the dependencies for packages works, but if you remove all the '.stamp_target_install' will that ensure that all the packages get installed in the correct order?

Busybox is the package that comes to my mind first that we need to ensure the installation order. From testing your scenario outlined above it appears that busybox gets installed before even dropbear would get built which would ensure that any full version of busybox would get install correctly over the symlinks to busybox.

docs/manual/clean-target-filesystem.txt line 25

Quoted 8 lines

docs/manual/clean-target-filesystem.txt new file +8 -0

+2. You enabled +PACKAGE_NTPD+ and set the local timezone to
+   +America/Montreal+ with +BR2_TARGET_LOCALTIME+ config. Then you find
+   out that +Montreal+ local has been replaced by +Toronto+ (French
+   looks like a rogue language). You enter +make menuconfig+, make the
+   change, save, and run +make+ again. Problem:
+   +output/target/etc/timezone+ still has +America/Montreal+ content!
+
+Solution to this is either to identify which package to reconfigure, or

I believe this first 'or' should be removed so that the sentence would read:

The solution to this is to either identify which package to reconfigure, remove files manually, or destroy the target filesystem and recreate it.

docs/manual/clean-target-filesystem.txt line 37

Quoted 13 lines

docs/manual/clean-target-filesystem.txt new file +12 -0

+remove files manually, or destroy the target filesystem and recreate it.
+This latter option is convenient: you don't have to care about which
+packages to rebuild, and it is fast. Here is how to recreate your
+target filesystem, and get rid of filesystem _stains_:
+
+------------------
+make clean-target
+make
+------------------
+
++clean-target+ removes +output/target+, tells buildroot to recreate the
+target skeleton, and install each package again.

This is a rather simple solution to the issue of recreating a 'clean' target filesystem so it will be interesting to hear what the maintainers (Peter/Thomas P) opinions are on this patch.

Thanks,
-Ryan

Quoted 5 lines · Ryan Barnett

Emeric,

In the future - could you put a version - such as 'v2' - into the title utilizing the -'-subject-prefix' parameter of git-send-email?

I did, using git format-patch:

  $ git format-patch -M -n -s --subject-prefix="PATCH v2" -o outgoing
buildroot/master

And I see [PATCH v2 1/1] in the title. Did you miss something?

If you had some files added to the targetfs (fs-overlay, new packages, ...) and you no longer need them, buildroot does not offer a simple way to remove these items from the output/target directory. The rule added by this commit allows you to clean the targetfs. Issuing 'make' afterward will generate a new and clean targetfs.

A section in the documentation is also added. It describes few examples when developers could need it. 'make help' is also updated accordingly.

Signed-off-by Emeric Vigier <[email protected]>

---
Changes v1 -> v2
- remove non-existent .built and deprecated .stamp_installed references (suggested by Baruch Siach)
- mention the new target in 'make help'

---
 Makefile                                |  5 +++++
 docs/manual/advanced.txt                |  2 ++
 docs/manual/clean-target-filesystem.txt | 37 +++++++++++++++++++++++++++++++++
 3 files changed, 44 insertions(+)
 create mode 100644 docs/manual/clean-target-filesystem.txt

Makefile +5 -0

index af043a3..45ad2aa 100644
@@ -679,6 +679,10 @@ graph-depends: graph-depends-requirements
        |tee $(BASE_DIR)/graphs/$(@).dot \
        |dot $(BR2_GRAPH_DOT_OPTS) -T$(BR_GRAPH_OUT) -o $(BASE_DIR)/graphs/$(@).$(BR_GRAPH_OUT)
 
+clean-target:
+       rm -rf $(TARGET_DIR) $(STAGING_DIR) $(BUILD_DIR)/.root \
+               $(BUILD_DIR)/*/.stamp_target_installed $(BUILD_DIR)/*/.stamp_staging_installed
+
 else # ifeq ($(BR2_HAVE_DOT_CONFIG),y)
 
 all: menuconfig

@@ -849,6 +853,7 @@ help:
        @echo 'Cleaning:'
        @echo '  clean                  - delete all files created by build'
        @echo '  distclean              - delete all non-source files (including .config)'
+       @echo '  clean-target           - delete targetfs, issuing make again will recreate it'
        @echo
        @echo 'Build:'
        @echo '  all                    - make world'

docs/manual/advanced.txt +2 -0

index b7bfc49..1cf0427 100644
@@ -14,3 +14,5 @@ include::download-location.txt[]
 include::package-make-target.txt[]
 
 include::using-buildroot-development.txt[]
+
+include::clean-target-filesystem.txt[]

docs/manual/clean-target-filesystem.txt new file +37 -1

new file mode 100644
index 0000000..0274474
@@ -0,0 +1,37 @@
+// -*- mode:doc; -*-
+// vim: set syntax=asciidoc:
+
+==== Clean target filesystem
+
+There are various cases where you would like to clean the target
+filesystem. Especially once you understood that buildroot build system
+is kept simple, and does not allow to uninstall files. Let's take some
+examples:
+
+1. You enabled +PACKAGE_OPENSSH+ to have a SSH server running on your
+   board. But you figure out that +sshd+ does not run well on your
+   target's architecture. Thus you decide to disable +PACKAGE_OPENSSH+
+   and enable +PACKAGE_DROPBEAR+ instead. You run +make+, +openssh+
+   gets built and installed. Problem: +openssh+ files are still present
+   in +output/target+!
+
+2. You enabled +PACKAGE_NTPD+ and set the local timezone to
+   +America/Montreal+ with +BR2_TARGET_LOCALTIME+ config. Then you find
+   out that +Montreal+ local has been replaced by +Toronto+ (French
+   looks like a rogue language). You enter +make menuconfig+, make the
+   change, save, and run +make+ again. Problem:
+   +output/target/etc/timezone+ still has +America/Montreal+ content!
+
+Solution to this is either to identify which package to reconfigure, or
+remove files manually, or destroy the target filesystem and recreate it.
+This latter option is convenient: you don't have to care about which
+packages to rebuild, and it is fast. Here is how to recreate your
+target filesystem, and get rid of filesystem _stains_:
+
+------------------
+make clean-target
+make
+------------------
+
++clean-target+ removes +output/target+, tells buildroot to recreate the
+target skeleton, and install each package again.
--

1.9.1

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

+make clean-target

If you had some files added to the targetfs (fs-overlay, new packages, ...) and you no longer need them, buildroot does not offer a simple way to remove these items from the output/target directory. The rule added by this commit allows you to clean the targetfs. Issuing 'make' afterward will generate a new and clean targetfs.

A section in the documentation is also added. It describes few examples when developers could need it. 'make help' is also updated accordingly. And full-rebuild section of the documentation is updated with content and links.

Change-Id: I42b640e845bb4195fb7b160281b16ee40883f01f

Signed-off-by Emeric Vigier <[email protected]>

docs/manual/clean-target-filesystem.txt new file +29 -0

new file mode 100644
index 0000000..3e8181e
@@ -0,0 +1,39 @@
+// -*- mode:doc; -*-
+// vim: set syntax=asciidoc:
+
+[[clean-target-filesystem]]
+==== Clean target filesystem
+
+There are various cases where you would like to clean the target
+filesystem. Especially once you understood that buildroot does not
+have rules to uninstall files. Let's take some examples:
+
+1. You enabled +PACKAGE_OPENSSH+ to have a SSH server running on your
+   board. But you figure out that +sshd+ does not run well on your
+   target's architecture. Thus you decide to disable +PACKAGE_OPENSSH+
+   and enable +PACKAGE_DROPBEAR+ instead. You run +make+: +dropbear+
+   gets built and installed, fine. But +openssh+ files are still present
+   in +output/target+!
+
+2. You enabled +PACKAGE_NTPD+ and set the local timezone to
+   +America/Montreal+ with +BR2_TARGET_LOCALTIME+ config. Then you find
+   out that +Montreal+ local has been replaced by +Toronto+ (French
+   looks like a rogue language). You enter +make menuconfig+, make the
+   change, save, and run +make+ again. Problem:
+   +output/target/etc/timezone+ still has +America/Montreal+ content!
+
+Solution to this is either to identify which package to reconfigure, or
+remove files manually, or destroy the target filesystem and recreate it.
+This latter option is convenient: you don't have to care about which
+packages to rebuild, and it is quite fast. Here is how to recreate your
+target filesystem, and get rid of filesystem _stains_:

docs/manual/rebuilding-packages.txt +0 -0

index 6faa67a..82385ae 100644

Hi Emeric,

Quoted 2 lines

Change-Id: I42b640e845bb4195fb7b160281b16ee40883f01f

This Gerrit specific line is not useful for upstream Buildroot. Please remove.

Quoted 2 lines

Signed-off-by Emeric Vigier <[email protected]>

[snip]

Quoted 40 lines

docs/manual/clean-target-filesystem.txt new file +33 -0

new file mode 100644
index 0000000..3e8181e
@@ -0,0 +1,39 @@
+// -*- mode:doc; -*-
+// vim: set syntax=asciidoc:
+
+[[clean-target-filesystem]]
+==== Clean target filesystem
+
+There are various cases where you would like to clean the target
+filesystem. Especially once you understood that buildroot does not
+have rules to uninstall files. Let's take some examples:
+
+1. You enabled +PACKAGE_OPENSSH+ to have a SSH server running on your
+   board. But you figure out that +sshd+ does not run well on your
+   target's architecture. Thus you decide to disable +PACKAGE_OPENSSH+
+   and enable +PACKAGE_DROPBEAR+ instead. You run +make+: +dropbear+
+   gets built and installed, fine. But +openssh+ files are still present
+   in +output/target+!
+
+2. You enabled +PACKAGE_NTPD+ and set the local timezone to
+   +America/Montreal+ with +BR2_TARGET_LOCALTIME+ config. Then you find
+   out that +Montreal+ local has been replaced by +Toronto+ (French
+   looks like a rogue language). You enter +make menuconfig+, make the
+   change, save, and run +make+ again. Problem:
+   +output/target/etc/timezone+ still has +America/Montreal+ content!
+
+Solution to this is either to identify which package to reconfigure, or
+remove files manually, or destroy the target filesystem and recreate it.
+This latter option is convenient: you don't have to care about which
+packages to rebuild, and it is quite fast. Here is how to recreate your
+target filesystem, and get rid of filesystem _stains_:
+
+------------------
+make target-clean
+make

docs/manual/clean-target-filesystem.txt line 39

Quoted 6 lines

docs/manual/clean-target-filesystem.txt new file +6 -0

+------------------
+
++target-clean+ removes +output/target+. That tells buildroot to recreate
+the target skeleton, and install each package again. This is a
+convenient way to rebuild your target filesystem cleanly without
+rebuilding everything.

The warning is worth repeating here, I think.

Quoted 6 lines

docs/manual/rebuilding-packages.txt +0 -0

index 6faa67a..82385ae 100644

docs/manual/rebuilding-packages.txt can help you understand how to work with Buildroot: · line 44

Quoted 9 lines

docs/manual/rebuilding-packages.txt +3 -3

@@ -39,17 +39,22 @@ can help you understand how to work with Buildroot:
     you would like to have SSL support in +ctorrent+, so you enable the
     +openssl+ package in Buildroot configuration and restart the
     build. Buildroot will detect that +openssl+ should be built and
-   will be build it, but it will not detect that +ctorrent+ should be
-   rebuilt to benefit from +openssl+ to add OpenSSL support. You will
-   either have to do a full rebuild, or rebuild +ctorrent+ itself.
+   will build it, but it will not detect that +ctorrent+ should be
+   rebuilt to benefit from +openssl+ support. You will either have to
+   do a full rebuild, or rebuild +ctorrent+ itself.

This is an unrelated fix. Please submit in a separate patch.

baruch

If you had some files added to the targetfs (fs-overlay, new packages, ...) and you no longer need them, buildroot does not offer a simple way to remove these items from the output/target directory. The rule added by this commit allows you to clean the targetfs. Issuing 'make' afterward will generate a new and clean targetfs.

A section in the documentation is also added. It describes few examples when developers could need it. 'make help' is also updated accordingly. And full-rebuild section of the documentation is updated with content and links.

Am 27.04.2015 um 02:45 schrieb Emeric Vigier:

Quoted 11 lines

If you had some files added to the targetfs (fs-overlay, new packages, ...) and you no longer need them, buildroot does not offer a simple way to remove these items from the output/target directory. The rule added by this commit allows you to clean the targetfs. Issuing 'make' afterward will generate a new and clean targetfs.

A section in the documentation is also added. It describes few examples when developers could need it. 'make help' is also updated accordingly. And full-rebuild section of the documentation is updated with content and links.

Hi Emeric,

this is very helpful, in fact i have a similar patch running for quite some time where I remove target/ and images/ and the corresponding stamps. There are some problems though and I want to share my findings with you.
To begin with i was not 100% sure if the recreated target is the same as after a clean build. So I wrote a small check script which compares the original with the re-installed target trees. Here's the comparison i do:

   rsync -rvncl --delete --exclude '*.pyc' --exclude '*.pyo' --exclude

'tmp/ldconfig/aux-cache' output/target/ target.orig/

As you see it already has some files excluded that are always recreated differently.
In addition I need to delete .stamp_host_installed from host-gcc-final* to force reinstall of libstdc++ into target (using external linaro toolchain).

Another problem that showed up was that some of the package install steps dont seem separated very well. E.g. qt5 examples copies everything from a certain staging-dir/* to target/..

A later qt5 module also creates files in that staging-dir, so next round there's more files in target.

I realize this is due to my way of not deleteing the staging dir any longer - I used to in the beginning. If I recall correctly this is because I noticed some packages copy files into staging/ during compile and these files are then missing after a reinstall. I'm not entirely sure though.

So my question is, in your approach, are you certain the re-installed staging/ is the same as the original for all packages? Maybe you can run some comparison like my rsync line for staging as well.

regards,
Andreas

Am 27.04.2015 um 02:45 schrieb Emeric Vigier:

Quoted 11 lines

If you had some files added to the targetfs (fs-overlay, new packages, ...) and you no longer need them, buildroot does not offer a simple way to remove these items from the output/target directory. The rule added by this commit allows you to clean the targetfs. Issuing 'make' afterward will generate a new and clean targetfs.

A section in the documentation is also added. It describes few examples when developers could need it. 'make help' is also updated accordingly. And full-rebuild section of the documentation is updated with content and links.

Hi Emeric,

this is very helpful, in fact i have a similar patch running for quite some time where I remove target/ and images/ and the corresponding stamps. There are some problems though and I want to share my findings with you.
To begin with i was not 100% sure if the recreated target is the same as after a clean build. So I wrote a small check script which compares the original with the re-installed target trees. Here's the comparison i do:

   rsync -rvncl --delete --exclude '*.pyc' --exclude '*.pyo' --exclude

'tmp/ldconfig/aux-cache' output/target/ target.orig/

As you see it already has some files excluded that are always recreated differently.
In addition I need to delete .stamp_host_installed from host-gcc-final* to force reinstall of libstdc++ into target (using external linaro toolchain).

Another problem that showed up was that some of the package install steps dont seem separated very well. E.g. qt5 examples copies everything from a certain staging-dir/* to target/..
A later qt5 module also creates files in that staging-dir, so next round there's more files in target.

I realize this is due to my way of not deleteing the staging dir any longer - I used to in the beginning. If I recall correctly this is because I noticed some packages copy files into staging/ during compile and these files are then missing after a reinstall. I'm not entirely sure though.

So my question is, in your approach, are you certain the re-installed staging/ is the same as the original for all packages? Maybe you can run some comparison like my rsync line for staging as well.

regards,
Andreas

Quoted 11 lines

Another problem that showed up was that some of the package install steps dont seem separated very well. E.g. qt5 examples copies everything from a certain staging-dir/* to target/..
A later qt5 module also creates files in that staging-dir, so next round there's more files in target.

I realize this is due to my way of not deleteing the staging dir any longer - I used to in the beginning. If I recall correctly this is because I noticed some packages copy files into staging/ during compile and these files are then missing after a reinstall. I'm not entirely sure though.

I enabled qt4 and compare the targetfs after a target-clean, no differences:

$ rsync -rvncl --delete --exclude '*.pyc' --exclude '*.pyo' --exclude
'tmp/ldconfig/aux-cache' output/target/ output/target.orig.qt4/
sending incremental file list

sent 15,095 bytes received 62 bytes 30,314.00 bytes/sec total size is 16,028,992 speedup is 1,057.53 (DRY RUN)

With what QT configs did you meet problems specifically?

Quoted 3 lines

So my question is, in your approach, are you certain the re-installed staging/ is the same as the original for all packages? Maybe you can run some comparison like my rsync line for staging as well.

Quoted 2 lines

regards,
Andreas

thanks,
--
Emeric

Quoted 15 lines

Quoted 11 lines

Another problem that showed up was that some of the package install steps dont seem separated very well. E.g. qt5 examples copies everything from a certain staging-dir/* to target/..
A later qt5 module also creates files in that staging-dir, so next round there's more files in target.

I realize this is due to my way of not deleteing the staging dir any longer - I used to in the beginning. If I recall correctly this is because I noticed some packages copy files into staging/ during compile and these files are then missing after a reinstall. I'm not entirely sure though.

I enabled qt4 and compare the targetfs after a target-clean, no differences:

Quoted 8 lines

$ rsync -rvncl --delete --exclude '*.pyc' --exclude '*.pyo' --exclude
'tmp/ldconfig/aux-cache' output/target/ output/target.orig.qt4/
sending incremental file list

sent 15,095 bytes received 62 bytes 30,314.00 bytes/sec total size is 16,028,992 speedup is 1,057.53 (DRY RUN)

With what QT configs did you meet problems specifically?

Qt4 showed no difference in the targetfs.

I disabled Qt4 and enabled Qt5 with several modules and saw no differences in targetfs either. Here is my Qt5 config:

   http://pastie.org/10296840

I had to switch from a buildroot toolchain to a Linaro-gcc-4.9 toolchain to build Qt5. I haven't enabled Qt5-webkit because I fell into this GCC-4.9 bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61207.

Quoted 4 lines

Quoted 4 lines

So my question is, in your approach, are you certain the re-installed staging/ is the same as the original for all packages? Maybe you can run some comparison like my rsync line for staging as well.

I only delete staging symlink, not the sysroot content. This patch does

not aim at fixing the black magic that some packages may be doing between the staging dir and the target dir, just recreating the targetfs.

So the only improvement I have in mind for this patch is this:

         rm -rf $(TARGET_DIR) $(STAGING_DIR) $(BUILD_DIR)/.root \

- $(BUILD_DIR)/*/.stamp_target_installed
$(BUILD_DIR)/*/.stamp_staging_installed
+ $(BUILD_DIR)/*/.stamp_target_installed
$(BUILD_DIR)/*/.stamp_staging_installed \
+ $(BUILD_DIR)/host-gcc-final-*/.stamp_host_installed

Do you have other things in mind?

Quoted 7 lines

Quoted 3 lines

regards,
Andreas

thanks,
--
Emeric

Emeric

If you had some files added to the targetfs (fs-overlay, new packages, ...) and you no longer need them, buildroot does not offer a simple way to remove these items from the output/target directory. The rule added by this commit allows you to clean the targetfs. Issuing 'make' afterward will generate a new and clean targetfs.

A section in the documentation is also added. It describes few examples when developers could need it. 'make help' is also updated accordingly. And full-rebuild section of the documentation is updated with content and links.

Change-Id I42b640e845bb4195fb7b160281b16ee40883f01f

Signed-off-by Emeric Vigier <[email protected]>