Skip to contentSkip to conversation

Misc accumulated changes

PATCH New Yann E. MORIN 6 participants 22 messages Jan 7, 2014 – Jan 28, 2014 Kbuild

lore
Share & tools
lore
#0 lore raw

cover Misc accumulated changes

From: "Yann E. MORIN" <[email protected]>

Hello All!

Here is a small series with a few un-assorted fixes for the toolchain wrapper and for graph-depends, a cleanup in the system sub-menu for /etc/{hostname,issue}, a new package, and a blurb in the manual about dealing efficiently with very large files.

Regards,
Yann E. MORIN.

----------------------------------------------------------------
Sagaert Johan (1):
system: default to empty issue/hostname for custom skeleton

Yann E. MORIN (4):
Makefile: support running graph-depends from out-of-tree toolchain/external: fix wrapper by not passing conflicting flags package/tzdata: install a default localtime
manual: add section about dealing efficiently with big image files

 Makefile                                           |  3 +-
 docs/manual/common-usage.txt                       | 39 ++++++++++++++++++++
 package/pkg-generic.mk                             |  3 +-
 package/tzdata/Config.in                           | 10 ++++++
 package/tzdata/tzdata.mk                           |  8 ++++-
 system/Config.in                                   | 32 ++++++++++-------
 .../toolchain-external/ext-toolchain-wrapper.c     | 41 ++++++++++++++++------
 7 files changed, 111 insertions(+), 25 deletions(-)
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#1 lore raw

PATCH 1/5 Makefile: support running graph-depends from out-of-tree

From: "Yann E. MORIN" <[email protected]>

---
 Makefile               | 3 ++-
 package/pkg-generic.mk | 3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)
Makefile +2 -1
index 4320e7b..dd21133 100644
@@ -647,7 +647,8 @@ graph-build: $(O)/build/build-time.log
 
 graph-depends:
 	@$(INSTALL) -d $(O)/graphs
-	@./support/scripts/graph-depends \
+	@cd "$(CONFIG_DIR)"; \
+	$(TOPDIR)/support/scripts/graph-depends \
 	|dot -T$(BR2_GRAPH_OUT) -o $(O)/graphs/$(@).$(BR2_GRAPH_OUT)
 
 else # ifeq ($(BR2_HAVE_DOT_CONFIG),y)
package/pkg-generic.mk +2 -1
index b519d69..1b99c2a 100644
@@ -470,7 +470,8 @@ $(1)-show-depends:
 
 $(1)-graph-depends:
 			@$(INSTALL) -d $(O)/graphs
-			@./support/scripts/graph-depends $(1) \
+			@cd "$(CONFIG_DIR)"; \
+			$(TOPDIR)/support/scripts/graph-depends $(1) \
 			|dot -T$(BR2_GRAPH_OUT) -o $(O)/graphs/$$(@).$(BR2_GRAPH_OUT)
 
 $(1)-dirclean:		$$($(2)_TARGET_DIRCLEAN)
Signature · 1 line
1.8.1.2
#3 lore raw

PATCH 2/5 toolchain/external: fix wrapper by not passing conflicting flags

In reply to Yann E. MORIN

From: "Yann E. MORIN" <[email protected]>

In our wrapper, we forcibly add the -march=, -mcpu= and-mtune= flags to the actual copmpiler, this in an attempt to always generate correct and optimised code for the target.

But in some cases, the caller knows better than we do, and passes its own set, or subset of those flags. In this case, some may conflict with the ones we pass. The most prominent offender being the Linux kernel.

For example, on the ARM Rapsberry Pi, the Linux kernel will set the -march=armv6 flag and no -mcpu= flag, but we pass -mcpu=arm1176jzf-s, which conflicts:

drivers/scsi/scsi_trace.c:1:0: warning: switch -mcpu=arm1176jzf-s conflicts with -march=armv6 switch

(and so for all the files the kernel compiles, pretty messy) (note: arm1176jzf-s is not an armv6, it is an armv6zk. Yeah...)

To avoid this situation, we scan our commandline for any occurence of the possibly conflicting flags. If none is found, then we add our owns. If any is found, then we don't add any of our owns.

The idea behind this is that we trust the caller to know better than we do what it is doing. Since the biggest, and sole so far, offender is the Linux kernel, then this is a rather safe bet.

Cc Thomas Petazzoni <[email protected]>
Cc Peter Korsgaard <[email protected]>
Cc Arnout Vandecappelle <[email protected]>
Cc Maxime Hadjinlian <[email protected]>
---
 .../toolchain-external/ext-toolchain-wrapper.c     | 41 ++++++++++++++++------
 1 file changed, 31 insertions(+), 10 deletions(-)
toolchain/toolchain-external/ext-toolchain-wrapper.c +31 -10
index d54f1f5..81c6ed1 100644
@@ -30,21 +30,17 @@ static char sysroot[PATH_MAX];
  * that we only pass the predefined one to the real compiler if the inverse
  * option isn't in the argument list.
  * This specifies the worst case number of extra arguments we might pass
+ * Currently, we have:
+ * 	-mfloat-abi=
+ * 	-march=
+ * 	-mtune=
+ * 	-mcpu=
  */
-#define EXCLUSIVE_ARGS	1
+#define EXCLUSIVE_ARGS	4
 
 static char *predef_args[] = {
 	path,
 	"--sysroot", sysroot,
-#ifdef BR_ARCH
-	"-march=" BR_ARCH,
-#endif /* BR_ARCH */
-#ifdef BR_TUNE
-	"-mtune=" BR_TUNE,
-#endif /* BR_TUNE */
-#ifdef BR_CPU
-	"-mcpu=" BR_CPU,
-#endif
 #ifdef BR_ABI
 	"-mabi=" BR_ABI,
 #endif
@@ -157,6 +153,31 @@ int main(int argc, char **argv)
 		*cur++ = "-mfloat-abi=" BR_FLOAT_ABI;
 #endif
 
+#if defined(BR_ARCH) || \
+    defined(BR_TUNE) || \
+    defined(BR_CPU)
+	/* Add our -march/cpu/tune/abi flags, but only if none are
+	 * already specified on the commandline
+	 */
+	for (i = 1; i < argc; i++) {
+		if (!strncmp(argv[i], "-march=", strlen("-march=")) ||
+		    !strncmp(argv[i], "-mtune=", strlen("-mtune=")) ||
+		    !strncmp(argv[i], "-mcpu=",  strlen("-mcpu=" )))
+			break;
+	}
+	if (i == argc) {
+#ifdef BR_ARCH
+		*cur++ = "-march=" BR_ARCH;
+#endif
+#ifdef BR_TUNE
+		*cur++ = "-mtune=" BR_TUNE;
+#endif
+#ifdef BR_CPU
+		*cur++ = "-mcpu=" BR_CPU;
+#endif
+	}
+#endif /* ARCH || TUNE || CPU */
+
 	/* append forward args */
 	memcpy(cur, &argv[1], sizeof(char *) * (argc - 1));
 	cur += argc - 1;
Signature · 1 line
1.8.1.2
#4 lore raw

PATCH 2/5 toolchain/external: fix wrapper by not passing conflicting flags

Hi Yann, all

Quoted 102 lines · Yann E. MORIN

From: "Yann E. MORIN" <[email protected]>

In our wrapper, we forcibly add the -march=, -mcpu= and-mtune= flags to the actual copmpiler, this in an attempt to always generate correct and optimised code for the target.

But in some cases, the caller knows better than we do, and passes its own set, or subset of those flags. In this case, some may conflict with the ones we pass. The most prominent offender being the Linux kernel.

For example, on the ARM Rapsberry Pi, the Linux kernel will set the -march=armv6 flag and no -mcpu= flag, but we pass -mcpu=arm1176jzf-s, which conflicts:

drivers/scsi/scsi_trace.c:1:0: warning: switch -mcpu=arm1176jzf-s conflicts with -march=armv6 switch

(and so for all the files the kernel compiles, pretty messy) (note: arm1176jzf-s is not an armv6, it is an armv6zk. Yeah...)

To avoid this situation, we scan our commandline for any occurence of the possibly conflicting flags. If none is found, then we add our owns. If any is found, then we don't add any of our owns.

The idea behind this is that we trust the caller to know better than we do what it is doing. Since the biggest, and sole so far, offender is the Linux kernel, then this is a rather safe bet.

Cc Thomas Petazzoni <[email protected]>
Cc Peter Korsgaard <[email protected]>
Cc Arnout Vandecappelle <[email protected]>
Cc Maxime Hadjinlian <[email protected]>
---
 .../toolchain-external/ext-toolchain-wrapper.c     | 41 ++++++++++++++++------
 1 file changed, 31 insertions(+), 10 deletions(-)
toolchain/toolchain-external/ext-toolchain-wrapper.c +31 -11
index d54f1f5..81c6ed1 100644
@@ -30,21 +30,17 @@ static char sysroot[PATH_MAX];
  * that we only pass the predefined one to the real compiler if the inverse
  * option isn't in the argument list.
  * This specifies the worst case number of extra arguments we might pass
+ * Currently, we have:
+ *     -mfloat-abi=
+ *     -march=
+ *     -mtune=
+ *     -mcpu=
  */
-#define EXCLUSIVE_ARGS 1
+#define EXCLUSIVE_ARGS 4
 
 static char *predef_args[] = {
        path,
        "--sysroot", sysroot,
-#ifdef BR_ARCH
-       "-march=" BR_ARCH,
-#endif /* BR_ARCH */
-#ifdef BR_TUNE
-       "-mtune=" BR_TUNE,
-#endif /* BR_TUNE */
-#ifdef BR_CPU
-       "-mcpu=" BR_CPU,
-#endif
 #ifdef BR_ABI
        "-mabi=" BR_ABI,
 #endif
@@ -157,6 +153,31 @@ int main(int argc, char **argv)
                *cur++ = "-mfloat-abi=" BR_FLOAT_ABI;
 #endif
 
+#if defined(BR_ARCH) || \
+    defined(BR_TUNE) || \
+    defined(BR_CPU)
+       /* Add our -march/cpu/tune/abi flags, but only if none are
+        * already specified on the commandline
+        */
+       for (i = 1; i < argc; i++) {
+               if (!strncmp(argv[i], "-march=", strlen("-march=")) ||
+                   !strncmp(argv[i], "-mtune=", strlen("-mtune=")) ||
+                   !strncmp(argv[i], "-mcpu=",  strlen("-mcpu=" )))
+                       break;
+       }
+       if (i == argc) {
+#ifdef BR_ARCH
+               *cur++ = "-march=" BR_ARCH;
+#endif
+#ifdef BR_TUNE
+               *cur++ = "-mtune=" BR_TUNE;
+#endif
+#ifdef BR_CPU
+               *cur++ = "-mcpu=" BR_CPU;
+#endif
+       }
+#endif /* ARCH || TUNE || CPU */
+
        /* append forward args */
        memcpy(cur, &argv[1], sizeof(char *) * (argc - 1));
        cur += argc - 1;
--
Quoted 2 lines

1.8.1.2

Seems good to me.
Shouldn't we change the different defconfig that exists ? At least the RaspberryPi so it would sets an example (and maybe add something about this in the manual ?)

#5 lore raw

PATCH 2/5 toolchain/external: fix wrapper by not passing conflicting flags

Dear Maxime Hadjinlian,

Quoted 3 lines · Maxime Hadjinlian

Shouldn't we change the different defconfig that exists ? At least the RaspberryPi so it would sets an example (and maybe add something about this in the manual ?)

Why would the defconfig need some changes following this patch?

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#6 lore raw

PATCH 2/5 toolchain/external: fix wrapper by not passing conflicting flags

Quoted 1 line · Thomas Petazzoni

Dear Maxime Hadjinlian,

Hi Thomas,

Quoted 8 lines · Maxime Hadjinlian
Quoted 4 lines

Shouldn't we change the different defconfig that exists ? At least the RaspberryPi so it would sets an example (and maybe add something about this in the manual ?)

Why would the defconfig need some changes following this patch?

To set a visible example of settings the correct -march/mcpu/mtune options ? Also, this would eliminate all the warnings that Yann saw while building a kernel for the rpi, so it seems like a good candidate.

Quoted 6 lines

Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

#7 lore raw

PATCH 2/5 toolchain/external: fix wrapper by not passing conflicting flags

Maxime, All,

On 2014-01-08 16:57 +0100, Maxime Hadjinlian spake thusly:

Quoted 7 lines
Quoted 1 line · Thomas Petazzoni

Dear Maxime Hadjinlian,

Hi Thomas,

Quoted 3 lines · Maxime Hadjinlian
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

Shouldn't we change the different defconfig that exists ? At least the RaspberryPi so it would sets an example (and maybe add something about this in the manual ?)

Why would the defconfig need some changes following this patch?

To set a visible example of settings the correct -march/mcpu/mtune options ? Also, this would eliminate all the warnings that Yann saw while building a kernel for the rpi, so it seems like a good candidate.

There is nothing in that patch that impacts the configuration. There is no reason to change any defconfig to 'activate' this change.

Rather, the config options BR2_GCC_TARGET_CPU, BR2_GCC_TARGET_ARCH, and BR2_GCC_TARGET_MTUNE, if any, are pased at the time we build the wrapper itself.

Those options are set automatically when you choose, in the menuconfig, the type of CPU your target has. And since that will automatically set the BR2_GCC_TARGET_{ARCH,CPU,MTUNE} options, they are forcibly set.

And since the raspberrypi_defconfig (as you use that as an example) already sets BR2_arm1176jzf_s=y, this will set BR2_GCC_TARGET_ARCH and BR2_GCC_TARGET_CPU.

And this is exactly this situation this patch fixes: our options forcing the compiler flags, when some are already provided on the command line.

So, there is no need (and no reason) to change any of our defconfigs to take advantage of this change.

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#8 lore raw

PATCH 2/5 toolchain/external: fix wrapper by not passing conflicting flags

Hi all,

Quoted 34 lines · Yann E. MORIN

Maxime, All,

On 2014-01-08 16:57 +0100, Maxime Hadjinlian spake thusly:

Quoted 7 lines
Quoted 1 line · Thomas Petazzoni

Dear Maxime Hadjinlian,

Hi Thomas,

Quoted 3 lines · Maxime Hadjinlian
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

Shouldn't we change the different defconfig that exists ? At least the RaspberryPi so it would sets an example (and maybe add something about this in the manual ?)

Why would the defconfig need some changes following this patch?

To set a visible example of settings the correct -march/mcpu/mtune options ? Also, this would eliminate all the warnings that Yann saw while building a kernel for the rpi, so it seems like a good candidate.

There is nothing in that patch that impacts the configuration. There is no reason to change any defconfig to 'activate' this change.

Rather, the config options BR2_GCC_TARGET_CPU, BR2_GCC_TARGET_ARCH, and BR2_GCC_TARGET_MTUNE, if any, are pased at the time we build the wrapper itself.

Those options are set automatically when you choose, in the menuconfig, the type of CPU your target has. And since that will automatically set the BR2_GCC_TARGET_{ARCH,CPU,MTUNE} options, they are forcibly set.

And since the raspberrypi_defconfig (as you use that as an example) already sets BR2_arm1176jzf_s=y, this will set BR2_GCC_TARGET_ARCH and BR2_GCC_TARGET_CPU.

Quoted 8 lines

And this is exactly this situation this patch fixes: our options forcing the compiler flags, when some are already provided on the command line.

So, there is no need (and no reason) to change any of our defconfigs to take advantage of this change.

Regards,
Yann E. MORIN.

My bad,
I spoke too quickly and I misunderstood some information.

Thanks for your clarifications.

Quoted 8 lines

--
.-----------------.--------------------.------------------.--------------------.

|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |

'------------------------------^-------^------------------^--------------------'

#9 lore raw

PATCH 2/5 toolchain/external: fix wrapper by not passing conflicting flags

In reply to Yann E. MORIN
Quoted 29 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

From: "Yann E. MORIN" <[email protected]>
In our wrapper, we forcibly add the -march=, -mcpu= and-mtune= flags to the actual copmpiler, this in an attempt to always generate correct and optimised code for the target.

But in some cases, the caller knows better than we do, and passes its own set, or subset of those flags. In this case, some may conflict with the ones we pass. The most prominent offender being the Linux kernel.

For example, on the ARM Rapsberry Pi, the Linux kernel will set the -march=armv6 flag and no -mcpu= flag, but we pass -mcpu=arm1176jzf-s, which conflicts:

drivers/scsi/scsi_trace.c:1:0: warning: switch -mcpu=arm1176jzf-s conflicts with -march=armv6 switch

(and so for all the files the kernel compiles, pretty messy) (note: arm1176jzf-s is not an armv6, it is an armv6zk. Yeah...)

To avoid this situation, we scan our commandline for any occurence of the possibly conflicting flags. If none is found, then we add our owns. If any is found, then we don't add any of our owns.

The idea behind this is that we trust the caller to know better than we do what it is doing. Since the biggest, and sole so far, offender is the Linux kernel, then this is a rather safe bet.

Quoted 5 lines
Cc Thomas Petazzoni <[email protected]>
Cc Peter Korsgaard <[email protected]>
Cc Arnout Vandecappelle <[email protected]>
Cc Maxime Hadjinlian <[email protected]>

Committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#10 lore raw

PATCH 3/5 package/tzdata: install a default localtime

In reply to Yann E. MORIN

From: "Yann E. MORIN" <[email protected]>

Add an option to install a default localtime.

since tzdata only makes sense for (e)glibc, we don't care that we install timezone files that uClibc can't understand.

---
 package/tzdata/Config.in | 10 ++++++++++
 package/tzdata/tzdata.mk |  8 +++++++-
 2 files changed, 17 insertions(+), 1 deletion(-)
package/tzdata/Config.in +10 -0
index 1be6814..c0ede1f 100644
@@ -23,4 +23,14 @@ config BR2_PACKAGE_TZDATA_ZONELIST
 	  The full list is the list of files in the time zone database source,
 	  not including the build and .tab files.
 
+config BR2_PACKAGE_TZDATA_LOCALTIME
+	string "default local time"
+	help
+	  The time zone to install as the default local time, expressed as a
+	  tzdata location, such as:
+	    GMT, Europe/Paris, America/New_York, Pacific/Wallis...
+
+	  If empty, no local time will be set, and the dates will be
+	  expressed in UTC.
+
 endif
package/tzdata/tzdata.mk +7 -1
index 1cb5b3f..74263e1 100644
@@ -39,8 +39,14 @@ define TZDATA_INSTALL_TARGET_CMDS
 	cp -a $(@D)/_output/* $(TARGET_DIR)/usr/share/zoneinfo
 	cd $(TARGET_DIR)/usr/share/zoneinfo;    \
 	for zone in posix/*; do                 \
-	    ln -sfn "$${zone}" "$${zone##*/}";    \
+	    ln -sfn "$${zone}" "$${zone##*/}";  \
 	done
+	if [ -n "$(BR2_PACKAGE_TZDATA_LOCALTIME)" ]; then                   \
+	    cd $(TARGET_DIR)/etc;                                           \
+	    ln -sf ../usr/share/zoneinfo/$(BR2_PACKAGE_TZDATA_LOCALTIME)    \
+	           localtime;                                               \
+	    echo "$(BR2_PACKAGE_TZDATA_LOCALTIME)" >timezone;               \
+	fi
 endef
 
 $(eval $(generic-package))
Signature · 1 line
1.8.1.2
#11 lore raw

PATCH 3/5 package/tzdata: install a default localtime

Quoted 18 lines · Yann E. MORIN

From: "Yann E. MORIN" <[email protected]>

Add an option to install a default localtime.

since tzdata only makes sense for (e)glibc, we don't care that we install timezone files that uClibc can't understand.

---
  package/tzdata/Config.in | 10 ++++++++++
  package/tzdata/tzdata.mk |  8 +++++++-
  2 files changed, 17 insertions(+), 1 deletion(-)
package/tzdata/Config.in +0 -0
index 1be6814..c0ede1f 100644

package/tzdata/Config.in config BR2_PACKAGE_TZDATA_ZONELIST · line 31

Quoted 9 lines
package/tzdata/Config.in +6 -0
@@ -23,4 +23,14 @@ config BR2_PACKAGE_TZDATA_ZONELIST
  	  The full list is the list of files in the time zone database source,
  	  not including the build and .tab files.
 
+config BR2_PACKAGE_TZDATA_LOCALTIME
+	string "default local time"
+	help
+	  The time zone to install as the default local time, expressed as a
+	  tzdata location, such as:
+	    GMT, Europe/Paris, America/New_York, Pacific/Wallis...

I hope this will not confuse users to think that they can specify
multiple values separated by a comma...

Quoted 10 lines
package/tzdata/Config.in +4 -0
+
+	  If empty, no local time will be set, and the dates will be
+	  expressed in UTC.
+
  endif
package/tzdata/tzdata.mk +0 -0
index 1cb5b3f..74263e1 100644

package/tzdata/tzdata.mk define TZDATA_INSTALL_TARGET_CMDS · line 44

Quoted 7 lines
package/tzdata/tzdata.mk +2 -1
@@ -39,8 +39,14 @@ define TZDATA_INSTALL_TARGET_CMDS
  	cp -a $(@D)/_output/* $(TARGET_DIR)/usr/share/zoneinfo
  	cd $(TARGET_DIR)/usr/share/zoneinfo;    \
  	for zone in posix/*; do                 \
-	    ln -sfn "$${zone}" "$${zone##*/}";    \
+	    ln -sfn "$${zone}" "$${zone##*/}";  \
  	done
+	if [ -n "$(BR2_PACKAGE_TZDATA_LOCALTIME)" ]; then                   \

Ick, this looks ugly... If it is unset, make will evaluate it to

	if [ -n """" ]; then ...

which is indeed an empty string, but not so nice... IOW, it's missing a qstrip.

Also, instead of a shell condition I'd prefer a make condition, but
that's probably a matter of taste.

package/tzdata/tzdata.mk define TZDATA_INSTALL_TARGET_CMDS · line 46

Quoted 2 lines
package/tzdata/tzdata.mk +2 -0
+	    cd $(TARGET_DIR)/etc;                                           \
+	    ln -sf ../usr/share/zoneinfo/$(BR2_PACKAGE_TZDATA_LOCALTIME)    \

Same here. But also quotes should be added in case the user put
something stupid there.

package/tzdata/tzdata.mk define TZDATA_INSTALL_TARGET_CMDS · line 48

Quoted 2 lines
package/tzdata/tzdata.mk +2 -0
+	           localtime;                                               \
+	    echo "$(BR2_PACKAGE_TZDATA_LOCALTIME)" >timezone;               \

Same here. The quotes are not actually quoting anything now...

Regards,
Arnout

Quoted 5 lines
package/tzdata/tzdata.mk +1 -0
+	fi
  endef
 
  $(eval $(generic-package))
Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
#12 lore raw

PATCH 3/5 package/tzdata: install a default localtime

Arnout, All,

On 2014-01-09 18:13 +0100, Arnout Vandecappelle spake thusly:

Quoted 3 lines · Yann E. MORIN
Quoted 2 lines

From: "Yann E. MORIN" <[email protected]>
Add an option to install a default localtime.

[--SNIP--]

Quoted 9 lines

package/tzdata/Config.in config BR2_PACKAGE_TZDATA_ZONELIST · line 31

Quoted 6 lines
package/tzdata/Config.in +6 -0
+config BR2_PACKAGE_TZDATA_LOCALTIME
+	string "default local time"
+	help
+	  The time zone to install as the default local time, expressed as a
+	  tzdata location, such as:
+	    GMT, Europe/Paris, America/New_York, Pacific/Wallis...

I hope this will not confuse users to think that they can specify multiple
values separated by a comma...

Now you point me to it, indeed, that may sound confusing.

OTOH, the sentence always use the singular 'a timezone", or "the timezone", so hopefully this is not so confusing... After all, it does not make sense to say that the current timezone is a multi-timezone, doesn't it?

Quoted 15 lines

package/tzdata/tzdata.mk define TZDATA_INSTALL_TARGET_CMDS · line 44

Quoted 12 lines
package/tzdata/tzdata.mk +2 -1
index 1cb5b3f..74263e1 100644
@@ -39,8 +39,14 @@ define TZDATA_INSTALL_TARGET_CMDS
 	cp -a $(@D)/_output/* $(TARGET_DIR)/usr/share/zoneinfo
 	cd $(TARGET_DIR)/usr/share/zoneinfo;    \
 	for zone in posix/*; do                 \
-	    ln -sfn "$${zone}" "$${zone##*/}";    \
+	    ln -sfn "$${zone}" "$${zone##*/}";  \
 	done
+	if [ -n "$(BR2_PACKAGE_TZDATA_LOCALTIME)" ]; then                   \

Ick, this looks ugly... If it is unset, make will evaluate it to

Quoted 4 lines
	if [ -n """" ]; then ...

which is indeed an empty string, but not so nice... IOW, it's missing a qstrip.

Doh, I always forget about this one. Sigh... Thanks for pointing this out.

Quoted 2 lines

Also, instead of a shell condition I'd prefer a make condition, but that's
probably a matter of taste.

I also pundered this, but it looke really ugly to me. Besides, we can't have ifeq-blocks in a define-block, that does not work (IIRC).

Quoted 5 lines

package/tzdata/tzdata.mk define TZDATA_INSTALL_TARGET_CMDS · line 46

Quoted 2 lines
package/tzdata/tzdata.mk +2 -0
+	    cd $(TARGET_DIR)/etc;                                           \
+	    ln -sf ../usr/share/zoneinfo/$(BR2_PACKAGE_TZDATA_LOCALTIME)    \

Same here. But also quotes should be added in case the user put something
stupid there.

Ditto, I just forget about qtrip everytime.

Quoted 4 lines

package/tzdata/tzdata.mk define TZDATA_INSTALL_TARGET_CMDS · line 48

Quoted 2 lines
package/tzdata/tzdata.mk +2 -0
+	           localtime;                                               \
+	    echo "$(BR2_PACKAGE_TZDATA_LOCALTIME)" >timezone;               \

Same here. The quotes are not actually quoting anything now...

Ditto.

I'll send a fix-patch soon.
Thank you!

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#13 lore raw

PATCH 3/5 package/tzdata: install a default localtime

In reply to Yann E. MORIN
Quoted 9 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

From: "Yann E. MORIN" <[email protected]>
Add an option to install a default localtime.

since tzdata only makes sense for (e)glibc, we don't care that we install timezone files that uClibc can't understand.

Committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#14 lore raw

PATCH 3/5 package/tzdata: install a default localtime

Dear Peter Korsgaard,

Quoted 11 lines · Peter Korsgaard
Quoted 9 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

From: "Yann E. MORIN" <[email protected]>
Add an option to install a default localtime.

since tzdata only makes sense for (e)glibc, we don't care that we install timezone files that uClibc can't understand.

Committed, thanks.

I thought I said somewhere I was not happy with this patch, because it creates a different solution to set the timezone for eglibc and uClibc. We should instead strive to have an option under "System configuration" to define the timezone, and ensure it does the right thing for both uClibc and eglibc.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#15 lore raw

PATCH 4/5 system: default to empty issue/hostname for custom skeleton

In reply to Yann E. MORIN

From: Sagaert Johan <[email protected]>

A custom skeleton may include /etc/issue and/or /etc/hostname, in which case we do not want to override them.

But those files could also be missing from the custom skeleton, or the user may want to override their content.

Make the current default value for isue and hostname depend on the generic skeleton, and default to empty for a custom skeleton.

[yann.morin.1998 at free.fr: change from hiding it behind SKELETON_DEFAULT
to use conditional default values; move below the skeleton choice; improve help entries]

---
 system/Config.in | 32 ++++++++++++++++++++------------
 1 file changed, 20 insertions(+), 12 deletions(-)
system/Config.in +20 -12
index 54c196c..4bdd610 100644
@@ -1,17 +1,5 @@
 menu "System configuration"
 
-config BR2_TARGET_GENERIC_HOSTNAME
-	string "System hostname"
-	default "buildroot"
-	help
-	  Select system hostname to be stored in /etc/hostname.
-
-config BR2_TARGET_GENERIC_ISSUE
-	string "System banner"
-	default "Welcome to Buildroot"
-	help
-	  Select system banner (/etc/issue) to be displayed at login.
-
 choice
 	bool "Passwords encoding"
 	default BR2_TARGET_GENERIC_PASSWD_MD5
@@ -179,6 +167,26 @@ config BR2_ROOTFS_SKELETON_CUSTOM_PATH
 	  Path custom target skeleton.
 endif
 
+config BR2_TARGET_GENERIC_HOSTNAME
+	string "System hostname"
+	default "buildroot" if BR2_ROOTFS_SKELETON_DEFAULT
+	default ""          if ! BR2_ROOTFS_SKELETON_DEFAULT
+	help
+	  Select system hostname to be stored in /etc/hostname.
+
+	  Leave empty to not create /etc/hostname, or to keep the
+	  one from a custom skeleton.
+
+config BR2_TARGET_GENERIC_ISSUE
+	string "System banner"
+	default "Welcome to Buildroot" if BR2_ROOTFS_SKELETON_DEFAULT
+	default ""                     if ! BR2_ROOTFS_SKELETON_DEFAULT
+	help
+	  Select system banner (/etc/issue) to be displayed at login.
+
+	  Leave empty to not create /etc/issue, or to keep the
+	  one from a custom skeleton.
+
 if BR2_ROOTFS_SKELETON_DEFAULT
 
 config BR2_TARGET_GENERIC_ROOT_PASSWD
Signature · 1 line
1.8.1.2
#16 lore raw

PATCH 4/5 system: default to empty issue/hostname for custom skeleton

Quoted 9 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

From: Sagaert Johan <[email protected]>
A custom skeleton may include /etc/issue and/or /etc/hostname, in which case we do not want to override them.

But those files could also be missing from the custom skeleton, or the user may want to override their content.

Quoted 8 lines

Make the current default value for isue and hostname depend on the generic skeleton, and default to empty for a custom skeleton.

[yann.morin.1998 at free.fr: change from hiding it behind SKELETON_DEFAULT
to use conditional default values; move below the skeleton choice; improve help entries]

Do we really need to add this extra complexity? A custom skeleton is not recommended (and in any case an advanced option), and the fix for people not wanting /etc/issues / hostname overwritten is very easy, they can just set it to the empty string.

What do others say?

Signature · 1 line
Bye, Peter Korsgaard
#17 lore raw

PATCH 4/5 system: default to empty issue/hostname for custom skeleton

Peter, All,

On 2014-01-09 21:32 +0100, Peter Korsgaard spake thusly:

Quoted 18 lines
Quoted 9 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

From: Sagaert Johan <[email protected]>
A custom skeleton may include /etc/issue and/or /etc/hostname, in which case we do not want to override them.

But those files could also be missing from the custom skeleton, or the user may want to override their content.

Quoted 8 lines

Make the current default value for isue and hostname depend on the generic skeleton, and default to empty for a custom skeleton.

[yann.morin.1998 at free.fr: change from hiding it behind SKELETON_DEFAULT
to use conditional default values; move below the skeleton choice; improve help entries]

Quoted 6 lines

Do we really need to add this extra complexity? A custom skeleton is not recommended (and in any case an advanced option), and the fix for people not wanting /etc/issues / hostname overwritten is very easy, they can just set it to the empty string.

What do others say?

Note that this is part of the patchwork-cleanup effort #5 by Thomas DS:

    http://lists.busybox.net/pipermail/buildroot/2014-January/086071.html

The original patch was:

    http://patchwork.ozlabs.org/patch/211644/

Which is now superseded by this very patch we're discussing.

I don't much care about it either. Sagaert Johan, maybe you could expose your arguments in favour of this change?

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#18 lore raw

PATCH 4/5 system: default to empty issue/hostname for custom skeleton

Hi All;

I made this patch because I found it made more sense if I selected a custom skeleton
that in that case buildroot should not insert its own hostname, issue,TZ etc into the target root directory. If I can achieve the same result by leaving the hostname etc... empty in buildroot menu then this is fine. I just wanted buildroot not to overwrite anything when I selected a custom skeleton.

Best Regards, Johan

-----Oorspronkelijk bericht-----
Van: Yann E. MORIN [mailto:yann.morin.1998 at gmail.com] Namens Yann E. MORIN
Verzonden: donderdag 9 januari 2014 21:45
Aan: Peter Korsgaard
CC: buildroot at busybox.net; Sagaert Johan
Onderwerp: Re: [PATCH 4/5] system: default to empty issue/hostname for custom skeleton

Peter, All,

On 2014-01-09 21:32 +0100, Peter Korsgaard spake thusly:

Quoted 18 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

From: Sagaert Johan <[email protected]> > A custom skeleton

may include /etc/issue and/or /etc/hostname, in > which case we do not want to override them.

Quoted 7 lines

But those files could also be missing from the custom skeleton, or the user may want to override their content.

Make the current default value for isue and hostname depend on the generic skeleton, and default to empty for a custom skeleton.

[yann.morin.1998 at free.fr: change from hiding it behind SKELETON_DEFAULT

Quoted 3 lines

to use conditional default values; move below the skeleton choice; improve help entries]

Quoted 6 lines

Do we really need to add this extra complexity? A custom skeleton is not recommended (and in any case an advanced option), and the fix for people not wanting /etc/issues / hostname overwritten is very easy, they can just set it to the empty string.

What do others say?

Note that this is part of the patchwork-cleanup effort #5 by Thomas DS:

    http://lists.busybox.net/pipermail/buildroot/2014-January/086071.html

The original patch was:

    http://patchwork.ozlabs.org/patch/211644/

Which is now superseded by this very patch we're discussing.

I don't much care about it either. Sagaert Johan, maybe you could expose your arguments in favour of this change?

Regards,
Yann E. MORIN.

--
.-----------------.--------------------.------------------.--------------------.

|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' 
| conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |

'------------------------------^-------^------------------^--------------------'

#19 lore raw

PATCH 4/5 system: default to empty issue/hostname for custom skeleton

Quoted 10 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Sagaert" == Sagaert Johan <[email protected]> writes:

Hi All;

I made this patch because I found it made more sense if I selected a custom skeleton that in that case buildroot should not insert its own hostname, issue,TZ etc into the target root directory. If I can achieve the same result by leaving the hostname etc... empty in buildroot menu then this is fine. I just wanted buildroot not to overwrite anything when I selected a custom skeleton.

Ok, thanks. Then I'll just take the part of the patch that updates the documention.

Signature · 1 line
Bye, Peter Korsgaard
#20 lore raw

PATCH 5/5 manual: add section about dealing efficiently with big image files

In reply to Yann E. MORIN

From: "Yann E. MORIN" <[email protected]>

As reported by Ryan, it is not well-known that most tools can deal efficiently with big sparse files.

Add a section in the manual about this, with tar and cp used as examples, and a hinting to the man pages for the others.

Cc Ryan Barnett <[email protected]>
Cc Peter Korsgaard <[email protected]>
Cc Thomas De Schampheleire <[email protected]>

---
changes v2 -> v3:
- fix typo (Thomas DS)
- better phrasing to direct the reader to the man pages (Thomas DS)

changes v1 -> v2:
- remove 'dd' since it can be dangerous (Peter)
- add a sentence that sparse files are to be used only on the build machine, not while transferring to the target device (Peter)

---
 docs/manual/common-usage.txt | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)
docs/manual/common-usage.txt +39 -0
index c7e4568..1d15c05 100644
@@ -104,3 +104,42 @@ or +g+++ for building helper-binaries on your host, then do
 --------------------
  $ make HOSTCXX=g++-4.3-HEAD HOSTCC=gcc-4.3-HEAD
 --------------------
+
+Dealing efficiently with filesystem images
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Filesystem images can get pretty big, depending on the filesystem you choose,
+the number of packages, whether you provisioned free space... Yet, some
+locations in the filesystems images may just be _empty_ (eg. a long run of
+'zeroes'); such a file is called a _sparse_ file.
+
+Most tools can handle sparse files efficiently, and will only store or write
+those parts of a sparse file that are not empty.
+
+For example:
+
+* +tar+ accepts the +-S+ option to tell it to only store non-zero blocks
+  of sparse files:
+** +tar cf archive.tar -S [files...]+ will efficiently store sparse files
+   in a tarball
+** +tar xf archive.tar -S+ will efficiently store sparse files extracted
+   from a tarball
+
+* +cp+ accepts the +--sparse=WHEN+ option (+WHEN+ is one of +auto+,
+  +never+ or +always+):
+** +cp --sparse=always source.file dest.file+ will make +dest.file+ a
+   sparse file if +source.file+ has long runs of zeroes
+
+Other tools may have similar options. Please consult their respective man
+pages.
+
+You can use sparse files if you need to store the filesystem images (eg.
+to transfer from one machine to another), of if you need to send them (eg.
+to the Q&A team).
+
+Note however that flashing a filesystem image to a device while using the
+sparse mode of +dd+ may result in a broken filesystem (eg. the block bitmap
+of an ext2 filesystem may be corrupted; or, if you have sparse files in
+your filesystem, those parts may not be all-zeroes when read back). You
+should only use sparse files when handling files on the build machine, not
+when transferring them to an actual device that will be used on the target.
Signature · 1 line
1.8.1.2
#21 lore raw

PATCH 5/5 manual: add section about dealing efficiently with big image files

Quoted 15 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

From: "Yann E. MORIN" <[email protected]>
As reported by Ryan, it is not well-known that most tools can deal efficiently with big sparse files.

Add a section in the manual about this, with tar and cp used as examples, and a hinting to the man pages for the others.

Cc Ryan Barnett <[email protected]>
Cc Peter Korsgaard <[email protected]>
Cc Thomas De Schampheleire <[email protected]>
Quoted 4 lines

---
changes v2 -> v3:
- fix typo (Thomas DS)
- better phrasing to direct the reader to the man pages (Thomas DS)

Committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard