Skip to contentSkip to conversation

fs: squashfs: Add config option to pass mksquashfs cmdln paramters.

PATCH New Heiko Stuebner 2 participants 4 messages Jun 12, 2020 – Jun 14, 2020 SquashFS file system

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 fs: squashfs: Add config option to pass mksquashfs cmdln paramters.

From: Christoph Muellner <[email protected]>

This can be use to specify additional command line argument to mksquashfs (e.g. -nopad). The implementation is inspired by a similar mechanism for ext file system images.

---
 fs/squashfs/Config.in   | 11 +++++++++++
 fs/squashfs/squashfs.mk |  8 +++++++-
 2 files changed, 18 insertions(+), 1 deletion(-)
fs/squashfs/Config.in +11 -0
index dde2097cb7..789b2cd24a 100644
@@ -5,6 +5,16 @@ config BR2_TARGET_ROOTFS_SQUASHFS
 
 if BR2_TARGET_ROOTFS_SQUASHFS
 
+config BR2_TARGET_ROOTFS_SQUASHFS_MKFS_OPTIONS
+	string "additional mksquashfs options"
+	default ""
+	help
+	  Specify a space-separated list of mksquashfs options, including
+	  any squashfs filesystem features.
+
+	  For more information about the mke2fs options, see the manual
+	  page mksquashfs(1).
+
 choice
 	prompt "Compression algorithm"
 	default BR2_TARGET_ROOTFS_SQUASHFS4_GZIP
@@ -31,4 +41,5 @@ config BR2_TARGET_ROOTFS_SQUASHFS4_ZSTD
 	bool "zstd"
 
 endchoice
+
 endif
fs/squashfs/squashfs.mk +7 -1
index 34ab048603..4bc6b338b0 100644
@@ -6,7 +6,13 @@
 
 ROOTFS_SQUASHFS_DEPENDENCIES = host-squashfs
 
-ROOTFS_SQUASHFS_ARGS = -noappend -processors $(PARALLEL_JOBS)
+SQUASHFS_MKFS_OPTS = $(call qstrip,$(BR2_TARGET_ROOTFS_SQUASHFS_MKFS_OPTIONS))
+
+ROOTFS_SQUASHFS_ARGS = \
+	-noappend \
+	-processors \
+	$(PARALLEL_JOBS) \
+	$(SQUASHFS_MKFS_OPTS)
 
 ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS4_LZ4),y)
 ROOTFS_SQUASHFS_ARGS += -comp lz4 -Xhc
Signature · 1 line
2.26.2
#1 lore raw

PATCH 1/1 fs: squashfs: Add config option to pass mksquashfs cmdln paramters.

Heiko, All,

On 2020-06-12 15:32 +0200, Heiko Stuebner spake thusly:

Quoted 5 lines

From: Christoph Muellner <[email protected]>

This can be use to specify additional command line argument to mksquashfs (e.g. -nopad). The implementation is inspired by a similar mechanism for ext file system images.

What kind of options do you expect to need to pass?

Usually, I'd prefer we have common options directly mapped to entries in the menuconfig, and keep this kind of "pass-through" options for more exotic options.

For example, if we are looking at the compression options, I'd like we add a config option like

config BR2_TARGET_ROOTFS_SQUASHFS_EXTRA_COMPRESS
bool "high-compression"
depends on !BR2_TARGET_ROOTFS_SQUASHFS4_LZMA # No option help
Say 'y' to compress even further, at the expense of compression time.

For example:
- for gzip, this enables all the -Xstrategy
- for lz4, this enables -Xhc
- for lzo, this enables -Xcompression-level 9
- for xz, this chooses appropriate -Xbcj filters
- for zstd, this enables -Xcompression-level 22

And maybe a few other such options, eventually adding the BR2_TARGET_ROOTFS_SQUASHFS_MKFS_OPTIONS for corner-cases only.

Quoted 13 lines
---
 fs/squashfs/Config.in   | 11 +++++++++++
 fs/squashfs/squashfs.mk |  8 +++++++-
 2 files changed, 18 insertions(+), 1 deletion(-)
fs/squashfs/Config.in +0 -0
index dde2097cb7..789b2cd24a 100644
@@ -5,6 +5,16 @@ config BR2_TARGET_ROOTFS_SQUASHFS
 

fs/squashfs/Config.in config BR2_TARGET_ROOTFS_SQUASHFS · line 15

Quoted 10 lines
fs/squashfs/Config.in +8 -0
 if BR2_TARGET_ROOTFS_SQUASHFS
 
+config BR2_TARGET_ROOTFS_SQUASHFS_MKFS_OPTIONS
+	string "additional mksquashfs options"
+	default ""
+	help
+	  Specify a space-separated list of mksquashfs options, including
+	  any squashfs filesystem features.
+
+	  For more information about the mke2fs options, see the manual

s/mke2fs/mksquashfs/

Quoted 16 lines
fs/squashfs/Config.in +3 -0
+	  page mksquashfs(1).
+
 choice
 	prompt "Compression algorithm"
 	default BR2_TARGET_ROOTFS_SQUASHFS4_GZIP
@@ -31,4 +41,5 @@ config BR2_TARGET_ROOTFS_SQUASHFS4_ZSTD
 	bool "zstd"
 
 endchoice
+
 endif
fs/squashfs/squashfs.mk +0 -0
index 34ab048603..4bc6b338b0 100644

fs/squashfs/squashfs.mk line 9

Quoted 5 lines
fs/squashfs/squashfs.mk +1 -1
@@ -6,7 +6,13 @@
 
 ROOTFS_SQUASHFS_DEPENDENCIES = host-squashfs
 
-ROOTFS_SQUASHFS_ARGS = -noappend -processors $(PARALLEL_JOBS)
+SQUASHFS_MKFS_OPTS = $(call qstrip,$(BR2_TARGET_ROOTFS_SQUASHFS_MKFS_OPTIONS))

Keep the vairalbes prefixed with ROOTFS_ here, to avoid clashing with the package namespace: SQUASHFS_MKFS_OPTS by name would be a variable of the "squashfs" package.

fs/squashfs/squashfs.mk line 14

Quoted 4 lines
fs/squashfs/squashfs.mk +4 -0
+ROOTFS_SQUASHFS_ARGS = \
+	-noappend \
+	-processors \
+	$(PARALLEL_JOBS) \

$(PARALLEL_JOBS) is a parameter to the --processor option, so it should be on the same line.

However, I don;t think you need to introduce an intermediate variable:

    ROOTFS_SQUASHFS_ARGS = \
        -noappend \
        -processors $(PARALLEL_JOBS) \
        $(call qstrip,$(BR2_TARGET_ROOTFS_SQUASHFS_MKFS_OPTIONS))

Regards,
Yann E. MORIN.

Quoted 11 lines
fs/squashfs/squashfs.mk +1 -0
+	$(SQUASHFS_MKFS_OPTS)
 
 ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS4_LZ4),y)
 ROOTFS_SQUASHFS_ARGS += -comp lz4 -Xhc
Signature · 6 lines
2.26.2

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#2 lore raw

PATCH 1/1 fs: squashfs: Add config option to pass mksquashfs cmdln paramters.

Hi,

Am Sonntag, 14. Juni 2020, 09:57:25 CEST schrieb Yann E. MORIN:

Quoted 10 lines

Heiko, All,

On 2020-06-12 15:32 +0200, Heiko Stuebner spake thusly:

Quoted 5 lines

From: Christoph Muellner <[email protected]>

This can be use to specify additional command line argument to mksquashfs (e.g. -nopad). The implementation is inspired by a similar mechanism for ext file system images.

What kind of options do you expect to need to pass?

In our case it is "-nopad" to omit padding to the
next 4k boundary.

Quoted 21 lines

Usually, I'd prefer we have common options directly mapped to entries in the menuconfig, and keep this kind of "pass-through" options for more exotic options.

For example, if we are looking at the compression options, I'd like we add a config option like

config BR2_TARGET_ROOTFS_SQUASHFS_EXTRA_COMPRESS
bool "high-compression"
depends on !BR2_TARGET_ROOTFS_SQUASHFS4_LZMA # No option help
Say 'y' to compress even further, at the expense of compression time.

For example:
- for gzip, this enables all the -Xstrategy
- for lz4, this enables -Xhc
- for lzo, this enables -Xcompression-level 9
- for xz, this chooses appropriate -Xbcj filters
- for zstd, this enables -Xcompression-level 22

Quoted 2 lines

And maybe a few other such options, eventually adding the BR2_TARGET_ROOTFS_SQUASHFS_MKFS_OPTIONS for corner-cases only.

ok, so if I'm reading you correctly, I should probably just introduce a
config BR2_TARGET_ROOTFS_SQUASHFS_NOPAD
option, right?

Quoted 15 lines
Quoted 15 lines
---
 fs/squashfs/Config.in   | 11 +++++++++++
 fs/squashfs/squashfs.mk |  8 +++++++-
 2 files changed, 18 insertions(+), 1 deletion(-)
fs/squashfs/Config.in +0 -0
index dde2097cb7..789b2cd24a 100644
@@ -5,6 +5,16 @@ config BR2_TARGET_ROOTFS_SQUASHFS
 
 if BR2_TARGET_ROOTFS_SQUASHFS
 
Quoted 10 lines

fs/squashfs/Config.in config BR2_TARGET_ROOTFS_SQUASHFS · line 15

Quoted 8 lines
fs/squashfs/Config.in +8 -0
+config BR2_TARGET_ROOTFS_SQUASHFS_MKFS_OPTIONS
+	string "additional mksquashfs options"
+	default ""
+	help
+	  Specify a space-separated list of mksquashfs options, including
+	  any squashfs filesystem features.
+
+	  For more information about the mke2fs options, see the manual

s/mke2fs/mksquashfs/

ok

Quoted 17 lines
Quoted 17 lines
fs/squashfs/Config.in +3 -0
+	  page mksquashfs(1).
+
 choice
 	prompt "Compression algorithm"
 	default BR2_TARGET_ROOTFS_SQUASHFS4_GZIP
@@ -31,4 +41,5 @@ config BR2_TARGET_ROOTFS_SQUASHFS4_ZSTD
 	bool "zstd"
 
 endchoice
+
 endif
fs/squashfs/squashfs.mk +0 -0
index 34ab048603..4bc6b338b0 100644
@@ -6,7 +6,13 @@
 
Quoted 8 lines

fs/squashfs/squashfs.mk line 9

Quoted 4 lines
fs/squashfs/squashfs.mk +1 -1
 ROOTFS_SQUASHFS_DEPENDENCIES = host-squashfs
 
-ROOTFS_SQUASHFS_ARGS = -noappend -processors $(PARALLEL_JOBS)
+SQUASHFS_MKFS_OPTS = $(call qstrip,$(BR2_TARGET_ROOTFS_SQUASHFS_MKFS_OPTIONS))

Keep the vairalbes prefixed with ROOTFS_ here, to avoid clashing with the package namespace: SQUASHFS_MKFS_OPTS by name would be a variable of the "squashfs" package.

ok, I wasn't sure here what the correct naming is, a lot of file-systems seem to use their options without ROOTFS_*

So I'll change this to ROOTFS_* then.

Quoted 8 lines

fs/squashfs/squashfs.mk line 14

Quoted 4 lines
fs/squashfs/squashfs.mk +4 -0
+ROOTFS_SQUASHFS_ARGS = \
+	-noappend \
+	-processors \
+	$(PARALLEL_JOBS) \

$(PARALLEL_JOBS) is a parameter to the --processor option, so it should be on the same line.

Quoted 6 lines

However, I don;t think you need to introduce an intermediate variable:

    ROOTFS_SQUASHFS_ARGS = \
        -noappend \
        -processors $(PARALLEL_JOBS) \
        $(call qstrip,$(BR2_TARGET_ROOTFS_SQUASHFS_MKFS_OPTIONS))

this was a bit "modelled" after (aka stolen from) how the extfs does this ;-) But yeah, we can make this nicer .

Heiko

#3 lore raw

PATCH 1/1 fs: squashfs: Add config option to pass mksquashfs cmdln paramters.

Heiko, All,

On 2020-06-14 12:20 +0200, Heiko St?bner spake thusly:

Quoted 6 lines

Am Sonntag, 14. Juni 2020, 09:57:25 CEST schrieb Yann E. MORIN:

Quoted 5 lines

Heiko, All,

On 2020-06-12 15:32 +0200, Heiko Stuebner spake thusly:

Quoted 2 lines

From: Christoph Muellner <[email protected]>

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

This can be use to specify additional command line argument to mksquashfs (e.g. -nopad). The implementation is inspired by a similar mechanism for ext file system images.

What kind of options do you expect to need to pass?

In our case it is "-nopad" to omit padding to the
next 4k boundary.

[--SNIP--]

Quoted 4 lines

ok, so if I'm reading you correctly, I should probably just introduce a
config BR2_TARGET_ROOTFS_SQUASHFS_NOPAD
option, right?

Basically, right. But usually, we go with positive logic (at least user-facing), so:

config BR2_TARGET_ROOTFS_SQUASHFS_PAD
bool "pad to a 4K boundary"
default y # legacy was always ON
help
Say 'y' here (the default) to pad the the filesystem image to a 4K boundary. Say 'n' to disable padding.

And then in the .mk :

    ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS_PAD),)
    ROOTFS_SQUASHFS_ARGS += -nopad
    endif

[--SNIP--]

Quoted 9 lines
Quoted 6 lines

fs/squashfs/squashfs.mk line 9

Quoted 2 lines
fs/squashfs/squashfs.mk +1 -1
-ROOTFS_SQUASHFS_ARGS = -noappend -processors $(PARALLEL_JOBS)
+SQUASHFS_MKFS_OPTS = $(call qstrip,$(BR2_TARGET_ROOTFS_SQUASHFS_MKFS_OPTIONS))

Keep the vairalbes prefixed with ROOTFS_ here, to avoid clashing with the package namespace: SQUASHFS_MKFS_OPTS by name would be a variable of the "squashfs" package.

ok, I wasn't sure here what the correct naming is, a lot of file-systems seem to use their options without ROOTFS_*

Yes, this is historical, and so far it works: we have no package named 'ext', so there are not actual clashing. But we do have a pakcage named 'squashfs', so there can be clashing.

Quoted 8 lines
Quoted 7 lines

$(PARALLEL_JOBS) is a parameter to the --processor option, so it should be on the same line.
However, I don;t think you need to introduce an intermediate variable:

    ROOTFS_SQUASHFS_ARGS = \
        -noappend \
        -processors $(PARALLEL_JOBS) \
        $(call qstrip,$(BR2_TARGET_ROOTFS_SQUASHFS_MKFS_OPTIONS))

this was a bit "modelled" after (aka stolen from) how the extfs does this ;-)

I see, but I don't see the point there either.

Quoted 1 line

But yeah, we can make this nicer .

Please! :-)

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'