Skip to contentSkip to conversation

fs/iso9660: fix transparent (de)compression

PATCH New Yann E. MORIN 2 participants 5 messages Jan 2, 2018

lore
Share & tools
lore

Yann E. MORIN

author
#0 lore raw

cover fs/iso9660: fix transparent (de)compression

Hello All!

We need mkzftree, from zisofs_tools, bundled in cdrkit. This was missing so far, so here we add it as a new package, and make iso9660 depend on it when needed.

The kernel also needs support for compressed isofs, so we update the test kernel config.

Finally, the kernel images must be stored uncompressed in the iso9660 filesystem, because grub does not decompresses the files it reads from the filesystem.

Regards,
Yann E. MORIN.

The following changes since commit 55d79ed93e44e1876334f795c6313ea6f406d229

eeprog: remove package (2018-01-02 10:10:25 +0100)

are available in the git repository at:

git://git.buildroot.org/~ymorin/git/buildroot.git

for you to fetch changes up to e793abe185c20ebd853db927552156e203813613

fs/iso9660: fix transparent (de)compression (2018-01-02 20:57:10 +0100)

----------------------------------------------------------------
Yann E. MORIN (2):
package/zisofs_tools: new package
fs/iso9660: fix transparent (de)compression

 fs/iso9660/iso9660.mk                               | 12 ++++++++----
 package/zisofs_tools/zisofs_tools.hash              |  5 +++++
 package/zisofs_tools/zisofs_tools.mk                | 15 +++++++++++++++
 support/testing/conf/minimal-x86-qemu-kernel.config |  1 +
 4 files changed, 29 insertions(+), 4 deletions(-)
 create mode 100644 package/zisofs_tools/zisofs_tools.hash
 create mode 100644 package/zisofs_tools/zisofs_tools.mk
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
author
#1 lore raw

PATCH 1/2 package/zisofs_tools: new package

To support transparent (de)compression in iso9660, we need mkzftree, which comes from zisofs_tools, biundled with cdrkit.

However, cdrkit is a cmake package, but zisofs_tools is an autotools package, so we need a separate package just to get mkzftree, but it is pretty lightweight.

We just need the host variant for now,so we just add that.

Signed-off-by "Yann E. MORIN" <[email protected]>
Cc Thomas Petazzoni <[email protected]>
---
 package/zisofs_tools/zisofs_tools.hash |  5 +++++
 package/zisofs_tools/zisofs_tools.mk   | 15 +++++++++++++++
 2 files changed, 20 insertions(+)
 create mode 100644 package/zisofs_tools/zisofs_tools.hash
 create mode 100644 package/zisofs_tools/zisofs_tools.mk
package/zisofs_tools/zisofs_tools.hash new file +5 -0
new file mode 100644
index 0000000000..d2f594c757
@@ -0,0 +1,5 @@
+# From http://snapshot.debian.org/archive/debian/20141023T043132Z/pool/main/c/cdrkit/cdrkit_1.1.11-3.dsc
+sha256  d1c030756ecc182defee9fe885638c1785d35a2c2a297b4604c0e0dcc78e47da  cdrkit_1.1.11.orig.tar.gz
+
+# Locally computed
+sha256  32b1062f7da84967e7019d01ab805935caa7ab7321a7ced0e30ebe75e5df1670  3rd-party/zisofs_tools/COPYING
package/zisofs_tools/zisofs_tools.mk new file +15 -0
new file mode 100644
index 0000000000..fd8c371fea
@@ -0,0 +1,15 @@
+################################################################################
+#
+# zisofs-tools
+#
+################################################################################
+
+ZISOFS_TOOLS_VERSION = 1.1.11
+ZISOFS_TOOLS_SOURCE = cdrkit_$(ZISOFS_TOOLS_VERSION).orig.tar.gz
+ZISOFS_TOOLS_SITE = http://snapshot.debian.org/archive/debian/20141023T043132Z/pool/main/c/cdrkit
+ZISOFS_TOOLS_LICENSE = GPL-2.0+
+ZISOFS_TOOLS_LICENSE_FILES = 3rd-party/zisofs_tools/COPYING
+ZISOFS_TOOLS_SUBDIR = 3rd-party/zisofs_tools
+HOST_ZISOFS_TOOLS_DEPENDENCIES = host-zlib
+
+$(eval $(host-autotools-package))
Signature · 1 line
2.11.0
#2 lore raw

PATCH 1/2 package/zisofs_tools: new package

Hello,

Quoted 11 lines · Yann E. MORIN

To support transparent (de)compression in iso9660, we need mkzftree, which comes from zisofs_tools, biundled with cdrkit.

However, cdrkit is a cmake package, but zisofs_tools is an autotools package, so we need a separate package just to get mkzftree, but it is pretty lightweight.

We just need the host variant for now,so we just add that.

Signed-off-by "Yann E. MORIN" <[email protected]>
Cc Thomas Petazzoni <[email protected]>

I've renamed to zisofs-tools (we more commonly use - as a separator, and indeed your comment header in the .mk file was using this name!), added an entry to the DEVELOPERS file, and applied. Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
author
#3 lore raw

PATCH 2/2 fs/iso9660: fix transparent (de)compression

In reply to Yann E. MORIN

It needs mkzftree from zisofs_tools, so we add a dependency to it, and we call that one explicitly (to avoid using the one from the host in PATH.

It also needs the the uncompressed kernel image, but because it is already in target/ so it gets compressed by mkzftree. We have two options:
- compress everything but the kernel image,
- compress everything, kernel included, and recopy it later.

We choose the latter, because it is the simplest solution. So, we alwats define the kernel-copy hook, but only register it when needed.

Finally, itneeds a kernel with support for transparent (de)compression, so we update the existing test config.

Reported-by Thomas Petazzoni <[email protected]>
Signed-off-by "Yann E. MORIN" <[email protected]>
---
 fs/iso9660/iso9660.mk                               | 12 ++++++++----
 support/testing/conf/minimal-x86-qemu-kernel.config |  1 +
 2 files changed, 9 insertions(+), 4 deletions(-)
fs/iso9660/iso9660.mk +8 -4
index 06467262e0..66929a9ccd 100644
@@ -42,11 +42,12 @@ define ROOTFS_ISO9660_CREATE_TEMPDIR
 endef
 ROOTFS_ISO9660_PRE_GEN_HOOKS += ROOTFS_ISO9660_CREATE_TEMPDIR
 else ifeq ($(BR2_TARGET_ROOTFS_ISO9660_TRANSPARENT_COMPRESSION),y)
+ROOTFS_ISO9660_DEPENDENCIES += host-zisofs_tools
 ROOTFS_ISO9660_TARGET_DIR = $(FS_DIR)/rootfs.iso9660.tmp
-# This must be early, before we copy the bootloader files
+# This must be early, before we copy the bootloader files.
 define ROOTFS_ISO9660_MKZFTREE
 	$(RM) -rf $(ROOTFS_ISO9660_TARGET_DIR)
-	mkzftree -X -z 9 -p $(PARALLEL_JOBS) \
+	$(HOST_DIR)/bin/mkzftree -X -z 9 -p $(PARALLEL_JOBS) \
 		$(TARGET_DIR) \
 		$(ROOTFS_ISO9660_TARGET_DIR)
 endef

@@ -92,14 +93,13 @@ define ROOTFS_ISO9660_DISABLE_EXTERNAL_INITRD
 	$(SED) '/__INITRD_PATH__/d'  $(ROOTFS_ISO9660_BOOTLOADER_CONFIG_PATH)
 endef
 
-ifeq ($(ROOTFS_ISO9660_USE_INITRD),YES)
-
 # Copy the kernel to temporary filesystem
 define ROOTFS_ISO9660_COPY_KERNEL
 	$(INSTALL) -D -m 0644 $(LINUX_IMAGE_PATH) \
 		$(ROOTFS_ISO9660_TARGET_DIR)/boot/$(LINUX_IMAGE_NAME)
 endef
 
+ifeq ($(ROOTFS_ISO9660_USE_INITRD),YES)
 ROOTFS_ISO9660_PRE_GEN_HOOKS += ROOTFS_ISO9660_COPY_KERNEL
 
 # If initramfs is used, disable loading the initrd as the rootfs is

@@ -119,6 +119,10 @@ ROOTFS_ISO9660_PRE_GEN_HOOKS += ROOTFS_ISO9660_COPY_INITRD
 endif
 
 else # ROOTFS_ISO9660_USE_INITRD
+ifeq ($(BR2_TARGET_ROOTFS_ISO9660_TRANSPARENT_COMPRESSION),y)
+# We must use the uncompressed kernel image
+ROOTFS_ISO9660_PRE_GEN_HOOKS += ROOTFS_ISO9660_COPY_KERNEL
+endif
 
 ROOTFS_ISO9660_PRE_GEN_HOOKS += ROOTFS_ISO9660_DISABLE_EXTERNAL_INITRD
support/testing/conf/minimal-x86-qemu-kernel.config +1 -0
index 4aaaab7218..cc8ee6a713 100644
@@ -43,6 +43,7 @@ CONFIG_VIRTIO_MMIO=y
 CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES=y
 CONFIG_EXT4_FS=y
 CONFIG_ISO9660_FS=y
+CONFIG_ZISOFS=y
 CONFIG_JOLIET=y
 CONFIG_TMPFS=y
 CONFIG_TMPFS_POSIX_ACL=y
Signature · 1 line
2.11.0
Applied
#4 lore raw

PATCH 2/2 fs/iso9660: fix transparent (de)compression

Hello,

Quoted 16 lines · Yann E. MORIN

It needs mkzftree from zisofs_tools, so we add a dependency to it, and we call that one explicitly (to avoid using the one from the host in PATH.

It also needs the the uncompressed kernel image, but because it is already in target/ so it gets compressed by mkzftree. We have two options:
- compress everything but the kernel image,
- compress everything, kernel included, and recopy it later.

We choose the latter, because it is the simplest solution. So, we alwats define the kernel-copy hook, but only register it when needed.

Finally, itneeds a kernel with support for transparent (de)compression, so we update the existing test config.

Quoted 6 lines
Reported-by Thomas Petazzoni <[email protected]>
Signed-off-by "Yann E. MORIN" <[email protected]>
---
 fs/iso9660/iso9660.mk                               | 12 ++++++++----
 support/testing/conf/minimal-x86-qemu-kernel.config |  1 +
 2 files changed, 9 insertions(+), 4 deletions(-)

Applied to master after changing the dependency on host-zisofs-tools following my change in PATCH 1/2, and fixing a few typos in the commit log.

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com