Manage big root directory trees by breaking them up.
Subsystems (7)
cover Manage big root directory trees by breaking them up.
From: David VomLehn <[email protected]>
These patches are not as tightly coupled as the usually series of patches, but I think the pieces make more sense when viewed in the context of the whole patchset. The problem to be solved is that Python is big and my root filesystem has to fit in a little flash device. So, I:
1. Created the ability to break the target filesystem into multiple
filesystems,
2. Enhanced autotools so that I could specify the --prefix and
--exec-prefix values. This allowed me to install packages in their own directory, which the above patch lets me split into a filesystem I can store on a much bigger flash device, and
3. I modified the Python package so that the installation directory
is a configurable.
With these in place, I configure Python to be in /usr/local and make that a separate filesystem, which I can deploy in a nice big flash device. Problem solved.
David VomLehn (3):
Add the capability to create sub-filesystems from the target
filesystem.
Allow specification of values for --prefix and --exec-prefix in
autotools.
Support installation of Python in a different directory.
Makefile | 5 +- docs/manual/adding-packages-autotools.txt | 8 ++ docs/manual/customize-rootfs.txt | 74 ++++++++++ fs/Config.in | 25 ++++ fs/cloop/Config.in | 12 ++ fs/cloop/cloop-post.mk | 13 ++ fs/cloop/cloop-pre.mk | 7 + fs/common.mk | 224 ++++++++++++++++++++++++++--- fs/cpio/Config.in | 12 ++ fs/cpio/cpio-post.mk | 42 ++++++ fs/cpio/cpio-pre.mk | 7 + fs/cramfs/Config.in | 12 ++ fs/cramfs/cramfs-post.mk | 19 +++ fs/cramfs/cramfs-pre.mk | 7 + fs/ext2/Config.in | 12 ++ fs/ext2/ext2-post.mk | 37 +++++ fs/ext2/ext2-pre.mk | 6 + fs/initramfs/initramfs-post.mk | 22 +++ fs/initramfs/initramfs-pre.mk | 5 + fs/iso9660/iso9660-post.mk | 51 +++++++ fs/iso9660/iso9660-pre.mk | 9 ++ fs/jffs2/Config.in | 12 ++ fs/jffs2/jffs2-post.mk | 49 +++++++ fs/jffs2/jffs2-pre.mk | 7 + fs/romfs/Config.in | 12 ++ fs/romfs/romfs-post.mk | 15 ++ fs/romfs/romfs-pre.mk | 7 + fs/squashfs/Config.in | 12 ++ fs/squashfs/squashfs-post.mk | 29 ++++ fs/squashfs/squashfs-pre.mk | 7 + fs/tar/Config.in | 12 ++ fs/tar/tar-post.mk | 13 ++ fs/tar/tar-pre.mk | 7 + fs/ubifs/Config.in | 12 ++ fs/ubifs/ubi-post.mk | 30 ++++ fs/ubifs/ubi-pre.mk | 7 + fs/ubifs/ubifs-post.mk | 27 ++++ fs/ubifs/ubifs-pre.mk | 7 + fs/yaffs2/Config.in | 12 ++ fs/yaffs2/yaffs-post.mk | 13 ++ fs/yaffs2/yaffs-pre.mk | 7 + package/pkg-autotools.mk | 7 +- package/python/Config.in | 7 + package/python/python.mk | 26 ++-- 44 files changed, 911 insertions(+), 35 deletions(-) create mode 100644 fs/cloop/cloop-post.mk create mode 100644 fs/cloop/cloop-pre.mk create mode 100644 fs/cpio/cpio-post.mk create mode 100644 fs/cpio/cpio-pre.mk create mode 100644 fs/cramfs/cramfs-post.mk create mode 100644 fs/cramfs/cramfs-pre.mk create mode 100644 fs/ext2/ext2-post.mk create mode 100644 fs/ext2/ext2-pre.mk create mode 100644 fs/initramfs/initramfs-post.mk create mode 100644 fs/initramfs/initramfs-pre.mk create mode 100644 fs/iso9660/iso9660-post.mk create mode 100644 fs/iso9660/iso9660-pre.mk create mode 100644 fs/jffs2/jffs2-post.mk create mode 100644 fs/jffs2/jffs2-pre.mk create mode 100644 fs/romfs/romfs-post.mk create mode 100644 fs/romfs/romfs-pre.mk create mode 100644 fs/squashfs/squashfs-post.mk create mode 100644 fs/squashfs/squashfs-pre.mk create mode 100644 fs/tar/tar-post.mk create mode 100644 fs/tar/tar-pre.mk create mode 100644 fs/ubifs/ubi-post.mk create mode 100644 fs/ubifs/ubi-pre.mk create mode 100644 fs/ubifs/ubifs-post.mk create mode 100644 fs/ubifs/ubifs-pre.mk create mode 100644 fs/yaffs2/yaffs-post.mk create mode 100644 fs/yaffs2/yaffs-pre.mk
Signature · 1 line
1.7.9.5
PATCH 1/3 Add the capability to create sub-filesystems from the target filesystem.
From: David VomLehn <[email protected]>
For example, say that the the root filesystem has a set of files and directories that are critical to being able to control, recover, and install software on an embedded system. These are redundantly stored in a small but reliable memory. The rest of the system is written in Python, which is huge (too big to fit in the reliable memory) but not a critical part of the system. It can, if necessary, be reloaded remotely, so it is stored without redundancy, in NAND flash.
This patch allows these two pieces to be split up so that the reliable core can be an initramfs filesystem (in cpio format) and a JFFS2 image can hold the rest of the root filesystem. It supports an arbitrary division of the root filesystem among one or more of the available filesystem formats.
--- Makefile | 5 +- docs/manual/customize-rootfs.txt | 74 +++++++++++++ fs/Config.in | 25 +++++ fs/cloop/Config.in | 12 ++ fs/cloop/cloop-post.mk | 13 +++ fs/cloop/cloop-pre.mk | 7 ++ fs/common.mk | 224 ++++++++++++++++++++++++++++++++++---- fs/cpio/Config.in | 12 ++ fs/cpio/cpio-post.mk | 42 +++++++ fs/cpio/cpio-pre.mk | 7 ++ fs/cramfs/Config.in | 12 ++ fs/cramfs/cramfs-post.mk | 19 ++++ fs/cramfs/cramfs-pre.mk | 7 ++ fs/ext2/Config.in | 12 ++ fs/ext2/ext2-post.mk | 37 +++++++ fs/ext2/ext2-pre.mk | 6 + fs/initramfs/initramfs-post.mk | 22 ++++ fs/initramfs/initramfs-pre.mk | 5 + fs/iso9660/iso9660-post.mk | 51 +++++++++ fs/iso9660/iso9660-pre.mk | 9 ++ fs/jffs2/Config.in | 12 ++ fs/jffs2/jffs2-post.mk | 49 +++++++++ fs/jffs2/jffs2-pre.mk | 7 ++ fs/romfs/Config.in | 12 ++ fs/romfs/romfs-post.mk | 15 +++ fs/romfs/romfs-pre.mk | 7 ++ fs/squashfs/Config.in | 12 ++ fs/squashfs/squashfs-post.mk | 29 +++++ fs/squashfs/squashfs-pre.mk | 7 ++ fs/tar/Config.in | 12 ++ fs/tar/tar-post.mk | 13 +++ fs/tar/tar-pre.mk | 7 ++ fs/ubifs/Config.in | 12 ++ fs/ubifs/ubi-post.mk | 30 +++++ fs/ubifs/ubi-pre.mk | 7 ++ fs/ubifs/ubifs-post.mk | 27 +++++ fs/ubifs/ubifs-pre.mk | 7 ++ fs/yaffs2/Config.in | 12 ++ fs/yaffs2/yaffs-post.mk | 13 +++ fs/yaffs2/yaffs-pre.mk | 7 ++ 40 files changed, 876 insertions(+), 22 deletions(-) create mode 100644 fs/cloop/cloop-post.mk create mode 100644 fs/cloop/cloop-pre.mk create mode 100644 fs/cpio/cpio-post.mk create mode 100644 fs/cpio/cpio-pre.mk create mode 100644 fs/cramfs/cramfs-post.mk create mode 100644 fs/cramfs/cramfs-pre.mk create mode 100644 fs/ext2/ext2-post.mk create mode 100644 fs/ext2/ext2-pre.mk create mode 100644 fs/initramfs/initramfs-post.mk create mode 100644 fs/initramfs/initramfs-pre.mk create mode 100644 fs/iso9660/iso9660-post.mk create mode 100644 fs/iso9660/iso9660-pre.mk create mode 100644 fs/jffs2/jffs2-post.mk create mode 100644 fs/jffs2/jffs2-pre.mk create mode 100644 fs/romfs/romfs-post.mk create mode 100644 fs/romfs/romfs-pre.mk create mode 100644 fs/squashfs/squashfs-post.mk create mode 100644 fs/squashfs/squashfs-pre.mk create mode 100644 fs/tar/tar-post.mk create mode 100644 fs/tar/tar-pre.mk create mode 100644 fs/ubifs/ubi-post.mk create mode 100644 fs/ubifs/ubi-pre.mk create mode 100644 fs/ubifs/ubifs-post.mk create mode 100644 fs/ubifs/ubifs-pre.mk create mode 100644 fs/yaffs2/yaffs-post.mk create mode 100644 fs/yaffs2/yaffs-pre.mk
Makefile +3 -2
@@ -332,7 +332,7 @@ TARGET_SKELETON = $(TOPDIR)/system/skeleton # Location of a file giving a big fat warning that output/target # should not be used as the root filesystem. -TARGET_DIR_WARNING_FILE = $(TARGET_DIR)/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM +TARGET_DIR_WARNING_FILE = THIS_IS_NOT_YOUR_ROOT_FILESYSTEM ifeq ($(BR2_CCACHE),y) CCACHE := $(HOST_DIR)/usr/bin/ccache
@@ -488,7 +488,8 @@ $(BUILD_DIR)/.root: rsync -a --ignore-times $(RSYNC_VCS_EXCLUSIONS) \ --chmod=Du+w --exclude .empty --exclude '*~' \ $(TARGET_SKELETON)/ $(TARGET_DIR)/ - $(INSTALL) -m 0644 support/misc/target-dir-warning.txt $(TARGET_DIR_WARNING_FILE) + $(INSTALL) -m 0644 support/misc/target-dir-warning.txt \ + $(TARGET_DIR)/$(TARGET_DIR_WARNING_FILE) @ln -snf lib $(TARGET_DIR)/$(LIB_SYMLINK) @mkdir -p $(TARGET_DIR)/usr @ln -snf lib $(TARGET_DIR)/usr/$(LIB_SYMLINK)
docs/manual/customize-rootfs.txt +74 -0
77 lines not on this page · raw message
fs/Config.in +25 -0
@@ -1,5 +1,30 @@ menu "Filesystem images" +config BR2_BUILD_SUB_FILESYSTEMS + bool "Build sub-filesystems" + help + Normally, all target files will be put into a single root filesystem. + Choosing this option allows sub-directories of the root filesystem + to be split into sub-filesystems, possible of different types. + These filesystems can then be written to different devices and + mounted at boot time to create the original view of the root + filesystem. + + Note that this will split up the root filesystems based on the + concatenation of filesystems listed under all filesystems types. + So, if "/ and /usr" are listed under cpio and "/ /usr/share" are + listed under JFFS2, four filesystem images will be created: + o rootfs.cpio: A cpio image containing everything under the + original root filesystem except for /usr + o rootfs.jffs: A JFFS2 image with everything under the original + root filesystem except for /usr, i.e the same contents as + rootfs.cpio, but as a different filesystem image type. + o rootfs-usrfs.cpio: Everything under the original /usr directory + except for files under /usr/share, but including the /usr/share + directory. + o rootfs-usr-sharefs.jffs2: Everything under the original + /usr/share directory. + source "fs/cloop/Config.in" source "fs/cpio/Config.in" source "fs/cramfs/Config.in"
fs/cloop/Config.in +12 -0
@@ -8,3 +8,15 @@ config BR2_TARGET_ROOTFS_CLOOP compressed filesystem like a block device and seamlessly decompress its data while accessing it. The majority of the software on an LNX-BBC is accessed in this fashion. + +if BR2_BUILD_SUB_FILESYSTEMS +config BR2_CLOOP_SUB_FILESYSTEMS + string "Sub-filesystems to build with cloop" + default "/" + depends on BR2_TARGET_ROOTFS_CLOOP + help + Provide a space-separated list of the root directories of all + filesystems. For example to build a root filesystem with a + separate /usr filesystem, enter: + / /usr +endif # BR2_BUILD_SUB_FILESYSTEMS
fs/cloop/cloop-post.mk new file +13 -0
@@ -0,0 +1,13 @@ +################################################################################ +# +# Build the compressed loop root filesystem image +# +################################################################################ + +ROOTFS_CLOOP_DEPENDENCIES = host-cloop host-cdrkit + +define ROOTFS_CLOOP_CMD + $(HOST_DIR)/usr/bin/genisoimage -r $(1) | $(HOST_DIR)/usr/bin/create_compressed_fs - 65536 > $@ +endef + +$(eval $(call ROOTFS_TARGETS,cloop,$(CLOOP_SUBFS)))
fs/cloop/cloop-pre.mk new file +7 -0
@@ -0,0 +1,7 @@ +################################################################################ +# +# Define cloop filesystems +# +################################################################################ + +CLOOP_SUBFS = $(if $(BR2_BUILD_SUB_FILESYSTEMS),$(BR2_CLOOP_SUB_FILESYSTEMS),/)
fs/common.mk +204 -20
263 lines not on this page · raw message
fs/cpio/Config.in +12 -0
@@ -5,6 +5,18 @@ config BR2_TARGET_ROOTFS_CPIO used for an initial RAM filesystem that is passed to the kernel by the bootloader. +if BR2_BUILD_SUB_FILESYSTEMS +config BR2_CPIO_SUB_FILESYSTEMS + string "Sub-filesystems to build with cpio" + default "/" + depends on BR2_TARGET_ROOTFS_CPIO + help + Provide a space-separated list of the root directories of all + filesystems. For example to build a root filesystem with a + separate /usr filesystem, enter: + / /usr +endif # BR2_BUILD_SUB_FILESYSTEMS + if BR2_TARGET_ROOTFS_CPIO choice
fs/cpio/cpio-post.mk new file +42 -0
42 lines not on this page · raw message
fs/cpio/cpio-pre.mk new file +7 -0
@@ -0,0 +1,7 @@ +################################################################################ +# +# Define cpio filesystems +# +################################################################################ + +CPIO_SUBFS = $(if $(BR2_BUILD_SUB_FILESYSTEMS),$(BR2_CPIO_SUB_FILESYSTEMS),/)
fs/cramfs/Config.in +12 -0
@@ -4,3 +4,15 @@ config BR2_TARGET_ROOTFS_CRAMFS Build a cramfs root filesystem http://sourceforge.net/projects/cramfs/ + +if BR2_BUILD_SUB_FILESYSTEMS +config BR2_CRAMFS_SUB_FILESYSTEMS + string "Sub-filesystems to build with cramfs" + default "/" + depends on BR2_TARGET_ROOTFS_CRAMFS + help + Provide a space-separated list of the root directories of all + filesystems. For example to build a root filesystem with a + separate /usr filesystem, enter: + / /usr +endif # BR2_BUILD_SUB_FILESYSTEMS
fs/cramfs/cramfs-post.mk new file +19 -0
@@ -0,0 +1,19 @@ +################################################################################ +# +# Build the cramfs root filesystem image +# +################################################################################ + +ifeq ($(BR2_ENDIAN),"BIG") +CRAMFS_OPTS = -b +else +CRAMFS_OPTS = -l +endif + +define ROOTFS_CRAMFS_CMD + $(HOST_DIR)/usr/bin/mkcramfs $(CRAMFS_OPTS) $(1) $@ +endef + +ROOTFS_CRAMFS_DEPENDENCIES = host-cramfs + +$(eval $(call ROOTFS_TARGETS,cramfs,$(CRAMFS_SUBFS)))
fs/cramfs/cramfs-pre.mk new file +7 -0
@@ -0,0 +1,7 @@ +################################################################################ +# +# Define cramfs filesystems +# +################################################################################ + +CRAMFS_SUBFS = $(if $(BR2_BUILD_SUB_FILESYSTEMS),$(BR2_CRAMFS_SUB_FILESYSTEMS),/)
fs/ext2/Config.in +12 -0
@@ -5,6 +5,18 @@ config BR2_TARGET_ROOTFS_EXT2 help Build an ext2/3/4 root filesystem +if BR2_BUILD_SUB_FILESYSTEMS +config BR2_EXT2_SUB_FILESYSTEMS + string "Sub-filesystems to build with ext2/3/4" + default "/" + depends on BR2_TARGET_ROOTFS_EXT2 + help + Provide a space-separated list of the root directories of all + filesystems. For example to build a root filesystem with a + separate /usr filesystem, enter: + / /usr +endif # BR2_BUILD_SUB_FILESYSTEMS + if BR2_TARGET_ROOTFS_EXT2 config BR2_TARGET_ROOTFS_EXT2_2
fs/ext2/ext2-post.mk new file +37 -0
@@ -0,0 +1,37 @@ +################################################################################ +# +# Build the ext2 root filesystem image +# +################################################################################ + +EXT2_OPTS := + +ifneq ($(strip $(BR2_TARGET_ROOTFS_EXT2_BLOCKS)),0) +EXT2_OPTS += -b $(BR2_TARGET_ROOTFS_EXT2_BLOCKS) +endif + +ifneq ($(strip $(BR2_TARGET_ROOTFS_EXT2_INODES)),0) +EXT2_OPTS += -N $(BR2_TARGET_ROOTFS_EXT2_INODES) +endif + +ifneq ($(strip $(BR2_TARGET_ROOTFS_EXT2_RESBLKS)),0) +EXT2_OPTS += -m $(BR2_TARGET_ROOTFS_EXT2_RESBLKS) +endif + +ROOTFS_EXT2_DEPENDENCIES = host-genext2fs host-e2fsprogs + +EXT2_ENV = GEN=$(BR2_TARGET_ROOTFS_EXT2_GEN) +EXT2_ENV += REV=$(BR2_TARGET_ROOTFS_EXT2_REV) + +define ROOTFS_EXT2_CMD + PATH=$(BR_PATH) $(EXT2_ENV) fs/ext2/genext2fs.sh -d $(1) $(EXT2_OPTS) $@ +endef + +rootfs-ext2-symlink: + ln -sf rootfs.ext2$(ROOTFS_EXT2_COMPRESS_EXT) $(BINARIES_DIR)/rootfs.ext$(BR2_TARGET_ROOTFS_EXT2_GEN)$(ROOTFS_EXT2_COMPRESS_EXT) + +ifneq ($(BR2_TARGET_ROOTFS_EXT2_GEN),2) +ROOTFS_EXT2_POST_TARGETS += rootfs-ext2-symlink +endif + +$(eval $(call ROOTFS_TARGETS,ext2,$(EXT2_SUBFS)))
fs/ext2/ext2-pre.mk new file +6 -0
@@ -0,0 +1,6 @@ +################################################################################ +# +# Define ext2/3/4 filesystems +# +################################################################################ +EXT2_SUBFS = $(if $(BR2_BUILD_SUB_FILESYSTEMS),$(BR2_EXT2_SUB_FILESYSTEMS),/)
fs/initramfs/initramfs-post.mk new file +22 -0
@@ -0,0 +1,22 @@ +################################################################################ +# +# Build a kernel with an integrated initial ramdisk +# filesystem based on cpio. +# +################################################################################ + +ROOTFS_INITRAMFS_DEPENDENCIES += rootfs-cpio + +ROOTFS_INITRAMFS_POST_TARGETS += linux26-rebuild-with-initramfs + + +# The generic fs infrastructure isn't very useful here. + +rootfs-initramfs: $(ROOTFS_INITRAMFS_DEPENDENCIES) $(ROOTFS_INITRAMFS_POST_TARGETS) + +rootfs-initramfs-show-depends: + @echo $(ROOTFS_INITRAMFS_DEPENDENCIES) + +ifeq ($(BR2_TARGET_ROOTFS_INITRAMFS),y) +TARGETS_ROOTFS += rootfs-initramfs +endif
fs/initramfs/initramfs-pre.mk new file +5 -0
@@ -0,0 +1,5 @@ +################################################################################ +# +# Define initramfs filesystems, which doesn't really fit in quite right +# +################################################################################
fs/iso9660/iso9660-post.mk new file +51 -0
51 lines not on this page · raw message
fs/iso9660/iso9660-pre.mk new file +9 -0
@@ -0,0 +1,9 @@ +################################################################################ +# +# Build ISO9660 filesystems +# +# Cannot be converted to the ROOTFS_TARGET infrastructure, because of +# the temporary construction in ISO9660_TARGET_DIR. +# +################################################################################ +
fs/jffs2/Config.in +12 -0
@@ -3,6 +3,18 @@ config BR2_TARGET_ROOTFS_JFFS2 help Build a jffs2 root filesystem +if BR2_BUILD_SUB_FILESYSTEMS +config BR2_JFFS2_SUB_FILESYSTEMS + string "Sub-filesystems to build with JFFS2" + default "/" + depends on BR2_TARGET_ROOTFS_JFFS2 + help + Provide a space-separated list of the root directories of all + filesystems. For example to build a root filesystem with a + separate /usr filesystem, enter: + / /usr +endif # BR2_BUILD_SUB_FILESYSTEMS + if BR2_TARGET_ROOTFS_JFFS2 choice
fs/jffs2/jffs2-post.mk new file +49 -0
49 lines not on this page · raw message
fs/jffs2/jffs2-pre.mk new file +7 -0
@@ -0,0 +1,7 @@ +################################################################################ +# +# Define JFFS2 filesystems +# +################################################################################ + +JFFS2_SUBFS = $(if $(BR2_BUILD_SUB_FILESYSTEMS),$(BR2_JFFS2_SUB_FILESYSTEMS),/)
fs/romfs/Config.in +12 -0
@@ -2,3 +2,15 @@ config BR2_TARGET_ROOTFS_ROMFS bool "romfs root filesystem" help Build a romfs image of the root filesystem. + +if BR2_BUILD_SUB_FILESYSTEMS +config BR2_ROMFS_SUB_FILESYSTEMS + string "Sub-filesystems to build as a ROMFS filesystem" + default "/" + depends on BR2_TARGET_ROOTFS_ROMFS + help + Provide a space-separated list of the root directories of all + filesystems. For example to build a root filesystem with a + separate /usr filesystem, enter: + / /usr +endif # BR2_BUILD_SUB_FILESYSTEMS
fs/romfs/romfs-post.mk new file +15 -0
@@ -0,0 +1,15 @@ +################################################################################ +# +# Build the romfs root filesystem image +# +################################################################################ + +ROMFS_TARGET=$(IMAGE).romfs + +ROOTFS_ROMFS_DEPENDENCIES = host-genromfs + +define ROOTFS_ROMFS_CMD + $(HOST_DIR)/usr/bin/genromfs -d $(1) -f $@ +endef + +$(eval $(call ROOTFS_TARGETS,romfs,$(ROMFS_SUBFS)))
fs/romfs/romfs-pre.mk new file +7 -0
@@ -0,0 +1,7 @@ +################################################################################ +# +# Define romfs filesystems +# +################################################################################ + +ROMFS_SUBFS = $(if $(BR2_BUILD_SUB_FILESYSTEMS),$(BR2_ROMFS_SUB_FILESYSTEMS),/)
fs/squashfs/Config.in +12 -0
@@ -3,6 +3,18 @@ config BR2_TARGET_ROOTFS_SQUASHFS help Build a squashfs root filesystem +if BR2_BUILD_SUB_FILESYSTEMS +config BR2_SQUASHFS_SUB_FILESYSTEMS + string "Sub-filesystems to build with squashfs" + default "/" + depends on BR2_TARGET_ROOTFS_SQUASHFS + help + Provide a space-separated list of the root directories of all + filesystems. For example to build a root filesystem with a + separate /usr filesystem, enter: + / /usr +endif # BR2_BUILD_SUB_FILESYSTEMS + if BR2_TARGET_ROOTFS_SQUASHFS choice
fs/squashfs/squashfs-post.mk new file +29 -0
@@ -0,0 +1,29 @@ +################################################################################ +# +# Build the squashfs root filesystem image +# +################################################################################ + +ROOTFS_SQUASHFS_DEPENDENCIES = host-squashfs + +ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS4_LZO),y) +ROOTFS_SQUASHFS_ARGS += -comp lzo +else +ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS4_LZMA),y) +ROOTFS_SQUASHFS_ARGS += -comp lzma +else +ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS4_XZ),y) +ROOTFS_SQUASHFS_ARGS += -comp xz +else +ROOTFS_SQUASHFS_ARGS += -comp gzip +endif +endif +endif + +define ROOTFS_SQUASHFS_CMD + $(HOST_DIR)/usr/bin/mksquashfs $(1) $@ -noappend \ + $(ROOTFS_SQUASHFS_ARGS) && \ + chmod 0644 $@ +endef + +$(eval $(call ROOTFS_TARGETS,squashfs,$(SQUASHFS_SUBFS)))
fs/squashfs/squashfs-pre.mk new file +7 -0
@@ -0,0 +1,7 @@ +################################################################################ +# +# Define squashfs filesystems +# +################################################################################ + +SQUASHFS_SUBFS = $(if $(BR2_BUILD_SUB_FILESYSTEMS),$(BR2_SQUASHFS_SUB_FILESYSTEMS),/)
fs/tar/Config.in +12 -0
@@ -4,6 +4,18 @@ config BR2_TARGET_ROOTFS_TAR help Build a tar archive of the root filesystem +if BR2_BUILD_SUB_FILESYSTEMS +config BR2_TAR_SUB_FILESYSTEMS + string "Sub-filesystems to build with tar" + default "/" + depends on BR2_TARGET_ROOTFS_TAR + help + Provide a space-separated list of the root directories of all + filesystems. For example to build a root filesystem with a + separate /usr filesystem, enter: + / /usr +endif # BR2_BUILD_SUB_FILESYSTEMS + choice prompt "Compression method" default BR2_TARGET_ROOTFS_TAR_NONE
fs/tar/tar-post.mk new file +13 -0
@@ -0,0 +1,13 @@ +################################################################################ +# +# tar to archive target filesystem +# +################################################################################ + +TAR_OPTS := $(BR2_TARGET_ROOTFS_TAR_OPTIONS) + +define ROOTFS_TAR_CMD + tar -c$(TAR_OPTS)f $@ -C $(1) . +endef + +$(eval $(call ROOTFS_TARGETS,tar,$(TAR_SUBFS)))
fs/tar/tar-pre.mk new file +7 -0
@@ -0,0 +1,7 @@ +################################################################################ +# +# Define tar-archived filesystems +# +################################################################################ + +TAR_SUBFS = $(if $(BR2_BUILD_SUB_FILESYSTEMS),$(BR2_TAR_SUB_FILESYSTEMS),/)
fs/ubifs/Config.in +12 -0
@@ -3,6 +3,18 @@ config BR2_TARGET_ROOTFS_UBIFS help Build a ubifs root filesystem +if BR2_BUILD_SUB_FILESYSTEMS +config BR2_UBIFS_SUB_FILESYSTEMS + string "Sub-filesystems to build with ubifs" + default "/" + depends on BR2_TARGET_ROOTFS_UBIFS + help + Provide a space-separated list of the root directories of all + filesystems. For example to build a root filesystem with a + separate /usr filesystem, enter: + / /usr +endif # BR2_BUILD_SUB_FILESYSTEMS + if BR2_TARGET_ROOTFS_UBIFS config BR2_TARGET_ROOTFS_UBIFS_LEBSIZE
fs/ubifs/ubi-post.mk new file +30 -0
@@ -0,0 +1,30 @@ +################################################################################ +# +# Embed the ubifs image into an ubi image +# +################################################################################ + +UBI_UBINIZE_OPTS := -m $(BR2_TARGET_ROOTFS_UBIFS_MINIOSIZE) +UBI_UBINIZE_OPTS += -p $(BR2_TARGET_ROOTFS_UBI_PEBSIZE) +ifneq ($(BR2_TARGET_ROOTFS_UBI_SUBSIZE),0) +UBI_UBINIZE_OPTS += -s $(BR2_TARGET_ROOTFS_UBI_SUBSIZE) +endif + +UBI_UBINIZE_OPTS += $(call qstrip,$(BR2_TARGET_ROOTFS_UBI_OPTS)) + +ROOTFS_UBI_DEPENDENCIES = rootfs-ubifs + +ifeq ($(BR2_TARGET_ROOTFS_UBI_USE_CUSTOM_CONFIG),y) +UBINIZE_CONFIG_FILE_PATH = $(call qstrip,$(BR2_TARGET_ROOTFS_UBI_CUSTOM_CONFIG_FILE)) +else +UBINIZE_CONFIG_FILE_PATH = fs/ubifs/ubinize.cfg +endif + +define ROOTFS_UBI_CMD + $(INSTALL) -m 0644 $(UBINIZE_CONFIG_FILE_PATH) $(BUILD_DIR)/ubinize.cfg ;\ + $(SED) 's;BR2_ROOTFS_UBIFS_PATH;$@fs;' $(BUILD_DIR)/ubinize.cfg ;\ + $(HOST_DIR)/usr/sbin/ubinize -o $@ $(UBI_UBINIZE_OPTS) $(BUILD_DIR)/ubinize.cfg ;\ + rm $(BUILD_DIR)/ubinize.cfg +endef + +$(eval $(call ROOTFS_TARGETS,ubi,$(UBI_SUBFS)))
fs/ubifs/ubi-pre.mk new file +7 -0
@@ -0,0 +1,7 @@ +################################################################################ +# +# Define embedded UBI filesystems +# +################################################################################ + +UBI_SUBFS = $(if $(BR2_BUILD_SUB_FILESYSTEMS),$(BR2_UBI_SUB_FILESYSTEMS),/)
fs/ubifs/ubifs-post.mk new file +27 -0
@@ -0,0 +1,27 @@ +################################################################################ +# +# Build the ubifs root filesystem image +# +################################################################################ + +UBIFS_OPTS := -e $(BR2_TARGET_ROOTFS_UBIFS_LEBSIZE) -c $(BR2_TARGET_ROOTFS_UBIFS_MAXLEBCNT) -m $(BR2_TARGET_ROOTFS_UBIFS_MINIOSIZE) + +ifeq ($(BR2_TARGET_ROOTFS_UBIFS_RT_ZLIB),y) +UBIFS_OPTS += -x zlib +endif +ifeq ($(BR2_TARGET_ROOTFS_UBIFS_RT_LZO),y) +UBIFS_OPTS += -x lzo +endif +ifeq ($(BR2_TARGET_ROOTFS_UBIFS_RT_NONE),y) +UBIFS_OPTS += -x none +endif + +UBIFS_OPTS += $(call qstrip,$(BR2_TARGET_ROOTFS_UBIFS_OPTS)) + +ROOTFS_UBIFS_DEPENDENCIES = host-mtd + +define ROOTFS_UBIFS_CMD + $(HOST_DIR)/usr/sbin/mkfs.ubifs -d $(1) $(UBIFS_OPTS) -o $@ +endef + +$(eval $(call ROOTFS_TARGETS,ubifs,$(UBIFS_SUBFS)))
fs/ubifs/ubifs-pre.mk new file +7 -0
@@ -0,0 +1,7 @@ +################################################################################ +# +# Define ubifs filesystems +# +################################################################################ + +UBIFS_SUBFS = $(if $(BR2_BUILD_SUB_FILESYSTEMS),$(BR2_UBIFS_SUB_FILESYSTEMS),/)
fs/yaffs2/Config.in +12 -0
@@ -2,3 +2,15 @@ config BR2_TARGET_ROOTFS_YAFFS2 bool "yaffs2 root filesystem" help Build a yaffs2 root filesystem + +if BR2_BUILD_SUB_FILESYSTEMS +config BR2_YAFFS2_SUB_FILESYSTEMS + string "Sub-filesystems to build with YAFFS2" + default "/" + depends on BR2_TARGET_ROOTFS_YAFFS2 + help + Provide a space-separated list of the root directories of all + filesystems. For example to build a root filesystem with a + separate /usr filesystem, enter: + / /usr +endif # BR2_BUILD_SUB_FILESYSTEMS
fs/yaffs2/yaffs-post.mk new file +13 -0
@@ -0,0 +1,13 @@ +################################################################################ +# +# Build the yaffs2 root filesystem image +# +################################################################################ + +ROOTFS_YAFFS2_DEPENDENCIES = host-yaffs2utils + +define ROOTFS_YAFFS2_CMD + $(HOST_DIR)/usr/bin/mkyaffs2 --all-root $(1) $@ +endef + +$(eval $(call ROOTFS_TARGETS,yaffs2,$(YAFFS2_SUBFS)))
fs/yaffs2/yaffs-pre.mk new file +7 -0
@@ -0,0 +1,7 @@ +################################################################################ +# +# Define YAFFS2 filesystems +# +################################################################################ + +YAFFS2_SUBFS = $(if $(BR2_BUILD_SUB_FILESYSTEMS),$(BR2_YAFFS2_SUB_FILESYSTEMS),/)
Signature · 1 line
1.7.9.5
PATCH 1/3 Add the capability to create sub-filesystems from the target filesystem.
Hello David,
Quoted 8 lines · On Wednesday 28 May 2014 22:24:12 vomlehn at canopus-us.com wrote:
For example, say that the the root filesystem has a set of files and directories that are critical to being able to control, recover, and install software on an embedded system. These are redundantly stored in a small but reliable memory. The rest of the system is written in Python, which is huge (too big to fit in the reliable memory) but not a critical part of the system. It can, if necessary, be reloaded remotely, so it is stored without redundancy, in NAND flash.
Quoted 4 lines
This patch allows these two pieces to be split up so that the reliable core can be an initramfs filesystem (in cpio format) and a JFFS2 image can hold the rest of the root filesystem. It supports an arbitrary division of the root filesystem among one or more of the available filesystem formats.
Did you check features of "genimage" tool? It looks like it do similar work. Maybe it would be better to add missing features to genimage?
BR,
Signature · 3 lines
J?r?me Pouiller, Sysmic Embedded Linux specialist http://www.sysmic.fr
PATCH 1/3 Add the capability to create sub-filesystems from the target filesystem.
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20140529/25c57a76/attachment.html>
PATCH 2/3 Allow specification of values for --prefix and --exec-prefix in autotools.
In reply to vomlehn at canopus-us.comFrom: David VomLehn <[email protected]>
The --prefix and --exec-prefix arguments to autotools configuration files allow the installation location to be specified. This is particularly useful when a package must be located in another directory so that it can be split into a separate filesystem image.
--- docs/manual/adding-packages-autotools.txt | 8 ++++++++ package/pkg-autotools.mk | 7 ++++--- 2 files changed, 12 insertions(+), 3 deletions(-)
docs/manual/adding-packages-autotools.txt +8 -0
@@ -99,6 +99,14 @@ cases, typical packages will therefore only use a few of them. * +LIBFOO_CONF_OPT+, to specify additional configure options to pass to the configure script. By default, empty. +* +LIBFOO_CONF_PREFIX+, to specify the installation directory for + architecture-independent files, i.e. the argument to --prefix. + By default, /usr. + +* +LIBFOO_CONF_EXEC_PREFIX+, to specify the installation directory for + architecture-dependent files, i.e. the argument to --exec-prefix. + By default, the value given for +LIBFOO_CONF_PREFIX+. + * +LIBFOO_MAKE+, to specify an alternate +make+ command. This is typically useful when parallel make is enabled in the configuration (using +BR2_JLEVEL+) but that this
package/pkg-autotools.mk +4 -3
@@ -95,7 +95,8 @@ $(2)_AUTORECONF_OPT ?= $($(3)_AUTORECONF_OPT) $(2)_INSTALL_OPT ?= install $(2)_INSTALL_STAGING_OPT ?= DESTDIR=$$(STAGING_DIR) install $(2)_INSTALL_TARGET_OPT ?= DESTDIR=$$(TARGET_DIR) install - +$(2)_CONF_PREFIX ?= /usr +$(2)_CONF_EXEC_PREFIX ?= $$($(2)_CONF_PREFIX) # # Configure step. Only define it if not already defined by the package
@@ -115,8 +116,8 @@ define $(2)_CONFIGURE_CMDS --target=$$(GNU_TARGET_NAME) \ --host=$$(GNU_TARGET_NAME) \ --build=$$(GNU_HOST_NAME) \ - --prefix=/usr \ - --exec-prefix=/usr \ + --prefix=$$($$(PKG)_CONF_PREFIX) \ + --exec-prefix=$$($$(PKG)_CONF_EXEC_PREFIX) \ --sysconfdir=/etc \ --program-prefix="" \ --disable-gtk-doc \
Signature · 1 line
1.7.9.5
PATCH 3/3 Support installation of Python in a different directory.
In reply to vomlehn at canopus-us.comFrom: David VomLehn <[email protected]>
Python is both useful and large. This patch allows installation in a different directory, which can then be installed on its own device by packaging it as part of a separated filesystem.
--- package/python/Config.in | 7 +++++++ package/python/python.mk | 26 ++++++++++++++++---------- 2 files changed, 23 insertions(+), 10 deletions(-)
package/python/Config.in +7 -0
@@ -15,6 +15,13 @@ comment "python needs a toolchain w/ wchar" if BR2_PACKAGE_PYTHON +config BR2_PACKAGE_PYTHON_PREFIX + string "Directory for generated files (executables, libraries, etc.)" + default "/usr" + help + Python is big! Installing Python in a different directory allows + it to be packaged as a separate filesystem. + choice prompt "python module format to install" default BR2_PACKAGE_PYTHON_PYC_ONLY
package/python/python.mk +16 -10
@@ -11,6 +11,9 @@ PYTHON_SITE = http://python.org/ftp/python/$(PYTHON_VERSION) PYTHON_LICENSE = Python software foundation license v2, others PYTHON_LICENSE_FILES = LICENSE +# Relocate to the desired location +PYTHON_CONF_PREFIX = $(BR2_PACKAGE_PYTHON_PREFIX) + # Python needs itself to be built, so in order to cross-compile # Python, we need to build a host Python first. This host Python is # also installed in $(HOST_DIR), as it is needed when cross-compiling
@@ -153,12 +156,12 @@ PYTHON_POST_PATCH_HOOKS += PYTHON_TOUCH_GRAMMAR_FILES # idle & smtpd.py have bad shebangs and are mostly samples # define PYTHON_REMOVE_USELESS_FILES - rm -f $(TARGET_DIR)/usr/bin/idle - rm -f $(TARGET_DIR)/usr/bin/python$(PYTHON_VERSION_MAJOR)-config - rm -f $(TARGET_DIR)/usr/bin/python2-config - rm -f $(TARGET_DIR)/usr/bin/python-config - rm -f $(TARGET_DIR)/usr/bin/smtpd.py - for i in `find $(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/config/ \ + rm -f $(TARGET_DIR)$(PYTHON_CONF_PREFIX)/bin/idle + rm -f $(TARGET_DIR)$(PYTHON_CONF_PREFIX)/bin/python$(PYTHON_VERSION_MAJOR)-config + rm -f $(TARGET_DIR)$(PYTHON_CONF_PREFIX)/bin/python2-config + rm -f $(TARGET_DIR)$(PYTHON_CONF_PREFIX)/bin/python-config + rm -f $(TARGET_DIR)$(PYTHON_CONF_PREFIX)/bin/smtpd.py + for i in `find $(TARGET_DIR)$(PYTHON_CONF_PREFIX)/lib/python$(PYTHON_VERSION_MAJOR)/config/ \ -type f -not -name pyconfig.h -a -not -name Makefile` ; do \ rm -f $$i ; \ done
@@ -170,21 +173,24 @@ PYTHON_POST_INSTALL_TARGET_HOOKS += PYTHON_REMOVE_USELESS_FILES # Make sure libpython gets stripped out on target # define PYTHON_ENSURE_LIBPYTHON_STRIPPED - chmod u+w $(TARGET_DIR)/usr/lib/libpython$(PYTHON_VERSION_MAJOR)*.so + mkdir -p $(TARGET_DIR)$(PYTHON_CONF_PREFIX)/lib + chmod u+w $(TARGET_DIR)$(PYTHON_CONF_PREFIX)/lib/libpython$(PYTHON_VERSION_MAJOR)*.so endef PYTHON_POST_INSTALL_TARGET_HOOKS += PYTHON_ENSURE_LIBPYTHON_STRIPPED # Always install the python symlink in the target tree define PYTHON_INSTALL_TARGET_PYTHON_SYMLINK - ln -sf python2 $(TARGET_DIR)/usr/bin/python + mkdir -p $(TARGET_DIR)$(PYTHON_CONF_PREFIX)/bin + ln -sf python2 $(TARGET_DIR)$(PYTHON_CONF_PREFIX)/bin/python endef PYTHON_POST_INSTALL_TARGET_HOOKS += PYTHON_INSTALL_TARGET_PYTHON_SYMLINK # Always install the python-config symlink in the staging tree define PYTHON_INSTALL_STAGING_PYTHON_CONFIG_SYMLINK - ln -sf python2-config $(STAGING_DIR)/usr/bin/python-config + mkdir -p $(TARGET_DIR)$(PYTHON_CONF_PREFIX)/bin + ln -sf python2-config $(STAGING_DIR)$(PYTHON_CONF_PREFIX)/bin/python-config endef PYTHON_POST_INSTALL_STAGING_HOOKS += PYTHON_INSTALL_STAGING_PYTHON_CONFIG_SYMLINK
@@ -204,7 +210,7 @@ HOST_PYTHON_POST_INSTALL_HOOKS += HOST_PYTHON_INSTALL_PYTHON_SYMLINK endif # Provided to other packages -PYTHON_PATH = $(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/sysconfigdata/:$(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/site-packages/ +PYTHON_PATH = $(TARGET_DIR)$(PYTHON_CONF_PREFIX)/lib/python$(PYTHON_VERSION_MAJOR)/sysconfigdata/:$(TARGET_DIR)$(PYTHON_CONF_PREFIX)/lib/python$(PYTHON_VERSION_MAJOR)/site-packages/ $(eval $(autotools-package)) $(eval $(host-autotools-package))
Signature · 1 line
1.7.9.5
PATCH 3/3 Support installation of Python in a different directory.
Hello,
Quoted 11 lines · vomlehn at canopus-us.com
From: David VomLehn <[email protected]>
Python is both useful and large. This patch allows installation in a different directory, which can then be installed on its own device by packaging it as part of a separated filesystem.
Signed-off-by David VomLehn <[email protected]>--- package/python/Config.in | 7 +++++++ package/python/python.mk | 26 ++++++++++++++++---------- 2 files changed, 23 insertions(+), 10 deletions(-)
The problem with this approach is that we would have to replicate this in *all* packages. You are personally interested by changing the installation location of Python. But I might be interested in changing the installation location of Qt. And then the next guy the installation location of X.org, and so on and so on.
That's a limitation of Kconfig: there is no way of creating "properties" that would apply to all packages without creating one option for each package. We have the same issue to define which package should be built statically or dynamically, which package should be built with debug symbols or not, and so on.
So I would prefer to see such advanced customization done by means of the BR2_PACKAGE_OVERRIDE_FILE option. It would avoid cluttering the Config.in files of packages with lots and lots of options.
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
cover Manage big root directory trees by breaking them up.
In reply to vomlehn at canopus-us.comQuoted 1 line · vomlehn at canopus-us.com
From: David VomLehn <[email protected]>
Hi David,
We've discussed this series at the Buildroot developer meeting, and we decided
to reject it. This is why:
Quoted 7 lines
These patches are not as tightly coupled as the usually series of patches, but I think the pieces make more sense when viewed in the context of the whole patchset. The problem to be solved is that Python is big and my root filesystem has to fit in a little flash device. So, I:
1. Created the ability to break the target filesystem into multiple
filesystems,
Although we do want to support the possibility to break down the target
filesystem into partitions, we don't want to add so many config options to do it. Basically, we want to keep buildroot as simple as possible. So instead, we want to provide the tools that are required to do this, like genpart, and then allow the user to do it in a post-build and post-image script.
Quoted 6 lines
2. Enhanced autotools so that I could specify the --prefix and
--exec-prefix values. This allowed me to install packages in their own directory, which the above patch lets me split into a filesystem I can store on a much bigger flash device, and
3. I modified the Python package so that the installation directory
is a configurable.
The problem with this is that it adds a user-visible feature that is only
useful for autotools packages. It is a bit similar to FOO_OVERRIDE_SRCDIR, but for that one we can guarantee that it works for all packages. The CONF_PREFIX only works for autotools packages and for python, we would need many many more changes to have it for all possible packages.
In addition, this is something that will break down easily, because it is not
tested. E.g. if a .mk file installs 10 files, and we forget to add the CONF_PREFIX for one of them, that's easy to miss.
Therefore we decided to reject these two patches as well.
That said, thank you for your contribution and thank you for using buildroot!
Regards,
Arnout
[snip]
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F