Pull request for branch yem/genimages
Subsystems (2)
From: "Yann E. MORIN" <[email protected]>
Hello All!
During the last developpers' day in Edimburgh, we discussed my Buildroot.config project, and especially the 'genimages' infrastructure I've done in there.
I've came up with 'genimages' after I came to the conclusion that the current images generation in Buildroot is limited to generating a single filesystem images, but most device do require a more complex setup.
For example. the Rapsberry Pi needs an SDcard (mmcblk0) formatted as thus:
mmcblk0
|- mmcblk0p1 (FAT32)
| |- bootcode.bin
| |- fixup.dat
| |- start.elf
| |- config.txt
| |- cmdline.txt
| `- kernel.bin
`- mmcblk0p2 (whatever, eg. ext3, btrfs...)
`- all of /With the current output of Buildroot, that requires a lot of fiddling with the partitioning of the SDcard (may require root), formatting the filesystems (may require root), extract rootfs.tar (may require root to mount, requires root to extract), copy select files from output/images/rpi-firmware/ to the first partition (may require root to mount).
Each user would either have to manually perform those actions (error prone), or write a small shell script to automate this (not trivial).
Also, some other devices require special, but different formatting as well.
Rather than have every users all suffer in silence (and sometimes not in silence) by re-inventing such a script over and over again, for each device that need such a setup, I came up with the 'genimages' idea: a description of the layout of the storage devices, partitioning schemes, and partitions content.
So, here is the 'genimages' from Buildroot.config, that I updated (in fact, trimmed) to include in Buildroot itself.
The basis is that we bundle some trivial partition description for the boards we have in board/ and reference it in the corresponding defconfigs.
How does 'genimages' work?
First, genimages extract the generated rootfs.tar. Hence, rootfs.tar is always generated if a partition description is configured.
Second, genimages generates one filesystem image for each partition listed in the description. Once the filesystem is generated, its mountpoint is emptied, so it does not appear in the upper filesystem.
Third, genimages aggregates all the partition of a device into a single image file
Finally, those device images can be flashed directly on the raw device of the storage, without manually fiddling with fdisk, mount, tar and so on...
How does 'genimages' work internally?
The partition layout is decribed in a .ini-like file.
genimages is a relatively simple shell script that parses that .ini file. Based on each device/partition description, it calls to 'handlers' (or hooks), for each kind of content we support.
For example, we have two types of partitioning scheme supported for
device: mbr and gpt. Each is implemented in its own shell-fragment. Ditto for partitions: we support different filesystems, ext, squashfs, and vfat; they are each implemented in their own shell-fragment.
Each handler has to define the 'do_image' shell function. That functions should expect two arguments: the base dir of the filesystem, and the filename of the image to generate, and it has access to the configuration from the .ini file.
How to use 'genimages'?
It's all explained in the manual. A pre-rendered version is available for reading, and easy review on-line there:
http://ymorin.is-a-geek.org/download/tmp/buildroot/manual/manual.html
The new section is "3.3.6 Customizing the generated filesystem images" and there is a detailed description of the partition table layout in appendix "12.3. Partition table layout description syntax".
Changes v2 -> v3:
- misc typoes in the manual
Changes v1 -> v2:
- drop rpi-firmware install into /boot (Thomas, Ryan)
- drop move of rpi-firmware in bootloader sub-menu (Thomas)
- add possibility to generate an unmounted filesystem (Thomas, Ryan)
- add ability to specify files to add in a filesystem instead of a sub-dir of target/ (Ryan, during a hacking session)
- properly depend on the host-tools if they are selected
- add GPT support
- add a third filesystem, squashfs (in its own patch, as an example of how easy it is to add one)
Regards,
Yann E. MORIN.
The following changes since commit 2746158497cdaf80a5b85584e3972857ac2e7e82:
util-linux: Add config switches for some more binaries (2013-12-20 22:45:31 +0100)
are available in the git repository at:
git://gitorious.org/buildroot/buildroot.git yem/genimages
for you to fetch changes up to 81062cef5d4d51b6f196fb7c3d9c171c7c362555:
fs/custom: add support for squashfs (2013-12-20 23:12:22 +0100)
----------------------------------------------------------------
Yann E. MORIN (4):
fs/custom: generate complete, partition-based device images board/raspberrypi: provide partition description for the new genimages package/squashfs: add selection for the host variant fs/custom: add support for squashfs
board/raspberrypi/partitions | 36 ++++ configs/raspberrypi_defconfig | 8 + docs/manual/appendix.txt | 1 + docs/manual/customize-filesystems.txt | 35 ++++ docs/manual/customize.txt | 2 + docs/manual/partition-layout.txt | 323 ++++++++++++++++++++++++++++++++++ fs/Config.in | 1 + fs/custom/Config.in | 16 ++ fs/custom/boot/gpt | 126 +++++++++++++ fs/custom/boot/mbr | 57 ++++++ fs/custom/boot/pre-post | 8 + fs/custom/custom.mk | 41 +++++ fs/custom/fs/ext | 22 +++ fs/custom/fs/pre-post | 67 +++++++ fs/custom/fs/squashfs | 15 ++ fs/custom/fs/vfat | 17 ++ fs/custom/functions | 47 +++++ fs/custom/genimages | 242 +++++++++++++++++++++++++ package/Config.in.host | 1 + package/squashfs/Config.in.host | 6 + 20 files changed, 1071 insertions(+) create mode 100644 board/raspberrypi/partitions create mode 100644 docs/manual/customize-filesystems.txt create mode 100644 docs/manual/partition-layout.txt create mode 100644 fs/custom/Config.in create mode 100644 fs/custom/boot/gpt create mode 100644 fs/custom/boot/mbr create mode 100644 fs/custom/boot/pre-post create mode 100644 fs/custom/custom.mk create mode 100644 fs/custom/fs/ext create mode 100644 fs/custom/fs/pre-post create mode 100644 fs/custom/fs/squashfs create mode 100644 fs/custom/fs/vfat create mode 100644 fs/custom/functions create mode 100755 fs/custom/genimages create mode 100644 package/squashfs/Config.in.host
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/4 fs/custom: generate complete, partition-based device images
From: "Yann E. MORIN" <[email protected]>
Contrary to the existing fs/ schemes, which each generate only a single filesystem image for the root filesystem, this new scheme allows the user to generate more complex images.
The basis behind this is a .ini-like description of the layout of the final target storage:
- the list of device(s)
- per-device, the list of partition(s)
- per-partition, the content
It is possible to create MBR- or GPT-based partitoining schemes. Adding new ones should be relatively easy (but would need adequate host tools).
For now, the only content possible for partitions is a filesystem. It should be pretty easy to add new types (eg. un-formated, or raw blob).
Also, only two filesystems are supported: ext{2,3,4} and vfat. Adding more will be relatively easy, provided we have the necessary host packages to deal with those filesystems.
The existing Buildroot filesystem generators are re-used as much as possible when it makes sense; when it does not (eg. for vfat), a specific generator is used.
--- docs/manual/appendix.txt | 1 + docs/manual/customize-filesystems.txt | 35 ++++ docs/manual/customize.txt | 2 + docs/manual/partition-layout.txt | 314 ++++++++++++++++++++++++++++++++++ fs/Config.in | 1 + fs/custom/Config.in | 16 ++ fs/custom/boot/gpt | 126 ++++++++++++++ fs/custom/boot/mbr | 57 ++++++ fs/custom/boot/pre-post | 8 + fs/custom/custom.mk | 38 ++++ fs/custom/fs/ext | 22 +++ fs/custom/fs/pre-post | 67 ++++++++ fs/custom/fs/vfat | 17 ++ fs/custom/functions | 47 +++++ fs/custom/genimages | 242 ++++++++++++++++++++++++++ 15 files changed, 993 insertions(+) create mode 100644 docs/manual/customize-filesystems.txt create mode 100644 docs/manual/partition-layout.txt create mode 100644 fs/custom/Config.in create mode 100644 fs/custom/boot/gpt create mode 100644 fs/custom/boot/mbr create mode 100644 fs/custom/boot/pre-post create mode 100644 fs/custom/custom.mk create mode 100644 fs/custom/fs/ext create mode 100644 fs/custom/fs/pre-post create mode 100644 fs/custom/fs/vfat create mode 100644 fs/custom/functions create mode 100755 fs/custom/genimages
docs/manual/appendix.txt +1 -0
@@ -6,6 +6,7 @@ Appendix include::makedev-syntax.txt[] include::makeusers-syntax.txt[] +include::partition-layout.txt[] // Automatically generated lists:
docs/manual/customize-filesystems.txt new file +35 -0
@@ -0,0 +1,35 @@ +// -*- mode:doc; -*- +// vim: set syntax=asciidoc: + +[[filesystem-custom]] +Customizing the generated filesystem images +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + ++Buildroot+ knows by default how to generate a few different kind of +filesystems, such as +squashfs+, +ext2/3/4+, +cramfs+... But those +filesystems are all generated to contain the complete target directory +hierarchy in a single filesystem, mounted as the root filesystem +/+. +That is, even if you select both an +ext2+ and a +squashfs+ filesystems, +the content of the two generated images will be the exact same, only the +types of the filesystems will be different. + +Most devices require a more complex setup, with different parts of the +directory structure split across different filesystems, each stored on +different partitions of one or more storage devices. + ++Buildroot+ can generate such complex setups, using a +partition table layout +description+. This is a simple text file, not unlike the +.ini+ style of +configuration files, that describes how the target directory hierarchy has +to be split across the target storage devices. It is a bit like a flattened +tree of the storage layout. + +Set the variable +BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE+ to the path of +the file containing your +partition table layout description+. + +See xref:part-layout-desc-syntax[] for the complete documentation of the ++partition table layout description+ syntax. + +[underline]*Note:* Although more versatile than the single filesystem image +mechanism, the +partition table layout description+ might be unable to +describe very complex setups. For example, it is not capable of handling ++initramfs+ based systems, or NFS-mounted filesystems.
docs/manual/customize.txt +2 -0
@@ -14,6 +14,8 @@ include::customize-kernel-config.txt[] include::customize-toolchain.txt[] +include::customize-filesystems.txt[] + include::customize-store.txt[] include::customize-packages.txt[]
docs/manual/partition-layout.txt new file +314 -0
314 lines not on this page · raw message
fs/Config.in +1 -0
@@ -3,6 +3,7 @@ menu "Filesystem images" source "fs/cloop/Config.in" source "fs/cpio/Config.in" source "fs/cramfs/Config.in" +source "fs/custom/Config.in" source "fs/ext2/Config.in" source "fs/initramfs/Config.in" source "fs/iso9660/Config.in"
fs/custom/Config.in new file +16 -0
@@ -0,0 +1,16 @@ +config BR2_TARGET_ROOTFS_CUSTOM + bool "Custom partition table layout" + select BR2_TARGET_ROOTFS_TAR + +config BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE + string "path to the custom partition table layout description" + depends on BR2_TARGET_ROOTFS_CUSTOM + help + Enter the path to a partition-table for your device. + + This will allow Buildroot to generate a more complex target + image, which may consist of more than one filesystem on more + than one partition. + + See docs/manual/bla-bla on how to construct such a partition + table.
fs/custom/boot/gpt new file +126 -0
126 lines not on this page · raw message
fs/custom/boot/mbr new file +57 -0
57 lines not on this page · raw message
fs/custom/boot/pre-post new file +8 -0
@@ -0,0 +1,8 @@ +do_image_pre() { + : +} +do_image_post() { + : +} + +#vim: set ft=sh
fs/custom/custom.mk new file +38 -0
@@ -0,0 +1,38 @@ +################################################################################ +# +# custom partitioning +# +################################################################################ + +define ROOTFS_CUSTOM_CMD + BUILD_DIR=$(BUILD_DIR) fs/custom/genimages \ + '$(call qstrip,$(BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE))' +endef + +# rootfs-custom uses rootfs.tar as the source to generate +# the resulting image(s), so we need to build it first. +ROOTFS_CUSTOM_DEPENDENCIES += rootfs-tar + +# All of the following filesystem generators, or partition managers, are +# optional, but if they are selected, we may need them, so we need to +# depend on them +ifeq ($(BR2_PACKAGE_HOST_DOSFSTOOLS),y) +ROOTFS_CUSTOM_DEPENDENCIES += host-dosfstools +endif +ifeq ($(BR2_PACKAGE_HOST_E2FSPROGS),y) +ROOTFS_CUSTOM_DEPENDENCIES += host-e2fsprogs +endif +ifeq ($(BR2_PACKAGE_HOST_GENEXT2FS),y) +ROOTFS_CUSTOM_DEPENDENCIES += host-genext2fs +endif +ifeq ($(BR2_PACKAGE_HOST_GENPART),y) +ROOTFS_CUSTOM_DEPENDENCIES += host-genpart +endif +ifeq ($(BR2_PACKAGE_HOST_MTOOLS),y) +ROOTFS_CUSTOM_DEPENDENCIES += host-mtools +endif +ifeq ($(BR2_PACKAGE_HOST_PARTED),y) +ROOTFS_CUSTOM_DEPENDENCIES += host-parted +endif + +$(eval $(call ROOTFS_TARGET,custom))
fs/custom/fs/ext new file +22 -0
@@ -0,0 +1,22 @@ +# Create an extended file system + +do_image() { + local root_dir="${1}" + local img="${2}" + local -a fs_opts + local gen rev + + fs_opts+=( -z ) + fs_opts+=( -d "${root_dir}" ) + [ -z "${size}" ] || fs_opts+=( -b $((size/1024)) ) + [ -n "${ext_gen}" ] || ext_gen=2 + [ -n "${ext_rev}" ] || ext_rev=1 + + # Remember, we're running from Buildroot's TOP_DIR + GEN=${ext_gen} REV=${ext_rev} \ + ./fs/ext2/genext2fs.sh "${fs_opts[@]}" "${img}" >/dev/null + + [ -z "${fs_label}" ] || tune2fs -L "${fs_label}" "${img}" >/dev/null +} + +# vim: ft=sh
fs/custom/fs/pre-post new file +67 -0
67 lines not on this page · raw message
fs/custom/fs/vfat new file +17 -0
@@ -0,0 +1,17 @@ +# Create a VFAT file system + +do_image() { + local root_dir="${1}" + local img="${2}" + local -a fs_opts + + dd if=/dev/zero of="${img}" bs=${size} count=0 seek=1 2>/dev/null + + [ -z "${vfat_size}" ] || fs_opts+=( -F ${vfat_size} ) + [ -z "${fs_label}" ] || fs_opts+=( -n "${fs_label}" ) + mkfs.vfat "${fs_opts[@]}" "${img}" >/dev/null + + mcopy -i "${img}" "${root_dir}/"* '::' +} + +# vim: ft=sh
fs/custom/functions new file +47 -0
47 lines not on this page · raw message
fs/custom/genimages new file +242 -0
242 lines not on this page · raw message
Signature · 1 line
1.8.1.2
PATCH 1/4 fs/custom: generate complete, partition-based device images
Dear Yann E. MORIN,
Quoted 5 lines · Yann E. MORIN
docs/manual/partition-layout.txt new file +5
-0+Requirements on host packages +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Depending on what types of partitioning and filesystems are used by your ++partition table layout description+, you may have to enable some
docs/manual/partition-layout.txt line 214
Quoted 10 lines
docs/manual/partition-layout.txt new file +10
-0+host-packages in the +Host utilities+ sub-menu. + +Since the content of a +partition table layout description+ is very +specific to a board and/or a project, there is no way for Buildroot to +automatically select those host packages, and thus it is your +responsibility to select the appropriate ones. + +For example, for an MBR partitioning, you will have to enable the +host +genpart+ package. For FAT filesystems, you will have to enable both of ++host dosfstools+ and +host mtools+.
I'll have other comments about the genimages stuff later on, but I got an idea this night about this specific problem. I don't really like the fact that the user has to manually enable in menuconfig the relevant host tools, depending on his genimage .ini configuration file.
What about doing something like:
ROOTFS_CUSTOM_DEPENDENCIES = $(shell fs/custom/genimages --show-depends \ '$(call qstrip,$(BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE))'
The --show-depends option of genimages would, instead of doing the normal genimages work, simply parse the .ini file, and return a space separated list of dependencies needed by the ROOTFS_CUSTOM stuff. Not only this would make the dependencies automatic for the user, but also it would allow you to remove the following piece of code from the custom.mk file:
+# All of the following filesystem generators, or partition managers, are +# optional, but if they are selected, we may need them, so we need to +# depend on them
+ifeq ($(BR2_PACKAGE_HOST_DOSFSTOOLS),y) +ROOTFS_CUSTOM_DEPENDENCIES += host-dosfstools
+endif
+ifeq ($(BR2_PACKAGE_HOST_E2FSPROGS),y) +ROOTFS_CUSTOM_DEPENDENCIES += host-e2fsprogs
+endif
+ifeq ($(BR2_PACKAGE_HOST_GENEXT2FS),y) +ROOTFS_CUSTOM_DEPENDENCIES += host-genext2fs
+endif
+ifeq ($(BR2_PACKAGE_HOST_GENPART),y) +ROOTFS_CUSTOM_DEPENDENCIES += host-genpart
+endif
+ifeq ($(BR2_PACKAGE_HOST_MTOOLS),y) +ROOTFS_CUSTOM_DEPENDENCIES += host-mtools
+endif
+ifeq ($(BR2_PACKAGE_HOST_PARTED),y) +ROOTFS_CUSTOM_DEPENDENCIES += host-parted
+endif
What do you think?
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
PATCH 1/4 fs/custom: generate complete, partition-based device images
Quoted 31 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Thomas" == Thomas Petazzoni <[email protected]> writes:
Dear Yann E. MORIN,
Quoted 5 lines · Yann E. MORIN
docs/manual/partition-layout.txt new file +5
-0+Requirements on host packages +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Depending on what types of partitioning and filesystems are used by your ++partition table layout description+, you may have to enable somedocs/manual/partition-layout.txt line 214
Quoted 10 lines
docs/manual/partition-layout.txt new file +10
-0+host-packages in the +Host utilities+ sub-menu. + +Since the content of a +partition table layout description+ is very +specific to a board and/or a project, there is no way for Buildroot to +automatically select those host packages, and thus it is your +responsibility to select the appropriate ones. + +For example, for an MBR partitioning, you will have to enable the +host +genpart+ package. For FAT filesystems, you will have to enable both of ++host dosfstools+ and +host mtools+.I'll have other comments about the genimages stuff later on, but I got an idea this night about this specific problem. I don't really like the fact that the user has to manually enable in menuconfig the relevant host tools, depending on his genimage .ini configuration file.
What about doing something like:
ROOTFS_CUSTOM_DEPENDENCIES = $(shell fs/custom/genimages --show-depends \ '$(call qstrip,$(BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE))'
Quoted 6 lines
The --show-depends option of genimages would, instead of doing the normal genimages work, simply parse the .ini file, and return a space separated list of dependencies needed by the
ROOTFS_CUSTOMstuff. Not only this would make the dependencies automatic for the user, but also it would allow you to remove the following piece of code from the custom.mk file:
Sounds good to me. They won't actumatically get selected in kconfig though, but that might not be such a big deal (do they still even need to be visible in kconfig?)
Signature · 1 line
Bye, Peter Korsgaard
PATCH 1/4 fs/custom: generate complete, partition-based device images
Thomas, Peter, All,
On 2013-12-21 17:40 +0100, Peter Korsgaard spake thusly:
Quoted 31 lines
Quoted 30 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Thomas" == Thomas Petazzoni <[email protected]> writes:
Dear Yann E. MORIN,
Quoted 5 lines · Yann E. MORIN
docs/manual/partition-layout.txt new file +5
-0+Requirements on host packages +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Depending on what types of partitioning and filesystems are used by your ++partition table layout description+, you may have to enable somedocs/manual/partition-layout.txt line 214
Quoted 10 lines
docs/manual/partition-layout.txt new file +10
-0+host-packages in the +Host utilities+ sub-menu. + +Since the content of a +partition table layout description+ is very +specific to a board and/or a project, there is no way for Buildroot to +automatically select those host packages, and thus it is your +responsibility to select the appropriate ones. + +For example, for an MBR partitioning, you will have to enable the +host +genpart+ package. For FAT filesystems, you will have to enable both of ++host dosfstools+ and +host mtools+.I'll have other comments about the genimages stuff later on, but I got an idea this night about this specific problem. I don't really like the fact that the user has to manually enable in menuconfig the relevant host tools, depending on his genimage .ini configuration file.
What about doing something like:
ROOTFS_CUSTOM_DEPENDENCIES = $(shell fs/custom/genimages --show-depends \ '$(call qstrip,$(BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE))'
Quoted 8 lines
Quoted 6 lines
The --show-depends option of genimages would, instead of doing the normal genimages work, simply parse the .ini file, and return a space separated list of dependencies needed by the
ROOTFS_CUSTOMstuff. Not only this would make the dependencies automatic for the user, but also it would allow you to remove the following piece of code from the custom.mk file:Sounds good to me.
Yep, I've implemented that now, and it seems to work pretty well.
Quoted 3 lines
They won't actumatically get selected in kconfig
though, but that might not be such a big deal (do they still even need to be visible in kconfig?)
Yes, in case someone wants to generate filesystem images with a post-build script, and not with genimages.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/4 fs/custom: generate complete, partition-based device images
In reply to Peter KorsgaardDear Peter Korsgaard,
Quoted 10 lines · Peter Korsgaard
Quoted 6 lines
The --show-depends option of genimages would, instead of doing the normal genimages work, simply parse the .ini file, and return a space separated list of dependencies needed by the
ROOTFS_CUSTOMstuff. Not only this would make the dependencies automatic for the user, but also it would allow you to remove the following piece of code from the custom.mk file:Sounds good to me. They won't actumatically get selected in kconfig though, but that might not be such a big deal (do they still even need to be visible in kconfig?)
I believe we have existing occurrences of host packages being listed in some <pkg>_DEPENDENCIES, but not necessarily selected at the Kconfig level. While we have enforced this not to happen for target packages, we've never enforced that for host packages.
The rationale is that host packages should normally be completely transparent to the user, and therefore not visible in Kconfig. The host package sub-menu is here only to make it easier for the user to enable *some* of the host packages that are actually relevant for the user (and not merely build dependencies of other packages).
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
PATCH 1/4 fs/custom: generate complete, partition-based device images
In reply to Thomas PetazzoniThomas, All,
On 2013-12-21 09:49 +0100, Thomas Petazzoni spake thusly:
Quoted 11 lines · Yann E. MORIN
Quoted 10 lines
docs/manual/partition-layout.txt new file +9
-0+Requirements on host packages +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Depending on what types of partitioning and filesystems are used by your ++partition table layout description+, you may have to enable some +host-packages in the +Host utilities+ sub-menu. + +Since the content of a +partition table layout description+ is very +specific to a board and/or a project, there is no way for Buildroot to
Quoted 8 lines
docs/manual/partition-layout.txt line 214
Quoted 6 lines
docs/manual/partition-layout.txt new file +6
-0+automatically select those host packages, and thus it is your +responsibility to select the appropriate ones. + +For example, for an MBR partitioning, you will have to enable the +host +genpart+ package. For FAT filesystems, you will have to enable both of ++host dosfstools+ and +host mtools+.I'll have other comments about the genimages stuff later on,
Hey! You can't let us all waiting like that! Speak up, now! ;-)
Quoted 1 line
but I got an idea this night
I don't know what you usually do at night, but I, for one, sleep. ;-)
Quoted 8 lines
about this specific problem. I don't really like the
fact that the user has to manually enable in menuconfig the relevant host tools, depending on his genimage .ini configuration file.What about doing something like:
ROOTFS_CUSTOM_DEPENDENCIES = $(shell fs/custom/genimages --show-depends \ '$(call qstrip,$(BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE))'
[--SNIP--]
Quoted 1 line
What do you think?
I was initially thinking about something like it, but found it a bit dubious. But since you suggested it (and now Peter concurred), I have now (re-)implemented it, and it works nicely.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 2/4 board/raspberrypi: provide partition description for the new genimages
In reply to Yann E. MORINFrom: "Yann E. MORIN" <[email protected]>
Now we can generate a complete target storage image with the genimages infra, add a partition table layout description for the Raspberry Pi as an example for how to use genimages.
--- board/raspberrypi/partitions | 36 ++++++++++++++++++++++++++++++++++++ configs/raspberrypi_defconfig | 8 ++++++++ 2 files changed, 44 insertions(+) create mode 100644 board/raspberrypi/partitions
board/raspberrypi/partitions new file +36 -0
@@ -0,0 +1,36 @@ +[global] +extract=tar +devices=mmcblk0 + +[mmcblk0] +type=boot +boot_type=mbr +partitions=boot,root +partalign=$((1048576)) + +[root] +type=fs +size=$((32*1048576)) +mbr_type=$((0x83)) +fs_type=ext +fs_vfstype=ext4 +fs_mntops=discard +fs_root=/ +fs_label=ROOT +ext_gen=4 +ext_rev=1 + +[boot] +type=fs +# FAT32 is not possible under 9MiB +size=$((9*1048576)) +mbr_type=$((0xc)) +fs_type=vfat +fs_files_0=rpi-firmware/bootcode.bin +fs_files_1=rpi-firmware/start.elf +fs_files_2=rpi-firmware/fixup.dat +fs_files_3=rpi-firmware/config.txt +fs_files_4=rpi-firmware/cmdline.txt +fs_files_5=zImage +fs_label=BOOT +vfat_size=16
configs/raspberrypi_defconfig +8 -0
@@ -21,3 +21,11 @@ BR2_LINUX_KERNEL_CUSTOM_REPO_VERSION="5ebf1db80e2c03e92d570605b53f91a06112574f" BR2_LINUX_KERNEL_USE_DEFCONFIG=y BR2_LINUX_KERNEL_DEFCONFIG="bcmrpi_quick" BR2_LINUX_KERNEL_ZIMAGE=y + +BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE="$(TOPDIR)/board/raspberrypi/partitions" + +BR2_PACKAGE_HOST_DOSFSTOOLS=y +BR2_PACKAGE_HOST_E2FSPROGS=y +BR2_PACKAGE_HOST_GENEXT2FS=y +BR2_PACKAGE_HOST_GENPART=y +BR2_PACKAGE_HOST_MTOOLS=y
Signature · 1 line
1.8.1.2
PATCH 3/4 package/squashfs: add selection for the host variant
In reply to Yann E. MORINFrom: "Yann E. MORIN" <[email protected]>
If any of the post-image scripts, or genimages, wants to handle squashfs filesystems, we need to expose an option for squashfs-tools to be user-selectable.
--- package/Config.in.host | 1 + package/squashfs/Config.in.host | 6 ++++++ 2 files changed, 7 insertions(+) create mode 100644 package/squashfs/Config.in.host
package/Config.in.host +1 -0
@@ -12,6 +12,7 @@ source "package/omap-u-boot-utils/Config.in.host" source "package/openocd/Config.in.host" source "package/parted/Config.in.host" source "package/sam-ba/Config.in.host" +source "package/squashfs/Config.in.host" source "package/sunxi-tools/Config.in.host" source "package/uboot-tools/Config.in.host"
package/squashfs/Config.in.host new file +6 -0
@@ -0,0 +1,6 @@ +config BR2_PACKAGE_HOST_SQUASHFS + bool "host squashfs" + help + Tools to generate SquashFS filesystems. + + http://squashfs.sourceforge.net/
Signature · 1 line
1.8.1.2
PATCH 4/4 fs/custom: add support for squashfs
In reply to Yann E. MORINFrom: "Yann E. MORIN" <[email protected]>
--- docs/manual/partition-layout.txt | 11 ++++++++++- fs/custom/custom.mk | 3 +++ fs/custom/fs/squashfs | 15 +++++++++++++++ 3 files changed, 28 insertions(+), 1 deletion(-) create mode 100644 fs/custom/fs/squashfs
docs/manual/partition-layout.txt +10 -1
@@ -133,7 +133,8 @@ Properties for +type=fs+ * +fs_type+ (mandatory): the type of filesystem to generate ** +ext+: generate an extended filesystem (ext2, ext3, ext4) -** +vfat+: generate a VFAT filesystem (FAT16, FAT32) +** +squashfs+: generate a squashfs (version 4) filesystem +** +vfat+: generate a VFAT filesystem (FAT12, FAT16 or FAT32) * +fs_label+ (optional): the label to assign to this filesystem, if that filesystem supports a label
@@ -177,6 +178,14 @@ Properties for +fs_type=ext+ ** +1+ (mandatory for ext3 or ext4): generate a revision 1 extended filesystem +Properties for +fs_type=squashfs+ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +* +squashfs_comp+ (mandatory): the compression type to use +** +gzip+, +xz+, +lzo+: gzip, xz or lzo compression + +* +squashfs_block+ (optional): the size of blocks, in bytes + Properties for +fs_type=vfat+ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
fs/custom/custom.mk +3 -0
@@ -34,5 +34,8 @@ endif ifeq ($(BR2_PACKAGE_HOST_PARTED),y) ROOTFS_CUSTOM_DEPENDENCIES += host-parted endif +ifeq ($(BR2_PACKAGE_HOST_SQUASHFS),y) +ROOTFS_CUSTOM_DEPENDENCIES += host-squashfs +endif $(eval $(call ROOTFS_TARGET,custom))
fs/custom/fs/squashfs new file +15 -0
@@ -0,0 +1,15 @@ +# Create a squashfs filesystem + +do_image() { + local root_dir="${1}" + local img="${2}" + local -a fs_opts + + fs_opts+=( -no-progress -noappend ) + [ -z "${squashfs_comp}" ] || fs_opts+=( -comp "${squashfs_comp}" ) + [ -z "${squashfs_block}" ] || fs_opts+=( -b ${squahsfs_block} ) + + mksquashfs "${root_dir}" "${img}" "${fs_opts[@]}" >/dev/null +} + +# vim: ft=sh
Signature · 1 line
1.8.1.2