Skip to contentSkip to conversation

rpi: image generation

Buildroot New Benoît Thébaudeau 9 participants 34 messages Aug 19, 2015 – Oct 26, 2015 1 message not in the archive Broadcom BCM2711/BCM2835 Arm architecture

lore
Share & tools
lore
#0 lore raw

Hi all,

I have local changes for the Raspberry Pi that can automatically generate an image to be flashed on the SD card (e.g. with dd).

This mechanism uses genimage, so no sudo is needed. It generates a FAT32 partition of 64 MiB and an ext4 partition that just fits its contents with a small free space. I also have scripts to auto-expand the ext4 partition and FS to fill the SD card.

Are you interested in any of this, or do you prefer to keep the current manual approach that makes it possible to do just anything custom?

Best regards,
Beno?t

#1 lore raw

Dear Beno?t Th?baudeau,

Quoted 10 lines · Beno?t Th?baudeau

I have local changes for the Raspberry Pi that can automatically generate an image to be flashed on the SD card (e.g. with dd).

This mechanism uses genimage, so no sudo is needed. It generates a FAT32 partition of 64 MiB and an ext4 partition that just fits its contents with a small free space. I also have scripts to auto-expand the ext4 partition and FS to fill the SD card.

Are you interested in any of this, or do you prefer to keep the current manual approach that makes it possible to do just anything custom?

I am interested in seeing more examples of genimage usage. I believe Vivien Didelot, who submitted the patch to use genimage for the Wandboard configuration, was also planning on submitting more genimage related patches.

Best regards,

Thomas Petazzoni

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#2 lore raw

Hi Beno?t, Thomas,

Quoted 12 lines · Thomas Petazzoni

Dear Beno?t Th?baudeau,

Quoted 8 lines · Beno?t Th?baudeau

I have local changes for the Raspberry Pi that can automatically generate an image to be flashed on the SD card (e.g. with dd).

This mechanism uses genimage, so no sudo is needed. It generates a FAT32 partition of 64 MiB and an ext4 partition that just fits its contents with a small free space. I also have scripts to auto-expand the ext4 partition and FS to fill the SD card.

Quoted 8 lines
Quoted 3 lines

Are you interested in any of this, or do you prefer to keep the current manual approach that makes it possible to do just anything custom?

I am interested in seeing more examples of genimage usage. I believe Vivien Didelot, who submitted the patch to use genimage for the Wandboard configuration, was also planning on submitting more genimage related patches.

Indeed, I initial had genimage configs for Rpi(s), Wanboard and BeagleBone. I didn't find the time yet to rebase all that.

Similar to the upstreamed Wandboard config (7fce297), the RPi(s) config looks like this: https://gist.github.com/vivien/752f85b563b02afd772c

Note, this was based on 8d00fe4. With some new upstreamed patches for RPi and DT, the patch 1/2 might not be relevant anymore.

Feel free to rebase mine or update yours. If you submit something, please put me in Cc, I'd be happy to review and test any of your patch.

Best,
-v

#3 lore raw

PATCH 1/4 configs/raspberrypi: use EABIhf

The SoC of the Raspberry Pi 1 is an ARM1176JZF-S, which features a VFPv2 FPU, so use the EABIhf target ABI, which is more efficient than EABI. This is also the default on Raspbian.

---
 configs/raspberrypi_defconfig | 1 +
 1 file changed, 1 insertion(+)
configs/raspberrypi_defconfig +1 -0
index af702a3..2d47058 100644
@@ -1,5 +1,6 @@
 BR2_arm=y
 BR2_arm1176jzf_s=y
+BR2_ARM_EABIHF=y
 
 BR2_TARGET_GENERIC_GETTY_PORT="tty1"
 
Signature · 1 line
2.1.4
#4 lore raw

PATCH 2/4 board/raspberrypi: prepare the image files for the target

Automate the preparation of the generated image files for the Raspberry Pi firmware:
- mark the kernel image as supporting Device Tree,
- place the marked kernel image and the DTBs into the rpi-firmware folder.

The instructions in readme.txt are updated accordingly.

---
 board/raspberrypi/post-image.sh | 17 ++++++++
 board/raspberrypi/readme.txt    | 90 ++++++++++++++++++-----------------------
 configs/raspberrypi2_defconfig  |  2 +
 configs/raspberrypi_defconfig   |  2 +
 4 files changed, 60 insertions(+), 51 deletions(-)
 create mode 100755 board/raspberrypi/post-image.sh
board/raspberrypi/post-image.sh new file +17 -0
new file mode 100755
index 0000000..db9c53c
@@ -0,0 +1,17 @@
+#!/bin/bash -e
+
+RPI_FW_BIN_DIR="${BINARIES_DIR}/rpi-firmware"
+KERNEL_IMG="${BINARIES_DIR}/zImage"
+KERNEL_RPI_FW_IMG="${RPI_FW_BIN_DIR}/zImage"
+
+# A special marker must be appended to the kernel image so that the firmware of
+# the Raspberry Pi knows that it supports Device Tree. Without this marker, the
+# firmware passes the ATAGS boot data instead of the appropriate DTB to the
+# kernel.
+# The final kernel image must be in the rpi-firmware folder.
+mkknlimg "${KERNEL_IMG}" "${KERNEL_RPI_FW_IMG}"
+
+# Move the DTBs where the firmware expects them to be, i.e. to the rpi-firmware
+# folder.
+find "${BINARIES_DIR}/" -maxdepth 1 -type f -name '*.dtb' -print0 | \
+	xargs -0r mv -ft "${RPI_FW_BIN_DIR}/"
board/raspberrypi/readme.txt +39 -51

137 lines not on this page · raw message

configs/raspberrypi2_defconfig +2 -0
index 7e49f0c..bfbf243 100644
@@ -26,3 +26,5 @@ BR2_LINUX_KERNEL_INTREE_DTS_NAME="bcm2709-rpi-2-b"
 
 BR2_PACKAGE_RPI_FIRMWARE=y
 # BR2_PACKAGE_RPI_FIRMWARE_INSTALL_DTB_OVERLAYS is not set
+
+BR2_ROOTFS_POST_IMAGE_SCRIPT="board/raspberrypi2/post-image.sh"
configs/raspberrypi_defconfig +2 -0
index 2d47058..5ded72f 100644
@@ -25,3 +25,5 @@ BR2_LINUX_KERNEL_INTREE_DTS_NAME="bcm2708-rpi-b bcm2708-rpi-b-plus"
 
 BR2_PACKAGE_RPI_FIRMWARE=y
 # BR2_PACKAGE_RPI_FIRMWARE_INSTALL_DTB_OVERLAYS is not set
+
+BR2_ROOTFS_POST_IMAGE_SCRIPT="board/raspberrypi/post-image.sh"
Signature · 1 line
2.1.4
#5 lore raw

PATCH 2/4 board/raspberrypi: prepare the image files for the target

Beno?t, All,

On 2015-08-22 22:01 +0200, Beno?t Th?baudeau spake thusly:

Quoted 30 lines

Automate the preparation of the generated image files for the Raspberry Pi firmware:
- mark the kernel image as supporting Device Tree,
- place the marked kernel image and the DTBs into the rpi-firmware folder.

The instructions in readme.txt are updated accordingly.

---
 board/raspberrypi/post-image.sh | 17 ++++++++
 board/raspberrypi/readme.txt    | 90 ++++++++++++++++++-----------------------
 configs/raspberrypi2_defconfig  |  2 +
 configs/raspberrypi_defconfig   |  2 +
 4 files changed, 60 insertions(+), 51 deletions(-)
 create mode 100755 board/raspberrypi/post-image.sh
board/raspberrypi/post-image.sh new file +7 -0
new file mode 100755
index 0000000..db9c53c
@@ -0,0 +1,17 @@
+#!/bin/bash -e
+
+RPI_FW_BIN_DIR="${BINARIES_DIR}/rpi-firmware"
+KERNEL_IMG="${BINARIES_DIR}/zImage"
+KERNEL_RPI_FW_IMG="${RPI_FW_BIN_DIR}/zImage"
+
+# A special marker must be appended to the kernel image so that the firmware of

board/raspberrypi/post-image.sh line 17

Quoted 10 lines
board/raspberrypi/post-image.sh new file +10 -0
+# the Raspberry Pi knows that it supports Device Tree. Without this marker, the
+# firmware passes the ATAGS boot data instead of the appropriate DTB to the
+# kernel.
+# The final kernel image must be in the rpi-firmware folder.
+mkknlimg "${KERNEL_IMG}" "${KERNEL_RPI_FW_IMG}"
+
+# Move the DTBs where the firmware expects them to be, i.e. to the rpi-firmware
+# folder.
+find "${BINARIES_DIR}/" -maxdepth 1 -type f -name '*.dtb' -print0 | \
+	xargs -0r mv -ft "${RPI_FW_BIN_DIR}/"

Why not simply:

    cp -l "${BINARIES_DIR}/"*.dtb "${RPI_FW_BIN_DIR}"

I'd prefer we keep the DTB files where they were installed, so the user does not get confused.

board/raspberrypi/readme.txt

Quoted 4 lines
board/raspberrypi/readme.txt +0 -0
index cf95879..168fab5 100644

[--SNIP--]

Quoted 9 lines
board/raspberrypi/readme.txt +3 -1
@@ -61,77 +61,65 @@ Result of the build
 After building, you should obtain this tree:
 
     output/images/
-    +-- rootfs.tar                  [0]
+    +-- rootfs.cpio                 [0]
+    +-- rootfs.tar                  [1]
     +-- rpi-firmware/
+    |   +-- bcm2708-rpi-b.dtb       [2]

board/raspberrypi/readme.txt Result of the build · line 72

Quoted 10 lines
board/raspberrypi/readme.txt +3 -4
+    |   +-- bcm2708-rpi-b-plus.dtb  [3]
+    |   +-- bcm2709-rpi-2-b.dtb     [4]
     |   +-- bootcode.bin
+    |   +-- cmdline.txt             [5]
     |   +-- config.txt
     |   +-- fixup.dat
-    |   `-- start.elf
-    +-- bcm2708-rpi-b.dtb           [1]
-    +-- bcm2708-rpi-b-plus.dtb      [1]
-    +-- bcm2709-rpi-2-b.dtb         [1]

As I said above, I'd prefer the DTB files be left in their original location, with the ones in rpi-firmware/ being only copies of, or hard-links to, the originals.

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#6 lore raw

PATCH 2/4 board/raspberrypi: prepare the image files for the target

Dear Yann E. MORIN,

On Sat, Oct 17, 2015 at 1:45 PM, Yann E. MORIN <[email protected]> wrote:
[...]

Quoted 5 lines

Why not simply:

    cp -l "${BINARIES_DIR}/"*.dtb "${RPI_FW_BIN_DIR}"

I'd prefer we keep the DTB files where they were installed, so the user does not get confused.

[...]

Quoted 3 lines

As I said above, I'd prefer the DTB files be left in their original location, with the ones in rpi-firmware/ being only copies of, or hard-links to, the originals.

OK, I'll do that.

Best regards,
Beno?t

#7 lore raw

PATCH 3/4 board/raspberrypi: generate a medium image

In reply to Benoît Thébaudeau

Automate the generation of an sdcard.img medium image, ready to be written to the SD card, while leaving the manual procedure available for custom needs.

genimage is used, so no commands need to be run as root by Buildroot.

The layout of the generated image is adjusted automatically if an initramfs is selected in the configuration.

The instructions in readme.txt are updated accordingly. The layout of the generated image is described in this file.

---
 board/raspberrypi/genimage-persistent-rootfs.cfg | 22 +++++++
 board/raspberrypi/genimage-volatile-rootfs.cfg   | 17 +++++
 board/raspberrypi/genimage.sh                    | 15 +++++
 board/raspberrypi/readme.txt                     | 79 +++++++++++++++++-------
 configs/raspberrypi2_defconfig                   |  6 +-
 configs/raspberrypi_defconfig                    |  6 +-
 6 files changed, 121 insertions(+), 24 deletions(-)
 create mode 100644 board/raspberrypi/genimage-persistent-rootfs.cfg
 create mode 100644 board/raspberrypi/genimage-volatile-rootfs.cfg
 create mode 100755 board/raspberrypi/genimage.sh
board/raspberrypi/genimage-persistent-rootfs.cfg new file +22 -0
new file mode 100644
index 0000000..e519ca6
@@ -0,0 +1,22 @@
+image boot.vfat {
+	vfat {
+		extraargs = "-F 16 -n BOOT"
+	}
+	size = 10M
+	mountpoint = "/"
+}
+
+image sdcard.img {
+	hdimage {}
+
+	partition boot {
+		partition-type = 0x0e
+		bootable = true
+		image = "boot.vfat"
+	}
+
+	partition rootfs {
+		partition-type = 0x83
+		image = "rootfs.ext4"
+	}
+}
board/raspberrypi/genimage-volatile-rootfs.cfg new file +17 -0
new file mode 100644
index 0000000..488bd81
@@ -0,0 +1,17 @@
+image boot.vfat {
+	vfat {
+		extraargs = "-F 32 -n BOOT"
+	}
+	size = 50M
+	mountpoint = "/"
+}
+
+image sdcard.img {
+	hdimage {}
+
+	partition boot {
+		partition-type = 0x0c
+		bootable = true
+		image = "boot.vfat"
+	}
+}
board/raspberrypi/genimage.sh new file +15 -0
new file mode 100755
index 0000000..663335c
@@ -0,0 +1,15 @@
+#!/bin/bash -e
+
+BOARD_DIR="board/raspberrypi"
+RPI_FW_BIN_DIR="${BINARIES_DIR}/rpi-firmware"
+if grep "^BR2_TARGET_ROOTFS_INITRAMFS=y$" "${BR2_CONFIG}" &>/dev/null; then
+	GENIMAGE_CFG="${BOARD_DIR}/genimage-volatile-rootfs.cfg"
+else
+	GENIMAGE_CFG="${BOARD_DIR}/genimage-persistent-rootfs.cfg"
+fi
+GENIMAGE_TMP="${BUILD_DIR}/genimage.tmp"
+
+rm -Rf "${GENIMAGE_TMP}"
+genimage --rootpath "${RPI_FW_BIN_DIR}" --tmppath "${GENIMAGE_TMP}" \
+	--inputpath "${BINARIES_DIR}" --outputpath "${BINARIES_DIR}" \
+	--config "${GENIMAGE_CFG}"
board/raspberrypi/readme.txt +57 -22

131 lines not on this page · raw message

configs/raspberrypi2_defconfig +5 -1
index bfbf243..73dc290 100644
@@ -27,4 +27,8 @@ BR2_LINUX_KERNEL_INTREE_DTS_NAME="bcm2709-rpi-2-b"
 BR2_PACKAGE_RPI_FIRMWARE=y
 # BR2_PACKAGE_RPI_FIRMWARE_INSTALL_DTB_OVERLAYS is not set
 
-BR2_ROOTFS_POST_IMAGE_SCRIPT="board/raspberrypi2/post-image.sh"
+BR2_TARGET_ROOTFS_EXT2=y
+BR2_TARGET_ROOTFS_EXT2_4=y
+BR2_TARGET_ROOTFS_EXT2_LABEL="rootfs"
+BR2_PACKAGE_HOST_GENIMAGE=y
+BR2_ROOTFS_POST_IMAGE_SCRIPT="board/raspberrypi2/post-image.sh board/raspberrypi2/genimage.sh"
configs/raspberrypi_defconfig +5 -1
index 5ded72f..58f4cab 100644
@@ -26,4 +26,8 @@ BR2_LINUX_KERNEL_INTREE_DTS_NAME="bcm2708-rpi-b bcm2708-rpi-b-plus"
 BR2_PACKAGE_RPI_FIRMWARE=y
 # BR2_PACKAGE_RPI_FIRMWARE_INSTALL_DTB_OVERLAYS is not set
 
-BR2_ROOTFS_POST_IMAGE_SCRIPT="board/raspberrypi/post-image.sh"
+BR2_TARGET_ROOTFS_EXT2=y
+BR2_TARGET_ROOTFS_EXT2_4=y
+BR2_TARGET_ROOTFS_EXT2_LABEL="rootfs"
+BR2_PACKAGE_HOST_GENIMAGE=y
+BR2_ROOTFS_POST_IMAGE_SCRIPT="board/raspberrypi/post-image.sh board/raspberrypi/genimage.sh"
Signature · 1 line
2.1.4
#8 lore raw

PATCH 4/4 board/raspberrypi: auto-expand rootfs on first boot

In reply to Benoît Thébaudeau

Add init scripts to auto-expand the persistent rootfs on the first boot to fill the medium.

These scripts follow the same procedure as raspi-config. The root partition is first expanded, then a reboot is required, then the rootfs is expanded. Each script removes itself once run.

The volatile (initramfs) rootfs is not affected by this change.

The instructions in readme.txt are updated accordingly.

---
 board/raspberrypi/post-build.sh                    |  7 +++++
 board/raspberrypi/readme.txt                       | 11 ++++---
 .../rootfs-overlay/etc/init.d/S22expand-rootpart   | 36 ++++++++++++++++++++++
 .../rootfs-overlay/etc/init.d/S23expand-rootfs     | 16 ++++++++++
 configs/raspberrypi2_defconfig                     |  8 +++++
 configs/raspberrypi_defconfig                      | 12 ++++++--
 6 files changed, 84 insertions(+), 6 deletions(-)
 create mode 100755 board/raspberrypi/post-build.sh
 create mode 100755 board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart
 create mode 100755 board/raspberrypi/rootfs-overlay/etc/init.d/S23expand-rootfs
board/raspberrypi/post-build.sh new file +7 -0
new file mode 100755
index 0000000..4b92b33
@@ -0,0 +1,7 @@
+#!/bin/bash -e
+
+# The init scripts used to auto-expand the persistent rootfs on the first boot
+# to fill the medium must not be used with the volatile (initramfs) rootfs.
+if grep "^BR2_TARGET_ROOTFS_INITRAMFS=y$" "${BR2_CONFIG}" &>/dev/null; then
+	rm -f "${TARGET_DIR}"/etc/init.d/{S22expand-rootpart,S23expand-rootfs}
+fi
board/raspberrypi/readme.txt +7 -4
index d63e71e..7ee783b 100644
@@ -34,10 +34,13 @@ And for model 2 B:
   $ make raspberrypi2_defconfig
 
 If you want to use a persistent rootfs, the generated ext4 rootfs image fits its
-contents with only a small (but sensible) margin of free space by default, so
-you might want to adjust "Filesystem images"/"ext2/3/4 root filesystem"/
-"size in blocks (leave at 0 for auto calculation)" depending on your needs, then
-skip to "Build the rootfs", below.
+contents with only a small (but sensible) margin of free space by default, but
+it is auto-expanded on the first boot to fill the medium. You can however adjust
+"Filesystem images"/"ext2/3/4 root filesystem"/
+"size in blocks (leave at 0 for auto calculation)" to fit your needs. The
+auto-expansion is controlled by
+board/raspberrypi/rootfs-overlay/etc/init.d/S*expand-root*. Skip to
+"Build the rootfs", below.
 
 For a volatile rootfs, you have to slightly adjust the configuration:
 
board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart new file +36 -0
new file mode 100755
index 0000000..7f19e7a
@@ -0,0 +1,36 @@
+#!/bin/sh
+
+case "$1" in
+	start)
+		echo -n "Expanding the root partition: "
+		BLOCK_DEV="/dev/mmcblk0"
+		PART_NUM="2"
+		PART_START="$(parted -ms "${BLOCK_DEV}" unit s p 2>/dev/null | \
+			grep "^${PART_NUM}:" | cut -d : -f 2 | sed 's/s$//')"
+		if [ -z "${PART_START}" ]; then
+			echo "failed"
+		else
+			fdisk "${BLOCK_DEV}" <<-EOF &>/dev/null
+				d
+				$PART_NUM
+				n
+				p
+				$PART_NUM
+				$PART_START
+
+				w
+			EOF
+			echo "done"
+		fi
+		rm -f /etc/init.d/S22expand-rootpart
+		if [ "${PART_START}" ]; then
+			reboot -f
+			while true; do :; done
+		fi
+		exit 2
+		;;
+	*)
+		echo "Usage: $0 start" >&2
+		exit 1
+		;;
+esac
board/raspberrypi/rootfs-overlay/etc/init.d/S23expand-rootfs new file +16 -0
new file mode 100755
index 0000000..7b3f286
@@ -0,0 +1,16 @@
+#!/bin/sh
+
+case "$1" in
+	start)
+		echo -n "Expanding the root FS: "
+		resize2fs /dev/mmcblk0p2 &>/dev/null
+		ret=$?
+		[ ${ret} -eq 0 ] && echo "done" || echo "failed"
+		rm -f /etc/init.d/S23expand-rootfs
+		exit ${ret}
+		;;
+	*)
+		echo "Usage: $0 start" >&2
+		exit 1
+		;;
+esac
configs/raspberrypi2_defconfig +8 -0
index 73dc290..2e064e0 100644
@@ -3,6 +3,7 @@ BR2_cortex_a7=y
 BR2_ARM_EABIHF=y
 BR2_ARM_FPU_NEON_VFPV4=y
 
+BR2_TOOLCHAIN_BUILDROOT_GLIBC=y
 BR2_TOOLCHAIN_BUILDROOT_CXX=y
 
 BR2_TARGET_GENERIC_GETTY_PORT="tty1"
@@ -27,6 +28,13 @@ BR2_LINUX_KERNEL_INTREE_DTS_NAME="bcm2709-rpi-2-b"
 BR2_PACKAGE_RPI_FIRMWARE=y
 # BR2_PACKAGE_RPI_FIRMWARE_INSTALL_DTB_OVERLAYS is not set
 
+BR2_PACKAGE_E2FSPROGS=y
+BR2_PACKAGE_E2FSPROGS_RESIZE2FS=y
+BR2_PACKAGE_PARTED=y
+BR2_PACKAGE_UTIL_LINUX_BINARIES=y
+
+BR2_ROOTFS_OVERLAY="board/raspberrypi2/rootfs-overlay"
+BR2_ROOTFS_POST_BUILD_SCRIPT="board/raspberrypi2/post-build.sh"
 BR2_TARGET_ROOTFS_EXT2=y
 BR2_TARGET_ROOTFS_EXT2_4=y
 BR2_TARGET_ROOTFS_EXT2_LABEL="rootfs"
configs/raspberrypi_defconfig +10 -2
index 58f4cab..7e744f3 100644
@@ -2,6 +2,9 @@ BR2_arm=y
 BR2_arm1176jzf_s=y
 BR2_ARM_EABIHF=y
 
+BR2_TOOLCHAIN_BUILDROOT_GLIBC=y
+BR2_TOOLCHAIN_BUILDROOT_CXX=y
+
 BR2_TARGET_GENERIC_GETTY_PORT="tty1"
 
 # Lock to 4.1 headers as the RPi kernel stable tree is
@@ -10,8 +13,6 @@ BR2_KERNEL_HEADERS_VERSION=y
 BR2_DEFAULT_KERNEL_VERSION="4.1.5"
 BR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_4_1=y
 
-BR2_TOOLCHAIN_BUILDROOT_CXX=y
-
 BR2_LINUX_KERNEL=y
 BR2_LINUX_KERNEL_CUSTOM_GIT=y
 BR2_LINUX_KERNEL_CUSTOM_REPO_URL="https://github.com/raspberrypi/linux.git"
@@ -26,6 +27,13 @@ BR2_LINUX_KERNEL_INTREE_DTS_NAME="bcm2708-rpi-b bcm2708-rpi-b-plus"
 BR2_PACKAGE_RPI_FIRMWARE=y
 # BR2_PACKAGE_RPI_FIRMWARE_INSTALL_DTB_OVERLAYS is not set
 
+BR2_PACKAGE_E2FSPROGS=y
+BR2_PACKAGE_E2FSPROGS_RESIZE2FS=y
+BR2_PACKAGE_PARTED=y
+BR2_PACKAGE_UTIL_LINUX_BINARIES=y
+
+BR2_ROOTFS_OVERLAY="board/raspberrypi/rootfs-overlay"
+BR2_ROOTFS_POST_BUILD_SCRIPT="board/raspberrypi/post-build.sh"
 BR2_TARGET_ROOTFS_EXT2=y
 BR2_TARGET_ROOTFS_EXT2_4=y
 BR2_TARGET_ROOTFS_EXT2_LABEL="rootfs"
Signature · 1 line
2.1.4
#9 lore raw

PATCH 4/4 board/raspberrypi: auto-expand rootfs on first boot

Hi Vivien, Yann, all,

Quoted 12 lines · Beno?t Th?baudeau

Add init scripts to auto-expand the persistent rootfs on the first boot to fill the medium.

These scripts follow the same procedure as raspi-config. The root partition is first expanded, then a reboot is required, then the rootfs is expanded. Each script removes itself once run.

The volatile (initramfs) rootfs is not affected by this change.

The instructions in readme.txt are updated accordingly.

[...]

board/raspberrypi/rootfs-overlay/etc/init.d/S23expand-rootfs line 6

Quoted 12 lines
board/raspberrypi/rootfs-overlay/etc/init.d/S23expand-rootfs new file +6 -0
new file mode 100755
index 0000000..7b3f286
@@ -0,0 +1,16 @@
+#!/bin/sh
+
+case "$1" in
+	start)
+		echo -n "Expanding the root FS: "
+		resize2fs /dev/mmcblk0p2 &>/dev/null

I had first tested this on a Raspberry Pi B using a 2-GiB SD card. It worked fine and it was very quick.

I've just tested this on a Raspberry Pi B+ using a 8-GiB SD card. First, I got:

    [    6.831228] EXT4-fs (mmcblk0p2): resizing filesystem from 130322 to 7611388 blocks
    [    6.838822] EXT4-fs (mmcblk0p2): Converting file system to meta_bg
    [   16.888893] EXT4-fs (mmcblk0p2): resized to 3407537 blocks
    [   20.794071] ------------[ cut here ]------------
    [   20.798723] kernel BUG at fs/jbd2/journal.c:766!
    [   20.803338] Internal error: Oops - BUG: 0 [#1] PREEMPT ARM
    Entering kdb (current=0xcf339b80, pid 44) Oops: (null)
    due to oops @ 0xc02216a8
    CPU: 0 PID: 44 Comm: jbd2/mmcblk0p2- Not tainted 4.1.5 #1
    Hardware name: BCM2708
    task: cf339b80 ti: cf356000 task.ti: cf356000
    PC is at jbd2_journal_next_log_block+0xb0/0xb4
    LR is at jbd2_journal_commit_transaction+0x978/0x1f08
    pc : [<c02216a8>]    lr : [<c0219cd0>]    psr: 60000013
    sp : cf357e00  ip : 00000001  fp : cf357e1c
    r10: ce93851c  r9 : cef22140  r8 : c0218e0c
    r7 : 00000021  r6 : ceedfbc0  r5 : cf357e88  r4 : cedb41e0
    r3 : 00000001  r2 : cf357e00  r1 : cf357e88  r0 : cf336000
    Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment kernel
    Control: 00c5387d  Table: 0e6fc008  DAC: 00000015
    CPU: 0 PID: 44 Comm: jbd2/mmcblk0p2- Not tainted 4.1.5 #1
    Hardware name: BCM2708
    [<c0016d08>] (unwind_backtrace) from [<c0013ad0>] (show_stack+0x20/0x24)
    [<c0013ad0>] (show_stack) from [<c0566070>] (dump_stack+0x20/0x28)
    [<c0566070>] (dump_stack) from [<c0010964>] (show_regs+0x1c/0x20)
    [<c0010964>] (show_regs) from [<c00a4b38>] (kdb_main_loop+0x3c8/0x730)
    [<c00a4b38>] (kdb_main_loop) from [<c00a75dc>] (kdb_stub+0x19c/0x3cc)
    [<c00a75dc>] (kdb_stub) from [<c009da00>] (kgdb_handle_exception+0x284/0x8a8)
    more> 

I retried several times with the same result. Then, I fsck'ed and reformatted the SD card, and I retried with the same sdcard.img as the first time. It worked fine, but it was extremely slow:

    [    6.803146] EXT4-fs (mmcblk0p2): resizing filesystem from 130322 to 7611388 blocks
    [    6.826546] EXT4-fs (mmcblk0p2): Converting file system to meta_bg
    [   16.884732] EXT4-fs (mmcblk0p2): resized to 3423841 blocks
    [   26.912318] EXT4-fs (mmcblk0p2): resized to 4442841 blocks
[...]
    [  768.704191] EXT4-fs (mmcblk0p2): resized to 7581361 blocks
    [  775.571684] EXT4-fs (mmcblk0p2): resized filesystem to 7611388

So the initial contents of the unused space on the SD card seem to have an influence, and there seems to be a bug somewhere (Linux, resize2fs, genext2fs, tune2fs, or lack of call to e2fsck before calling resize2fs but this would not be reliable with an online partition).

Then I retried with a 16-GiB SD card on a Raspberry Pi B, and it was very slow too. Same if the resize is performed on a PC. However, it's very quick with raspi-config and the latest Raspbian image. The main difference seemed to be the block size (4 KiB on Raspbian vs. 1 KiB with mke2img), so I hacked mke2img to test with genext2fs -B 4096 and tune2fs -J size=4, but it did not make things faster, only the image bigger (about 512 MiB, probably the minimal size for 4-KiB blocks). So I don't know which ones, but it's probably some ext4 parameters that are having an influence on the resize2fs speed here. Any idea?

So what do you think would be the best thing to do here?
1) Find the set of ext4 parameters making resize2fs faster and change mke2img and fs/ext2/* to make it possible to use these parameters.
2) Just drop this patch from the series because the resize will probably always be slow for some larger SD cards.

If 2), would you keep the auto ext4 size as in the current 3/4, or would you set a fixed size with some significant free space margin (e.g. 512 MiB)?

Best regards,
Beno?t

#10 lore raw

PATCH 4/4 board/raspberrypi: auto-expand rootfs on first boot

Hello Beno?t,

Quoted 3 lines · On Thursday 27 August 2015 11:02:02 Beno?t Th?baudeau wrote:

Hi Vivien, Yann, all,

On 22/08/2015 22:01, Beno?t Th?baudeau wrote:

[...]

Quoted 5 lines

So the initial contents of the unused space on the SD card seem to have an influence, and there seems to be a bug somewhere (Linux, resize2fs, genext2fs, tune2fs, or lack of call to e2fsck before calling resize2fs but this would not be reliable with an online partition).

Quoted 9 lines

Then I retried with a 16-GiB SD card on a Raspberry Pi B, and it was very slow too. Same if the resize is performed on a PC. However, it's very quick with raspi-config and the latest Raspbian image. The main difference seemed to be the block size (4 KiB on Raspbian vs. 1 KiB with mke2img), so I hacked mke2img to test with genext2fs -B 4096 and tune2fs -J size=4, but it did not make things faster, only the image bigger (about 512 MiB, probably the minimal size for 4-KiB blocks). So I don't know which ones, but it's probably some ext4 parameters that are having an influence on the resize2fs speed here. Any idea?

Yes, I have an idea :-) Did you try sparse_super?

In fact a did nearly same task a few month ago and I found same problem. For information, I use following script to fill disk with last partition:

sdflash.sh new file +48 -0
new file mode 100755
index 0000000..5f9a9d5
@@ -0,0 +1,48 @@
+#!/bin/bash
+#
+# Copy a disk image to sdcard card and resize last partition in order to use
+# whole disk
+#
+
+device=$1
+image=$2
+
+if [ "$device" = "" ]; then
+    echo "Usage: $0 DEVICE IMAGE"
+    echo "Example: $0 /dev/sdd images/disk.img"
+    exit 1
+fi
+
+sfdisk -R $device       || exit 1
+echo "Copying $image to $device"
+dd if=$image of=$device conv=fsync,notrunc || exit 1
+sfdisk -R $device       || exit 1
+for part in 4 3 2 1; do
+    for file in $device$part ${device}p${part}; do
+        if [ -e $file ]; then
+            ID=$(sfdisk --print-id $device $part)
+            if [[ "0x$ID" -eq 0x83 ]]; then
+                e2fsck -f $file
+                [ $? -le 1 ] || exit 1
+                tune2fs -O sparse_super -c 0 -i 0 $file
+                [ $? -eq 0 ] || exit 1
+                e2fsck -yf $file > /dev/null
+                [ $? -le 1 ] || exit 1
+                sleep 1
+                echo ',+,' | sfdisk -L -D -uM -N $part $device
+                [ $? -eq 0 ] || exit 1
+                resize2fs $file
+                [ $? -eq 0 ] || exit 1
+                e2fsck -f $file
+                [ $? -le 1 ] || exit 1
+                exit 0
+            else
+                echo "Unknown partition type"
+                exit 1
+            fi
+        fi
+    done
+done
+echo "Cannot find partition on $device"
+exit 1
+
Signature · 3 lines
J?r?me Pouiller, Sysmic
Embedded Linux specialist
http://www.sysmic.fr
#11 lore raw

PATCH 4/4 board/raspberrypi: auto-expand rootfs on first boot

Hi J?r?me,

Quoted 19 lines · J?r?me Pouiller
Quoted 3 lines · On Thursday 27 August 2015 11:02:02 Beno?t Th?baudeau wrote:

Hi Vivien, Yann, all,

On 22/08/2015 22:01, Beno?t Th?baudeau wrote:

[...]

Quoted 5 lines

So the initial contents of the unused space on the SD card seem to have an influence, and there seems to be a bug somewhere (Linux, resize2fs, genext2fs, tune2fs, or lack of call to e2fsck before calling resize2fs but this would not be reliable with an online partition).

Quoted 9 lines

Then I retried with a 16-GiB SD card on a Raspberry Pi B, and it was very slow too. Same if the resize is performed on a PC. However, it's very quick with raspi-config and the latest Raspbian image. The main difference seemed to be the block size (4 KiB on Raspbian vs. 1 KiB with mke2img), so I hacked mke2img to test with genext2fs -B 4096 and tune2fs -J size=4, but it did not make things faster, only the image bigger (about 512 MiB, probably the minimal size for 4-KiB blocks). So I don't know which ones, but it's probably some ext4 parameters that are having an influence on the resize2fs speed here. Any idea?

Quoted 1 line

Yes, I have an idea :-) Did you try sparse_super?

[...]

Awesome, it works, even if resizing the partition online. Thanks for the tip!

I'll send a patch changing mke2img to allow the use of this option.

Best regards,
Beno?t

#12 lore raw

PATCH 4/4 board/raspberrypi: auto-expand rootfs on first boot

Quoted 20 lines · Beno?t Th?baudeau
Quoted 19 lines · J?r?me Pouiller
Quoted 3 lines · On Thursday 27 August 2015 11:02:02 Beno?t Th?baudeau wrote:

Hi Vivien, Yann, all,

On 22/08/2015 22:01, Beno?t Th?baudeau wrote:

[...]

Quoted 14 lines

So the initial contents of the unused space on the SD card seem to have an influence, and there seems to be a bug somewhere (Linux, resize2fs, genext2fs, tune2fs, or lack of call to e2fsck before calling resize2fs but this would not be reliable with an online partition).

Then I retried with a 16-GiB SD card on a Raspberry Pi B, and it was very slow too. Same if the resize is performed on a PC. However, it's very quick with raspi-config and the latest Raspbian image. The main difference seemed to be the block size (4 KiB on Raspbian vs. 1 KiB with mke2img), so I hacked mke2img to test with genext2fs -B 4096 and tune2fs -J size=4, but it did not make things faster, only the image bigger (about 512 MiB, probably the minimal size for 4-KiB blocks). So I don't know which ones, but it's probably some ext4 parameters that are having an influence on the resize2fs speed here. Any idea?

Quoted 7 lines
Quoted 1 line

Yes, I have an idea :-) Did you try sparse_super?

[...]

Awesome, it works, even if resizing the partition online. Thanks for the tip!

I'll send a patch changing mke2img to allow the use of this option.

Please apply the patch below before this series in order to avoid the resize speed issue described above:
http://patchwork.ozlabs.org/patch/511861/

Best regards,
Beno?t

A message that is not in the archive (sent off-list or not yet received): 4057940.fE2DsQZqLb@sagittea

#14 lore raw

PATCH 4/4 board/raspberrypi: auto-expand rootfs on first boot

Hello J?r?me,

Quoted 30 lines · J?r?me Pouiller
Quoted 6 lines · On Saturday 22 August 2015 22:01:28 Beno?t Th?baudeau wrote:

Add init scripts to auto-expand the persistent rootfs on the first boot to fill the medium.

These scripts follow the same procedure as raspi-config. The root partition is first expanded, then a reboot is required, then the rootfs is expanded. Each script removes itself once run.

[..]

Quoted 16 lines
board/raspberrypi/readme.txt +0 -0
index d63e71e..7ee783b 100644
@@ -34,10 +34,13 @@ And for model 2 B:
   $ make raspberrypi2_defconfig
 
 If you want to use a persistent rootfs, the generated ext4 rootfs image

fits its -contents with only a small (but sensible) margin of free space by default, so -you might want to adjust "Filesystem images"/"ext2/3/4 root filesystem"/ -"size in blocks (leave at 0 for auto calculation)" depending on your needs, then -skip to "Build the rootfs", below. +contents with only a small (but sensible) margin of free space by default, but +it is auto-expanded on the first boot to fill the medium. You can however adjust +"Filesystem images"/"ext2/3/4 root filesystem"/ +"size in blocks (leave at 0 for auto calculation)" to fit your needs. The

board/raspberrypi/readme.txt And for model 2 B: · line 43

Quoted 3 lines
board/raspberrypi/readme.txt +3 -0
+auto-expansion is controlled by
+board/raspberrypi/rootfs-overlay/etc/init.d/S*expand-root*. Skip to
+"Build the rootfs", below.

Isn't safer to auto-expand partition while flashing SD Card instead of doing it online?

Quoted 2 lines

Maybe you can explain how to resize partition in README or provide a script like the one I sent you a few days ago (named sdflash.sh), no?

It would be safer if users pull the power plug during the few seconds that the expansion lasts, without looking at their console output, which is not a big risk. Doing it while flashing the SD card would have a major limitation: it prevents the publication of an autonomous image file that you can program by any means to get the same result.

Resizing a partition on a PC is trivial if the image is programmed first. There are plenty of tools like gparted and tutorials for that on the web, so I don't think adding such information to the readme.txt would be worth it.

Regarding a script writing the SD card and auto-expanding the rootfs, it might be useful. Since it would require superuser rights, it should not be added to the post-image scripts, but launched manually. That would give users a choice. This script would just write the generated image file, auto-expand the rootfs and remove the auto-expansion scripts from the rootfs. I'll look into it.

Best regards,
Beno?t

#15 lore raw

PATCH 4/4 board/raspberrypi: auto-expand rootfs on first boot

In reply to Benoît Thébaudeau
Quoted 2 lines · Beno?t Th?baudeau

Add init scripts to auto-expand the persistent rootfs on the first boot to fill the medium.

Wonder if it wouldn't be better to have a package section in buildroot for commonly used init scripts like this?

I don't think resizing the rootfs is specific to only this board. And by making it a package you could do "depends on !BR2_TARGET_ROOTFS_INITRAMFS" instead of grep magic in the post build script.

Yours sincerely,

Floris Bos

#16 lore raw

PATCH 4/4 board/raspberrypi: auto-expand rootfs on first boot

Dear Floris Bos,

Quoted 5 lines · Floris Bos
Quoted 2 lines · Beno?t Th?baudeau

Add init scripts to auto-expand the persistent rootfs on the first boot to fill the medium.

Quoted 7 lines

Wonder if it wouldn't be better to have a package section in buildroot for commonly used init scripts like this?

I don't think resizing the rootfs is specific to only this board. And by making it a package you could do "depends on !BR2_TARGET_ROOTFS_INITRAMFS" instead of grep magic in the post build script.

That makes sense. It would do "depends on BR2_TARGET_ROOTFS_EXT2" and select the required target utils. It would also have to ask the user for the block device to use (or it could autodetect at runtime the block device on which '/' is mounted). Should this be a new package, or just new options in the ext2 rootfs menu? I think that the latter makes more sense for now.

But regarding the "!BR2_TARGET_ROOTFS_INITRAMFS", it would not be needed at all. This is only on RPi to make user choices easier by not having to unselect ext4fs and the rootfs overlay besides selecting initramfs. On RPi, we could just force users to select only one of them, and say so in readme.txt, and there would be no rootfs overlay to manage with this feature as new config options.

Best regards,
Beno?t

#17 lore raw

PATCH 4/4 board/raspberrypi: auto-expand rootfs on first boot

In reply to Benoît Thébaudeau
Quoted 2 lines · Beno?t Th?baudeau

Add init scripts to auto-expand the persistent rootfs on the first boot to fill the medium.

Had a look at your script, but think it has more dependencies than strictly necessary.

Quoted 8 lines
board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart new file +1 -0
new file mode 100755
index 0000000..7f19e7a
@@ -0,0 +1,36 @@
+#!/bin/sh

board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart line 11

Quoted 10 lines
board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart new file +10 -0
+
+case "$1" in
+	start)
+		echo -n "Expanding the root partition: "
+		BLOCK_DEV="/dev/mmcblk0"
+		PART_NUM="2"
+		PART_START="$(parted -ms "${BLOCK_DEV}" unit s p 2>/dev/null | \
+			grep "^${PART_NUM}:" | cut -d : -f 2 | sed 's/s$//')"
+		if [ -z "${PART_START}" ]; then
+			echo "failed"

Note that the existing start sector can also be found in /sys/class/block/mmcblk0p2/start
So can remove the parted dependency.

Quoted 52 lines
board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart new file +25 -0
+		else
+			fdisk "${BLOCK_DEV}" <<-EOF &>/dev/null
+				d
+				$PART_NUM
+				n
+				p
+				$PART_NUM
+				$PART_START
+
+				w
+			EOF
+			echo "done"
+		fi
+		rm -f /etc/init.d/S22expand-rootpart
+		if [ "${PART_START}" ]; then
+			reboot -f
+			while true; do :; done
+		fi
+		exit 2
+		;;
+	*)
+		echo "Usage: $0 start" >&2
+		exit 1
+		;;
+esac
board/raspberrypi/rootfs-overlay/etc/init.d/S23expand-rootfs new file +16 -0
new file mode 100755
index 0000000..7b3f286
@@ -0,0 +1,16 @@
+#!/bin/sh
+
+case "$1" in
+	start)
+		echo -n "Expanding the root FS: "
+		resize2fs /dev/mmcblk0p2 &>/dev/null
+		ret=$?
+		[ ${ret} -eq 0 ] && echo "done" || echo "failed"
+		rm -f /etc/init.d/S23expand-rootfs
+		exit ${ret}
+		;;
+	*)
+		echo "Usage: $0 start" >&2
+		exit 1
+		;;
+esac
configs/raspberrypi2_defconfig +0 -0
index 73dc290..2e064e0 100644

configs/raspberrypi2_defconfig BR2_cortex_a7=y · line 6

Quoted 4 lines
configs/raspberrypi2_defconfig +1 -0
@@ -3,6 +3,7 @@ BR2_cortex_a7=y
  BR2_ARM_EABIHF=y
  BR2_ARM_FPU_NEON_VFPV4=y
 
+BR2_TOOLCHAIN_BUILDROOT_GLIBC=y

Why is glibc required?
Isn't it possible to just set the right options (like wchar, largefile) for uclibc?

Quoted 4 lines
configs/raspberrypi2_defconfig +0 -0
  BR2_TOOLCHAIN_BUILDROOT_CXX=y
 
  BR2_TARGET_GENERIC_GETTY_PORT="tty1"

configs/raspberrypi2_defconfig BR2_LINUX_KERNEL_INTREE_DTS_NAME="bcm2709-rpi-2-b" · line 37

Quoted 10 lines
configs/raspberrypi2_defconfig +7 -0
@@ -27,6 +28,13 @@ BR2_LINUX_KERNEL_INTREE_DTS_NAME="bcm2709-rpi-2-b"
  BR2_PACKAGE_RPI_FIRMWARE=y
  # BR2_PACKAGE_RPI_FIRMWARE_INSTALL_DTB_OVERLAYS is not set
 
+BR2_PACKAGE_E2FSPROGS=y
+BR2_PACKAGE_E2FSPROGS_RESIZE2FS=y
+BR2_PACKAGE_PARTED=y
+BR2_PACKAGE_UTIL_LINUX_BINARIES=y
+
+BR2_ROOTFS_OVERLAY="board/raspberrypi2/rootfs-overlay"
+BR2_ROOTFS_POST_BUILD_SCRIPT="board/raspberrypi2/post-build.sh"

Why is UTIL_LINUX_BINARIES necessary?
If for fdisk, doesn't busybox provides this as well?

Yours sincerely,

Floris Bos

#18 lore raw

PATCH 4/4 board/raspberrypi: auto-expand rootfs on first boot

Dear Floris Bos,

Quoted 7 lines · Floris Bos
Quoted 2 lines · Beno?t Th?baudeau

Add init scripts to auto-expand the persistent rootfs on the first boot to fill the medium.

Had a look at your script, but think it has more dependencies than strictly necessary.

Certainly. I've not tried to optimize dependencies in this patch.

Quoted 18 lines
Quoted 18 lines

diff --git
a/board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart b/board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart new file mode 100755
index 0000000..7f19e7a

board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart new file +8 -0
@@ -0,0 +1,36 @@
+#!/bin/sh
+
+case "$1" in
+       start)
+               echo -n "Expanding the root partition: "
+               BLOCK_DEV="/dev/mmcblk0"
+               PART_NUM="2"
+               PART_START="$(parted -ms "${BLOCK_DEV}" unit s p

2>/dev/null | \
+ grep "^${PART_NUM}:" | cut -d : -f 2 | sed

Quoted 8 lines

board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart line 11

Quoted 3 lines

's/s$//')"

board/raspberrypi/rootfs-overlay/etc/init.d/S22expand-rootpart new file +2 -0
+               if [ -z "${PART_START}" ]; then
+                       echo "failed"

Note that the existing start sector can also be found in /sys/class/block/mmcblk0p2/start
So can remove the parted dependency.

Yes, but we would have to check how portable this is among all Linux versions if we want to move that to fs/ext2/ or packages/.

[...]

Quoted 6 lines
Quoted 6 lines
configs/raspberrypi2_defconfig +0 -0
index 73dc290..2e064e0 100644
@@ -3,6 +3,7 @@ BR2_cortex_a7=y
Quoted 8 lines

configs/raspberrypi2_defconfig BR2_cortex_a7=y

Quoted 3 lines
configs/raspberrypi2_defconfig +0 -0
  BR2_ARM_EABIHF=y
  BR2_ARM_FPU_NEON_VFPV4=y
  +BR2_TOOLCHAIN_BUILDROOT_GLIBC=y

Why is glibc required?
Isn't it possible to just set the right options (like wchar, largefile) for uclibc?

Maybe. It was for e2fsprogs or parted. I will check that.

Quoted 8 lines
Quoted 8 lines
configs/raspberrypi2_defconfig +2 -0
  BR2_TOOLCHAIN_BUILDROOT_CXX=y
    BR2_TARGET_GENERIC_GETTY_PORT="tty1"
@@ -27,6 +28,13 @@ BR2_LINUX_KERNEL_INTREE_DTS_NAME="bcm2709-rpi-2-b"
  BR2_PACKAGE_RPI_FIRMWARE=y
  # BR2_PACKAGE_RPI_FIRMWARE_INSTALL_DTB_OVERLAYS is not set
  +BR2_PACKAGE_E2FSPROGS=y
+BR2_PACKAGE_E2FSPROGS_RESIZE2FS=y
+BR2_PACKAGE_PARTED=y
Quoted 8 lines

configs/raspberrypi2_defconfig BR2_LINUX_KERNEL_INTREE_DTS_NAME="bcm2709-rpi-2-b" · line 37

Quoted 4 lines
configs/raspberrypi2_defconfig +4 -0
+BR2_PACKAGE_UTIL_LINUX_BINARIES=y
+
+BR2_ROOTFS_OVERLAY="board/raspberrypi2/rootfs-overlay"
+BR2_ROOTFS_POST_BUILD_SCRIPT="board/raspberrypi2/post-build.sh"

Why is UTIL_LINUX_BINARIES necessary?
If for fdisk,

Yes.

Quoted 1 line

doesn't busybox provides this as well?

Yes, it does, and it's selected in the default Busybox configuration provided by Buildroot and used on RPi. But I've not checked whether it is compatible with the one from util-linux regarding the command line interactions. I will check this.

Best regards,
Beno?t

#19 lore raw

PATCH 1/4 configs/raspberrypi: use EABIhf

In reply to Benoît Thébaudeau

Dear Beno?t Th?baudeau,

Quoted 8 lines · Beno?t Th?baudeau

The SoC of the Raspberry Pi 1 is an ARM1176JZF-S, which features a VFPv2 FPU, so use the EABIhf target ABI, which is more efficient than EABI. This is also the default on Raspbian.

---
 configs/raspberrypi_defconfig | 1 +
 1 file changed, 1 insertion(+)

Applied, thanks. I actually believe we should be doing the same for all ARMv6/ARMv7 platforms, except if there is a specific issue with using EABIhf (such as existing pre-built binaries that are EABI only).

Maybe we should even use EABIhf by default on ARMv6+.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#20 lore raw

PATCH 1/4 configs/raspberrypi: use EABIhf

Dear Thomas Petazzoni,

Quoted 3 lines · Thomas Petazzoni

Applied, thanks. I actually believe we should be doing the same for all ARMv6/ARMv7 platforms, except if there is a specific issue with using EABIhf (such as existing pre-built binaries that are EABI only).

Yes, there is such an issue because some hardware vendors provide closed libraries that use soft-FP. It was the case e.g. for i.MX6 (audio/video CODEC and GPU libraries), but they now provide hard-float binaries IIRC. So such a change should be checked on a per-platform basis.

Quoted 1 line

Maybe we should even use EABIhf by default on ARMv6+.

Yes, I think that it would be a sane default, and it would still be possible to set EABI for a specific platform for which EABIhf would not apply.

Best regards,
Beno?t

#21 lore raw

PATCH 1/4 configs/raspberrypi: use EABIhf

Quoted 8 lines · Beno?t Th?baudeau

Dear Thomas Petazzoni,

Quoted 3 lines · Thomas Petazzoni

Applied, thanks. I actually believe we should be doing the same for all ARMv6/ARMv7 platforms, except if there is a specific issue with using EABIhf (such as existing pre-built binaries that are EABI only).

Quoted 5 lines

Yes, there is such an issue because some hardware vendors provide closed libraries that use soft-FP. It was the case e.g. for i.MX6 (audio/video CODEC and GPU libraries), but they now provide hard-float binaries IIRC. So such a change should be checked on a per-platform basis.

Actually, I think at the BR developer meeting we decided that even if there are
binary-only packages that require softfloat, our defconfig should still use hardfloat if possible (since the defconfig doesn't enable that package). Anyway the point is moot since those binary packages will in my experience also depend on glibc, so the defconfig will not work for them anyway. Also, I can't find this back in the report :-(

Regards,
Arnout

Quoted 14 lines
Quoted 2 lines

Maybe we should even use EABIhf by default on ARMv6+.

Yes, I think that it would be a sane default, and it would still be possible to set EABI for a specific platform for which EABIhf would not apply.

Best regards,
Beno?t
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
#22 lore raw

PATCH 1/4 configs/raspberrypi: use EABIhf

In reply to Thomas Petazzoni
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Hi,

Quoted 5 lines

Applied, thanks. I actually believe we should be doing the same for all ARMv6/ARMv7 platforms, except if there is a specific issue with using EABIhf (such as existing pre-built binaries that are EABI only).

Maybe we should even use EABIhf by default on ARMv6+.

Agreed!

Signature · 1 line
Bye, Peter Korsgaard
#23 lore raw

PATCH 1/4 configs/raspberrypi: use EABIhf

Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

"Peter" == Peter Korsgaard <[email protected]> writes:

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Hi,

Quoted 7 lines
Quoted 5 lines

Applied, thanks. I actually believe we should be doing the same for all ARMv6/ARMv7 platforms, except if there is a specific issue with using EABIhf (such as existing pre-built binaries that are EABI only).

Maybe we should even use EABIhf by default on ARMv6+.

Agreed!

Only potential disadvantage is that we currently don't have any preconfigured external ARMv6 EABIhf toolchains :/

Signature · 1 line
Bye, Peter Korsgaard
#24 lore raw

PATCH 1/1 arch/arm: use EABIhf by default with VFP

In reply to Peter Korsgaard

Set EABIhf as the default target ABI for the ARM processors that have or may have a VFP unit, since this ABI is the most efficient in that case. Of course, EABI can still be selected manually if needed.

---
 arch/Config.in.arm | 1 +
 1 file changed, 1 insertion(+)
arch/Config.in.arm +1 -0
index 4d10f4c..f0110b1 100644
@@ -188,6 +188,7 @@ endchoice
 choice
 	prompt "Target ABI"
 	depends on BR2_arm || BR2_armeb
+	default BR2_ARM_EABIHF if BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2
 	default BR2_ARM_EABI
 	help
 	  Application Binary Interface to use. The Application Binary
Signature · 1 line
2.1.4
#25 lore raw

PATCH 1/1 arch/arm: use EABIhf by default with VFP

Quoted 16 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Beno?t" == Beno?t Th?baudeau <[email protected]> writes:

Set EABIhf as the default target ABI for the ARM processors that have or may have a VFP unit, since this ABI is the most efficient in that case. Of course, EABI can still be selected manually if needed.

---
 arch/Config.in.arm | 1 +
 1 file changed, 1 insertion(+)
arch/Config.in.arm +0 -0
index 4d10f4c..f0110b1 100644

arch/Config.in.arm endchoice · line 191

Quoted 4 lines
arch/Config.in.arm +1 -0
@@ -188,6 +188,7 @@ endchoice
 choice
 	prompt "Target ABI"
 	depends on BR2_arm || BR2_armeb
+	default BR2_ARM_EABIHF if BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2

I think we should only do it when we are certain it will run, E.G. arm926 (our default variant) selects MAYBE_HAS_VFPV2, but it is afaik quite rare to see arm9s with a vfp.

Committed with that changed, thanks.

Signature · 2 lines
Venlig hilsen,
Peter Korsgaard 
#26 lore raw

PATCH 1/1 arch/arm: use EABIhf by default with VFP

Dear Peter Korsgaard,

Quoted 14 lines · Peter Korsgaard

arch/Config.in.arm endchoice · line 191

Quoted 9 lines
arch/Config.in.arm +1 -0
index 4d10f4c..f0110b1 100644
@@ -188,6 +188,7 @@ endchoice
 choice
 	prompt "Target ABI"
 	depends on BR2_arm || BR2_armeb
+	default BR2_ARM_EABIHF if BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2

I think we should only do it when we are certain it will run, E.G. arm926 (our default variant) selects MAYBE_HAS_VFPV2, but it is afaik quite rare to see arm9s with a vfp.

Quoted 1 line

Committed with that changed, thanks.

default BR2_ARM_EABIHF if BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2

was indeed wrong, since having BR2_ARM_CPU_MAYBE_HAS_VFPV2 does not indicate that the SoC has VFP.

However, unlike BR2_ARM_CPU_HAS_NEON which gets selected even on ARM cores for which NEON is optional, but for which the user has explicitly requested NEON support to be enabled (through BR2_ARM_ENABLE_NEON), it is not the case for the VFP option. I think we should do that. Otherwise, BR2_ARM_CPU_HAS_VFPV2 will not be true, even if the VFPv2 unit is actually available and used.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#27 lore raw

PATCH 1/1 arch/arm: use EABIhf by default with VFP

Quoted 19 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Dear Peter Korsgaard,

Quoted 14 lines · Peter Korsgaard

arch/Config.in.arm endchoice · line 191

Quoted 9 lines
arch/Config.in.arm +1 -0
index 4d10f4c..f0110b1 100644
@@ -188,6 +188,7 @@ endchoice
 choice
 	prompt "Target ABI"
 	depends on BR2_arm || BR2_armeb
+	default BR2_ARM_EABIHF if BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2

I think we should only do it when we are certain it will run, E.G. arm926 (our default variant) selects MAYBE_HAS_VFPV2, but it is afaik quite rare to see arm9s with a vfp.

Quoted 6 lines
Quoted 1 line

Committed with that changed, thanks.

default BR2_ARM_EABIHF if BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2

was indeed wrong, since having BR2_ARM_CPU_MAYBE_HAS_VFPV2 does not indicate that the SoC has VFP.

Indeed.

Quoted 6 lines

However, unlike BR2_ARM_CPU_HAS_NEON which gets selected even on ARM cores for which NEON is optional, but for which the user has explicitly requested NEON support to be enabled (through BR2_ARM_ENABLE_NEON), it is not the case for the VFP option. I think we should do that. Otherwise, BR2_ARM_CPU_HAS_VFPV2 will not be true, even if the VFPv2 unit is actually available and used.

But packages should use the BR2_ARM_FPU_VFPV2 (and 3/4) symbols to check if they should use the VFP, and not BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2

I don't think kconfig will let us both have these options depend on BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2 and select BR2_ARM_CPU_HAS_VFPV2.

Signature · 2 lines
Venlig hilsen,
Peter Korsgaard 
#28 lore raw

PATCH 1/1 arch/arm: use EABIhf by default with VFP

Peter,

Quoted 9 lines · Peter Korsgaard
Quoted 6 lines

However, unlike BR2_ARM_CPU_HAS_NEON which gets selected even on ARM cores for which NEON is optional, but for which the user has explicitly requested NEON support to be enabled (through BR2_ARM_ENABLE_NEON), it is not the case for the VFP option. I think we should do that. Otherwise, BR2_ARM_CPU_HAS_VFPV2 will not be true, even if the VFPv2 unit is actually available and used.

But packages should use the BR2_ARM_FPU_VFPV2 (and 3/4) symbols to check if they should use the VFP, and not BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2

But then they have to test BR2_ARM_FPU_VFPV2 || BR2_ARM_FPU_VFPV3 || BR2_ARM_FPU_VFPV3D16 || ...

If they just want to know if a VFP is available.

Quoted 3 lines

I don't think kconfig will let us both have these options depend on BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2 and select BR2_ARM_CPU_HAS_VFPV2.

Indeed, it doesn't. Some better solution is needed :)

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#29 lore raw

PATCH 1/1 arch/arm: use EABIhf by default with VFP

Quoted 12 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Peter,

Quoted 7 lines · Peter Korsgaard
Quoted 6 lines

However, unlike BR2_ARM_CPU_HAS_NEON which gets selected even on ARM cores for which NEON is optional, but for which the user has explicitly requested NEON support to be enabled (through BR2_ARM_ENABLE_NEON), it is not the case for the VFP option. I think we should do that. Otherwise, BR2_ARM_CPU_HAS_VFPV2 will not be true, even if the VFPv2 unit is actually available and used.

Quoted 7 lines
Quoted 2 lines

But packages should use the BR2_ARM_FPU_VFPV2 (and 3/4) symbols to check if they should use the VFP, and not BR2_ARM_CPU_MAYBE_HAS_VFPV2 || BR2_ARM_CPU_HAS_VFPV2

But then they have to test BR2_ARM_FPU_VFPV2 || BR2_ARM_FPU_VFPV3 || BR2_ARM_FPU_VFPV3D16 || ...

If they just want to know if a VFP is available.

If that is the case we could make a hidden BR2_ARM_FPU_VFP that gets selected by those, or simply check for !ARM_SOFT_FLOAT.

Signature · 2 lines
Venlig hilsen,
Peter Korsgaard 
#30 lore raw

PATCH 1/1 arch/arm: use EABIhf by default with VFP

In reply to Benoît Thébaudeau
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Beno?t" == Beno?t Th?baudeau <[email protected]> writes:

Set EABIhf as the default target ABI for the ARM processors that have or may have a VFP unit, since this ABI is the most efficient in that case. Of course, EABI can still be selected manually if needed.

On a related note, should we default to thumb2 if the CPU supports it?

Signature · 2 lines
Venlig hilsen,
Peter Korsgaard 
#31 lore raw

PATCH 1/1 arch/arm: use EABIhf by default with VFP

Quoted 7 lines · Peter Korsgaard
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Beno?t" == Beno?t Th?baudeau <[email protected]> writes:

Set EABIhf as the default target ABI for the ARM processors that have or may have a VFP unit, since this ABI is the most efficient in that case. Of course, EABI can still be selected manually if needed.

On a related note, should we default to thumb2 if the CPU supports it?

I would say:

choice
prompt "ARM instruction set"
default BR2_ARM_INSTRUCTIONS_ARM if BR2_ARM_CPU_HAS_ARM default BR2_ARM_INSTRUCTIONS_THUMBS2 if BR2_ARM_CPU_HAS_THUMBS2 default BR2_ARM_INSTRUCTIONS_THUMBS if BR2_ARM_CPU_HAS_THUMBS

Best regards,
Beno?t

#32 lore raw

PATCH 1/1 arch/arm: use EABIhf by default with VFP

Quoted 11 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Beno?t" == Beno?t Th?baudeau <[email protected]> writes:

Quoted 7 lines · Peter Korsgaard
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Beno?t" == Beno?t Th?baudeau <[email protected]> writes:

Set EABIhf as the default target ABI for the ARM processors that have or may have a VFP unit, since this ABI is the most efficient in that case. Of course, EABI can still be selected manually if needed.

On a related note, should we default to thumb2 if the CPU supports it?

Quoted 7 lines

I would say:

choice
prompt "ARM instruction set"
default BR2_ARM_INSTRUCTIONS_ARM if BR2_ARM_CPU_HAS_ARM default BR2_ARM_INSTRUCTIONS_THUMBS2 if BR2_ARM_CPU_HAS_THUMBS2 default BR2_ARM_INSTRUCTIONS_THUMBS if BR2_ARM_CPU_HAS_THUMBS

I'm not sure it makes sense to default to thumb1 even if the CPU supports it because of various compatibility issues - So simply:

default BR2_ARM_INSTRUCTIONS_THUMBS2 if BR2_ARM_CPU_HAS_THUMBS2

(if no thumb2 is available, the first available option (ARM) is used).

Signature · 2 lines
Venlig hilsen,
Peter Korsgaard 
#33 lore raw

PATCH 1/1 arch/arm: use EABIhf by default with VFP

Quoted 13 lines · Peter Korsgaard
Quoted 12 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Beno?t" == Beno?t Th?baudeau <[email protected]> writes:

Quoted 7 lines · Peter Korsgaard
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Beno?t" == Beno?t Th?baudeau <[email protected]> writes:

Set EABIhf as the default target ABI for the ARM processors that have or may have a VFP unit, since this ABI is the most efficient in that case. Of course, EABI can still be selected manually if needed.

On a related note, should we default to thumb2 if the CPU supports it?

I would say:

Quoted 8 lines
Quoted 5 lines

choice
prompt "ARM instruction set"
default BR2_ARM_INSTRUCTIONS_ARM if BR2_ARM_CPU_HAS_ARM default BR2_ARM_INSTRUCTIONS_THUMBS2 if BR2_ARM_CPU_HAS_THUMBS2 default BR2_ARM_INSTRUCTIONS_THUMBS if BR2_ARM_CPU_HAS_THUMBS

I'm not sure it makes sense to default to thumb1 even if the CPU supports it because of various compatibility issues

It's the first default that wins, not the last one, so above it's ARM over Thumb1.

Quoted 5 lines

- So simply:

default BR2_ARM_INSTRUCTIONS_THUMBS2 if BR2_ARM_CPU_HAS_THUMBS2

(if no thumb2 is available, the first available option (ARM) is used).

With ARM + Thumb2, I would rather use ARM to avoid compatibility issues, except on platforms very constrained on memory. On Cortex-M3, there's not even ARM. Or is it common to use Thumb2 on bigger platforms like Cortex-A*?

Best regards,
Beno?t

#34 lore raw

PATCH 1/1 arch/arm: use EABIhf by default with VFP

Beno?t, Peter, All,

On 2015-10-25 21:52 +0100, Beno?t Th?baudeau spake thusly:

Quoted 20 lines · Peter Korsgaard
Quoted 19 lines
Quoted 18 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Beno?t" == Beno?t Th?baudeau <[email protected]> writes:

Quoted 7 lines · Peter Korsgaard
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Beno?t" == Beno?t Th?baudeau <[email protected]> writes:

Set EABIhf as the default target ABI for the ARM processors that have or may have a VFP unit, since this ABI is the most efficient in that case. Of course, EABI can still be selected manually if needed.

On a related note, should we default to thumb2 if the CPU supports it?

I would say:

choice
prompt "ARM instruction set"
default BR2_ARM_INSTRUCTIONS_ARM if BR2_ARM_CPU_HAS_ARM default BR2_ARM_INSTRUCTIONS_THUMBS2 if BR2_ARM_CPU_HAS_THUMBS2 default BR2_ARM_INSTRUCTIONS_THUMBS if BR2_ARM_CPU_HAS_THUMBS

Quoted 4 lines
Quoted 2 lines

I'm not sure it makes sense to default to thumb1 even if the CPU supports it because of various compatibility issues

It's the first default that wins, not the last one,

To be more precise: it's the first default which dependencies are fullfiled that wins.

Quoted 4 lines

so above it's ARM over Thumb1.

Quoted 2 lines

- So simply:

Quoted 8 lines
Quoted 3 lines

default BR2_ARM_INSTRUCTIONS_THUMBS2 if BR2_ARM_CPU_HAS_THUMBS2

(if no thumb2 is available, the first available option (ARM) is used).

With ARM + Thumb2, I would rather use ARM to avoid compatibility issues, except on platforms very constrained on memory. On Cortex-M3, there's not even ARM. Or is it common to use Thumb2 on bigger platforms like Cortex-A*?

I agree that we should default to ARM if available. If not, then default to Thumb2 if available. Finally, fallback to Thumb1.

We know that if neither ARM not Thumb2 is available, then we must have Thumb1 available. otherwise it would not make sense to have an ARM CPU that does not support any of those three.

So, slight ammendment to Beno?t's proposal:

choice
prompt "ARM instruction set"
default BR2_ARM_INSTRUCTIONS_ARM if BR2_ARM_CPU_HAS_ARM
default BR2_ARM_INSTRUCTIONS_THUMB2 if BR2_ARM_CPU_HAS_THUMB2 default BR2_ARM_INSTRUCTIONS_THUMB

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'