Skip to contentSkip to conversation

post-build scripts: limitations, and proposal

RFC Yann E. MORIN 4 participants 5 messages Jun 20, 2013 – Jun 23, 2013

lore
Share & tools
lore
#0 lore raw

Hello All!

Currently, I'm using a set of post-build scripts, and it's becoming to be slightly tedious.

For example, here are a few of them:

- rpi-fixup-rootfs
- adds /dev/mmcblk0p1 on /boot to fstab, don't mount _netdev filesystems
- set root's PS1
- store config's tree hash in /etc/rpi-config.version [0]

- rpi-fixup-rootfs-yem
- stores my ssh pub key in root's ~/.ssh/authorized_keys

- rpi-fixup-rootfs-tvheadend
- add a script to wait for DVB devices
- add a CIFS mount to fstab, to access my NAS

- rpi-fixup-rootfs-wpa
- add my wpa-supplicant conf file to connect to my LAN

So, when I build my tvheadend filesystem, I want to run scripts:
rpi-fixup-rootfs rpi-fixup-rootfs-yem rpi-fixup-rootfs-tvheadend

But when I want to build my mpd filesystem, I want to run scripts:
rpi-fixup-rootfs rpi-fixup-rootfs-yem rpi-fixup-rootfs-wpa

Since the scripts lie out of the buildroot tree, I set (all on one line, but broken-out for readability):

    BR2_ROOTFS_POST_BUILD_SCRIPT="
        $(CONFIG_DIR)/../scripts/rpi-fixup-rootfs
        $(CONFIG_DIR)/../scripts/rpi-fixup-rootfs-yem
        $(CONFIG_DIR)/../scripts/rpi-fixup-rootfs-tvheadend"

Now, I am adding a new script. This is starting to be ridiculous...

What I'd like is to be able to pass one or more options to each post-build script, so I can have a wrapper and tell it what to run:

    BR2_ROOTFS_POST_BUILD_SCRIPT="
        $(CONFIG_DIR)/../scripts/fixup-rootfs rpi ssh-key rpi-tvheadend"

to call those scripts:
fixup-rootfs-rpi
fixup-rootfs-ssh-key
fixup-rootfs-rpi-tvheadend

But it is not possible, as Buildroot interprets BR2_ROOTFS_POST_BUILD_SCRIPT as a space-separated list, not a single command. So, it is not possible to pass any option to the post-build scripts.

So, I was going to change it from space-separated to colon-separated. But that would break existing configuration. Sigh... :-(

I can see only one way to solve this: rename the variable (and add to legacy), and make it a colon-separated list.

Opinions?

[0] My config files (and scripts...) are stored in a git tree, so I want to know which to rebuild only by looking at a file on a running board. Having only the Buildroot tree hash is not enough.

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#1 lore raw

Hi Yann,

Quoted 32 lines · Yann E. MORIN

Hello All!

Currently, I'm using a set of post-build scripts, and it's becoming to be slightly tedious.

For example, here are a few of them:

- rpi-fixup-rootfs
- adds /dev/mmcblk0p1 on /boot to fstab, don't mount _netdev filesystems
- set root's PS1
- store config's tree hash in /etc/rpi-config.version [0]

- rpi-fixup-rootfs-yem
- stores my ssh pub key in root's ~/.ssh/authorized_keys

- rpi-fixup-rootfs-tvheadend
- add a script to wait for DVB devices
- add a CIFS mount to fstab, to access my NAS

- rpi-fixup-rootfs-wpa
- add my wpa-supplicant conf file to connect to my LAN

So, when I build my tvheadend filesystem, I want to run scripts:
rpi-fixup-rootfs rpi-fixup-rootfs-yem rpi-fixup-rootfs-tvheadend

But when I want to build my mpd filesystem, I want to run scripts:
rpi-fixup-rootfs rpi-fixup-rootfs-yem rpi-fixup-rootfs-wpa

Since the scripts lie out of the buildroot tree, I set (all on one line, but broken-out for readability):

Quoted 6 lines
    BR2_ROOTFS_POST_BUILD_SCRIPT="
        $(CONFIG_DIR)/../scripts/rpi-fixup-rootfs
        $(CONFIG_DIR)/../scripts/rpi-fixup-rootfs-yem
        $(CONFIG_DIR)/../scripts/rpi-fixup-rootfs-tvheadend"

Now, I am adding a new script. This is starting to be ridiculous...

Just to be clear: what you find ridiculous is the length and non-readability of this option, is that correct?

Besides the proposal you are making below, here is an alternate one (I'm not saying it is superior, just thinking out loud): if we add an extra option like BR2_ROOTFS_POST_BUILD_SCRIPT_BASE="$(CONFIG_DIR)/../scripts/", then:
BR2_ROOTFS_POST_BUILD_SCRIPT="rpi-fixup-rootfs rpi-fixup-rootfs-yem rpi-fixup-rootfs-tvheadend".
This already is much more readable.
The logic I'd use for this is: when parsing the BR2_ROOTFS_POST_BUILD_SCRIPT, prepend BR2_ROOTFS_POST_BUILD_SCRIPT_BASE to each item. BASE can be empty (legacy) in which case the entries in the normal variable should be the full path, as is currently the case.

Quoted 16 lines

What I'd like is to be able to pass one or more options to each post-build script, so I can have a wrapper and tell it what to run:

    BR2_ROOTFS_POST_BUILD_SCRIPT="
        $(CONFIG_DIR)/../scripts/fixup-rootfs rpi ssh-key rpi-tvheadend"

to call those scripts:
fixup-rootfs-rpi
fixup-rootfs-ssh-key
fixup-rootfs-rpi-tvheadend

But it is not possible, as Buildroot interprets BR2_ROOTFS_POST_BUILD_SCRIPT as a space-separated list, not a single command. So, it is not possible to pass any option to the post-build scripts.

Quoted 7 lines

So, I was going to change it from space-separated to colon-separated. But that would break existing configuration. Sigh... :-(

I can see only one way to solve this: rename the variable (and add to legacy), and make it a colon-separated list.

Opinions?

It does indeed solve your use case. I'm not really against it, but there certainly are other possibilities. Above I gave one, here's another:
Instead of passing multiple scripts, you could just pass one wrapper post build script, without options. This wrapper would be different in your different configurations, and would call all relevant scripts in turn. Adding a script for a certain configuration now no longer means updating the buildroot configuration, but instead updating the wrapper script.
So for example you'd have:

fixup-rootfs-tvheadend:
#!/bin/sh
targetdir=$1
CONFIG_DIR=<to be hardcoded here, or imported from elsewhere> $CONFIG_DIR/../scripts/rpi-fixup-rootfs $targetdir $CONFIG_DIR/../scripts/rpi-fixup-rootfs-yem $targetdir $CONFIG_DIR/../scripts/rpi-fixup-rootfs-tvheadend $targetdir

fixup-rootfs-mpd:
#!/bin/sh
targetdir=$1
CONFIG_DIR=<to be hardcoded here, or imported from elsewhere> $CONFIG_DIR/../scripts/rpi-fixup-rootfs $targetdir $CONFIG_DIR/../scripts/rpi-fixup-rootfs-yem $targetdir $CONFIG_DIR/../scripts/rpi-fixup-rootfs-wpa $targetdir

With this alternative, there is no need to change anything in buildroot itself, there is just a change in your workflow.

Again, I'm not saying this is superior, just opening up the discussion...

Best regards,
Thomas

#2 lore raw

Yann, Thomas,

Quoted 6 lines · Thomas De Schampheleire

Instead of passing multiple scripts, you could just pass one wrapper post build script, without options. This wrapper would be different in your different configurations, and would call all relevant scripts in turn. Adding a script for a certain configuration now no longer means updating the buildroot configuration, but instead updating the wrapper script.

This is absolutely what should be done in Yann's case. There's no need to make the post-build script mechanism in Buildroot more complicated, just handle the complexity in our own scripts, it is much more flexible.

Best regards,

Thomas

Signature · 4 lines
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
#3 lore raw
In reply to Thomas De Schampheleire
Quoted 4 lines · Thomas De Schampheleire

Instead of passing multiple scripts, you could just pass one wrapper post build script, without options. This wrapper would be different in your different configurations, and would call all relevant scripts in turn.

Or alternatively: a wrapper script that observes $0 and decides which
other scripts to call based on that. So you'd have rpi-fixup-rootfs at yem@tvheadend for the one configuration, and rpi-fixup-rootfs at yem@wpa for the other one.

Regards,
Arnout

Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
#4 lore raw
Quoted 8 lines · Arnout Vandecappelle
Quoted 5 lines · Thomas De Schampheleire

Instead of passing multiple scripts, you could just pass one wrapper post build script, without options. This wrapper would be different in your different configurations, and would call all relevant scripts in turn.

Quoted 5 lines

Or alternatively: a wrapper script that observes $0 and decides which other
scripts to call based on that. So you'd have
rpi-fixup-rootfs at yem@tvheadend for the one configuration, and rpi-fixup-rootfs at yem@wpa for the other one.

But this requires a set of symbolic links matching each different configuration, right?