post-build scripts: limitations, and proposal
Hello All!
Currently, I'm using a set of post-build scripts, and it's becoming to be slightly tedious.
For example, here are a few of them:
- rpi-fixup-rootfs
- adds /dev/mmcblk0p1 on /boot to fstab, don't mount _netdev filesystems
- set root's PS1
- store config's tree hash in /etc/rpi-config.version [0]
- rpi-fixup-rootfs-yem
- stores my ssh pub key in root's ~/.ssh/authorized_keys
- rpi-fixup-rootfs-tvheadend
- add a script to wait for DVB devices
- add a CIFS mount to fstab, to access my NAS
- rpi-fixup-rootfs-wpa
- add my wpa-supplicant conf file to connect to my LAN
So, when I build my tvheadend filesystem, I want to run scripts:
rpi-fixup-rootfs rpi-fixup-rootfs-yem rpi-fixup-rootfs-tvheadend
But when I want to build my mpd filesystem, I want to run scripts:
rpi-fixup-rootfs rpi-fixup-rootfs-yem rpi-fixup-rootfs-wpa
Since the scripts lie out of the buildroot tree, I set (all on one line, but broken-out for readability):
BR2_ROOTFS_POST_BUILD_SCRIPT="
$(CONFIG_DIR)/../scripts/rpi-fixup-rootfs
$(CONFIG_DIR)/../scripts/rpi-fixup-rootfs-yem
$(CONFIG_DIR)/../scripts/rpi-fixup-rootfs-tvheadend"Now, I am adding a new script. This is starting to be ridiculous...
What I'd like is to be able to pass one or more options to each post-build script, so I can have a wrapper and tell it what to run:
BR2_ROOTFS_POST_BUILD_SCRIPT="
$(CONFIG_DIR)/../scripts/fixup-rootfs rpi ssh-key rpi-tvheadend"to call those scripts:
fixup-rootfs-rpi
fixup-rootfs-ssh-key
fixup-rootfs-rpi-tvheadend
But it is not possible, as Buildroot interprets BR2_ROOTFS_POST_BUILD_SCRIPT as a space-separated list, not a single command. So, it is not possible to pass any option to the post-build scripts.
So, I was going to change it from space-separated to colon-separated. But that would break existing configuration. Sigh... :-(
I can see only one way to solve this: rename the variable (and add to legacy), and make it a colon-separated list.
Opinions?
[0] My config files (and scripts...) are stored in a git tree, so I want to know which to rebuild only by looking at a file on a running board. Having only the Buildroot tree hash is not enough.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
Hi Yann,
Quoted 32 lines · Yann E. MORIN
Hello All!
Currently, I'm using a set of post-build scripts, and it's becoming to be slightly tedious.
For example, here are a few of them:
- rpi-fixup-rootfs
- adds/dev/mmcblk0p1on /boot to fstab, don't mount_netdevfilesystems
- set root's PS1
- store config's tree hash in/etc/rpi-config.version[0]- rpi-fixup-rootfs-yem
- stores my ssh pub key in root's ~/.ssh/authorized_keys- rpi-fixup-rootfs-tvheadend
- add a script to wait for DVB devices
- add a CIFS mount to fstab, to access my NAS- rpi-fixup-rootfs-wpa
- add my wpa-supplicant conf file to connect to my LANSo, when I build my tvheadend filesystem, I want to run scripts:
rpi-fixup-rootfs rpi-fixup-rootfs-yem rpi-fixup-rootfs-tvheadendBut when I want to build my mpd filesystem, I want to run scripts:
rpi-fixup-rootfs rpi-fixup-rootfs-yem rpi-fixup-rootfs-wpaSince the scripts lie out of the buildroot tree, I set (all on one line, but broken-out for readability):
Quoted 6 lines
BR2_ROOTFS_POST_BUILD_SCRIPT=" $(CONFIG_DIR)/../scripts/rpi-fixup-rootfs $(CONFIG_DIR)/../scripts/rpi-fixup-rootfs-yem $(CONFIG_DIR)/../scripts/rpi-fixup-rootfs-tvheadend"Now, I am adding a new script. This is starting to be ridiculous...
Just to be clear: what you find ridiculous is the length and non-readability of this option, is that correct?
Besides the proposal you are making below, here is an alternate one (I'm not saying it is superior, just thinking out loud): if we add an extra option like BR2_ROOTFS_POST_BUILD_SCRIPT_BASE="$(CONFIG_DIR)/../scripts/", then:
BR2_ROOTFS_POST_BUILD_SCRIPT="rpi-fixup-rootfs rpi-fixup-rootfs-yem rpi-fixup-rootfs-tvheadend".
This already is much more readable.
The logic I'd use for this is: when parsing the BR2_ROOTFS_POST_BUILD_SCRIPT, prepend BR2_ROOTFS_POST_BUILD_SCRIPT_BASE to each item. BASE can be empty (legacy) in which case the entries in the normal variable should be the full path, as is currently the case.
Quoted 16 lines
What I'd like is to be able to pass one or more options to each post-build script, so I can have a wrapper and tell it what to run:
BR2_ROOTFS_POST_BUILD_SCRIPT=" $(CONFIG_DIR)/../scripts/fixup-rootfs rpi ssh-key rpi-tvheadend"to call those scripts:
fixup-rootfs-rpi
fixup-rootfs-ssh-key
fixup-rootfs-rpi-tvheadendBut it is not possible, as Buildroot interprets
BR2_ROOTFS_POST_BUILD_SCRIPTas a space-separated list, not a single command. So, it is not possible to pass any option to the post-build scripts.
Quoted 7 lines
So, I was going to change it from space-separated to colon-separated. But that would break existing configuration. Sigh... :-(
I can see only one way to solve this: rename the variable (and add to legacy), and make it a colon-separated list.
Opinions?
It does indeed solve your use case. I'm not really against it, but there certainly are other possibilities. Above I gave one, here's another:
Instead of passing multiple scripts, you could just pass one wrapper post build script, without options. This wrapper would be different in your different configurations, and would call all relevant scripts in turn. Adding a script for a certain configuration now no longer means updating the buildroot configuration, but instead updating the wrapper script.
So for example you'd have:
fixup-rootfs-tvheadend:
#!/bin/sh
targetdir=$1CONFIG_DIR=<to be hardcoded here, or imported from elsewhere> $CONFIG_DIR/../scripts/rpi-fixup-rootfs $targetdir $CONFIG_DIR/../scripts/rpi-fixup-rootfs-yem $targetdir $CONFIG_DIR/../scripts/rpi-fixup-rootfs-tvheadend $targetdir
fixup-rootfs-mpd:
#!/bin/sh
targetdir=$1CONFIG_DIR=<to be hardcoded here, or imported from elsewhere> $CONFIG_DIR/../scripts/rpi-fixup-rootfs $targetdir $CONFIG_DIR/../scripts/rpi-fixup-rootfs-yem $targetdir $CONFIG_DIR/../scripts/rpi-fixup-rootfs-wpa $targetdir
With this alternative, there is no need to change anything in buildroot itself, there is just a change in your workflow.
Again, I'm not saying this is superior, just opening up the discussion...
Best regards,
Thomas
Yann, Thomas,
Quoted 6 lines · Thomas De Schampheleire
Instead of passing multiple scripts, you could just pass one wrapper post build script, without options. This wrapper would be different in your different configurations, and would call all relevant scripts in turn. Adding a script for a certain configuration now no longer means updating the buildroot configuration, but instead updating the wrapper script.
This is absolutely what should be done in Yann's case. There's no need to make the post-build script mechanism in Buildroot more complicated, just handle the complexity in our own scripts, it is much more flexible.
Best regards,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
Quoted 4 lines · Thomas De Schampheleire
Instead of passing multiple scripts, you could just pass one wrapper post build script, without options. This wrapper would be different in your different configurations, and would call all relevant scripts in turn.
Or alternatively: a wrapper script that observes $0 and decides which
other scripts to call based on that. So you'd have rpi-fixup-rootfs at yem@tvheadend for the one configuration, and rpi-fixup-rootfs at yem@wpa for the other one.
Regards,
Arnout
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
Quoted 8 lines · Arnout Vandecappelle
Quoted 5 lines · Thomas De Schampheleire
Instead of passing multiple scripts, you could just pass one wrapper post build script, without options. This wrapper would be different in your different configurations, and would call all relevant scripts in turn.
Quoted 5 lines
Or alternatively: a wrapper script that observes $0 and decides which other
scripts to call based on that. So you'd have
rpi-fixup-rootfs at yem@tvheadend for the one configuration, and rpi-fixup-rootfs at yem@wpa for the other one.
But this requires a set of symbolic links matching each different configuration, right?