udev providers: compile the hwdb at build time (branch yem/host-eudev-2)
v2 cover udev providers: compile the hwdb at build time (branch yem/host-eudev-2)
Hello All!
The *.hwdb files installed by eudev or systemd are "source" files; they need to be compiled into a binary form to be useful.
This needs udevadm to be available, so we introduce a eudev host variant to provide it, and use that from both eudev and systemd to compile the database during target-finalize.
---
Changes v1 -> v2:
- do a custom configure and install of host-eudev, to simplify the hooks (Carlos)
Regards,
Yann E. MORIN.
The following changes since commit 1f89ab5c0cf44017955ebf72c89c31df60b1b0a5
package/uclibc-ng-test: fix build with latest glibc (2019-10-29 22:55:36 +0100)
are available in the git repository at:
git://git.buildroot.org/~ymorin/git/buildroot.git
for you to fetch changes up to 803cedc8ae9597d0ae287e65d873cb9e0f6a91d9
packagesystemd: generate the hwdb.bin (2019-10-29 23:06:24 +0100)
----------------------------------------------------------------
Yann E. MORIN (3):
package/eudev: introduce a host variant
package/eudev: generate the hwdb.bin
packagesystemd: generate the hwdb.bin
package/Config.in.host | 1 + package/eudev/Config.in | 3 ++- package/eudev/Config.in.host | 3 +++ package/eudev/eudev.mk | 36 ++++++++++++++++++++++++++++++++++++ package/systemd/Config.in | 1 + 5 files changed, 43 insertions(+), 1 deletion(-) create mode 100644 package/eudev/Config.in.host
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH v2 1/3 package/eudev: introduce a host variant
eudev and systemd provide a hardware database (hwdb) as a set of multiple files. Various other utilities may also use that database.
Those files have to be "compiled" into a binary to be useful; libudev (and thus all utilities based on it) only use the compiled hwdb.
Compiling the hwdb is done with udevadm, using the hwdb sub-command:
udevadm hwdb --update
Provide a simple host-variant of eudev, so that we can call udevadm at build time.
When it is configured, eudev will shoehorn its --prefix path as the base location where the .hwdb file will be searched from, as well as where the hwdb.bin will be generated in. This means that with the usual --prefix=$(HOST_DIR), it would look into there.
udevadm also accepts a --root=/path option at runtime, which prepends /path to all the paths it uses to find and generate files.
Obviously, combining --root=$(TARGET_DIR) and --prefix=$(HOST_DIR) would not do what we want: all files would be searched for, and generated, in $(HOST_DIR)$(TARGET_DIR)/ . Avoiding use of --root would not help much either, as files would still searched in $(HOST_DIR) (we could use a trick to copy files there, generate and then move the hwdb.bin, but that's not nice).
However, since we only need udevadm, and since udevadm has no internal and no external dependency, we can use a less dirty trick, an configure host-eudev with --prefix=/usr (and similar for the other paths), manually copy udevadm to HOST_DIR, and then use --root when calling it.
Then, we get a udevadm that can read files from, and generate files into $(TARGET_DIR). We register a target-finalize hook to generate the hwdb.bin, so that any pakage may install its .hwdb files (currently only eudev and systemd do, but other packages might (e.g. sane is known to do so on standard desktop distros))
The *.hwdb source files consume a lot of space, roughly the same as the generated database, i.e. ~8MiB as of today, and they are totally useless on the target; only the generated hwdb.bin is useful. So we want to get rid of them.
However, we also want to be able to complete a build (e.g. make foo-reinstall to reinstall more hwdb files), so we don't want to irremediably lose them. As such, we register a pre-rootfs-cmd hook, that removes them just before assembling the filesystems, when we're only using a copy of the target directory.
Note that this is the first host package to register a target-finalize hook, and also the first to register a pre-rootfs-cmd hook. This avoids duplicating these hooks logic in both eudev and systemd.
---
Changes v1 -> v2:
- tweak and trick the configure and install (Carlos)
- include the hooks to avoid duplication (Peter)
- expand the commit log accordingly
--- package/Config.in.host | 1 + package/eudev/Config.in.host | 3 +++ package/eudev/eudev.mk | 36 ++++++++++++++++++++++++++++++++++++ 3 files changed, 40 insertions(+) create mode 100644 package/eudev/Config.in.host
package/Config.in.host +1 -0
@@ -17,6 +17,7 @@ menu "Host utilities" source "package/dtc/Config.in.host" source "package/e2fsprogs/Config.in.host" source "package/e2tools/Config.in.host" + source "package/eudev/Config.in.host" source "package/f2fs-tools/Config.in.host" source "package/faketime/Config.in.host" source "package/fatcat/Config.in.host"
package/eudev/Config.in.host new file +3 -0
@@ -0,0 +1,3 @@ +# Select this if you need host eudev tools (e.g. udevadm) +config BR2_PACKAGE_HOST_EUDEV + bool
package/eudev/eudev.mk +36 -0
@@ -55,4 +55,40 @@ define EUDEV_USERS - - kvm -1 * - - - kvm nodes endef +HOST_EUDEV_DEPENDENCIES = host-gperf host-pkgconf + +HOST_EUDEV_SYSCONFDIR = $(if $(BR2_PACKAGE_SYSTEMD),/usr/lib,/etc) +HOST_EUDEV_CONF_OPTS = \ + --prefix=/usr \ + --sbindir=/sbin \ + --libexecdir=/lib \ + --with-rootlibdir=/lib \ + --sysconfdir=$(HOST_EUDEV_SYSCONFDIR) \ + --disable-blkid \ + --disable-introspection \ + --disable-kmod \ + --disable-manpages \ + --disable-rule-generator \ + --disable-selinux \ + --enable-hwdb + +define HOST_EUDEV_INSTALL_CMDS + $(INSTALL) -D -m 0755 $(@D)/src/udev/udevadm \ + $(HOST_DIR)/bin/udevadm +endef + +define HOST_EUDEV_BUILD_HWDB + $(HOST_DIR)/bin/udevadm hwdb --update --root $(TARGET_DIR) +endef +HOST_EUDEV_TARGET_FINALIZE_HOOKS += HOST_EUDEV_BUILD_HWDB + +# Note: this will run in the filesystem context, so will use a copy +# of tharget/, not the real one, so the files are still available on +# re-builds (foo-rebuild, etc...) +define HOST_EUDEV_RM_HWDB_SRC + rm -rf $(TARGET_DIR)/$(HOST_EUDEV_SYSCONFDIR)/udev/hwdb.d/ +endef +HOST_EUDEV_ROOTFS_PRE_CMD_HOOKS += HOST_EUDEV_RM_HWDB_SRC + $(eval $(autotools-package)) +$(eval $(host-autotools-package))
Signature · 1 line
2.20.1
PATCH v2 1/3 package/eudev: introduce a host variant
Quoted 99 lines · Yann E. MORIN
eudev and systemd provide a hardware database (hwdb) as a set of multiple files. Various other utilities may also use that database.
Those files have to be "compiled" into a binary to be useful; libudev (and thus all utilities based on it) only use the compiled hwdb.
Compiling the hwdb is done with udevadm, using the hwdb sub-command:
udevadm hwdb --updateProvide a simple host-variant of eudev, so that we can call udevadm at build time.
When it is configured, eudev will shoehorn its --prefix path as the base location where the .hwdb file will be searched from, as well as where the hwdb.bin will be generated in. This means that with the usual --prefix=$(HOST_DIR), it would look into there.
udevadm also accepts a --root=/path option at runtime, which prepends /path to all the paths it uses to find and generate files.
Obviously, combining --root=$(TARGET_DIR) and --prefix=$(HOST_DIR) would not do what we want: all files would be searched for, and generated, in $(HOST_DIR)$(TARGET_DIR)/ . Avoiding use of --root would not help much either, as files would still searched in $(HOST_DIR) (we could use a trick to copy files there, generate and then move the hwdb.bin, but that's not nice).
However, since we only need udevadm, and since udevadm has no internal and no external dependency, we can use a less dirty trick, an configure host-eudev with --prefix=/usr (and similar for the other paths), manually copy udevadm to
HOST_DIR, and then use --root when calling it.Then, we get a udevadm that can read files from, and generate files into $(TARGET_DIR). We register a target-finalize hook to generate the hwdb.bin, so that any pakage may install its .hwdb files (currently only eudev and systemd do, but other packages might (e.g. sane is known to do so on standard desktop distros))
The *.hwdb source files consume a lot of space, roughly the same as the generated database, i.e. ~8MiB as of today, and they are totally useless on the target; only the generated hwdb.bin is useful. So we want to get rid of them.
However, we also want to be able to complete a build (e.g. make foo-reinstall to reinstall more hwdb files), so we don't want to irremediably lose them. As such, we register a pre-rootfs-cmd hook, that removes them just before assembling the filesystems, when we're only using a copy of the target directory.
Note that this is the first host package to register a target-finalize hook, and also the first to register a pre-rootfs-cmd hook. This avoids duplicating these hooks logic in both eudev and systemd.
Signed-off-by Yann E. MORIN <[email protected]>Cc Peter Korsgaard <[email protected]>Cc Thomas Petazzoni <[email protected]>Cc Carlos Santos <[email protected]>---
Changes v1 -> v2:
- tweak and trick the configure and install (Carlos)
- include the hooks to avoid duplication (Peter)
- expand the commit log accordingly--- package/Config.in.host | 1 + package/eudev/Config.in.host | 3 +++ package/eudev/eudev.mk | 36 ++++++++++++++++++++++++++++++++++++ 3 files changed, 40 insertions(+) create mode 100644 package/eudev/Config.in.hostpackage/Config.in.host +1
-0@@ -17,6 +17,7 @@ menu "Host utilities" source "package/dtc/Config.in.host" source "package/e2fsprogs/Config.in.host" source "package/e2tools/Config.in.host" + source "package/eudev/Config.in.host" source "package/f2fs-tools/Config.in.host" source "package/faketime/Config.in.host" source "package/fatcat/Config.in.host"package/eudev/Config.in.host new file +3
-0@@ -0,0 +1,3 @@ +# Select this if you need host eudev tools (e.g. udevadm) +config BR2_PACKAGE_HOST_EUDEV + boolpackage/eudev/eudev.mk +0
-0@@ -55,4 +55,40 @@ define EUDEV_USERS - - kvm -1 * - - - kvm nodes endef
package/eudev/eudev.mk define EUDEV_USERS · line 66
Quoted 10 lines
package/eudev/eudev.mk +9
-0+HOST_EUDEV_DEPENDENCIES = host-gperf host-pkgconf + +HOST_EUDEV_SYSCONFDIR = $(if $(BR2_PACKAGE_SYSTEMD),/usr/lib,/etc) +HOST_EUDEV_CONF_OPTS = \ + --prefix=/usr \ + --sbindir=/sbin \ + --libexecdir=/lib \ + --with-rootlibdir=/lib \ + --sysconfdir=$(HOST_EUDEV_SYSCONFDIR) \
We don't actually need this, since /etc/udev/udevdb.bin is a valid location for systemd too. It was my idea, but I take it back. :-)
Quoted 31 lines
package/eudev/eudev.mk +27
-1+ --disable-blkid \ + --disable-introspection \ + --disable-kmod \ + --disable-manpages \ + --disable-rule-generator \ + --disable-selinux \ + --enable-hwdb + +define HOST_EUDEV_INSTALL_CMDS + $(INSTALL) -D -m 0755 $(@D)/src/udev/udevadm \ + $(HOST_DIR)/bin/udevadm +endef + +define HOST_EUDEV_BUILD_HWDB + $(HOST_DIR)/bin/udevadm hwdb --update --root $(TARGET_DIR) +endef +HOST_EUDEV_TARGET_FINALIZE_HOOKS += HOST_EUDEV_BUILD_HWDB + +# Note: this will run in the filesystem context, so will use a copy +# of tharget/, not the real one, so the files are still available on +# re-builds (foo-rebuild, etc...) +define HOST_EUDEV_RM_HWDB_SRC + rm -rf $(TARGET_DIR)/$(HOST_EUDEV_SYSCONFDIR)/udev/hwdb.d/ +endef +HOST_EUDEV_ROOTFS_PRE_CMD_HOOKS += HOST_EUDEV_RM_HWDB_SRC + $(eval $(autotools-package)) +$(eval $(host-autotools-package)) --2.20.1
Signature · 1 line
Carlos Santos <[email protected]>
PATCH v2 1/3 package/eudev: introduce a host variant
Quoted 7 lines
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Carlos" == Carlos Santos <[email protected]> writes:
+++
b/package/eudev/eudev.mk(quoted diff) +0
-1@@ -55,4 +55,40 @@ define EUDEV_USERS - - kvm -1 * - - - kvm nodesendef
Quoted 12 lines
package/eudev/eudev.mk define EUDEV_USERS · line 66
Quoted 9 lines
package/eudev/eudev.mk +9
-0+HOST_EUDEV_DEPENDENCIES = host-gperf host-pkgconf + +HOST_EUDEV_SYSCONFDIR = $(if $(BR2_PACKAGE_SYSTEMD),/usr/lib,/etc) +HOST_EUDEV_CONF_OPTS = \ + --prefix=/usr \ + --sbindir=/sbin \ + --libexecdir=/lib \ + --with-rootlibdir=/lib \ + --sysconfdir=$(HOST_EUDEV_SYSCONFDIR) \We don't actually need this, since
/etc/udev/udevdb.binis a valid location for systemd too. It was my idea, but I take it back. :-)
Are you sure that works? The sysconfdir is used both for the input and output location. If you use systemd then the files provided by systemd-udev presumably gets installed under /usr/lib/udev/hwdb.d as well?
Signature · 1 line
Bye, Peter Korsgaard
PATCH v2 1/3 package/eudev: introduce a host variant
In reply to Yann E. MORINQuoted 51 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
eudev and systemd provide a hardware database (hwdb) as a set of multiple files. Various other utilities may also use that database.
Those files have to be "compiled" into a binary to be useful; libudev (and thus all utilities based on it) only use the compiled hwdb.
Compiling the hwdb is done with udevadm, using the hwdb sub-command:
udevadm hwdb --updateProvide a simple host-variant of eudev, so that we can call udevadm at build time.
When it is configured, eudev will shoehorn its --prefix path as the base location where the .hwdb file will be searched from, as well as where the hwdb.bin will be generated in. This means that with the usual --prefix=$(HOST_DIR), it would look into there.
udevadm also accepts a --root=/path option at runtime, which prepends /path to all the paths it uses to find and generate files.
Obviously, combining --root=$(TARGET_DIR) and --prefix=$(HOST_DIR) would not do what we want: all files would be searched for, and generated, in $(HOST_DIR)$(TARGET_DIR)/ . Avoiding use of --root would not help much either, as files would still searched in $(HOST_DIR) (we could use a trick to copy files there, generate and then move the hwdb.bin, but that's not nice).
However, since we only need udevadm, and since udevadm has no internal and no external dependency, we can use a less dirty trick, an configure host-eudev with --prefix=/usr (and similar for the other paths), manually copy udevadm to
HOST_DIR, and then use --root when calling it.Then, we get a udevadm that can read files from, and generate files into $(TARGET_DIR). We register a target-finalize hook to generate the hwdb.bin, so that any pakage may install its .hwdb files (currently only eudev and systemd do, but other packages might (e.g. sane is known to do so on standard desktop distros))
The *.hwdb source files consume a lot of space, roughly the same as the generated database, i.e. ~8MiB as of today, and they are totally useless on the target; only the generated hwdb.bin is useful. So we want to get rid of them.
However, we also want to be able to complete a build (e.g. make foo-reinstall to reinstall more hwdb files), so we don't want to irremediably lose them. As such, we register a pre-rootfs-cmd hook, that removes them just before assembling the filesystems, when we're only using a copy of the target directory.
Quoted 8 lines
Note that this is the first host package to register a target-finalize hook, and also the first to register a pre-rootfs-cmd hook. This avoids duplicating these hooks logic in both eudev and systemd.
Signed-off-by Yann E. MORIN <[email protected]>Cc Peter Korsgaard <[email protected]>Cc Thomas Petazzoni <[email protected]>Cc Carlos Santos <[email protected]>
Committed, thanks.
Signature · 1 line
Bye, Peter Korsgaard
PATCH v2 1/3 package/eudev: introduce a host variant
In reply to Yann E. MORINQuoted 60 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
eudev and systemd provide a hardware database (hwdb) as a set of multiple files. Various other utilities may also use that database.
Those files have to be "compiled" into a binary to be useful; libudev (and thus all utilities based on it) only use the compiled hwdb.
Compiling the hwdb is done with udevadm, using the hwdb sub-command:
udevadm hwdb --updateProvide a simple host-variant of eudev, so that we can call udevadm at build time.
When it is configured, eudev will shoehorn its --prefix path as the base location where the .hwdb file will be searched from, as well as where the hwdb.bin will be generated in. This means that with the usual --prefix=$(HOST_DIR), it would look into there.
udevadm also accepts a --root=/path option at runtime, which prepends /path to all the paths it uses to find and generate files.
Obviously, combining --root=$(TARGET_DIR) and --prefix=$(HOST_DIR) would not do what we want: all files would be searched for, and generated, in $(HOST_DIR)$(TARGET_DIR)/ . Avoiding use of --root would not help much either, as files would still searched in $(HOST_DIR) (we could use a trick to copy files there, generate and then move the hwdb.bin, but that's not nice).
However, since we only need udevadm, and since udevadm has no internal and no external dependency, we can use a less dirty trick, an configure host-eudev with --prefix=/usr (and similar for the other paths), manually copy udevadm to
HOST_DIR, and then use --root when calling it.Then, we get a udevadm that can read files from, and generate files into $(TARGET_DIR). We register a target-finalize hook to generate the hwdb.bin, so that any pakage may install its .hwdb files (currently only eudev and systemd do, but other packages might (e.g. sane is known to do so on standard desktop distros))
The *.hwdb source files consume a lot of space, roughly the same as the generated database, i.e. ~8MiB as of today, and they are totally useless on the target; only the generated hwdb.bin is useful. So we want to get rid of them.
However, we also want to be able to complete a build (e.g. make foo-reinstall to reinstall more hwdb files), so we don't want to irremediably lose them. As such, we register a pre-rootfs-cmd hook, that removes them just before assembling the filesystems, when we're only using a copy of the target directory.
Note that this is the first host package to register a target-finalize hook, and also the first to register a pre-rootfs-cmd hook. This avoids duplicating these hooks logic in both eudev and systemd.
Signed-off-by Yann E. MORIN <[email protected]>Cc Peter Korsgaard <[email protected]>Cc Thomas Petazzoni <[email protected]>Cc Carlos Santos <[email protected]>
Quoted 5 lines
---
Changes v1 -> v2:
- tweak and trick the configure and install (Carlos)
- include the hooks to avoid duplication (Peter)
- expand the commit log accordingly
Committed to 2019.02.x and 2019.08.x, thanks.
Signature · 1 line
Bye, Peter Korsgaard
PATCH v2 2/3 package/eudev: generate the hwdb.bin
In reply to Yann E. MORIN---
Changes v1 -> v2:
- drop the post hooks
--- package/eudev/Config.in | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
package/eudev/Config.in +2 -1
@@ -29,8 +29,9 @@ config BR2_PACKAGE_EUDEV_RULES_GEN config BR2_PACKAGE_EUDEV_ENABLE_HWDB bool "enable hwdb installation" default y + select BR2_PACKAGE_HOST_EUDEV # for udevadm, during target-finalize help - Enables hardware database installation to /etc/udev/hwdb.d + Enables hardware database installation to /etc/udev/hwdb.bin endif
Signature · 1 line
2.20.1
PATCH v2 2/3 package/eudev: generate the hwdb.bin
Quoted 10 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
Signed-off-by Yann E. MORIN <[email protected]>Cc Peter Korsgaard <[email protected]>Cc Thomas Petazzoni <[email protected]>Cc Carlos Santos <[email protected]>---
Changes v1 -> v2:
- drop the post hooks
Committed, thanks.
Signature · 1 line
Bye, Peter Korsgaard
PATCH v2 2/3 package/eudev: generate the hwdb.bin
In reply to Yann E. MORINQuoted 10 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
Signed-off-by Yann E. MORIN <[email protected]>Cc Peter Korsgaard <[email protected]>Cc Thomas Petazzoni <[email protected]>Cc Carlos Santos <[email protected]>---
Changes v1 -> v2:
- drop the post hooks
Committed to 2019.02.x and 2019.08.x, thanks.
Signature · 1 line
Bye, Peter Korsgaard
PATCH v2 3/3 packagesystemd: generate the hwdb.bin
In reply to Yann E. MORIN---
Changes v1 -> v2:
- drop the post hooks
--- package/systemd/Config.in | 1 + 1 file changed, 1 insertion(+)
package/systemd/Config.in +1 -0
@@ -199,6 +199,7 @@ config BR2_PACKAGE_SYSTEMD_HOSTNAMED config BR2_PACKAGE_SYSTEMD_HWDB bool "enable hwdb installation" default y + select BR2_PACKAGE_HOST_EUDEV # for udevadm, during target-finalize help Enables hardware database installation to /usr/lib/udev/hwdb.d
Signature · 1 line
2.20.1
PATCH v2 3/3 packagesystemd: generate the hwdb.bin
Quoted 10 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
Signed-off-by Yann E. MORIN <[email protected]>Cc Peter Korsgaard <[email protected]>Cc Thomas Petazzoni <[email protected]>Cc Carlos Santos <[email protected]>---
Changes v1 -> v2:
- drop the post hooks
Committed, thanks.
Signature · 1 line
Bye, Peter Korsgaard
PATCH v2 3/3 packagesystemd: generate the hwdb.bin
In reply to Yann E. MORINQuoted 10 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
Signed-off-by Yann E. MORIN <[email protected]>Cc Peter Korsgaard <[email protected]>Cc Thomas Petazzoni <[email protected]>Cc Carlos Santos <[email protected]>---
Changes v1 -> v2:
- drop the post hooks
Committed to 2019.02.x and 2019.08.x, thanks.
Signature · 1 line
Bye, Peter Korsgaard