package/pkg-generic.mk: fix absolute paths in per-package config files

PATCH In review Adam Duskett 2 participants 2 messages Jan 6, 2022 – Jan 8, 2022

Share & tools

Adam Duskett

author

2022-01-06 17:17 UTC #0 lore raw

PATCH v4 1/1 package/pkg-generic.mk: fix absolute paths in per-package config files

Some packages (notably qmake packages) generate .cmake, .pc, and .pri files with absolute paths. Unfortunately, doing so breaks per-package builds because the paths in those files point outside the per-package sysroot for packages that have rsynced those same files.

- Rename FIXUP_PYTHON_SYSCONFIGDATA to FIXUP_PER_PACKAGE_FILES in pkg-generic.mk to avoid more calls to ifeq ($(BR2_PER_PACKAGE_DIRECTORIES),y)

- Add .cmake, .pc, and .pri files to the files of which to find to sed the per-package paths.

This simple fix allows qt5 submodule packages to build with per-package enabled.

Based off of Louis-Paul's earlier work found here:
https://lore.kernel.org/buildroot/[email protected]/

And Yann Morin's feedback found here:
https://patchwork.ozlabs.org/project/buildroot/patch/[email protected]/

Signed-off-by Adam Duskett <[email protected]>

---
changes v3 -> v4:
- Further compress the method by simply adding the files to the already existing find command. (Yann)

changes v2 -> v3:
- Rename FIXUP_PYTHON_SYSCONFIGDATA to FIXUP_PER_PACKAGE_FILES in pkg-generic.mk (Yann)

- Combine FIXUP_CONFIG_FILES_ABSOLUTE_PATHS with FIXUP_PER_PACKAGE_FILES (Yann)

changes v1 -> v2
- Drop patches 2 and 3 and combine them into a single macro in pkg-generic.mk
- Move the PRE_CONFIGURE_HOOK to inner-generic-package so it actually runs
- Rename FIXUP_PC_FILES_ABSOLUTE_PATHS to FIXUP_CONFIG_FILES_ABSOLUTE_PATHS
- Use $(HOST_DIR) instead of $(STAGING_DIR) to cover both stagin and host directories in a single step.

 package/pkg-generic.mk | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

package/pkg-generic.mk +13 -2

index 6a5fe5507b..9952ff1963 100644
@@ -113,8 +113,16 @@ endif
 # fail.
 # So we just use HOST_DIR as a starting point, and filter on the two directories
 # of interest.
+#
+# When per-package is enabled, packages may generate files with absolute paths
+# that point to the packages per-package sysroot. However, this breaks
+# per-package isolation when the files are rsynced to a new package that depends
+# on the first package. because the hard-coded paths point to directories
+# outside the packages sysroot. Run sed on all .cmake, .pc, and .pri files in a
+# given packages sysroot containing the word "per-package" and replace the path
+# with the appropriate per-package directory.
 ifeq ($(BR2_PER_PACKAGE_DIRECTORIES),y)
-define FIXUP_PYTHON_SYSCONFIGDATA
+define FIXUP_PER_PACKAGE_FILES
 	$(Q)find $(HOST_DIR) \
 		\(    -path '$(HOST_DIR)/lib/python*' \
 		   -o -path '$(STAGING_DIR)/usr/lib/python*' \

@@ -122,6 +130,9 @@ define FIXUP_PYTHON_SYSCONFIGDATA
 		\(    \( -name "_sysconfigdata*.pyc" -delete \) \
 		   -o \( -name "_sysconfigdata*.py" -print0 \) \
 		\) \
+		-o -name "*.cmake" -print0 \
+		-o -name "*.pc" -print0 \
+		-o -name "*.pri" -print0 \
 	| xargs -0 --no-run-if-empty \
 		$(SED) 's:$(PER_PACKAGE_DIR)/[^/]\+/:$(PER_PACKAGE_DIR)/$($(PKG)_NAME)/:g'
 endef

@@ -836,7 +847,7 @@ $(2)_EXTRACT_CMDS ?= \
 		$$(TAR_OPTIONS) -)
 
 # pre/post-steps hooks
-$(2)_POST_PREPARE_HOOKS += FIXUP_PYTHON_SYSCONFIGDATA
+$(2)_POST_PREPARE_HOOKS += FIXUP_PER_PACKAGE_FILES
 
 ifeq ($$($(2)_TYPE),target)
 ifneq ($$(HOST_$(2)_KCONFIG_VAR),)

Signature · 6 lines

2.33.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

Yann E. MORIN

2022-01-08 18:22 UTC #1 lore raw

Adam, All,

On 2022-01-06 09:17 -0800, Adam Duskett spake thusly:

Quoted 13 lines

Some packages (notably qmake packages) generate .cmake, .pc, and .pri files with absolute paths. Unfortunately, doing so breaks per-package builds because the paths in those files point outside the per-package sysroot for packages that have rsynced those same files.
- Rename FIXUP_PYTHON_SYSCONFIGDATA to FIXUP_PER_PACKAGE_FILES in pkg-generic.mk to avoid more calls to ifeq ($(BR2_PER_PACKAGE_DIRECTORIES),y)
- Add .cmake, .pc, and .pri files to the files of which to find to sed the per-package paths.
This simple fix allows qt5 submodule packages to build with per-package enabled.

Quoted 7 lines

Based off of Louis-Paul's earlier work found here:
https://lore.kernel.org/buildroot/[email protected]/
And Yann Morin's feedback found here:
https://patchwork.ozlabs.org/project/buildroot/patch/[email protected]/
Signed-off-by Adam Duskett <[email protected]>

Thanks for this iteration!

While discussing with the otehr maintainers int he room, we concluded that we did want an even more generic soltuion, so I've sent a further series;

    https://lore.kernel.org/buildroot/[email protected]/T/#t

Thanks for your persistence! ;-)

Regards,
Yann E. MORIN.

Quoted 69 lines

---
changes v3 -> v4:
- Further compress the method by simply adding the files to the already existing find command. (Yann)

changes v2 -> v3:
- Rename FIXUP_PYTHON_SYSCONFIGDATA to FIXUP_PER_PACKAGE_FILES in pkg-generic.mk (Yann)

- Combine FIXUP_CONFIG_FILES_ABSOLUTE_PATHS with FIXUP_PER_PACKAGE_FILES (Yann)

 package/pkg-generic.mk | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

package/pkg-generic.mk +13 -2

index 6a5fe5507b..9952ff1963 100644
@@ -113,8 +113,16 @@ endif
 # fail.
 # So we just use HOST_DIR as a starting point, and filter on the two directories
 # of interest.
+#
+# When per-package is enabled, packages may generate files with absolute paths
+# that point to the packages per-package sysroot. However, this breaks
+# per-package isolation when the files are rsynced to a new package that depends
+# on the first package. because the hard-coded paths point to directories
+# outside the packages sysroot. Run sed on all .cmake, .pc, and .pri files in a
+# given packages sysroot containing the word "per-package" and replace the path
+# with the appropriate per-package directory.
 ifeq ($(BR2_PER_PACKAGE_DIRECTORIES),y)
-define FIXUP_PYTHON_SYSCONFIGDATA
+define FIXUP_PER_PACKAGE_FILES
 	$(Q)find $(HOST_DIR) \
 		\(    -path '$(HOST_DIR)/lib/python*' \
 		   -o -path '$(STAGING_DIR)/usr/lib/python*' \

@@ -122,6 +130,9 @@ define FIXUP_PYTHON_SYSCONFIGDATA
 		\(    \( -name "_sysconfigdata*.pyc" -delete \) \
 		   -o \( -name "_sysconfigdata*.py" -print0 \) \
 		\) \
+		-o -name "*.cmake" -print0 \
+		-o -name "*.pc" -print0 \
+		-o -name "*.pri" -print0 \
 	| xargs -0 --no-run-if-empty \
 		$(SED) 's:$(PER_PACKAGE_DIR)/[^/]\+/:$(PER_PACKAGE_DIR)/$($(PKG)_NAME)/:g'
 endef

@@ -836,7 +847,7 @@ $(2)_EXTRACT_CMDS ?= \
 		$$(TAR_OPTIONS) -)
 
 # pre/post-steps hooks
-$(2)_POST_PREPARE_HOOKS += FIXUP_PYTHON_SYSCONFIGDATA
+$(2)_POST_PREPARE_HOOKS += FIXUP_PER_PACKAGE_FILES
 
 ifeq ($$($(2)_TYPE),target)
 ifneq ($$(HOST_$(2)_KCONFIG_VAR),)

Signature · 6 lines

2.33.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

Signature · 10 lines

.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot