Skip to contentSkip to conversation

package/qemu: force the host compiler to look at HOST_DIR first

PATCH In review Romain Naour 2 participants 3 messages Aug 18, 2023 – Aug 19, 2023

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 package/qemu: force the host compiler to look at HOST_DIR first

Since meson 1.2.0, compiler's search path are retrieved and used to link a library. If the same library is installed on the host and build by Buildroot (HOST_DIR/lib), meson now link against the one found in the compiler's search path instead of linking agains the one provided by Buildroot.

See: https://github.com/mesonbuild/meson/issues/11914#issuecomment-1678090566

Help the compiler by setting -Bprefix that specifies a path where to find the executables, libraries, include files, and data files of the compiler itself. For each subprogram to be run, the compiler driver first tries the -B prefix, if any. If that name is not found, or if -B is not specified, the driver tries two standard prefixes, /usr/lib/gcc/ and /usr/local/lib/gcc/.

Fixes:
host/lib/libgio-2.0.so: undefined reference to `g_module_open_full'
collect2: error: ld returned 1 exit statu

The problem only appear if we have two version of the glib2 library installed on the system:

[Buildroot]/host/lib/libgio-2.0.so.0.7600.1
/usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.6600.8

This is probably a meson bug introduced by [1] since the error message is wrong!

$ strings output/host/lib/libgio-2.0.so | grep g_module_open_full
g_module_open_full

The libgio-2.0.so provided by Buildroot contains the g_module_open_full symbol while the libgio-2.0.so installed on the host is too old to have it:

$ strings /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.6600.8 | grep g_module_open_full

[1] https://github.com/mesonbuild/meson/commit/59cfbf68e00aa774a9868101f423bd662938c15d

Cc Nicolas CARRIER <[email protected]>

---
This meson issue may be not specific to host-qemu...

---
 package/qemu/qemu.mk | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
package/qemu/qemu.mk +12 -1
index b341d0220d..8a8dddee58 100644
@@ -401,7 +401,18 @@ HOST_QEMU_ARCH = sh4eb
 endif
 HOST_QEMU_SYS_ARCH ?= $(HOST_QEMU_ARCH)
 
-HOST_QEMU_CFLAGS = $(HOST_CFLAGS)
+# Since meson 1.2.0, compiler's search path are retrieved and used
+# to link a library. If the same library is installed on the host
+# and build by Buildroot (HOST_DIR/lib), meson now link against
+# the one found in the compiler's search path instead of linking
+# agains the one provided by Buildroot.
+# https://github.com/mesonbuild/meson/issues/11914#issuecomment-1678090566
+# Help the compiler by setting -Bprefix that specifies where to find the
+# executables, libraries, include files, and data files of the compiler itself.
+# For each subprogram to be run, the compiler driver first tries the -B prefix,
+# if any. If that name is not found, or if -B is not specified, the driver
+# tries two standard prefixes, /usr/lib/gcc/ and /usr/local/lib/gcc/.
+HOST_QEMU_CFLAGS = $(HOST_CFLAGS) -B$(HOST_DIR)/lib
 
 ifeq ($(BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE),y)
 HOST_QEMU_TARGETS += $(HOST_QEMU_SYS_ARCH)-softmmu
Signature · 6 lines
2.41.0

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#1 lore raw

Romain, All,

+Bernd for the meson bump

On 2023-08-19 01:46 +0200, Romain Naour spake thusly:

Quoted 6 lines

Since meson 1.2.0, compiler's search path are retrieved and used to link a library. If the same library is installed on the host and build by Buildroot (HOST_DIR/lib), meson now link against the one found in the compiler's search path instead of linking agains the one provided by Buildroot.

Quoted 7 lines

See: https://github.com/mesonbuild/meson/issues/11914#issuecomment-1678090566

Help the compiler by setting -Bprefix that specifies a path where to find the executables, libraries, include files, and data files of the compiler itself. For each subprogram to be run, the compiler driver first tries the -B prefix, if any. If that name is not found, or if -B is not specified, the driver tries two standard prefixes, /usr/lib/gcc/ and /usr/local/lib/gcc/.

So, I understand that this makes qemu actually build, but I am afraid this is just hidign the problem.

Indeed, it is possible that we have other host packages that depend on some of the host libs we build and that may exist on the system, and those packages are also going to break.

For example, host-efl is a host-meson-package, and depends on host-libglib2, so presumably we would hav e the issue there too.

We can't fix the breakage by adding -B to all the host packages one by one. If -B is idneed the correct fix (I doubt it), then we will have to add it to the common HOST_CFLAGS, HOST_CXXFLAGS and so on...

However, what bothers me slighhtly, is why the qemu link step uses two different ways to link with the libglib2 libraries, e.g.:

    -l/path/to/host/lib/libgio-2.0.so
    -L/path/to/host/lib -lgmodule-2.0

So, why is libgio handled with an absolute path, while libgmodule is not? Note that libgio does have a DT_NEEDED on libgmodule.

However, let's have a look at the pkg-config files, let's start with gio-2.0.pc:

    Requires: glib-2.0, gobject-2.0
    Requires.private: gmodule-no-export-2.0, zlib, mount >= 2.23
    Libs: -L${libdir} -lgio-2.0
    Libs.private: -ldl -lresolv

OK, then gmodule-no-export-2.0.pc:

    Requires: glib-2.0
    Libs: -L${libdir} -lgmodule-2.0 -pthread
    Libs.private: -ldl

So, the two are similar. I am not sure to understand how we end up in this situation, though... :-/

In the meantime, since the bump to meson 1.2.0 is causing the issue with out an easy fix in sight, I'll revert the meson bump.

Regards,
Yann E. MORIN.

Quoted 61 lines

Fixes:
host/lib/libgio-2.0.so: undefined reference to `g_module_open_full'
collect2: error: ld returned 1 exit statu

The problem only appear if we have two version of the glib2 library installed on the system:

[Buildroot]/host/lib/libgio-2.0.so.0.7600.1
/usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.6600.8

This is probably a meson bug introduced by [1] since the error message is wrong!

$ strings output/host/lib/libgio-2.0.so | grep g_module_open_full
g_module_open_full

The libgio-2.0.so provided by Buildroot contains the g_module_open_full symbol while the libgio-2.0.so installed on the host is too old to have it:

$ strings /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.6600.8 | grep g_module_open_full

[1] https://github.com/mesonbuild/meson/commit/59cfbf68e00aa774a9868101f423bd662938c15d

Cc Nicolas CARRIER <[email protected]>

---
This meson issue may be not specific to host-qemu...

---
 package/qemu/qemu.mk | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
package/qemu/qemu.mk +12 -1
index b341d0220d..8a8dddee58 100644
@@ -401,7 +401,18 @@ HOST_QEMU_ARCH = sh4eb
 endif
 HOST_QEMU_SYS_ARCH ?= $(HOST_QEMU_ARCH)
 
-HOST_QEMU_CFLAGS = $(HOST_CFLAGS)
+# Since meson 1.2.0, compiler's search path are retrieved and used
+# to link a library. If the same library is installed on the host
+# and build by Buildroot (HOST_DIR/lib), meson now link against
+# the one found in the compiler's search path instead of linking
+# agains the one provided by Buildroot.
+# https://github.com/mesonbuild/meson/issues/11914#issuecomment-1678090566
+# Help the compiler by setting -Bprefix that specifies where to find the
+# executables, libraries, include files, and data files of the compiler itself.
+# For each subprogram to be run, the compiler driver first tries the -B prefix,
+# if any. If that name is not found, or if -B is not specified, the driver
+# tries two standard prefixes, /usr/lib/gcc/ and /usr/local/lib/gcc/.
+HOST_QEMU_CFLAGS = $(HOST_CFLAGS) -B$(HOST_DIR)/lib
 
 ifeq ($(BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE),y)
 HOST_QEMU_TARGETS += $(HOST_QEMU_SYS_ARCH)-softmmu
Signature · 6 lines
2.41.0

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#2 lore raw

Hello,

Le sam. 19 août 2023, 15:54, Yann E. MORIN <[email protected]> a écrit :

Quoted 30 lines

Romain, All,

+Bernd for the meson bump

On 2023-08-19 01:46 +0200, Romain Naour spake thusly:

Quoted 7 lines

Since meson 1.2.0, compiler's search path are retrieved and used to link a library. If the same library is installed on the host and build by Buildroot (HOST_DIR/lib), meson now link against the one found in the compiler's search path instead of linking agains the one provided by Buildroot.

See:

https://github.com/mesonbuild/meson/issues/11914#issuecomment-1678090566

Quoted 2 lines

Help the compiler by setting -Bprefix that specifies a path where to

find the

Quoted 1 line

executables, libraries, include files, and data files of the compiler

itself.

Quoted 1 line

For each subprogram to be run, the compiler driver first tries the -B

prefix,

Quoted 2 lines

if any. If that name is not found, or if -B is not specified, the driver tries two standard prefixes, /usr/lib/gcc/ and /usr/local/lib/gcc/.

So, I understand that this makes qemu actually build, but I am afraid this is just hidign the problem.

Indeed, it is possible that we have other host packages that depend on some of the host libs we build and that may exist on the system, and those packages are also going to break.

Quoted 7 lines

For example, host-efl is a host-meson-package, and depends on host-libglib2, so presumably we would hav e the issue there too.

We can't fix the breakage by adding -B to all the host packages one by one. If -B is idneed the correct fix (I doubt it), then we will have to add it to the common HOST_CFLAGS, HOST_CXXFLAGS and so on...

I'm not convinced either by this patch (see the comment bellow the --- sign). Indeed host-qemu may not the only package having such issue...

Quoted 18 lines

However, what bothers me slighhtly, is why the qemu link step uses two different ways to link with the libglib2 libraries, e.g.:

    -l/path/to/host/lib/libgio-2.0.so
    -L/path/to/host/lib -lgmodule-2.0

So, why is libgio handled with an absolute path, while libgmodule is not? Note that libgio does have a DT_NEEDED on libgmodule.

However, let's have a look at the pkg-config files, let's start with gio-2.0.pc:

    Requires: glib-2.0, gobject-2.0
    Requires.private: gmodule-no-export-2.0, zlib, mount >= 2.23
    Libs: -L${libdir} -lgio-2.0
    Libs.private: -ldl -lresolv
Quoted 9 lines

OK, then gmodule-no-export-2.0.pc:

    Requires: glib-2.0
    Libs: -L${libdir} -lgmodule-2.0 -pthread
    Libs.private: -ldl

So, the two are similar. I am not sure to understand how we end up in this situation, though... :-/

I'm not sure to understand your comment about pc files, this may be another problem?

This issue could be reproduced with another library.

Do you know any buildsystem using compiler's default paths like this ?

Quoted 3 lines

In the meantime, since the bump to meson 1.2.0 is causing the issue with out an easy fix in sight, I'll revert the meson bump.

I believe this is the safe solution for the upcoming release.

Best regards,
Romain

Quoted 86 lines

Regards,
Yann E. MORIN.

Quoted 5 lines

Fixes:
host/lib/libgio-2.0.so: undefined reference to `g_module_open_full'
collect2: error: ld returned 1 exit statu

The problem only appear if we have two version of the glib2 library

installed

Quoted 6 lines

on the system:

[Buildroot]/host/lib/libgio-2.0.so.0.7600.1
/usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.6600.8

This is probably a meson bug introduced by [1] since the error message

is wrong!

Quoted 5 lines
$ strings output/host/lib/libgio-2.0.so | grep g_module_open_full
g_module_open_full

The libgio-2.0.so provided by Buildroot contains the g_module_open_full

symbol

Quoted 3 lines

while the libgio-2.0.so installed on the host is too old to have it:

$ strings /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.6600.8 | grep

g_module_open_full

Quoted 2 lines

[1]

https://github.com/mesonbuild/meson/commit/59cfbf68e00aa774a9868101f423bd662938c15d

Quoted 14 lines
Cc Nicolas CARRIER <[email protected]>

---
This meson issue may be not specific to host-qemu...

---
 package/qemu/qemu.mk | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
package/qemu/qemu.mk +0 -0
index b341d0220d..8a8dddee58 100644

package/qemu/qemu.mk HOST_QEMU_ARCH = sh4eb · line 409

Quoted 10 lines
package/qemu/qemu.mk +6 -1
@@ -401,7 +401,18 @@ HOST_QEMU_ARCH = sh4eb
 endif
 HOST_QEMU_SYS_ARCH ?= $(HOST_QEMU_ARCH)
 
-HOST_QEMU_CFLAGS = $(HOST_CFLAGS)
+# Since meson 1.2.0, compiler's search path are retrieved and used
+# to link a library. If the same library is installed on the host
+# and build by Buildroot (HOST_DIR/lib), meson now link against
+# the one found in the compiler's search path instead of linking
+# agains the one provided by Buildroot.
+#

https://github.com/mesonbuild/meson/issues/11914#issuecomment-1678090566

package/qemu/qemu.mk HOST_QEMU_ARCH = sh4eb · line 411

Quoted 2 lines
package/qemu/qemu.mk +2 -0
+# Help the compiler by setting -Bprefix that specifies where to find the
+# executables, libraries, include files, and data files of the compiler

itself.

package/qemu/qemu.mk HOST_QEMU_ARCH = sh4eb · line 412

Quoted 1 line
package/qemu/qemu.mk +1 -0
+# For each subprogram to be run, the compiler driver first tries the -B

prefix,

package/qemu/qemu.mk HOST_QEMU_ARCH = sh4eb · line 413

Quoted 1 line
package/qemu/qemu.mk +1 -0
+# if any. If that name is not found, or if -B is not specified, the

driver

Quoted 8 lines
package/qemu/qemu.mk +2 -1
+# tries two standard prefixes, /usr/lib/gcc/ and /usr/local/lib/gcc/.
+HOST_QEMU_CFLAGS = $(HOST_CFLAGS) -B$(HOST_DIR)/lib
 
 ifeq ($(BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE),y)
 HOST_QEMU_TARGETS += $(HOST_QEMU_SYS_ARCH)-softmmu
--

2.41.0

Quoted 4 lines

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

--

.-----------------.--------------------.------------------.--------------------.

|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics'
conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___
     |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is
no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v
 conspiracy.  |
Quoted 2 lines

'------------------------------^-------^------------------^--------------------'

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot