buildtools-tarball: add nativesdk-libxml2-utils

For build-sysroots.bb, the xmlcatalog would not be in its staging directory. Causing the following error for eSDK.

ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst-docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status 127.
ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function failed: do_build_native_sysroot

The problem could be reproduced by the following steps.
1. Add in local.conf:
IMAGE_INSTALL_append = " btrfs-tools" DISTRO_FEATURES_append = " api-documentation" INHERIT += "testsdk"
2. bitbake core-image-minimal -c populate_sdk_ext
3. bitbake core-image-minimal -c testsdkext

So we add nativesdk-libxml2-utils to buildtools-tarball to ensure the existence of xmlcatalog. Also add it to HOSTTOOLS_NONFATAL so it could be seen by bitbake.

Signed-off-by Chen Qi <[email protected]>

---
 meta/conf/bitbake.conf                       | 3 +++
 meta/recipes-core/meta/buildtools-tarball.bb | 1 +
 2 files changed, 4 insertions(+)

meta/conf/bitbake.conf +3 -0

index 5e93f5c..2f64eae 100644
@@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
 # Link to git-lfs if present
 HOSTTOOLS_NONFATAL += "git-lfs"
 
+# build-sysroot needs xmlcatalog in order for eSDK installation
+HOSTTOOLS_NONFATAL += "xmlcatalog"
+
 CCACHE ??= ""
 
 TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"

meta/recipes-core/meta/buildtools-tarball.bb +1 -0

index 91df6f1..d39d6f7 100644
@@ -25,6 +25,7 @@ TOOLCHAIN_HOST_TASK ?= "\
     nativesdk-texinfo \
     nativesdk-libnss-nis \
     nativesdk-rpcsvc-proto \
+    nativesdk-libxml2-utils \
     "
 
 MULTIMACH_TARGET_SYS = "${SDK_ARCH}-nativesdk${SDK_VENDOR}-${SDK_OS}"

Quoted 10 lines · Chen Qi

For build-sysroots.bb, the xmlcatalog would not be in its staging directory. Causing the following error for eSDK.

ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst- docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status 127.
ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
failed: do_build_native_sysroot

Quoted 7 lines

The problem could be reproduced by the following steps.
1. Add in local.conf:
IMAGE_INSTALL_append = " btrfs-tools" DISTRO_FEATURES_append = " api-documentation" INHERIT += "testsdk"
2. bitbake core-image-minimal -c populate_sdk_ext
3. bitbake core-image-minimal -c testsdkext

should these steps be added to eSDK tests?

For build-sysroots.bb, the xmlcatalog would not be in its staging directory. Causing the following error for eSDK.

ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst- docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status 127.
ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
failed: do_build_native_sysroot

The problem could be reproduced by the following steps.
1. Add in local.conf:
IMAGE_INSTALL_append = " btrfs-tools" DISTRO_FEATURES_append = " api-documentation" INHERIT += "testsdk"
2. bitbake core-image-minimal -c populate_sdk_ext
3. bitbake core-image-minimal -c testsdkext

So we add nativesdk-libxml2-utils to buildtools-tarball to ensure the existence of xmlcatalog. Also add it to HOSTTOOLS_NONFATAL so it could be seen by bitbake.

Signed-off-by Chen Qi <[email protected]>

---
 meta/conf/bitbake.conf                       | 3 +++
 meta/recipes-core/meta/buildtools-tarball.bb | 1 +
 2 files changed, 4 insertions(+)

meta/conf/bitbake.conf +0 -0

index 5e93f5c..2f64eae 100644

Quoted 33 lines · Chen Qi

For build-sysroots.bb, the xmlcatalog would not be in its staging directory. Causing the following error for eSDK.

ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst- docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status 127.
ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
failed: do_build_native_sysroot

The problem could be reproduced by the following steps.
1. Add in local.conf:
IMAGE_INSTALL_append = " btrfs-tools" DISTRO_FEATURES_append = " api-documentation" INHERIT += "testsdk"
2. bitbake core-image-minimal -c populate_sdk_ext
3. bitbake core-image-minimal -c testsdkext

So we add nativesdk-libxml2-utils to buildtools-tarball to ensure the existence of xmlcatalog. Also add it to HOSTTOOLS_NONFATAL so it could be seen by bitbake.

Signed-off-by Chen Qi <[email protected]>

---
 meta/conf/bitbake.conf                       | 3 +++
 meta/recipes-core/meta/buildtools-tarball.bb | 1 +
 2 files changed, 4 insertions(+)

meta/conf/bitbake.conf +0 -0

index 5e93f5c..2f64eae 100644

meta/conf/bitbake.conf HOSTTOOLS_NONFATAL += "scp" · line 521

Quoted 5 lines

meta/conf/bitbake.conf +2 -0

@@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
 # Link to git-lfs if present
 HOSTTOOLS_NONFATAL += "git-lfs"
 
+# build-sysroot needs xmlcatalog in order for eSDK installation
+HOSTTOOLS_NONFATAL += "xmlcatalog"

I don't mind the buildtools-tarball change but HOSTTOOLS is starting to grow to contain far too much random stuff which could impact reproduciblity.

Is there some other way we could fix this? It still feels like a dependency problem which we chould fix by adding a missing dependency although I appreciate its far from being that simple.

I wonder if there is a way we could teach build-sysroots to be cleverer about dependencies?

Cheers,

Richard

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Agreed, making it a non-fatal dependency just means that some eSDKs break, not all. There needs to be a better fix.

Ross

Quoted 59 lines · Richard Purdie

Quoted 33 lines · Chen Qi

For build-sysroots.bb, the xmlcatalog would not be in its staging directory. Causing the following error for eSDK.

ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst- docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status 127.
ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
failed: do_build_native_sysroot

The problem could be reproduced by the following steps.
1. Add in local.conf:
IMAGE_INSTALL_append = " btrfs-tools" DISTRO_FEATURES_append = " api-documentation" INHERIT += "testsdk"
2. bitbake core-image-minimal -c populate_sdk_ext
3. bitbake core-image-minimal -c testsdkext

So we add nativesdk-libxml2-utils to buildtools-tarball to ensure the existence of xmlcatalog. Also add it to HOSTTOOLS_NONFATAL so it could be seen by bitbake.

Signed-off-by Chen Qi <[email protected]>

---
  meta/conf/bitbake.conf                       | 3 +++
  meta/recipes-core/meta/buildtools-tarball.bb | 1 +
  2 files changed, 4 insertions(+)

meta/conf/bitbake.conf +0 -0

index 5e93f5c..2f64eae 100644

meta/conf/bitbake.conf HOSTTOOLS_NONFATAL += "scp" · line 521

Quoted 5 lines

meta/conf/bitbake.conf +2 -0

@@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
  # Link to git-lfs if present
  HOSTTOOLS_NONFATAL += "git-lfs"
 
+# build-sysroot needs xmlcatalog in order for eSDK installation
+HOSTTOOLS_NONFATAL += "xmlcatalog"

I don't mind the buildtools-tarball change but HOSTTOOLS is starting to grow to contain far too much random stuff which could impact reproduciblity.

Is there some other way we could fix this? It still feels like a dependency problem which we chould fix by adding a missing dependency although I appreciate its far from being that simple.

I wonder if there is a way we could teach build-sysroots to be cleverer about dependencies?

Cheers,

Richard

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Quoted 33 lines · Chen Qi

For build-sysroots.bb, the xmlcatalog would not be in its staging directory. Causing the following error for eSDK.

ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst- docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status 127.
ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
failed: do_build_native_sysroot

The problem could be reproduced by the following steps.
1. Add in local.conf:
IMAGE_INSTALL_append = " btrfs-tools" DISTRO_FEATURES_append = " api-documentation" INHERIT += "testsdk"
2. bitbake core-image-minimal -c populate_sdk_ext
3. bitbake core-image-minimal -c testsdkext

So we add nativesdk-libxml2-utils to buildtools-tarball to ensure the existence of xmlcatalog. Also add it to HOSTTOOLS_NONFATAL so it could be seen by bitbake.

Signed-off-by Chen Qi <[email protected]>

---
  meta/conf/bitbake.conf                       | 3 +++
  meta/recipes-core/meta/buildtools-tarball.bb | 1 +
  2 files changed, 4 insertions(+)

meta/conf/bitbake.conf +0 -0

index 5e93f5c..2f64eae 100644

meta/conf/bitbake.conf HOSTTOOLS_NONFATAL += "scp" · line 521

Quoted 5 lines

meta/conf/bitbake.conf +2 -0

@@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
  # Link to git-lfs if present
  HOSTTOOLS_NONFATAL += "git-lfs"
 
+# build-sysroot needs xmlcatalog in order for eSDK installation
+HOSTTOOLS_NONFATAL += "xmlcatalog"

I don't mind the buildtools-tarball change but HOSTTOOLS is starting to grow to contain far too much random stuff which could impact reproduciblity.

Is there some other way we could fix this? It still feels like a dependency problem which we chould fix by adding a missing dependency although I appreciate its far from being that simple.

I wonder if there is a way we could teach build-sysroots to be cleverer about dependencies?

Cheers,

Richard

Quoted 42 lines · Richard Purdie

Quoted 33 lines · Chen Qi

For build-sysroots.bb, the xmlcatalog would not be in its staging directory. Causing the following error for eSDK.

ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst- docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status 127.
ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
failed: do_build_native_sysroot

The problem could be reproduced by the following steps.
1. Add in local.conf:
IMAGE_INSTALL_append = " btrfs-tools" DISTRO_FEATURES_append = " api-documentation" INHERIT += "testsdk"
2. bitbake core-image-minimal -c populate_sdk_ext
3. bitbake core-image-minimal -c testsdkext

So we add nativesdk-libxml2-utils to buildtools-tarball to ensure the existence of xmlcatalog. Also add it to HOSTTOOLS_NONFATAL so it could be seen by bitbake.

Signed-off-by Chen Qi <[email protected]>

---
  meta/conf/bitbake.conf                       | 3 +++
  meta/recipes-core/meta/buildtools-tarball.bb | 1 +
  2 files changed, 4 insertions(+)

meta/conf/bitbake.conf +0 -0

index 5e93f5c..2f64eae 100644

meta/conf/bitbake.conf HOSTTOOLS_NONFATAL += "scp" · line 520

Quoted 4 lines

meta/conf/bitbake.conf +1 -0

@@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
  # Link to git-lfs if present
  HOSTTOOLS_NONFATAL += "git-lfs"
  +# build-sysroot needs xmlcatalog in order for eSDK installation
+HOSTTOOLS_NONFATAL += "xmlcatalog"

I don't mind the buildtools-tarball change but HOSTTOOLS is starting to grow to contain far too much random stuff which could impact reproduciblity.

Quoted 6 lines

Is there some other way we could fix this? It still feels like a dependency problem which we chould fix by adding a missing dependency although I appreciate its far from being that simple.

I wonder if there is a way we could teach build-sysroots to be cleverer about dependencies?

I'll try to implement it and send out a new patch.

Regards,
Chen Qi

Quoted 5 lines

Cheers,

Richard

Quoted 52 lines · ChenQi

Quoted 51 lines · Richard Purdie

Quoted 38 lines · Chen Qi

For build-sysroots.bb, the xmlcatalog would not be in its staging directory. Causing the following error for eSDK.

ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst -
docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status
127.
ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
failed: do_build_native_sysroot

The problem could be reproduced by the following steps.
1. Add in local.conf:
IMAGE_INSTALL_append = " btrfs-tools" DISTRO_FEATURES_append = " api-documentation" INHERIT += "testsdk"
2. bitbake core-image-minimal -c populate_sdk_ext
3. bitbake core-image-minimal -c testsdkext

So we add nativesdk-libxml2-utils to buildtools-tarball to ensure the existence of xmlcatalog. Also add it to HOSTTOOLS_NONFATAL so it could be seen by bitbake.

Signed-off-by Chen Qi <[email protected]>

---
  meta/conf/bitbake.conf                       | 3 +++
  meta/recipes-core/meta/buildtools-tarball.bb | 1 +
  2 files changed, 4 insertions(+)

meta/conf/bitbake.conf +0 -0

index 5e93f5c..2f64eae 100644
@@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
  # Link to git-lfs if present
  HOSTTOOLS_NONFATAL += "git-lfs"
  +# build-sysroot needs xmlcatalog in order for eSDK

Quoted 2 lines

installation
+HOSTTOOLS_NONFATAL += "xmlcatalog"

I don't mind the buildtools-tarball change but HOSTTOOLS is starting to
grow to contain far too much random stuff which could impact reproduciblity.

Is there some other way we could fix this? It still feels like a dependency problem which we chould fix by adding a missing dependency
although I appreciate its far from being that simple.

Quoted 5 lines

Quoted 3 lines

I wonder if there is a way we could teach build-sysroots to be cleverer
about dependencies?

I'll try to implement it and send out a new patch.

Unfortunately I cannot figure out a good way to implement this. Could you please help giving me some hints?

Quoted 53 lines · ChenQi

Quoted 52 lines · ChenQi

Quoted 51 lines · Richard Purdie

Quoted 38 lines · Chen Qi

For build-sysroots.bb, the xmlcatalog would not be in its staging directory. Causing the following error for eSDK.

ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Command '/PATH/TO/IMAGE/testsdkext/tmp/sysroots/x86_64/usr/bin/postinst -
docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status
127.
ERROR: build-sysroots-1.0-r0 do_build_native_sysroot: Function
failed: do_build_native_sysroot

The problem could be reproduced by the following steps.
1. Add in local.conf:
IMAGE_INSTALL_append = " btrfs-tools" DISTRO_FEATURES_append = " api-documentation" INHERIT += "testsdk"
2. bitbake core-image-minimal -c populate_sdk_ext
3. bitbake core-image-minimal -c testsdkext

So we add nativesdk-libxml2-utils to buildtools-tarball to ensure the existence of xmlcatalog. Also add it to HOSTTOOLS_NONFATAL so it could be seen by bitbake.

Signed-off-by Chen Qi <[email protected]>

---
   meta/conf/bitbake.conf                       | 3 +++
   meta/recipes-core/meta/buildtools-tarball.bb | 1 +
   2 files changed, 4 insertions(+)

meta/conf/bitbake.conf +0 -0

index 5e93f5c..2f64eae 100644
@@ -517,6 +517,9 @@ HOSTTOOLS_NONFATAL += "scp"
   # Link to git-lfs if present
   HOSTTOOLS_NONFATAL += "git-lfs"
   +# build-sysroot needs xmlcatalog in order for eSDK

Quoted 2 lines

installation
+HOSTTOOLS_NONFATAL += "xmlcatalog"

I don't mind the buildtools-tarball change but HOSTTOOLS is starting to
grow to contain far too much random stuff which could impact reproduciblity.

Is there some other way we could fix this? It still feels like a dependency problem which we chould fix by adding a missing dependency
although I appreciate its far from being that simple.

Quoted 6 lines

Quoted 4 lines

Quoted 3 lines

I wonder if there is a way we could teach build-sysroots to be cleverer
about dependencies?

I'll try to implement it and send out a new patch.

Unfortunately I cannot figure out a good way to implement this. Could you please help giving me some hints?

What happens if we add:

${@bb.utils.contains('DISTRO_FEATURES', 'api-documentation', 'nativesdk-libxml2', '', d)} \

to
` meta/recipes-core/packagegroups/nativesdk-packagegroup-sdk-host.bb

?

Cheers,

Richard

Quoted 6 lines · Richard Purdie

On Tue, 2019-07-09 at 17:36 +0800, ChenQi wrote:
What happens if we add:

${@bb.utils.contains('DISTRO_FEATURES', 'api-documentation', 'nativesdk-libxml2', '', d)} \

Quoted 5 lines

to
` meta/recipes-core/packagegroups/nativesdk-packagegroup-sdk-host.bb

?

This does not work. Using nativesdk-libxml2-utils does not work too. Both buildtools and eSDK do not use this packagegroup.

I've sent out a V2 of this patch, with change to put the 'libxml2-native' dependency in xmlcatalog.bbclass.
In this way, we can ensure that the xmlcatalog is from our recipe. In normal build, it's from libxml2-native; in build-sysroots in eSDK, it's from nativesdk-libxml2-utils. Do you think this is acceptable?