Skip to contentSkip to conversation

vulkan-samples: enable network during do_unpack

PATCH In review Julien STEPHAN 4 participants 10 messages Aug 17, 2022 – Oct 17, 2022 1 message not in the archive

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 vulkan-samples: enable network during do_unpack

vulkan-samples uses git submodules and one of the submodules uses lfs objects. During the unpack step, the lfs objects are retrieved leading to a network failure.

The correct fix is to fetch lfs objects during the fetch step, but let's workaround this issue for now

---
 meta/recipes-graphics/vulkan/vulkan-samples_git.bb | 5 +++++
 1 file changed, 5 insertions(+)
meta/recipes-graphics/vulkan/vulkan-samples_git.bb +5 -0
index 41cb4a437d..b45ee22a69 100644
@@ -11,6 +11,11 @@ SRC_URI = "gitsm://github.com/KhronosGroup/Vulkan-Samples.git;branch=master;prot
            file://0001-framework-core-hpp_vulkan_resource.h-add-header-incl.patch \
            "
 
+# Workaround for network access issue during the do_unpack step
+# this needs to be fixed in the recipes buildsystem to move
+# this such that it can be accomplished during do_fetch task
+do_unpack[network] = "1"
+
 UPSTREAM_CHECK_COMMITS = "1"
 SRCREV = "ee6c1522af8ba9b56f8416c2eedafb440f681085"
 
Signature · 1 line
2.37.1
#1 lore raw

This has not been observed on the autobuilder or independently reported, can you show how to reproduce or the failure log?

Alex

Quoted 41 lines · Julien STEPHAN

vulkan-samples uses git submodules and one of the submodules uses lfs objects. During the unpack step, the lfs objects are retrieved leading to a network failure.

The correct fix is to fetch lfs objects during the fetch step, but let's workaround this issue for now

---
 meta/recipes-graphics/vulkan/vulkan-samples_git.bb | 5 +++++
 1 file changed, 5 insertions(+)
meta/recipes-graphics/vulkan/vulkan-samples_git.bb +5 -1
index 41cb4a437d..b45ee22a69 100644
@@ -11,6 +11,11 @@ SRC_URI = "gitsm://github.com/KhronosGroup/Vulkan-Samples.git;branch=master;prot
            file://0001-framework-core-hpp_vulkan_resource.h-add-header-incl.patch \
            "
 
+# Workaround for network access issue during the do_unpack step
+# this needs to be fixed in the recipes buildsystem to move
+# this such that it can be accomplished during do_fetch task
+do_unpack[network] = "1"
+
 UPSTREAM_CHECK_COMMITS = "1"
 SRCREV = "ee6c1522af8ba9b56f8416c2eedafb440f681085"
 
--

2.37.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169500): https://lists.openembedded.org/g/openembedded-core/message/169500
Mute This Topic: https://lists.openembedded.org/mt/93089137/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#2 lore raw

Hi Alexander,

I observe this by just trying to build the vulkan-samples recipe:
DISTRO=poky bitbake vulkan-samples.

Here is the log output (sorry it is a bit verbose):
ERROR: vulkan-samples-git-r0 do_unpack: Bitbake Fetcher Error:
FetchError('Fetch command export PSEUDO_DISABLED=1; export
SSH_AUTH_SOCK="/home/jstephan/.ssh/ssh_auth_sock"; export PATH="/home/jstephan/kirkstone/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/jstephan/kirkstone/src/poky/scripts:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot/usr/bin/crossscripts:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/sbin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/bin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/sbin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/bin:/home/jstephan/kirkstone/src/poky/bitbake/bin:/home/jstephan/kirkstone/build/tmp/hosttools"; export HOME="/home/jstephan"; git -c core.fsyncobjectfiles=0 -c
gc.autoDetach=false -c core.pager=cat submodule update --recursive
--no-fetch failed with exit code 1, output:\nSubmodule path \'assets\':
checked out \'8a6b7f952d7786a305f89d375a55598e995ee972\'\nSubmodule path
\'third_party/CTPL\': checked out
\'437e135dbd94eb65b45533d9ce8ee28b5bd37b6d\'\nSubmodule path
\'third_party/astc\': checked out
\'4861c3db1ca50eb5f5767d95d1cd1bacb98ac8a5\'\nSubmodule path
\'third_party/cli11\': checked out
\'4af78beef777e313814b4daff70e2da9171a385a\'\nSubmodule path
\'third_party/glfw\': checked out
\'6876cf8d7e0e70dc3e4d7b0224d08312c9f78099\'\nSubmodule path
\'third_party/glm\': checked out
\'fce2abd01ce21063bd25ba67c9318be83bf48813\'\nSubmodule path
\'third_party/glslang\': checked out
\'6baa85f6ceec5fddd2e13b43dd1edc944dd930af\'\nSubmodule path
\'third_party/hwcpipe\': checked out
\'5a0c3d0b527bea96154998f29177f3b449cebd3a\'\nSubmodule path
\'third_party/imgui\': checked out
\'c23a19c26fa9cc778f755e76799f0cafdcb5a13b\'\nSubmodule path
\'third_party/opencl\': checked out
\'1aa1139b58a515877a923cce6b254e09d1b2fb2c\'\nSubmodule path
\'third_party/spdlog\': checked out
\'7bf8f148795d2571b4ae4aa9c6d2a8111fbb7077\'\nSubmodule path
\'third_party/spirv-cross\': checked
out \'84f56d0b43e5a6874de45e11be2bff9dc336fd7e\'\nSubmodule path
\'third_party/stb\': checked out
\'2c2908f50515dcd939f24be261c3ccbcd277bb49\'\nSubmodule path
\'third_party/tinygltf\': checked out
\'c0d0251e2c7f2273a894aa1b125af9313a88eae4\'\nSubmodule path
\'third_party/vma\': checked out
\'f9921aefddee2437cc2e3303d3175bd8ef23e22c\'\nSubmodule path
\'third_party/volk\': checked out
\'b18f3192886981f28d17f8f41c26c5a53ac3b977\'\nSubmodule path
\'third_party/vulkan\': checked out
\'76f00ef6cbb1886eb1162d1fa39bee8b51e22ee8\'\n\nDownloading
other_lib/ios/Debug-iphoneos/libSDL2.a (8.2 MB)\nError downloading object:
other_lib/ios/Debug-iphoneos/libSDL2.a (5045776): Smudge error: Error
downloading other_lib/ios/Debug-iphoneos/libSDL2.a
(5045776556ca57f2f7125d89c37a8b94f79cdebdc040d420513f0491787c4ea7): batch
response: Post
https://github.com/KhronosGroup/KTX-Software.git/info/lfs/objects/batch:
dial tcp: lookup github.com: Temporary failure in name resolution\n\nErrors
logged to
/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/git/.git/modules/third_party/ktx/lfs/logs/20220817T152514.337649981.log\nUse
git lfs logs last to view the log.\nerror: external filter \'git-lfs
filter-process\' failed\nfatal: other_lib/ios/Debug-iphoneos/libSDL2.a:
smudge filter lfs failed\nUnable to checkout
\'6a38a0694b4d73d22c5d1e22f865d03545e808ea\' in submodule path
\'third_party/ktx\'\n', None)
ERROR: Logfile of failure stored in:
/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/temp/log.do_unpack.1796011
ERROR: Task
(/home/jstephan/kirkstone/src/poky/meta/recipes-graphics/vulkan/vulkan-samples_git.bb:do_unpack)
failed with exit code '1'

Julien

Le mer. 17 août 2022 à 21:09, Alexander Kanavin <[email protected]> a écrit :

Quoted 54 lines

This has not been observed on the autobuilder or independently reported, can you show how to reproduce or the failure log?

Alex

Quoted 8 lines · Julien STEPHAN

vulkan-samples uses git submodules and one of the submodules uses lfs objects. During the unpack step, the lfs objects are retrieved leading to a network failure.

The correct fix is to fetch lfs objects during the fetch step, but let's workaround this issue for now

Review

Quoted 6 lines
---
 meta/recipes-graphics/vulkan/vulkan-samples_git.bb | 5 +++++
 1 file changed, 5 insertions(+)
(quoted diff) +0 -0

b/meta/recipes-graphics/vulkan/vulkan-samples_git.bb

meta/recipes-graphics/vulkan/vulkan-samples_git.bb SRC_URI = "gitsm://

Quoted 4 lines

index 41cb4a437d..b45ee22a69 100644

meta/recipes-graphics/vulkan/vulkan-samples_git.bb +0 -0
@@ -11,6 +11,11 @@ SRC_URI = "gitsm://

github.com/KhronosGroup/Vulkan-Samples.git;branch=master;prot

Quoted 1 line

file://0001-framework-core-hpp_vulkan_resource.h-add-header-incl.patch \

Quoted 14 lines
            "
meta/recipes-graphics/vulkan/vulkan-samples_git.bb +5 -1
+# Workaround for network access issue during the do_unpack step
+# this needs to be fixed in the recipes buildsystem to move
+# this such that it can be accomplished during do_fetch task
+do_unpack[network] = "1"
+
 UPSTREAM_CHECK_COMMITS = "1"
 SRCREV = "ee6c1522af8ba9b56f8416c2eedafb440f681085"
 
--

2.37.1

Quoted 3 lines

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#169500):

https://lists.openembedded.org/g/openembedded-core/message/169500

Quoted 3 lines

Mute This Topic: https://lists.openembedded.org/mt/93089137/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [

[email protected]]

Quoted 2 lines

-=-=-=-=-=-=-=-=-=-=-=-

#3 lore raw

Just to be sure, can you repeat this on a pristine poky master checkout with no cache and no downloads? 'bitbake -c unpack vulkan-samples' should do.

Also what does git --version say? I wonder if there's something in too old or too new versions of it.

Alex

Quoted 61 lines · Julien Stephan

Hi Alexander,

I observe this by just trying to build the vulkan-samples recipe: DISTRO=poky bitbake vulkan-samples.

Here is the log output (sorry it is a bit verbose):
ERROR: vulkan-samples-git-r0 do_unpack: Bitbake Fetcher Error: FetchError('Fetch command export PSEUDO_DISABLED=1; export SSH_AUTH_SOCK="/home/jstephan/.ssh/ssh_auth_sock"; export PATH="/home/jstephan/kirkstone/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/jstephan/kirkstone/src/poky/scripts:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot/usr/bin/crossscripts:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/sbin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/bin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/sbin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/bin:/home/jstephan/kirkstone/src/poky/bitbake/bin:/home/jstephan/kirkstone/build/tmp/hosttools"; export HOME="/home/jstephan"; git -c core.fsyncobjectfiles=0 -c gc.autoDetach=false -c core.pager=cat submodule update --recursive --no-fetch failed with exit code 1, output:\nSubmodule path \'assets\': checked out \'8a6b7f952d7786a305f89d375a55598e995ee972\'\nSubmodule path \'third_party/CTPL\': checked out \'437e135dbd94eb65b45533d9ce8ee28b5bd37b6d\'\nSubmodule path \'third_party/astc\': checked out \'4861c3db1ca50eb5f5767d95d1cd1bacb98ac8a5\'\nSubmodule path \'third_party/cli11\': checked out \'4af78beef777e313814b4daff70e2da9171a385a\'\nSubmodule path \'third_party/glfw\': checked out \'6876cf8d7e0e70dc3e4d7b0224d08312c9f78099\'\nSubmodule path \'third_party/glm\': checked out \'fce2abd01ce21063bd25ba67c9318be83bf48813\'\nSubmodule path \'third_party/glslang\': checked out \'6baa85f6ceec5fddd2e13b43dd1edc944dd930af\'\nSubmodule path \'third_party/hwcpipe\': checked out \'5a0c3d0b527bea96154998f29177f3b449cebd3a\'\nSubmodule path \'third_party/imgui\': checked out \'c23a19c26fa9cc778f755e76799f0cafdcb5a13b\'\nSubmodule path \'third_party/opencl\': checked out \'1aa1139b58a515877a923cce6b254e09d1b2fb2c\'\nSubmodule path \'third_party/spdlog\': checked out \'7bf8f148795d2571b4ae4aa9c6d2a8111fbb7077\'\nSubmodule path \'third_party/spirv-cross\': checked out \'84f56d0b43e5a6874de45e11be2bff9dc336fd7e\'\nSubmodule path \'third_party/stb\': checked out \'2c2908f50515dcd939f24be261c3ccbcd277bb49\'\nSubmodule path \'third_party/tinygltf\': checked out \'c0d0251e2c7f2273a894aa1b125af9313a88eae4\'\nSubmodule path \'third_party/vma\': checked out \'f9921aefddee2437cc2e3303d3175bd8ef23e22c\'\nSubmodule path \'third_party/volk\': checked out \'b18f3192886981f28d17f8f41c26c5a53ac3b977\'\nSubmodule path \'third_party/vulkan\': checked out \'76f00ef6cbb1886eb1162d1fa39bee8b51e22ee8\'\n\nDownloading other_lib/ios/Debug-iphoneos/libSDL2.a (8.2 MB)\nError downloading object: other_lib/ios/Debug-iphoneos/libSDL2.a (5045776): Smudge error: Error downloading other_lib/ios/Debug-iphoneos/libSDL2.a (5045776556ca57f2f7125d89c37a8b94f79cdebdc040d420513f0491787c4ea7): batch response: Post https://github.com/KhronosGroup/KTX-Software.git/info/lfs/objects/batch: dial tcp: lookup github.com: Temporary failure in name resolution\n\nErrors logged to /home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/git/.git/modules/third_party/ktx/lfs/logs/20220817T152514.337649981.log\nUse git lfs logs last to view the log.\nerror: external filter \'git-lfs filter-process\' failed\nfatal: other_lib/ios/Debug-iphoneos/libSDL2.a: smudge filter lfs failed\nUnable to checkout \'6a38a0694b4d73d22c5d1e22f865d03545e808ea\' in submodule path \'third_party/ktx\'\n', None)
ERROR: Logfile of failure stored in: /home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/temp/log.do_unpack.1796011
ERROR: Task (/home/jstephan/kirkstone/src/poky/meta/recipes-graphics/vulkan/vulkan-samples_git.bb:do_unpack) failed with exit code '1'

Julien

Quoted 48 lines · Alexander Kanavin

This has not been observed on the autobuilder or independently reported, can you show how to reproduce or the failure log?

Alex

Quoted 41 lines · Julien STEPHAN

vulkan-samples uses git submodules and one of the submodules uses lfs objects. During the unpack step, the lfs objects are retrieved leading to a network failure.

The correct fix is to fetch lfs objects during the fetch step, but let's workaround this issue for now

---
 meta/recipes-graphics/vulkan/vulkan-samples_git.bb | 5 +++++
 1 file changed, 5 insertions(+)
meta/recipes-graphics/vulkan/vulkan-samples_git.bb +5 -1
index 41cb4a437d..b45ee22a69 100644
@@ -11,6 +11,11 @@ SRC_URI = "gitsm://github.com/KhronosGroup/Vulkan-Samples.git;branch=master;prot
            file://0001-framework-core-hpp_vulkan_resource.h-add-header-incl.patch \
            "
 
+# Workaround for network access issue during the do_unpack step
+# this needs to be fixed in the recipes buildsystem to move
+# this such that it can be accomplished during do_fetch task
+do_unpack[network] = "1"
+
 UPSTREAM_CHECK_COMMITS = "1"
 SRCREV = "ee6c1522af8ba9b56f8416c2eedafb440f681085"
 
--

2.37.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169500): https://lists.openembedded.org/g/openembedded-core/message/169500
Mute This Topic: https://lists.openembedded.org/mt/93089137/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

A message that is not in the archive (sent off-list or not yet received): [email protected]

#5 lore raw

I reproduced. This happens if all three are true:
- recipe is using gitsm://
- one of the submodules contains lfs objects
- git-lfs is installed on the host (that's why it's not seen on the AB)

There's logic in plain git:// fetcher to deal with lfs, but not in gitsm://. So that logic needs to be added there as well. A workaround like you suggested is not the right fix.

Can you look into it please? Or at least a ticket would be good to have.

Alex

Quoted 75 lines · Alexander Kanavin via lists.openembedded.org

Just to be sure, can you repeat this on a pristine poky master checkout with no cache and no downloads? 'bitbake -c unpack vulkan-samples' should do.

Also what does git --version say? I wonder if there's something in too old or too new versions of it.

Alex

Quoted 49 lines · Julien Stephan

Hi Alexander,

I observe this by just trying to build the vulkan-samples recipe: DISTRO=poky bitbake vulkan-samples.

Here is the log output (sorry it is a bit verbose):
ERROR: vulkan-samples-git-r0 do_unpack: Bitbake Fetcher Error: FetchError('Fetch command export PSEUDO_DISABLED=1; export SSH_AUTH_SOCK="/home/jstephan/.ssh/ssh_auth_sock"; export PATH="/home/jstephan/kirkstone/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/jstephan/kirkstone/src/poky/scripts:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot/usr/bin/crossscripts:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/sbin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/bin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/sbin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/bin:/home/jstephan/kirkstone/src/poky/bitbake/bin:/home/jstephan/kirkstone/build/tmp/hosttools"; export HOME="/home/jstephan"; git -c core.fsyncobjectfiles=0 -c gc.autoDetach=false -c core.pager=cat submodule update --recursive --no-fetch failed with exit code 1, output:\nSubmodule path \'assets\': checked out \'8a6b7f952d7786a305f89d375a55598e995ee972\'\nSubmodule path \'third_party/CTPL\': checked out \'437e135dbd94eb65b45533d9ce8ee28b5bd37b6d\'\nSubmodule path \'third_party/astc\': checked out \'4861c3db1ca50eb5f5767d95d1cd1bacb98ac8a5\'\nSubmodule path \'third_party/cli11\': checked out \'4af78beef777e313814b4daff70e2da9171a385a\'\nSubmodule path \'third_party/glfw\': checked out \'6876cf8d7e0e70dc3e4d7b0224d08312c9f78099\'\nSubmodule path \'third_party/glm\': checked out \'fce2abd01ce21063bd25ba67c9318be83bf48813\'\nSubmodule path \'third_party/glslang\': checked out \'6baa85f6ceec5fddd2e13b43dd1edc944dd930af\'\nSubmodule path \'third_party/hwcpipe\': checked out \'5a0c3d0b527bea96154998f29177f3b449cebd3a\'\nSubmodule path \'third_party/imgui\': checked out \'c23a19c26fa9cc778f755e76799f0cafdcb5a13b\'\nSubmodule path \'third_party/opencl\': checked out \'1aa1139b58a515877a923cce6b254e09d1b2fb2c\'\nSubmodule path \'third_party/spdlog\': checked out \'7bf8f148795d2571b4ae4aa9c6d2a8111fbb7077\'\nSubmodule path \'third_party/spirv-cross\': checked out \'84f56d0b43e5a6874de45e11be2bff9dc336fd7e\'\nSubmodule path \'third_party/stb\': checked out \'2c2908f50515dcd939f24be261c3ccbcd277bb49\'\nSubmodule path \'third_party/tinygltf\': checked out \'c0d0251e2c7f2273a894aa1b125af9313a88eae4\'\nSubmodule path \'third_party/vma\': checked out \'f9921aefddee2437cc2e3303d3175bd8ef23e22c\'\nSubmodule path \'third_party/volk\': checked out \'b18f3192886981f28d17f8f41c26c5a53ac3b977\'\nSubmodule path \'third_party/vulkan\': checked out \'76f00ef6cbb1886eb1162d1fa39bee8b51e22ee8\'\n\nDownloading other_lib/ios/Debug-iphoneos/libSDL2.a (8.2 MB)\nError downloading object: other_lib/ios/Debug-iphoneos/libSDL2.a (5045776): Smudge error: Error downloading other_lib/ios/Debug-iphoneos/libSDL2.a (5045776556ca57f2f7125d89c37a8b94f79cdebdc040d420513f0491787c4ea7): batch response: Post https://github.com/KhronosGroup/KTX-Software.git/info/lfs/objects/batch: dial tcp: lookup github.com: Temporary failure in name resolution\n\nErrors logged to /home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/git/.git/modules/third_party/ktx/lfs/logs/20220817T152514.337649981.log\nUse git lfs logs last to view the log.\nerror: external filter \'git-lfs filter-process\' failed\nfatal: other_lib/ios/Debug-iphoneos/libSDL2.a: smudge filter lfs failed\nUnable to checkout \'6a38a0694b4d73d22c5d1e22f865d03545e808ea\' in submodule path \'third_party/ktx\'\n', None)
ERROR: Logfile of failure stored in: /home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/temp/log.do_unpack.1796011
ERROR: Task (/home/jstephan/kirkstone/src/poky/meta/recipes-graphics/vulkan/vulkan-samples_git.bb:do_unpack) failed with exit code '1'

Julien

Quoted 36 lines · Alexander Kanavin

This has not been observed on the autobuilder or independently reported, can you show how to reproduce or the failure log?

Alex

Quoted 29 lines · Julien STEPHAN

vulkan-samples uses git submodules and one of the submodules uses lfs objects. During the unpack step, the lfs objects are retrieved leading to a network failure.

The correct fix is to fetch lfs objects during the fetch step, but let's workaround this issue for now

---
 meta/recipes-graphics/vulkan/vulkan-samples_git.bb | 5 +++++
 1 file changed, 5 insertions(+)
meta/recipes-graphics/vulkan/vulkan-samples_git.bb +5 -0
index 41cb4a437d..b45ee22a69 100644
@@ -11,6 +11,11 @@ SRC_URI = "gitsm://github.com/KhronosGroup/Vulkan-Samples.git;branch=master;prot
            file://0001-framework-core-hpp_vulkan_resource.h-add-header-incl.patch \
            "
 
+# Workaround for network access issue during the do_unpack step
+# this needs to be fixed in the recipes buildsystem to move
+# this such that it can be accomplished during do_fetch task
+do_unpack[network] = "1"
+
 UPSTREAM_CHECK_COMMITS = "1"
 SRCREV = "ee6c1522af8ba9b56f8416c2eedafb440f681085"
 
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

--
2.37.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169504): https://lists.openembedded.org/g/openembedded-core/message/169504
Mute This Topic: https://lists.openembedded.org/mt/93089137/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#6 lore raw

Le mer. 17 août 2022 à 22:33, Alexander Kanavin <[email protected]> a écrit :

Quoted 8 lines

I reproduced. This happens if all three are true:
- recipe is using gitsm://
- one of the submodules contains lfs objects
- git-lfs is installed on the host (that's why it's not seen on the AB)

Nice! I figured out the first two points, but not the last one.

Quoted 7 lines

There's logic in plain git:// fetcher to deal with lfs, but not in gitsm://. So that logic needs to be added there as well. A workaround like you suggested is not the right fix.

Can you look into it please? Or at least a ticket would be good to have.

Sure, I will try to look at it

Julien

Quoted 149 lines

Alex

Quoted 11 lines · Alexander Kanavin via lists.openembedded.org

Just to be sure, can you repeat this on a pristine poky master checkout with no cache and no downloads? 'bitbake -c unpack vulkan-samples' should do.

Also what does git --version say? I wonder if there's something in too old or too new versions of it.

Alex

On Wed, 17 Aug 2022 at 21:22, Julien Stephan <[email protected]>

Quoted 4 lines · wrote:
Quoted 4 lines

Hi Alexander,

I observe this by just trying to build the vulkan-samples recipe:

DISTRO=poky bitbake vulkan-samples.

Quoted 3 lines
Quoted 3 lines

Here is the log output (sorry it is a bit verbose):
ERROR: vulkan-samples-git-r0 do_unpack: Bitbake Fetcher Error:

FetchError('Fetch command export PSEUDO_DISABLED=1; export
SSH_AUTH_SOCK="/home/jstephan/.ssh/ssh_auth_sock"; export PATH="/home/jstephan/kirkstone/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/jstephan/kirkstone/src/poky/scripts:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot/usr/bin/crossscripts:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/sbin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/bin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/sbin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/bin:/home/jstephan/kirkstone/src/poky/bitbake/bin:/home/jstephan/kirkstone/build/tmp/hosttools"; export HOME="/home/jstephan"; git -c core.fsyncobjectfiles=0 -c
gc.autoDetach=false -c core.pager=cat submodule update --recursive
--no-fetch failed with exit code 1, output:\nSubmodule path \'assets\':
checked out \'8a6b7f952d7786a305f89d375a55598e995ee972\'\nSubmodule path
\'third_party/CTPL\': checked out
\'437e135dbd94eb65b45533d9ce8ee28b5bd37b6d\'\nSubmodule path
\'third_party/astc\': checked out
\'4861c3db1ca50eb5f5767d95d1cd1bacb98ac8a5\'\nSubmodule path
\'third_party/cli11\': checked out
\'4af78beef777e313814b4daff70e2da9171a385a\'\nSubmodule path
\'third_party/glfw\': checked out
\'6876cf8d7e0e70dc3e4d7b0224d08312c9f78099\'\nSubmodule path
\'third_party/glm\': checked out
\'fce2abd01ce21063bd25ba67c9318be83bf48813\'\nSubmodule path
\'third_party/glslang\': checked out
\'6baa85f6ceec5fddd2e13b43dd1edc944dd930af\'\nSubmodule path
\'third_party/hwcpipe\': checked out
\'5a0c3d0b527bea96154998f29177f3b449cebd3a\'\nSubmodule path
\'third_party/imgui\': checked out
\'c23a19c26fa9cc778f755e76799f0cafdcb5a13b\'\nSubmodule path
\'third_party/opencl\': checked out
\'1aa1139b58a515877a923cce6b254e09d1b2fb2c\'\nSubmodule path
\'third_party/spdlog\': checked out
\'7bf8f148795d2571b4ae4aa9c6d2a8111fbb7077\'\nSubmodule path
\'third_party/spirv-cross\': checked out
\'84f56d0b43e5a6874de45e11be2bff9dc336fd7e\'\nSubmodule path
\'third_party/stb\': checked out
\'2c2908f50515dcd939f24be261c3ccbcd277bb49\'\nSubmodule path
\'third_party/tinygltf\': checked out
\'c0d0251e2c7f2273a894aa1b125af9313a88eae4\'\nSubmodule path
\'third_party/vma\': checked out
\'f9921aefddee2437cc2e3303d3175bd8ef23e22c\'\nSubmodule path
\'third_party/volk\': checked out
\'b18f3192886981f28d17f8f41c26c5a53ac3b977\'\nSubmodule path
\'third_party/vulkan\': checked out
\'76f00ef6cbb1886eb1162d1fa39bee8b51e22ee8\'\n\nDownloading
other_lib/ios/Debug-iphoneos/libSDL2.a (8.2 MB)\nError downloading object:
other_lib/ios/Debug-iphoneos/libSDL2.a (5045776): Smudge error: Error
downloading other_lib/ios/Debug-iphoneos/libSDL2.a
(5045776556ca57f2f7125d89c37a8b94f79cdebdc040d420513f0491787c4ea7): batch
response: Post
https://github.com/KhronosGroup/KTX-Software.git/info/lfs/objects/batch:
dial tcp: lookup github.com: Temporary failure in name
resolution\n\nErrors logged to
/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/git/.git/modules/third_party/ktx/lfs/logs/20220817T152514.337649981.log\nUse
git lfs logs last to view the log.\nerror: external filter \'git-lfs
filter-process\' failed\nfatal: other_lib/ios/Debug-iphoneos/libSDL2.a:
smudge filter lfs failed\nUnable to checkout
\'6a38a0694b4d73d22c5d1e22f865d03545e808ea\' in submodule path
\'third_party/ktx\'\n', None)

Quoted 1 line
Quoted 1 line

ERROR: Logfile of failure stored in:

/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/temp/log.do_unpack.1796011

Quoted 1 line
Quoted 1 line

ERROR: Task

(/home/jstephan/kirkstone/src/poky/meta/recipes-graphics/vulkan/vulkan-samples_git.bb:do_unpack) failed with exit code '1'

Quoted 4 lines
Quoted 4 lines

Julien

Le mer. 17 août 2022 à 21:09, Alexander Kanavin <

Quoted 7 lines · [email protected]> a écrit :
Quoted 7 lines
Quoted 7 lines

This has not been observed on the autobuilder or independently reported, can you show how to reproduce or the failure log?

Alex

On Wed, 17 Aug 2022 at 21:01, Julien STEPHAN <[email protected]>

Quoted 2 lines · wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

vulkan-samples uses git submodules and one of the submodules uses

lfs

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

objects. During the unpack step, the lfs objects are retrieved

leading to

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

a network failure.

The correct fix is to fetch lfs objects during the fetch step, but

let's

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Review

Quoted 8 lines

workaround this issue for now

---
 meta/recipes-graphics/vulkan/vulkan-samples_git.bb | 5 +++++
 1 file changed, 5 insertions(+)
(quoted diff) +0 -0

b/meta/recipes-graphics/vulkan/vulkan-samples_git.bb

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

meta/recipes-graphics/vulkan/vulkan-samples_git.bb SRC_URI = "gitsm://

Quoted 4 lines

index 41cb4a437d..b45ee22a69 100644

meta/recipes-graphics/vulkan/vulkan-samples_git.bb +0 -0
@@ -11,6 +11,11 @@ SRC_URI = "gitsm://

github.com/KhronosGroup/Vulkan-Samples.git;branch=master;prot

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

file://0001-framework-core-hpp_vulkan_resource.h-add-header-incl.patch \

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
            "
meta/recipes-graphics/vulkan/vulkan-samples_git.bb +5 -0
+# Workaround for network access issue during the do_unpack step
+# this needs to be fixed in the recipes buildsystem to move
+# this such that it can be accomplished during do_fetch task
+do_unpack[network] = "1"
+
 UPSTREAM_CHECK_COMMITS = "1"
 SRCREV = "ee6c1522af8ba9b56f8416c2eedafb440f681085"
Quoted 10 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

--
2.37.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#169504):

https://lists.openembedded.org/g/openembedded-core/message/169504

Quoted 3 lines

Mute This Topic: https://lists.openembedded.org/mt/93089137/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [

[email protected]]

Quoted 2 lines

-=-=-=-=-=-=-=-=-=-=-=-

#7 lore raw

There is ticket about this already, see
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14283

Quoted 175 lines · Julien STEPHAN
Quoted 8 lines · Alexander Kanavin

I reproduced. This happens if all three are true:
- recipe is using gitsm://
- one of the submodules contains lfs objects
- git-lfs is installed on the host (that's why it's not seen on the AB)

Nice! I figured out the first two points, but not the last one.

Quoted 7 lines

There's logic in plain git:// fetcher to deal with lfs, but not in gitsm://. So that logic needs to be added there as well. A workaround like you suggested is not the right fix.

Can you look into it please? Or at least a ticket would be good to have.

Sure, I will try to look at it

Julien

Quoted 114 lines

Alex

Quoted 11 lines · Alexander Kanavin via lists.openembedded.org

Just to be sure, can you repeat this on a pristine poky master checkout with no cache and no downloads? 'bitbake -c unpack vulkan-samples' should do.

Also what does git --version say? I wonder if there's something in too old or too new versions of it.

Alex

On Wed, 17 Aug 2022 at 21:22, Julien Stephan <[email protected]>

Quoted 4 lines · wrote:
Quoted 4 lines

Hi Alexander,

I observe this by just trying to build the vulkan-samples recipe:

DISTRO=poky bitbake vulkan-samples.

Quoted 3 lines
Quoted 3 lines

Here is the log output (sorry it is a bit verbose):
ERROR: vulkan-samples-git-r0 do_unpack: Bitbake Fetcher Error:

FetchError('Fetch command export PSEUDO_DISABLED=1; export
SSH_AUTH_SOCK="/home/jstephan/.ssh/ssh_auth_sock"; export PATH="/home/jstephan/kirkstone/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/jstephan/kirkstone/src/poky/scripts:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot/usr/bin/crossscripts:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/sbin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/bin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/sbin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/bin:/home/jstephan/kirkstone/src/poky/bitbake/bin:/home/jstephan/kirkstone/build/tmp/hosttools"; export HOME="/home/jstephan"; git -c core.fsyncobjectfiles=0 -c
gc.autoDetach=false -c core.pager=cat submodule update --recursive
--no-fetch failed with exit code 1, output:\nSubmodule path \'assets\':
checked out \'8a6b7f952d7786a305f89d375a55598e995ee972\'\nSubmodule path
\'third_party/CTPL\': checked out
\'437e135dbd94eb65b45533d9ce8ee28b5bd37b6d\'\nSubmodule path
\'third_party/astc\': checked out
\'4861c3db1ca50eb5f5767d95d1cd1bacb98ac8a5\'\nSubmodule path
\'third_party/cli11\': checked out
\'4af78beef777e313814b4daff70e2da9171a385a\'\nSubmodule path
\'third_party/glfw\': checked out
\'6876cf8d7e0e70dc3e4d7b0224d08312c9f78099\'\nSubmodule path
\'third_party/glm\': checked out
\'fce2abd01ce21063bd25ba67c9318be83bf48813\'\nSubmodule path
\'third_party/glslang\': checked out
\'6baa85f6ceec5fddd2e13b43dd1edc944dd930af\'\nSubmodule path
\'third_party/hwcpipe\': checked out
\'5a0c3d0b527bea96154998f29177f3b449cebd3a\'\nSubmodule path
\'third_party/imgui\': checked out
\'c23a19c26fa9cc778f755e76799f0cafdcb5a13b\'\nSubmodule path
\'third_party/opencl\': checked out
\'1aa1139b58a515877a923cce6b254e09d1b2fb2c\'\nSubmodule path
\'third_party/spdlog\': checked out
\'7bf8f148795d2571b4ae4aa9c6d2a8111fbb7077\'\nSubmodule path
\'third_party/spirv-cross\': checked out
\'84f56d0b43e5a6874de45e11be2bff9dc336fd7e\'\nSubmodule path
\'third_party/stb\': checked out
\'2c2908f50515dcd939f24be261c3ccbcd277bb49\'\nSubmodule path
\'third_party/tinygltf\': checked out
\'c0d0251e2c7f2273a894aa1b125af9313a88eae4\'\nSubmodule path
\'third_party/vma\': checked out
\'f9921aefddee2437cc2e3303d3175bd8ef23e22c\'\nSubmodule path
\'third_party/volk\': checked out
\'b18f3192886981f28d17f8f41c26c5a53ac3b977\'\nSubmodule path
\'third_party/vulkan\': checked out
\'76f00ef6cbb1886eb1162d1fa39bee8b51e22ee8\'\n\nDownloading
other_lib/ios/Debug-iphoneos/libSDL2.a (8.2 MB)\nError downloading object:
other_lib/ios/Debug-iphoneos/libSDL2.a (5045776): Smudge error: Error
downloading other_lib/ios/Debug-iphoneos/libSDL2.a
(5045776556ca57f2f7125d89c37a8b94f79cdebdc040d420513f0491787c4ea7): batch
response: Post
https://github.com/KhronosGroup/KTX-Software.git/info/lfs/objects/batch:
dial tcp: lookup github.com: Temporary failure in name
resolution\n\nErrors logged to
/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/git/.git/modules/third_party/ktx/lfs/logs/20220817T152514.337649981.log\nUse
git lfs logs last to view the log.\nerror: external filter \'git-lfs
filter-process\' failed\nfatal: other_lib/ios/Debug-iphoneos/libSDL2.a:
smudge filter lfs failed\nUnable to checkout
\'6a38a0694b4d73d22c5d1e22f865d03545e808ea\' in submodule path
\'third_party/ktx\'\n', None)

Quoted 1 line
Quoted 1 line

ERROR: Logfile of failure stored in:

/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/temp/log.do_unpack.1796011

Quoted 1 line
Quoted 1 line

ERROR: Task

(/home/jstephan/kirkstone/src/poky/meta/recipes-graphics/vulkan/vulkan-samples_git.bb:do_unpack) failed with exit code '1'

Quoted 4 lines
Quoted 4 lines

Julien

Le mer. 17 août 2022 à 21:09, Alexander Kanavin <

Quoted 7 lines · [email protected]> a écrit :
Quoted 7 lines
Quoted 7 lines

This has not been observed on the autobuilder or independently reported, can you show how to reproduce or the failure log?

Alex

On Wed, 17 Aug 2022 at 21:01, Julien STEPHAN <[email protected]>

Quoted 2 lines · wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

vulkan-samples uses git submodules and one of the submodules uses

lfs

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

objects. During the unpack step, the lfs objects are retrieved

leading to

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

a network failure.

The correct fix is to fetch lfs objects during the fetch step, but

let's

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Review

Quoted 8 lines

workaround this issue for now

---
 meta/recipes-graphics/vulkan/vulkan-samples_git.bb | 5 +++++
 1 file changed, 5 insertions(+)
(quoted diff) +0 -0

b/meta/recipes-graphics/vulkan/vulkan-samples_git.bb

Quoted 27 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

meta/recipes-graphics/vulkan/vulkan-samples_git.bb SRC_URI = "gitsm://

Quoted 4 lines

index 41cb4a437d..b45ee22a69 100644

meta/recipes-graphics/vulkan/vulkan-samples_git.bb +0 -0
@@ -11,6 +11,11 @@ SRC_URI = "gitsm://

github.com/KhronosGroup/Vulkan-Samples.git;branch=master;prot

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

file://0001-framework-core-hpp_vulkan_resource.h-add-header-incl.patch \

Quoted 19 lines
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
            "
meta/recipes-graphics/vulkan/vulkan-samples_git.bb +5 -1
+# Workaround for network access issue during the do_unpack step
+# this needs to be fixed in the recipes buildsystem to move
+# this such that it can be accomplished during do_fetch task
+do_unpack[network] = "1"
+
 UPSTREAM_CHECK_COMMITS = "1"
 SRCREV = "ee6c1522af8ba9b56f8416c2eedafb440f681085"
 
--

2.37.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#169514):
https://lists.openembedded.org/g/openembedded-core/message/169514
Mute This Topic: https://lists.openembedded.org/mt/93089137/3617156
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#8 lore raw

The ticket is RESOLVED/FIXED, which is surely not the case here?

Alex

Quoted 111 lines · Martin Jansa

There is ticket about this already, see
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14283

Quoted 106 lines · Julien STEPHAN
Quoted 6 lines · Alexander Kanavin

I reproduced. This happens if all three are true:
- recipe is using gitsm://
- one of the submodules contains lfs objects
- git-lfs is installed on the host (that's why it's not seen on the AB)

Nice! I figured out the first two points, but not the last one.

Quoted 7 lines

There's logic in plain git:// fetcher to deal with lfs, but not in gitsm://. So that logic needs to be added there as well. A workaround like you suggested is not the right fix.

Can you look into it please? Or at least a ticket would be good to have.

Sure, I will try to look at it
Julien

Quoted 75 lines

Alex

Quoted 69 lines · Alexander Kanavin via lists.openembedded.org

Just to be sure, can you repeat this on a pristine poky master checkout with no cache and no downloads? 'bitbake -c unpack vulkan-samples' should do.

Also what does git --version say? I wonder if there's something in too old or too new versions of it.

Alex

Quoted 55 lines · Julien Stephan

Hi Alexander,

I observe this by just trying to build the vulkan-samples recipe: DISTRO=poky bitbake vulkan-samples.

Here is the log output (sorry it is a bit verbose):
ERROR: vulkan-samples-git-r0 do_unpack: Bitbake Fetcher Error: FetchError('Fetch command export PSEUDO_DISABLED=1; export SSH_AUTH_SOCK="/home/jstephan/.ssh/ssh_auth_sock"; export PATH="/home/jstephan/kirkstone/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/jstephan/kirkstone/src/poky/scripts:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot/usr/bin/crossscripts:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/sbin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/bin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/sbin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/bin:/home/jstephan/kirkstone/src/poky/bitbake/bin:/home/jstephan/kirkstone/build/tmp/hosttools"; export HOME="/home/jstephan"; git -c core.fsyncobjectfiles=0 -c gc.autoDetach=false -c core.pager=cat submodule update --recursive --no-fetch failed with exit code 1, output:\nSubmodule path \'assets\': checked out \'8a6b7f952d7786a305f89d375a55598e995ee972\'\nSubmodule path \'third_party/CTPL\': checked out \'437e135dbd94eb65b45533d9ce8ee28b5bd37b6d\'\nSubmodule path \'third_party/astc\': checked out \'4861c3db1ca50eb5f5767d95d1cd1bacb98ac8a5\'\nSubmodule path \'third_party/cli11\': checked out \'4af78beef777e313814b4daff70e2da9171a385a\'\nSubmodule path \'third_party/glfw\': checked out \'6876cf8d7e0e70dc3e4d7b0224d08312c9f78099\'\nSubmodule path \'third_party/glm\': checked out \'fce2abd01ce21063bd25ba67c9318be83bf48813\'\nSubmodule path \'third_party/glslang\': checked out \'6baa85f6ceec5fddd2e13b43dd1edc944dd930af\'\nSubmodule path \'third_party/hwcpipe\': checked out \'5a0c3d0b527bea96154998f29177f3b449cebd3a\'\nSubmodule path \'third_party/imgui\': checked out \'c23a19c26fa9cc778f755e76799f0cafdcb5a13b\'\nSubmodule path \'third_party/opencl\': checked out \'1aa1139b58a515877a923cce6b254e09d1b2fb2c\'\nSubmodule path \'third_party/spdlog\': checked out \'7bf8f148795d2571b4ae4aa9c6d2a8111fbb7077\'\nSubmodule path \'third_party/spirv-cross\': checked out \'84f56d0b43e5a6874de45e11be2bff9dc336fd7e\'\nSubmodule path \'third_party/stb\': checked out \'2c2908f50515dcd939f24be261c3ccbcd277bb49\'\nSubmodule path \'third_party/tinygltf\': checked out \'c0d0251e2c7f2273a894aa1b125af9313a88eae4\'\nSubmodule path \'third_party/vma\': checked out \'f9921aefddee2437cc2e3303d3175bd8ef23e22c\'\nSubmodule path \'third_party/volk\': checked out \'b18f3192886981f28d17f8f41c26c5a53ac3b977\'\nSubmodule path \'third_party/vulkan\': checked out \'76f00ef6cbb1886eb1162d1fa39bee8b51e22ee8\'\n\nDownloading other_lib/ios/Debug-iphoneos/libSDL2.a (8.2 MB)\nError downloading object: other_lib/ios/Debug-iphoneos/libSDL2.a (5045776): Smudge error: Error downloading other_lib/ios/Debug-iphoneos/libSDL2.a (5045776556ca57f2f7125d89c37a8b94f79cdebdc040d420513f0491787c4ea7): batch response: Post https://github.com/KhronosGroup/KTX-Software.git/info/lfs/objects/batch: dial tcp: lookup github.com: Temporary failure in name resolution\n\nErrors logged to /home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/git/.git/modules/third_party/ktx/lfs/logs/20220817T152514.337649981.log\nUse git lfs logs last to view the log.\nerror: external filter \'git-lfs filter-process\' failed\nfatal: other_lib/ios/Debug-iphoneos/libSDL2.a: smudge filter lfs failed\nUnable to checkout \'6a38a0694b4d73d22c5d1e22f865d03545e808ea\' in submodule path \'third_party/ktx\'\n', None)
ERROR: Logfile of failure stored in: /home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/temp/log.do_unpack.1796011
ERROR: Task (/home/jstephan/kirkstone/src/poky/meta/recipes-graphics/vulkan/vulkan-samples_git.bb:do_unpack) failed with exit code '1'

Julien

Quoted 42 lines · Alexander Kanavin

This has not been observed on the autobuilder or independently reported, can you show how to reproduce or the failure log?

Alex

Quoted 35 lines · Julien STEPHAN

vulkan-samples uses git submodules and one of the submodules uses lfs objects. During the unpack step, the lfs objects are retrieved leading to a network failure.

The correct fix is to fetch lfs objects during the fetch step, but let's workaround this issue for now

---
 meta/recipes-graphics/vulkan/vulkan-samples_git.bb | 5 +++++
 1 file changed, 5 insertions(+)
meta/recipes-graphics/vulkan/vulkan-samples_git.bb +5 -1
index 41cb4a437d..b45ee22a69 100644
@@ -11,6 +11,11 @@ SRC_URI = "gitsm://github.com/KhronosGroup/Vulkan-Samples.git;branch=master;prot
            file://0001-framework-core-hpp_vulkan_resource.h-add-header-incl.patch \
            "
 
+# Workaround for network access issue during the do_unpack step
+# this needs to be fixed in the recipes buildsystem to move
+# this such that it can be accomplished during do_fetch task
+do_unpack[network] = "1"
+
 UPSTREAM_CHECK_COMMITS = "1"
 SRCREV = "ee6c1522af8ba9b56f8416c2eedafb440f681085"
 
--

2.37.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169514): https://lists.openembedded.org/g/openembedded-core/message/169514
Mute This Topic: https://lists.openembedded.org/mt/93089137/3617156
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#9 lore raw

https://bugzilla.yoctoproject.org/show_bug.cgi?id=14938

Quoted 189 lines · Alexander Kanavin

The ticket is RESOLVED/FIXED, which is surely not the case here?

Alex

Quoted 5 lines · Martin Jansa

There is ticket about this already, see
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14283

On Thu, Aug 18, 2022 at 9:02 AM Julien STEPHAN <[email protected]>

Quoted 4 lines · wrote:
Quoted 4 lines

Le mer. 17 août 2022 à 22:33, Alexander Kanavin <[email protected]>

Quoted 6 lines · a écrit :
Quoted 6 lines
Quoted 6 lines

I reproduced. This happens if all three are true:
- recipe is using gitsm://
- one of the submodules contains lfs objects
- git-lfs is installed on the host (that's why it's not seen on the AB)

Quoted 8 lines
Quoted 8 lines

Nice! I figured out the first two points, but not the last one.

Quoted 6 lines

There's logic in plain git:// fetcher to deal with lfs, but not in gitsm://. So that logic needs to be added there as well. A workaround like you suggested is not the right fix.

Can you look into it please? Or at least a ticket would be good to

have.

Quoted 10 lines
Quoted 10 lines
Quoted 1 line

Sure, I will try to look at it
Julien

Quoted 7 lines

Alex

Quoted 1 line · Alexander Kanavin via lists.openembedded.org
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Just to be sure, can you repeat this on a pristine poky master checkout with no cache and no downloads? 'bitbake -c unpack vulkan-samples' should do.

Also what does git --version say? I wonder if there's something in

too

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

old or too new versions of it.

Alex

On Wed, 17 Aug 2022 at 21:22, Julien Stephan <[email protected]>

Quoted 4 lines · wrote:
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Hi Alexander,

I observe this by just trying to build the vulkan-samples recipe:

DISTRO=poky bitbake vulkan-samples.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Here is the log output (sorry it is a bit verbose):
ERROR: vulkan-samples-git-r0 do_unpack: Bitbake Fetcher Error:

FetchError('Fetch command export PSEUDO_DISABLED=1; export
SSH_AUTH_SOCK="/home/jstephan/.ssh/ssh_auth_sock"; export PATH="/home/jstephan/kirkstone/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/jstephan/kirkstone/src/poky/scripts:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/bin/aarch64-poky-linux:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot/usr/bin/crossscripts:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/sbin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/usr/bin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/sbin:/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/recipe-sysroot-native/bin:/home/jstephan/kirkstone/src/poky/bitbake/bin:/home/jstephan/kirkstone/build/tmp/hosttools"; export HOME="/home/jstephan"; git -c core.fsyncobjectfiles=0 -c
gc.autoDetach=false -c core.pager=cat submodule update --recursive
--no-fetch failed with exit code 1, output:\nSubmodule path \'assets\':
checked out \'8a6b7f952d7786a305f89d375a55598e995ee972\'\nSubmodule path
\'third_party/CTPL\': checked out
\'437e135dbd94eb65b45533d9ce8ee28b5bd37b6d\'\nSubmodule path
\'third_party/astc\': checked out
\'4861c3db1ca50eb5f5767d95d1cd1bacb98ac8a5\'\nSubmodule path
\'third_party/cli11\': checked out
\'4af78beef777e313814b4daff70e2da9171a385a\'\nSubmodule path
\'third_party/glfw\': checked out
\'6876cf8d7e0e70dc3e4d7b0224d08312c9f78099\'\nSubmodule path
\'third_party/glm\': checked out
\'fce2abd01ce21063bd25ba67c9318be83bf48813\'\nSubmodule path
\'third_party/glslang\': checked out
\'6baa85f6ceec5fddd2e13b43dd1edc944dd930af\'\nSubmodule path
\'third_party/hwcpipe\': checked out
\'5a0c3d0b527bea96154998f29177f3b449cebd3a\'\nSubmodule path
\'third_party/imgui\': checked out
\'c23a19c26fa9cc778f755e76799f0cafdcb5a13b\'\nSubmodule path
\'third_party/opencl\': checked out
\'1aa1139b58a515877a923cce6b254e09d1b2fb2c\'\nSubmodule path
\'third_party/spdlog\': checked out
\'7bf8f148795d2571b4ae4aa9c6d2a8111fbb7077\'\nSubmodule path
\'third_party/spirv-cross\': checked out
\'84f56d0b43e5a6874de45e11be2bff9dc336fd7e\'\nSubmodule path
\'third_party/stb\': checked out
\'2c2908f50515dcd939f24be261c3ccbcd277bb49\'\nSubmodule path
\'third_party/tinygltf\': checked out
\'c0d0251e2c7f2273a894aa1b125af9313a88eae4\'\nSubmodule path
\'third_party/vma\': checked out
\'f9921aefddee2437cc2e3303d3175bd8ef23e22c\'\nSubmodule path
\'third_party/volk\': checked out
\'b18f3192886981f28d17f8f41c26c5a53ac3b977\'\nSubmodule path
\'third_party/vulkan\': checked out
\'76f00ef6cbb1886eb1162d1fa39bee8b51e22ee8\'\n\nDownloading
other_lib/ios/Debug-iphoneos/libSDL2.a (8.2 MB)\nError downloading object:
other_lib/ios/Debug-iphoneos/libSDL2.a (5045776): Smudge error: Error
downloading other_lib/ios/Debug-iphoneos/libSDL2.a
(5045776556ca57f2f7125d89c37a8b94f79cdebdc040d420513f0491787c4ea7): batch
response: Post
https://github.com/KhronosGroup/KTX-Software.git/info/lfs/objects/batch:
dial tcp: lookup github.com: Temporary failure in name
resolution\n\nErrors logged to
/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/git/.git/modules/third_party/ktx/lfs/logs/20220817T152514.337649981.log\nUse
git lfs logs last to view the log.\nerror: external filter \'git-lfs
filter-process\' failed\nfatal: other_lib/ios/Debug-iphoneos/libSDL2.a:
smudge filter lfs failed\nUnable to checkout
\'6a38a0694b4d73d22c5d1e22f865d03545e808ea\' in submodule path
\'third_party/ktx\'\n', None)

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

ERROR: Logfile of failure stored in:

/home/jstephan/kirkstone/build/tmp/work/cortexa53-poky-linux/vulkan-samples/git-r0/temp/log.do_unpack.1796011

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

ERROR: Task

(/home/jstephan/kirkstone/src/poky/meta/recipes-graphics/vulkan/vulkan-samples_git.bb:do_unpack) failed with exit code '1'

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Julien

Le mer. 17 août 2022 à 21:09, Alexander Kanavin <

Quoted 7 lines · [email protected]> a écrit :
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

This has not been observed on the autobuilder or independently reported, can you show how to reproduce or the failure log?

Alex

On Wed, 17 Aug 2022 at 21:01, Julien STEPHAN <

Quoted 2 lines · [email protected]> wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

vulkan-samples uses git submodules and one of the submodules

uses lfs

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

objects. During the unpack step, the lfs objects are retrieved

leading to

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

a network failure.

The correct fix is to fetch lfs objects during the fetch step,

but let's

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Review

Quoted 8 lines

workaround this issue for now

---
 meta/recipes-graphics/vulkan/vulkan-samples_git.bb | 5 +++++
 1 file changed, 5 insertions(+)
(quoted diff) +0 -0

b/meta/recipes-graphics/vulkan/vulkan-samples_git.bb

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

meta/recipes-graphics/vulkan/vulkan-samples_git.bb SRC_URI = "gitsm://

Quoted 4 lines

index 41cb4a437d..b45ee22a69 100644

meta/recipes-graphics/vulkan/vulkan-samples_git.bb +0 -0
@@ -11,6 +11,11 @@ SRC_URI = "gitsm://

github.com/KhronosGroup/Vulkan-Samples.git;branch=master;prot

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

file://0001-framework-core-hpp_vulkan_resource.h-add-header-incl.patch \

Quoted 21 lines
Quoted 21 lines
Quoted 19 lines
Quoted 19 lines
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
            "
meta/recipes-graphics/vulkan/vulkan-samples_git.bb +5 -1
+# Workaround for network access issue during the do_unpack step
+# this needs to be fixed in the recipes buildsystem to move
+# this such that it can be accomplished during do_fetch task
+do_unpack[network] = "1"
+
 UPSTREAM_CHECK_COMMITS = "1"
 SRCREV = "ee6c1522af8ba9b56f8416c2eedafb440f681085"
 
--

2.37.1

Quoted 3 lines
Quoted 3 lines

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#169514):

https://lists.openembedded.org/g/openembedded-core/message/169514

Quoted 3 lines
Quoted 3 lines

Mute This Topic: https://lists.openembedded.org/mt/93089137/3617156
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [

[email protected]]

Quoted 2 lines
Quoted 2 lines

-=-=-=-=-=-=-=-=-=-=-=-

#10 lore raw
In reply to Julien STEPHAN
Quoted 7 lines · Julien STEPHAN

vulkan-samples uses git submodules and one of the submodules uses lfs objects. During the unpack step, the lfs objects are retrieved leading to a network failure.

The correct fix is to fetch lfs objects during the fetch step, but let's workaround this issue for now

How can you ensure you are not relying on third parties to sotre the code you need to build your images then?

Quoted 34 lines
---
 meta/recipes-graphics/vulkan/vulkan-samples_git.bb | 5 +++++
 1 file changed, 5 insertions(+)
meta/recipes-graphics/vulkan/vulkan-samples_git.bb +5 -0
index 41cb4a437d..b45ee22a69 100644
@@ -11,6 +11,11 @@ SRC_URI = "gitsm://github.com/KhronosGroup/Vulkan-Samples.git;branch=master;prot
            file://0001-framework-core-hpp_vulkan_resource.h-add-header-incl.patch \
            "
 
+# Workaround for network access issue during the do_unpack step
+# this needs to be fixed in the recipes buildsystem to move
+# this such that it can be accomplished during do_fetch task
+do_unpack[network] = "1"
+
 UPSTREAM_CHECK_COMMITS = "1"
 SRCREV = "ee6c1522af8ba9b56f8416c2eedafb440f681085"
 
Signature · 11 lines
2.37.1



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#169500): https://lists.openembedded.org/g/openembedded-core/message/169500
Mute This Topic: https://lists.openembedded.org/mt/93089137/3617179
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com