Skip to contentSkip to conversation

Revert "package/pkg-generic.mk: create directories upfront in the configure step"

PATCH New Yann E. MORIN 1 participant 2 messages Aug 1, 2020 – Aug 2, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Revert "package/pkg-generic.mk: create directories upfront in the configure step"

This _partially_ reverts commit 2dbb0660fb4d89f8eac86fcff08e9b3d0a705917.

Indeed, before 2dbb0660fb, the per-package ../host/ directory was explicitly created right before it was populated. With 2dbb0660fb, it is now created only at the begining of the configure step.

However, some packages may have download or extract host dependencies, and when we are trying to rsync them, the destination .../host/ directory does not yet exist at the download or wxtract steps, thus the build fails. E.g., with wget which needs host-lzip if the system does not have it yet:

    rsync -a --link-dest=/home/ymorin/dev/buildroot/O/per-package/host-lzip/host/ /home/ymorin/dev/buildroot/O/per-package/host-lzip/host/ /home/ymorin/dev/buildroot/O/per-package/wget/host
    rsync: mkdir "/home/ymorin/dev/buildroot/O/per-package/wget/host" failed: No such file or directory (2)
    rsync error: error in file IO (code 11) at main.c(682) [Receiver=3.1.3]
    make[1]: *** [package/pkg-generic.mk:170: /home/ymorin/dev/buildroot/O/build/wget-1.20.3/.stamp_extracted] Error 11

Kudos to Raphael for pin-pointing the culprit commit.

Fixes:
http://autobuild.buildroot.org/results/4c8/4c895c7ac0a635d38f5015b5df0393581ad7b508/
and a slew of similar failures

Notes:
An alternatative solution would be to move the creation of the directories from the configure step to the download step *and* to the rsync step, but this is not so nice in the end... Let's just ensure they are created where needed: at the configure step, and when populating them.

---
 package/pkg-utils.mk | 1 +
 1 file changed, 1 insertion(+)
package/pkg-utils.mk +1 -0
index f6de295b7f..ad7a84475a 100644
@@ -167,6 +167,7 @@ ifeq ($(BR2_PER_PACKAGE_DIRECTORIES),y)
 # $2: 'host' or 'target'
 # $3: destination directory
 define per-package-rsync
+	mkdir -p $(3)
 	$(foreach pkg,$(1),\
 		rsync -a --link-dest=$(PER_PACKAGE_DIR)/$(pkg)/$(2)/ \
 		$(PER_PACKAGE_DIR)/$(pkg)/$(2)/ \
Signature · 1 line
2.20.1
author Applied
#1 lore raw

PATCH 1/1 Revert "package/pkg-generic.mk: create directories upfront in the configure step"

All,

On 2020-08-01 14:01 +0200, Yann E. MORIN spake thusly:

Quoted 30 lines

This _partially_ reverts commit 2dbb0660fb4d89f8eac86fcff08e9b3d0a705917.

Indeed, before 2dbb0660fb, the per-package ../host/ directory was explicitly created right before it was populated. With 2dbb0660fb, it is now created only at the begining of the configure step.

However, some packages may have download or extract host dependencies, and when we are trying to rsync them, the destination .../host/ directory does not yet exist at the download or wxtract steps, thus the build fails. E.g., with wget which needs host-lzip if the system does not have it yet:

    rsync -a --link-dest=/home/ymorin/dev/buildroot/O/per-package/host-lzip/host/ /home/ymorin/dev/buildroot/O/per-package/host-lzip/host/ /home/ymorin/dev/buildroot/O/per-package/wget/host
    rsync: mkdir "/home/ymorin/dev/buildroot/O/per-package/wget/host" failed: No such file or directory (2)
    rsync error: error in file IO (code 11) at main.c(682) [Receiver=3.1.3]
    make[1]: *** [package/pkg-generic.mk:170: /home/ymorin/dev/buildroot/O/build/wget-1.20.3/.stamp_extracted] Error 11

Kudos to Raphael for pin-pointing the culprit commit.

Fixes:
http://autobuild.buildroot.org/results/4c8/4c895c7ac0a635d38f5015b5df0393581ad7b508/
and a slew of similar failures

Notes:
An alternatative solution would be to move the creation of the directories from the configure step to the download step *and* to the rsync step, but this is not so nice in the end... Let's just ensure they are created where needed: at the configure step, and when populating them.

Quoted 3 lines

Applied to master.

Thomas, I applied as a stop-gap measure, but also because I believe this is semantically correct to do so.

Should you think otherwise, this leaves us a bit more time to devise a better solution.

Regards,
Yann E. MORIN.

Quoted 19 lines
---
 package/pkg-utils.mk | 1 +
 1 file changed, 1 insertion(+)
package/pkg-utils.mk +1 -0
index f6de295b7f..ad7a84475a 100644
@@ -167,6 +167,7 @@ ifeq ($(BR2_PER_PACKAGE_DIRECTORIES),y)
 # $2: 'host' or 'target'
 # $3: destination directory
 define per-package-rsync
+	mkdir -p $(3)
 	$(foreach pkg,$(1),\
 		rsync -a --link-dest=$(PER_PACKAGE_DIR)/$(pkg)/$(2)/ \
 		$(PER_PACKAGE_DIR)/$(pkg)/$(2)/ \
Signature · 1 line
2.20.1
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'