Skip to contentSkip to conversation

core/download: fix when the BR2_DL_DIR does not accept hardlinks

RFC PATCH New Yann E. MORIN 2 participants 2 messages Apr 3, 2018

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 core/download: fix when the BR2_DL_DIR does not accept hardlinks

When the BR2_DL_DIR is a mountpoint (presumably shared between various machine, or mounted from the local host when running in a VM), it is possible that it does not support hardlinks (e.g. samba, or the VMWare VMFS, etc...).

If the hardlink fails, fallback to copying the file. As a last resort, if that also fails, eventually fallback to doing the download.

Note: this means that the dl-wrapper is no longer atomic-safe: the code suffers of a TOCTTOU condition: the file may be created in-between the check and the moment we try to ln/cp it. Fortnately, the dl-wrapper is now run under an flock, so we're still safe. If we eventually go for a more fine-grained implementation, we'll have to be careful then.

Cc Arnout Vandecappelle <[email protected]>
Cc Peter Korsgaard <[email protected]>
Cc Maxime Hadjinlian <[email protected]>

---
Note: the TOCTTOU condition has been introduced e80d1d0af448.

Note: RFC, because I didn't test it...

---
 support/download/dl-wrapper | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
support/download/dl-wrapper +6 -1
index ce44752df0..38430738eb 100755
@@ -56,8 +56,13 @@ main() {
     # Legacy handling: check if the file already exists in the global
     # download directory. If it does, hard-link it. If it turns out it
     # was an incorrect download, we'd still check it below anyway.
+    # If we can neither link nor copy, fallback to doing a download.
+    # NOTE! This is not atomic, is subject to TOCTTOU, but the whole
+    # dl-wrapper runs under an flock, so we're safe.
     if [ ! -e "${output}" -a -e "${old_dl_dir}/${filename}" ]; then
-        ln "${old_dl_dir}/${filename}" "${output}"
+        ln "${old_dl_dir}/${filename}" "${output}" || \
+        cp "${old_dl_dir}/${filename}" "${output}" || \
+        true
     fi
 
     # If the output file already exists and:
Signature · 1 line
2.14.1
#1 lore raw

RFC PATCH 1/1 core/download: fix when the BR2_DL_DIR does not accept hardlinks

Quoted 7 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

When the BR2_DL_DIR is a mountpoint (presumably shared between various machine, or mounted from the local host when running in a VM), it is possible that it does not support hardlinks (e.g. samba, or the VMWare VMFS, etc...).

Quoted 6 lines

If the hardlink fails, fallback to copying the file. As a last resort, if that also fails, eventually fallback to doing the download.

Note: this means that the dl-wrapper is no longer atomic-safe: the code suffers of a TOCTTOU condition: the file may be created in-between the check and the moment we try to ln/cp it. Fortnately, the dl-wrapper is

s/Fortnately/Fortunately/

Committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard