Skip to contentSkip to conversation

packages/pkg-download.mk: fix file locking over NFS

PATCH New Maxim Cournoyer 3 participants 4 messages Jun 25, 2020 – Jul 16, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 packages/pkg-download.mk: fix file locking over NFS

Hello,

The attached patch fixes the NFS lock mechanism not working.

Tested using NFS 4.2 and multiple build machines caching the same source artifacts in parallel to an empty NFS directory.

Thank you,

Maxim
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-packages-pkg-download.mk-fix-file-locking-over-NFS.patch
Type: text/x-patch
Size: 1274 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20200625/7ed64a2c/attachment.bin>

#1 lore raw

PATCH 1/1 packages/pkg-download.mk: fix file locking over NFS

Maxim, All,

On 2020-06-25 14:11 -0400, Maxim Cournoyer spake thusly:

Quoted 1 line

The attached patch fixes the NFS lock mechanism not working.

You should talk to your Savoir-faire Linux colleagues: they manage to send patches to the list.

Quoted 17 lines

Tested using NFS 4.2 and multiple build machines caching the same source artifacts in parallel to an empty NFS directory.

Thank you,

Maxim

From 0f2dcace04d7c386b25888a4c851feddd187588f Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <[email protected]>
Date: Thu, 4 Jun 2020 16:14:26 -0400
Subject: [PATCH] packages/pkg-download.mk: fix file locking over NFS

BSD style locks such as implemented by flock are translated to POSIX advisory file locks (implemented by the fcntl system call on Linux). It is not possible to lock a directory using POSIX advisory file locks. Hence, the lock strategy used by Buildroot doesn't work when used over NFS.

Quoted 6 lines

Using flock on a simple file works correctly though, so use a '.lock' file inside the download directory instead.

Tested using NFS v4.2 and Linux 5.4.43.

I've added a blurb about the creation of the lock file when it is missing, and applied to master. Thanks.

Regards,
Yann E. MORIN.

Quoted 25 lines
---
 package/pkg-download.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
package/pkg-download.mk +1 -1
index de619ba90a..951d2fb554 100644
@@ -99,7 +99,7 @@ endif
 
 define DOWNLOAD
 	$(Q)mkdir -p $($(2)_DL_DIR)
-	$(Q)$(EXTRA_ENV) flock $($(2)_DL_DIR)/ $(DL_WRAPPER) \
+	$(Q)$(EXTRA_ENV) flock $($(2)_DL_DIR)/.lock $(DL_WRAPPER) \
 		-c '$($(2)_DL_VERSION)' \
 		-d '$($(2)_DL_DIR)' \
 		-D '$(DL_DIR)' \
Signature · 7 lines
2.26.2


_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#2 lore raw

PATCH 1/1 packages/pkg-download.mk: fix file locking over NFS

Hello Yann,

----- Original Message -----

Quoted 6 lines

From: "Yann E. MORIN" <[email protected]>
To: "Maxim Cournoyer" <[email protected]>
Cc: "buildroot" <[email protected]>
Sent: Thursday, June 25, 2020 4:27:18 PM
Subject: Re: [Buildroot] [PATCH] packages/pkg-download.mk: fix file locking over NFS

Quoted 7 lines

Maxim, All,

On 2020-06-25 14:11 -0400, Maxim Cournoyer spake thusly:

Quoted 1 line

The attached patch fixes the NFS lock mechanism not working.

You should talk to your Savoir-faire Linux colleagues: they manage to send patches to the list.

I'll definitly look into it prior to my next contribution :-).

Quoted 2 lines

I've added a blurb about the creation of the lock file when it is missing, and applied to master. Thanks.

That was quick! Thank you for the prompt resolution.

Cheers,

Maxim

#3 lore raw

PATCH 1/1 packages/pkg-download.mk: fix file locking over NFS

In reply to Maxim Cournoyer
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Maxim" == Maxim Cournoyer <[email protected]> writes:

Hello,
The attached patch fixes the NFS lock mechanism not working.

Tested using NFS 4.2 and multiple build machines caching the same source artifacts in parallel to an empty NFS directory.

Committed to 2020.02.x and 2020.05.x, thanks.

Signature · 1 line
Bye, Peter Korsgaard