package/pkg-download: restore user's original umask for the whole download process
cover package/pkg-download: restore user's original umask for the whole download process
Hi Yann, all,
this is the revised patch based on Yann's suggestion from this thread:
https://lists.buildroot.org/pipermail/buildroot/2022-December/656833.html
There is one more thing to it, though: the original user's umask is saved only if it is more permissive than BR minimum requirement (0022). This is to guarantee that the download process can always create files/dirs just as before. The logic of this check is a bit tricky, and requires bitwise operations on octal numbers... I had to do it in shell :(
Regards,
Luca
Luca Pesce (1):
package/pkg-download: restore user's original umask for the whole
download process
Makefile | 17 ++++++++++++++++- package/pkg-download.mk | 6 ++++-- 2 files changed, 20 insertions(+), 3 deletions(-)
Signature · 6 lines
2.7.4 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
PATCH 1/1 package/pkg-download: restore user's original umask for the whole download process
Root makefile imposes 'umask 0022', and this may be more restrictive than the user's original umask - which could have provisions set to share files/dirs with other users.
As an example, the imposed value makes the per-package download directories not writeable for the group, but just for the owner - the user that issued the first build that populated the per-package dl dir for the first time (say user A). Thus, if a BR package changes its version (e.g. for buildroot update), and another user (say user B, in the same group of A) starts a build, BR fails the creation of package-xxx.tar.gz inside the dl dir, because user B has no write permissions on that path. Furthermore, in the case of the git backend, this makes the git cache not updatable by a different user. This is disruptive for a host used by many users, all belonging to a certain group.
So, to allow sharing of a rw BR2_DL_DIR location among users, we save the original umask value (but only if it is more permissive than our REQ_UMASK, which is our minimum working requirement) and restore it during the download process.
--- Makefile | 17 ++++++++++++++++- package/pkg-download.mk | 6 ++++-- 2 files changed, 20 insertions(+), 3 deletions(-)
Makefile +16 -1
@@ -68,21 +68,31 @@ endif CANONICAL_CURDIR = $(realpath $(CURDIR)) REQ_UMASK = 0022 +CURR_UMASK = $(shell umask) # Make sure O= is passed (with its absolute canonical path) everywhere the # toplevel makefile is called back. EXTRAMAKEARGS := O=$(CANONICAL_O) # Check Buildroot execution pre-requisites here. -ifneq ($(shell umask):$(CURDIR):$(O),$(REQ_UMASK):$(CANONICAL_CURDIR):$(CANONICAL_O)) +ifneq ($(CURR_UMASK):$(CURDIR):$(O),$(REQ_UMASK):$(CANONICAL_CURDIR):$(CANONICAL_O)) .PHONY: _all $(MAKECMDGOALS) +# Save the user's original umask (but only if it is more permissive than or +# equal to REQ_UMASK, which is our minimum working requirement) +ifeq ($(shell printf "%04o" "$$(( $(CURR_UMASK) | $(REQ_UMASK) ))"),$(REQ_UMASK)) +BR2_ORIG_UMASK := $(CURR_UMASK) +else +BR2_ORIG_UMASK := $(REQ_UMASK) +endif + $(MAKECMDGOALS): _all @: _all: @umask $(REQ_UMASK) && \ $(MAKE) -C $(CANONICAL_CURDIR) --no-print-directory \ + BR2_ORIG_UMASK=$(BR2_ORIG_UMASK) \ $(MAKECMDGOALS) $(EXTRAMAKEARGS) else # umask / $(CURDIR) / $(O)
@@ -91,6 +101,11 @@ else # umask / $(CURDIR) / $(O) all: .PHONY: all +# Export the user's original umask: we may need it later to honour user's file +# creation permissions when creating non-target files/dirs (e.g. during the +# execution of dl-wrapper script) +export BR2_ORIG_UMASK := $(or $(BR2_ORIG_UMASK),$(CURR_UMASK)) + # Set and export the version string export BR2_VERSION := 2023.02-git # Actual time the release is cut (for reproducible builds)
package/pkg-download.mk +4 -2
@@ -107,9 +107,11 @@ endif # ################################################################################ +# Restore the user's original umask during the whole download, in case he has +# provisions set to share the download directory with his group (or others). define DOWNLOAD - $(Q)mkdir -p $($(2)_DL_DIR) - $(Q)$(EXTRA_ENV) $($(2)_DL_ENV) \ + $(Q)umask $(BR2_ORIG_UMASK); mkdir -p $($(2)_DL_DIR) + $(Q)umask $(BR2_ORIG_UMASK); $(EXTRA_ENV) $($(2)_DL_ENV) \ flock $($(2)_DL_DIR)/.lock $(DL_WRAPPER) \ -c '$($(2)_DL_VERSION)' \ -d '$($(2)_DL_DIR)' \
Signature · 6 lines
2.7.4 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Hi Luca,
Sorry for the extremely long turnaround time for this. Messing with umasks is
something that nobody really wants to think about because it entails all kinds of corner cases... In the end, though, it took me just a little more than half an hour to get this merged.
Quoted 13 lines · Luca Pesce via buildroot
Root makefile imposes 'umask 0022', and this may be more restrictive than the user's original umask - which could have provisions set to share files/dirs with other users.
As an example, the imposed value makes the per-package download directories not writeable for the group, but just for the owner - the user that issued the first build that populated the per-package dl dir for the first time (say user A). Thus, if a BR package changes its version (e.g. for buildroot update), and another user (say user B, in the same group of A) starts a build, BR fails the creation of package-xxx.tar.gz inside the dl dir, because user B has no write permissions on that path. Furthermore, in the case of the git backend, this makes the git cache not updatable by a different user. This is disruptive for a host used by many users, all belonging to a certain group.
Quoted 6 lines
So, to allow sharing of a rw
BR2_DL_DIRlocation among users, we save the original umask value (but only if it is more permissive than ourREQ_UMASK, which is our minimum working requirement) and restore it during the download process.Signed-off-by Luca Pesce <[email protected]>
Applied to master, thanks, with the following changes:
- CURR_UMASK -> CUR_UMASK.
- BR2_ORIG_UMASK -> BR_ORIG_UMASK.
- Don't check if the umask is more permissive, apply it regardless. If the user explicitly doesn't want to make their DL_DIR readable by others, that's fine.
- Don't export BR_ORIG_UMASK.
- Only set BR_ORIG_UMASK if it we recurse, and only set umask if BR_ORIG_UMASK is set.
- Add DOWNLOAD_SET_UMASK to simplify the latter.
I hope it helps you!
Regards,
Arnout
Quoted 62 lines
--- Makefile | 17 ++++++++++++++++- package/pkg-download.mk | 6 ++++-- 2 files changed, 20 insertions(+), 3 deletions(-)Makefile +16
-1@@ -68,21 +68,31 @@ endif CANONICAL_CURDIR = $(realpath $(CURDIR)) REQ_UMASK = 0022 +CURR_UMASK = $(shell umask) # Make sure O= is passed (with its absolute canonical path) everywhere the # toplevel makefile is called back. EXTRAMAKEARGS := O=$(CANONICAL_O) # Check Buildroot execution pre-requisites here. -ifneq ($(shell umask):$(CURDIR):$(O),$(REQ_UMASK):$(CANONICAL_CURDIR):$(CANONICAL_O)) +ifneq ($(CURR_UMASK):$(CURDIR):$(O),$(REQ_UMASK):$(CANONICAL_CURDIR):$(CANONICAL_O)) .PHONY: _all $(MAKECMDGOALS) +# Save the user's original umask (but only if it is more permissive than or +# equal to REQ_UMASK, which is our minimum working requirement) +ifeq ($(shell printf "%04o" "$$(( $(CURR_UMASK) | $(REQ_UMASK) ))"),$(REQ_UMASK)) +BR2_ORIG_UMASK := $(CURR_UMASK) +else +BR2_ORIG_UMASK := $(REQ_UMASK) +endif + $(MAKECMDGOALS): _all @: _all: @umask $(REQ_UMASK) && \ $(MAKE) -C $(CANONICAL_CURDIR) --no-print-directory \ + BR2_ORIG_UMASK=$(BR2_ORIG_UMASK) \ $(MAKECMDGOALS) $(EXTRAMAKEARGS) else # umask / $(CURDIR) / $(O)@@ -91,6 +101,11 @@ else # umask / $(CURDIR) / $(O) all: .PHONY: all +# Export the user's original umask: we may need it later to honour user's file +# creation permissions when creating non-target files/dirs (e.g. during the +# execution of dl-wrapper script) +export BR2_ORIG_UMASK := $(or $(BR2_ORIG_UMASK),$(CURR_UMASK)) + # Set and export the version string export BR2_VERSION := 2023.02-git # Actual time the release is cut (for reproducible builds)package/pkg-download.mk +0
-0@@ -107,9 +107,11 @@ endif # ################################################################################
package/pkg-download.mk endif · line 117
Quoted 10 lines
package/pkg-download.mk +4
-2+# Restore the user's original umask during the whole download, in case he has +# provisions set to share the download directory with his group (or others). define DOWNLOAD - $(Q)mkdir -p $($(2)_DL_DIR) - $(Q)$(EXTRA_ENV) $($(2)_DL_ENV) \ + $(Q)umask $(BR2_ORIG_UMASK); mkdir -p $($(2)_DL_DIR) + $(Q)umask $(BR2_ORIG_UMASK); $(EXTRA_ENV) $($(2)_DL_ENV) \ flock $($(2)_DL_DIR)/.lock $(DL_WRAPPER) \ -c '$($(2)_DL_VERSION)' \ -d '$($(2)_DL_DIR)' \
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
[Buildroot] R: [PATCH 1/1] package/pkg-download: restore user's original umask for the whole download process
Hi Arnout,
Quoted 3 lines
Sorry for the extremely long turnaround time for this. Messing with umasks is something that nobody really wants to think about because it entails all kinds of corner cases...
yes, indeed - also, the patch was addressing a minor issue, so I was prepared for a certain review delay, no problem.
Quoted 1 line
I hope it helps you!
Of course it does, thank you!
Regards,
Luca
________________________________________
Da: Arnout Vandecappelle <[email protected]>
Inviato: domenica 14 luglio 2024 15:13
A: Pesce Luca; [email protected]
Cc: Yann E . MORIN
Oggetto: Re: [Buildroot] [PATCH 1/1] package/pkg-download: restore user's original umask for the whole download process
Hi Luca,
Sorry for the extremely long turnaround time for this. Messing with umasks is
something that nobody really wants to think about because it entails all kinds of corner cases... In the end, though, it took me just a little more than half an hour to get this merged.
Quoted 13 lines · Luca Pesce via buildroot
Root makefile imposes 'umask 0022', and this may be more restrictive than the user's original umask - which could have provisions set to share files/dirs with other users.
As an example, the imposed value makes the per-package download directories not writeable for the group, but just for the owner - the user that issued the first build that populated the per-package dl dir for the first time (say user A). Thus, if a BR package changes its version (e.g. for buildroot update), and another user (say user B, in the same group of A) starts a build, BR fails the creation of package-xxx.tar.gz inside the dl dir, because user B has no write permissions on that path. Furthermore, in the case of the git backend, this makes the git cache not updatable by a different user. This is disruptive for a host used by many users, all belonging to a certain group.
Quoted 6 lines
So, to allow sharing of a rw
BR2_DL_DIRlocation among users, we save the original umask value (but only if it is more permissive than ourREQ_UMASK, which is our minimum working requirement) and restore it during the download process.Signed-off-by Luca Pesce <[email protected]>
Applied to master, thanks, with the following changes:
- CURR_UMASK -> CUR_UMASK.
- BR2_ORIG_UMASK -> BR_ORIG_UMASK.
- Don't check if the umask is more permissive, apply it regardless. If the user explicitly doesn't want to make their DL_DIR readable by others, that's fine.
- Don't export BR_ORIG_UMASK.
- Only set BR_ORIG_UMASK if it we recurse, and only set umask if BR_ORIG_UMASK is set.
- Add DOWNLOAD_SET_UMASK to simplify the latter.
I hope it helps you!
Regards,
Arnout
Quoted 62 lines
--- Makefile | 17 ++++++++++++++++- package/pkg-download.mk | 6 ++++-- 2 files changed, 20 insertions(+), 3 deletions(-)Makefile +16
-1@@ -68,21 +68,31 @@ endif CANONICAL_CURDIR = $(realpath $(CURDIR)) REQ_UMASK = 0022 +CURR_UMASK = $(shell umask) # Make sure O= is passed (with its absolute canonical path) everywhere the # toplevel makefile is called back. EXTRAMAKEARGS := O=$(CANONICAL_O) # Check Buildroot execution pre-requisites here. -ifneq ($(shell umask):$(CURDIR):$(O),$(REQ_UMASK):$(CANONICAL_CURDIR):$(CANONICAL_O)) +ifneq ($(CURR_UMASK):$(CURDIR):$(O),$(REQ_UMASK):$(CANONICAL_CURDIR):$(CANONICAL_O)) .PHONY: _all $(MAKECMDGOALS) +# Save the user's original umask (but only if it is more permissive than or +# equal to REQ_UMASK, which is our minimum working requirement) +ifeq ($(shell printf "%04o" "$$(( $(CURR_UMASK) | $(REQ_UMASK) ))"),$(REQ_UMASK)) +BR2_ORIG_UMASK := $(CURR_UMASK) +else +BR2_ORIG_UMASK := $(REQ_UMASK) +endif + $(MAKECMDGOALS): _all @: _all: @umask $(REQ_UMASK) && \ $(MAKE) -C $(CANONICAL_CURDIR) --no-print-directory \ + BR2_ORIG_UMASK=$(BR2_ORIG_UMASK) \ $(MAKECMDGOALS) $(EXTRAMAKEARGS) else # umask / $(CURDIR) / $(O)@@ -91,6 +101,11 @@ else # umask / $(CURDIR) / $(O) all: .PHONY: all +# Export the user's original umask: we may need it later to honour user's file +# creation permissions when creating non-target files/dirs (e.g. during the +# execution of dl-wrapper script) +export BR2_ORIG_UMASK := $(or $(BR2_ORIG_UMASK),$(CURR_UMASK)) + # Set and export the version string export BR2_VERSION := 2023.02-git # Actual time the release is cut (for reproducible builds)package/pkg-download.mk +0
-0@@ -107,9 +107,11 @@ endif # ################################################################################
package/pkg-download.mk endif · line 117
Quoted 10 lines
package/pkg-download.mk +4
-2+# Restore the user's original umask during the whole download, in case he has +# provisions set to share the download directory with his group (or others). define DOWNLOAD - $(Q)mkdir -p $($(2)_DL_DIR) - $(Q)$(EXTRA_ENV) $($(2)_DL_ENV) \ + $(Q)umask $(BR2_ORIG_UMASK); mkdir -p $($(2)_DL_DIR) + $(Q)umask $(BR2_ORIG_UMASK); $(EXTRA_ENV) $($(2)_DL_ENV) \ flock $($(2)_DL_DIR)/.lock $(DL_WRAPPER) \ -c '$($(2)_DL_VERSION)' \ -d '$($(2)_DL_DIR)' \
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot