Skip to contentSkip to conversation

package/pkg-download: restore user's original umask for the whole download process

PATCH In review Luca Pesce via buildroot 1 participant 4 messages Dec 6, 2022 – Jul 17, 2024 Kbuild

lore
Share & tools
lore
#0 lore raw

cover package/pkg-download: restore user's original umask for the whole download process

Hi Yann, all,
this is the revised patch based on Yann's suggestion from this thread:
https://lists.buildroot.org/pipermail/buildroot/2022-December/656833.html

There is one more thing to it, though: the original user's umask is saved only if it is more permissive than BR minimum requirement (0022). This is to guarantee that the download process can always create files/dirs just as before. The logic of this check is a bit tricky, and requires bitwise operations on octal numbers... I had to do it in shell :(

Regards,
Luca

Luca Pesce (1):
package/pkg-download: restore user's original umask for the whole
download process

 Makefile                | 17 ++++++++++++++++-
 package/pkg-download.mk |  6 ++++--
 2 files changed, 20 insertions(+), 3 deletions(-)
Signature · 6 lines
2.7.4

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#1 lore raw

PATCH 1/1 package/pkg-download: restore user's original umask for the whole download process

Root makefile imposes 'umask 0022', and this may be more restrictive than the user's original umask - which could have provisions set to share files/dirs with other users.
As an example, the imposed value makes the per-package download directories not writeable for the group, but just for the owner - the user that issued the first build that populated the per-package dl dir for the first time (say user A). Thus, if a BR package changes its version (e.g. for buildroot update), and another user (say user B, in the same group of A) starts a build, BR fails the creation of package-xxx.tar.gz inside the dl dir, because user B has no write permissions on that path. Furthermore, in the case of the git backend, this makes the git cache not updatable by a different user. This is disruptive for a host used by many users, all belonging to a certain group.

So, to allow sharing of a rw BR2_DL_DIR location among users, we save the original umask value (but only if it is more permissive than our REQ_UMASK, which is our minimum working requirement) and restore it during the download process.

---
 Makefile                | 17 ++++++++++++++++-
 package/pkg-download.mk |  6 ++++--
 2 files changed, 20 insertions(+), 3 deletions(-)
Makefile +16 -1
index 88f90cd..ec45589 100644
@@ -68,21 +68,31 @@ endif
 CANONICAL_CURDIR = $(realpath $(CURDIR))
 
 REQ_UMASK = 0022
+CURR_UMASK = $(shell umask)
 
 # Make sure O= is passed (with its absolute canonical path) everywhere the
 # toplevel makefile is called back.
 EXTRAMAKEARGS := O=$(CANONICAL_O)
 
 # Check Buildroot execution pre-requisites here.
-ifneq ($(shell umask):$(CURDIR):$(O),$(REQ_UMASK):$(CANONICAL_CURDIR):$(CANONICAL_O))
+ifneq ($(CURR_UMASK):$(CURDIR):$(O),$(REQ_UMASK):$(CANONICAL_CURDIR):$(CANONICAL_O))
 .PHONY: _all $(MAKECMDGOALS)
 
+# Save the user's original umask (but only if it is more permissive than or
+# equal to REQ_UMASK, which is our minimum working requirement)
+ifeq ($(shell printf "%04o" "$$(( $(CURR_UMASK) | $(REQ_UMASK) ))"),$(REQ_UMASK))
+BR2_ORIG_UMASK := $(CURR_UMASK)
+else
+BR2_ORIG_UMASK := $(REQ_UMASK)
+endif
+
 $(MAKECMDGOALS): _all
 	@:
 
 _all:
 	@umask $(REQ_UMASK) && \
 		$(MAKE) -C $(CANONICAL_CURDIR) --no-print-directory \
+			BR2_ORIG_UMASK=$(BR2_ORIG_UMASK) \
 			$(MAKECMDGOALS) $(EXTRAMAKEARGS)
 
 else # umask / $(CURDIR) / $(O)
@@ -91,6 +101,11 @@ else # umask / $(CURDIR) / $(O)
 all:
 .PHONY: all
 
+# Export the user's original umask: we may need it later to honour user's file
+# creation permissions when creating non-target files/dirs (e.g. during the
+# execution of dl-wrapper script)
+export BR2_ORIG_UMASK := $(or $(BR2_ORIG_UMASK),$(CURR_UMASK))
+
 # Set and export the version string
 export BR2_VERSION := 2023.02-git
 # Actual time the release is cut (for reproducible builds)
package/pkg-download.mk +4 -2
index 0718f21..eb2a242 100644
@@ -107,9 +107,11 @@ endif
 #
 ################################################################################
 
+# Restore the user's original umask during the whole download, in case he has
+# provisions set to share the download directory with his group (or others).
 define DOWNLOAD
-	$(Q)mkdir -p $($(2)_DL_DIR)
-	$(Q)$(EXTRA_ENV) $($(2)_DL_ENV) \
+	$(Q)umask $(BR2_ORIG_UMASK); mkdir -p $($(2)_DL_DIR)
+	$(Q)umask $(BR2_ORIG_UMASK); $(EXTRA_ENV) $($(2)_DL_ENV) \
 		flock $($(2)_DL_DIR)/.lock $(DL_WRAPPER) \
 		-c '$($(2)_DL_VERSION)' \
 		-d '$($(2)_DL_DIR)' \
Signature · 6 lines
2.7.4

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#2 lore raw

Hi Luca,

Sorry for the extremely long turnaround time for this. Messing with umasks is
something that nobody really wants to think about because it entails all kinds of corner cases... In the end, though, it took me just a little more than half an hour to get this merged.

Quoted 13 lines · Luca Pesce via buildroot

Root makefile imposes 'umask 0022', and this may be more restrictive than the user's original umask - which could have provisions set to share files/dirs with other users.
As an example, the imposed value makes the per-package download directories not writeable for the group, but just for the owner - the user that issued the first build that populated the per-package dl dir for the first time (say user A). Thus, if a BR package changes its version (e.g. for buildroot update), and another user (say user B, in the same group of A) starts a build, BR fails the creation of package-xxx.tar.gz inside the dl dir, because user B has no write permissions on that path. Furthermore, in the case of the git backend, this makes the git cache not updatable by a different user. This is disruptive for a host used by many users, all belonging to a certain group.

Quoted 6 lines

So, to allow sharing of a rw BR2_DL_DIR location among users, we save the original umask value (but only if it is more permissive than our REQ_UMASK, which is our minimum working requirement) and restore it during the download process.

Applied to master, thanks, with the following changes:
- CURR_UMASK -> CUR_UMASK.
- BR2_ORIG_UMASK -> BR_ORIG_UMASK.
- Don't check if the umask is more permissive, apply it regardless. If the user explicitly doesn't want to make their DL_DIR readable by others, that's fine.
- Don't export BR_ORIG_UMASK.
- Only set BR_ORIG_UMASK if it we recurse, and only set umask if BR_ORIG_UMASK is set.
- Add DOWNLOAD_SET_UMASK to simplify the latter.

I hope it helps you!

Regards,
Arnout

Quoted 62 lines
---
  Makefile                | 17 ++++++++++++++++-
  package/pkg-download.mk |  6 ++++--
  2 files changed, 20 insertions(+), 3 deletions(-)
Makefile +16 -1
index 88f90cd..ec45589 100644
@@ -68,21 +68,31 @@ endif
  CANONICAL_CURDIR = $(realpath $(CURDIR))
 
  REQ_UMASK = 0022
+CURR_UMASK = $(shell umask)
 
  # Make sure O= is passed (with its absolute canonical path) everywhere the
  # toplevel makefile is called back.
  EXTRAMAKEARGS := O=$(CANONICAL_O)
 
  # Check Buildroot execution pre-requisites here.
-ifneq ($(shell umask):$(CURDIR):$(O),$(REQ_UMASK):$(CANONICAL_CURDIR):$(CANONICAL_O))
+ifneq ($(CURR_UMASK):$(CURDIR):$(O),$(REQ_UMASK):$(CANONICAL_CURDIR):$(CANONICAL_O))
  .PHONY: _all $(MAKECMDGOALS)
 
+# Save the user's original umask (but only if it is more permissive than or
+# equal to REQ_UMASK, which is our minimum working requirement)
+ifeq ($(shell printf "%04o" "$$(( $(CURR_UMASK) | $(REQ_UMASK) ))"),$(REQ_UMASK))
+BR2_ORIG_UMASK := $(CURR_UMASK)
+else
+BR2_ORIG_UMASK := $(REQ_UMASK)
+endif
+
  $(MAKECMDGOALS): _all
  	@:
 
  _all:
  	@umask $(REQ_UMASK) && \
  		$(MAKE) -C $(CANONICAL_CURDIR) --no-print-directory \
+			BR2_ORIG_UMASK=$(BR2_ORIG_UMASK) \
  			$(MAKECMDGOALS) $(EXTRAMAKEARGS)
 
  else # umask / $(CURDIR) / $(O)
@@ -91,6 +101,11 @@ else # umask / $(CURDIR) / $(O)
  all:
  .PHONY: all
 
+# Export the user's original umask: we may need it later to honour user's file
+# creation permissions when creating non-target files/dirs (e.g. during the
+# execution of dl-wrapper script)
+export BR2_ORIG_UMASK := $(or $(BR2_ORIG_UMASK),$(CURR_UMASK))
+
  # Set and export the version string
  export BR2_VERSION := 2023.02-git
  # Actual time the release is cut (for reproducible builds)
package/pkg-download.mk +0 -0
index 0718f21..eb2a242 100644
@@ -107,9 +107,11 @@ endif
  #
  ################################################################################
 

package/pkg-download.mk endif · line 117

Quoted 10 lines
package/pkg-download.mk +4 -2
+# Restore the user's original umask during the whole download, in case he has
+# provisions set to share the download directory with his group (or others).
  define DOWNLOAD
-	$(Q)mkdir -p $($(2)_DL_DIR)
-	$(Q)$(EXTRA_ENV) $($(2)_DL_ENV) \
+	$(Q)umask $(BR2_ORIG_UMASK); mkdir -p $($(2)_DL_DIR)
+	$(Q)umask $(BR2_ORIG_UMASK); $(EXTRA_ENV) $($(2)_DL_ENV) \
  		flock $($(2)_DL_DIR)/.lock $(DL_WRAPPER) \
  		-c '$($(2)_DL_VERSION)' \
  		-d '$($(2)_DL_DIR)' \

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#3 lore raw

[Buildroot] R: [PATCH 1/1] package/pkg-download: restore user's original umask for the whole download process

Hi Arnout,

Quoted 3 lines

Sorry for the extremely long turnaround time for this. Messing with umasks is something that nobody really wants to think about because it entails all kinds of corner cases...

yes, indeed - also, the patch was addressing a minor issue, so I was prepared for a certain review delay, no problem.

Quoted 1 line

I hope it helps you!

Of course it does, thank you!

Regards,
Luca

________________________________________
Da: Arnout Vandecappelle <[email protected]>
Inviato: domenica 14 luglio 2024 15:13
A: Pesce Luca; [email protected]
Cc: Yann E . MORIN
Oggetto: Re: [Buildroot] [PATCH 1/1] package/pkg-download: restore user's original umask for the whole download process

Hi Luca,

Sorry for the extremely long turnaround time for this. Messing with umasks is
something that nobody really wants to think about because it entails all kinds of corner cases... In the end, though, it took me just a little more than half an hour to get this merged.

Quoted 13 lines · Luca Pesce via buildroot

Root makefile imposes 'umask 0022', and this may be more restrictive than the user's original umask - which could have provisions set to share files/dirs with other users.
As an example, the imposed value makes the per-package download directories not writeable for the group, but just for the owner - the user that issued the first build that populated the per-package dl dir for the first time (say user A). Thus, if a BR package changes its version (e.g. for buildroot update), and another user (say user B, in the same group of A) starts a build, BR fails the creation of package-xxx.tar.gz inside the dl dir, because user B has no write permissions on that path. Furthermore, in the case of the git backend, this makes the git cache not updatable by a different user. This is disruptive for a host used by many users, all belonging to a certain group.

Quoted 6 lines

So, to allow sharing of a rw BR2_DL_DIR location among users, we save the original umask value (but only if it is more permissive than our REQ_UMASK, which is our minimum working requirement) and restore it during the download process.

Applied to master, thanks, with the following changes:
- CURR_UMASK -> CUR_UMASK.
- BR2_ORIG_UMASK -> BR_ORIG_UMASK.
- Don't check if the umask is more permissive, apply it regardless. If the user explicitly doesn't want to make their DL_DIR readable by others, that's fine.
- Don't export BR_ORIG_UMASK.
- Only set BR_ORIG_UMASK if it we recurse, and only set umask if BR_ORIG_UMASK is set.
- Add DOWNLOAD_SET_UMASK to simplify the latter.

I hope it helps you!

Regards,
Arnout

Quoted 62 lines
---
  Makefile                | 17 ++++++++++++++++-
  package/pkg-download.mk |  6 ++++--
  2 files changed, 20 insertions(+), 3 deletions(-)
Makefile +16 -1
index 88f90cd..ec45589 100644
@@ -68,21 +68,31 @@ endif
  CANONICAL_CURDIR = $(realpath $(CURDIR))
 
  REQ_UMASK = 0022
+CURR_UMASK = $(shell umask)
 
  # Make sure O= is passed (with its absolute canonical path) everywhere the
  # toplevel makefile is called back.
  EXTRAMAKEARGS := O=$(CANONICAL_O)
 
  # Check Buildroot execution pre-requisites here.
-ifneq ($(shell umask):$(CURDIR):$(O),$(REQ_UMASK):$(CANONICAL_CURDIR):$(CANONICAL_O))
+ifneq ($(CURR_UMASK):$(CURDIR):$(O),$(REQ_UMASK):$(CANONICAL_CURDIR):$(CANONICAL_O))
  .PHONY: _all $(MAKECMDGOALS)
 
+# Save the user's original umask (but only if it is more permissive than or
+# equal to REQ_UMASK, which is our minimum working requirement)
+ifeq ($(shell printf "%04o" "$$(( $(CURR_UMASK) | $(REQ_UMASK) ))"),$(REQ_UMASK))
+BR2_ORIG_UMASK := $(CURR_UMASK)
+else
+BR2_ORIG_UMASK := $(REQ_UMASK)
+endif
+
  $(MAKECMDGOALS): _all
      @:
 
  _all:
      @umask $(REQ_UMASK) && \
              $(MAKE) -C $(CANONICAL_CURDIR) --no-print-directory \
+                     BR2_ORIG_UMASK=$(BR2_ORIG_UMASK) \
                      $(MAKECMDGOALS) $(EXTRAMAKEARGS)
 
  else # umask / $(CURDIR) / $(O)
@@ -91,6 +101,11 @@ else # umask / $(CURDIR) / $(O)
  all:
  .PHONY: all
 
+# Export the user's original umask: we may need it later to honour user's file
+# creation permissions when creating non-target files/dirs (e.g. during the
+# execution of dl-wrapper script)
+export BR2_ORIG_UMASK := $(or $(BR2_ORIG_UMASK),$(CURR_UMASK))
+
  # Set and export the version string
  export BR2_VERSION := 2023.02-git
  # Actual time the release is cut (for reproducible builds)
package/pkg-download.mk +0 -0
index 0718f21..eb2a242 100644
@@ -107,9 +107,11 @@ endif
  #
  ################################################################################
 

package/pkg-download.mk endif · line 117

Quoted 10 lines
package/pkg-download.mk +4 -2
+# Restore the user's original umask during the whole download, in case he has
+# provisions set to share the download directory with his group (or others).
  define DOWNLOAD
-     $(Q)mkdir -p $($(2)_DL_DIR)
-     $(Q)$(EXTRA_ENV) $($(2)_DL_ENV) \
+     $(Q)umask $(BR2_ORIG_UMASK); mkdir -p $($(2)_DL_DIR)
+     $(Q)umask $(BR2_ORIG_UMASK); $(EXTRA_ENV) $($(2)_DL_ENV) \
              flock $($(2)_DL_DIR)/.lock $(DL_WRAPPER) \
              -c '$($(2)_DL_VERSION)' \
              -d '$($(2)_DL_DIR)' \

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot