Skip to contentSkip to conversation

CodingGuidelines: document a shell that "fails" "VAR=VAL shell_func"

PATCH Superseded Junio C Hamano 4 participants 7 messages Jul 22, 2024 – Jul 23, 2024 Documentation

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 CodingGuidelines: document a shell that "fails" "VAR=VAL shell_func"

Over the years, we accumulated the community wisdom to avoid the common "one-short export" construct for shell functions, but seem to have lost on which exact platform it is known to fail. Now during an investigation on a breakage for a recent topic, we found one example of failing shell. Let's document that.

This does *not* mean that we can freely start using the construct once Ubuntu 20.04 is retired. But it does mean that we cannot use the construct until Ubuntu 20.04 is fully retired from the machines that matter.

---
 Documentation/CodingGuidelines | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
Documentation/CodingGuidelines +24 -0
index 1d92b2da03..203ef49364 100644
@@ -204,6 +204,30 @@ For shell scripts specifically (not exhaustive):
 	local variable="$value"
 	local variable="$(command args)"
 
+ - The common construct
+
+	VAR=VAL command args
+
+   to temporarily set and export environment variable VAR only while
+   "command args" is running is handy, but some versions of dash (like
+   0.5.10.2-6 found on Ubuntu 20.04) makes a temporary assignment
+   without exporting the variable, when command is *not* an external
+   command.  Do not use it for shell functions.  A common workaround
+   is to do an explicit export in a subshell, like so:
+
+	(incorrect)
+	VAR=VAL func args
+
+	(correct)
+	(
+		VAR=VAL &&
+		export VAR &&
+		func args
+	)
+
+   but be careful that the effect "func" makes to the variables in the
+   current shell will be lost across the subshell boundary.
+
  - Use octal escape sequences (e.g. "\302\242"), not hexadecimal (e.g.
    "\xc2\xa2") in printf format strings, since hexadecimal escape
    sequences are not portable.

Range-diff:
1: 78a3847e80 ! 1: 75d07c05c7 CodingGuidelines: give an example shell that "fails" "VAR=VAL shell_func"
@@ Metadata
Author: Junio C Hamano <[email protected]>

## Commit message ##
- CodingGuidelines: give an example shell that "fails" "VAR=VAL shell_func"
+ CodingGuidelines: document a shell that "fails" "VAR=VAL shell_func"

Over the years, we accumulated the community wisdom to avoid the common "one-short export" construct for shell functions, but seem to have lost on which exact platform it is known to fail. Now during
- an investigation on a breakage for a recent topic, let's document
- one example of failing shell.
+ an investigation on a breakage for a recent topic, we found one
+ example of failing shell. Let's document that.

This does *not* mean that we can freely start using the construct once Ubuntu 20.04 is retired. But it does mean that we cannot use
@@ Documentation/CodingGuidelines: For shell scripts specifically (not exhaustive):
+
+ (correct)
+ (
-+ VAR=VAL && export VAR &&
++ VAR=VAL &&
++ export VAR &&
+ func args
+ )
+

Signature · 1 line
2.46.0-rc1-52-g816ffef0a1
#1 lore raw

PATCH v3 1/1 CodingGuidelines: document a shell that "fails" "VAR=VAL shell_func"

Over the years, we accumulated the community wisdom to avoid the common "one-short export" construct for shell functions, but seem to have lost on which exact platform it is known to fail. Now during an investigation on a breakage for a recent topic, we found one example of failing shell. Let's document that.

This does *not* mean that we can freely start using the construct once Ubuntu 20.04 is retired. But it does mean that we cannot use the construct until Ubuntu 20.04 is fully retired from the machines that matter. Moreover, posix explicitly says that the behaviour for the construct is unspecified.

---
 Documentation/CodingGuidelines | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
Documentation/CodingGuidelines +27 -0
index 1d92b2da03..ad71c26152 100644
@@ -204,6 +204,33 @@ For shell scripts specifically (not exhaustive):
 	local variable="$value"
 	local variable="$(command args)"
 
+ - The common construct
+
+	VAR=VAL command args
+
+   to temporarily set and export environment variable VAR only while
+   "command args" is running is handy, but this triggers an
+   unspecified behaviour accoreding to POSIX when used for a command
+   that is not an external command (like shell functions).  Indeed,
+   some versions of dash (like 0.5.10.2-6 found on Ubuntu 20.04) and
+   AT&T ksh do make a temporary assignment without exporting the
+   variable, in such a case.  Do not use it for shell functions.  A
+   common workaround is to do an explicit export in a subshell, like
+   so:
+
+	(incorrect)
+	VAR=VAL func args
+
+	(correct)
+	(
+		VAR=VAL &&
+		export VAR &&
+		func args
+	)
+
+   but be careful that the effect "func" makes to the variables in the
+   current shell will be lost across the subshell boundary.
+
  - Use octal escape sequences (e.g. "\302\242"), not hexadecimal (e.g.
    "\xc2\xa2") in printf format strings, since hexadecimal escape
    sequences are not portable.

Range-diff:
1: 75d07c05c7 ! 1: 8462cbb740 CodingGuidelines: document a shell that "fails" "VAR=VAL shell_func"
@@ Commit message
This does *not* mean that we can freely start using the construct
once Ubuntu 20.04 is retired. But it does mean that we cannot use
the construct until Ubuntu 20.04 is fully retired from the machines
- that matter.
+ that matter. Moreover, posix explicitly says that the behaviour for
+ the construct is unspecified.

@@ Documentation/CodingGuidelines: For shell scripts specifically (not exhaustive):
+ VAR=VAL command args
+
+ to temporarily set and export environment variable VAR only while
-+ "command args" is running is handy, but some versions of dash (like
-+ 0.5.10.2-6 found on Ubuntu 20.04) makes a temporary assignment
-+ without exporting the variable, when command is *not* an external
-+ command. Do not use it for shell functions. A common workaround
-+ is to do an explicit export in a subshell, like so:
++ "command args" is running is handy, but this triggers an
++ unspecified behaviour accoreding to POSIX when used for a command
++ that is not an external command (like shell functions). Indeed,
++ some versions of dash (like 0.5.10.2-6 found on Ubuntu 20.04) and
++ AT&T ksh do make a temporary assignment without exporting the
++ variable, in such a case. Do not use it for shell functions. A
++ common workaround is to do an explicit export in a subshell, like
++ so:
+
+ (incorrect)
+ VAR=VAL func args

Signature · 1 line
2.46.0-rc1-52-g816ffef0a1
#2 lore raw
Quoted 10 lines · Junio C Hamano

Over the years, we accumulated the community wisdom to avoid the common "one-short export" construct for shell functions, but seem to have lost on which exact platform it is known to fail. Now during an investigation on a breakage for a recent topic, we found one example of failing shell. Let's document that.

---

Documentation/CodingGuidelines For shell scripts specifically (not exhaustive): · line 210

Quoted 7 lines
Documentation/CodingGuidelines +7 -0
@@ -204,6 +204,33 @@ For shell scripts specifically (not exhaustive):
+ - The common construct
+
+       VAR=VAL command args
+
+   to temporarily set and export environment variable VAR only while
+   "command args" is running is handy, but this triggers an
+   unspecified behaviour accoreding to POSIX when used for a command

s/accoreding/according/

Quoted 19 lines
Documentation/CodingGuidelines +19 -0
+   that is not an external command (like shell functions).  Indeed,
+   some versions of dash (like 0.5.10.2-6 found on Ubuntu 20.04) and
+   AT&T ksh do make a temporary assignment without exporting the
+   variable, in such a case.  Do not use it for shell functions.  A
+   common workaround is to do an explicit export in a subshell, like
+   so:
+
+       (incorrect)
+       VAR=VAL func args
+
+       (correct)
+       (
+               VAR=VAL &&
+               export VAR &&
+               func args
+       )
+
+   but be careful that the effect "func" makes to the variables in the
+   current shell will be lost across the subshell boundary.
#3 lore raw
Quoted 5 lines · Eric Sunshine

Documentation/CodingGuidelines For shell scripts specifically (not exhaustive): · line 213

Quoted 3 lines
Documentation/CodingGuidelines +3 -0
+   to temporarily set and export environment variable VAR only while
+   "command args" is running is handy, but this triggers an
+   unspecified behaviour accoreding to POSIX when used for a command

s/accoreding/according/

Thanks.

#4 lore raw
In reply to Junio C Hamano
Quoted 5 lines · Junio C Hamano

Over the years, we accumulated the community wisdom to avoid the common "one-short export" construct for shell functions, but seem to have lost on which exact platform it is known to fail. Now during an investigation on a breakage for a recent topic, we found one example of failing shell. Let's document that.

My recollection was that FreeBSD's /bin/sh was the culprit, but I couldn't find any mention digging in the archive. However, I just checked on a FreeBSD 13 VM, and it does have the same problem (that the one-shot variable is not exported). I don't think that changes anything for your patch, but just reinforces this part:

Quoted 4 lines

This does *not* mean that we can freely start using the construct once Ubuntu 20.04 is retired. But it does mean that we cannot use the construct until Ubuntu 20.04 is fully retired from the machines that matter.

since now we have one other instance.

I thought it also had the issue that the variable would remain set in the caller after the function returned, but it does not seem to do so now (if it ever did).

-Peff

#5 lore raw
Quoted 19 lines · Jeff King
Quoted 5 lines · Junio C Hamano

Over the years, we accumulated the community wisdom to avoid the common "one-short export" construct for shell functions, but seem to have lost on which exact platform it is known to fail. Now during an investigation on a breakage for a recent topic, we found one example of failing shell. Let's document that.

My recollection was that FreeBSD's /bin/sh was the culprit, but I couldn't find any mention digging in the archive. However, I just checked on a FreeBSD 13 VM, and it does have the same problem (that the one-shot variable is not exported). I don't think that changes anything for your patch, but just reinforces this part:

Quoted 4 lines

This does *not* mean that we can freely start using the construct once Ubuntu 20.04 is retired. But it does mean that we cannot use the construct until Ubuntu 20.04 is fully retired from the machines that matter.

Quoted 5 lines

since now we have one other instance.

I thought it also had the issue that the variable would remain set in the caller after the function returned, but it does not seem to do so now (if it ever did).

Yeah, that one is also what POSIX leaves to the implementation, if I recall what I read there.

So here is how the part looks like in my tree right now.

Thanks.

Documentation/CodingGuidelines +5 -5
index 2151ec51b8..52afb2725f 100644
@@ -212,11 +212,11 @@ For shell scripts specifically (not exhaustive):
    "command args" is running is handy, but this triggers an
    unspecified behaviour according to POSIX when used for a command
    that is not an external command (like shell functions).  Indeed,
-   some versions of dash (like 0.5.10.2-6 found on Ubuntu 20.04) and
-   AT&T ksh do make a temporary assignment without exporting the
-   variable, in such a case.  Do not use it for shell functions.  A
-   common workaround is to do an explicit export in a subshell, like
-   so:
+   dash 0.5.10.2-6 on Ubuntu 20.04, /bin/sh on FreeBSD 13, and AT&T
+   ksh all make a temporary assignment without exporting the variable,
+   in such a case.  As it does not work portably across shells, do not
+   use this syntax for shell functions.  A common workaround is to do
+   an explicit export in a subshell, like so:
 
 	(incorrect)
 	VAR=VAL func args
#6 lore raw
In reply to Jeff King
Quoted 8 lines · Jeff King
Quoted 5 lines · Junio C Hamano

Over the years, we accumulated the community wisdom to avoid the common "one-short export" construct for shell functions, but seem to have lost on which exact platform it is known to fail. Now during an investigation on a breakage for a recent topic, we found one example of failing shell. Let's document that.

Quoted 10 lines

My recollection was that FreeBSD's /bin/sh was the culprit, but I couldn't find any mention digging in the archive. However, I just checked on a FreeBSD 13 VM, and it does have the same problem (that the one-shot variable is not exported). I don't think that changes anything for your patch, but just reinforces this part:

Quoted 4 lines

This does *not* mean that we can freely start using the construct once Ubuntu 20.04 is retired. But it does mean that we cannot use the construct until Ubuntu 20.04 is fully retired from the machines that matter.

And adding more confusion, which further reinforces the need to avoid the construct and the usefulness of Eric's change [1], it sometimes appears to function differently:

** dash 0.5.10.2-6 **

   $ f() { echo $A; bash -c 'echo $A'; }
   $ A=2
   $ A=1 f
   1
   $ export A
   $ f
   2
   2
   $ A=1 f
   1
   1

1.- https://lore.kernel.org/git/[email protected]/T/#m3470fc6c1df59d29312bbe3de0444f1f608f3611

Quoted 8 lines

since now we have one other instance.

I thought it also had the issue that the variable would remain set in the caller after the function returned, but it does not seem to do so now (if it ever did).

-Peff