Restrict tar transform to regular files Without this
PATCH 1/1 Restrict tar transform to regular files Without this
restriction, symlinks are rewritten and corrupted.
Example without the restriction:
Input tree (valid):
package-githash/file1
package-githash/link -> ./file1
Output tree (broken):
package-githash/file1
package-githash/link -> package-githash/file1
--- support/download/git | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
support/download/git +1 -1
@@ -190,7 +190,7 @@ LC_ALL=C sort <"${output}.list" >"${output}.list.sorted" # Create GNU-format tarballs, since that's the format of the tarballs on # sources.buildroot.org and used in the *.hash files -tar cf - --transform="s#^\./#${basename}/#" \ +tar cf - --transform="flags=r;s#^\./#${basename}/#" \ --numeric-owner --owner=0 --group=0 --mtime="${date}" --format=gnu \ -T "${output}.list.sorted" >"${output}.tar" gzip -6 -n <"${output}.tar" >"${output}"
PATCH 1/1 Restrict tar transform to regular files Without this
Timothy, All,
On 2019-04-26 11:49 -0500, Timothy Pearson spake thusly:
Quoted 1 line
restriction, symlinks are rewritten and corrupted.
Something is amiss in your commit log. It shoul;d be:
titile on a single line
Some text on
multiple lines.
Quoted 18 lines
Example without the restriction:
Input tree (valid):
package-githash/file1
package-githash/link -> ./file1Output tree (broken):
package-githash/file1
package-githash/link -> package-githash/file1--- support/download/git | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)support/download/git +0
-0
support/download/git LC_ALL=C sort <"${output}.list" >"${output}.list.sorted" · line 193
Quoted 5 lines
support/download/git +1
-1@@ -190,7 +190,7 @@ LC_ALL=C sort <"${output}.list" >"${output}.list.sorted" # Create GNU-format tarballs, since that's the format of the tarballs on # sources.buildroot.org and used in the *.hash files -tar cf - --transform="s#^\./#${basename}/#" \ +tar cf - --transform="flags=r;s#^\./#${basename}/#" \
OK, so this is a nice trick and all, but I can't find any reference to it in the man page. But I see it's in the git tree in doc/tar.texi...
Your proposal would exclude hardlinks too, which is not correct I think. In fact, we would need to use 'S' (upper-case 'S') here, because we want it to apply to anything but symlinks.
Also, the texi file states that "Default scope flags can also be changed using flags= statement in the transform expression." However, they are standard flags too, quoting:
Supported flags are:
g [...]
i [...]
x [...]
<number> [...]
[...]
In addition, several transformation scope flags are supported, that control to what files transformations apply. These are:
r Apply transformation to regular archive members.
R Do not apply transformation to regular archive members.
s Apply transformation to symbolic link targets.
S Do not apply transformation to symbolic link targets.
h Apply transformation to hard link targets.
H Do not apply transformation to hard link targets.
So I think we could simply write:
--transform="s#^\./#${basename}/#S'Regards,
Yann E. MORIN.
Quoted 7 lines
support/download/git +0
-0--numeric-owner --owner=0 --group=0 --mtime="${date}" --format=gnu \ -T "${output}.list.sorted" >"${output}.tar" gzip -6 -n <"${output}.tar" >"${output}"_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/1 Restrict tar transform to regular files Without this
Quoted 5 lines
Timothy, All,
On 2019-04-26 11:49 -0500, Timothy Pearson spake thusly:
Quoted 1 line
restriction, symlinks are rewritten and corrupted.
Quoted 8 lines
Something is amiss in your commit log. It shoul;d be:
titile on a single line
Some text on
multiple lines.Signed-off-by Your NAME <your@email>
Looks like the signed off line wasn't part of the commit...sorry about that. I've re-uploaded it. The rest of the formatting was generated by git format-patch and should be correct?
Quoted 52 lines
Quoted 18 lines
Example without the restriction:
Input tree (valid):
package-githash/file1
package-githash/link -> ./file1Output tree (broken):
package-githash/file1
package-githash/link -> package-githash/file1--- support/download/git | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)support/download/git +0
-0support/download/git LC_ALL=C sort <"${output}.list" >"${output}.list.sorted" · line 193
Quoted 5 lines
support/download/git +1
-1@@ -190,7 +190,7 @@ LC_ALL=C sort <"${output}.list" >"${output}.list.sorted" # Create GNU-format tarballs, since that's the format of the tarballs on # sources.buildroot.org and used in the *.hash files -tar cf - --transform="s#^\./#${basename}/#" \ +tar cf - --transform="flags=r;s#^\./#${basename}/#" \OK, so this is a nice trick and all, but I can't find any reference to it in the man page. But I see it's in the git tree in doc/tar.texi...
Your proposal would exclude hardlinks too, which is not correct I think. In fact, we would need to use 'S' (upper-case 'S') here, because we want it to apply to anything but symlinks.
Also, the texi file states that "Default scope flags can also be changed using flags= statement in the transform expression." However, they are standard flags too, quoting:
Supported flags are:
g [...] i [...] x [...] <number> [...][...]
In addition, several transformation scope flags are supported, that control to what files transformations apply. These are:
r Apply transformation to regular archive members. R Do not apply transformation to regular archive members. s Apply transformation to symbolic link targets. S Do not apply transformation to symbolic link targets. h Apply transformation to hard link targets. H Do not apply transformation to hard link targets.
Quoted 3 lines
So I think we could simply write:
--transform="s#^\./#${basename}/#S'
If that is functionally equivalent I'd be OK with it. Let me run a quick test against the original failing downloads first.
Quoted 18 lines
Regards,
Yann E. MORIN.Quoted 7 lines
support/download/git +0
-0--numeric-owner --owner=0 --group=0 --mtime="${date}" --format=gnu \ -T "${output}.list.sorted" >"${output}.tar" gzip -6 -n <"${output}.tar" >"${output}"_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot--
.-----------------.--------------------.------------------.--------------------.| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |'------------------------------^-------^------------------^--------------------'
PATCH 1/1 Restrict tar transform to regular files Without this
Timothy, All,
[Please, wrap your messages at ~72 chars]
On 2019-04-27 03:49 -0500, Timothy Pearson spake thusly:
Quoted 7 lines
Quoted 7 lines
On 2019-04-26 11:49 -0500, Timothy Pearson spake thusly:
Quoted 1 line
restriction, symlinks are rewritten and corrupted.
Something is amiss in your commit log. It shoul;d be:
titile on a single line
Quoted 8 lines
Quoted 4 lines
Some text on
multiple lines.Signed-off-by Your NAME <your@email>Looks like the signed off line wasn't part of the commit...sorry about that. I've re-uploaded it. The rest of the formatting was generated by git format-patch and should be correct?
No it's not correct, see the subject of the mail which ends up in the middle of a sentence, and see how patchwork stored it (even your v2):
https://patchwork.ozlabs.org/patch/1091900/
By the way, why did you resend a v2 that does not address my comment, which suggests putting the flag at the end of the expression?
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'