scalar: implement the subcommand "diagnose"
Subsystems (2)
Re: [PATCH v6+ 2/7] archive --add-virtual-file: allow paths containing colons
In reply to Johannes SchindelinQuoted 1 line · Johannes Schindelin
I'd rather avoid changing FUNNYNAMES at this stage, if we can help it.
I wonder if it is sufficient to ask "unzip -l" the names of the files in the archive, without having to materialize these files on the filesystem. Would that bypass the whole FUNNYNAMES business, or is "unzip" paranoid enough to reject an archive, even when it is not extracting into the local filesystem, with a path that it would not be able to extract if it were asked to?
I do not know how standardized different implementations of "unzip" is, and how similar output "unzip -l" implementations produce are, but the following seems to pass for me locally.
t/t5003-archive-zip.sh | 18 ++++-------------- 1 file changed, 4 insertions(+), 14 deletions(-)
t/t5003-archive-zip.sh +4 -14
@@ -207,23 +207,13 @@ check_zip with_untracked check_added with_untracked untracked untracked test_expect_success UNZIP 'git archive --format=zip --add-virtual-file' ' - if test_have_prereq FUNNYNAMES - then - PATHNAME="pathname with : colon" - else - PATHNAME="pathname without colon" - fi && + PATHNAME="pathname with : colon" && git archive --format=zip >with_file_with_content.zip \ --add-virtual-file=\""$PATHNAME"\": \ --add-virtual-file=hello:world $EMPTY_TREE && - test_when_finished "rm -rf tmp-unpack" && - mkdir tmp-unpack && ( - cd tmp-unpack && - "$GIT_UNZIP" ../with_file_with_content.zip && - test_path_is_file hello && - test_path_is_file "$PATHNAME" && - test world = $(cat hello) - ) + "$GIT_UNZIP" -l with_file_with_content.zip >toc && + grep -e " $PATHNAME\$" toc && + grep -e " hello\$" toc ' test_expect_success 'git archive --format=zip --add-file twice' '
Re: [PATCH v6+ 2/7] archive --add-virtual-file: allow paths containing colons
In reply to Johannes SchindelinQuoted 60 lines · Johannes Schindelin
Hi Adam,
Quoted 45 lines · Adam Dinwoodie
Quoted 38 lines · Junio C Hamano
Quoted 23 lines · Adam Dinwoodie
Quoted 23 lines
t/t5003-archive-zip.sh +8
-0@@ -207,13 +207,21 @@ check_zip with_untracked check_added with_untracked untracked untracked test_expect_success UNZIP 'git archive --format=zip --add-virtual-file' ' + if test_have_prereq FUNNYNAMES + then + PATHNAME="pathname with : colon" + else + PATHNAME="pathname without colon" + fi && git archive --format=zip >with_file_with_content.zip \ + --add-virtual-file=\""$PATHNAME"\": \ --add-virtual-file=hello:world $EMPTY_TREE && test_when_finished "rm -rf tmp-unpack" && mkdir tmp-unpack && ( cd tmp-unpack && "$GIT_UNZIP" ../with_file_with_content.zip && test_path_is_file hello && + test_path_is_file "$PATHNAME" &&Quoted 8 lines
t/t5003-archive-zip.sh check_zip with_untracked · line 227
Quoted 3 lines
t/t5003-archive-zip.sh +0
-0test world = $(cat hello) ) 'This test is currently failing on Cygwin: it looks like it's exposing a bug in Cygwin that means files with colons in their name aren't correctly extracted from zip archives. I'm going to report that to the Cygwin mailing list, but I wanted to note it for the record here, too.
Does this mean that our code to set FUNNYNAMES prerequiste is slightly broken? IOW, should we check with a path with a colon in it, as well as whatever we use currently for FUNNYNAMES?
Quoted 5 lines
Something like the attached patch?
Or does Cygwin otherwise work perfectly well with a path with a colon in it, but only $GIT_UNZIP command has problem with it? If that is the case, then please disregard the attached.
Quoted 5 lines
The latter: Cygwin works perfectly with paths containing colons, except that Cygwin's
unzipis seemingly buggy and doesn't work. The file systems Cygwin runs on don't support colons in paths, but Cygwin hides that problem by rewriting ASCII colons to some high Unicode code point on the filesystem,Let me throw in a bit more detail: The forbidden characters are mapped into the Unicode page U+f0XX, which is supposed to be used "for private purposes". Even more detail can be found here:
https://github.com/cygwin/cygwin/blob/cygwin-3_3_5-release/winsup/cygwin/strfuncs.cc#L19-L23
Quoted 8 lines
Quoted 3 lines
meaning Cygwin-native applications see a regular colon, while Windows-native applications see an unusual but perfectly valid Unicode character.
Now, I have two questions:
- Why does
unzipnot use Cygwin's regular functions (which should all be aware of that U+f0XX <-> U+00XX mapping)?
That is an excellent question! This behaviour came from an #ifdef __CYGWIN__ in the upstream unzip package; with that #ifdef removed, everything works as expected. The folk on the Cygwin mailing list had no idea *why* that #ifdef was there, given it's evidently unnecessary; my best guess is that it was added a long time ago before Cygwin could handle those characters in the general case.
Since my report, the Cygwin package has picked up a new maintainer who has released a version of the unzip package with that #ifdef removed, so this test is now passing.
Quoted 2 lines
- Even more importantly: would the test case pass if we simply used another forbidden character, such as
?or*?
The set of characters that had special handling in unzip was "*:?|<> all of which are handled appropriately by Cygwin applications in general, and all of which had this unnecessary handling in unzip
Quoted 5 lines
Quoted 5 lines
I tested the same patch to FUNNYNAMES myself before reporting, and the test fails exactly the same way. If we wanted to catch this, I think we'd need a test that explicitly attempted to unzip an archive containing a path with a colon.
Quoted 8 lines
Quoted 6 lines
(The code to set FUNNYNAMES *is* slightly broken, per the discussions around
6d340dfaef("t9902: split test to run on appropriate systems", 2022-04-08), and my to-do list still features tidying up and resubmitting the patch Ævar wrote in that discussion thread. But it wouldn't help here because this issue is specific to Cygwin'sunzip, rather than a general limitation of running on Cygwin.)I'd rather avoid changing FUNNYNAMES at this stage, if we can help it.
Oh yes, I definitely wasn't proposing changing things for 2.37.0! I just wanted to acknowledge that there is a known issue here that has been discussed on this list previously, that we (I) would hopefully get around to fixing at some point.
Adam