Git 2.13.0 segfaults on Solaris SPARC due to DC_SHA1=YesPlease being on by default
Since 2.13.0 just running "git status" on a newly init'd repo on Solaris SPARC[1] segfaults. Michael (CC'd) reported this issue on #git and I helped him debug it.
Just compiling with BLK_SHA1=YesPlease solves the issue.
There are at least two different issues with DC_SHA1 here:
* We don't properly detect that this platform is big endian. The
check at the top of sha1dc/sha1.c needs to test for _BIG_ENDIAN. This comes from sys/isa_defs.h which (I'm told by #solaris) is included on Solaris by default, at least by stdio.h.
Hacking the endian detection makes t0013-sha1dc.sh pass.
* Even with that & the test passing just a plain "git init x && cd x
&& touch A && git add A && git commit" will segfault.
This is some bug in the sha1dc code, presumably some big endian issue that's not resolved by the change above. Backtrace for that (censored actual author info):
Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 1 (LWP 1)]
0x002f8c84 in sha1_compression_states (ihv=0xffbf8268, m=0x398ad5,
W=0xffbf82fc, states=0xffbf857c) at sha1dc/sha1.c:291
291 SHA1COMPRESS_FULL_ROUND1_STEP_LOAD(a, b, c, d, e, m,
W, 0, temp);
(gdb) bt
#0 0x002f8c84 in sha1_compression_states (ihv=0xffbf8268,
m=0x398ad5, W=0xffbf82fc, states=0xffbf857c) at sha1dc/sha1.c:291
#1 0x00300b60 in sha1_process (ctx=0xffbf8260, block=0x398ad5) atsha1dc/sha1.c:1616
#2 0x00301188 in SHA1DCUpdate (ctx=0xffbf8260,
buf=0x398ad5 "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef\nauthor Au Thor <[email protected]> 123456789 +0000\ncommitter Au Thor <[email protected]> 123456789 +0000\n\nBlah Blah"..., len=220) at sha1dc/sha1.c:1731
#3 0x0030168c in git_SHA1DCUpdate (ctx=0xffbf8260, vdata=0x398aa0,
len=273) at sha1dc/sha1.c:1808
#4 0x002a6f7c in write_sha1_file_prepare (buf=0x398aa0, len=273,
type=0x959c8 "commit", sha1=0xffbfd630 "",
hdr=0xffbf8c28 "commit 273", hdrlen=0xffbf8c24) at sha1_file.c:3207
#5 0x002a71ac in hash_sha1_file (buf=0x398aa0, len=273, type=0x959c8
"commit", sha1=0xffbfd630 "") at sha1_file.c:3266
#6 0x002a25f8 in check_sha1_signature (sha1=0xffbfdbb8
"\375\067\356\337\002", map=0x398aa0, size=273, type=0x959c8 "commit")
at sha1_file.c:1644
#7 0x0022816c in parse_object (sha1=0xffbfdbb8
"\375\067\356\337\002") at object.c:269
#8 0x0027c258 in get_reference (revs=0xffbfdc88, name=0xa87f0
"HEAD", sha1=0xffbfdbb8 "\375\067\356\337\002", flags=0)
at revision.c:196
#9 0x00284714 in setup_revisions (argc=0, argv=0x0, revs=0xffbfdc88,
opt=0xffbfdc74) at revision.c:2295
#10 0x002ee4d8 in wt_status_collect_changes_index (s=0x348ea8
<s.24114>) at wt-status.c:585
#11 0x002eeae8 in wt_status_collect (s=0x348ea8 <s.24114>) at wt-status.c:701 #12 0x000db4fc in cmd_status (argc=0, argv=0xffbfe6fc, prefix=0x0) atbuiltin/commit.c:1393
#13 0x000acc24 in run_builtin (p=0x340ccc <commands+1200>, argc=1,
argv=0xffbfe6fc) at git.c:371
#14 0x000acf98 in handle_builtin (argc=1, argv=0xffbfe6fc) at git.c:572 #15 0x000ad1a4 in run_argv (argcp=0xffbfe66c, argv=0xffbfe670) at git.c:624 #16 0x000ad3fc in cmd_main (argc=1, argv=0xffbfe6fc) at git.c:701 #17 0x001747a0 in main (argc=5, argv=0xffbfe6ec) at common-main.c:43
1. Relevant part of uname -a: "SunOS <hostname> 5.11 11.3 sun4v sparc sun4v".
Hi Aevar,
Thank you for notifying us of this issue.
Big endianness is a tricky issue, also since I don't have access or accurate knowledge about all big endian systems. Our github repo does check correct functioning, including an endianness mistake, with 'make test'. But I guess this is not included for SHA1DC in Git.
Anyway, we can easily add the _BIG_ENDIAN macrotest to the git repo and will do so soon.
I don't think the segfault is caused by buffer overflow, inproper access, or the endianness issue. But I did notice an unexpected issue: the message block pointer m=0x398ad5 is odd.
Can you confirm whether loading an uint32_t from an odd address triggers a hardware interrupt on your platform? This is not problem for x86, but maybe for your platform it is?
If it is then we should always copy buffer contents to the sha1context to avoid this issue.
Best regards,
Marc Stevens
-----Oorspronkelijk bericht-----
Van: Ævar Arnfjörð Bjarmason [mailto:[email protected]]
Verzonden: maandag 15 mei 2017 14:49
Aan: Git Mailing List <[email protected]>
CC: [email protected]; Jeff King <[email protected]>; Marc Stevens <[email protected]>
Onderwerp: Git 2.13.0 segfaults on Solaris SPARC due to DC_SHA1=YesPlease being on by default
Since 2.13.0 just running "git status" on a newly init'd repo on Solaris SPARC[1] segfaults. Michael (CC'd) reported this issue on #git and I helped him debug it.
Just compiling with BLK_SHA1=YesPlease solves the issue.
There are at least two different issues with DC_SHA1 here:
* We don't properly detect that this platform is big endian. The check at the top of sha1dc/sha1.c needs to test for _BIG_ENDIAN. This comes from sys/isa_defs.h which (I'm told by #solaris) is included on Solaris by default, at least by stdio.h.
Hacking the endian detection makes t0013-sha1dc.sh pass.
* Even with that & the test passing just a plain "git init x && cd x && touch A && git add A && git commit" will segfault.
This is some bug in the sha1dc code, presumably some big endian issue that's not resolved by the change above. Backtrace for that (censored actual author info):
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1 (LWP 1)]
0x002f8c84 in sha1_compression_states (ihv=0xffbf8268, m=0x398ad5, W=0xffbf82fc, states=0xffbf857c) at sha1dc/sha1.c:291
291 SHA1COMPRESS_FULL_ROUND1_STEP_LOAD(a, b, c, d, e, m,
W, 0, temp);
(gdb) bt
#0 0x002f8c84 in sha1_compression_states (ihv=0xffbf8268, m=0x398ad5, W=0xffbf82fc, states=0xffbf857c) at sha1dc/sha1.c:291 #1 0x00300b60 in sha1_process (ctx=0xffbf8260, block=0x398ad5) atsha1dc/sha1.c:1616
#2 0x00301188 in SHA1DCUpdate (ctx=0xffbf8260,
buf=0x398ad5 "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef\nauthor Au Thor <[email protected]> 123456789 +0000\ncommitter Au Thor <[email protected]> 123456789 +0000\n\nBlah Blah"..., len=220) at sha1dc/sha1.c:1731
#3 0x0030168c in git_SHA1DCUpdate (ctx=0xffbf8260, vdata=0x398aa0,
len=273) at sha1dc/sha1.c:1808
#4 0x002a6f7c in write_sha1_file_prepare (buf=0x398aa0, len=273,
type=0x959c8 "commit", sha1=0xffbfd630 "",
hdr=0xffbf8c28 "commit 273", hdrlen=0xffbf8c24) at sha1_file.c:3207
#5 0x002a71ac in hash_sha1_file (buf=0x398aa0, len=273, type=0x959c8 "commit", sha1=0xffbfd630 "") at sha1_file.c:3266
#6 0x002a25f8 in check_sha1_signature (sha1=0xffbfdbb8 "\375\067\356\337\002", map=0x398aa0, size=273, type=0x959c8 "commit")
at sha1_file.c:1644
#7 0x0022816c in parse_object (sha1=0xffbfdbb8
"\375\067\356\337\002") at object.c:269
#8 0x0027c258 in get_reference (revs=0xffbfdc88, name=0xa87f0 "HEAD", sha1=0xffbfdbb8 "\375\067\356\337\002", flags=0)
at revision.c:196 #9 0x00284714 in setup_revisions (argc=0, argv=0x0, revs=0xffbfdc88,
opt=0xffbfdc74) at revision.c:2295
#10 0x002ee4d8 in wt_status_collect_changes_index (s=0x348ea8
<s.24114>) at wt-status.c:585
#11 0x002eeae8 in wt_status_collect (s=0x348ea8 <s.24114>) at wt-status.c:701 #12 0x000db4fc in cmd_status (argc=0, argv=0xffbfe6fc, prefix=0x0) atbuiltin/commit.c:1393
#13 0x000acc24 in run_builtin (p=0x340ccc <commands+1200>, argc=1,
argv=0xffbfe6fc) at git.c:371
#14 0x000acf98 in handle_builtin (argc=1, argv=0xffbfe6fc) at git.c:572 #15 0x000ad1a4 in run_argv (argcp=0xffbfe66c, argv=0xffbfe670) at git.c:624 #16 0x000ad3fc in cmd_main (argc=1, argv=0xffbfe6fc) at git.c:701 #17 0x001747a0 in main (argc=5, argv=0xffbfe6ec) at common-main.c:43
1. Relevant part of uname -a: "SunOS <hostname> 5.11 11.3 sun4v sparc sun4v".
Quoted 7 lines · Marc Stevens
Hi Aevar,
Thank you for notifying us of this issue.
Big endianness is a tricky issue, also since I don't have access or accurate knowledge about all big endian systems. Our github repo does check correct functioning, including an endianness mistake, with 'make test'. But I guess this is not included for SHA1DC in Git.
Quoted 7 lines
Anyway, we can easily add the
_BIG_ENDIANmacrotest to the git repo and will do so soon.I don't think the segfault is caused by buffer overflow, inproper access, or the endianness issue. But I did notice an unexpected issue: the message block pointer m=0x398ad5 is odd.
Can you confirm whether loading anuint32_tfrom an odd address triggers a hardware interrupt on your platform? This is not problem for x86, but maybe for your platform it is?
If it is then we should always copy buffer contents to the sha1context to avoid this issue.
I don't have access to the box in question, Michael was testing this code for me. But unaligned access is probably the cause, although according to some info I found online that should give a SIGBUS not a SIGSEGV, but that may have changed:
https://bugs.python.org/issue12181
https://github.com/magnumripper/JohnTheRipper/issues/2187
Quoted 65 lines
Best regards,
Marc Stevens-----Oorspronkelijk bericht-----
Van: Ævar Arnfjörð Bjarmason [mailto:[email protected]]
Verzonden: maandag 15 mei 2017 14:49
Aan: Git Mailing List <[email protected]>
CC: [email protected]; Jeff King <[email protected]>; Marc Stevens <[email protected]>
Onderwerp: Git 2.13.0 segfaults on Solaris SPARC due to DC_SHA1=YesPlease being on by defaultSince 2.13.0 just running "git status" on a newly init'd repo on Solaris SPARC[1] segfaults. Michael (CC'd) reported this issue on #git and I helped him debug it.
Just compiling with BLK_SHA1=YesPlease solves the issue.
There are at least two different issues with
DC_SHA1here:* We don't properly detect that this platform is big endian. The check at the top of
sha1dc/sha1.cneeds to test for_BIG_ENDIAN. This comes fromsys/isa_defs.hwhich (I'm told by #solaris) is included on Solaris by default, at least bystdio.h.Hacking the endian detection makes
t0013-sha1dc.shpass.* Even with that & the test passing just a plain "git init x && cd x && touch A && git add A && git commit" will segfault.
This is some bug in the sha1dc code, presumably some big endian issue that's not resolved by the change above. Backtrace for that (censored actual author info):
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1 (LWP 1)]
0x002f8c84insha1_compression_states(ihv=0xffbf8268, m=0x398ad5, W=0xffbf82fc, states=0xffbf857c) atsha1dc/sha1.c:291
291 SHA1COMPRESS_FULL_ROUND1_STEP_LOAD(a, b, c, d, e, m,
W, 0, temp);
(gdb) bt
#00x002f8c84insha1_compression_states(ihv=0xffbf8268, m=0x398ad5, W=0xffbf82fc, states=0xffbf857c) atsha1dc/sha1.c:291#10x00300b60insha1_process(ctx=0xffbf8260, block=0x398ad5) atsha1dc/sha1.c:1616
#20x00301188in SHA1DCUpdate (ctx=0xffbf8260,buf=0x398ad5 "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef\nauthor Au Thor <[email protected]> 123456789 +0000\ncommitter Au Thor <[email protected]> 123456789 +0000\n\nBlah Blah"..., len=220) at sha1dc/sha1.c:1731#3
0x0030168cingit_SHA1DCUpdate(ctx=0xffbf8260, vdata=0x398aa0,
len=273) atsha1dc/sha1.c:1808
#40x002a6f7cinwrite_sha1_file_prepare(buf=0x398aa0, len=273,
type=0x959c8 "commit", sha1=0xffbfd630 "",hdr=0xffbf8c28 "commit 273", hdrlen=0xffbf8c24) at sha1_file.c:3207 #5 0x002a71ac in hash_sha1_file (buf=0x398aa0, len=273, type=0x959c8 "commit", sha1=0xffbfd630 "") at sha1_file.c:3266 #6 0x002a25f8 in check_sha1_signature (sha1=0xffbfdbb8 "\375\067\356\337\002", map=0x398aa0, size=273, type=0x959c8 "commit") at sha1_file.c:1644 #7 0x0022816c in parse_object (sha1=0xffbfdbb8 "\375\067\356\337\002") at object.c:269 #8 0x0027c258 in get_reference (revs=0xffbfdc88, name=0xa87f0 "HEAD", sha1=0xffbfdbb8 "\375\067\356\337\002", flags=0) at revision.c:196#9
0x00284714insetup_revisions(argc=0, argv=0x0, revs=0xffbfdc88,
opt=0xffbfdc74) at revision.c:2295
#100x002ee4d8inwt_status_collect_changes_index(s=0x348ea8
<s.24114>) at wt-status.c:585
#110x002eeae8inwt_status_collect(s=0x348ea8 <s.24114>) at wt-status.c:701 #120x000db4fcincmd_status(argc=0, argv=0xffbfe6fc, prefix=0x0) atbuiltin/commit.c:1393
#130x000acc24inrun_builtin(p=0x340ccc <commands+1200>, argc=1,
argv=0xffbfe6fc) at git.c:371
#140x000acf98inhandle_builtin(argc=1, argv=0xffbfe6fc) at git.c:572 #150x000ad1a4inrun_argv(argcp=0xffbfe66c, argv=0xffbfe670) at git.c:624 #160x000ad3fcincmd_main(argc=1, argv=0xffbfe6fc) at git.c:701 #170x001747a0in main (argc=5, argv=0xffbfe6ec) at common-main.c:431. Relevant part of uname -a: "SunOS <hostname> 5.11 11.3 sun4v sparc sun4v".
Quoted 16 lines · Ævar Arnfjörð Bjarmason
Quoted 7 lines · Marc Stevens
Hi Aevar,
Thank you for notifying us of this issue.
Big endianness is a tricky issue, also since I don't have access or accurate knowledge about all big endian systems. Our github repo does check correct functioning, including an endianness mistake, with 'make test'. But I guess this is not included for SHA1DC in Git.Quoted 7 lines
Anyway, we can easily add the
_BIG_ENDIANmacrotest to the git repo and will do so soon.I don't think the segfault is caused by buffer overflow, inproper access, or the endianness issue. But I did notice an unexpected issue: the message block pointer m=0x398ad5 is odd.
Can you confirm whether loading anuint32_tfrom an odd address triggers a hardware interrupt on your platform? This is not problem for x86, but maybe for your platform it is?
If it is then we should always copy buffer contents to the sha1context to avoid this issue.
Quoted 4 lines
I don't have access to the box in question, Michael was testing this code for me. But unaligned access is probably the cause, although according to some info I found online that should give a SIGBUS not a SIGSEGV, but that may have changed:
Yeah, I would have expected SIGBUS there. If we have alignment issues, though, I'd expect that ARM systems will experience problems.
Block-sha1 uses a macro which allows unaligned loads on platforms that support it, and otherwise does the endian conversion on the fly as we load the bytes into a local variable (which presumably happens all in-register). That may be faster than doing a mass copy of the buffer.
-Peff
Quoted 23 lines · Jeff King
Quoted 16 lines · Ævar Arnfjörð Bjarmason
Quoted 7 lines · Marc Stevens
Hi Aevar,
Thank you for notifying us of this issue.
Big endianness is a tricky issue, also since I don't have access or accurate knowledge about all big endian systems. Our github repo does check correct functioning, including an endianness mistake, with 'make test'. But I guess this is not included for SHA1DC in Git.Quoted 7 lines
Anyway, we can easily add the
_BIG_ENDIANmacrotest to the git repo and will do so soon.I don't think the segfault is caused by buffer overflow, inproper access, or the endianness issue. But I did notice an unexpected issue: the message block pointer m=0x398ad5 is odd.
Can you confirm whether loading anuint32_tfrom an odd address triggers a hardware interrupt on your platform? This is not problem for x86, but maybe for your platform it is?
If it is then we should always copy buffer contents to the sha1context to avoid this issue.Quoted 4 lines
I don't have access to the box in question, Michael was testing this code for me. But unaligned access is probably the cause, although according to some info I found online that should give a SIGBUS not a SIGSEGV, but that may have changed:
Quoted 7 lines
Yeah, I would have expected SIGBUS there. If we have alignment issues, though, I'd expect that ARM systems will experience problems.
Block-sha1 uses a macro which allows unaligned loads on platforms that support it, and otherwise does the endian conversion on the fly as we load the bytes into a local variable (which presumably happens all in-register). That may be faster than doing a mass copy of the buffer.
I agree. It is fairly normal to use that kind of macro and not do a memcpy with hash functions.
In fact many hash functions ONLY use that kind of macro, as decent compilers will automagically convert the macro into an unaligned load on platforms that support fast unaligned loads.
The only reason to expose the direct unaligned load is to make sure that the hashing code is fast on such platforms even when compiled under -g.
Yves
Signature · 1 line
perl -Mre=debug -e "/just|another|perl|hacker/"
A message that is not in the archive (sent off-list or not yet received): CAKKM46vwM9pxyMxTc4jA0z_8vGKdDGCGg9ziKkFAsqr5ULYJxA@mail.gmail.com
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hi Marc,
works like a charm!
Michael
2017-05-15 16:33 GMT+02:00 Marc Stevens <[email protected]>:
Quoted 157 lines
Hi Michael,
See the latest commit to the SHA1DC repo:
Just apply this same patch to SHA1DC in Git.
Best regards,
Marc Stevens
Van: Michael Kebe [mailto:[email protected]]
Verzonden: maandag 15 mei 2017 16:22
Aan: Ævar Arnfjörð Bjarmason <[email protected]>
CC: Jeff King <[email protected]>; Git Mailing List <[email protected]>; Marc Stevens <[email protected]>
Onderwerp: Re: Git 2.13.0 segfaults on Solaris SPARC due to DC_SHA1=YesPlease being on by defaultJust give me patch or repo, I will test it tomorrow.
Michael
Am 15.05.2017 4:14 nachm. schrieb "Ævar Arnfjörð Bjarmason" <[email protected]>:
Quoted 12 lines · Marc Stevens
Hi Aevar,
Thank you for notifying us of this issue.
Big endianness is a tricky issue, also since I don't have access or accurate knowledge about all big endian systems.
Our github repo does check correct functioning, including an endianness mistake, with 'make test'.
But I guess this is not included for SHA1DC in Git.Anyway, we can easily add the
_BIG_ENDIANmacrotest to the git repo and will do so soon.Quoted 9 lines
I don't think the segfault is caused by buffer overflow, inproper access, or the endianness issue.
But I did notice an unexpected issue: the message block pointer m=0x398ad5 is odd.
Can you confirm whether loading anuint32_tfrom an odd address triggers a hardware interrupt on your platform?
This is not problem for x86, but maybe for your platform it is? If it is then we should always copy buffer contents to the sha1context to avoid this issue.I don't have access to the box in question, Michael was testing this code for me. But unaligned access is probably the cause, although according to some info I found online that should give a SIGBUS not a SIGSEGV, but that may have changed:
https://bugs.python.org/issue12181
https://github.com/magnumripper/JohnTheRipper/issues/2187Quoted 85 lines
Best regards,
Marc Stevens-----Oorspronkelijk bericht-----
Van: Ævar Arnfjörð Bjarmason [mailto:[email protected]]
Verzonden: maandag 15 mei 2017 14:49
Aan: Git Mailing List <[email protected]>
CC: [email protected]; Jeff King <[email protected]>; Marc Stevens <[email protected]>
Onderwerp: Git 2.13.0 segfaults on Solaris SPARC due to DC_SHA1=YesPlease being on by defaultSince 2.13.0 just running "git status" on a newly init'd repo on Solaris SPARC[1] segfaults. Michael (CC'd) reported this issue on #git and I helped him debug it.
Just compiling with BLK_SHA1=YesPlease solves the issue.
There are at least two different issues with
DC_SHA1here:* We don't properly detect that this platform is big endian. The check at
the top ofsha1dc/sha1.cneeds to test for_BIG_ENDIAN. This comes fromsys/isa_defs.hwhich (I'm told by #solaris) is included on Solaris by default, at least bystdio.h.Hacking the endian detection makes
t0013-sha1dc.shpass.* Even with that & the test passing just a plain "git init x && cd x &&
touch A && git add A && git commit" will segfault.This is some bug in the sha1dc code, presumably some big endian issue that's not resolved by the change above. Backtrace for that (censored actual author info):
Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 1 (LWP 1)]
0x002f8c84insha1_compression_states(ihv=0xffbf8268, m=0x398ad5,
W=0xffbf82fc, states=0xffbf857c) atsha1dc/sha1.c:291
291 SHA1COMPRESS_FULL_ROUND1_STEP_LOAD(a, b, c, d, e, m,
W, 0, temp);
(gdb) bt
#00x002f8c84insha1_compression_states(ihv=0xffbf8268, m=0x398ad5,
W=0xffbf82fc, states=0xffbf857c) atsha1dc/sha1.c:291
#10x00300b60insha1_process(ctx=0xffbf8260, block=0x398ad5) atsha1dc/sha1.c:1616
#20x00301188in SHA1DCUpdate (ctx=0xffbf8260,buf=0x398ad5 "deadbeefdeadbeefdeadbeefdeadbeefdeadbeef\nauthor Au Thor <[email protected]> 123456789 +0000\ncommitter Au Thor <[email protected]> 123456789 +0000\n\nBlah Blah"..., len=220) at sha1dc/sha1.c:1731#3
0x0030168cingit_SHA1DCUpdate(ctx=0xffbf8260, vdata=0x398aa0,
len=273) atsha1dc/sha1.c:1808
#40x002a6f7cinwrite_sha1_file_prepare(buf=0x398aa0, len=273,
type=0x959c8 "commit", sha1=0xffbfd630 "",hdr=0xffbf8c28 "commit 273", hdrlen=0xffbf8c24) at sha1_file.c:3207#5
0x002a71acinhash_sha1_file(buf=0x398aa0, len=273, type=0x959c8
"commit", sha1=0xffbfd630 "") at sha1_file.c:3266
#60x002a25f8incheck_sha1_signature(sha1=0xffbfdbb8
"\375\067\356\337\002", map=0x398aa0, size=273, type=0x959c8 "commit")at sha1_file.c:1644#7
0x0022816cinparse_object(sha1=0xffbfdbb8
"\375\067\356\337\002") at object.c:269
#80x0027c258inget_reference(revs=0xffbfdc88, name=0xa87f0 "HEAD",
sha1=0xffbfdbb8 "\375\067\356\337\002", flags=0)at revision.c:196#9
0x00284714insetup_revisions(argc=0, argv=0x0, revs=0xffbfdc88,
opt=0xffbfdc74) at revision.c:2295
#100x002ee4d8inwt_status_collect_changes_index(s=0x348ea8
<s.24114>) at wt-status.c:585
#110x002eeae8inwt_status_collect(s=0x348ea8 <s.24114>) at
wt-status.c:701
#120x000db4fcincmd_status(argc=0, argv=0xffbfe6fc, prefix=0x0) atbuiltin/commit.c:1393
#130x000acc24inrun_builtin(p=0x340ccc <commands+1200>, argc=1,
argv=0xffbfe6fc) at git.c:371
#140x000acf98inhandle_builtin(argc=1, argv=0xffbfe6fc) at git.c:572 #150x000ad1a4inrun_argv(argcp=0xffbfe66c, argv=0xffbfe670) at
git.c:624
#160x000ad3fcincmd_main(argc=1, argv=0xffbfe6fc) at git.c:701 #170x001747a0in main (argc=5, argv=0xffbfe6ec) at common-main.c:431. Relevant part of uname -a: "SunOS <hostname> 5.11 11.3 sun4v sparc sun4v".
PATCH 1/1 sha1dc: fix issues with a big endian platform
From: Marc Stevens <[email protected]>
Some big-endian platforms define _BIG_ENDIAN, which the test at the beginning of file has missed. Also, when the input is not aligned, some platforms trigger SIGBUS.
This change corresponds to 33a694a9 ("Fix issues with a big endian platform", 2017-05-15) in the history of the upstream repository
https://github.com/cr-marcstevens/sha1collisiondetection
---
* So here is my attempt to clarify the log message (I left the title as-is, but this change deals both with endianness and alignment requirement).
Please look it over, and then sign-off your patch ;-)
Thanks.
P.S. I wonder how often "buf" is not aligned---could we somehow optimize out memcpy when it is not necessary, or is it not worth it?
sha1dc/sha1.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
sha1dc/sha1.c +3 -2
@@ -20,7 +20,7 @@*/ #if (defined(__BYTE_ORDER) && (__BYTE_ORDER == __BIG_ENDIAN)) || \ (defined(__BYTE_ORDER__) && (__BYTE_ORDER__ == __BIG_ENDIAN__)) || \ - defined(__BIG_ENDIAN__) || defined(__ARMEB__) || defined(__THUMBEB__) || defined(__AARCH64EB__) || \ + defined(_BIG_ENDIAN) || defined(__BIG_ENDIAN__) || defined(__ARMEB__) || defined(__THUMBEB__) || defined(__AARCH64EB__) || \ defined(_MIPSEB) || defined(__MIPSEB) || defined(__MIPSEB__)#define SHA1DC_BIGENDIAN 1
@@ -1728,7 +1728,8 @@ void SHA1DCUpdate(SHA1_CTX* ctx, const char* buf, size_t len)while (len >=64 ) { ctx->total +=64 ; -sha1_process (ctx, (uint32_t *)(buf)); +memcpy (ctx->buffer, buf,64 ); +sha1_process (ctx, (uint32_t *)(ctx->buffer)); buf +=64 ; len -=64 ; }
Signature · 1 line
2.13.0-416-g4c6b804423
Re: [PATCH] sha1dc: fix issues with a big endian platform
Quoted 9 lines · Junio C Hamano
From: Marc Stevens <[email protected]>
Some big-endian platforms define
_BIG_ENDIAN, which the test at the beginning of file has missed. Also, when the input is not aligned, some platforms trigger SIGBUS.This change corresponds to
33a694a9("Fix issues with a big endian platform", 2017-05-15) in the history of the upstream repository
https://github.com/cr-marcstevens/sha1collisiondetection
Then why not just update sha1dc from upstream instead of cherry-picking one patch from them? My update-sha1dc change does this:
https://github.com/git/git/compare/master...avar:update-sha1dc
This follows the instructions Jeff detailed in 28dc98e343. I solved a few merge conflicts, I think I got them right, but review of that especially welcome.
Maybe you just meant this patch for maint, but if that's the case it's not clear from your commit message or E-Mail.
Quoted 43 lines
---
* So here is my attempt to clarify the log message (I left the title as-is, but this change deals both with endianness and alignment requirement).
Please look it over, and then sign-off your patch ;-)
Thanks.
P.S. I wonder how often "buf" is not aligned---could we somehow optimize out memcpy when it is not necessary, or is it not worth it?sha1dc/sha1.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)sha1dc/sha1.c +3
-3@@ -20,7 +20,7 @@*/ #if (defined(__BYTE_ORDER) && (__BYTE_ORDER == __BIG_ENDIAN)) || \ (defined(__BYTE_ORDER__) && (__BYTE_ORDER__ == __BIG_ENDIAN__)) || \ - defined(__BIG_ENDIAN__) || defined(__ARMEB__) || defined(__THUMBEB__) || defined(__AARCH64EB__) || \ + defined(_BIG_ENDIAN) || defined(__BIG_ENDIAN__) || defined(__ARMEB__) || defined(__THUMBEB__) || defined(__AARCH64EB__) || \ defined(_MIPSEB) || defined(__MIPSEB) || defined(__MIPSEB__)#define SHA1DC_BIGENDIAN 1@@ -1728,7 +1728,8 @@ void SHA1DCUpdate(SHA1_CTX* ctx, const char* buf, size_t len)while (len >=64 ) { ctx->total +=64 ; -sha1_process (ctx, (uint32_t *)(buf)); +memcpy (ctx->buffer, buf,64 ); +sha1_process (ctx, (uint32_t *)(ctx->buffer)); buf +=64 ; len -=64 ; } --2.13.0-416-g4c6b804423
Re: [PATCH] sha1dc: fix issues with a big endian platform
Quoted 7 lines · Ævar Arnfjörð Bjarmason
Quoted 6 lines · Junio C Hamano
From: Marc Stevens <[email protected]>
Some big-endian platforms define
_BIG_ENDIAN, which the test at the beginning of file has missed. Also, when the input is not aligned, some platforms trigger SIGBUS.
Quoted 6 lines
Quoted 3 lines
This change corresponds to
33a694a9("Fix issues with a big endian platform", 2017-05-15) in the history of the upstream repository
https://github.com/cr-marcstevens/sha1collisiondetectionThen why not just update sha1dc from upstream instead of cherry-picking one patch from them?
See the original message upthread. I did the cherry-pick simply because that was what Marc instructed the patch recipient to do ;-).
cover Use sha1collisiondetection as a submodule
Quoted 13 lines · Junio C Hamano
Quoted 11 lines · Ævar Arnfjörð Bjarmason
Quoted 9 lines · Junio C Hamano
From: Marc Stevens <[email protected]>
Some big-endian platforms define
_BIG_ENDIAN, which the test at the beginning of file has missed. Also, when the input is not aligned, some platforms trigger SIGBUS.This change corresponds to
33a694a9("Fix issues with a big endian platform", 2017-05-15) in the history of the upstream repository
https://github.com/cr-marcstevens/sha1collisiondetection
Quoted 5 lines
Quoted 2 lines
Then why not just update sha1dc from upstream instead of cherry-picking one patch from them?
See the original message upthread. I did the cherry-pick simply because that was what Marc instructed the patch recipient to do ;-).
Since that patch is now in Marc's upstream code we can just update to that.
While we're at it let's see if Marc will take a patch so that we can use his code as-is with some Makefile trickery of our own, instead of having to solve merge conflicts each time we update from him. I'll submit a pull request for that shortly.
And since if and when that pull request gets accepted we're at the point of being able to use the upstream code as-is & don't need to locally modify it, we can just use a submodule to track it.
Ævar Arnfjörð Bjarmason (3):
sha1dc: update from my fork of upstream
sha1dc: use sha1collisiondetection as a submodule
sha1dc: remove our old copy of the sha1dc code
.gitmodules | 4 + Makefile | 13 +- hash.h | 2 +- sha1collisiondetection | 1 + sha1dc/LICENSE.txt | 30 - sha1dc/sha1.c | 1809 ------------------------------------------------ sha1dc/sha1.h | 122 ---- sha1dc/ubc_check.c | 363 ---------- sha1dc/ubc_check.h | 44 -- sha1dc_git.c | 19 + sha1dc_git.h | 14 + 11 files changed, 49 insertions(+), 2372 deletions(-) create mode 100644 .gitmodules create mode 160000 sha1collisiondetection delete mode 100644 sha1dc/LICENSE.txt delete mode 100644 sha1dc/sha1.c delete mode 100644 sha1dc/sha1.h delete mode 100644 sha1dc/ubc_check.c delete mode 100644 sha1dc/ubc_check.h create mode 100644 sha1dc_git.c create mode 100644 sha1dc_git.h
Signature · 1 line
2.13.0.303.g4ebf302169
PATCH 1/3 sha1dc: update from my fork of upstream
Upgrade the sha1dc code from my fork of the project, which has one patch I'm hoping will be integrated upstream after this series demonstrates its usefulness:
That minor change allows us to in the future just use the upstream code as-is, without any local modifications.
Now all the local modifications we've done in this directory (see the git history of sha1dc/) are done either with defines, or via the newly added sha1dc_git.[ch] files.
--- Makefile | 9 +++++- sha1dc/sha1.c | 79 ++++++++++++++++++++++++++--------------------- sha1dc/sha1.h | 90 +++++++++++++++++++++++------------------------------- sha1dc/ubc_check.c | 9 ++++-- sha1dc/ubc_check.h | 10 ++++-- sha1dc_git.c | 19 ++++++++++++ sha1dc_git.h | 14 +++++++++ 7 files changed, 139 insertions(+), 91 deletions(-) create mode 100644 sha1dc_git.c create mode 100644 sha1dc_git.h
Makefile +8 -1
@@ -1414,7 +1414,14 @@ else DC_SHA1 := YesPlease LIB_OBJS += sha1dc/sha1.o LIB_OBJS += sha1dc/ubc_check.o - BASIC_CFLAGS += -DSHA1_DC + BASIC_CFLAGS += \ + -DSHA1_DC \ + -DSHA1DC_NO_STANDARD_INCLUDES \ + -DSHA1DC_INIT_SAFE_HASH_DEFAULT=0 \ + -DSHA1DC_CUSTOM_INCLUDE_SHA1_C="\"cache.h\"" \ + -DSHA1DC_CUSTOM_TRAILING_INCLUDE_SHA1_C="\"../sha1dc_git.c\"" \ + -DSHA1DC_CUSTOM_TRAILING_INCLUDE_SHA1_H="\"../sha1dc_git.h\"" \ + -DSHA1DC_CUSTOM_INCLUDE_UBC_CHECK_C="\"git-compat-util.h\"" endif endif endif
sha1dc/sha1.c +45 -34
137 lines not on this page · raw message
sha1dc/sha1.h +39 -51
136 lines not on this page · raw message
sha1dc/ubc_check.c +7 -2
@@ -24,8 +24,13 @@// ubc_check has been verified against ubc_check_verify using the 'ubc_check_test' program in the tools section */ -#include "git-compat-util.h" -#include "sha1dc/ubc_check.h" +#ifndef SHA1DC_NO_STANDARD_INCLUDES +#include <stdint.h> +#endif +#ifdef SHA1DC_CUSTOM_INCLUDE_UBC_CHECK_C +#include SHA1DC_CUSTOM_INCLUDE_UBC_CHECK_C +#endif +#include "ubc_check.h" static const uint32_t DV_I_43_0_bit = (uint32_t )(1 ) <<0 ;static const uint32_t DV_I_44_0_bit = (uint32_t )(1 ) <<1 ;
sha1dc/ubc_check.h +7 -3
@@ -20,13 +20,17 @@// thus one needs to do the recompression check for each DV that has its bit set */ -#ifndef UBC_CHECK_H -#define UBC_CHECK_H +#ifndef SHA1DC_UBC_CHECK_H +#define SHA1DC_UBC_CHECK_H #if defined(__cplusplus) extern "C" { #endif +#ifndef SHA1DC_NO_STANDARD_INCLUDES +#include <stdint.h> +#endif + #define DVMASKSIZE 1 typedef struct { int dvType; int dvK; int dvB; int testt; int maski; int maskb; uint32_t dm[80]; } dv_info_t; extern dv_info_t sha1_dvs[];
@@ -41,4 +45,4 @@ void ubc_check(const uint32_t W[80], uint32_t dvmask[DVMASKSIZE]); }#endif -#endif /* UBC_CHECK_H */ +#endif
sha1dc_git.c new file +19 -0
@@ -0,0 +1,19 @@ +void git_SHA1DCFinal (unsigned char hash[20 ],SHA1_CTX *ctx) +{ +if (!SHA1DCFinal (hash, ctx)) +return ; +die ("SHA-1 appears to be part of a collision attack: %s" , +sha1_to_hex (hash)); +} + +void git_SHA1DCUpdate (SHA1_CTX *ctx,const void *vdata,unsigned long len) +{ +const char *data = vdata; +/* We expect an unsigned long, but sha1dc only takes an int */ +while (len >INT_MAX ) { +SHA1DCUpdate (ctx, data,INT_MAX ); + data +=INT_MAX ; + len -=INT_MAX ; + } +SHA1DCUpdate (ctx, data, len); +}
sha1dc_git.h new file +14 -0
@@ -0,0 +1,14 @@ +/* +* Same as SHA1DCFinal, but convert collision attack case into a verbose die(). +*/ +void git_SHA1DCFinal (unsigned char [20 ],SHA1_CTX *); + +/* +* Same as SHA1DCUpdate, but adjust types to match git's usual interface. +*/ +void git_SHA1DCUpdate (SHA1_CTX *ctx,const void *data,unsigned long len); + +#define platform_SHA_CTX SHA1_CTX +#define platform_SHA1_Init SHA1DCInit +#define platform_SHA1_Update git_SHA1DCUpdate +#define platform_SHA1_Final git_SHA1DCFinal
Signature · 1 line
2.13.0.303.g4ebf302169
PATCH 2/3 sha1dc: use sha1collisiondetection as a submodule
In reply to Ævar Arnfjörð BjarmasonNow that we don't need local modifications to the code anymore with my avar/sha1collisiondetection:easier-inclusion-in-other-programs branch we can use the upstream code-as is.
If and when this change gets accepted by upstream we should of course point to the https://github.com/cr-marcstevens/sha1collisiondetection
master branch instead.
--- .gitmodules | 4 ++++ Makefile | 8 ++++---- hash.h | 2 +- sha1collisiondetection | 1 + 4 files changed, 10 insertions(+), 5 deletions(-) create mode 100644 .gitmodules create mode 160000 sha1collisiondetection
.gitmodules new file +4 -0
@@ -0,0 +1,4 @@ +[submodule "sha1collisiondetection"] + path = sha1collisiondetection + url = https://github.com/avar/sha1collisiondetection.git + branch = easier-inclusion-in-other-programs
Makefile +4 -4
@@ -1412,15 +1412,15 @@ ifdef APPLE_COMMON_CRYPTO BASIC_CFLAGS += -DSHA1_APPLE else DC_SHA1 := YesPlease - LIB_OBJS += sha1dc/sha1.o - LIB_OBJS += sha1dc/ubc_check.o + LIB_OBJS += sha1dc/lib/sha1.o + LIB_OBJS += sha1dc/lib/ubc_check.o BASIC_CFLAGS += \ -DSHA1_DC \ -DSHA1DC_NO_STANDARD_INCLUDES \ -DSHA1DC_INIT_SAFE_HASH_DEFAULT=0 \ -DSHA1DC_CUSTOM_INCLUDE_SHA1_C="\"cache.h\"" \ - -DSHA1DC_CUSTOM_TRAILING_INCLUDE_SHA1_C="\"../sha1dc_git.c\"" \ - -DSHA1DC_CUSTOM_TRAILING_INCLUDE_SHA1_H="\"../sha1dc_git.h\"" \ + -DSHA1DC_CUSTOM_TRAILING_INCLUDE_SHA1_C="\"../../sha1dc_git.c\"" \ + -DSHA1DC_CUSTOM_TRAILING_INCLUDE_SHA1_H="\"../../sha1dc_git.h\"" \ -DSHA1DC_CUSTOM_INCLUDE_UBC_CHECK_C="\"git-compat-util.h\"" endif endif
hash.h +1 -1
@@ -8,7 +8,7 @@#elif defined(SHA1_OPENSSL )#include <openssl/sha.h> #elif defined(SHA1_DC ) -#include "sha1dc/sha1.h" +#include "sha1dc/lib/sha1.h" #else /* SHA1_BLK */ #include "block-sha1/sha1.h" #endif
sha1collisiondetection new file +1 -0
@@ -0,0 +1 @@ +Subproject commit 5423dd8bcf3cdbd029dfa08d5d5f6dc044f7ac6d
Signature · 1 line
2.13.0.303.g4ebf302169
Re: [PATCH/RFC 0/3] Use sha1collisiondetection as a submodule
In reply to Ævar Arnfjörð BjarmasonQuoted 23 lines · Ævar Arnfjörð Bjarmason
Quoted 13 lines · Junio C Hamano
Quoted 11 lines · Ævar Arnfjörð Bjarmason
Quoted 9 lines · Junio C Hamano
From: Marc Stevens <[email protected]>
Some big-endian platforms define
_BIG_ENDIAN, which the test at the beginning of file has missed. Also, when the input is not aligned, some platforms trigger SIGBUS.This change corresponds to
33a694a9("Fix issues with a big endian platform", 2017-05-15) in the history of the upstream repository
https://github.com/cr-marcstevens/sha1collisiondetectionQuoted 5 lines
Quoted 2 lines
Then why not just update sha1dc from upstream instead of cherry-picking one patch from them?
See the original message upthread. I did the cherry-pick simply because that was what Marc instructed the patch recipient to do ;-).
Since that patch is now in Marc's upstream code we can just update to that.
Quoted 8 lines
While we're at it let's see if Marc will take a patch so that we can use his code as-is with some
Makefiletrickery of our own, instead of having to solve merge conflicts each time we update from him. I'll submit a pull request for that shortly.And since if and when that pull request gets accepted we're at the point of being able to use the upstream code as-is & don't need to locally modify it, we can just use a submodule to track it.
As someone who works on submodules: YAY! This sounds wonderful in the sense that more git developers experience the sharp edges (if any) of submodules and would fix them.
With the reset work on submodules (checkout, reset, ls-files, grep) and more in the works (better push/pull) we may be close to prime time.
However we do distribute tarballs (well, Junio does), which is not yet supported to include submodules.
I did follow the SHA1DC discussion just from the sideline, how crucial is that library for us?
Also now that we discuss having submodules:
Would we just point the submodule URL to github\
and call it a day?
We could make a friendly fork of it and put it next to all the repositories of https://git-blame.blogspot.com/p/git-public-repositories.html
and then use relative URLs in the .gitmodules file.
On a tangent, in an off-list discussion we discussed having the git-annex tests as an optional submodule instead of an "external" test, but in conclusion we considered that idea not ideal to implement (because our tests are like contracts, we should actually write our own tests and not rely on downstream to write tests for us)
looking forward for a discussion here,
Stefan
Re: [PATCH/RFC 0/3] Use sha1collisiondetection as a submodule
Quoted 30 lines · Stefan Beller
Quoted 28 lines · Ævar Arnfjörð Bjarmason
Quoted 13 lines · Junio C Hamano
Quoted 11 lines · Ævar Arnfjörð Bjarmason
Quoted 9 lines · Junio C Hamano
From: Marc Stevens <[email protected]>
Some big-endian platforms define
_BIG_ENDIAN, which the test at the beginning of file has missed. Also, when the input is not aligned, some platforms trigger SIGBUS.This change corresponds to
33a694a9("Fix issues with a big endian platform", 2017-05-15) in the history of the upstream repository
https://github.com/cr-marcstevens/sha1collisiondetectionQuoted 5 lines
Quoted 2 lines
Then why not just update sha1dc from upstream instead of cherry-picking one patch from them?
See the original message upthread. I did the cherry-pick simply because that was what Marc instructed the patch recipient to do ;-).
Since that patch is now in Marc's upstream code we can just update to that.
While we're at it let's see if Marc will take a patch so that we can use his code as-is with some
Makefiletrickery of our own, instead of having to solve merge conflicts each time we update from him. I'll submit a pull request for that shortly.
Quoted 7 lines
Quoted 3 lines
And since if and when that pull request gets accepted we're at the point of being able to use the upstream code as-is & don't need to locally modify it, we can just use a submodule to track it.
As someone who works on submodules: YAY! This sounds wonderful in the sense that more git developers experience the sharp edges (if any) of submodules and would fix them.
Yeah I agree git.git should dogfood submodules, and this seemed like a perfect opportunity for thaht.
As noted later in your mail everything may not be ready, so when I submit a non-RFC version of this (after upstream pulls my changes, hopefully), the 2nd and 3rd patches can just be dropped.
Quoted 6 lines
With the reset work on submodules (checkout, reset, ls-files, grep) and more in the works (better push/pull) we may be close to prime time.
However we do distribute tarballs (well, Junio does), which is not yet supported to include submodules.
Ouch. So there's no non-manual way with git-archive to make a tarball release of a git project using submodules?
Quoted 2 lines
I did follow the SHA1DC discussion just from the sideline, how crucial is that library for us?
Since 2.13.0 it's git's default sha1 implementation.
Quoted 7 lines
Also now that we discuss having submodules:
Would we just point the submodule URL to github\
and call it a day?We could make a friendly fork of it and put it next to all the repositories of https://git-blame.blogspot.com/p/git-public-repositories.html
and then use relative URLs in the .gitmodules file.
Wouldn't we lose the ability to just "pull" the module to see if upstream changed, i.e. now we'd need to manually munge the URL first.
If so having a fetchurl be a relative URL similar to git-push's pushurl would solve it wouldn't it? I.e. a project like git could say "here's my mirrors" different from "here's my upstream".
Re: [PATCH/RFC 0/3] Use sha1collisiondetection as a submodule
Quoted 50 lines · Ævar Arnfjörð Bjarmason
Quoted 30 lines · Stefan Beller
Quoted 28 lines · Ævar Arnfjörð Bjarmason
Quoted 13 lines · Junio C Hamano
Quoted 11 lines · Ævar Arnfjörð Bjarmason
Quoted 9 lines · Junio C Hamano
From: Marc Stevens <[email protected]>
Some big-endian platforms define
_BIG_ENDIAN, which the test at the beginning of file has missed. Also, when the input is not aligned, some platforms trigger SIGBUS.This change corresponds to
33a694a9("Fix issues with a big endian platform", 2017-05-15) in the history of the upstream repository
https://github.com/cr-marcstevens/sha1collisiondetectionQuoted 5 lines
Quoted 2 lines
Then why not just update sha1dc from upstream instead of cherry-picking one patch from them?
See the original message upthread. I did the cherry-pick simply because that was what Marc instructed the patch recipient to do ;-).
Since that patch is now in Marc's upstream code we can just update to that.
While we're at it let's see if Marc will take a patch so that we can use his code as-is with some
Makefiletrickery of our own, instead of having to solve merge conflicts each time we update from him. I'll submit a pull request for that shortly.Quoted 7 lines
Quoted 3 lines
And since if and when that pull request gets accepted we're at the point of being able to use the upstream code as-is & don't need to locally modify it, we can just use a submodule to track it.
As someone who works on submodules: YAY! This sounds wonderful in the sense that more git developers experience the sharp edges (if any) of submodules and would fix them.
Yeah I agree git.git should dogfood submodules, and this seemed like a perfect opportunity for thaht.
As noted later in your mail everything may not be ready, so when I submit a non-RFC version of this (after upstream pulls my changes, hopefully), the 2nd and 3rd patches can just be dropped.
Quoted 4 lines
With the reset work on submodules (checkout, reset, ls-files, grep) and more in the works (better push/pull) we may be close to prime time.
Quoted 5 lines
Quoted 2 lines
However we do distribute tarballs (well, Junio does), which is not yet supported to include submodules.
Ouch. So there's no non-manual way with git-archive to make a tarball release of a git project using submodules?
Yes. Ouch.
Quoted 6 lines
Quoted 3 lines
We could make a friendly fork of it and put it next to all the repositories of https://git-blame.blogspot.com/p/git-public-repositories.html
and then use relative URLs in the .gitmodules file.Wouldn't we lose the ability to just "pull" the module to see if upstream changed, i.e. now we'd need to manually munge the URL first.
I assumed more people would fetch git and trust the community rather than people checking if SHA1DC is up-to-date, so I thought it is easier for the minority to rewrite a URL.
Quoted 3 lines
If so having a fetchurl be a relative URL similar to git-push's pushurl would solve it wouldn't it? I.e. a project like git could say "here's my mirrors" different from "here's my upstream".
I would think so.
Re: [PATCH/RFC 0/3] Use sha1collisiondetection as a submodule
In reply to Stefan BellerQuoted 30 lines · Stefan Beller
Quoted 28 lines · Ævar Arnfjörð Bjarmason
Quoted 13 lines · Junio C Hamano
Quoted 11 lines · Ævar Arnfjörð Bjarmason
Quoted 9 lines · Junio C Hamano
From: Marc Stevens <[email protected]>
Some big-endian platforms define
_BIG_ENDIAN, which the test at the beginning of file has missed. Also, when the input is not aligned, some platforms trigger SIGBUS.This change corresponds to
33a694a9("Fix issues with a big endian platform", 2017-05-15) in the history of the upstream repository
https://github.com/cr-marcstevens/sha1collisiondetectionQuoted 5 lines
Quoted 2 lines
Then why not just update sha1dc from upstream instead of cherry-picking one patch from them?
See the original message upthread. I did the cherry-pick simply because that was what Marc instructed the patch recipient to do ;-).
Since that patch is now in Marc's upstream code we can just update to that.
While we're at it let's see if Marc will take a patch so that we can use his code as-is with some
Makefiletrickery of our own, instead of having to solve merge conflicts each time we update from him. I'll submit a pull request for that shortly.
Quoted 7 lines
Quoted 3 lines
And since if and when that pull request gets accepted we're at the point of being able to use the upstream code as-is & don't need to locally modify it, we can just use a submodule to track it.
As someone who works on submodules: YAY! This sounds wonderful in the sense that more git developers experience the sharp edges (if any) of submodules and would fix them.
Haha, I wish I had your enthusiasm for this. That is a good argument for us to include them in git.git, but I still feel apprehensive. There is still so much to do in submodule land! We'll make them great though!
Quoted 28 lines
With the reset work on submodules (checkout, reset, ls-files, grep) and more in the works (better push/pull) we may be close to prime time.
However we do distribute tarballs (well, Junio does), which is not yet supported to include submodules.
I did follow the SHA1DC discussion just from the sideline, how crucial is that library for us?
Also now that we discuss having submodules:
Would we just point the submodule URL to github\
and call it a day?We could make a friendly fork of it and put it next to all the repositories of https://git-blame.blogspot.com/p/git-public-repositories.html
and then use relative URLs in the .gitmodules file.On a tangent, in an off-list discussion we discussed having the git-annex tests as an optional submodule instead of an "external" test, but in conclusion we considered that idea not ideal to implement (because our tests are like contracts, we should actually write our own tests and not rely on downstream to write tests for us)
looking forward for a discussion here,
Stefan
Signature · 1 line
Brandon Williams
Re: [PATCH] sha1dc: fix issues with a big endian platform
In reply to Junio C HamanoHi Junio,
Quoted 14 lines · Junio C Hamano
sha1dc/sha1.c +1
-1@@ -20,7 +20,7 @@*/ #if (defined(__BYTE_ORDER) && (__BYTE_ORDER == __BIG_ENDIAN)) || \ (defined(__BYTE_ORDER__) && (__BYTE_ORDER__ == __BIG_ENDIAN__)) || \ - defined(__BIG_ENDIAN__) || defined(__ARMEB__) || defined(__THUMBEB__) || defined(__AARCH64EB__) || \ + defined(_BIG_ENDIAN) || defined(__BIG_ENDIAN__) || defined(__ARMEB__) || defined(__THUMBEB__) || defined(__AARCH64EB__) || \ defined(_MIPSEB) || defined(__MIPSEB) || defined(__MIPSEB__)#define SHA1DC_BIGENDIAN 1
sha1dc/sha1.c void SHA1DCUpdate(SHA1_CTX* ctx, const char* buf, size_t len) · line 1733
Quoted 7 lines
sha1dc/sha1.c +2
-1@@ -1728,7 +1728,8 @@ void SHA1DCUpdate(SHA1_CTX* ctx, const char* buf, size_t len)while (len >=64 ) { ctx->total +=64 ; -sha1_process (ctx, (uint32_t *)(buf)); +memcpy (ctx->buffer, buf,64 ); +sha1_process (ctx, (uint32_t *)(ctx->buffer)); buf +=64 ;
There is actually a discussion going on about this. See
for details.
The fixup commit under discussion is here:
Ciao,
dscho