SHA1dc on mac
Hi,
If I'm not mistaken in my reading of the various files involved, it looks like for some reason, building git on mac leads to using Apple Common Crypto for SHA1, rather than SHA1dc, which seems unfortunate. Is that really expected? More generally, at this point, should anything other than SHA1dc be supported as a build option at all?
Mike
Quoted 4 lines · Mike Hommey
If I'm not mistaken in my reading of the various files involved, it looks like for some reason, building git on mac leads to using Apple Common Crypto for SHA1, rather than SHA1dc, which seems unfortunate. Is that really expected?
There was a discussion on this topic a while back[1], and it does seem that the behavior you describe is intentional[2].
Quoted 2 lines
More generally, at this point, should anything other than SHA1dc be supported as a build option at all?
The conclusion [2,3] was that it likely would make sense to drop support for Apple's CommonCrypto altogether, although nobody has yet stepped up to do the work.
[1]: https://lore.kernel.org/git/CAMYxyaVQyVRQb-b0nVv412tMZ3rEnOfUPRakg2dEREg5_Ba5Ag@mail.gmail.com/T/
[2]: https://lore.kernel.org/git/[email protected]/
[3]: https://lore.kernel.org/git/CAPig+cQ5kKAt2_RQnqT7Rn=uGmHV9VvxpQ+UgDPOj=D=pq6arg@mail.gmail.com/
Quoted 8 lines · Eric Sunshine
Quoted 4 lines · Mike Hommey
If I'm not mistaken in my reading of the various files involved, it looks like for some reason, building git on mac leads to using Apple Common Crypto for SHA1, rather than SHA1dc, which seems unfortunate. Is that really expected?
There was a discussion on this topic a while back[1], and it does seem that the behavior you describe is intentional[2].
That discussion predates SHA1dc, though.
Quoted 6 lines
Quoted 2 lines
More generally, at this point, should anything other than SHA1dc be supported as a build option at all?
The conclusion [2,3] was that it likely would make sense to drop support for Apple's CommonCrypto altogether, although nobody has yet stepped up to do the work.
I wasn't explicit in my question, but I meant more broadly than Apple Common Crypto. There is still opt-in support for openssl sha1 and PPC sha1.
Mike
Quoted 11 lines · Mike Hommey
Quoted 8 lines · Eric Sunshine
Quoted 4 lines · Mike Hommey
If I'm not mistaken in my reading of the various files involved, it looks like for some reason, building git on mac leads to using Apple Common Crypto for SHA1, rather than SHA1dc, which seems unfortunate. Is that really expected?
There was a discussion on this topic a while back[1], and it does seem that the behavior you describe is intentional[2].
That discussion predates SHA1dc, though.
Yes, but the essense is the same. It was phrased as "is there a good reason to prefer CommonCrypto over block-sha1?" but it really was "is there a good reason to prefer CommonCrypto over the best we offer?" And the best we offer, which used to be block-sha1, is now sha1dc.
PATCH 1/1 Remove non-SHA1dc sha1 implementations
It is 2020, and with the weakening of SHA1 security-wise, there doesn't seem to be a reason to support anything else than SHA1dc, with collision detection.
---
Note: I only tested building on Linux.
INSTALL | 5 - Makefile | 67 ++----------- block-sha1/sha1.c | 251 ---------------------------------------------- block-sha1/sha1.h | 22 ---- config.mak.uname | 1 - configure.ac | 3 - hash.h | 24 ----- ppc/sha1.c | 72 ------------- ppc/sha1.h | 25 ----- ppc/sha1ppc.S | 224 ----------------------------------------- 10 files changed, 6 insertions(+), 688 deletions(-) delete mode 100644 block-sha1/sha1.c delete mode 100644 block-sha1/sha1.h delete mode 100644 ppc/sha1.c delete mode 100644 ppc/sha1.h delete mode 100644 ppc/sha1ppc.S
INSTALL +0 -5
@@ -133,11 +133,6 @@ Issues of note: you are using libcurl older than 7.34.0. Otherwise you can use NO_OPENSSL without losing git-imap-send. - By default, git uses OpenSSL for SHA1 but it will use its own - library (inspired by Mozilla's) with either NO_OPENSSL or - BLK_SHA1. Also included is a version optimized for PowerPC - (PPC_SHA1). - - "libcurl" library is used by git-http-fetch, git-fetch, and, if the curl version >= 7.34.0, for git-imap-send. You might also want the "curl" executable for debugging purposes. If you do not
Makefile +6 -61
100 lines not on this page · raw message
block-sha1/sha1.c deleted +0 -251
251 lines not on this page · raw message
block-sha1/sha1.h deleted +0 -22
@@ -1,22 +0,0 @@ -/* -* SHA1 routine optimized to do word accesses rather than byte accesses, -* and to avoid unnecessary copies into the context array. -* -* This was initially based on the Mozilla SHA1 implementation, although -* none of the original Mozilla code remains. -*/ - -typedef struct { -unsigned long long size; -unsigned int H[5 ]; -unsigned int W[16 ]; -}blk_SHA_CTX ; - -void blk_SHA1_Init (blk_SHA_CTX *ctx); -void blk_SHA1_Update (blk_SHA_CTX *ctx,const void *dataIn,unsigned long len); -void blk_SHA1_Final (unsigned char hashout[20 ],blk_SHA_CTX *ctx); - -#define platform_SHA_CTX blk_SHA_CTX -#define platform_SHA1_Init blk_SHA1_Init -#define platform_SHA1_Update blk_SHA1_Update -#define platform_SHA1_Final blk_SHA1_Final
config.mak.uname +0 -1
@@ -116,7 +116,6 @@ ifeq ($(uname_S),Darwin) # i.e. "begins with [15678] and a dot" means "10.4.* or older". ifeq ($(shell expr "$(uname_R)" : '[15678]\.'),2) OLD_ICONV = UnfortunatelyYes - NO_APPLE_COMMON_CRYPTO = YesPlease endif ifeq ($(shell expr "$(uname_R)" : '[15]\.'),2) NO_STRLCPY = YesPlease
configure.ac +0 -3
@@ -237,9 +237,6 @@ AC_MSG_NOTICE([CHECKS for site configuration]) # tests. These tests take up a significant amount of the total test time # but are not needed unless you plan to talk to SVN repos. # -# Define PPC_SHA1 environment variable when running make to make use of -# a bundled SHA1 routine optimized for PowerPC. -# # Define NO_OPENSSL environment variable if you do not have OpenSSL. # # Define OPENSSLDIR=/foo/bar if your openssl header and library files are in
hash.h +0 -24
@@ -3,17 +3,7 @@#include "git-compat-util.h" -#if defined(SHA1_PPC ) -#include "ppc/sha1.h" -#elif defined(SHA1_APPLE ) -#include <CommonCrypto/CommonDigest.h> -#elif defined(SHA1_OPENSSL ) -#include <openssl/sha.h> -#elif defined(SHA1_DC )#include "sha1dc_git.h" -#else /* SHA1_BLK */ -#include "block-sha1/sha1.h" -#endif #if defined(SHA256_GCRYPT )#include "sha256/gcrypt.h"
@@ -23,20 +13,6 @@#include "sha256/block/sha256.h" #endif -#ifndef platform_SHA_CTX -/* -* platform's underlying implementation of SHA-1; could be OpenSSL, -* blk_SHA, Apple CommonCrypto, etc... Note that the relevant -* SHA-1 header may have already defined platform_SHA_CTX for our -* own implementations like block-sha1 and ppc-sha1, so we list -* the default for OpenSSL compatible SHA-1 implementations here. -*/ -#define platform_SHA_CTX SHA_CTX -#define platform_SHA1_Init SHA1_Init -#define platform_SHA1_Update SHA1_Update -#define platform_SHA1_Final SHA1_Final -#endif -#define git_SHA_CTX platform_SHA_CTX#define git_SHA1_Init platform_SHA1_Init#define git_SHA1_Update platform_SHA1_Update
ppc/sha1.c deleted +0 -72
72 lines not on this page · raw message
ppc/sha1.h deleted +0 -25
@@ -1,25 +0,0 @@ -/* -* SHA-1 implementation. -* -* Copyright (C) 2005 Paul Mackerras <[email protected]> -*/ -#include <stdint.h> - -typedef struct { -uint32_t hash[5 ]; -uint32_t cnt; -uint64_t len; -union { -unsigned char b[64 ]; -uint64_t l[8 ]; - } buf; -}ppc_SHA_CTX ; - -int ppc_SHA1_Init (ppc_SHA_CTX *c); -int ppc_SHA1_Update (ppc_SHA_CTX *c,const void *p,unsigned long n); -int ppc_SHA1_Final (unsigned char *hash,ppc_SHA_CTX *c); - -#define platform_SHA_CTX ppc_SHA_CTX -#define platform_SHA1_Init ppc_SHA1_Init -#define platform_SHA1_Update ppc_SHA1_Update -#define platform_SHA1_Final ppc_SHA1_Final
ppc/sha1ppc.S deleted +0 -224
224 lines not on this page · raw message
Signature · 1 line
2.24.0.424.g559c6fc317.dirty
Re: [PATCH] Remove non-SHA1dc sha1 implementations
Quoted 3 lines · Mike Hommey
It is 2020, and with the weakening of SHA1 security-wise, there doesn't seem to be a reason to support anything else than SHA1dc, with collision detection.
One possible reason is that they're way faster than sha1dc (block-sha1 maybe only a little, but openssl's sha1 is over twice as fast).
To be clear, I think the slowdown is worth the extra safety, but:
- do we still want to care about people who prefer to make the tradeoff differently?
- when we first switched the default to sha1dc, the idea was raised of continuing to use a faster implementation for non-security checksums (e.g., the checksums at the end of packfiles, index files, etc). I don't think anybody ever implemented that, but it's not a terrible idea. OTOH, if nobody noticed the bottleneck enough to care, maybe it's not worth worrying about.
I'm not convinced the answer to those questions is "yes", but I think it's worth at least raising them (and arguing against them in the commit message).
One thing that compels me is the recent report that we still build with common crypto by default on macOS, which was definitely _not_ intended. That's a bug that can be fixed, but it wouldn't have happened in the first place if we only supported sha1dc.
-Peff
Re: [PATCH] Remove non-SHA1dc sha1 implementations
Quoted 4 lines · Jeff King
One thing that compels me is the recent report that we still build with common crypto by default on macOS, which was definitely _not_ intended. That's a bug that can be fixed, but it wouldn't have happened in the first place if we only supported sha1dc.
I just noticed you were the original reporter there, too. So I guess it compelled you, too. ;)
If we do want to keep the other implementations around, another thing that might be worth doing is to teach t0013 to complain when the collision-detecting sha1 is not in use (i.e., rather than auto-skipping when built without DC_SHA1, require the user to set a special NO_REALLY_I_CHOOSE_NOT_TO_USE_DC_SHA1_AND_AM_AWARE_OF_THE_IMPLICATIONS variable). That would provide a cross-check on the build flags.
-Peff
PATCH 1/1 ppc: remove custom SHA-1 implementation
In reply to Mike HommeyRemove the PPC_SHA1 implementation added in a6ef3518f9a ([PATCH] PPC assembly implementation of SHA1, 2005-04-22). When this was added Apple consumer hardware used the PPC architecture, and the implementation was intended to improve SHA-1 speed there.
Since it was added we've moved to DC_SHA1 by default, and anyone wanting hard-rolled non-DC SHA-1 implementation can use OpenSSL's via the OPENSSL_SHA1 knob.
I'm unsure if this was ever supposed to work on 64-bit PPC. It clearly originally targeted 32 bit PPC, but there's some mailing list references to this being tried on G5 (PPC 970). I can't get it to do anything but segfault on the BE POWER8 machine in the GCC compile farm. Anyone caring about speed on PPC these days is likely to be using IBM's POWER, not PPC 970.
There have been proposals to entirely remove non-DC_SHA1 implementations from the tree[1]. I think per [2] that would be a bit overzealous. I.e. there are various set-ups git's speed is going to be more important than the relatively implausible SHA-1 collision attack, or where such attacks are entirely mitigated by other means (e.g. by incoming objects being checked with DC_SHA1).
The main reason for doing so at this point is to simplify follow-up Makefile change. Since PPC_SHA1 included the only in-tree *.S assembly file we needed to keep around special support for building objects from it. By getting rid of it we know we'll always build *.o from *.c files, which makes the build process simpler.
As an aside the code being removed here was also throwing warnings with the "-pedantic" flag, but let's remove it instead of fixing it, as 544d93bc3b4 (block-sha1: remove use of obsolete x86 assembly, 2022-03-10) did for block-sha1/*.
1. https://lore.kernel.org/git/[email protected]/
2. https://lore.kernel.org/git/[email protected]/
--- INSTALL | 3 +- Makefile | 35 ++++---- configure.ac | 3 - hash.h | 6 +- ppc/sha1.c | 72 ---------------- ppc/sha1.h | 25 ------ ppc/sha1ppc.S | 224 -------------------------------------------------- 7 files changed, 17 insertions(+), 351 deletions(-) delete mode 100644 ppc/sha1.c delete mode 100644 ppc/sha1.h delete mode 100644 ppc/sha1ppc.S
INSTALL +1 -2
@@ -135,8 +135,7 @@ Issues of note: By default, git uses OpenSSL for SHA1 but it will use its own library (inspired by Mozilla's) with either NO_OPENSSL or - BLK_SHA1. Also included is a version optimized for PowerPC - (PPC_SHA1). + BLK_SHA1. - "libcurl" library is used for fetching and pushing repositories over http:// or https://, as well as by
Makefile +14 -21
@@ -155,9 +155,6 @@ include shared.mak # Define BLK_SHA1 environment variable to make use of the bundled # optimized C SHA1 routine. # -# Define PPC_SHA1 environment variable when running make to make use of -# a bundled SHA1 routine optimized for PowerPC. -# # Define DC_SHA1 to unconditionally enable the collision-detecting sha1 # algorithm. This is slower, but may detect attempted collision attacks. # Takes priority over other *_SHA1 knobs.
@@ -1770,14 +1767,14 @@ ifdef OPENSSL_SHA1 EXTLIBS += $(LIB_4_CRYPTO) BASIC_CFLAGS += -DSHA1_OPENSSL else +ifdef PPC_SHA1 +$(error PPC_SHA1 has been removed! You should almost definitely remove that \ +knob and use the DC_SHA1 default! See INSTALL for more information) +endif ifdef BLK_SHA1 LIB_OBJS += block-sha1/sha1.o BASIC_CFLAGS += -DSHA1_BLK else -ifdef PPC_SHA1 - LIB_OBJS += ppc/sha1.o ppc/sha1ppc.o - BASIC_CFLAGS += -DSHA1_PPC -else ifdef APPLE_COMMON_CRYPTO COMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL BASIC_CFLAGS += -DSHA1_APPLE
@@ -1811,7 +1808,6 @@ endif endif endif endif -endif ifdef OPENSSL_SHA256 EXTLIBS += $(LIB_4_CRYPTO)
@@ -2509,6 +2505,11 @@ OBJECTS += $(SCALAR_OBJECTS) .PHONY: objects objects: $(OBJECTS) +# Derived from $(OBJECTS) +OBJECTS_C = $(OBJECTS:%.o=%.c) +OBJECTS_S = $(OBJECTS:%.o=%.s) +OBJECTS_SP = $(OBJECTS:%.o=%.sp) + dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d) dep_dirs := $(addsuffix .depend,$(sort $(dir $(OBJECTS))))
@@ -2540,13 +2541,7 @@ missing_compdb_dir = compdb_args = endif -ASM_SRC := $(wildcard $(OBJECTS:o=S)) -ASM_OBJ := $(ASM_SRC:S=o) -C_OBJ := $(filter-out $(ASM_OBJ),$(OBJECTS)) - -$(C_OBJ): %.o: %.c GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir) - $(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(compdb_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $< -$(ASM_OBJ): %.o: %.S GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir) +$(OBJECTS): %.o: %.c GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir) $(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(compdb_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $< %.s: %.c GIT-CFLAGS FORCE
@@ -2692,7 +2687,7 @@ XGETTEXT_FLAGS_SH = $(XGETTEXT_FLAGS) --language=Shell \ --keyword=gettextln --keyword=eval_gettextln XGETTEXT_FLAGS_PERL = $(XGETTEXT_FLAGS) --language=Perl \ --keyword=__ --keyword=N__ --keyword="__n:1,2" -LOCALIZED_C = $(C_OBJ:o=c) $(LIB_H) $(GENERATED_H) +LOCALIZED_C = $(OBJECTS_C) $(LIB_H) $(GENERATED_H) LOCALIZED_SH = $(SCRIPT_SH) LOCALIZED_SH += git-sh-setup.sh LOCALIZED_PERL = $(SCRIPT_PERL)
@@ -2939,16 +2934,14 @@ t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS) $(REFTABLE_TEST_LIB) check-sha1:: t/helper/test-tool$X t/helper/test-sha1.sh -SP_OBJ = $(patsubst %.o,%.sp,$(C_OBJ)) - -$(SP_OBJ): %.sp: %.c %.o +$(OBJECTS_SP): %.sp: %.c %.o $(QUIET_SP)cgcc -no-compile $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) \ -Wsparse-error \ $(SPARSE_FLAGS) $(SP_EXTRA_FLAGS) $< && \ >$@ .PHONY: sparse -sparse: $(SP_OBJ) +sparse: $(OBJECTS_SP) EXCEPT_HDRS := $(GENERATED_H) unicode-width.h compat/% xdiff/% ifndef GCRYPT_SHA256
@@ -3272,7 +3265,7 @@ clean: profile-clean coverage-clean cocciclean $(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) git$X $(RM) $(TEST_PROGRAMS) $(RM) $(FUZZ_PROGRAMS) - $(RM) $(SP_OBJ) + $(RM) $(OBJECTS_SP) $(RM) $(HCC) $(RM) -r bin-wrappers $(dep_dirs) $(compdb_dir) compile_commands.json $(RM) -r po/build/
configure.ac +0 -3
@@ -237,9 +237,6 @@ AC_MSG_NOTICE([CHECKS for site configuration]) # tests. These tests take up a significant amount of the total test time # but are not needed unless you plan to talk to SVN repos. # -# Define PPC_SHA1 environment variable when running make to make use of -# a bundled SHA1 routine optimized for PowerPC. -# # Define NO_OPENSSL environment variable if you do not have OpenSSL. # # Define OPENSSLDIR=/foo/bar if your openssl header and library files are in
hash.h +2 -4
@@ -4,9 +4,7 @@#include "git-compat-util.h" #include "repository.h" -#if defined(SHA1_PPC ) -#include "ppc/sha1.h" -#elif defined(SHA1_APPLE ) +#if defined(SHA1_APPLE )#include <CommonCrypto/CommonDigest.h> #elif defined(SHA1_OPENSSL )#include <openssl/sha.h>
@@ -30,7 +28,7 @@* platform's underlying implementation of SHA-1; could be OpenSSL, * blk_SHA, Apple CommonCrypto, etc... Note that the relevant * SHA-1 header may have already defined platform_SHA_CTX for our -* own implementations like block-sha1 and ppc-sha1, so we list +* own implementations like block-sha1, so we list * the default for OpenSSL compatible SHA-1 implementations here. */ #define platform_SHA_CTX SHA_CTX
ppc/sha1.c deleted +0 -72
72 lines not on this page · raw message
ppc/sha1.h deleted +0 -25
@@ -1,25 +0,0 @@ -/* -* SHA-1 implementation. -* -* Copyright (C) 2005 Paul Mackerras <[email protected]> -*/ -#include <stdint.h> - -typedef struct { -uint32_t hash[5 ]; -uint32_t cnt; -uint64_t len; -union { -unsigned char b[64 ]; -uint64_t l[8 ]; - } buf; -}ppc_SHA_CTX ; - -int ppc_SHA1_Init (ppc_SHA_CTX *c); -int ppc_SHA1_Update (ppc_SHA_CTX *c,const void *p,unsigned long n); -int ppc_SHA1_Final (unsigned char *hash,ppc_SHA_CTX *c); - -#define platform_SHA_CTX ppc_SHA_CTX -#define platform_SHA1_Init ppc_SHA1_Init -#define platform_SHA1_Update ppc_SHA1_Update -#define platform_SHA1_Final ppc_SHA1_Final
ppc/sha1ppc.S deleted +0 -224
224 lines not on this page · raw message
Signature · 1 line
2.35.1.1438.g8874c8eeb35
Re: [PATCH] ppc: remove custom SHA-1 implementation
Quoted 5 lines · Ævar Arnfjörð Bjarmason
The main reason for doing so at this point is to simplify follow-up
Makefilechange. SincePPC_SHA1included the only in-tree *.S assembly file we needed to keep around special support for building objects from it. By getting rid of it we know we'll always build *.o from *.c files, which makes the build process simpler.
Yuck.
Quoted 15 lines
Makefile +0
-3@@ -155,9 +155,6 @@ include shared.mak # Define BLK_SHA1 environment variable to make use of the bundled # optimized C SHA1 routine. # -# Define PPC_SHA1 environment variable when running make to make use of -# a bundled SHA1 routine optimized for PowerPC. -# # Define DC_SHA1 to unconditionally enable the collision-detecting sha1 # algorithm. This is slower, but may detect attempted collision attacks. # Takes priority over other *_SHA1 knobs.
Makefile ifdef OPENSSL_SHA1 · line 1773
Quoted 7 lines
Makefile +4
-0@@ -1770,14 +1767,14 @@ ifdef OPENSSL_SHA1 EXTLIBS += $(LIB_4_CRYPTO) BASIC_CFLAGS += -DSHA1_OPENSSL else +ifdef PPC_SHA1 +$(error PPC_SHA1 has been removed! You should almost definitely remove that \ +knob and use the DC_SHA1 default! See INSTALL for more information) +endif
"use the DC_SHA1 default"?
PPC_SHA1 support is being removed. Use DC_SHA1 instead, which is the default.
I am wondering if we can make only these four lines the first step of remova, without doing anything else. It would give us a good feel on how many users we may be inconveniencing (not necessarily hurting, as switching to DC_SHA1 would be a good move) by the removal.
Makefile OBJECTS += $(SCALAR_OBJECTS) · line 2511
Quoted 8 lines
Makefile +4
-0@@ -2509,6 +2505,11 @@ OBJECTS += $(SCALAR_OBJECTS) .PHONY: objects objects: $(OBJECTS) +# Derived from $(OBJECTS) +OBJECTS_C = $(OBJECTS:%.o=%.c) +OBJECTS_S = $(OBJECTS:%.o=%.s) +OBJECTS_SP = $(OBJECTS:%.o=%.sp)
Usually we build objects from sources, so "derived from" is a puzzling way to call them. I understand we are deriving a list from another list, but it still feels confusing (see below for ASM_OBJ).
This seems to have nothing to do with "we no longer have *.S files" and looks more like "now we have excuse to touch Makefile, make random changes that look subjectively good to me, burying them in the noise so that we can sneak them in without justifying them much in the proposed log message".
Makefile missing_compdb_dir = · line 2545
Quoted 10 lines
Makefile +0
-3dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d) dep_dirs := $(addsuffix .depend,$(sort $(dir $(OBJECTS))))@@ -2540,13 +2541,7 @@ missing_compdb_dir = compdb_args = endif -ASM_SRC := $(wildcard $(OBJECTS:o=S)) -ASM_OBJ := $(ASM_SRC:S=o) -C_OBJ := $(filter-out $(ASM_OBJ),$(OBJECTS))
I tend to agree with this patch that these three lines are ugly in multiple ways.
It's a confusing construct. The list of OBJECTS is used as the single source of truth and others are derived by filtering the list and futzing the suffix of the resulting subset of elements; it makes me wonder if it should be the other way around (i.e. we have a list of source files in various languages, all get turned into objects, rather than we have a list of object files, and we see if corresponding source file in possible languages, if any, exists on disk). It is pleasing that we can see them go.
Unfortunately the new "Derived from $(OBJECTS) lists are in the same spirit as used by ASM_SRC we are removing here, aren't they?
Makefile missing_compdb_dir = · line 2545
Quoted 5 lines
Makefile +1
-3-$(C_OBJ): %.o: %.c GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir) - $(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(compdb_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $< -$(ASM_OBJ): %.o: %.S GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir) +$(OBJECTS): %.o: %.c GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir) $(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(compdb_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<
Now we deal only with *.c sources and all objects come from them, and the action is the same as the old C_OBJ rule, naturally.
Makefile missing_compdb_dir = · line 2547
Quoted 1 line
Makefile +0
-0%.s: %.c GIT-CFLAGS FORCE
This is the only remaining rule regarding assembly and it is the target to generate for debugging, not even used as a source to create object files. Do we need OBJECTS_S defined above? I somehow doubt it. FWIW, I do not see the need for OBJECTS_SP or OBJECTS_C, either. E.g.
Makefile XGETTEXT_FLAGS_SH = $(XGETTEXT_FLAGS) --language=Shell \ · line 2690
Quoted 6 lines
Makefile +1
-1@@ -2692,7 +2687,7 @@ XGETTEXT_FLAGS_SH = $(XGETTEXT_FLAGS) --language=Shell \ --keyword=gettextln --keyword=eval_gettextln XGETTEXT_FLAGS_PERL = $(XGETTEXT_FLAGS) --language=Perl \ --keyword=__ --keyword=N__ --keyword="__n:1,2" -LOCALIZED_C = $(C_OBJ:o=c) $(LIB_H) $(GENERATED_H) +LOCALIZED_C = $(OBJECTS_C) $(LIB_H) $(GENERATED_H)
Shouldn't it be sufficient to use %(OBJECTS:o=c) here, i.e. equating the old C_OBJ with OBJECTS, now we know all objects come from C? The same question for SP_OBJ below, but I won't repeat.
PATCH v2 1/1 ppc: remove custom SHA-1 implementation
In reply to Ævar Arnfjörð BjarmasonRemove the PPC_SHA1 implementation added in a6ef3518f9a ([PATCH] PPC assembly implementation of SHA1, 2005-04-22). When this was added Apple consumer hardware used the PPC architecture, and the implementation was intended to improve SHA-1 speed there.
Since it was added we've moved to DC_SHA1 by default, and anyone wanting hard-rolled non-DC SHA-1 implementation can use OpenSSL's via the OPENSSL_SHA1 knob.
I'm unsure if this was ever supposed to work on 64-bit PPC. It clearly originally targeted 32 bit PPC, but there's some mailing list references to this being tried on G5 (PPC 970). I can't get it to do anything but segfault on the BE POWER8 machine in the GCC compile farm. Anyone caring about speed on PPC these days is likely to be using IBM's POWER, not PPC 970.
There have been proposals to entirely remove non-DC_SHA1 implementations from the tree[1]. I think per [2] that would be a bit overzealous. I.e. there are various set-ups git's speed is going to be more important than the relatively implausible SHA-1 collision attack, or where such attacks are entirely mitigated by other means (e.g. by incoming objects being checked with DC_SHA1).
The main reason for doing so at this point is to simplify follow-up Makefile change. Since PPC_SHA1 included the only in-tree *.S assembly file we needed to keep around special support for building objects from it. By getting rid of it we know we'll always build *.o from *.c files, which makes the build process simpler.
As an aside the code being removed here was also throwing warnings with the "-pedantic" flag, but let's remove it instead of fixing it, as 544d93bc3b4 (block-sha1: remove use of obsolete x86 assembly, 2022-03-10) did for block-sha1/*.
1. https://lore.kernel.org/git/[email protected]/
2. https://lore.kernel.org/git/[email protected]/
---
A more minimal change to the Makefile than that in v1. In v1 I thought it was a bit odd to leave in-place the C_OBJ variable that was only declared for now-removed code. But sure, we can keep it and keep defining SP_OBJ etc. as being derived from it.
Range-diff against v1:
1: 05dcdca3877 ! 1: e77fd23a824 ppc: remove custom SHA-1 implementation
@@ Makefile: ifdef OPENSSL_SHA1
BASIC_CFLAGS += -DSHA1_OPENSSL
else
+ifdef PPC_SHA1
-+$(error PPC_SHA1 has been removed! You should almost definitely remove that \
-+knob and use the DC_SHA1 default! See INSTALL for more information)
++$(error PPC_SHA1 has been removed! Use DC_SHA1 instead, which is the default)
+endif
ifdef BLK_SHA1
LIB_OBJS += block-sha1/sha1.o
@@ Makefile: endif
ifdef OPENSSL_SHA256
EXTLIBS += $(LIB_4_CRYPTO)
-@@ Makefile: OBJECTS += $(SCALAR_OBJECTS)
- .PHONY: objects
- objects: $(OBJECTS)
-
-+# Derived from $(OBJECTS)
-+OBJECTS_C = $(OBJECTS:%.o=%.c)
-+OBJECTS_S = $(OBJECTS:%.o=%.s)
-+OBJECTS_SP = $(OBJECTS:%.o=%.sp)
-+
- dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)
- dep_dirs := $(addsuffix .depend,$(sort $(dir $(OBJECTS))))
-
@@ Makefile: missing_compdb_dir =
compdb_args =
endif
@@ Makefile: missing_compdb_dir =
-ASM_SRC := $(wildcard $(OBJECTS:o=S))
-ASM_OBJ := $(ASM_SRC:S=o)
-C_OBJ := $(filter-out $(ASM_OBJ),$(OBJECTS))
--
--$(C_OBJ): %.o: %.c GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir)
-- $(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(compdb_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<
--$(ASM_OBJ): %.o: %.S GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir)
-+$(OBJECTS): %.o: %.c GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir)
++C_OBJ = $(OBJECTS)
+
+ $(C_OBJ): %.o: %.c GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir)
$(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(compdb_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<
+-$(ASM_OBJ): %.o: %.S GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir)
+- $(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(compdb_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<
%.s: %.c GIT-CFLAGS FORCE
-@@ Makefile: XGETTEXT_FLAGS_SH = $(XGETTEXT_FLAGS) --language=Shell \
- --keyword=gettextln --keyword=eval_gettextln
- XGETTEXT_FLAGS_PERL = $(XGETTEXT_FLAGS) --language=Perl \
- --keyword=__ --keyword=N__ --keyword="__n:1,2"
--LOCALIZED_C = $(C_OBJ:o=c) $(LIB_H) $(GENERATED_H)
-+LOCALIZED_C = $(OBJECTS_C) $(LIB_H) $(GENERATED_H)
- LOCALIZED_SH = $(SCRIPT_SH)
- LOCALIZED_SH += git-sh-setup.sh
- LOCALIZED_PERL = $(SCRIPT_PERL)
-@@ Makefile: t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS) $(REFTABLE_TEST_LIB)
- check-sha1:: t/helper/test-tool$X
- t/helper/test-sha1.sh
-
--SP_OBJ = $(patsubst %.o,%.sp,$(C_OBJ))
--
--$(SP_OBJ): %.sp: %.c %.o
-+$(OBJECTS_SP): %.sp: %.c %.o
- $(QUIET_SP)cgcc -no-compile $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) \
- -Wsparse-error \
- $(SPARSE_FLAGS) $(SP_EXTRA_FLAGS) $< && \
- >$@
-
- .PHONY: sparse
--sparse: $(SP_OBJ)
-+sparse: $(OBJECTS_SP)
-
- EXCEPT_HDRS := $(GENERATED_H) unicode-width.h compat/% xdiff/%
- ifndef GCRYPT_SHA256
-@@ Makefile: clean: profile-clean coverage-clean cocciclean
- $(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) git$X
- $(RM) $(TEST_PROGRAMS)
- $(RM) $(FUZZ_PROGRAMS)
-- $(RM) $(SP_OBJ)
-+ $(RM) $(OBJECTS_SP)
- $(RM) $(HCC)
- $(RM) -r bin-wrappers $(dep_dirs) $(compdb_dir) compile_commands.json
- $(RM) -r po/build/
+ $(QUIET_CC)$(CC) -o $@ -S $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<
## configure.ac ##
@@ configure.ac: AC_MSG_NOTICE([CHECKS for site configuration])INSTALL | 3 +- Makefile | 17 +--- configure.ac | 3 - hash.h | 6 +- ppc/sha1.c | 72 ---------------- ppc/sha1.h | 25 ------ ppc/sha1ppc.S | 224 -------------------------------------------------- 7 files changed, 7 insertions(+), 343 deletions(-) delete mode 100644 ppc/sha1.c delete mode 100644 ppc/sha1.h delete mode 100644 ppc/sha1ppc.S
INSTALL +1 -2
@@ -135,8 +135,7 @@ Issues of note: By default, git uses OpenSSL for SHA1 but it will use its own library (inspired by Mozilla's) with either NO_OPENSSL or - BLK_SHA1. Also included is a version optimized for PowerPC - (PPC_SHA1). + BLK_SHA1. - "libcurl" library is used for fetching and pushing repositories over http:// or https://, as well as by
Makefile +4 -13
@@ -155,9 +155,6 @@ include shared.mak # Define BLK_SHA1 environment variable to make use of the bundled # optimized C SHA1 routine. # -# Define PPC_SHA1 environment variable when running make to make use of -# a bundled SHA1 routine optimized for PowerPC. -# # Define DC_SHA1 to unconditionally enable the collision-detecting sha1 # algorithm. This is slower, but may detect attempted collision attacks. # Takes priority over other *_SHA1 knobs.
@@ -1770,14 +1767,13 @@ ifdef OPENSSL_SHA1 EXTLIBS += $(LIB_4_CRYPTO) BASIC_CFLAGS += -DSHA1_OPENSSL else +ifdef PPC_SHA1 +$(error PPC_SHA1 has been removed! Use DC_SHA1 instead, which is the default) +endif ifdef BLK_SHA1 LIB_OBJS += block-sha1/sha1.o BASIC_CFLAGS += -DSHA1_BLK else -ifdef PPC_SHA1 - LIB_OBJS += ppc/sha1.o ppc/sha1ppc.o - BASIC_CFLAGS += -DSHA1_PPC -else ifdef APPLE_COMMON_CRYPTO COMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL BASIC_CFLAGS += -DSHA1_APPLE
@@ -1811,7 +1807,6 @@ endif endif endif endif -endif ifdef OPENSSL_SHA256 EXTLIBS += $(LIB_4_CRYPTO)
@@ -2540,14 +2535,10 @@ missing_compdb_dir = compdb_args = endif -ASM_SRC := $(wildcard $(OBJECTS:o=S)) -ASM_OBJ := $(ASM_SRC:S=o) -C_OBJ := $(filter-out $(ASM_OBJ),$(OBJECTS)) +C_OBJ = $(OBJECTS) $(C_OBJ): %.o: %.c GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir) $(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(compdb_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $< -$(ASM_OBJ): %.o: %.S GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir) - $(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(compdb_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $< %.s: %.c GIT-CFLAGS FORCE $(QUIET_CC)$(CC) -o $@ -S $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<
configure.ac +0 -3
@@ -237,9 +237,6 @@ AC_MSG_NOTICE([CHECKS for site configuration]) # tests. These tests take up a significant amount of the total test time # but are not needed unless you plan to talk to SVN repos. # -# Define PPC_SHA1 environment variable when running make to make use of -# a bundled SHA1 routine optimized for PowerPC. -# # Define NO_OPENSSL environment variable if you do not have OpenSSL. # # Define OPENSSLDIR=/foo/bar if your openssl header and library files are in
hash.h +2 -4
@@ -4,9 +4,7 @@#include "git-compat-util.h" #include "repository.h" -#if defined(SHA1_PPC ) -#include "ppc/sha1.h" -#elif defined(SHA1_APPLE ) +#if defined(SHA1_APPLE )#include <CommonCrypto/CommonDigest.h> #elif defined(SHA1_OPENSSL )#include <openssl/sha.h>
@@ -30,7 +28,7 @@* platform's underlying implementation of SHA-1; could be OpenSSL, * blk_SHA, Apple CommonCrypto, etc... Note that the relevant * SHA-1 header may have already defined platform_SHA_CTX for our -* own implementations like block-sha1 and ppc-sha1, so we list +* own implementations like block-sha1, so we list * the default for OpenSSL compatible SHA-1 implementations here. */ #define platform_SHA_CTX SHA_CTX
ppc/sha1.c deleted +0 -72
72 lines not on this page · raw message
ppc/sha1.h deleted +0 -25
@@ -1,25 +0,0 @@ -/* -* SHA-1 implementation. -* -* Copyright (C) 2005 Paul Mackerras <[email protected]> -*/ -#include <stdint.h> - -typedef struct { -uint32_t hash[5 ]; -uint32_t cnt; -uint64_t len; -union { -unsigned char b[64 ]; -uint64_t l[8 ]; - } buf; -}ppc_SHA_CTX ; - -int ppc_SHA1_Init (ppc_SHA_CTX *c); -int ppc_SHA1_Update (ppc_SHA_CTX *c,const void *p,unsigned long n); -int ppc_SHA1_Final (unsigned char *hash,ppc_SHA_CTX *c); - -#define platform_SHA_CTX ppc_SHA_CTX -#define platform_SHA1_Init ppc_SHA1_Init -#define platform_SHA1_Update ppc_SHA1_Update -#define platform_SHA1_Final ppc_SHA1_Final
ppc/sha1ppc.S deleted +0 -224
224 lines not on this page · raw message
Signature · 1 line
2.35.1.1441.g83331fcb493
Re: [PATCH v2] ppc: remove custom SHA-1 implementation
Quoted 29 lines · Ævar Arnfjörð Bjarmason
Remove the
PPC_SHA1implementation added ina6ef3518f9a([PATCH] PPC assembly implementation of SHA1, 2005-04-22). When this was added Apple consumer hardware used the PPC architecture, and the implementation was intended to improve SHA-1 speed there.Since it was added we've moved to
DC_SHA1by default, and anyone wanting hard-rolled non-DC SHA-1 implementation can use OpenSSL's via theOPENSSL_SHA1knob.I'm unsure if this was ever supposed to work on 64-bit PPC. It clearly originally targeted 32 bit PPC, but there's some mailing list references to this being tried on G5 (PPC 970). I can't get it to do anything but segfault on the BE POWER8 machine in the GCC compile farm. Anyone caring about speed on PPC these days is likely to be using IBM's POWER, not PPC 970.
There have been proposals to entirely remove non-DC_SHA1 implementations from the tree[1]. I think per [2] that would be a bit overzealous. I.e. there are various set-ups git's speed is going to be more important than the relatively implausible SHA-1 collision attack, or where such attacks are entirely mitigated by other means (e.g. by incoming objects being checked with
DC_SHA1).The main reason for doing so at this point is to simplify follow-up
Makefilechange. SincePPC_SHA1included the only in-tree *.S assembly file we needed to keep around special support for building objects from it. By getting rid of it we know we'll always build *.o from *.c files, which makes the build process simpler.
Quoted 4 lines
As an aside the code being removed here was also throwing warnings with the "-pedantic" flag, but let's remove it instead of fixing it, as
544d93bc3b4(block-sha1: remove use of obsolete x86 assembly, 2022-03-10) did for block-sha1/*.
While I don't agree that we shouldn't remove the other non-DC SHA-1 implementations, I do agree that we should remove this one. Given the testing you've done and the fact that almost everyone desiring speed is using a 64-bit machine these days, I think it's unlikely that anyone is using this in the real world.
Signature · 2 lines
brian m. carlson (he/him or they/them) Toronto, Ontario, CA
v3 cover Makefile + hash.h: remove PPC_SHA1 implementation
In reply to Ævar Arnfjörð BjarmasonThis is a re-roll of [1] sent back in March. It removes the PPC_SHA1 implementation, which as noted in in 1/2 is likely completely unused at this point, if anyone's using it we have other better tested (and just as fast) SHA-1 implementations.
I then included this change in a larger series sent in late April[2], which stalled for lack of feedback from the OSX crowd[3].
But this part should be uncontroversial, and is an obvious win in terms of the diffstat, and in reducing our complexity surface in this area.
Changes since v2:
* Much rewritten commit message (see range-diff)
* Rebased on upstream changes
* More PPC_SHA1 removal of comments/docs that I missed the first time around.
* The s/C_OBJS/OBJECTS/ Makefile change is now split into a new 2/2.
* Added brian's Acked-by, per [4] (not explicitly an "Acked-by", but I thought adding amounted to a fair paraphrasing of the feedback there)
1. https://lore.kernel.org/git/[email protected]/
2. https://lore.kernel.org/git/[email protected]/
3. https://lore.kernel.org/git/[email protected]/
4. https://lore.kernel.org/git/[email protected]/
Ævar Arnfjörð Bjarmason (2):
Makefile + hash.h: remove PPC_SHA1 implementation
Makefile: use $(OBJECTS) instead of $(C_OBJ)
INSTALL | 3 +- Makefile | 22 ++--- block-sha1/sha1.c | 4 - configure.ac | 3 - hash.h | 6 +- ppc/sha1.c | 72 --------------- ppc/sha1.h | 25 ------ ppc/sha1ppc.S | 224 ---------------------------------------------- 8 files changed, 9 insertions(+), 350 deletions(-) delete mode 100644 ppc/sha1.c delete mode 100644 ppc/sha1.h delete mode 100644 ppc/sha1ppc.S
Range-diff against v2:
1: e77fd23a824 ! 1: 87a204b8937 ppc: remove custom SHA-1 implementation
@@ Metadata
Author: Ævar Arnfjörð Bjarmason <[email protected]>
## Commit message ##
- ppc: remove custom SHA-1 implementation
+ Makefile + hash.h: remove PPC_SHA1 implementation
Remove the PPC_SHA1 implementation added in a6ef3518f9a ([PATCH] PPC assembly implementation of SHA1, 2005-04-22). When this was added Apple consumer hardware used the PPC architecture, and the implementation was intended to improve SHA-1 speed there.
- Since it was added we've moved to DC_SHA1 by default, and anyone
- wanting hard-rolled non-DC SHA-1 implementation can use OpenSSL's via
- the OPENSSL_SHA1 knob.
+ Since it was added we've moved to using sha1collisiondetection by
+ default, and anyone wanting hard-rolled non-DC SHA-1 implementation
+ can use OpenSSL's via the OPENSSL_SHA1 knob.
- I'm unsure if this was ever supposed to work on 64-bit PPC. It clearly
- originally targeted 32 bit PPC, but there's some mailing list
- references to this being tried on G5 (PPC 970). I can't get it to do
- anything but segfault on the BE POWER8 machine in the GCC compile
- farm. Anyone caring about speed on PPC these days is likely to be
- using IBM's POWER, not PPC 970.
+ The PPC_SHA1 originally originally targeted 32 bit PPC, and later the
+ 64 bit PPC 970 (a.k.a. Apple PowerPC G5). See 926172c5e48 (block-sha1:
+ improve code on large-register-set machines, 2009-08-10) for a
+ reference about the performance on G5 (a comment in block-sha1/sha1.c
+ being removed here).
- There have been proposals to entirely remove non-DC_SHA1
- implementations from the tree[1]. I think per [2] that would be a bit
+ I can't get it to do anything but segfault on both the BE and LE POWER
+ machines in the GCC compile farm[1]. Anyone who's concerned about
+ performance on PPC these days is likely to be using the IBM POWER
+ processors.
+
+ There have been proposals to entirely remove non-sha1collisiondetection
+ implementations from the tree[2]. I think per [3] that would be a bit
overzealous. I.e. there are various set-ups git's speed is going to be more important than the relatively implausible SHA-1 collision attack, or where such attacks are entirely mitigated by other means (e.g. by incoming objects being checked with DC_SHA1).
- The main reason for doing so at this point is to simplify follow-up
- Makefile change. Since PPC_SHA1 included the only in-tree *.S assembly
- file we needed to keep around special support for building objects
- from it. By getting rid of it we know we'll always build *.o from *.c
- files, which makes the build process simpler.
+ But that really doesn't apply to PPC_SHA1 in particular, which seems
+ to have outlived its usefulness.
+
+ As this gets rid of the only in-tree *.S assembly file we can remove
+ the small bits of logic from the Makefile needed to build objects
+ from *.S (as opposed to *.c)
- As an aside the code being removed here was also throwing warnings
- with the "-pedantic" flag, but let's remove it instead of fixing it,
- as 544d93bc3b4 (block-sha1: remove use of obsolete x86 assembly,
- 2022-03-10) did for block-sha1/*.
+ The code being removed here was also throwing warnings with the
+ "-pedantic" flag, it could have been fixed as 544d93bc3b4 (block-sha1:
+ remove use of obsolete x86 assembly, 2022-03-10) did for block-sha1/*,
+ but as noted above let's remove it instead.
- 1. https://lore.kernel.org/git/[email protected]/
- 2. https://lore.kernel.org/git/[email protected]/
+ 1. https://cfarm.tetaneutral.net/machines/list/
+ Tested on gcc{110,112,135,203}, a mixture of POWER [789] ppc64 and
+ ppc64le. All segfault in anything needing object
+ hashing (e.g. t/t1007-hash-object.sh) when compiled with
+ PPC_SHA1=Y.
+ 2. https://lore.kernel.org/git/[email protected]/
+ 3. https://lore.kernel.org/git/[email protected]/
+ Acked-by: brian m. carlson" <[email protected]>
## INSTALL ##
@@ Makefile: include shared.mak
# Define DC_SHA1 to unconditionally enable the collision-detecting sha1
# algorithm. This is slower, but may detect attempted collision attacks.
# Takes priority over other *_SHA1 knobs.
-@@ Makefile: ifdef OPENSSL_SHA1
- EXTLIBS += $(LIB_4_CRYPTO)
- BASIC_CFLAGS += -DSHA1_OPENSSL
- else
+@@ Makefile: ifdef APPLE_COMMON_CRYPTO
+ SHA1_MAX_BLOCK_SIZE = 1024L*1024L*1024L
+ endif
+
+ifdef PPC_SHA1
-+$(error PPC_SHA1 has been removed! Use DC_SHA1 instead, which is the default) ++$(error the PPC_SHA1 flag has been removed along with the PowerPC-specific SHA-1 implementation.)
+endif
- ifdef BLK_SHA1
++
+ ifdef OPENSSL_SHA1
+ EXTLIBS += $(LIB_4_CRYPTO)
+ BASIC_CFLAGS += -DSHA1_OPENSSL
+@@ Makefile: ifdef BLK_SHA1
LIB_OBJS += block-sha1/sha1.o
BASIC_CFLAGS += -DSHA1_BLK
else
@@ Makefile: missing_compdb_dir =
-ASM_SRC := $(wildcard $(OBJECTS:o=S))
-ASM_OBJ := $(ASM_SRC:S=o)
-C_OBJ := $(filter-out $(ASM_OBJ),$(OBJECTS))
-+C_OBJ = $(OBJECTS)
++C_OBJ := $(OBJECTS)
$(C_OBJ): %.o: %.c GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir)
$(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(compdb_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<
@@ Makefile: missing_compdb_dir =
%.s: %.c GIT-CFLAGS FORCE
$(QUIET_CC)$(CC) -o $@ -S $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<
+ ## block-sha1/sha1.c ##
+@@
+ * try to do the silly "optimize away loads" part because it won't
+ * see what the value will be).
+ *
+- * Ben Herrenschmidt reports that on PPC, the C version comes close +- * to the optimized asm with this (ie on PPC you don't want that +- * 'volatile', since there are lots of registers).
+- *
+ * On ARM we get the best code generation by forcing a full memory barrier
+ * between each SHA_ROUND, otherwise gcc happily get wild with spilling and
+ * the stack frame size simply explode and performance goes down the drain. +
## configure.ac ##
@@ configure.ac: AC_MSG_NOTICE([CHECKS for site configuration])
# tests. These tests take up a significant amount of the total test time
-: ----------- > 2: cb3bc8b5029 Makefile: use $(OBJECTS) instead of $(C_OBJ)
Signature · 1 line
2.37.3.1406.g184357183a6
PATCH v3 1/2 Makefile + hash.h: remove PPC_SHA1 implementation
Remove the PPC_SHA1 implementation added in a6ef3518f9a ([PATCH] PPC assembly implementation of SHA1, 2005-04-22). When this was added Apple consumer hardware used the PPC architecture, and the implementation was intended to improve SHA-1 speed there.
Since it was added we've moved to using sha1collisiondetection by default, and anyone wanting hard-rolled non-DC SHA-1 implementation can use OpenSSL's via the OPENSSL_SHA1 knob.
The PPC_SHA1 originally originally targeted 32 bit PPC, and later the 64 bit PPC 970 (a.k.a. Apple PowerPC G5). See 926172c5e48 (block-sha1:
improve code on large-register-set machines, 2009-08-10) for a reference about the performance on G5 (a comment in block-sha1/sha1.c being removed here).
I can't get it to do anything but segfault on both the BE and LE POWER machines in the GCC compile farm[1]. Anyone who's concerned about performance on PPC these days is likely to be using the IBM POWER processors.
There have been proposals to entirely remove non-sha1collisiondetection implementations from the tree[2]. I think per [3] that would be a bit overzealous. I.e. there are various set-ups git's speed is going to be more important than the relatively implausible SHA-1 collision attack, or where such attacks are entirely mitigated by other means (e.g. by incoming objects being checked with DC_SHA1).
But that really doesn't apply to PPC_SHA1 in particular, which seems to have outlived its usefulness.
As this gets rid of the only in-tree *.S assembly file we can remove the small bits of logic from the Makefile needed to build objects from *.S (as opposed to *.c)
The code being removed here was also throwing warnings with the "-pedantic" flag, it could have been fixed as 544d93bc3b4 (block-sha1:
remove use of obsolete x86 assembly, 2022-03-10) did for block-sha1/*, but as noted above let's remove it instead.
1. https://cfarm.tetaneutral.net/machines/list/
Tested on gcc{110,112,135,203}, a mixture of POWER [789] ppc64 and ppc64le. All segfault in anything needing object
hashing (e.g. t/t1007-hash-object.sh) when compiled with PPC_SHA1=Y.
2. https://lore.kernel.org/git/[email protected]/
3. https://lore.kernel.org/git/[email protected]/
--- INSTALL | 3 +- Makefile | 18 ++-- block-sha1/sha1.c | 4 - configure.ac | 3 - hash.h | 6 +- ppc/sha1.c | 72 --------------- ppc/sha1.h | 25 ------ ppc/sha1ppc.S | 224 ---------------------------------------------- 8 files changed, 8 insertions(+), 347 deletions(-) delete mode 100644 ppc/sha1.c delete mode 100644 ppc/sha1.h delete mode 100644 ppc/sha1ppc.S
INSTALL +1 -2
@@ -135,8 +135,7 @@ Issues of note: By default, git uses OpenSSL for SHA1 but it will use its own library (inspired by Mozilla's) with either NO_OPENSSL or - BLK_SHA1. Also included is a version optimized for PowerPC - (PPC_SHA1). + BLK_SHA1. - "libcurl" library is used for fetching and pushing repositories over http:// or https://, as well as by
Makefile +5 -13
@@ -155,9 +155,6 @@ include shared.mak # Define BLK_SHA1 environment variable to make use of the bundled # optimized C SHA1 routine. # -# Define PPC_SHA1 environment variable when running make to make use of -# a bundled SHA1 routine optimized for PowerPC. -# # Define DC_SHA1 to unconditionally enable the collision-detecting sha1 # algorithm. This is slower, but may detect attempted collision attacks. # Takes priority over other *_SHA1 knobs.
@@ -1802,6 +1799,10 @@ ifdef APPLE_COMMON_CRYPTO SHA1_MAX_BLOCK_SIZE = 1024L*1024L*1024L endif +ifdef PPC_SHA1 +$(error the PPC_SHA1 flag has been removed along with the PowerPC-specific SHA-1 implementation.) +endif + ifdef OPENSSL_SHA1 EXTLIBS += $(LIB_4_CRYPTO) BASIC_CFLAGS += -DSHA1_OPENSSL
@@ -1810,10 +1811,6 @@ ifdef BLK_SHA1 LIB_OBJS += block-sha1/sha1.o BASIC_CFLAGS += -DSHA1_BLK else -ifdef PPC_SHA1 - LIB_OBJS += ppc/sha1.o ppc/sha1ppc.o - BASIC_CFLAGS += -DSHA1_PPC -else ifdef APPLE_COMMON_CRYPTO COMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL BASIC_CFLAGS += -DSHA1_APPLE
@@ -1847,7 +1844,6 @@ endif endif endif endif -endif ifdef OPENSSL_SHA256 EXTLIBS += $(LIB_4_CRYPTO)
@@ -2594,14 +2590,10 @@ missing_compdb_dir = compdb_args = endif -ASM_SRC := $(wildcard $(OBJECTS:o=S)) -ASM_OBJ := $(ASM_SRC:S=o) -C_OBJ := $(filter-out $(ASM_OBJ),$(OBJECTS)) +C_OBJ := $(OBJECTS) $(C_OBJ): %.o: %.c GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir) $(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(compdb_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $< -$(ASM_OBJ): %.o: %.S GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir) - $(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(compdb_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $< %.s: %.c GIT-CFLAGS FORCE $(QUIET_CC)$(CC) -o $@ -S $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<
block-sha1/sha1.c +0 -4
@@ -28,10 +28,6 @@* try to do the silly "optimize away loads" part because it won't * see what the value will be). * -* Ben Herrenschmidt reports that on PPC, the C version comes close -* to the optimized asm with this (ie on PPC you don't want that -* 'volatile', since there are lots of registers). -* * On ARM we get the best code generation by forcing a full memory barrier * between each SHA_ROUND, otherwise gcc happily get wild with spilling and * the stack frame size simply explode and performance goes down the drain.
configure.ac +0 -3
@@ -237,9 +237,6 @@ AC_MSG_NOTICE([CHECKS for site configuration]) # tests. These tests take up a significant amount of the total test time # but are not needed unless you plan to talk to SVN repos. # -# Define PPC_SHA1 environment variable when running make to make use of -# a bundled SHA1 routine optimized for PowerPC. -# # Define NO_OPENSSL environment variable if you do not have OpenSSL. # # Define OPENSSLDIR=/foo/bar if your openssl header and library files are in
hash.h +2 -4
@@ -4,9 +4,7 @@#include "git-compat-util.h" #include "repository.h" -#if defined(SHA1_PPC ) -#include "ppc/sha1.h" -#elif defined(SHA1_APPLE ) +#if defined(SHA1_APPLE )#include <CommonCrypto/CommonDigest.h> #elif defined(SHA1_OPENSSL )#include <openssl/sha.h>
@@ -32,7 +30,7 @@* platform's underlying implementation of SHA-1; could be OpenSSL, * blk_SHA, Apple CommonCrypto, etc... Note that the relevant * SHA-1 header may have already defined platform_SHA_CTX for our -* own implementations like block-sha1 and ppc-sha1, so we list +* own implementations like block-sha1, so we list * the default for OpenSSL compatible SHA-1 implementations here. */ #define platform_SHA_CTX SHA_CTX
ppc/sha1.c deleted +0 -72
72 lines not on this page · raw message
ppc/sha1.h deleted +0 -25
@@ -1,25 +0,0 @@ -/* -* SHA-1 implementation. -* -* Copyright (C) 2005 Paul Mackerras <[email protected]> -*/ -#include <stdint.h> - -typedef struct { -uint32_t hash[5 ]; -uint32_t cnt; -uint64_t len; -union { -unsigned char b[64 ]; -uint64_t l[8 ]; - } buf; -}ppc_SHA_CTX ; - -int ppc_SHA1_Init (ppc_SHA_CTX *c); -int ppc_SHA1_Update (ppc_SHA_CTX *c,const void *p,unsigned long n); -int ppc_SHA1_Final (unsigned char *hash,ppc_SHA_CTX *c); - -#define platform_SHA_CTX ppc_SHA_CTX -#define platform_SHA1_Init ppc_SHA1_Init -#define platform_SHA1_Update ppc_SHA1_Update -#define platform_SHA1_Final ppc_SHA1_Final
ppc/sha1ppc.S deleted +0 -224
224 lines not on this page · raw message
Signature · 1 line
2.37.3.1406.g184357183a6
PATCH v3 2/2 Makefile: use $(OBJECTS) instead of $(C_OBJ)
In reply to Ævar Arnfjörð BjarmasonIn the preceding commit $(C_OBJ) added in c373991375a (Makefile: list generated object files in OBJECTS, 2010-01-26) became synonymous with $(OBJECTS). Let's avoid the indirection and use the $(OBJECTS) variable directly instead.
--- Makefile | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
Makefile +2 -4
@@ -2590,9 +2590,7 @@ missing_compdb_dir = compdb_args = endif -C_OBJ := $(OBJECTS) - -$(C_OBJ): %.o: %.c GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir) +$(OBJECTS): %.o: %.c GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir) $(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(compdb_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $< %.s: %.c GIT-CFLAGS FORCE
@@ -3084,7 +3082,7 @@ t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS) $(REFTABLE_TEST_LIB) check-sha1:: t/helper/test-tool$X t/helper/test-sha1.sh -SP_OBJ = $(patsubst %.o,%.sp,$(C_OBJ)) +SP_OBJ = $(patsubst %.o,%.sp,$(OBJECTS)) $(SP_OBJ): %.sp: %.c %.o $(QUIET_SP)cgcc -no-compile $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) \
Signature · 1 line
2.37.3.1406.g184357183a6
Re: [PATCH v3 2/2] Makefile: use $(OBJECTS) instead of $(C_OBJ)
Quoted 9 lines · Ævar Arnfjörð Bjarmason
In the preceding commit $(C_OBJ) added in
c373991375a(Makefile: list generated object files in OBJECTS, 2010-01-26) became synonymous with $(OBJECTS). Let's avoid the indirection and use the $(OBJECTS) variable directly instead.Signed-off-by Ævar Arnfjörð Bjarmason <[email protected]>--- Makefile | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
This is a declaration that we would never ever build .o files out of sources other than .c files. While it does make sense to have it outside the scope of [PATCH 1/2], I am not sure if it even belongs to the same series.
Re: [PATCH v3 2/2] Makefile: use $(OBJECTS) instead of $(C_OBJ)
Quoted 12 lines · Junio C Hamano
Quoted 9 lines · Ævar Arnfjörð Bjarmason
In the preceding commit $(C_OBJ) added in
c373991375a(Makefile: list generated object files in OBJECTS, 2010-01-26) became synonymous with $(OBJECTS). Let's avoid the indirection and use the $(OBJECTS) variable directly instead.Signed-off-by Ævar Arnfjörð Bjarmason <[email protected]>--- Makefile | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
Quoted 4 lines
This is a declaration that we would never ever build .o files out of sources other than .c files. While it does make sense to have it outside the scope of [PATCH 1/2], I am not sure if it even belongs to the same series.
I think it does. Before this the C_OBJ would be:
C_OBJ := $(filter-out $(ASM_OBJ),$(OBJECTS))
but after 1/2 it's the same as $(OBJECTS). An earlier iteration of this did this cleanup "while we're at it" (which I do think makes sense as an atomic change), but I got the feedback that the cleanup wasn't strictly necessary.
But as 1/2 has removed the ability to build those $(ASM_OBJ), as we had only one of those, I don't think keeping this particular bit of indirection makes sense.
Of course it doesn't really matter at all, the real change is the removal of $(ASM_OBJ).
If we do start building *.o files out of *.S files (or other non-*.c) again we'll need new rules anyway. I think we should just add any such variables back then, and not keep this small bit of dead husk around.
Re: [PATCH v3 2/2] Makefile: use $(OBJECTS) instead of $(C_OBJ)
Quoted 9 lines · Ævar Arnfjörð Bjarmason
Quoted 4 lines
This is a declaration that we would never ever build .o files out of sources other than .c files. While it does make sense to have it outside the scope of [PATCH 1/2], I am not sure if it even belongs to the same series.
I think it does. Before this the
C_OBJwould be:C_OBJ := $(filter-out $(ASM_OBJ),$(OBJECTS))
Quoted 8 lines
but after 1/2 it's the same as $(OBJECTS). An earlier iteration of this did this cleanup "while we're at it" (which I do think makes sense as an atomic change), but I got the feedback that the cleanup wasn't strictly necessary.
But as 1/2 has removed the ability to build those $(ASM_OBJ), as we had only one of those, I don't think keeping this particular bit of indirection makes sense.
You are not thinking for longer term to help project maintenance.
This change removes distinction between C_OBJ and OBJECTS, only because the sources to the objects we HAPPEN TO have are only C files. It is premature and short sighted to declare that it has to stay that way forever. And such a declaration is not something we would casually make "while at it" in a topic like this.
When we add a source written in another language, say xyzzy, to be compiled into an object file, we'd add $(XYZZY_OBJ), and they will become part of $(OBJECTS), but the current rule to create $(C_OBJ) will not apply to $(XYZZY_OBJ). But you do this:
-$(C_OBJ): %.o: %.c GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir)
+$(OBJECTS): %.o: %.c GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir)
$(QUIET_CC)$(CC) -o $*.o -c ... $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<Right now, we know where this patch affected the build procedure, because the patch highlights what is being changed. But when future developers need to produce some files that belong to $(OBJECTS) out of source files that are not .c, they first need to locate the above hunk and revert it. I do not see the benefit of being hostile to future developers with this patch. Not before we know that it is not likely that we would add any non-C sources in the future, by running with 1/2 alone for a year or two.