git broken for AIX somewhere between 2.13.2 and 2.13.3
Hi,
Several years ago I had downloaded and packaged git-2.10.1 with no real issues, and it has been working fine. Deciding it was time for an update I downloaded git-2.18.0 and built from scratch.
After testing lots of version - the the last 2.12 version worked; git-2.13.2 works but git-2.13.3 does not.
root@x066:[/tmp/xxx]git clone [email protected]:aixtools/hello-world.git root@x066:[/tmp/xxx]git --version
git version 2.13.2
root@x066:[/tmp/xxx]ls -l
total 0
drwxr-xr-x 3 root system 256 Jul 29 16:37 hello-world
root@x066:[/tmp/xxx]git --version
git version 2.13.3
root@x066:[/tmp/xxx]git clone [email protected]:aixtools/hello-world.git Cloning into 'hello-world'...
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 3 Receiving objects: 100% (3/3), done.
fatal: pack is corrupted (SHA1 mismatch)
fatal: index-pack failed
p.s. - what surprises me re: git-2.13.2 - no messages about 'Cloning into ...', which version 2.13.1 did give.
I guess a bisect is the next step - between version 2.13.2 and 2.13.3. Other suggestions welcome!
Michael
Quoted 10 lines · Michael
root@x066:[/tmp/xxx]git --version
git version 2.13.3
root@x066:[/tmp/xxx]git clone [email protected]:aixtools/hello-world.git Cloning into 'hello-world'...
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 3 Receiving objects: 100% (3/3), done.
fatal: pack is corrupted (SHA1 mismatch)
fatal: index-pack failed
Quoted 5 lines
p.s. - what surprises me re: git-2.13.2 - no messages about 'Cloning into ...', which version 2.13.1 did give.
I guess a bisect is the next step - between version 2.13.2 and 2.13.3. Other suggestions welcome!
Are you using SHA1DC on that system, and does compiling with another SHA-1 implementation help? There was a change to the SHA1DC code big endian detection in that commit, which might be the cause of your problems if you're using a POWER or PowerPC system.
Signature · 2 lines
brian m. carlson: Houston, Texas, US OpenPGP: https://keybase.io/bk2204
Quoted 14 lines · brian m. carlson
Quoted 13 lines · Michael
root@x066:[/tmp/xxx]git --version
git version 2.13.3
root@x066:[/tmp/xxx]git clone [email protected]:aixtools/hello-world.git Cloning into 'hello-world'...
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 3 Receiving objects: 100% (3/3), done.
fatal: pack is corrupted (SHA1 mismatch)
fatal: index-pack failedp.s. - what surprises me re: git-2.13.2 - no messages about 'Cloning into ...', which version 2.13.1 did give.
Quoted 6 lines
Quoted 2 lines
I guess a bisect is the next step - between version 2.13.2 and 2.13.3. Other suggestions welcome!
Are you using SHA1DC on that system, and does compiling with another SHA-1 implementation help? There was a change to the SHA1DC code big endian detection in that commit, which might be the cause of your problems if you're using a POWER or PowerPC system.
I was thinking it might be an 'endian' issue. So, yes - AIX runs on POWER, only as BigEndian.
git bisect returns:
michael@x071:[/data/prj/aixtools/git/github/git-master]git bisect bad
Bisecting: 1 revision left to test after this (roughly 1 step) [35049a2343948f686861e176a8c395f9f67da7b6] Merge branch 'aw/contrib-subtree-doc-asciidoctor' into maint michael@x071:[/data/prj/aixtools/git/github/git-master]git bisect good
Bisecting: 0 revisions left to test after this (roughly 0 steps) [9936c1b52a39fa14fca04f937df3e75f7498ac66] sha1dc: update from upstream
michael@x071:[/data/prj/aixtools/git/github/git-master]git bisect bad 9936c1b52a39fa14fca04f937df3e75f7498ac66 is the first bad commit commit 9936c1b52a39fa14fca04f937df3e75f7498ac66
Author: Ævar Arnfjörð Bjarmason <[email protected]>
Date: Sat Jul 1 22:05:45 2017 +0000
sha1dc: update from upstream
Update sha1dc from the latest version by the upstream maintainer[1].
See commit 6b851e536b ("sha1dc: update from upstream", 2017-06-06) for the last update.
This solves the Big Endian detection on Solaris reported against v2.13.2[2], hopefully without any regressions. A version of this has been tested on two Solaris SPARC installations, Cygwin (by jturney on cygwin@Freenode), and on numerous more boring systems (mainly linux/x86_64). See [3] for a discussion of the implementation and platform-specific issues.
See commit a0103914c2 ("sha1dc: update from upstream", 2017-05-20) and 6b851e536b ("sha1dc: update from upstream", 2017-06-06) for previous attempts in the 2.13 series to address various compile-time feature detection in this library.
2. <CAKKM46tHq13XiW5C8sux3=PZ1VHSu_npG8ExfWwcPD7rkZkyRQ@mail.gmail.com>
(https://public-inbox.org/git/CAKKM46tHq13XiW5C8sux3=PZ1VHSu_npG8ExfWwcPD7rkZkyRQ@mail.gmail.com/)
3. https://github.com/cr-marcstevens/sha1collisiondetection/pull/34
:040000 040000 a84797967fb742e4ca9618a641d53ce3a6c6589b 32efa656d78901da961e4a47d84b6d82fede064b M sha1dc
Quoted 68 lines · Michael
Quoted 14 lines · brian m. carlson
Quoted 13 lines · Michael
root@x066:[/tmp/xxx]git --version
git version 2.13.3
root@x066:[/tmp/xxx]git clone [email protected]:aixtools/hello-world.git Cloning into 'hello-world'...
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 3 Receiving objects: 100% (3/3), done.
fatal: pack is corrupted (SHA1 mismatch)
fatal: index-pack failedp.s. - what surprises me re: git-2.13.2 - no messages about 'Cloning into ...', which version 2.13.1 did give.
Quoted 6 lines
Quoted 2 lines
I guess a bisect is the next step - between version 2.13.2 and 2.13.3. Other suggestions welcome!
Are you using SHA1DC on that system, and does compiling with another SHA-1 implementation help? There was a change to the SHA1DC code big endian detection in that commit, which might be the cause of your problems if you're using a POWER or PowerPC system.
I was thinking it might be an 'endian' issue. So, yes - AIX runs on POWER, only as BigEndian.
git bisect returns:
michael@x071:[/data/prj/aixtools/git/github/git-master]git bisect bad
Bisecting: 1 revision left to test after this (roughly 1 step) [35049a2343948f686861e176a8c395f9f67da7b6] Merge branch 'aw/contrib-subtree-doc-asciidoctor' into maint michael@x071:[/data/prj/aixtools/git/github/git-master]git bisect good
Bisecting: 0 revisions left to test after this (roughly 0 steps) [9936c1b52a39fa14fca04f937df3e75f7498ac66] sha1dc: update from upstreammichael@x071:[/data/prj/aixtools/git/github/git-master]git bisect bad
9936c1b52a39fa14fca04f937df3e75f7498ac66is the first bad commit commit9936c1b52a39fa14fca04f937df3e75f7498ac66
Author: Ævar Arnfjörð Bjarmason <[email protected]>
Date: Sat Jul 1 22:05:45 2017 +0000sha1dc: update from upstream
Update sha1dc from the latest version by the upstream maintainer[1].
See commit
6b851e536b("sha1dc: update from upstream", 2017-06-06) for the last update.This solves the Big Endian detection on Solaris reported against v2.13.2[2], hopefully without any regressions. A version of this has been tested on two Solaris SPARC installations, Cygwin (by jturney on cygwin@Freenode), and on numerous more boring systems (mainly linux/x86_64). See [3] for a discussion of the implementation and platform-specific issues.
See commit
a0103914c2("sha1dc: update from upstream", 2017-05-20) and6b851e536b("sha1dc: update from upstream", 2017-06-06) for previous attempts in the 2.13 series to address various compile-time feature detection in this library.2. <CAKKM46tHq13XiW5C8sux3=PZ1VHSu_npG8ExfWwcPD7rkZkyRQ@mail.gmail.com>
(https://public-inbox.org/git/CAKKM46tHq13XiW5C8sux3=PZ1VHSu_npG8ExfWwcPD7rkZkyRQ@mail.gmail.com/)
Quoted 7 lines
3. https://github.com/cr-marcstevens/sha1collisiondetection/pull/34
Signed-off-by Ævar Arnfjörð Bjarmason <[email protected]>Signed-off-by Junio C Hamano <[email protected]>:040000 040000
a84797967fb742e4ca9618a641d53ce3a6c6589b32efa656d78901da961e4a47d84b6d82fede064bM sha1dc
Sorry about that. As can be seen from those PRs and the "git log" detecting whether something is big endian or not can be quite tricky, we figured out how to do it on both BSD and Solaris, but apparently broke AIX as a result.
You should be able to define -DSHA1DC_FORCE_LITTLEENDIAN or -DSHA1DC_FORCE_BIGENDIAN (looks like you'll need the latter) to get the latest version to compile, but I and upstream cr-marcstevens would be very interested to know from someone who knows AIX how this broke.
Also, to you and anyone else with access to AIX: I'd be happy to figure these issues out pro-actively if you give me a login to an AIX machine. I promise not to do anything except compile/debug/test git on it.
I used to have access to an AIX box through a previous job ages ago, it's a very interesting OS and like with Solaris it's easy to discover a lot of portability issues.
Quoted 4 lines · Ævar Arnfjörð Bjarmason
Also, to you and anyone else with access to AIX: I'd be happy to figure these issues out pro-actively if you give me a login to an AIX machine. I promise not to do anything except compile/debug/test git on it.
The GCC compile farm <http://gcc.gnu.org/wiki/CompileFarm> has a machine running AIX, and is free to use for anyone working on free software.
Andreas.
Signature · 3 lines
Andreas Schwab, [email protected] GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Quoted 8 lines · Andreas Schwab
Quoted 4 lines · Ævar Arnfjörð Bjarmason
Also, to you and anyone else with access to AIX: I'd be happy to figure these issues out pro-actively if you give me a login to an AIX machine. I promise not to do anything except compile/debug/test git on it.
The GCC compile farm <http://gcc.gnu.org/wiki/CompileFarm> has a machine running AIX, and is free to use for anyone working on free software.
My goal is less "to work on", more, "to package" and/or "to work with". Most others, including IBM, seem to use gcc as compiler - which is fine. However, on AIX I often see side-effects introduced by the GNU run-time environment needed on top of the xlc.rte provided as part of AIX.
In any case - my testing is using the xlc complier - so there are syntax differences. the compiler has many modes - by default I use 'xlc_r' that has the following default settings:
xlc_r: use = DEFLT_C
crt = /lib/crt0.o
mcrt = /lib/mcrt0.o
gcrt = /lib/gcrt0.o
libraries = -L/usr/vac/lib,-lxlopt,-lxlipa,-lxl,-lpthreads,-lc
proflibs = -L/lib/profiled,-L/usr/lib/profiled
hdlibs = -L/usr/vac/lib,-lhmd
options = -qlanglvl=extc99,-qcpluscmt,-qkeyword=inline,-qalias=ansi,-qthreaded,-D_THREAD_SAFE,-D__VACPP_MULTI__
DEFLT_C (for the curious, the default options is perhaps most interesting) is:
* common definitions
DEFLT_C:
use =DEFLT
xlurt_cfg_path=/usr/vac/urt
xlurt_cfg_name=urt_client.cfg
DEFLT_CPP:
use =DEFLT
xlurt_cfg_path=/usr/vacpp/urt
xlurt_cfg_name=urt_client.cfg
DEFLT:
cppcomp = /usr/vacpp/exe/xlCentry
ccomp = /usr/vac/exe/xlcentry
code = /usr/vac/exe/xlCcode
cpp = /usr/vac/exe/xlCcpp
munch = /usr/vacpp/exe/munch
dis = /usr/vac/exe/dis
xlC = /usr/vac/bin/xlc
list = /usr/vac/exe/xllist
xslt = /usr/vac/exe/XALAN
transforms = /usr/vac/listings
listlibs = /usr/vac/lib
cppinc = /usr/vacpp/include
ipa = /usr/vac/exe/ipa
cppfilt = /usr/vacpp/bin/c++filt
bolt = /usr/vac/exe/bolt
as = /bin/as
ld = /bin/ld
artool = /bin/ar
options = -D_AIX,-D_AIX32,-D_AIX41,-D_AIX43,-D_AIX50,-D_AIX51,-D_AIX52,-D_AIX53,-D_AIX61,-D_IBMR2,-D_POWER options32 = -bpT:0x10000000,-bpD:0x20000000
options32_bmaxdata = -bpT:0x10000000,-bpD:0x30000000
options64 = -bpT:0x100000000,-bpD:0x110000000
ldopt = "b:o:e:u:R:H:Y:Z:L:T:A:k:j:"
hdlibs = -L/usr/vac/lib,-lhmd
xlCcopt = -qlanglvl=extc99,-qcpluscmt,-qkeyword=inline,-qalias=ansi
crt_64 = /lib/crt0_64.o
mcrt_64 = /lib/mcrt0_64.o
gcrt_64 = /lib/gcrt0_64.o
smplibraries = -lxlsmp
palibraries = -L/usr/vatools/lib,-lpahooks
resexp = /usr/vacpp/lib/res.exp
genexports = /usr/vac/bin/CreateExportList
vac_path = /usr/vac
vacpp_path = /usr/vacpp
xlcmp_path = /usr/vac:/usr/vacpp
xlc_c_stdinc = /usr/vac/include:/usr/include
xlc_cpp_stdinc = /usr/vacpp/include:/usr/include
xlurt_msg_cat_name=vacumsg.cat
__GNUC_MINOR__ = 3
Quoted 3 lines
Andreas.
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 11 lines · brian m. carlson
Quoted 7 lines · Michael
Quoted 4 lines · brian m. carlson
Are you using SHA1DC on that system, and does compiling with another SHA-1 implementation help? There was a change to the SHA1DC code big endian detection in that commit, which might be the cause of your problems if you're using a POWER or PowerPC system.
I was thinking it might be an 'endian' issue. So, yes - AIX runs on POWER, only as BigEndian.
Well, that explains it. I would recommend submitting a patch to
https://github.com/cr-marcstevens/sha1collisiondetection, and the we can pull in the updated submodule with that fix.
Not sure I am smart enough to do that. I'll have to download, build, and see what it says.
Quoted 5 lines
In the mean time, you could build using OpenSSL or the block SHA-1 implementation, and switch back once things are in a good state. I do recommend using SHA1DC for things long term, though, as attacks on SHA-1 are only going to get better.
Any suggestions on where/how to do this?
root@x066:[/data/prj/aixtools/git/git-2.13.2]./configure --help | grep -i sha --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com] --datarootdir=DIR read-only arch.-independent data root [PREFIX/share]
root@x066:[/data/prj/aixtools/git/git-2.13.2]./configure --help | grep ssl --with-openssl use OpenSSL library (default is YES) ARG can be prefix for openssl library and headers
Quoted 6 lines · Michael
Quoted 3 lines · brian m. carlson
Well, that explains it. I would recommend submitting a patch to
https://github.com/cr-marcstevens/sha1collisiondetection, and the we can pull in the updated submodule with that fix.Not sure I am smart enough to do that. I'll have to download, build, and see what it says.
The issue is that somewhere in lib/sha1.c, you need to cause SHA1DC_BIGENDIAN to be set. That means you need to figure out what compiler macro might indicate that. I can tell you that a POWER- or PowerPC-specific one is going to be a bad choice unless it includes the endianness, since those chips come in little-endian versions as well.
_AIX might be a fine choice if you know that it only ever runs on big-endian chips.
Quoted 13 lines
Quoted 4 lines
In the mean time, you could build using OpenSSL or the block SHA-1 implementation, and switch back once things are in a good state. I do recommend using SHA1DC for things long term, though, as attacks on SHA-1 are only going to get better.
Any suggestions on where/how to do this?
root@x066:[/data/prj/aixtools/git/git-2.13.2]./configure --help | grep -i sha
--sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com]
--datarootdir=DIR read-only arch.-independent data root [PREFIX/share]
Quoted 3 lines
root@x066:[/data/prj/aixtools/git/git-2.13.2]./configure --help | grep ssl --with-openssl use OpenSSL library (default is YES) ARG can be prefix for openssl library and headers
If you're using configure, you can use --with-openssl, or --with-openssl=PREFIX if your OpenSSL isn't in the standard location but is instead in PREFIX.
Signature · 2 lines
brian m. carlson: Houston, Texas, US OpenPGP: https://keybase.io/bk2204
Quoted 10 lines · brian m. carlson
Quoted 6 lines · Michael
Quoted 3 lines · brian m. carlson
Well, that explains it. I would recommend submitting a patch to
https://github.com/cr-marcstevens/sha1collisiondetection, and the we can pull in the updated submodule with that fix.Not sure I am smart enough to do that. I'll have to download, build, and see what it says.
The issue is that somewhere in
lib/sha1.c, you need to causeSHA1DC_BIGENDIANto be set. That means you need to figure out what compiler macro might indicate that.
I remember - roughly - a few decades back - having an assignment to write code to determine endianness. PDP and VAC were different iirc, and many other micro-processors besides the 8088/8086/z85/68k/etc..
If you are looking for a compiler macro as a way to determine this - maybe you have one for gcc, but not for xlc. I do not know it - currently :)
Quoted 3 lines
I can tell you that a POWER- or
PowerPC-specific one is going to be a bad choice unless it includes the endianness, since those chips come in little-endian versions as well.
Actually, the POWER8 and POWER9 (and I expect all future versions) support both. There are not separate chips. Per virtual machine - a mode is determined during boot (virtual power on) e.g., SLES11 only ran in BigEndian and SLES12 only runs in LittleEndian. afaik, RHEL was supplying both BE and LE distributions. AIX, as an OS, only runs in BE mode, and I expect IBM i (was os/400) is also only running in BE.
Quoted 3 lines
_AIXmight be a fine choice if you know that it only ever runs on big-endian chips.
Do you mean just testing for _AIX. That would be very very easy - yes!
Quoted 13 lines
Quoted 13 lines
Quoted 4 lines
In the mean time, you could build using OpenSSL or the block SHA-1 implementation, and switch back once things are in a good state. I do recommend using SHA1DC for things long term, though, as attacks on SHA-1 are only going to get better.
Any suggestions on where/how to do this?
root@x066:[/data/prj/aixtools/git/git-2.13.2]./configure --help | grep -i sha
--sharedstatedir=DIR modifiable architecture-independent data
[PREFIX/com]
--datarootdir=DIR read-only arch.-independent data root
[PREFIX/share]
Quoted 6 lines
Quoted 3 lines
root@x066:[/data/prj/aixtools/git/git-2.13.2]./configure --help | grep ssl
--with-openssl use OpenSSL library (default is YES) ARG can be prefix for openssl library and headersIf you're using configure, you can use --with-openssl, or --with-openssl=PREFIX if your OpenSSL isn't in the standard location but is instead in PREFIX.
I'll look in configure to see if it is not finding openssl. I was assuming it was found - as everything else using GNU "auto" tools find it okay. i.e., /var/lib/libssl.a, etc..
Tomorrow!
Is detecting endianness at compile-time unworkable?
Quoted 11 lines · Michael
Quoted 10 lines · brian m. carlson
Quoted 6 lines · Michael
Quoted 3 lines · brian m. carlson
Well, that explains it. I would recommend submitting a patch to
https://github.com/cr-marcstevens/sha1collisiondetection, and the we can pull in the updated submodule with that fix.Not sure I am smart enough to do that. I'll have to download, build, and see what it says.
The issue is that somewhere in
lib/sha1.c, you need to causeSHA1DC_BIGENDIANto be set. That means you need to figure out what compiler macro might indicate that.
Quoted 6 lines
I remember - roughly - a few decades back - having an assignment to write code to determine endianness. PDP and VAC were different iirc, and many other micro-processors besides the 8088/8086/z85/68k/etc..
If you are looking for a compiler macro as a way to determine this - maybe you have one for gcc, but not for xlc. I do not know it - currently :)
I'm not familiar with AIX, but from searching around I found this porting manual from IBM:
http://www.redbooks.ibm.com/redbooks/pdfs/sg246034.pdf
There they suggest either defining your own macros, or testing the memory layout at runtime (see section "2.2.2.3 Technique 3: Testing memory layout" and surrounding sections).
Perhaps it's worth taking a step back here and thinking about whether this whole thing is unworkable. It was hard enough to get this to work on the combination of Linux, *BSD and Solaris, but I suspect we'll run into increasingly obscure platforms where this is hard or impossible (AIX, HP/UX etc.)
The reason we're in this hole is because we use this sha1collisiondetection library to do SHA-1, and the reason we have issues with it specifically (not OpenSSL et al) is because its only method of detecting endianness is at compile time.
This didn't use to be the case, it was changed in this commit:
https://github.com/cr-marcstevens/sha1collisiondetection/commit/d597672
Dan Shumow: Since the commit message doesn't say why, can you elaborate a bit on why this was done, i.e. is determining this at runtime harmful for performance? If not, perhaps it would be best to bring this back, at least as an option.
And, as an aside, the reason we can't easily make it better ourselves is because the build process for git.git doesn't have a facility to run code to detect this type of stuff (the configure script is always optional). So we can't just run this test ourselves.
Junio: I've barked up that particular tree before in
https://public-inbox.org/git/[email protected]/ and I won't bore you all by repeating myself, except to say that this is yet another case where I wish we had a hard dependency on some way of doing checks via compiled code in our build system.
Re: Is detecting endianness at compile-time unworkable?
Quoted 4 lines · Ævar Arnfjörð Bjarmason
And, as an aside, the reason we can't easily make it better ourselves is because the build process for git.git doesn't have a facility to run code to detect this type of stuff (the configure script is always optional). So we can't just run this test ourselves.
It won't help those who cross-compile anyway. I thought we declared "we make a reasonable effort to guess the target endianness from the system header by inspecting usual macros, but will not aim to cover every system on the planet---instead there is a knob to tweak it for those on exotic platforms" last time we discussed this?
Re: Is detecting endianness at compile-time unworkable?
Quoted 7 lines · Junio C Hamano
Quoted 4 lines · Ævar Arnfjörð Bjarmason
And, as an aside, the reason we can't easily make it better ourselves is because the build process for git.git doesn't have a facility to run code to detect this type of stuff (the configure script is always optional). So we can't just run this test ourselves.
Quoted 5 lines
It won't help those who cross-compile anyway. I thought we declared "we make a reasonable effort to guess the target endianness from the system header by inspecting usual macros, but will not aim to cover every system on the planet---instead there is a knob to tweak it for those on exotic platforms" last time we discussed this?
Well, having said all that, I do not think I personally mind if ./configure learned to include a "compile small program and run it to determine byte order on the build machine" as part of "we make a reasonable effort" as long as it cleanly excludes cross building case (and the result is made overridable just in case we misdetect the "cross-ness" of the build).
Re: Is detecting endianness at compile-time unworkable?
The change was definitely made for performance. Removing the if statements, conditioned upon endianess was an approx 10% improvement, which was very important to getting this library accepted into git.
Thanks,
Dan
Quoted 23 lines · Junio C Hamano
Quoted 7 lines · Junio C Hamano
Quoted 4 lines · Ævar Arnfjörð Bjarmason
And, as an aside, the reason we can't easily make it better ourselves is because the build process for git.git doesn't have a facility to run code to detect this type of stuff (the configure script is always optional). So we can't just run this test ourselves.
Quoted 5 lines
It won't help those who cross-compile anyway. I thought we declared "we make a reasonable effort to guess the target endianness from the system header by inspecting usual macros, but will not aim to cover every system on the planet---instead there is a knob to tweak it for those on exotic platforms" last time we discussed this?
Well, having said all that, I do not think I personally mind if ./configure learned to include a "compile small program and run it to determine byte order on the build machine" as part of "we make a reasonable effort" as long as it cleanly excludes cross building case (and the result is made overridable just in case we misdetect the "cross-ness" of the build).
Re: Is detecting endianness at compile-time unworkable?
I have just replied to https://github.com/cr-marcstevens/sha1collisiondetection/pull/42
I checked a gcc compiler on AIX, and I have the defines for vac.
I do not have access yet to SLES or RHEL (or Ubuntu), just a "free Debian" on my Power6.
* my conclusions|recommendations:
a) AIX is always Big Endian, the define _AIX can be used to determine if AIX
b) POWER7 and earlier are always Big Endian
c) assuming lscpu is always available on Linux systems a command (in configure?) could be used:
root@x074:/usr/bin# lscpu | grep -i endian Byte Order: Big Endian
d) some linux systems (in any case latest versions of RHEL and SLES enterprise) should have a file named lparcfg in /proc (/proc/{powerppc|ppc64|ppc64le|ppc64el}/lparcfg - and it might be in that file. Need to get onto a (POWER8|POWER9) system to check.
Hope this helps:
details re: define of _AIX
root@x068:[/data/httpd/gcc]gcc -dM -E - < /dev/null | grep AIX | head -1 #define _AIX 1
michael@x071:[/home/michael]/usr/bin/grep -p DEFLT: /etc/vac.cfg.[567][123] | grep options\
options = -D_AIX,-D_AIX32,-D_AIX41,-D_AIX43,-D_AIX50,-D_AIX51,-D_AIX52,-D_AIX53,-D_IBMR2,-D_POWER options = -D_AIX,-D_AIX32,-D_AIX41,-D_AIX43,-D_AIX50,-D_AIX51,-D_AIX52,-D_AIX53,-D_AIX61,-D_IBMR2,-D_POWER options = -D_AIX,-D_AIX32,-D_AIX41,-D_AIX43,-D_AIX50,-D_AIX51,-D_AIX52,-D_AIX53,-D_AIX61,-D_AIX71,-D_IBMR2,-D_POWER options = -D_AIX,-D_AIX32,-D_AIX41,-D_AIX43,-D_AIX50,-D_AIX51,-D_AIX52,-D_AIX53,-D_AIX61,-D_AIX71,-D_AIX72,-D_IBMR2,-D_POWER
michael@x071:[/home/michael]ls /etc/vac.cfg.[567][123] /etc/vac.cfg.53 /etc/vac.cfg.61 /etc/vac.cfg.71 /etc/vac.cfg.72
Quoted 34 lines · Daniel Shumow
The change was definitely made for performance. Removing the if statements, conditioned upon endianess was an approx 10% improvement, which was very important to getting this library accepted into git.
Thanks,
DanOn Mon, Jul 30, 2018 at 11:32 AM, Junio C Hamano <[email protected] <mailto:[email protected]>> wrote:
Junio C Hamano <[email protected] <mailto:[email protected]>> writes:
> Ævar Arnfjörð Bjarmason <[email protected] <mailto:[email protected]>> writes:
>
>> And, as an aside, the reason we can't easily make it better ourselves is
>> because the build process for git.git doesn't have a facility to run
>> code to detect this type of stuff (the configure script is always >> optional). So we can't just run this test ourselves.
>
> It won't help those who cross-compile anyway. I thought we declared > "we make a reasonable effort to guess the target endianness from the > system header by inspecting usual macros, but will not aim to cover > every system on the planet---instead there is a knob to tweak it for > those on exotic platforms" last time we discussed this?Well, having said all that, I do not think I personally mind if ./configure learned to include a "compile small program and run it to determine byte order on the build machine" as part of "we make a reasonable effort" as long as it cleanly excludes cross building case (and the result is made overridable just in case we misdetect the "cross-ness" of the build).
Re: Is detecting endianness at compile-time unworkable?
In reply to Junio C HamanoQuoted 6 lines · Junio C Hamano
Well, having said all that, I do not think I personally mind if ./configure learned to include a "compile small program and run it to determine byte order on the build machine" as part of "we make a reasonable effort" as long as it cleanly excludes cross building case (and the result is made overridable just in case we misdetect the "cross-ness" of the build).
No need to run the program for cross-compiles, grepping a object file seems to work for autoconf:
git clone https://git.sv.gnu.org/git/autoconf.git
$PAGER autoconf/lib/autoconf/c.m4
# look for "BIGenDianSyS"
Re: Is detecting endianness at compile-time unworkable?
In reply to Junio C HamanoQuoted 8 lines · Junio C Hamano
Quoted 4 lines · Ævar Arnfjörð Bjarmason
And, as an aside, the reason we can't easily make it better ourselves is because the build process for git.git doesn't have a facility to run code to detect this type of stuff (the configure script is always optional). So we can't just run this test ourselves.
It won't help those who cross-compile anyway.
I was being unclear, what I mean by having "a hard dependency on some way of doing checks via compiled code in our build system" is that we would do some equivalent of this:
diff --git a/Makefile b/Makefile
index 08e5c54549..b021b6e1b6 100644
--- a/Makefile
+++ b/Makefile
@@ -1107,7 +1107,7 @@ DC_SHA1_SUBMODULE = auto
endif
include config.mak.uname
--include config.mak.autogen
+include config.mak.autogen
-include config.mak
ifdef DEVELOPERAnd document that in order to build git you needed to do './configure && make' instead of just 'make', and we'd error out by default if config.mak.autogen wasn't there.
Now obviously that would need some sort of escape hatch. I.e. you could invoke 'make' like this:
make CONFIGURE_MAK_AUTOGEN_FILE=some-file
That's how you would do cross-compilation, you'd arrange to run './configure' on some system, save the output, and ferry over this 'some-file' to where you're building git, or you would manually prepare a file that had all the settings we'd expect to have been set already set.
Now, whether we do this with autoconf or not is just an implementation detail. Looking at this some more I think since we already use the $(shell) construct we could just have some 'configure-detect' Makefile target which would compile various test programs, and we'd use their output to set various settings, a sort of home-grown autoconf (because people are bound to have objections to a hard dependency on it...).
Quoted 5 lines
I thought we declared "we make a reasonable effort to guess the target endianness from the system header by inspecting usual macros, but will not aim to cover every system on the planet---instead there is a knob to tweak it for those on exotic platforms" last time we discussed this?
Yes, but I think it's worth re-visiting that decision, which was made with the constraints that we don't have a build system that can do checks via compiled code, so we need this hack in the first place instead of things Just Working.
And as I pointed out in the linked E-Mail this also impacts us in other ways, and will cause other issues in the future, so it's worth thinking about if this is the right path to take.
Re: Is detecting endianness at compile-time unworkable?
In reply to Ævar Arnfjörð BjarmasonA small step back...
Quoted 17 lines · Ævar Arnfjörð Bjarmason
Quoted 15 lines · Michael
Quoted 10 lines · brian m. carlson
Quoted 6 lines · Michael
Quoted 3 lines · brian m. carlson
Well, that explains it. I would recommend submitting a patch to
https://github.com/cr-marcstevens/sha1collisiondetection, and the we can pull in the updated submodule with that fix.Not sure I am smart enough to do that. I'll have to download, build, and see what it says.
The issue is that somewhere in
lib/sha1.c, you need to causeSHA1DC_BIGENDIANto be set. That means you need to figure out what compiler macro might indicate that.I remember - roughly - a few decades back - having an assignment to write code to determine endianness. PDP and VAC were different iirc, and many other micro-processors besides the 8088/8086/z85/68k/etc..
Quoted 5 lines
Quoted 2 lines
If you are looking for a compiler macro as a way to determine this - maybe you have one for gcc, but not for xlc. I do not know it - currently :)
I'm not familiar with AIX, but from searching around I found this porting manual from IBM:
http://www.redbooks.ibm.com/redbooks/pdfs/sg246034.pdf
This is from July 2001 - when AIX 5L, for Linux affinity, was new. AIX was (nearly) the #1 posix system, and linux was a minor player in the data center (in or out (now as IAAS)). IMHO, the recommendations made in 2001 are probably no longer applicable (64-bit was fairly new, e.g., rather than common).
Quoted 11 lines
There they suggest either defining your own macros, or testing the memory layout at runtime (see section "2.2.2.3 Technique 3: Testing memory layout" and surrounding sections).
Perhaps it's worth taking a step back here and thinking about whether this whole thing is unworkable. It was hard enough to get this to work on the combination of Linux, *BSD and Solaris, but I suspect we'll run into increasingly obscure platforms where this is hard or impossible (AIX, HP/UX etc.)
Quoted 4 lines
The reason we're in this hole is because we use this sha1collisiondetection library to do SHA-1, and the reason we have issues with it specifically (not OpenSSL et al) is because its only method of detecting endianness is at compile time.
Cannot speak for the "others", but as I have mentioned before - as AIX in only on POWER it is also only Big Endian - so a compiletime #if testing for _AIX will work fine
Quoted 8 lines
This didn't use to be the case, it was changed in this commit:
https://github.com/cr-marcstevens/sha1collisiondetection/commit/d597672Dan Shumow: Since the commit message doesn't say why, can you elaborate a bit on why this was done, i.e. is determining this at runtime harmful for performance? If not, perhaps it would be best to bring this back, at least as an option.
Quoted 4 lines
And, as an aside, the reason we can't easily make it better ourselves is because the build process for git.git doesn't have a facility to run code to detect this type of stuff (the configure script is always optional). So we can't just run this test ourselves.
On AIX - I am required to run configure, and frankly, I am amazed that not everyone is running it. Among other things I an modifying the prefix (to /opt) and many of the others to different /var/git/* areas as I do not want to "polute" the BOS (base OS) and/or other packages/packagers. Officially, according the Linux FHS-3.0 I sould be using /opt/aixtools as prefix.
FYI: my current build process is:
wget git-{version}.tar.xz
xz -dc git-{version}.tar.xz
cd git-{version}.tar.xz
mv Makefile Makefile.git #my build scripts only run configure when Makefile does not exist ./configure ...
ln Makefile.git Makefile
make
I am amazed, as it rarely happens (maybe git is my first encounter) - that configure does not create a Makefile. This also complicates building git "out of tree".
Quoted 6 lines
Junio: I've barked up that particular tree before in
https://public-inbox.org/git/[email protected]/ and I won't bore you all by repeating myself, except to say that this is yet another case where I wish we had a hard dependency on some way of doing checks via compiled code in our build system.
For AIX: again - the determination is simple. If _AIX is set to 1 then use BigEndian, or, use:
michael@x071:[/home/michael]uname
AIX
i.e., something like:
$(uname) == "AIX" && BigEndian=1
Re: Is detecting endianness at compile-time unworkable?
Quoted 6 lines · Michael Felt
For AIX: again - the determination is simple. If
_AIXis set to 1 then use BigEndian, or, use:
michael@x071:[/home/michael]uname
AIX
i.e., something like:
$(uname) == "AIX" && BigEndian=1
In lieu of some "let's test this with a compile-test" solution, that seems like a viable way forward for now. Can you please test the merge request I have at
https://github.com/cr-marcstevens/sha1collisiondetection/pull/45 by manually applying that change to the relevant file in sha1dc/ and building/testing git on AIX?
It should work, but as noted in the MR please test it so we can make sure, and then (if you have a GitHub account) comment on the MR saying it works for you.
PATCH 1/1 sha1dc: update from upstream
Update sha1dc from the latest version by the upstream maintainer[1]. See 2db87328ef ("Merge branch 'ab/sha1dc'", 2017-07-10) for the last update.
This fixes an issue where AIX was wrongly detected as a Little-endian instead of a Big-endian system. See [2][3][4].
1. https://github.com/cr-marcstevens/sha1collisiondetection/commit/232357eb2ea0397388254a4b188333a227bf5b10
2. https://github.com/cr-marcstevens/sha1collisiondetection/pull/45
3. https://github.com/cr-marcstevens/sha1collisiondetection/pull/42
4. https://public-inbox.org/git/[email protected]/
---
Quoted 5 lines · Ævar Arnfjörð Bjarmason
https://github.com/cr-marcstevens/sha1collisiondetection/pull/45
[...]
It should work, but as noted in the MR please test it so we can make sure, and then (if you have a GitHub account) comment on the MR saying it works for you.
This got merged upstream, and as noted in that upstream PR I've personally tested this on AIX under both GCC and IBM's xlc on the GCC compile farm, it works.
sha1collisiondetection | 2 +- sha1dc/sha1.c | 12 +++++++++++- 2 files changed, 12 insertions(+), 2 deletions(-)
sha1collisiondetection +1 -1
@@ -1 +1 @@ -Subproject commit 19d97bf5af05312267c2e874ee6bcf584d9e9681 +Subproject commit 232357eb2ea0397388254a4b188333a227bf5b10
sha1dc/sha1.c +11 -1
@@ -93,13 +93,23 @@#define SHA1DC_BIGENDIAN /* Not under GCC-alike or glibc or *BSD or newlib or <processor whitelist> */ +#elif (defined(_AIX)) + +/* +* Defines Big Endian on a whitelist of OSs that are known to be Big +* Endian-only. See +* https://public-inbox.org/git/[email protected]/ +*/ +#define SHA1DC_BIGENDIAN + +/* Not under GCC-alike or glibc or *BSD or newlib or <processor whitelist> or <os whitelist> */ #elif defined(SHA1DC_ON_INTEL_LIKE_PROCESSOR )/* * As a last resort before we do anything else we're not 100% sure * about below, we blacklist specific processors here. We could add * more, see e.g. https://wiki.debian.org/ArchitectureSpecificsMemo */ -#else /* Not under GCC-alike or glibc or *BSD or newlib or <processor whitelist> or <processor blacklist> */ +#else /* Not under GCC-alike or glibc or *BSD or newlib or <processor whitelist> or <os whitelist> or <processor blacklist> */ /* We do nothing more here for now */ /*#error "Uncomment this to see if you fall through all the detection"*/
Signature · 1 line
2.18.0.345.g5c9ce644c3
Re: [PATCH] sha1dc: update from upstream
Sent from my iPhone
Quoted 24 lines · Ævar Arnfjörð Bjarmason
Update sha1dc from the latest version by the upstream maintainer[1]. See
2db87328ef("Merge branch 'ab/sha1dc'", 2017-07-10) for the last update.This fixes an issue where AIX was wrongly detected as a Little-endian instead of a Big-endian system. See [2][3][4].
1. https://github.com/cr-marcstevens/sha1collisiondetection/commit/232357eb2ea0397388254a4b188333a227bf5b10
2. https://github.com/cr-marcstevens/sha1collisiondetection/pull/45
3. https://github.com/cr-marcstevens/sha1collisiondetection/pull/42
4. https://public-inbox.org/git/[email protected]/Signed-off-by Ævar Arnfjörð Bjarmason <[email protected]>---
Quoted 6 lines · Ævar Arnfjörð Bjarmason
https://github.com/cr-marcstevens/sha1collisiondetection/pull/45
[...]
It should work, but as noted in the MR please test it so we can make sure, and then (if you have a GitHub account) comment on the MR saying it works for you.
Quoted 4 lines
This got merged upstream, and as noted in that upstream PR I've personally tested this on AIX under both GCC and IBM's xlc on the GCC compile farm, it works.
Thanks. I already have git 2.18 in use with the manual patch.
Quoted 43 lines
sha1collisiondetection | 2 +-
sha1dc/sha1.c| 12 +++++++++++-2 files changed, 12 insertions(+), 2 deletions(-)sha1collisiondetection +1
-1@@ -1 +1 @@ -Subproject commit 19d97bf5af05312267c2e874ee6bcf584d9e9681 +Subproject commit 232357eb2ea0397388254a4b188333a227bf5b10sha1dc/sha1.c +11
-1@@ -93,13 +93,23 @@#define SHA1DC_BIGENDIAN /* Not under GCC-alike or glibc or *BSD or newlib or <processor whitelist> */ +#elif (defined(_AIX)) + +/* +* Defines Big Endian on a whitelist of OSs that are known to be Big +* Endian-only. See +* https://public-inbox.org/git/[email protected]/ +*/ +#define SHA1DC_BIGENDIAN + +/* Not under GCC-alike or glibc or *BSD or newlib or <processor whitelist> or <os whitelist> */ #elif defined(SHA1DC_ON_INTEL_LIKE_PROCESSOR )/* * As a last resort before we do anything else we're not 100% sure * about below, we blacklist specific processors here. We could add * more, see e.g. https://wiki.debian.org/ArchitectureSpecificsMemo */ -#else /* Not under GCC-alike or glibc or *BSD or newlib or <processor whitelist> or <processor blacklist> */ +#else /* Not under GCC-alike or glibc or *BSD or newlib or <processor whitelist> or <os whitelist> or <processor blacklist> */ /* We do nothing more here for now */ /*#error "Uncomment this to see if you fall through all the detection"*/ Signature · 1 line
2.18.0.345.g5c9ce644c3
Re: [PATCH] sha1dc: update from upstream
In reply to Ævar Arnfjörð Bjarmasonsha1collisiondetection line 1
Quoted 7 lines · Ævar Arnfjörð Bjarmason
sha1collisiondetection +1
-1@@ -1 +1 @@ -Subproject commit 19d97bf5af05312267c2e874ee6bcf584d9e9681 +Subproject commit 232357eb2ea0397388254a4b188333a227bf5b10
Offtopic:
I wonder if we want to extend diffs a little here and similar to a diffstat that is not technically needed, but rather aiding the reviewers eyes. So maybe we could add a concise shortlog, e.g. after this line:
19d97bf5..232357eb (2 dots indicating it is a fast forward instead of 3 dots) 3 commits from "Merge pull request #37 from avar/fixup-pull-request-34" to "Merge pull request #45 from avar/aix-big-endian-detection"
Re: Is detecting endianness at compile-time unworkable?
In reply to Ævar Arnfjörð BjarmasonQuoted 4 lines · Ævar Arnfjörð Bjarmason
The reason we're in this hole is because we use this sha1collisiondetection library to do SHA-1, and the reason we have issues with it specifically (not OpenSSL et al) is because its only method of detecting endianness is at compile time.
When using gcc (no xlc available for Linux on Power)
POWER6 (Big Endian by definition) root@x068:[/data/httpd/gcc]gcc -dM -E - < /dev/null | grep -i end #define __ORDER_LITTLE_ENDIAN__ 1234
#define __BIG_ENDIAN__ 1
#define __FLOAT_WORD_ORDER__ __ORDER_BIG_ENDIAN__ #define __ORDER_PDP_ENDIAN__ 3412
#define _BIG_ENDIAN 1
#define __ORDER_BIG_ENDIAN__ 4321
#define __BYTE_ORDER__ __ORDER_BIG_ENDIAN__
SLES12 on POWER8
suse12test:~ # gcc -dM -E - < /dev/null | grep -i end #define __ORDER_LITTLE_ENDIAN__ 1234
#define _LITTLE_ENDIAN 1
#define __FLOAT_WORD_ORDER__ __ORDER_LITTLE_ENDIAN__ #define __ORDER_PDP_ENDIAN__ 3412
#define __LITTLE_ENDIAN__ 1
#define __ORDER_BIG_ENDIAN__ 4321
#define __BYTE_ORDER__ __ORDER_LITTLE_ENDIAN__
*So, for compile time tests, when gcc is the compiler it seems the following defines are available**
**__BIG_ENDIAN__, _BIG_ENDIAN, __LITTLE__ENDIAN__, _LITTLE_ENDIAN** **or something based on the value of __BYTE_ORDER__*
I'll see if I can find something similar for xlc, but will only be able to test xlc on AIX.
Quoted 8 lines
This didn't use to be the case, it was changed in this commit:
https://github.com/cr-marcstevens/sha1collisiondetection/commit/d597672Dan Shumow: Since the commit message doesn't say why, can you elaborate a bit on why this was done, i.e. is determining this at runtime harmful for performance? If not, perhaps it would be best to bring this back, at least as an option.
Re: Is detecting endianness at compile-time unworkable?
In reply to Ævar Arnfjörð BjarmasonI hope a I have a "leap forward"
Quoted 5 lines · Ævar Arnfjörð Bjarmason
Perhaps it's worth taking a step back here and thinking about whether this whole thing is unworkable. It was hard enough to get this to work on the combination of Linux, *BSD and Solaris, but I suspect we'll run into increasingly obscure platforms where this is hard or impossible (AIX, HP/UX etc.)
While I still cannot say for HP/UX it does seem there is a potential solution based on the status for _LITTLE_ENDIAN and _BIG_ENDIAN. At least, gcc on POWER and xlc on POWER provides one or the other - and my hope is that gcc on other platforms also provides them.
For "other" compilers that do not provide them - a modification to CFLAGS to define one or the other should make "make" work.
Details (note - I am not a programmer, so by definition at least one of my "macros" will be wrong :)
AIX and xlc
root@x066:[/]xlc -qshowmacros -E /dev/null | grep -i endi 1506-297 (S) Unable to open input file null. A file or directory in the path name does not exist..
#define __HHW_BIG_ENDIAN__ 1
#define __BIG_ENDIAN__ 1
#define __THW_BIG_ENDIAN__ 1
#define _BIG_ENDIAN 1
On SLES12 (le) and xlc suse12test:~/images/littleEndian/sles # xlc -qshowmacros -dM -E x.c | grep -i endi
#define _LITTLE_ENDIAN 1
#define __BYTE_ORDER__ __ORDER_LITTLE_ENDIAN__
#define __FLOAT_WORD_ORDER__ __ORDER_LITTLE_ENDIAN__ #define __LITTLE_ENDIAN__ 1
#define __ORDER_BIG_ENDIAN__ 4321
#define __ORDER_LITTLE_ENDIAN__ 1234
#define __ORDER_PDP_ENDIAN__ 3412
#define __VEC_ELEMENT_REG_ORDER__ __ORDER_LITTLE_ENDIAN__
Based on what I can see on gcc on POWER and xlc on POWER I think an approach (simplified) can be:
#if undefined(_BIG_ENDIAN) && undef(_LITTLE_ENDIAN)
#error "one of _BIG_ENDIAN or _LITTLE_ENDIAN must be defined. Try adding the correct value to CFLAGS"
#else defined(_BIG_ENDIAN) && defined(_LITTLE_ENDIAN)
#error "Only one of _BIG_ENDIAN and _LITTLE_ENDIAN may be defined, not both" #endif
And then logic based on the value set.
This should also make cross-compile possible by unsetting an incorrect default and setting the correct value.
p.s. Is there a setting I need to set somewhere so I receive a copy of the email sent after it is received by the list. I could send myself a copy, but I much prefer it comes from the maillist - as verification it was received.
Re: Is detecting endianness at compile-time unworkable?
Quoted 9 lines · Michael Felt
I hope a I have a "leap forward"
Quoted 5 lines · Ævar Arnfjörð Bjarmason
Perhaps it's worth taking a step back here and thinking about whether this whole thing is unworkable. It was hard enough to get this to work on the combination of Linux, *BSD and Solaris, but I suspect we'll run into increasingly obscure platforms where this is hard or impossible (AIX, HP/UX etc.)
Quoted 4 lines
While I still cannot say for HP/UX it does seem there is a potential solution based on the status for
_LITTLE_ENDIANand_BIG_ENDIAN. At least, gcc on POWER and xlc on POWER provides one or the other - and my hope is that gcc on other platforms also provides them.
Yeah with GCC this is relatively easy, see
https://github.com/cr-marcstevens/sha1collisiondetection/blame/c3e1304/lib/sha1.c#L29-L115
Quoted 38 lines
For "other" compilers that do not provide them - a modification to CFLAGS to define one or the other should make "make" work.
Details (note - I am not a programmer, so by definition at least one of my "macros" will be wrong :)
AIX and xlc
root@x066:[/]xlc -qshowmacros -E/dev/null| grep -i endi 1506-297 (S) Unable to open input file null. A file or directory in the path name does not exist..
#define__HHW_BIG_ENDIAN__1
#define__BIG_ENDIAN__1
#define__THW_BIG_ENDIAN__1
#define_BIG_ENDIAN1On SLES12 (le) and xlc suse12test:~/images/littleEndian/sles # xlc -qshowmacros -dM -E
x.c| grep -i endi
#define_LITTLE_ENDIAN1
#define__BYTE_ORDER____ORDER_LITTLE_ENDIAN__
#define__FLOAT_WORD_ORDER____ORDER_LITTLE_ENDIAN__#define__LITTLE_ENDIAN__1
#define__ORDER_BIG_ENDIAN__4321
#define__ORDER_LITTLE_ENDIAN__1234
#define__ORDER_PDP_ENDIAN__3412
#define__VEC_ELEMENT_REG_ORDER____ORDER_LITTLE_ENDIAN__Based on what I can see on gcc on POWER and xlc on POWER I think an approach (simplified) can be:
#if
undefined(_BIG_ENDIAN)&&undef(_LITTLE_ENDIAN)
#error "one of_BIG_ENDIANor_LITTLE_ENDIANmust be defined. Try adding the correct value to CFLAGS"
#elsedefined(_BIG_ENDIAN)&&defined(_LITTLE_ENDIAN)
#error "Only one of_BIG_ENDIANand_LITTLE_ENDIANmay be defined, not both" #endif
Quoted 3 lines
And then logic based on the value set.
This should also make cross-compile possible by unsetting an incorrect default and setting the correct value.
...the real trick is using these macros outside of GCC / glibc and on older GCC versions. See the github link above, you basically end up with a whitelist of how it looks on different systems / compilers. Sometimes both are defined, sometimes only both etc.
It can be done, but as that code shows it's somewhat complex macro soup to get right.
Quoted 4 lines
p.s. Is there a setting I need to set somewhere so I receive a copy of the email sent after it is received by the list. I could send myself a copy, but I much prefer it comes from the maillist - as verification it was received.
You should get that, but maybe your mailer ignores Message-Ids it already has, but you can go to https://public-inbox.org/git/ and search for the Message-Id or your name to see E-Mails you've sent that made it to the list, e.g.:
https://public-inbox.org/git/?q=aixtools%40felt.demon.nl
Re: Is detecting endianness at compile-time unworkable?
Quoted 7 lines · Ævar Arnfjörð Bjarmason
...the real trick is using these macros outside of GCC / glibc and on older GCC versions. See the github link above, you basically end up with a whitelist of how it looks on different systems / compilers. Sometimes both are defined, sometimes only both etc.
It can be done, but as that code shows it's somewhat complex macro soup to get right.
FYI - the gcc I was using is 4.7.4.
And, the reason I suggest the test for both not being defined is so that 'make' stops and whoever is running make just sets one or the other. Let them 'file a bug' When they come with a compiler that does not work - and find out what could be used.
For example, _AIX is the same as _BIG_ENDIAN. In the meantime, the code to test is simple.
Either one of _BIG_ENDIAN or _LITTLE_ENDIAN is provided by the compiler or the builder supplies one of the two using CFLAGS. I assume there is also a "undefine" flag, maybe -U - so hopefully a -U and a -D combination could be used for cross-compiling.
re: my mailer blocking things - it would only be for this list, as other lists come through with no extra work from me. At least I am not aware of anything special I could do.