Skip to contentSkip to conversation

compat/bswap.h: Simplify MSVC endianness detection

PATCH Superseded Daniel Gurney 5 participants 13 messages Nov 7, 2020 – Nov 10, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 compat/bswap.h: Simplify MSVC endianness detection

Modern MSVC or Windows versions don't support big-endian, so it's unnecessary to consider architectures when using it.

---
 compat/bswap.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
compat/bswap.h +1 -1
index c0bb744adc..72f225eaa8 100644
@@ -74,7 +74,7 @@ static inline uint64_t git_bswap64(uint64_t x)
 }
 #endif
 
-#elif defined(_MSC_VER) && (defined(_M_IX86) || defined(_M_X64))
+#elif defined(_MSC_VER)
 
 #include <stdlib.h>
 
Signature · 1 line
2.29.2
#1 lore raw
Quoted 13 lines · Daniel Gurney

Modern MSVC or Windows versions don't support big-endian, so it's unnecessary to consider architectures when using it.

---
 compat/bswap.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
compat/bswap.h +0 -0
index c0bb744adc..72f225eaa8 100644

compat/bswap.h static inline uint64_t git_bswap64(uint64_t x) · line 80

Quoted 8 lines
compat/bswap.h +1 -1
@@ -74,7 +74,7 @@ static inline uint64_t git_bswap64(uint64_t x)
 }
 #endif
 
-#elif defined(_MSC_VER) && (defined(_M_IX86) || defined(_M_X64))
+#elif defined(_MSC_VER)
 
 #include <stdlib.h>
 

The patch itself seems fine to me, of course. For the message, it may be valuable to mention that this enables the proper behavior for ARM64, which was your original goal. Also, by convention, we typically use a lower case letter after the colon in the summary.

And of course, thanks for the patch. I always have a soft spot for patches that improve architecture support.

Signature · 2 lines
brian m. carlson (he/him or they/them)
Houston, Texas, US
#2 lore raw

PATCH v2 1/1 compat/bswap.h: simplify MSVC endianness detection

In reply to Daniel Gurney

Modern MSVC or Windows versions don't support big-endian, so it's unnecessary to consider architectures when using it.

This also makes ARM64 MSVC builds succeed.

---
v2: just edited the commit message as suggested by Brian.

 compat/bswap.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
compat/bswap.h +1 -1
index c0bb744adc..72f225eaa8 100644
@@ -74,7 +74,7 @@ static inline uint64_t git_bswap64(uint64_t x)
 }
 #endif
 
-#elif defined(_MSC_VER) && (defined(_M_IX86) || defined(_M_X64))
+#elif defined(_MSC_VER)
 
 #include <stdlib.h>
 
Signature · 1 line
2.29.2
#3 lore raw

Re: [PATCH v2] compat/bswap.h: simplify MSVC endianness detection

Quoted 2 lines · Daniel Gurney

Modern MSVC or Windows versions don't support big-endian, so it's unnecessary to consider architectures when using it.

This made me wonder if we support any non-modern versions (which would be negatively impacted).

From the earlier thread at [1], it sounds like probably not, but I wonder if we can offer a stronger argument there (or just define "modern" a little more clearly).

-Peff

[1] https://lore.kernel.org/git/[email protected]/

#4 lore raw

Re: [PATCH v2] compat/bswap.h: simplify MSVC endianness detection

Quoted 7 lines · Jeff King
Quoted 2 lines · Daniel Gurney

Modern MSVC or Windows versions don't support big-endian, so it's unnecessary to consider architectures when using it.

This made me wonder if we support any non-modern versions (which would be negatively impacted).

I'm pretty sure we don't. As I said, we're using several C99 features and that version precedes the C99 standard (and 1999).

Quoted 3 lines

From the earlier thread at [1], it sounds like probably not, but I wonder if we can offer a stronger argument there (or just define "modern" a little more clearly).

According to Wikipedia[0]:

Visual C++ 2013 [12.0] finally added support for various C99 features in its C mode (including designated initializers, compound literals, and the _Bool type), though it was still not complete. Visual C++ 2015 further improved the C99 support, with full support of the C99 Standard Library, except for features that require C99 language features not yet supported by the compiler.

The version mentioned that supported MIPS, Alpha, and m68k was Visual C++ 2.0 RISC Edition. While Wikipedia doesn't mention its release date, its successor, Visual C++ 4.0, was released in 1995. The m68k version ran on Macs using those processors, and Apple abandoned m68k for PowerPC in 1994[1].

I'm entirely comfortable with requiring that people use a compiler and operating system newer than 25 years old to compile Git correctly. As I've said or implied in previous threads, I'm also fine requiring C99 (vendors having had over two decades to implement it) and only supporting OSes with vendor security support, although obviously these latter two items are much more controversial.

I'm fine leaving the commit message as it stands, given the brevity of the patch and that in the technology field, the affected versions are not in any way "modern," but of course I wouldn't object to a reroll. It's fine, should that happen, to include any of this email in the commit message.

[0] https://en.wikipedia.org/wiki/Microsoft_Visual_C%2B%2B
[1] https://en.wikipedia.org/wiki/Macintosh

Signature · 2 lines
brian m. carlson (he/him or they/them)
Houston, Texas, US
#5 lore raw

Re: [PATCH v2] compat/bswap.h: simplify MSVC endianness detection

Quoted 25 lines · brian m. carlson
Quoted 7 lines · Jeff King
Quoted 2 lines · Daniel Gurney

Modern MSVC or Windows versions don't support big-endian, so it's unnecessary to consider architectures when using it.

This made me wonder if we support any non-modern versions (which would be negatively impacted).

I'm pretty sure we don't. As I said, we're using several C99 features and that version precedes the C99 standard (and 1999).

Quoted 3 lines

From the earlier thread at [1], it sounds like probably not, but I wonder if we can offer a stronger argument there (or just define "modern" a little more clearly).

According to Wikipedia[0]:

Visual C++ 2013 [12.0] finally added support for various C99 features in its C mode (including designated initializers, compound literals, and the _Bool type), though it was still not complete. Visual C++ 2015 further improved the C99 support, with full support of the C99 Standard Library, except for features that require C99 language features not yet supported by the compiler.

Quoted 5 lines

The version mentioned that supported MIPS, Alpha, and m68k was Visual C++ 2.0 RISC Edition. While Wikipedia doesn't mention its release date, its successor, Visual C++ 4.0, was released in 1995. The m68k version ran on Macs using those processors, and Apple abandoned m68k for PowerPC in 1994[1].

So,

The only versions of MSVC that support big-endian are too ancient and do not understand some C99 features we use in our codebase, so it is unnecessary...

would be sufficient?

Quoted 6 lines

I'm entirely comfortable with requiring that people use a compiler and operating system newer than 25 years old to compile Git correctly. As I've said or implied in previous threads, I'm also fine requiring C99 (vendors having had over two decades to implement it) and only supporting OSes with vendor security support, although obviously these latter two items are much more controversial.

Maybe controversial, but worth at least laying the ground rules ahead of time?

Do we have any specific feature we avoid only due to portability concerns? Declaring an identifier in the first part of for() is the only thing that comes to my mind, but there may be others. I think we should consider how well each individual feature is supported by systems we care about as we feel the need.

Thanks.

#6 lore raw

Re: [PATCH v2] compat/bswap.h: simplify MSVC endianness detection

Hi Junio,

Quoted 48 lines · Junio C Hamano
Quoted 25 lines · brian m. carlson
Quoted 7 lines · Jeff King
Quoted 2 lines · Daniel Gurney

Modern MSVC or Windows versions don't support big-endian, so it's unnecessary to consider architectures when using it.

This made me wonder if we support any non-modern versions (which would be negatively impacted).

I'm pretty sure we don't. As I said, we're using several C99 features and that version precedes the C99 standard (and 1999).

Quoted 3 lines

From the earlier thread at [1], it sounds like probably not, but I wonder if we can offer a stronger argument there (or just define "modern" a little more clearly).

According to Wikipedia[0]:

Visual C++ 2013 [12.0] finally added support for various C99 features in its C mode (including designated initializers, compound literals, and the _Bool type), though it was still not complete. Visual C++ 2015 further improved the C99 support, with full support of the C99 Standard Library, except for features that require C99 language features not yet supported by the compiler.

Quoted 5 lines

The version mentioned that supported MIPS, Alpha, and m68k was Visual C++ 2.0 RISC Edition. While Wikipedia doesn't mention its release date, its successor, Visual C++ 4.0, was released in 1995. The m68k version ran on Macs using those processors, and Apple abandoned m68k for PowerPC in 1994[1].

So,

The only versions of MSVC that support big-endian are too ancient and do not understand some C99 features we use in our codebase, so it is unnecessary...

would be sufficient?

Quoted 6 lines

I'm entirely comfortable with requiring that people use a compiler and operating system newer than 25 years old to compile Git correctly. As I've said or implied in previous threads, I'm also fine requiring C99 (vendors having had over two decades to implement it) and only supporting OSes with vendor security support, although obviously these latter two items are much more controversial.

Quoted 8 lines

Maybe controversial, but worth at least laying the ground rules ahead of time?

Do we have any specific feature we avoid only due to portability concerns? Declaring an identifier in the first part of for() is the only thing that comes to my mind, but there may be others. I think we should consider how well each individual feature is supported by systems we care about as we feel the need.

I would feel a bit more comfortable reintroducing the part that specifically checks for x86, x86_64 and ARM64, for the reasons I outlined in my reply to a previous version of this patch: just because the MSVC versions we can currently use to build Git currently only supports little endian does not mean that all future versions will do. Point in reference:
you can build Linux applications in Visual Studio like _right now_ [*1*].

Ciao,
Dscho

Footnote *1*: It currently uses GCC, but who says it always will?
https://docs.microsoft.com/en-us/cpp/linux/cmake-linux-project

#7 lore raw

Re: [PATCH v2] compat/bswap.h: simplify MSVC endianness detection

Hello,

Quoted 8 lines · Johannes Schindelin

Point in reference:
you can build Linux applications in Visual Studio like _right now_ [*1*].

Ciao,
Dscho

Footnote *1*: It currently uses GCC, but who says it always will?
https://docs.microsoft.com/en-us/cpp/linux/cmake-linux-project

When it comes to building C++ code MSVC uses Microsoft's C++ Standard Library, and one of its stated non-goals[1] is being ported to other systems. I assume the same applies for their C library. Therefore to me a scenario where a future version of MSVC would build non-Windows code, let alone for a big-endian architecture, seems extremely unlikely.

That said, I understand your overall point of view perfectly, and I'm fine with either one of my patches being applied since the end result today is the same.

- Daniel

[1] https://github.com/microsoft/STL#non-goals

#8 lore raw

Re: [PATCH v2] compat/bswap.h: simplify MSVC endianness detection

Hi Daniel,

Quoted 7 lines · Daniel Gurney
Quoted 6 lines · Johannes Schindelin

Point in reference:
you can build Linux applications in Visual Studio like _right now_ [*1*].

Ciao,
Dscho

Quoted 8 lines
Quoted 2 lines

Footnote *1*: It currently uses GCC, but who says it always will?
https://docs.microsoft.com/en-us/cpp/linux/cmake-linux-project

When it comes to building C++ code MSVC uses Microsoft's C++ Standard Library, and one of its stated non-goals[1] is being ported to other systems. I assume the same applies for their C library. Therefore to me a scenario where a future version of MSVC would build non-Windows code, let alone for a big-endian architecture, seems extremely unlikely.

Well, we know for a fact that at least XBox is a big-endian system and ther are rumors out there that Visual Studio _might_ be used to build software for that system.

Of course, I am not suggesting that we should build Git for XBox. But it seems dangerous to me to make too many assumptions.

Quoted 3 lines

That said, I understand your overall point of view perfectly, and I'm fine with either one of my patches being applied since the end result today is the same.

As a maintainer, I am less concerned about the "result today" than I am about keeping things easy and effortless to maintain. One of your patches accomplishes that. The other one made it into next:
https://github.com/git/git/commit/91a67b86f77

Ciao,
Dscho

#9 lore raw

Re: [PATCH v2] compat/bswap.h: simplify MSVC endianness detection

Quoted 4 lines · Johannes Schindelin

As a maintainer, I am less concerned about the "result today" than I am about keeping things easy and effortless to maintain. One of your patches accomplishes that. The other one made it into next:
https://github.com/git/git/commit/91a67b86f77

I do not think reverting it and requeuing

https://lore.kernel.org/git/[email protected]/

would help future folks why we ignore _MSC_VER as any sign usable to detect endianness, so I'd prefer to see a patch *on top* of 1af265f0 (compat/bswap.h: simplify MSVC endianness detection, 2020-11-08), which is 91a67b86f77^2, that explains why we prefer to list archs explicitly in its log message, which would be the primary value of that commit.

Something along this line, perhaps?

-- >8 --

Subject: compat/bswap.h: do not assume MSVC is little-endian only

Earlier, with 1af265f0 (compat/bswap.h: simplify MSVC endianness detection, 2020-11-08), we tried to simplify endianness detection used in compat/bswap.h by assuming that any version Git compiled by MSVC (detected by _MSC_VER preprocessor macro) is meant to run on little endian boxes, as the versions of old MSVC that support m68k and MIPS do not support some C99 features used in the codebase anyway.

While it might hold true that modern versions of Windows are all little-endian, MSVC is and/or can be ported to build for big-endian boxes, so tying _MSC_VER with endianness is a bit too restrictive.

Let's go back to the old way to use _MSC_VER to learn what preprocessor macros compiler uses to tell us which arch we are building for, and list these arches that are little-endian explicitly.

... signed-off-by from you and helped-by from others ...

---
diffstat
patch

#10 lore raw

Re: [PATCH v2] compat/bswap.h: simplify MSVC endianness detection

Quoted 41 lines · Junio C Hamano
Quoted 4 lines · Johannes Schindelin

As a maintainer, I am less concerned about the "result today" than I am about keeping things easy and effortless to maintain. One of your patches accomplishes that. The other one made it into next:
https://github.com/git/git/commit/91a67b86f77

I do not think reverting it and requeuing

https://lore.kernel.org/git/[email protected]/

would help future folks why we ignore _MSC_VER as any sign usable to detect endianness, so I'd prefer to see a patch *on top* of 1af265f0 (compat/bswap.h: simplify MSVC endianness detection, 2020-11-08), which is 91a67b86f77^2, that explains why we prefer to list archs explicitly in its log message, which would be the primary value of that commit.

Something along this line, perhaps?

-- >8 --

Subject: compat/bswap.h: do not assume MSVC is little-endian only

Earlier, with 1af265f0 (compat/bswap.h: simplify MSVC endianness detection, 2020-11-08), we tried to simplify endianness detection used in compat/bswap.h by assuming that any version Git compiled by MSVC (detected by _MSC_VER preprocessor macro) is meant to run on little endian boxes, as the versions of old MSVC that support m68k and MIPS do not support some C99 features used in the codebase anyway.

While it might hold true that modern versions of Windows are all little-endian, MSVC is and/or can be ported to build for big-endian boxes, so tying _MSC_VER with endianness is a bit too restrictive.

Let's go back to the old way to use _MSC_VER to learn what preprocessor macros compiler uses to tell us which arch we are building for, and list these arches that are little-endian explicitly.

Quoted 5 lines

... signed-off-by from you and helped-by from others ...

---
diffstat
patch

Daniel's patch adds _M_ARM64 to the list, but do we need to do anything further to tell the endian on such a bi-endian arch, or does MSVC only support little-endian for that architecture?

Just double-checking as the "confusion" that started this thread came from an assumption that MSVC == Windows == big-endian, and you told us MSVC != Windows. Now the patch assumes ARM64-on-MSVC is little-endian only and we want to make sure that assumption is true.

And perhaps it is worth documenting in the log, perhaps

... that are little-endian explicitly. Note that ARM64 is bi-endian in nature but we treat it little-endian as MSVC does not treat the arch as bi-endian.

or something like that at the end (I do not know what MSVC actually does---just illustrating the level of details I expect in the explanation).

Thanks.

#11 lore raw

Re: [PATCH v2] compat/bswap.h: simplify MSVC endianness detection

Hi Junio,

Quoted 49 lines · Junio C Hamano
Quoted 41 lines · Junio C Hamano
Quoted 4 lines · Johannes Schindelin

As a maintainer, I am less concerned about the "result today" than I am about keeping things easy and effortless to maintain. One of your patches accomplishes that. The other one made it into next:
https://github.com/git/git/commit/91a67b86f77

I do not think reverting it and requeuing

https://lore.kernel.org/git/[email protected]/

would help future folks why we ignore _MSC_VER as any sign usable to detect endianness, so I'd prefer to see a patch *on top* of 1af265f0 (compat/bswap.h: simplify MSVC endianness detection, 2020-11-08), which is 91a67b86f77^2, that explains why we prefer to list archs explicitly in its log message, which would be the primary value of that commit.

Something along this line, perhaps?

-- >8 --

Subject: compat/bswap.h: do not assume MSVC is little-endian only

Earlier, with 1af265f0 (compat/bswap.h: simplify MSVC endianness detection, 2020-11-08), we tried to simplify endianness detection used in compat/bswap.h by assuming that any version Git compiled by MSVC (detected by _MSC_VER preprocessor macro) is meant to run on little endian boxes, as the versions of old MSVC that support m68k and MIPS do not support some C99 features used in the codebase anyway.

While it might hold true that modern versions of Windows are all little-endian, MSVC is and/or can be ported to build for big-endian boxes, so tying _MSC_VER with endianness is a bit too restrictive.

Let's go back to the old way to use _MSC_VER to learn what preprocessor macros compiler uses to tell us which arch we are building for, and list these arches that are little-endian explicitly.

Quoted 5 lines

... signed-off-by from you and helped-by from others ...

---
diffstat
patch

Quoted 3 lines

Daniel's patch adds _M_ARM64 to the list, but do we need to do anything further to tell the endian on such a bi-endian arch, or does MSVC only support little-endian for that architecture?

Yes, MSVC only supports little-endian for that architecture. From
https://docs.microsoft.com/en-us/cpp/build/arm64-windows-abi-conventions?view=msvc-160#endianness:

Endianness

As with the ARM32 version of Windows, on ARM64 Windows executes in little-endian mode. Switching endianness is difficult to achieve without kernel mode support in AArch64, so it's easier to enforce.

Quoted 4 lines

Just double-checking as the "confusion" that started this thread came from an assumption that MSVC == Windows == big-endian, and you told us MSVC != Windows. Now the patch assumes ARM64-on-MSVC is little-endian only and we want to make sure that assumption is true.

I did not say that MSVC != Windows, but Visual Studio != Windows. But I did say that I do not want to assume MSVC == Windows for all eternity.

Quoted 9 lines

And perhaps it is worth documenting in the log, perhaps

... that are little-endian explicitly. Note that ARM64 is bi-endian in nature but we treat it little-endian as MSVC does not treat the arch as bi-endian.

or something like that at the end (I do not know what MSVC actually does---just illustrating the level of details I expect in the explanation).

Absolutely. If nobody beats me to the punch, I hope to get around to prepare the patch that you suggested.

Thanks,
Dscho

#12 lore raw

Re: [PATCH v2] compat/bswap.h: simplify MSVC endianness detection

In reply to brian m. carlson
Quoted 8 lines · brian m. carlson
Quoted 5 lines
Quoted 2 lines

Modern MSVC or Windows versions don't support big-endian, so it's unnecessary to consider architectures when using it.

This made me wonder if we support any non-modern versions (which would be negatively impacted).

I'm pretty sure we don't. As I said, we're using several C99 features and that version precedes the C99 standard (and 1999).

Your response was much more thorough than I anticipated. What I was really going for in the commit message was just laying out "modern" a bit more clearly, like:

No version of MSVC or Windows has supported a big-endian platform since the mid-90's. Git wouldn't build with these pre-C99 compilers these days anyway, so we can assume that MSVC is always little-endian.

Quoted 5 lines

I'm fine leaving the commit message as it stands, given the brevity of the patch and that in the technology field, the affected versions are not in any way "modern," but of course I wouldn't object to a reroll. It's fine, should that happen, to include any of this email in the commit message.

I often find that the briefer the patch, the more I need to revisit the assumptions of the author (myself included!). :)

But I am also OK either way; when commit messages are insufficient, I often go back to the list archive to get more details, and now it's well documented here.

-Peff