pending bug fixes for nolibc
Subsystems (2)
Willy Tarreau
maintainercover pending bug fixes for nolibc
Hello Paul,
please consider the current patch series for merging into your fixes queue. The intent is to get them before 6.2, then backported where relevant.
It addresses the following bugs:
- fd_set was incorrectly defined as arrays of u32 instead of long, which breaks BE64. Fix courtesy of Sven Schnelle.
- S_ISxxx macros were incorrectly testing the bits after applying them instead of applying S_ISFMT to the value. Fix from Warner Losh.
- the mips code was randomly broken due to an unprotected "noreorder" directive in the _start code that would prevent the assembler from filling delayed slots, and randomly leaving other instructions there
- since the split of the single include file into multiple files, we're implicitly refraining from including some which are not explicitly added in the code. It causes build failures when such files contain definitions for functions that may be used e.g. by libgcc, such as raise() or memset(), which are often called only by a few archs at certain optimization levels only.
- gcc 11.3 in ARM thumb2 mode at -O2 was able to recognize a memset() construction inside the memset() definition, and it replaced it with a call to... memset(). We cannot impose to userland to build with -ffreestanding so the introduction of an empty asm() statement in the loop was enough to stop this.
- most of the O_* macros were wrong on RISCV because their octal value was used as a hexadecimal one when the platform was introduced. This was revealed by the selftest breaking in getdents64().
The series was tested on x86_64, i386, armv5, armv7, thumb1, thumb2, mips and riscv, all at -O0, -Os and -O3. This is based on the "nolibc" branch of your linux-rcu tree. Do not hesitate to let me know if you prefer that I rebase it on a different one.
Thank you!
Willy
---
Sven Schnelle (1):
nolibc: fix fd_set type
Warner Losh (1):
tools/nolibc: Fix S_ISxxx macros
Willy Tarreau (4):
tools/nolibc: restore mips branch ordering in the _start block tools/nolibc: fix missing includes causing build issues at -O0 tools/nolibc: prevent gcc from making memset() loop over itself tools/nolibc: fix the O_* fcntl/open macro definitions for riscv
tools/include/nolibc/arch-mips.h | 2 + tools/include/nolibc/arch-riscv.h | 14 +++---- tools/include/nolibc/ctype.h | 3 ++ tools/include/nolibc/errno.h | 3 ++ tools/include/nolibc/signal.h | 3 ++ tools/include/nolibc/stdio.h | 3 ++ tools/include/nolibc/stdlib.h | 3 ++ tools/include/nolibc/string.h | 8 +++- tools/include/nolibc/sys.h | 2 + tools/include/nolibc/time.h | 3 ++ tools/include/nolibc/types.h | 70 ++++++++++++++++++------------- tools/include/nolibc/unistd.h | 3 ++ 12 files changed, 79 insertions(+), 38 deletions(-)
Signature · 1 line
2.35.3
Willy Tarreau
maintainerPATCH 1/6 nolibc: fix fd_set type
From: Sven Schnelle <[email protected]>
The kernel uses unsigned long for the fd_set bitmap, but nolibc use u32. This works fine on little endian machines, but fails on big endian. Convert to unsigned long to fix this.
--- tools/include/nolibc/types.h | 53 ++++++++++++++++++++---------------- 1 file changed, 30 insertions(+), 23 deletions(-)
tools/include/nolibc/types.h +30 -23
@@ -89,39 +89,46 @@#define EXIT_SUCCESS 0#define EXIT_FAILURE 1 +#define FD_SETIDXMASK (8 * sizeof(unsigned long)) +#define FD_SETBITMASK (8 * sizeof(unsigned long)-1) +/* for select() */ typedef struct { -uint32_t fd32[(FD_SETSIZE +31 ) /32 ]; +unsigned long fds[(FD_SETSIZE +FD_SETBITMASK ) /FD_SETIDXMASK ]; }fd_set ; -#define FD_CLR (fd, set) do { \ - fd_set *__set = (set); \ - int __fd = (fd); \ - if (__fd >= 0) \ - __set->fd32[__fd / 32] &= ~(1U << (__fd & 31)); \ +#define FD_CLR (fd, set) do { \ + fd_set *__set = (set); \ + int __fd = (fd); \ + if (__fd >= 0) \ + __set->fds[__fd / FD_SETIDXMASK] &= \ + ~(1U << (__fd & FX_SETBITMASK)); \ } while (0) -#define FD_SET (fd, set) do { \ - fd_set *__set = (set); \ - int __fd = (fd); \ - if (__fd >= 0) \ - __set->fd32[__fd / 32] |= 1U << (__fd & 31); \ +#define FD_SET (fd, set) do { \ + fd_set *__set = (set); \ + int __fd = (fd); \ + if (__fd >= 0) \ + __set->fds[__fd / FD_SETIDXMASK] |= \ + 1 << (__fd & FD_SETBITMASK); \ } while (0) -#define FD_ISSET (fd, set) ({ \ - fd_set *__set = (set); \ - int __fd = (fd); \ - int __r = 0; \ - if (__fd >= 0) \ - __r = !!(__set->fd32[__fd / 32] & 1U << (__fd & 31)); \ - __r; \ +#define FD_ISSET (fd, set) ({ \ + fd_set *__set = (set); \ + int __fd = (fd); \ + int __r = 0; \ + if (__fd >= 0) \ + __r = !!(__set->fds[__fd / FD_SETIDXMASK] & \ +1U << (__fd & FD_SET_BITMASK)); \ + __r; \ }) -#define FD_ZERO (set) do { \ - fd_set *__set = (set); \ - int __idx; \ - for (__idx = 0; __idx < (FD_SETSIZE+31) / 32; __idx ++) \ - __set->fd32[__idx] = 0; \ +#define FD_ZERO (set) do { \ + fd_set *__set = (set); \ + int __idx; \ + int __size = (FD_SETSIZE+FD_SETBITMASK) / FD_SETIDXMASK;\ + for (__idx = 0; __idx < __size; __idx++) \ + __set->fds[__idx] = 0; \ } while (0)/* for poll() */
Signature · 1 line
2.35.3
Willy Tarreau
maintainerPATCH 2/6 tools/nolibc: Fix S_ISxxx macros
In reply to Willy TarreauFrom: Warner Losh <[email protected]>
The mode field has the type encoded as an value in a field, not as a bit mask. Mask the mode with S_IFMT instead of each type to test. Otherwise, false positives are possible: eg S_ISDIR will return true for block devices because S_IFDIR = 0040000 and S_IFBLK = 0060000 since mode is masked with S_IFDIR instead of S_IFMT. These macros now match the similar definitions in tools/include/uapi/linux/stat.h.
--- tools/include/nolibc/types.h | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
tools/include/nolibc/types.h +7 -7
@@ -26,13 +26,13 @@#define S_IFSOCK 0140000#define S_IFMT 0170000 -#define S_ISDIR (mode) (((mode) & S_IFDIR) == S_IFDIR) -#define S_ISCHR (mode) (((mode) & S_IFCHR) == S_IFCHR) -#define S_ISBLK (mode) (((mode) & S_IFBLK) == S_IFBLK) -#define S_ISREG (mode) (((mode) & S_IFREG) == S_IFREG) -#define S_ISFIFO (mode) (((mode) & S_IFIFO) == S_IFIFO) -#define S_ISLNK (mode) (((mode) & S_IFLNK) == S_IFLNK) -#define S_ISSOCK (mode) (((mode) & S_IFSOCK) == S_IFSOCK) +#define S_ISDIR (mode) (((mode) & S_IFMT) == S_IFDIR) +#define S_ISCHR (mode) (((mode) & S_IFMT) == S_IFCHR) +#define S_ISBLK (mode) (((mode) & S_IFMT) == S_IFBLK) +#define S_ISREG (mode) (((mode) & S_IFMT) == S_IFREG) +#define S_ISFIFO (mode) (((mode) & S_IFMT) == S_IFIFO) +#define S_ISLNK (mode) (((mode) & S_IFMT) == S_IFLNK) +#define S_ISSOCK (mode) (((mode) & S_IFMT) == S_IFSOCK)/* dirent types */ #define DT_UNKNOWN 0x0
Signature · 1 line
2.35.3
Willy Tarreau
maintainerPATCH 3/6 tools/nolibc: restore mips branch ordering in the _start block
In reply to Willy TarreauDepending on the compiler used and the optimization options, the sbrk() test was crashing, both on real hardware (mips-24kc) and in qemu. One such example is kernel.org toolchain in version 11.3 optimizing at -Os.
Inspecting the sys_brk() call shows the following code:
0040047c <sys_brk>:
40047c: 24020fcd li v0,4045
400480: 27bdffe0 addiu sp,sp,-32
400484: 0000000c syscall
400488: 27bd0020 addiu sp,sp,32
40048c: 10e00001 beqz a3,400494 <sys_brk+0x18>
400490: 00021023 negu v0,v0
400494: 03e00008 jr raIt is obviously wrong, the "negu" instruction is placed in beqz's delayed slot, and worse, there's no nop nor instruction after the return, so the next function's first instruction (addiu sip,sip,-32) will also be executed as part of the delayed slot that follows the return.
This is caused by the ".set noreorder" directive in the _start block, that applies to the whole program. The compiler emits code without the delayed slots and relies on the compiler to swap instructions when this option is not set. Removing the option would require to change the startup code in a way that wouldn't make it look like the resulting code, which would not be easy to debug. Instead let's just save the default ordering before changing it, and restore it at the end of the _start block. Now the code is correct:
0040047c <sys_brk>:
40047c: 24020fcd li v0,4045
400480: 27bdffe0 addiu sp,sp,-32
400484: 0000000c syscall
400488: 10e00002 beqz a3,400494 <sys_brk+0x18>
40048c: 27bd0020 addiu sp,sp,32
400490: 00021023 negu v0,v0
400494: 03e00008 jr ra
400498: 00000000 nop--- tools/include/nolibc/arch-mips.h | 2 ++ 1 file changed, 2 insertions(+)
tools/include/nolibc/arch-mips.h +2 -0
@@ -192,6 +192,7 @@ struct sys_stat_struct { __asm__ (".section .text\n" ".weak __start\n" ".set nomips16\n" +".set push\n" ".set noreorder\n" ".option pic0\n" ".ent __start\n"
@@ -210,6 +211,7 @@ __asm__ (".section .text\n""li $v0, 4001\n" // NR_exit == 4001 "syscall\n" ".end __start\n" +".set pop\n" "");#endif // _NOLIBC_ARCH_MIPS_H
Signature · 1 line
2.35.3
Willy Tarreau
maintainerPATCH 4/6 tools/nolibc: fix missing includes causing build issues at -O0
In reply to Willy TarreauAfter the nolibc includes were split to facilitate portability from standard libcs, programs that include only what they need may miss some symbols which are needed by libgcc. This is the case for raise() which is needed by the divide by zero code in some architectures for example.
Regardless, being able to include only the apparently needed files is convenient.
Instead of trying to move all exported definitions to a single file, since this can change over time, this patch takes another approach consisting in including the nolibc header at the end of all standard include files. This way their types and functions are already known at the moment of inclusion, and including any single one of them is sufficient to bring all the required ones.
--- tools/include/nolibc/ctype.h | 3 +++ tools/include/nolibc/errno.h | 3 +++ tools/include/nolibc/signal.h | 3 +++ tools/include/nolibc/stdio.h | 3 +++ tools/include/nolibc/stdlib.h | 3 +++ tools/include/nolibc/string.h | 3 +++ tools/include/nolibc/sys.h | 2 ++ tools/include/nolibc/time.h | 3 +++ tools/include/nolibc/types.h | 3 +++ tools/include/nolibc/unistd.h | 3 +++ 10 files changed, 29 insertions(+)
tools/include/nolibc/ctype.h +3 -0
@@ -96,4 +96,7 @@ int ispunct(int c) return isgraph(c) && !isalnum(c); } +/* make sure to include all global symbols */ +#include "nolibc.h" +#endif /* _NOLIBC_CTYPE_H */
tools/include/nolibc/errno.h +3 -0
@@ -24,4 +24,7 @@ static int errno;*/ #define MAX_ERRNO 4095 +/* make sure to include all global symbols */ +#include "nolibc.h" +#endif /* _NOLIBC_ERRNO_H */
tools/include/nolibc/signal.h +3 -0
@@ -19,4 +19,7 @@ int raise(int signal) return sys_kill(sys_getpid(), signal); } +/* make sure to include all global symbols */ +#include "nolibc.h" +#endif /* _NOLIBC_SIGNAL_H */
tools/include/nolibc/stdio.h +3 -0
@@ -303,4 +303,7 @@ void perror(const char *msg) fprintf(stderr, "%s%serrno=%d\n", (msg && *msg) ? msg : "", (msg && *msg) ? ": " : "", errno); } +/* make sure to include all global symbols */ +#include "nolibc.h" +#endif /* _NOLIBC_STDIO_H */
tools/include/nolibc/stdlib.h +3 -0
@@ -419,4 +419,7 @@ char *u64toa(uint64_t in) return itoa_buffer; } +/* make sure to include all global symbols */ +#include "nolibc.h" +#endif /* _NOLIBC_STDLIB_H */
tools/include/nolibc/string.h +3 -0
@@ -285,4 +285,7 @@ char *strrchr(const char *s, int c) return (char *)ret; } +/* make sure to include all global symbols */ +#include "nolibc.h" +#endif /* _NOLIBC_STRING_H */
tools/include/nolibc/sys.h +2 -0
@@ -1243,5 +1243,7 @@ ssize_t write(int fd, const void *buf, size_t count) return ret; } +/* make sure to include all global symbols */ +#include "nolibc.h" #endif /* _NOLIBC_SYS_H */
tools/include/nolibc/time.h +3 -0
@@ -25,4 +25,7 @@ time_t time(time_t *tptr) return tv.tv_sec; } +/* make sure to include all global symbols */ +#include "nolibc.h" +#endif /* _NOLIBC_TIME_H */
tools/include/nolibc/types.h +3 -0
@@ -209,4 +209,7 @@ struct stat { })#endif +/* make sure to include all global symbols */ +#include "nolibc.h" +#endif /* _NOLIBC_TYPES_H */
tools/include/nolibc/unistd.h +3 -0
@@ -51,4 +51,7 @@ int tcsetpgrp(int fd, pid_t pid) return ioctl(fd, TIOCSPGRP, &pid); } +/* make sure to include all global symbols */ +#include "nolibc.h" +#endif /* _NOLIBC_UNISTD_H */
Signature · 1 line
2.35.3
Willy Tarreau
maintainerPATCH 5/6 tools/nolibc: prevent gcc from making memset() loop over itself
In reply to Willy TarreauWhen building on ARM in thumb mode with gcc-11.3 at -O2 or -O3, nolibc-test segfaults during the select() tests. It turns out that at this level, gcc recognizes an opportunity for using memset() to zero the fd_set, but it miscompiles it because it also recognizes a memset pattern as well, and decides to call memset() from the memset() code:
000122bc <memset>:
122bc: b510 push {r4, lr}
122be: 0004 movs r4, r0
122c0: 2a00 cmp r2, #0
122c2: d003 beq.n 122cc <memset+0x10>
122c4: 23ff movs r3, #255 ; 0xff
122c6: 4019 ands r1, r3
122c8: f7ff fff8 bl 122bc <memset>
122cc: 0020 movs r0, r4
122ce: bd10 pop {r4, pc}Simply placing an empty asm() statement inside the loop suffices to avoid this.
--- tools/include/nolibc/string.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
tools/include/nolibc/string.h +4 -1
@@ -88,8 +88,11 @@ void *memset(void *dst, int b, size_t len) {char *p = dst; -while (len--) +while (len--) { +/* prevent gcc from recognizing memset() here */ + asmvolatile ("" ); *(p++) = b; + }return dst; }
Signature · 1 line
2.35.3
Willy Tarreau
maintainerPATCH 6/6 tools/nolibc: fix the O_* fcntl/open macro definitions for riscv
In reply to Willy TarreauWhen RISCV port was imported in 5.2, the O_* macros were taken with their octal value and written as-is in hex, resulting in the getdents64() to fail in nolibc-test.
--- tools/include/nolibc/arch-riscv.h | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
tools/include/nolibc/arch-riscv.h +7 -7
@@ -11,13 +11,13 @@#define O_RDONLY 0#define O_WRONLY 1#define O_RDWR 2 -#define O_CREAT 0x100 -#define O_EXCL 0x200 -#define O_NOCTTY 0x400 -#define O_TRUNC 0x1000 -#define O_APPEND 0x2000 -#define O_NONBLOCK 0x4000 -#define O_DIRECTORY 0x200000 +#define O_CREAT 0x40 +#define O_EXCL 0x80 +#define O_NOCTTY 0x100 +#define O_TRUNC 0x200 +#define O_APPEND 0x400 +#define O_NONBLOCK 0x800 +#define O_DIRECTORY 0x10000struct sys_stat_struct {unsigned long st_dev;/* Device. */
Signature · 1 line
2.35.3
Quoted 29 lines · Willy Tarreau
Hello Paul,
please consider the current patch series for merging into your fixes queue. The intent is to get them before 6.2, then backported where relevant.
It addresses the following bugs:
-fd_setwas incorrectly defined as arrays of u32 instead of long, which breaks BE64. Fix courtesy of Sven Schnelle.-
S_ISxxxmacros were incorrectly testing the bits after applying them instead of applyingS_ISFMTto the value. Fix from Warner Losh.- the mips code was randomly broken due to an unprotected "noreorder" directive in the
_startcode that would prevent the assembler from filling delayed slots, and randomly leaving other instructions there- since the split of the single include file into multiple files, we're implicitly refraining from including some which are not explicitly added in the code. It causes build failures when such files contain definitions for functions that may be used e.g. by libgcc, such as
raise()ormemset(), which are often called only by a few archs at certain optimization levels only.- gcc 11.3 in ARM thumb2 mode at -O2 was able to recognize a
memset()construction inside thememset()definition, and it replaced it with a call to...memset(). We cannot impose to userland to build with -ffreestanding so the introduction of an emptyasm()statement in the loop was enough to stop this.
Quoted 8 lines
- most of the O_* macros were wrong on RISCV because their octal value was used as a hexadecimal one when the platform was introduced. This was revealed by the selftest breaking in
getdents64().The series was tested on
x86_64, i386, armv5, armv7, thumb1, thumb2, mips and riscv, all at -O0, -Os and -O3. This is based on the "nolibc" branch of your linux-rcu tree. Do not hesitate to let me know if you prefer that I rebase it on a different one.
"81 test(s) passed", so queued at urgent-nolibc.2023.01.09a, thank you!
Also, thank you for the detailed cover letter, which I co-opted into the signed tag. But please check to make sure that my wordsmithing didn't break anything.
If all goes well, I will send the pull request to Linus before the end of this week.
Thanx, Paul
Quoted 33 lines
Thank you!
Willy---
Sven Schnelle (1):
nolibc: fixfd_settypeWarner Losh (1):
tools/nolibc: FixS_ISxxxmacrosWilly Tarreau (4):
tools/nolibc: restore mips branch ordering in the_startblocktools/nolibc: fix missing includes causing build issues at -O0tools/nolibc: prevent gcc from makingmemset()loop over itselftools/nolibc: fix the O_* fcntl/open macro definitions for riscvtools/include/nolibc/arch-mips.h | 2 + tools/include/nolibc/arch-riscv.h | 14 +++---- tools/include/nolibc/ctype.h | 3 ++ tools/include/nolibc/errno.h | 3 ++ tools/include/nolibc/signal.h | 3 ++ tools/include/nolibc/stdio.h | 3 ++ tools/include/nolibc/stdlib.h | 3 ++ tools/include/nolibc/string.h | 8 +++- tools/include/nolibc/sys.h | 2 + tools/include/nolibc/time.h | 3 ++ tools/include/nolibc/types.h | 70 ++++++++++++++++++------------- tools/include/nolibc/unistd.h | 3 ++ 12 files changed, 79 insertions(+), 38 deletions(-)Signature · 1 line
2.35.3
Willy Tarreau
maintainerHello Paul,
Quoted 39 lines · Paul E. McKenney
Quoted 29 lines · Willy Tarreau
Hello Paul,
please consider the current patch series for merging into your fixes queue. The intent is to get them before 6.2, then backported where relevant.
It addresses the following bugs:
-fd_setwas incorrectly defined as arrays of u32 instead of long, which breaks BE64. Fix courtesy of Sven Schnelle.-
S_ISxxxmacros were incorrectly testing the bits after applying them instead of applyingS_ISFMTto the value. Fix from Warner Losh.- the mips code was randomly broken due to an unprotected "noreorder" directive in the
_startcode that would prevent the assembler from filling delayed slots, and randomly leaving other instructions there- since the split of the single include file into multiple files, we're implicitly refraining from including some which are not explicitly added in the code. It causes build failures when such files contain definitions for functions that may be used e.g. by libgcc, such as
raise()ormemset(), which are often called only by a few archs at certain optimization levels only.- gcc 11.3 in ARM thumb2 mode at -O2 was able to recognize a
memset()construction inside thememset()definition, and it replaced it with a call to...memset(). We cannot impose to userland to build with -ffreestanding so the introduction of an emptyasm()statement in the loop was enough to stop this.Quoted 8 lines
- most of the O_* macros were wrong on RISCV because their octal value was used as a hexadecimal one when the platform was introduced. This was revealed by the selftest breaking in
getdents64().The series was tested on
x86_64, i386, armv5, armv7, thumb1, thumb2, mips and riscv, all at -O0, -Os and -O3. This is based on the "nolibc" branch of your linux-rcu tree. Do not hesitate to let me know if you prefer that I rebase it on a different one.
Quoted 4 lines
"81 test(s) passed", so queued at urgent-nolibc.2023.01.09a, thank you!
Also, thank you for the detailed cover letter, which I co-opted into the signed tag.
You're welcome!
Quoted 2 lines
But please check to make sure that my wordsmithing didn't break anything.
It all looks perfect to me.
Quoted 2 lines
If all goes well, I will send the pull request to Linus before the end of this week.
Great, thank you!
Willy