Skip to contentSkip to conversation

nolibc: small updates required for the self-tests

PATCH In review Willy Tarreau 2 participants 12 messages Mar 21, 2022 Nolibc header file

lore
Share & tools
lore

Willy Tarreau

author maintainer
#0 lore raw

cover nolibc: small updates required for the self-tests

Hello Paul,

while developing the nolibc self-tests, I naturally faced a few limitations ranging from missing POLL* flags to missing strcmp() or getenv(), as well as the ability for printf() to print "(null)" on "%s" instead of crashing.

This series adds a makefile and a headers installation target that creates a sysroot made of kernel headers and nolibc headers, which significantly ease building programs.

I already have some test code with roughly 75 tests, but I find it misplaced in the nolibc dir, I need to move it to testing/selftests/ before sending it to you.

This series is intended to be an add-on to what you already have in your dev branch. I don't intend to send you much more stuff on top of this, but I expect to see an update from Ammar's patch set (CCed). With all this I think we'll have a good basis to easily add new tests.

Thanks!
Willy

---
Willy Tarreau (8):
tools/nolibc/stdio: make printf(%s) accept NULL tools/nolibc/stdlib: add a simple getenv() implementation tools/nolibc/stdio: add support for '%p' to vfprintf() tools/nolibc/string: add strcmp() and strncmp() tools/nolibc/sys: add syscall definition for getppid() tools/nolibc/types: add poll() and waitpid() flag definitions tools/nolibc: add a makefile to install headers tools/nolibc: add the nolibc subdir to the common Makefile

 tools/Makefile                |  4 ++++
 tools/include/nolibc/Makefile | 43 +++++++++++++++++++++++++++++++++++
 tools/include/nolibc/stdio.h  | 32 +++++++++++++++++---------
 tools/include/nolibc/stdlib.h | 23 +++++++++++++++++++
 tools/include/nolibc/string.h | 23 +++++++++++++++++++
 tools/include/nolibc/sys.h    | 17 ++++++++++++++
 tools/include/nolibc/types.h  | 10 ++++++++
 7 files changed, 141 insertions(+), 11 deletions(-)
 create mode 100644 tools/include/nolibc/Makefile
Signature · 1 line
2.35.1
author maintainer
#1 lore raw

PATCH 1/8 tools/nolibc/stdio: make printf(%s) accept NULL

It's often convenient to support this, especially in test programs where a NULL may correspond to an allocation error or a non-existing value. Let's make printf("%s") support being passed a NULL. In this case it prints "(null)" like glibc's printf().

---
 tools/include/nolibc/stdio.h | 2 ++
 1 file changed, 2 insertions(+)
tools/include/nolibc/stdio.h +2 -0
index cb4d3ab3a565..559ebe052a75 100644
@@ -220,6 +220,8 @@ int vfprintf(FILE *stream, const char *fmt, va_list args)
 			}
 			else if (c == 's') {
 				outstr = va_arg(args, char *);
+				if (!outstr)
+					outstr="(null)";
 			}
 			else if (c == '%') {
 				/* queue it verbatim */
Signature · 1 line
2.35.1
author maintainer
#2 lore raw

PATCH 2/8 tools/nolibc/stdlib: add a simple getenv() implementation

In reply to Willy Tarreau

This implementation relies on an extern definition of the environ variable, that the caller must declare and initialize from envp.

---
 tools/include/nolibc/stdlib.h | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
tools/include/nolibc/stdlib.h +23 -0
index 733105c574ee..aca8616335e3 100644
@@ -60,6 +60,29 @@ int atoi(const char *s)
 	return atol(s);
 }
 
+/* Tries to find the environment variable named <name> in the environment array
+ * pointed to by global variable "environ" which must be declared as a char **,
+ * and must be terminated by a NULL (it is recommended to set this variable to
+ * the "envp" argument of main()). If the requested environment variable exists
+ * its value is returned otherwise NULL is returned.
+ */
+static __attribute__((unused))
+char *getenv(const char *name)
+{
+	extern char **environ;
+	int idx, i;
+
+	if (environ) {
+		for (idx = 0; environ[idx]; idx++) {
+			for (i = 0; name[i] && name[i] == environ[idx][i];)
+				i++;
+			if (!name[i] && environ[idx][i] == '=')
+				return &environ[idx][i+1];
+		}
+	}
+	return NULL;
+}
+
 /* Converts the unsigned long integer <in> to its hex representation into
  * buffer <buffer>, which must be long enough to store the number and the
  * trailing zero (17 bytes for "ffffffffffffffff" or 9 for "ffffffff"). The
Signature · 1 line
2.35.1
author maintainer
#3 lore raw

PATCH 3/8 tools/nolibc/stdio: add support for '%p' to vfprintf()

In reply to Willy Tarreau

%p remains quite useful in test code, and the code path can easily be merged with the existing "%x" thus only adds ~50 bytes, thus let's add it.

---
 tools/include/nolibc/stdio.h | 30 +++++++++++++++++++-----------
 1 file changed, 19 insertions(+), 11 deletions(-)
tools/include/nolibc/stdio.h +19 -11
index 559ebe052a75..15dedf8d0902 100644
@@ -163,7 +163,7 @@ char *fgets(char *s, int size, FILE *stream)
 
 
 /* minimal vfprintf(). It supports the following formats:
- *  - %[l*]{d,u,c,x}
+ *  - %[l*]{d,u,c,x,p}
  *  - %s
  *  - unknown modifiers are ignored.
  */
@@ -184,8 +184,12 @@ int vfprintf(FILE *stream, const char *fmt, va_list args)
 		if (escape) {
 			/* we're in an escape sequence, ofs == 1 */
 			escape = 0;
-			if (c == 'c' || c == 'd' || c == 'u' || c == 'x') {
-				if (lpref) {
+			if (c == 'c' || c == 'd' || c == 'u' || c == 'x' || c == 'p') {
+				char *out = tmpbuf;
+
+				if (c == 'p')
+					v = va_arg(args, unsigned long);
+				else if (lpref) {
 					if (lpref > 1)
 						v = va_arg(args, unsigned long long);
 					else
@@ -202,18 +206,22 @@ int vfprintf(FILE *stream, const char *fmt, va_list args)
 				}
 
 				switch (c) {
+				case 'c':
+					out[0] = v;
+					out[1] = 0;
+					break;
 				case 'd':
-					i64toa_r(v, tmpbuf);
+					i64toa_r(v, out);
 					break;
 				case 'u':
-					u64toa_r(v, tmpbuf);
+					u64toa_r(v, out);
 					break;
-				case 'x':
-					u64toh_r(v, tmpbuf);
-					break;
-				default: /* 'c' */
-					tmpbuf[0] = v;
-					tmpbuf[1] = 0;
+				case 'p':
+					*(out++) = '0';
+					*(out++) = 'x';
+					/* fall through */
+				default: /* 'x' and 'p' above */
+					u64toh_r(v, out);
 					break;
 				}
 				outstr = tmpbuf;
Signature · 1 line
2.35.1
author maintainer
#4 lore raw

PATCH 4/8 tools/nolibc/string: add strcmp() and strncmp()

In reply to Willy Tarreau

We need these functions all the time, including when checking environment variables and parsing command-line arguments. These implementations were optimized to show optimal code size on a wide range of compilers (22 bytes return included for strcmp(), 33 for strncmp()).

---
 tools/include/nolibc/string.h | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
tools/include/nolibc/string.h +23 -0
index 4554b6fcb400..0d5e870c7c0b 100644
@@ -102,6 +102,17 @@ char *strchr(const char *s, int c)
 	return NULL;
 }
 
+static __attribute__((unused))
+int strcmp(const char *a, const char *b)
+{
+	unsigned int c;
+	int diff;
+
+	while (!(diff = (unsigned char)*a++ - (c = (unsigned char)*b++)) && c)
+		;
+	return diff;
+}
+
 static __attribute__((unused))
 char *strcpy(char *dst, const char *src)
 {
@@ -184,6 +195,18 @@ char *strncat(char *dst, const char *src, size_t size)
 	return orig;
 }
 
+static __attribute__((unused))
+int strncmp(const char *a, const char *b, size_t size)
+{
+	unsigned int c;
+	int diff = 0;
+
+	while (size-- &&
+	       !(diff = (unsigned char)*a++ - (c = (unsigned char)*b++)) && c)
+		;
+
+	return diff;
+}
 
 static __attribute__((unused))
 char *strncpy(char *dst, const char *src, size_t size)
Signature · 1 line
2.35.1
author maintainer
#5 lore raw

PATCH 5/8 tools/nolibc/sys: add syscall definition for getppid()

In reply to Willy Tarreau

This is essentially for completeness as it's not the most often used in regtests.

---
 tools/include/nolibc/sys.h | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
tools/include/nolibc/sys.h +17 -0
index 28437863c63f..4d4308d5d111 100644
@@ -464,6 +464,23 @@ pid_t getpid(void)
 }
 
 
+/*
+ * pid_t getppid(void);
+ */
+
+static __attribute__((unused))
+pid_t sys_getppid(void)
+{
+	return my_syscall0(__NR_getppid);
+}
+
+static __attribute__((unused))
+pid_t getppid(void)
+{
+	return sys_getppid();
+}
+
+
 /*
  * pid_t gettid(void);
  */
Signature · 1 line
2.35.1
author maintainer
#6 lore raw

PATCH 6/8 tools/nolibc/types: add poll() and waitpid() flag definitions

In reply to Willy Tarreau

- POLLIN etc were missing, so poll() could only be used with timeouts.
- WNOHANG was not defined and is convenient to check if a child is still running

---
 tools/include/nolibc/types.h | 10 ++++++++++
 1 file changed, 10 insertions(+)
tools/include/nolibc/types.h +10 -0
index e4026e740b56..357e60ad38a8 100644
@@ -82,6 +82,9 @@
 #define WEXITSTATUS(status) (((status) & 0xff00) >> 8)
 #define WIFEXITED(status)   (((status) & 0x7f) == 0)
 
+/* waitpid() flags */
+#define WNOHANG      1
+
 /* standard exit() codes */
 #define EXIT_SUCCESS 0
 #define EXIT_FAILURE 1
@@ -122,6 +125,13 @@ typedef struct {
 	} while (0)
 
 /* for poll() */
+#define POLLIN          0x0001
+#define POLLPRI         0x0002
+#define POLLOUT         0x0004
+#define POLLERR         0x0008
+#define POLLHUP         0x0010
+#define POLLNVAL        0x0020
+
 struct pollfd {
 	int fd;
 	short int events;
Signature · 1 line
2.35.1
author maintainer
#7 lore raw

PATCH 7/8 tools/nolibc: add a makefile to install headers

In reply to Willy Tarreau

This provides a target "headers_standalone" which installs the nolibc's arch-specific headers with "arch.h" taken from the current arch (or a concatenation of both i386 and x86_64 for arch=x86), then installs kernel headers. This creates a convenient sysroot which is directly usable by a bare-metal compiler to create any executable.

---
 tools/include/nolibc/Makefile | 43 +++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
 create mode 100644 tools/include/nolibc/Makefile
tools/include/nolibc/Makefile new file +43 -0
new file mode 100644
index 000000000000..7d4d61b08637
@@ -0,0 +1,43 @@
+# SPDX-License-Identifier: GPL-2.0
+# Makefile for nolibc installation and tests
+include ../../scripts/Makefile.include
+
+# we're in ".../tools/include/nolibc"
+ifeq ($(srctree),)
+srctree := $(patsubst %/tools/include/,%,$(dir $(CURDIR)))
+endif
+
+nolibc_arch := $(patsubst arm64,aarch64,$(ARCH))
+arch_file := arch-$(nolibc_arch).h
+all_files := ctype.h errno.h nolibc.h signal.h std.h stdio.h stdlib.h string.h \
+             sys.h time.h types.h unistd.h
+
+# install all headers needed to support a bare-metal compiler
+all:
+
+# Note: when ARCH is "x86" we concatenate both x86_64 and i386
+headers:
+	$(Q)mkdir -p $(OUTPUT)sysroot
+	$(Q)mkdir -p $(OUTPUT)sysroot/include
+	$(Q)cp $(all_files) $(OUTPUT)sysroot/include/
+	$(Q)if [ "$(ARCH)" = "x86" ]; then      \
+		sed -e                          \
+		  's,^#ifndef _NOLIBC_ARCH_X86_64_H,#if !defined(_NOLIBC_ARCH_X86_64_H) \&\& defined(__x86_64__),' \
+		  arch-x86_64.h;                \
+		sed -e                          \
+		  's,^#ifndef _NOLIBC_ARCH_I386_H,#if !defined(_NOLIBC_ARCH_I386_H) \&\& !defined(__x86_64__),' \
+		  arch-i386.h;                  \
+	elif [ -e "$(arch_file)" ]; then        \
+		cat $(arch_file);               \
+	else                                    \
+		echo "Fatal: architecture $(ARCH) not yet supported by nolibc." >&2; \
+		exit 1;                         \
+	fi > $(OUTPUT)sysroot/include/arch.h
+
+headers_standalone: headers
+	$(Q)$(MAKE) -C $(srctree) headers
+	$(Q)$(MAKE) -C $(srctree) headers_install INSTALL_HDR_PATH=$(OUTPUT)/sysroot
+
+clean:
+	$(call QUIET_CLEAN, nolibc) rm -rf "$(OUTPUT)sysroot"
+
Signature · 1 line
2.35.1
author maintainer
#8 lore raw

PATCH 8/8 tools/nolibc: add the nolibc subdir to the common Makefile

In reply to Willy Tarreau

The Makefile in tools/ is used to forward options to the makefiles in the various subdirs. Let's add nolibc there so that it becomes possible to make tools/nolibc_headers_standalone from the main tree to simply create a completely usable sysroot.

---
 tools/Makefile | 4 ++++
 1 file changed, 4 insertions(+)
tools/Makefile +4 -0
index db2f7b8ebed5..724134f0e56c 100644
@@ -24,6 +24,7 @@ help:
 	@echo '  intel-speed-select     - Intel Speed Select tool'
 	@echo '  kvm_stat               - top-like utility for displaying kvm statistics'
 	@echo '  leds                   - LEDs  tools'
+	@echo '  nolibc                 - nolibc headers testing and installation'
 	@echo '  objtool                - an ELF object analysis tool'
 	@echo '  pci                    - PCI tools'
 	@echo '  perf                   - Linux performance measurement and analysis tool'
@@ -74,6 +75,9 @@ bpf/%: FORCE
 libapi: FORCE
 	$(call descend,lib/api)
 
+nolibc_%: FORCE
+	$(call descend,include/nolibc,$(patsubst nolibc_%,%,$@))
+
 # The perf build does not follow the descend function setup,
 # invoking it via it's own make rule.
 PERF_O   = $(if $(O),$(O)/tools/perf,)
Signature · 1 line
2.35.1
#9 lore raw
In reply to Willy Tarreau
Quoted 11 lines · Willy Tarreau

Hello Paul,

while developing the nolibc self-tests, I naturally faced a few limitations ranging from missing POLL* flags to missing strcmp() or getenv(), as well as the ability for printf() to print "(null)" on "%s" instead of crashing.

This series adds a makefile and a headers installation target that creates a sysroot made of kernel headers and nolibc headers, which significantly ease building programs.

Quoted 8 lines

I already have some test code with roughly 75 tests, but I find it misplaced in the nolibc dir, I need to move it to testing/selftests/ before sending it to you.

This series is intended to be an add-on to what you already have in your dev branch. I don't intend to send you much more stuff on top of this, but I expect to see an update from Ammar's patch set (CCed). With all this I think we'll have a good basis to easily add new tests.

I have pulled this series in for review and testing, thank you!

It won't go into -next until v5.18-rc1 comes out, in about two weeks, but it will at least be in -rcu.

I removed an ostensibly extraneous blank line from the end of tools/include/nolibc/Makefile. So please let me know if that blank line is actually necessary.

Thanx, Paul

Quoted 27 lines

Thanks!
Willy

---
Willy Tarreau (8):
tools/nolibc/stdio: make printf(%s) accept NULL tools/nolibc/stdlib: add a simple getenv() implementation tools/nolibc/stdio: add support for '%p' to vfprintf() tools/nolibc/string: add strcmp() and strncmp() tools/nolibc/sys: add syscall definition for getppid() tools/nolibc/types: add poll() and waitpid() flag definitions tools/nolibc: add a makefile to install headers tools/nolibc: add the nolibc subdir to the common Makefile

 tools/Makefile                |  4 ++++
 tools/include/nolibc/Makefile | 43 +++++++++++++++++++++++++++++++++++
 tools/include/nolibc/stdio.h  | 32 +++++++++++++++++---------
 tools/include/nolibc/stdlib.h | 23 +++++++++++++++++++
 tools/include/nolibc/string.h | 23 +++++++++++++++++++
 tools/include/nolibc/sys.h    | 17 ++++++++++++++
 tools/include/nolibc/types.h  | 10 ++++++++
 7 files changed, 141 insertions(+), 11 deletions(-)
 create mode 100644 tools/include/nolibc/Makefile
Signature · 1 line
2.35.1
author maintainer
#10 lore raw
Quoted 21 lines · Paul E. McKenney
Quoted 11 lines · Willy Tarreau

Hello Paul,

while developing the nolibc self-tests, I naturally faced a few limitations ranging from missing POLL* flags to missing strcmp() or getenv(), as well as the ability for printf() to print "(null)" on "%s" instead of crashing.

This series adds a makefile and a headers installation target that creates a sysroot made of kernel headers and nolibc headers, which significantly ease building programs.

Quoted 8 lines

I already have some test code with roughly 75 tests, but I find it misplaced in the nolibc dir, I need to move it to testing/selftests/ before sending it to you.

This series is intended to be an add-on to what you already have in your dev branch. I don't intend to send you much more stuff on top of this, but I expect to see an update from Ammar's patch set (CCed). With all this I think we'll have a good basis to easily add new tests.

Quoted 4 lines

I have pulled this series in for review and testing, thank you!

It won't go into -next until v5.18-rc1 comes out, in about two weeks, but it will at least be in -rcu.

Perfect, thank you!

Quoted 3 lines

I removed an ostensibly extraneous blank line from the end of tools/include/nolibc/Makefile. So please let me know if that blank line is actually necessary.

Oh I'm very sorry, that the type of thing I'm very careful about so it has definitely escaped my checks.

Thanks,
Willy

#11 lore raw
Quoted 27 lines · Willy Tarreau
Quoted 21 lines · Paul E. McKenney
Quoted 11 lines · Willy Tarreau

Hello Paul,

while developing the nolibc self-tests, I naturally faced a few limitations ranging from missing POLL* flags to missing strcmp() or getenv(), as well as the ability for printf() to print "(null)" on "%s" instead of crashing.

This series adds a makefile and a headers installation target that creates a sysroot made of kernel headers and nolibc headers, which significantly ease building programs.

Quoted 8 lines

I already have some test code with roughly 75 tests, but I find it misplaced in the nolibc dir, I need to move it to testing/selftests/ before sending it to you.

This series is intended to be an add-on to what you already have in your dev branch. I don't intend to send you much more stuff on top of this, but I expect to see an update from Ammar's patch set (CCed). With all this I think we'll have a good basis to easily add new tests.

Quoted 4 lines

I have pulled this series in for review and testing, thank you!

It won't go into -next until v5.18-rc1 comes out, in about two weeks, but it will at least be in -rcu.

Quoted 8 lines

Perfect, thank you!

Quoted 3 lines

I removed an ostensibly extraneous blank line from the end of tools/include/nolibc/Makefile. So please let me know if that blank line is actually necessary.

Oh I'm very sorry, that the type of thing I'm very careful about so it has definitely escaped my checks.

"To err is human!" ;-)

Thanx, Paul