Skip to contentSkip to conversation

Add dynamic memory allocator support for nolibc

PATCH In review Ammar Faizi 3 participants 14 messages Mar 29, 2022 – Mar 31, 2022

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v2 cover Add dynamic memory allocator support for nolibc

Hi,

This is a patchset v2 to add dynamic memory allocator support for nolibc after 2 RFCs, please review the changes carefully.

@@ Changelog:
---
Link v1: https://lore.kernel.org/lkml/[email protected]
v1 -> v2:
- Sync with Paul's tree.
- Drop 2 patches that tried to remove register variables
(comment from Willy).
- Make the patch that replaces asm with __asm__ the
second patch (comment from Willy).

Link RFC v2: https://lore.kernel.org/lkml/[email protected]
RFC v2 -> v1:
- Rebase, sync with Paul's tree.
- Add new 3 patches [PATCH 03/11], [PATCH 04/11], [PATCH 05/11].

[PATCH 02/11]
- Append Reviewed-by tag from Nick.
- s/Removing/remove/

[PATCH 06/11]
- Use the same pattern for syscall6, regardless using GCC or Clang (comment from David).
- Use appropriate constraints for syscall6 instead of always using register variables (comment from David).

[PATCH 09/11]
- Round up the malloc() allocation to 4096 (comment from David).
- Don't realloc() if we still have enough memory to contain the requested new size (comment from David).
- Fix conflict with getenv() fix (after rebase).

Link RFC v1: https://lore.kernel.org/lkml/[email protected]
RFC v1 -> RFC v2:
- Add 2 new patches [PATCH 5/8] and [PATCH 7/8].

[PATCH 2/8]
- Remove all .global _start for all build (GCC and Clang) instead of removing all .weak _start for clang build (Comment from Willy).

[PATCH 3/8]
- Fix %ebp saving method. Don't use redzone, i386 doesn't have a redzone (comment from David and Alviro).

[PATCH 6/8]
- Move container_of() and offsetof() macro to types.h with a separate patch (comment from Willy).

[PATCH 8/8]
- Update strdup and strndup implementation, use strlen and strnlen to get the string length first (comment from Willy and Alviro).
- Fix the subject line prefix, it was "tools/include/string: ", it should be "tools/nolibc/string: ".
- Update the commit message.

---
Ammar Faizi (9):
tools/nolibc: x86-64: Update System V ABI document link tools/nolibc: Replace asm with __asm__
tools/nolibc: Remove .global _start from the entry point code tools/nolibc: i386: Implement syscall with 6 arguments tools/nolibc/sys: Implement mmap() and munmap() tools/nolibc/types: Implement offsetof() and container_of() macro tools/nolibc/stdlib: Implement malloc(), calloc(), realloc() and free() tools/nolibc/string: Implement strnlen()
tools/include/string: Implement strdup() and strndup()

 tools/include/nolibc/arch-aarch64.h | 75 +++++++++++++-------------
 tools/include/nolibc/arch-arm.h     | 59 +++++++++++----------
 tools/include/nolibc/arch-i386.h    | 80 +++++++++++++++++-----------
 tools/include/nolibc/arch-mips.h    | 63 +++++++++++-----------
 tools/include/nolibc/arch-riscv.h   | 75 +++++++++++++-------------
 tools/include/nolibc/arch-x86_64.h  | 75 +++++++++++++-------------
 tools/include/nolibc/stdlib.h       | 81 +++++++++++++++++++++++++++++
 tools/include/nolibc/string.h       | 41 +++++++++++++++
 tools/include/nolibc/sys.h          | 62 ++++++++++++++++++++++
 tools/include/nolibc/types.h        | 11 ++++
 10 files changed, 417 insertions(+), 205 deletions(-)

base-commit: 2eb9d6a49acd4f12078967c33e9786e084fe6407

Signature · 1 line
Ammar Faizi
#1 lore raw

PATCH v2 1/9 tools/nolibc: x86-64: Update System V ABI document link

The old link no longer works, update it.

===

@@ Changelog:

Link v1: https://lore.kernel.org/lkml/[email protected]
v1 -> v2:
* No changes *

Link RFC v2: https://lore.kernel.org/lkml/[email protected]
RFC v2 -> v1:
* No changes *

Link RFC v1: https://lore.kernel.org/lkml/[email protected]
RFC v1 -> RFC v2:
* No changes *

---
 tools/include/nolibc/arch-x86_64.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/include/nolibc/arch-x86_64.h +1 -1
index fe517c16cd4d..a7b70ea51b68 100644
@@ -61,7 +61,7 @@ struct sys_stat_struct {
  *   - see also x86-64 ABI section A.2 AMD64 Linux Kernel Conventions, A.2.1
  *     Calling Conventions.
  *
- * Link x86-64 ABI: https://gitlab.com/x86-psABIs/x86-64-ABI/-/wikis/x86-64-psABI
+ * Link x86-64 ABI: https://gitlab.com/x86-psABIs/x86-64-ABI/-/wikis/home
  *
  */
 
Signature · 1 line
Ammar Faizi
#2 lore raw

PATCH v2 2/9 tools/nolibc: Replace `asm` with `__asm__`

In reply to Ammar Faizi

Replace asm with __asm__ to support compilation with -std flag. Using asm with -std flag makes GCC think asm() is a function call instead of an inline assembly.

GCC doc says:

For the C language, the asm keyword is a GNU extension. When writing C code that can be compiled with -ansi and the -std options that select C dialects without GNU extensions, use __asm__ instead of asm.

===

In a private conversation with Alviro, we found that using -std flag hits the following errors:

```
    In file included from arch.h:21,
                   from nolibc.h:97,
                   from test.c:2:
  arch-i386.h:177:5: error: expected declaration specifiers or ‘...’ before string constant
    177 | asm(".section .text\n"
        |     ^~~~~~~~~~~~~~~~~~
  sys.h: In function ‘sys_brk’:
  arch-i386.h:83:28: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘asm’
     83 |         register long _num asm("eax") = (num);                                \
        |                            ^~~
  sys.h:57:24: note: in expansion of macro ‘my_syscall1’
     57 |         return (void *)my_syscall1(__NR_brk, addr);
        |                        ^~~~~~~~~~~
  arch-i386.h:83:28: warning: implicit declaration of function ‘asm’ [-Wimplicit-function-declaration]
     83 |         register long _num asm("eax") = (num);                                \
        |                            ^~~
  sys.h:57:24: note: in expansion of macro ‘my_syscall1’
     57 |         return (void *)my_syscall1(__NR_brk, addr);
        |                        ^~~~~~~~~~~
  arch-i386.h:83:39: error: lvalue required as left operand of assignment
     83 |         register long _num asm("eax") = (num);                                \
        |                                       ^
  sys.h:57:24: note: in expansion of macro ‘my_syscall1’
     57 |         return (void *)my_syscall1(__NR_brk, addr);
        |                        ^~~~~~~~~~~
  arch-i386.h:84:29: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘asm’
     84 |         register long _arg1 asm("ebx") = (long)(arg1);                        \
```

It is because inline ASM is a GNU extension. So GCC thinks asm() is a function call. To make it always available for any kind of compilation, use __asm__ instead of asm.

@@ Changelog:

Link v1: https://lore.gnuweeb.org/gwml/[email protected]
v1 -> v2:
- Switch the order with the patch that fixes clang build.

---
 tools/include/nolibc/arch-aarch64.h | 74 ++++++++++++++---------------
 tools/include/nolibc/arch-arm.h     | 58 +++++++++++-----------
 tools/include/nolibc/arch-i386.h    | 56 +++++++++++-----------
 tools/include/nolibc/arch-mips.h    | 62 ++++++++++++------------
 tools/include/nolibc/arch-riscv.h   | 74 ++++++++++++++---------------
 tools/include/nolibc/arch-x86_64.h  | 72 ++++++++++++++--------------
 6 files changed, 198 insertions(+), 198 deletions(-)
tools/include/nolibc/arch-aarch64.h +37 -37
index 87d9e434820c..b68443f63980 100644
@@ -64,10 +64,10 @@ struct sys_stat_struct {
 
 #define my_syscall0(num)                                                      \
 ({                                                                            \
-	register long _num  asm("x8") = (num);                                \
-	register long _arg1 asm("x0");                                        \
+	register long _num  __asm__ ("x8") = (num);                           \
+	register long _arg1 __asm__ ("x0");                                   \
 	                                                                      \
-	asm volatile (                                                        \
+	__asm__  volatile (                                                   \
 		"svc #0\n"                                                    \
 		: "=r"(_arg1)                                                 \
 		: "r"(_num)                                                   \
@@ -78,10 +78,10 @@ struct sys_stat_struct {
 
 #define my_syscall1(num, arg1)                                                \
 ({                                                                            \
-	register long _num  asm("x8") = (num);                                \
-	register long _arg1 asm("x0") = (long)(arg1);                         \
+	register long _num  __asm__ ("x8") = (num);                           \
+	register long _arg1 __asm__ ("x0") = (long)(arg1);                    \
 	                                                                      \
-	asm volatile (                                                        \
+	__asm__  volatile (                                                   \
 		"svc #0\n"                                                    \
 		: "=r"(_arg1)                                                 \
 		: "r"(_arg1),                                                 \
@@ -93,11 +93,11 @@ struct sys_stat_struct {
 
 #define my_syscall2(num, arg1, arg2)                                          \
 ({                                                                            \
-	register long _num  asm("x8") = (num);                                \
-	register long _arg1 asm("x0") = (long)(arg1);                         \
-	register long _arg2 asm("x1") = (long)(arg2);                         \
+	register long _num  __asm__ ("x8") = (num);                           \
+	register long _arg1 __asm__ ("x0") = (long)(arg1);                    \
+	register long _arg2 __asm__ ("x1") = (long)(arg2);                    \
 	                                                                      \
-	asm volatile (                                                        \
+	__asm__  volatile (                                                   \
 		"svc #0\n"                                                    \
 		: "=r"(_arg1)                                                 \
 		: "r"(_arg1), "r"(_arg2),                                     \
@@ -109,12 +109,12 @@ struct sys_stat_struct {
 
 #define my_syscall3(num, arg1, arg2, arg3)                                    \
 ({                                                                            \
-	register long _num  asm("x8") = (num);                                \
-	register long _arg1 asm("x0") = (long)(arg1);                         \
-	register long _arg2 asm("x1") = (long)(arg2);                         \
-	register long _arg3 asm("x2") = (long)(arg3);                         \
+	register long _num  __asm__ ("x8") = (num);                           \
+	register long _arg1 __asm__ ("x0") = (long)(arg1);                    \
+	register long _arg2 __asm__ ("x1") = (long)(arg2);                    \
+	register long _arg3 __asm__ ("x2") = (long)(arg3);                    \
 	                                                                      \
-	asm volatile (                                                        \
+	__asm__  volatile (                                                   \
 		"svc #0\n"                                                    \
 		: "=r"(_arg1)                                                 \
 		: "r"(_arg1), "r"(_arg2), "r"(_arg3),                         \
@@ -126,13 +126,13 @@ struct sys_stat_struct {
 
 #define my_syscall4(num, arg1, arg2, arg3, arg4)                              \
 ({                                                                            \
-	register long _num  asm("x8") = (num);                                \
-	register long _arg1 asm("x0") = (long)(arg1);                         \
-	register long _arg2 asm("x1") = (long)(arg2);                         \
-	register long _arg3 asm("x2") = (long)(arg3);                         \
-	register long _arg4 asm("x3") = (long)(arg4);                         \
+	register long _num  __asm__ ("x8") = (num);                           \
+	register long _arg1 __asm__ ("x0") = (long)(arg1);                    \
+	register long _arg2 __asm__ ("x1") = (long)(arg2);                    \
+	register long _arg3 __asm__ ("x2") = (long)(arg3);                    \
+	register long _arg4 __asm__ ("x3") = (long)(arg4);                    \
 	                                                                      \
-	asm volatile (                                                        \
+	__asm__  volatile (                                                   \
 		"svc #0\n"                                                    \
 		: "=r"(_arg1)                                                 \
 		: "r"(_arg1), "r"(_arg2), "r"(_arg3), "r"(_arg4),             \
@@ -144,14 +144,14 @@ struct sys_stat_struct {
 
 #define my_syscall5(num, arg1, arg2, arg3, arg4, arg5)                        \
 ({                                                                            \
-	register long _num  asm("x8") = (num);                                \
-	register long _arg1 asm("x0") = (long)(arg1);                         \
-	register long _arg2 asm("x1") = (long)(arg2);                         \
-	register long _arg3 asm("x2") = (long)(arg3);                         \
-	register long _arg4 asm("x3") = (long)(arg4);                         \
-	register long _arg5 asm("x4") = (long)(arg5);                         \
+	register long _num  __asm__ ("x8") = (num);                           \
+	register long _arg1 __asm__ ("x0") = (long)(arg1);                    \
+	register long _arg2 __asm__ ("x1") = (long)(arg2);                    \
+	register long _arg3 __asm__ ("x2") = (long)(arg3);                    \
+	register long _arg4 __asm__ ("x3") = (long)(arg4);                    \
+	register long _arg5 __asm__ ("x4") = (long)(arg5);                    \
 	                                                                      \
-	asm volatile (                                                        \
+	__asm__  volatile (                                                   \
 		"svc #0\n"                                                    \
 		: "=r" (_arg1)                                                \
 		: "r"(_arg1), "r"(_arg2), "r"(_arg3), "r"(_arg4), "r"(_arg5), \
@@ -163,15 +163,15 @@ struct sys_stat_struct {
 
 #define my_syscall6(num, arg1, arg2, arg3, arg4, arg5, arg6)                  \
 ({                                                                            \
-	register long _num  asm("x8") = (num);                                \
-	register long _arg1 asm("x0") = (long)(arg1);                         \
-	register long _arg2 asm("x1") = (long)(arg2);                         \
-	register long _arg3 asm("x2") = (long)(arg3);                         \
-	register long _arg4 asm("x3") = (long)(arg4);                         \
-	register long _arg5 asm("x4") = (long)(arg5);                         \
-	register long _arg6 asm("x5") = (long)(arg6);                         \
+	register long _num  __asm__ ("x8") = (num);                           \
+	register long _arg1 __asm__ ("x0") = (long)(arg1);                    \
+	register long _arg2 __asm__ ("x1") = (long)(arg2);                    \
+	register long _arg3 __asm__ ("x2") = (long)(arg3);                    \
+	register long _arg4 __asm__ ("x3") = (long)(arg4);                    \
+	register long _arg5 __asm__ ("x4") = (long)(arg5);                    \
+	register long _arg6 __asm__ ("x5") = (long)(arg6);                    \
 	                                                                      \
-	asm volatile (                                                        \
+	__asm__  volatile (                                                   \
 		"svc #0\n"                                                    \
 		: "=r" (_arg1)                                                \
 		: "r"(_arg1), "r"(_arg2), "r"(_arg3), "r"(_arg4), "r"(_arg5), \
@@ -182,7 +182,7 @@ struct sys_stat_struct {
 })
 
 /* startup code */
-asm(".section .text\n"
+__asm__ (".section .text\n"
     ".weak _start\n"
     ".global _start\n"
     "_start:\n"
tools/include/nolibc/arch-arm.h +29 -29

106 lines not on this page · raw message

tools/include/nolibc/arch-i386.h +28 -28

104 lines not on this page · raw message

tools/include/nolibc/arch-mips.h +31 -31

111 lines not on this page · raw message

tools/include/nolibc/arch-riscv.h +37 -37

129 lines not on this page · raw message

tools/include/nolibc/arch-x86_64.h +36 -36

127 lines not on this page · raw message

Signature · 1 line
Ammar Faizi
#3 lore raw

PATCH v2 3/9 tools/nolibc: Remove .global _start from the entry point code

In reply to Ammar Faizi

Building with clang yields the following error:

```
  <inline asm>:3:1: error: _start changed binding to STB_GLOBAL
  .global _start
  ^
  1 error generated.
```

Make sure only specify one between .global _start and .weak _start. Remove .global _start.

===

@@ Changelog:

Link v1: https://lore.gnuweeb.org/gwml/[email protected]
v1 -> v2:
- Switch the order with the patch that replaces asm with __asm__.

Link RFC v2: https://lore.kernel.org/llvm/[email protected]
RFC v2 -> v1:
- Append Reviewed-by tag from Nick.
- s/Removing/remove/

Link RFC v1: https://lore.kernel.org/llvm/[email protected]
RFC v1 -> RFC v2:
- Remove all .global _start for all build (GCC and Clang) instead of removing all .weak _start for clang build (Comment from Willy).

---
 tools/include/nolibc/arch-aarch64.h | 1 -
 tools/include/nolibc/arch-arm.h     | 1 -
 tools/include/nolibc/arch-i386.h    | 1 -
 tools/include/nolibc/arch-mips.h    | 1 -
 tools/include/nolibc/arch-riscv.h   | 1 -
 tools/include/nolibc/arch-x86_64.h  | 1 -
 6 files changed, 6 deletions(-)
tools/include/nolibc/arch-aarch64.h +0 -1
index b68443f63980..f68baf8f395f 100644
@@ -184,7 +184,6 @@ struct sys_stat_struct {
 /* startup code */
 __asm__ (".section .text\n"
     ".weak _start\n"
-    ".global _start\n"
     "_start:\n"
     "ldr x0, [sp]\n"              // argc (x0) was in the stack
     "add x1, sp, 8\n"             // argv (x1) = sp
tools/include/nolibc/arch-arm.h +0 -1
index 55fd9439b2e2..f31be8e967d6 100644
@@ -177,7 +177,6 @@ struct sys_stat_struct {
 /* startup code */
 __asm__ (".section .text\n"
     ".weak _start\n"
-    ".global _start\n"
     "_start:\n"
 #if defined(__THUMBEB__) || defined(__THUMBEL__)
     /* We enter here in 32-bit mode but if some previous functions were in
tools/include/nolibc/arch-i386.h +0 -1
index 136d5739e456..10aada40680d 100644
@@ -176,7 +176,6 @@ struct sys_stat_struct {
  */
 __asm__ (".section .text\n"
     ".weak _start\n"
-    ".global _start\n"
     "_start:\n"
     "pop %eax\n"                // argc   (first arg, %eax)
     "mov %esp, %ebx\n"          // argv[] (second arg, %ebx)
tools/include/nolibc/arch-mips.h +0 -1
index cd587e539d7d..5fc5b8029bff 100644
@@ -192,7 +192,6 @@ struct sys_stat_struct {
 __asm__ (".section .text\n"
     ".weak __start\n"
     ".set nomips16\n"
-    ".global __start\n"
     ".set    noreorder\n"
     ".option pic0\n"
     ".ent __start\n"
tools/include/nolibc/arch-riscv.h +0 -1
index 8ec4c05fa69b..95e2b7924925 100644
@@ -185,7 +185,6 @@ struct sys_stat_struct {
 /* startup code */
 __asm__ (".section .text\n"
     ".weak _start\n"
-    ".global _start\n"
     "_start:\n"
     ".option push\n"
     ".option norelax\n"
tools/include/nolibc/arch-x86_64.h +0 -1
index 490530429ac9..0e1e9eb8545d 100644
@@ -199,7 +199,6 @@ struct sys_stat_struct {
  */
 __asm__ (".section .text\n"
     ".weak _start\n"
-    ".global _start\n"
     "_start:\n"
     "pop %rdi\n"                // argc   (first arg, %rdi)
     "mov %rsp, %rsi\n"          // argv[] (second arg, %rsi)
Signature · 1 line
Ammar Faizi
#4 lore raw

PATCH v2 4/9 tools/nolibc: i386: Implement syscall with 6 arguments

In reply to Ammar Faizi

On i386, the 6th argument of syscall goes in %ebp. However, both Clang and GCC cannot use %ebp in the clobber list and in the "r" constraint without using -fomit-frame-pointer. To make it always available for any kind of compilation, the below workaround is implemented.

1) Push the 6-th argument.
2) Push %ebp.
3) Load the 6-th argument from 4(%esp) to %ebp.
4) Do the syscall (int $0x80).
5) Pop %ebp (restore the old value of %ebp).
6) Add %esp by 4 (undo the stack pointer).

===
@@ Changelog:

Link v1: https://lore.gnuweeb.org/gwml/[email protected]
v1 -> v2:
* No changes *

Link RFC v2: https://lore.kernel.org/lkml/[email protected]
RFC v2 -> v1:
- Use the same pattern for syscall6, regardless using GCC or Clang (comment from David).
- Use appropriate constraints for syscall6 instead of always using register variables (comment from David).

Link RFC v1: https://lore.kernel.org/llvm/[email protected]
RFC v1 -> RFC v2:
- Fix %ebp saving method. Don't use redzone, i386 doesn't have a redzone (comment from David and Alviro).

---
 tools/include/nolibc/arch-i386.h | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
tools/include/nolibc/arch-i386.h +23 -0
index 10aada40680d..d7e7212346e2 100644
@@ -167,6 +167,29 @@ struct sys_stat_struct {
 	_ret;                                                                 \
 })
 
+#define my_syscall6(num, arg1, arg2, arg3, arg4, arg5, arg6)	\
+({								\
+	long _eax  = (long)(num);				\
+	long _arg6 = (long)(arg6); /* Always in memory */	\
+	__asm__ volatile (					\
+		"pushl	%[_arg6]\n\t"				\
+		"pushl	%%ebp\n\t"				\
+		"movl	4(%%esp),%%ebp\n\t"			\
+		"int	$0x80\n\t"				\
+		"popl	%%ebp\n\t"				\
+		"addl	$4,%%esp\n\t"				\
+		: "+a"(_eax)		/* %eax */		\
+		: "b"(arg1),		/* %ebx */		\
+		  "c"(arg2),		/* %ecx */		\
+		  "d"(arg3),		/* %edx */		\
+		  "S"(arg4),		/* %esi */		\
+		  "D"(arg5),		/* %edi */		\
+		  [_arg6]"m"(_arg6)	/* memory */		\
+		: "memory", "cc"				\
+	);							\
+	_eax;							\
+})
+
 /* startup code */
 /*
  * i386 System V ABI mandates:
Signature · 1 line
Ammar Faizi
#5 lore raw

PATCH v2 5/9 tools/nolibc/sys: Implement `mmap()` and `munmap()`

In reply to Ammar Faizi

Implement mmap() and munmap(). Currently, they are only available for architecures that have my_syscall6 macro. For architectures that don't have, this function will return -1 with errno set to ENOSYS (Function not implemented).

This has been tested on x86 and i386.

Notes for i386:
1) The common mmap() syscall implementation uses __NR_mmap2 instead

    of __NR_mmap.

2) The offset must be shifted-right by 12-bit.

===
@@ Changelog:

Link v1: https://lore.gnuweeb.org/gwml/[email protected]
v1 -> v2:
* No changes *

Link RFC v2: https://lore.kernel.org/lkml/[email protected]
RFC v2 -> v1:
* No changes *

Link RFC v1: https://lore.kernel.org/lkml/[email protected]
RFC v1 -> RFC v2:
* No changes *

---
 tools/include/nolibc/sys.h | 62 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 62 insertions(+)
tools/include/nolibc/sys.h +62 -0
index 4d4308d5d111..08491070387b 100644
@@ -14,6 +14,7 @@
 #include <asm/unistd.h>
 #include <asm/signal.h>  // for SIGCHLD
 #include <asm/ioctls.h>
+#include <asm/mman.h>
 #include <linux/fs.h>
 #include <linux/loop.h>
 #include <linux/time.h>
@@ -675,6 +676,67 @@ int mknod(const char *path, mode_t mode, dev_t dev)
 	return ret;
 }
 
+#ifndef MAP_SHARED
+#define MAP_SHARED		0x01	/* Share changes */
+#define MAP_PRIVATE		0x02	/* Changes are private */
+#define MAP_SHARED_VALIDATE	0x03	/* share + validate extension flags */
+#endif
+
+#ifndef MAP_FAILED
+#define MAP_FAILED ((void *)-1)
+#endif
+
+static __attribute__((unused))
+void *sys_mmap(void *addr, size_t length, int prot, int flags, int fd,
+	       off_t offset)
+{
+#ifndef my_syscall6
+	/* Function not implemented. */
+	return -ENOSYS;
+#else
+
+	int n;
+
+#if defined(__i386__)
+	n = __NR_mmap2;
+	offset >>= 12;
+#else
+	n = __NR_mmap;
+#endif
+
+	return (void *)my_syscall6(n, addr, length, prot, flags, fd, offset);
+#endif
+}
+
+static __attribute__((unused))
+void *mmap(void *addr, size_t length, int prot, int flags, int fd, off_t offset)
+{
+	void *ret = sys_mmap(addr, length, prot, flags, fd, offset);
+
+	if ((unsigned long)ret >= -4095UL) {
+		SET_ERRNO(-(long)ret);
+		ret = MAP_FAILED;
+	}
+	return ret;
+}
+
+static __attribute__((unused))
+int sys_munmap(void *addr, size_t length)
+{
+	return my_syscall2(__NR_munmap, addr, length);
+}
+
+static __attribute__((unused))
+int munmap(void *addr, size_t length)
+{
+	int ret = sys_munmap(addr, length);
+
+	if (ret < 0) {
+		SET_ERRNO(-ret);
+		ret = -1;
+	}
+	return ret;
+}
 
 /*
  * int mount(const char *source, const char *target,
Signature · 1 line
Ammar Faizi
#6 lore raw

PATCH v2 6/9 tools/nolibc/types: Implement `offsetof()` and `container_of()` macro

In reply to Ammar Faizi

Implement offsetof() and container_of() macro. The first use case of these macros is for malloc(), realloc() and free().

===
@@ Changelog:

Link v1: https://lore.gnuweeb.org/gwml/[email protected]
v1 -> v2:
* No changes *

Link RFC v2: https://lore.kernel.org/lkml/[email protected]
RFC v2 -> v1:
* No changes *

---
 tools/include/nolibc/types.h | 11 +++++++++++
 1 file changed, 11 insertions(+)
tools/include/nolibc/types.h +11 -0
index 357e60ad38a8..959997034e55 100644
@@ -191,4 +191,15 @@ struct stat {
 #define major(dev) ((unsigned int)(((dev) >> 8) & 0xfff))
 #define minor(dev) ((unsigned int)(((dev) & 0xff))
 
+#ifndef offsetof
+#define offsetof(TYPE, FIELD) ((size_t) &((TYPE *)0)->FIELD)
+#endif
+
+#ifndef container_of
+#define container_of(PTR, TYPE, FIELD) ({			\
+	__typeof__(((TYPE *)0)->FIELD) *__FIELD_PTR = (PTR);	\
+	(TYPE *)((char *) __FIELD_PTR - offsetof(TYPE, FIELD));	\
+})
+#endif
+
 #endif /* _NOLIBC_TYPES_H */
Signature · 1 line
Ammar Faizi
#7 lore raw

PATCH v2 7/9 tools/nolibc/stdlib: Implement `malloc()`, `calloc()`, `realloc()` and `free()`

In reply to Ammar Faizi

Implement basic dynamic allocator functions. These functions are currently only available on architectures that have nolibc mmap() syscall implemented. These are not a super-fast memory allocator, but at least they can satisfy basic needs for having heap without libc.

===

@@ Changelog:

Link v1: https://lore.gnuweeb.org/gwml/[email protected]
v1 -> v2:
* No changes *

Link v2: https://lore.kernel.org/lkml/[email protected]
RFC v2 -> v1:
- Round up the malloc() allocation to 4096 (comment from David).
- Don't realloc() if we still have enough memory to contain the requested new size (comment from David).
- Fix conflict with getenv() fix (after rebase).

Link v1: https://lore.kernel.org/lkml/[email protected]
RFC v1 -> RFC v2:
- Move container_of() and offsetof() macro to types.h with a
separate patch (comment from Willy).

---
 tools/include/nolibc/stdlib.h | 81 +++++++++++++++++++++++++++++++++++
 1 file changed, 81 insertions(+)
tools/include/nolibc/stdlib.h +81 -0
index 8a07e263f0d0..8fd32eaf8037 100644
@@ -11,7 +11,12 @@
 #include "arch.h"
 #include "types.h"
 #include "sys.h"
+#include "string.h"
 
+struct nolibc_heap {
+	size_t	len;
+	char	user_p[] __attribute__((__aligned__));
+};
 
 /* Buffer used to store int-to-ASCII conversions. Will only be implemented if
  * any of the related functions is implemented. The area is large enough to
@@ -60,6 +65,18 @@ int atoi(const char *s)
 	return atol(s);
 }
 
+static __attribute__((unused))
+void free(void *ptr)
+{
+	struct nolibc_heap *heap;
+
+	if (!ptr)
+		return;
+
+	heap = container_of(ptr, struct nolibc_heap, user_p);
+	munmap(heap, heap->len);
+}
+
 /* getenv() tries to find the environment variable named <name> in the
  * environment array pointed to by global variable "environ" which must be
  * declared as a char **, and must be terminated by a NULL (it is recommended
@@ -91,6 +108,70 @@ char *getenv(const char *name)
 	return _getenv(name, environ);
 }
 
+static __attribute__((unused))
+void *malloc(size_t len)
+{
+	struct nolibc_heap *heap;
+
+	/* Always allocate memory with size multiple of 4096. */
+	len  = sizeof(*heap) + len;
+	len  = (len + 4095UL) & -4096UL;
+	heap = mmap(NULL, len, PROT_READ|PROT_WRITE, MAP_ANONYMOUS|MAP_PRIVATE,
+		    -1, 0);
+	if (__builtin_expect(heap == MAP_FAILED, 0))
+		return NULL;
+
+	heap->len = len;
+	return heap->user_p;
+}
+
+static __attribute__((unused))
+void *calloc(size_t size, size_t nmemb)
+{
+	void *orig;
+	size_t res = 0;
+
+	if (__builtin_expect(__builtin_mul_overflow(nmemb, size, &res), 0)) {
+		SET_ERRNO(ENOMEM);
+		return NULL;
+	}
+
+	/*
+	 * No need to zero the heap, the MAP_ANONYMOUS in malloc()
+	 * already does it.
+	 */
+	return malloc(res);
+}
+
+static __attribute__((unused))
+void *realloc(void *old_ptr, size_t new_size)
+{
+	struct nolibc_heap *heap;
+	size_t user_p_len;
+	void *ret;
+
+	if (!old_ptr)
+		return malloc(new_size);
+
+	heap = container_of(old_ptr, struct nolibc_heap, user_p);
+	user_p_len = heap->len - sizeof(*heap);
+	/*
+	 * Don't realloc() if @user_p_len >= @new_size, this block of
+	 * memory is still enough to handle the @new_size. Just return
+	 * the same pointer.
+	 */
+	if (user_p_len >= new_size)
+		return old_ptr;
+
+	ret = malloc(new_size);
+	if (__builtin_expect(!ret, 0))
+		return NULL;
+
+	memcpy(ret, heap->user_p, heap->len);
+	munmap(heap, heap->len);
+	return ret;
+}
+
 /* Converts the unsigned long integer <in> to its hex representation into
  * buffer <buffer>, which must be long enough to store the number and the
  * trailing zero (17 bytes for "ffffffffffffffff" or 9 for "ffffffff"). The
Signature · 1 line
Ammar Faizi
#8 lore raw

PATCH v2 8/9 tools/nolibc/string: Implement `strnlen()`

In reply to Ammar Faizi
  size_t strnlen(const char *str, size_t maxlen);

The strnlen() function returns the number of bytes in the string pointed to by sstr, excluding the terminating null byte ('\0'), but at most maxlen. In doing this, strnlen() looks only at the first maxlen characters in the string pointed to by str and never beyond str[maxlen-1].

The first use case of this function is for determining the memory allocation size in the strndup() function.

===

@@ Changelog:

Link v1: https://lore.gnuweeb.org/gwml/[email protected]
v1 -> v2:
* No changes *

Link v2: https://lore.kernel.org/lkml/[email protected]
RFC v2 -> v1:
* No changes *

---
 tools/include/nolibc/string.h | 9 +++++++++
 1 file changed, 9 insertions(+)
tools/include/nolibc/string.h +9 -0
index 75a453870498..f43d52a44d09 100644
@@ -147,6 +147,15 @@ size_t nolibc_strlen(const char *str)
 #define strlen(str) nolibc_strlen((str))
 #endif
 
+static __attribute__((unused))
+size_t strnlen(const char *str, size_t maxlen)
+{
+	size_t len;
+
+	for (len = 0; (len < maxlen) && str[len]; len++);
+	return len;
+}
+
 static __attribute__((unused))
 size_t strlcat(char *dst, const char *src, size_t size)
 {
Signature · 1 line
Ammar Faizi
#9 lore raw

PATCH v2 9/9 tools/include/string: Implement `strdup()` and `strndup()`

In reply to Ammar Faizi

These functions are currently only available on architectures that have my_syscall6() macro implemented. Since these functions use malloc(), malloc() uses mmap(), mmap() depends on my_syscall6() macro.

On architectures that don't support my_syscall6(), these function will always return NULL with errno set to ENOSYS.

===

@@ Changelog:

Link v1: https://lore.gnuweeb.org/gwml/[email protected]
v1 -> v2:
* No changes *

Link RFC v2: https://lore.kernel.org/lkml/[email protected]
RFC v2 -> v1:
* No changes *

Link RFC v1: https://lore.kernel.org/lkml/[email protected]
RFC v1 -> RFC v2:
- Update strdup and strndup implementation, use strlen and strnlen to get the string length first (comment from Willy and Alviro).
- Fix the subject line prefix, it was "tools/include/string: ", it should be "tools/nolibc/string: ".

---
 tools/include/nolibc/string.h | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
tools/include/nolibc/string.h +32 -0
index f43d52a44d09..bef35bee9c44 100644
@@ -9,6 +9,8 @@
 
 #include "std.h"
 
+static void *malloc(size_t len);
+
 /*
  * As much as possible, please keep functions alphabetically sorted.
  */
@@ -156,6 +158,36 @@ size_t strnlen(const char *str, size_t maxlen)
 	return len;
 }
 
+static __attribute__((unused))
+char *strdup(const char *str)
+{
+	size_t len;
+	char *ret;
+
+	len = strlen(str);
+	ret = malloc(len + 1);
+	if (__builtin_expect(ret != NULL, 1))
+		memcpy(ret, str, len + 1);
+
+	return ret;
+}
+
+static __attribute__((unused))
+char *strndup(const char *str, size_t maxlen)
+{
+	size_t len;
+	char *ret;
+
+	len = strnlen(str, maxlen);
+	ret = malloc(len + 1);
+	if (__builtin_expect(ret != NULL, 1)) {
+		memcpy(ret, str, len);
+		ret[len] = '\0';
+	}
+
+	return ret;
+}
+
 static __attribute__((unused))
 size_t strlcat(char *dst, const char *src, size_t size)
 {
Signature · 1 line
Ammar Faizi
#10 lore raw
In reply to Ammar Faizi
Quoted 4 lines · Ammar Faizi

Hi,

This is a patchset v2 to add dynamic memory allocator support for nolibc after 2 RFCs, please review the changes carefully.

Sorry, forgot to replace === with --- in for each patch. Should I resend?

Signature · 1 line
Ammar Faizi
maintainer Acked-by
#11 lore raw

Hi Ammar,

Quoted 5 lines · Ammar Faizi
Quoted 4 lines · Ammar Faizi

Hi,

This is a patchset v2 to add dynamic memory allocator support for nolibc after 2 RFCs, please review the changes carefully.

Thank you! For me it's OK for all the series:

I do have a minor comment about the use of __builtin_mul_overflow() here. While this code is included in the kernel and mostly for use with kernel related stuff, till now I've been careful to support older compilers (I'm still seeing 4.8, 4.7 and 4.4 commonly in field). I don't find it urgent, but I think that sooner or later it would be nice to implement an alternative for compilers missing this builtin, especially if it's the only one that prevents older compilers from being used. Probably that something like this (untested) would do the job:

	if (nmemb && ~(size_t)0 / nmemb < size) {
		SET_ERRNO(ENOMEM);
		return NULL;
	}
	size *= nmemb;

But again, for me it's not a showstopper and can be improved later.

Quoted 2 lines

Sorry, forgot to replace === with --- in for each patch. Should I resend?

Let's see what Paul prefers. sed 's/===/---/' on the mbox should fix it otherwise a resend will be needed.

Thanks!
Willy

#12 lore raw
Quoted 28 lines · Willy Tarreau

Hi Ammar,

Quoted 5 lines · Ammar Faizi
Quoted 4 lines · Ammar Faizi

Hi,

This is a patchset v2 to add dynamic memory allocator support for nolibc after 2 RFCs, please review the changes carefully.

Thank you! For me it's OK for all the series:

I do have a minor comment about the use of __builtin_mul_overflow() here. While this code is included in the kernel and mostly for use with kernel related stuff, till now I've been careful to support older compilers (I'm still seeing 4.8, 4.7 and 4.4 commonly in field). I don't find it urgent, but I think that sooner or later it would be nice to implement an alternative for compilers missing this builtin, especially if it's the only one that prevents older compilers from being used. Probably that something like this (untested) would do the job:

	if (nmemb && ~(size_t)0 / nmemb < size) {
		SET_ERRNO(ENOMEM);
		return NULL;
	}
	size *= nmemb;
Quoted 7 lines

But again, for me it's not a showstopper and can be improved later.

Quoted 2 lines

Sorry, forgot to replace === with --- in for each patch. Should I resend?

Let's see what Paul prefers. sed 's/===/---/' on the mbox should fix it otherwise a resend will be needed.

Given that I am adding your Acked-by anyway, why not? ;-)

But please check the commits to make sure that this had the desired effect.

Thanx, Paul

maintainer
#13 lore raw
Quoted 7 lines · Paul E. McKenney
Quoted 5 lines
Quoted 2 lines

Sorry, forgot to replace === with --- in for each patch. Should I resend?

Let's see what Paul prefers. sed 's/===/---/' on the mbox should fix it otherwise a resend will be needed.

Given that I am adding your Acked-by anyway, why not? ;-)

:-)

Quoted 2 lines

But please check the commits to make sure that this had the desired effect.

All of it looks perfect to me, many thanks Paul!

Willy