Skip to contentSkip to conversation

aarch64 support

PATCH Superseded Ammar Faizi 2 participants 15 messages Jul 4, 2022 – Jul 5, 2022 io_uring

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v4 cover aarch64 support

From: Ammar Faizi <[email protected]>

Hi Jens,

This is v4 revision of aarch64 support.

This series contains nolibc support for aarch64 and one extra irrelevant cleanup (patch #1). The missing bit from aarch64 is get_page_size() which is a bit complicated to implement without libc.

aarch64 supports three values of page size: 4K, 16K, and 64K which are selected at kernel compilation time. Therefore, we can't hard code the page size for this arch. In this series we utilize open(), read() and close() syscall to find the page size from /proc/self/auxv.

The auxiliary vector contains information about the page size, it is located at AT_PAGESZ keyval pair.

For more details about the auxv data structure, check the link below.

There are 10 patches in this series. Summary:

- Patch 1 is just a trivial changelog fix.
- Patch 2 and 3 are to add open() and read() syscall.
- Patch 4 is to remove __INTERNAL__LIBURING_SYSCALL_H checks.
- Patch 5 is to add get_page_size() function.
- Patch 6 is just a trivial style fixup for #ifdef and friends.
- Patch 7 is to enable the nolibc support for aarch64.
- Patch 8 is to add nolibc test.
- Patch 9 is for GitHub bot build.
- Patch 10 is to update the CHANGELOG to note aarch64 nolibc support.

I have built it with GitHub bot and it compiles just fine. But I don't have an aarch64 machine to run the tests. Since you are using aarch64, I can rely on you to test it.

How to test this?

  make clean;
  ./configure --nolibc;
  make -j8;

ldd src/liburing.so.2.3;
# Make sure you don't see libc.so from the ldd command.

make runtests;

Please give it a test, thanks!

## Changelog

v3 -> v4:
- Simplify __get_page_size() function.
- Change __sys_read() return type from int to ssize_t.
- Append Reviewed-by tags from Alviro Iskandar Setiawan.
- Fix git am complain: space before tab in indent.

v2 -> v3:
- No need to init the static var to zero.
- Split style fixup into a separate patch.
- Add nolibc test to test get_page_size() function.
- Use __NR_openat when __NR_open is not defined on the arch.
- Split open/read/close in get_page_size() into a new function.
- Cache the get_page_size() fallback value when we fail on the syscalls.

v1 -> v2:
- Drop aarch64 renaming directory patch.
- Fallback the page size to 4K if we fail to get it.
- Cache the page size after we read it from /proc/self/auxv.
- Massage commit messages.
- Note aarch64 nolibc support in CHANGELOG.

---

Ammar Faizi (10):
CHANGELOG: Fixup missing space
arch: syscall: Add __sys_open() syscall
arch: syscall: Add __sys_read() syscall
arch: Remove __INTERNAL__LIBURING_LIB_H checks arch/aarch64: lib: Add get_page_size() function
lib: Style fixup for #if / #elif / #else / #endif
lib: Enable nolibc support for aarch64
test: Add nolibc test
.github: Enable aarch64 nolibc build for GitHub bot
CHANGELOG: Note about aarch64 support

 .github/workflows/build.yml |  2 +-
 CHANGELOG                   |  7 ++++-
 src/arch/aarch64/lib.h      | 48 +++++++++++++++++++++++++++++
 src/arch/aarch64/syscall.h  |  4 ---
 src/arch/generic/lib.h      |  4 ---
 src/arch/generic/syscall.h  | 20 ++++++++++---
 src/arch/syscall-defs.h     | 19 ++++++++++++
 src/arch/x86/lib.h          |  4 ---
 src/arch/x86/syscall.h      |  4 ---
 src/lib.h                   | 32 ++++++++++----------
 src/syscall.h               |  2 --
 test/Makefile               |  1 +
 test/nolibc.c               | 60 +++++++++++++++++++++++++++++++++++++
 13 files changed, 167 insertions(+), 40 deletions(-)
 create mode 100644 src/arch/aarch64/lib.h
 create mode 100644 test/nolibc.c

base-commit: f8eb5f804288e10ae7ef442ef482e4dd8b18fee7

Signature · 1 line
Ammar Faizi
#1 lore raw

PATCH v4 1/10 CHANGELOG: Fixup missing space

From: Ammar Faizi <[email protected]>

s/reducingthe/reducing the/

---
 CHANGELOG | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
CHANGELOG +1 -1
index 01cb677..efb3ff3 100644
@@ -6,7 +6,7 @@ liburing-2.2 release
 - Add support for multishot accept.
 - io_uring_register_files() will set RLIMIT_NOFILE if necessary.
 - Add support for registered ring fds, io_uring_register_ring_fd(),
-  reducingthe overhead of an io_uring_enter() system call.
+  reducing the overhead of an io_uring_enter() system call.
 - Add support for the message ring opcode.
 - Add support for newer request cancelation features.
 - Add support for IORING_SETUP_COOP_TASKRUN, which can help reduce the
Signature · 1 line
Ammar Faizi
#2 lore raw

PATCH v4 2/10 arch: syscall: Add `__sys_open()` syscall

In reply to Ammar Faizi

From: Ammar Faizi <[email protected]>

A prep patch to support aarch64 nolibc. We will use this to get the page size by reading /proc/self/auxv. For some reason __NR_open is not defined, so also define it in aarch64 syscall specific file.

v3:
- Use __NR_openat if __NR_open is not defined.

---
 src/arch/generic/syscall.h |  9 +++++++++
 src/arch/syscall-defs.h    | 14 ++++++++++++++
 2 files changed, 23 insertions(+)
src/arch/generic/syscall.h +9 -0
index fa93064..71b2234 100644
@@ -7,6 +7,8 @@
 #ifndef LIBURING_ARCH_GENERIC_SYSCALL_H
 #define LIBURING_ARCH_GENERIC_SYSCALL_H
 
+#include <fcntl.h>
+
 static inline int ____sys_io_uring_register(int fd, unsigned opcode,
 					    const void *arg, unsigned nr_args)
 {
@@ -41,6 +43,13 @@ static inline int ____sys_io_uring_enter(int fd, unsigned to_submit,
 				       _NSIG / 8);
 }
 
+static inline int __sys_open(const char *pathname, int flags, mode_t mode)
+{
+	int ret;
+	ret = open(pathname, flags, mode);
+	return (ret < 0) ? -errno : ret;
+}
+
 static inline void *__sys_mmap(void *addr, size_t length, int prot, int flags,
 			       int fd, off_t offset)
 {
src/arch/syscall-defs.h +14 -0
index 1e8ae1b..9d4424d 100644
@@ -3,6 +3,20 @@
 #ifndef LIBURING_ARCH_SYSCALL_DEFS_H
 #define LIBURING_ARCH_SYSCALL_DEFS_H
 
+#include <fcntl.h>
+
+static inline int __sys_open(const char *pathname, int flags, mode_t mode)
+{
+	/*
+	 * Some architectures don't have __NR_open, but __NR_openat.
+	 */
+#ifdef __NR_open
+	return __do_syscall3(__NR_open, pathname, flags, mode);
+#else
+	return __do_syscall4(__NR_openat, AT_FDCWD, pathname, flags, mode);
+#endif
+}
+
 static inline void *__sys_mmap(void *addr, size_t length, int prot, int flags,
 			       int fd, off_t offset)
 {
Signature · 1 line
Ammar Faizi
#3 lore raw

Re: [PATCH liburing v4 02/10] arch: syscall: Add `__sys_open()` syscall

Quoted 5 lines · Ammar Faizi

From: Ammar Faizi <[email protected]>

A prep patch to support aarch64 nolibc. We will use this to get the page size by reading /proc/self/auxv. For some reason __NR_open is not defined, so also define it in aarch64 syscall specific file.

The commit message should also be updated, no extra definition for __NR_open anymore in this version.

Quoted 5 lines

v3:
- Use __NR_openat if __NR_open is not defined.

tq

-- Viro

#4 lore raw

Re: [PATCH liburing v4 02/10] arch: syscall: Add `__sys_open()` syscall

Quoted 9 lines · Alviro Iskandar Setiawan
Quoted 5 lines · Ammar Faizi

From: Ammar Faizi <[email protected]>

A prep patch to support aarch64 nolibc. We will use this to get the page size by reading /proc/self/auxv. For some reason __NR_open is not defined, so also define it in aarch64 syscall specific file.

The commit message should also be updated, no extra definition for __NR_open anymore in this version.

Ah right, forgot to remove that from the commit message, will do it in v5. Thanks!

Signature · 1 line
Ammar Faizi
#5 lore raw

PATCH v4 3/10 arch: syscall: Add `__sys_read()` syscall

In reply to Ammar Faizi

From: Ammar Faizi <[email protected]>

A prep patch to support aarch64 nolibc. We will use this to get the page size by reading /proc/self/auxv.

v2:
- Fix return type, __sys_read() returns ssize_t, not int.

---
 src/arch/generic/syscall.h | 7 +++++++
 src/arch/syscall-defs.h    | 5 +++++
 2 files changed, 12 insertions(+)
src/arch/generic/syscall.h +7 -0
index 71b2234..22252a1 100644
@@ -50,6 +50,13 @@ static inline int __sys_open(const char *pathname, int flags, mode_t mode)
 	return (ret < 0) ? -errno : ret;
 }
 
+static inline ssize_t __sys_read(int fd, void *buffer, size_t size)
+{
+	ssize_t ret;
+	ret = read(fd, buffer, size);
+	return (ret < 0) ? -errno : ret;
+}
+
 static inline void *__sys_mmap(void *addr, size_t length, int prot, int flags,
 			       int fd, off_t offset)
 {
src/arch/syscall-defs.h +5 -0
index 9d4424d..bde69fa 100644
@@ -17,6 +17,11 @@ static inline int __sys_open(const char *pathname, int flags, mode_t mode)
 #endif
 }
 
+static inline ssize_t __sys_read(int fd, void *buffer, size_t size)
+{
+	return (ssize_t) __do_syscall3(__NR_read, fd, buffer, size);
+}
+
 static inline void *__sys_mmap(void *addr, size_t length, int prot, int flags,
 			       int fd, off_t offset)
 {
Signature · 1 line
Ammar Faizi
#6 lore raw

PATCH v4 4/10 arch: Remove `__INTERNAL__LIBURING_LIB_H` checks

In reply to Ammar Faizi

From: Ammar Faizi <[email protected]>

We will include the syscall.h from another place as well. This check was added by me when adding the x86 syscalls. For aarch64 we will include this header from lib.h but we are restricted by this check. Let's just remove it for all archs. User shouldn't touch this code directly anyway.

---
 src/arch/aarch64/syscall.h | 4 ----
 src/arch/generic/lib.h     | 4 ----
 src/arch/generic/syscall.h | 4 ----
 src/arch/x86/lib.h         | 4 ----
 src/arch/x86/syscall.h     | 4 ----
 src/lib.h                  | 2 --
 src/syscall.h              | 2 --
 7 files changed, 24 deletions(-)
src/arch/aarch64/syscall.h +0 -4
index c0ab7e2..b00e90b 100644
@@ -1,9 +1,5 @@
 /* SPDX-License-Identifier: MIT */
 
-#ifndef __INTERNAL__LIBURING_SYSCALL_H
-	#error "This file should be included from src/syscall.h (liburing)"
-#endif
-
 #ifndef LIBURING_ARCH_AARCH64_SYSCALL_H
 #define LIBURING_ARCH_AARCH64_SYSCALL_H
 
src/arch/generic/lib.h +0 -4
index 737e795..6b006c6 100644
@@ -1,9 +1,5 @@
 /* SPDX-License-Identifier: MIT */
 
-#ifndef __INTERNAL__LIBURING_LIB_H
-	#error "This file should be included from src/lib.h (liburing)"
-#endif
-
 #ifndef LIBURING_ARCH_GENERIC_LIB_H
 #define LIBURING_ARCH_GENERIC_LIB_H
 
src/arch/generic/syscall.h +0 -4
index 22252a1..e637890 100644
@@ -1,9 +1,5 @@
 /* SPDX-License-Identifier: MIT */
 
-#ifndef __INTERNAL__LIBURING_SYSCALL_H
-	#error "This file should be included from src/syscall.h (liburing)"
-#endif
-
 #ifndef LIBURING_ARCH_GENERIC_SYSCALL_H
 #define LIBURING_ARCH_GENERIC_SYSCALL_H
 
src/arch/x86/lib.h +0 -4
index e6a74f3..6ece2d4 100644
@@ -1,9 +1,5 @@
 /* SPDX-License-Identifier: MIT */
 
-#ifndef __INTERNAL__LIBURING_LIB_H
-	#error "This file should be included from src/lib.h (liburing)"
-#endif
-
 #ifndef LIBURING_ARCH_X86_LIB_H
 #define LIBURING_ARCH_X86_LIB_H
 
src/arch/x86/syscall.h +0 -4
index 43c576b..cb8fb91 100644
@@ -1,9 +1,5 @@
 /* SPDX-License-Identifier: MIT */
 
-#ifndef __INTERNAL__LIBURING_SYSCALL_H
-	#error "This file should be included from src/syscall.h (liburing)"
-#endif
-
 #ifndef LIBURING_ARCH_X86_SYSCALL_H
 #define LIBURING_ARCH_X86_SYSCALL_H
 
src/lib.h +0 -2
index 89a40f2..7bbacb9 100644
@@ -6,7 +6,6 @@
 #include <string.h>
 #include <unistd.h>
 
-#define __INTERNAL__LIBURING_LIB_H
 #if defined(__x86_64__) || defined(__i386__)
 	#include "arch/x86/lib.h"
 #else
@@ -19,7 +18,6 @@
 	/* libc wrappers. */
 	#include "arch/generic/lib.h"
 #endif
-#undef __INTERNAL__LIBURING_LIB_H
 
 
 #ifndef offsetof
src/syscall.h +0 -2
index 214789d..73b04b4 100644
@@ -70,7 +70,6 @@ static inline bool IS_ERR(const void *ptr)
 	return uring_unlikely((uintptr_t) ptr >= (uintptr_t) -4095UL);
 }
 
-#define __INTERNAL__LIBURING_SYSCALL_H
 #if defined(__x86_64__) || defined(__i386__)
 #include "arch/x86/syscall.h"
 #elif defined(__aarch64__)
@@ -86,7 +85,6 @@ static inline bool IS_ERR(const void *ptr)
 /* libc syscall wrappers. */
 #include "arch/generic/syscall.h"
 #endif
-#undef __INTERNAL__LIBURING_SYSCALL_H
 
 /*
  * For backward compatibility.
Signature · 1 line
Ammar Faizi
Reviewed-by on v4 4/10
#7 lore raw

Re: [PATCH liburing v4 04/10] arch: Remove `__INTERNAL__LIBURING_LIB_H` checks

Quoted 9 lines · Ammar Faizi

From: Ammar Faizi <[email protected]>

We will include the syscall.h from another place as well. This check was added by me when adding the x86 syscalls. For aarch64 we will include this header from lib.h but we are restricted by this check. Let's just remove it for all archs. User shouldn't touch this code directly anyway.

tq

-- Viro

#8 lore raw

PATCH v4 5/10 arch/aarch64: lib: Add `get_page_size()` function

In reply to Ammar Faizi

From: Ammar Faizi <[email protected]>

A prep patch to add aarch64 nolibc support.

aarch64 supports three values of page size: 4K, 16K, and 64K which are selected at kernel compilation time. Therefore, we can't hard code the page size for this arch. Utilize open(), read() and close() syscall to find the page size from /proc/self/auxv. For more details about the auxv data structure, check the link below [1].

v4:
- Simplify __get_page_size() function (review from Alviro).

v3:
- Split open/read/close in get_page_size() into a new function.
- Cache the fallback value when we fail on the syscalls.
- No need to init the static var to zero.

v2:
- Fallback to 4K if the syscall fails.
- Cache the page size after read as suggested by Jens.

---
 src/arch/aarch64/lib.h | 48 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)
 create mode 100644 src/arch/aarch64/lib.h
src/arch/aarch64/lib.h new file +48 -0
new file mode 100644
index 0000000..5a75c1a
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: MIT */
+
+#ifndef LIBURING_ARCH_AARCH64_LIB_H
+#define LIBURING_ARCH_AARCH64_LIB_H
+
+#include <elf.h>
+#include <sys/auxv.h>
+#include "../../syscall.h"
+
+static inline long __get_page_size(void)
+{
+	Elf64_Off buf[2];
+	long ret = 4096;
+	int fd;
+
+	fd = __sys_open("/proc/self/auxv", O_RDONLY, 0);
+	if (fd < 0)
+		return ret;
+
+	while (1) {
+		ssize_t x;
+
+		x = __sys_read(fd, buf, sizeof(buf));
+		if (x < sizeof(buf))
+			break;
+
+		if (buf[0] == AT_PAGESZ) {
+			ret = buf[1];
+			break;
+		}
+	}
+
+	__sys_close(fd);
+	return ret;
+}
+
+static inline long get_page_size(void)
+{
+	static long cache_val;
+
+	if (cache_val)
+		return cache_val;
+
+	cache_val = __get_page_size();
+	return cache_val;
+}
+
+#endif /* #ifndef LIBURING_ARCH_AARCH64_LIB_H */
Signature · 1 line
Ammar Faizi
#9 lore raw

PATCH v4 6/10 lib: Style fixup for #if / #elif / #else / #endif

In reply to Ammar Faizi

From: Ammar Faizi <[email protected]>

Don't indent the block inside #if / #elif / #else / #endif.

---
 src/lib.h | 28 ++++++++++++++--------------
 1 file changed, 14 insertions(+), 14 deletions(-)
src/lib.h +14 -14
index 7bbacb9..a966c77 100644
@@ -7,28 +7,28 @@
 #include <unistd.h>
 
 #if defined(__x86_64__) || defined(__i386__)
-	#include "arch/x86/lib.h"
+#include "arch/x86/lib.h"
 #else
-	/*
-	 * We don't have nolibc support for this arch. Must use libc!
-	 */
-	#ifdef CONFIG_NOLIBC
-		#error "This arch doesn't support building liburing without libc"
-	#endif
-	/* libc wrappers. */
-	#include "arch/generic/lib.h"
+/*
+ * We don't have nolibc support for this arch. Must use libc!
+ */
+#ifdef CONFIG_NOLIBC
+#error "This arch doesn't support building liburing without libc"
+#endif
+/* libc wrappers. */
+#include "arch/generic/lib.h"
 #endif
 
 
 #ifndef offsetof
-	#define offsetof(TYPE, FIELD) ((size_t) &((TYPE *)0)->FIELD)
+#define offsetof(TYPE, FIELD) ((size_t) &((TYPE *)0)->FIELD)
 #endif
 
 #ifndef container_of
-	#define container_of(PTR, TYPE, FIELD) ({			\
-		__typeof__(((TYPE *)0)->FIELD) *__FIELD_PTR = (PTR);	\
-		(TYPE *)((char *) __FIELD_PTR - offsetof(TYPE, FIELD));	\
-	})
+#define container_of(PTR, TYPE, FIELD) ({			\
+	__typeof__(((TYPE *)0)->FIELD) *__FIELD_PTR = (PTR);	\
+	(TYPE *)((char *) __FIELD_PTR - offsetof(TYPE, FIELD));	\
+})
 #endif
 
 #define __maybe_unused		__attribute__((__unused__))
Signature · 1 line
Ammar Faizi
#10 lore raw

PATCH v4 7/10 lib: Enable nolibc support for aarch64

In reply to Ammar Faizi

From: Ammar Faizi <[email protected]>

A previous patch adds get_page_size() function which is the missing bit for aarch64 nolibc support. Now we have a full set of functions to enable nolibc build support for aarch64.

---
 src/lib.h | 2 ++
 1 file changed, 2 insertions(+)
src/lib.h +2 -0
index a966c77..f347191 100644
@@ -8,6 +8,8 @@
 
 #if defined(__x86_64__) || defined(__i386__)
 #include "arch/x86/lib.h"
+#elif defined(__aarch64__)
+#include "arch/aarch64/lib.h"
 #else
 /*
  * We don't have nolibc support for this arch. Must use libc!
Signature · 1 line
Ammar Faizi
#11 lore raw

PATCH v4 8/10 test: Add nolibc test

In reply to Ammar Faizi

From: Ammar Faizi <[email protected]>

This test is used to test liburing nolibc functionality. The first use case is test get_page_size() function, especially for aarch64 which relies on reading /proc/self/auxv. We don't seem to have a test that tests that function, so let's do it here.

We may add more nolibc tests in this file in the future.

---
 test/Makefile |  1 +
 test/nolibc.c | 60 +++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 61 insertions(+)
 create mode 100644 test/nolibc.c
test/Makefile +1 -0
index 9590e1e..45674c3 100644
@@ -101,6 +101,7 @@ test_srcs := \
 	mkdir.c \
 	msg-ring.c \
 	multicqes_drain.c \
+	nolibc.c \
 	nop-all-sizes.c \
 	nop.c \
 	openat2.c \
test/nolibc.c new file +60 -0

60 lines not on this page · raw message

Signature · 1 line
Ammar Faizi
Reviewed-by on v4 8/10
#12 lore raw

Re: [PATCH liburing v4 08/10] test: Add nolibc test

Quoted 10 lines · Ammar Faizi

From: Ammar Faizi <[email protected]>

This test is used to test liburing nolibc functionality. The first use case is test get_page_size() function, especially for aarch64 which relies on reading /proc/self/auxv. We don't seem to have a test that tests that function, so let's do it here.

We may add more nolibc tests in this file in the future.

tq

-- Viro

#13 lore raw

PATCH v4 9/10 .github: Enable aarch64 nolibc build for GitHub bot

In reply to Ammar Faizi

From: Ammar Faizi <[email protected]>

---
 .github/workflows/build.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
.github/workflows/build.yml +1 -1
index 88192ff..fc119cb 100644
@@ -114,7 +114,7 @@ jobs:
 
     - name: Build nolibc
       run: |
-        if [[ "${{matrix.arch}}" == "x86_64" || "${{matrix.arch}}" == "i686" ]]; then \
+        if [[ "${{matrix.arch}}" == "x86_64" || "${{matrix.arch}}" == "i686" || "${{matrix.arch}}" == "aarch64" ]]; then \
             make clean; \
             ./configure --cc=${{matrix.cc}} --cxx=${{matrix.cxx}} --nolibc; \
             make -j$(nproc) V=1 CPPFLAGS="-Werror" CFLAGS="$FLAGS" CXXFLAGS="$FLAGS"; \
Signature · 1 line
Ammar Faizi
#14 lore raw

PATCH v4 10/10 CHANGELOG: Note about aarch64 support

In reply to Ammar Faizi

From: Ammar Faizi <[email protected]>

---
 CHANGELOG | 5 +++++
 1 file changed, 5 insertions(+)
CHANGELOG +5 -0
index efb3ff3..9c054b0 100644
@@ -1,3 +1,8 @@
+liburing-2.3 release
+
+- Support non-libc build for aarch64.
+
+
 liburing-2.2 release
 
 - Support non-libc builds.
Signature · 1 line
Ammar Faizi